Integrated Filter Methods

The following table shows the available methods for calculating the feature importance. Columns Classif, Regr and Surv indicate if classification, regression or survival analysis problems are supported. Columns Fac., Num. and Ord. show if a particular method can deal with factor, numeric and ordered factor features.

Current methods

Method	Package	Description	Classif	Regr	Surv	Fac.	Num.	Ord.
anova.test		ANOVA Test for binary and multiclass classification tasks	X				X
auc		AUC filter for binary classification tasks	X				X
carscore	care	CAR scores		X			X
FSelector_chi.squared	FSelector	Chi-squared statistic of independence between feature and target	X	X		X	X
FSelector_gain.ratio	FSelector	Chi-squared statistic of independence between feature and target	X	X		X	X
FSelector_information.gain	FSelector	Entropy-based information gain between feature and target	X	X		X	X
FSelector_oneR	FSelector	oneR association rule	X	X		X	X
FSelector_relief	FSelector	RELIEF algorithm	X	X		X	X
FSelector_symmetrical.uncertainty	FSelector	Entropy-based symmetrical uncertainty between feature and target	X	X		X	X
FSelectorRcpp_gain.ratio	FSelectorRcpp	Entropy-based Filters: Algorithms that find ranks of importance of discrete attributes, basing on their entropy with a continous class attribute	X	X		X	X
FSelectorRcpp_information.gain	FSelectorRcpp	Entropy-based Filters: Algorithms that find ranks of importance of discrete attributes, basing on their entropy with a continous class attribute	X	X		X	X
FSelectorRcpp_relief	FSelectorRcpp	RELIEF algorithm	X	X		X	X
FSelectorRcpp_symmetrical.uncertainty	FSelectorRcpp	Entropy-based Filters: Algorithms that find ranks of importance of discrete attributes, basing on their entropy with a continous class attribute	X	X		X	X
kruskal.test		Kruskal Test for binary and multiclass classification tasks	X			X	X
linear.correlation		Pearson correlation between feature and target		X			X
mrmr	mRMRe	Minimum redundancy, maximum relevance filter		X	X		X	X
party_cforest.importance	party	Permutation importance of random forest fitted in package ‘party’	X	X	X	X	X	X
permutation.importance		Aggregated difference between feature permuted and unpermuted predictions	X	X	X	X	X	X
praznik_CMIM	praznik	Minimal conditional mutual information maximisation filter	X	X		X	X
praznik_DISR	praznik	Double input symmetrical relevance filter	X	X		X	X
praznik_JMI	praznik	Joint mutual information filter	X	X		X	X
praznik_JMIM	praznik	Minimal joint mutual information maximisation filter	X	X		X	X
praznik_MIM	praznik	conditional mutual information based feature selection filters	X	X		X	X
praznik_MRMR	praznik	Minimum redundancy maximal relevancy filter	X	X		X	X
praznik_NJMIM	praznik	Minimal normalised joint mutual information maximisation filter	X	X		X	X
randomForest_importance	randomForest	Importance based on OOB-accuracy or node inpurity of random forest fitted in package ‘randomForest’.	X	X		X	X
randomForestSRC_importance	randomForestSRC	Importance of random forests fitted in package ‘randomForestSRC’. Importance is calculated using argument ‘permute’.	X	X	X	X	X	X
randomForestSRC_var.select	randomForestSRC	Minimal depth of / variable hunting via method var.select on random forests fitted in package ‘randomForestSRC’.	X	X	X	X	X	X
ranger_impurity	ranger	Variable importance based on ranger impurity importance	X	X		X	X	X
ranger_permutation	ranger	Variable importance based on ranger permutation importance	X	X	X	X	X	X
rank.correlation		Spearman’s correlation between feature and target		X			X
univariate.model.score		Resamples an mlr learner for each input feature individually. The resampling performance is used as filter score, with rpart as default learner.	X	X	X	X	X	X
variance		A simple variance filter	X	X	X		X

Ensemble methods

Name	Description
E-Borda	Borda ensemble filter. Takes the sum across all base filter methods for each feature.
E-max	Maximum ensemble filter. Takes the best maximum value across all base filter methods for each feature.
E-mean	Mean ensemble filter. Takes the mean across all base filter methods for each feature.
E-median	Median ensemble filter. Takes the median across all base filter methods for each feature.
E-min	Minimum ensemble filter. Takes the best minimum value across all base filter methods for each feature.

Deprecated methods

Method	Package	Description	Classif	Regr	Surv	Fac.	Num.	Ord.
cforest.importance	party	(DEPRECATED)	X	X	X	X	X	X
chi.squared	FSelector	(DEPRECATED)	X	X		X	X
gain.ratio	FSelector	(DEPRECATED)	X	X		X	X
information.gain	FSelector	(DEPRECATED)	X	X		X	X
oneR	FSelector	(DEPRECATED)	X	X		X	X
randomForest.importance	randomForest	(DEPRECATED)	X	X		X	X
randomForestSRC.rfsrc	randomForestSRC	(DEPRECATED)	X	X	X	X	X	X
randomForestSRC.var.select	randomForestSRC	(DEPRECATED)	X	X	X	X	X	X
ranger.impurity	ranger	(DEPRECATED)	X	X		X	X	X
ranger.permutation	ranger	(DEPRECATED)	X	X	X	X	X	X
relief	FSelector	(DEPRECATED)	X	X		X	X
rf.importance	randomForestSRC	(DEPRECATED)	X	X	X	X	X	X
rf.min.depth	randomForestSRC	(DEPRECATED)	X	X	X	X	X	X
symmetrical.uncertainty	FSelector	(DEPRECATED)	X	X		X	X
univariate		(DEPRECATED)	X	X	X	X	X	X