利用scikit-learn进行FeatureSelection

1.单变量特征选择(Univariate feature selection)

>>> from sklearn.datasets import load_iris
>>> from sklearn.feature_selection import SelectKBest
>>> from sklearn.feature_selection import chi2
>>> iris = load_iris()
>>> X, y = iris.data, iris.target
>>> X.shape
(150, 4)
>>> X_new = SelectKBest(chi2, k=2).fit_transform(X, y)
>>> X_new.shape
(150, 2)

除了使用SelectKBest之外，还可以使用SelectPercentile，她是按百分比进行选择的。

2.基于树的特征选择(Tree-based Feature Selection)

>>> from sklearn.ensemble import ExtraTreesClassifier
>>> from sklearn.datasets import load_iris
>>> iris = load_iris()
>>> X, y = iris.data, iris.target
>>> X.shape
(150, 4)
>>> clf = ExtraTreesClassifier()
>>> X_new = clf.fit(X, y).transform(X)
>>> clf.feature_importances_  
array([ 0.04...,  0.05...,  0.4...,  0.4...])
>>> X_new.shape               
(150, 2)

两种方法我都试过，感觉对于人群标签来说，第二种方法要好些。

参考资料：http://scikit-learn.org/dev/modules/feature_selection.html#feature-selection

                    http://scikit-learn.org/dev/auto_examples/ensemble/plot_forest_importances.html#example-ensemble-plot-forest-importances-py