參考：JMLR的paper《an introduction to variable and feature selection》

we summarize the steps that may be taken to solve a feature selection problem in a check list:

1. Do you have domain knowledge?

If yes, construct a better set of “ad hoc” features.

2. Are your features commensurate（能够同单位度量的）?

If no, consider normalizing them.

3. Do you suspect interdependence of features? If yes, expand your feature set by constructing conjunctive features or products of features（通过构建联合特征<应该是多个variables当做一个feature>或高次特征。扩展您的功能集）, as much as your computer resources
allow you(see example of use in Section 4.4).

4. Do you need to prune（裁剪） the input variables (e.g. for cost, speed or data understanding reasons)? If no, construct disjunctive features or weighted sums of features（构建析取特征<应该是一个variables当做一个feature>或加权和特征） (e.g. by clustering or matrix factorization, see
Section 5).

5. Do you need to assess features individually（单独评估每一个feature） (e.g. to understand their influence on the system or because their number is so large that you need to do a first filtering)? If yes, use a variable ranking method (Section 2 and Section 7.2); else,
do it anyway to get baseline results.

6. Do you need a predictor? If no, stop.

7. Do you suspect your data is “dirty” (has a few meaningless input patterns and/or noisy outputs or wrong class labels)? If yes, detect the outlier examples using the top ranking variables obtained in step 5 as representation; check and/or discard them（注意：这里的them是example的意思，不是feature。

。

。）.

8. Do you know what to try first? If no, use a linear predictor. Use a forward selection method(Section 4.2) with the “probe” method as a stopping criterion (Section 6) or use the L0-norm embedded
method (Section 4.3). For comparison, following the ranking of step 5, construct a sequence of predictors of same nature using increasing subsets of features. Can you match or improve performance with a smaller subset?

If yes, try a non-linear predictor with
that subset.

9. Do you have new ideas, time, computational resources, and enough examples? If yes, compare several feature selection methods, including your new idea, correlation coefficients, backward selection and embedded methods (Section 4). Use linear and non-linear
predictors. Select the best approach with model selection (Section 6).

10. Do you want a stable solution (to improve performance and/or understanding)? If yes, sub-sample your data and redo your analysis for several “bootstraps” (Section 7.1)

Section 2：describing
filters that select variables by ranking them with correlation coefficients.（经常使用的标准有皮尔逊相关系数、互信息等）

Section 3：Limitations of such approaches（filters） are illustrated by a set of constructed examples. （通过以上标准每次筛选一个“最好的”variable是有局限的，由于variables的组合往往能够比一个variable效果好。即使是看起来无用的variable。怎样和实用的variables结合。或者几个无用的variables结合，都能够provide
a significant performance improvement）

Section 4：Subset selection methods are then introduced. These include wrapper methods that assess subsets
of variables according to their usefulness to a given predictor（就是很easy的逐步添加或者候选消除：http://blog.csdn.net/mmc2015/article/details/47426437）. We show how some
embedded methods implement

the same idea, but proceed more efficiently by directly optimizing a two-part objective function with

a goodness-of-fit term and a penalty for a large number of variables（就是所谓的L0-norm、L1-norm等）.

Section 5：We then turn to the problem of feature construction, whose goals include increasing the predictor performance and building more compact
feature subsets. All of the previous steps benefit from reliably assessing the statistical significance of the relevance of features. （常见的方法有：聚类，本质思想是。将多个相似的variables用他们的聚类中心取代，最经常使用的是k-mean和层次聚类；矩阵分解法，本质思想是对输入的variables进行线性转换，如PCA/SVD/LDA等；非线性变换，kernel方法。

。。）

Section 6：We briefly review model selection methods and statistical tests used to that effect.

Section 7：Finally, we conclude the paper with a discussion section in which we go over more advanced issues.

we recommend using a linear predictor of your choice (e.g. a linear SVM) and select variables in two alternate ways: (1) with a variable ranking method using a correlation coefficient or mutual information; (2) with a nested
subset selection method performing forward or backward selection or with multiplicative updates