scikit-learn:3.2. Grid Search: Searching for estimator parameters
參考:http://scikit-learn.org/stable/modules/grid_search.html
GridSearchCV通过(蛮力)搜索參数空间(參数的全部可能组合)。寻找最好的 Cross-validation:
evaluating estimator performance score相应的超參数(翻译文章參考:http://blog.csdn.net/mmc2015/article/details/47099275)。比如Support
Vector Classifier的 C, kernel and gamma ,Lasso的alpha。etc。
A search consists of:
- an estimator (regressor or classifier such as sklearn.svm.SVC());
- a parameter space;
- a method for searching or sampling candidates;
- a cross-validation scheme
- a score
function.
RandomizedSearchCV 通过一定的分布sample候选參数。而不是搜索全部參数组合。
本节我们介绍 GridSearchCV、RandomizedSearchCV 、以及parameter
search的小Tips,最后介绍蛮力搜索的alternatives。
1、Exhaustive
Grid Search
GridSearchCV的參数param_grid定义搜索网格。
两个样例说明一切:
- See Parameter
estimation using grid search with cross-validation for an example of Grid Search computation on the digits dataset. - See Sample
pipeline for text feature extraction and evaluation for an example of Grid Search coupling parameters from a text documents feature extractor (n-gram count vectorizer and TF-IDF transformer) with a classifier (here a linear SVM trained with SGD with
either elastic net or L2 penalty) using a pipeline.Pipeline instance.
2、Randomized
Parameter Optimization
RandomizedSearchCV 通过在參数可能的取值的某个分布中sample一组參数。优点是:能够设定独立于參数(及全部取值)详细数量的一个搜索次数;加入无效的參数也不会减少效率。
搜索的次数通过 n_iter 设定,对于每个參数,假设是连续的取值。则通过一定的分布sample,假设是离散的取值,则通过uniform分布sample,比如:
[{'C': scipy.stats.expon(scale=100), 'gamma': scipy.stats.expon(scale=.1),
'kernel': ['rbf'], 'class_weight':['auto', None]}]
scipy.stats module提供了非常多用来sample參数的distributions,如expon, gamma, uniform or randint.
对于连续的參数,如 C ,一定要选择连续的分布来sample,而且适当增大 n_iter 通常会搜索到更好的參数组合。
给个样例:
- Comparing
randomized search and grid search for hyperparameter estimation compares the usage and efficiency of randomized search and grid search.
3、Tips
for parameter search(这几个建议很靠谱。。。
)
1)详细化目标函数
參数搜索默认使用score function(
即,分类用sklearn.metrics.accuracy_score 回归用sklearn.metrics.r2_score )来衡量參数的好坏对于有些应用(比方分类unbalance,score不是非常好的标准),通过详细化GridSearchCV和RandomizedSearchCV 的scoring parameter。See The
scoring parameter: defining model evaluation rules for more details.
2)综合estimators和parameter sapces(同一时候考虑预測器和參数空间)
Pipeline:
chaining estimators describes building composite estimators whose parameter space can be searched with these tools.
3)模型选择:先训练、再评估
用训练集选择模型。用測试集验证模型(using
the cross_validation.train_test_split utility
function.)(it is recommended to split the data into a development set (to be
fed to the GridSearchCV instance)
and an evaluation set to compute performance metrics.)
4)并行搜索
n_jobs=-1.
自己主动使用全部核。
5)robustness to failure(增强搜索错误的鲁棒性)
有些參数组合对于某些folds
of the data会failure,进而导致整个search failure,虽然其它的參数组合没有问题。
设定 error_score=0 (or =np.NaN)
能够使search过程忽略这种failure,只抛出一个warning,并将这种search结果设为0 (or =np.NaN)
,可以提高搜索遇到错误时的鲁棒性!
4、Alternatives
to brute force parameter search(没太看懂,还是不翻译了)
3.2.4.1. Model specific cross-validation
Some models can fit data for a range of value of some parameter almost as efficiently as fitting the estimator for a single value of the parameter. This feature can be leveraged to perform
a more efficient cross-validation used for model selection of this parameter.
The most common parameter amenable to this strategy is the parameter encoding the strength of the regularizer. In this case we say that we compute theregularization path of
the estimator.
Here is the list of such models:
linear_model.ElasticNetCV([l1_ratio, eps, ...]) | Elastic Net model with iterative fitting along a regularization path |
linear_model.LarsCV([fit_intercept, ...]) | Cross-validated Least Angle Regression model |
linear_model.LassoCV([eps, n_alphas, ...]) | Lasso linear model with iterative fitting along a regularization path |
linear_model.LassoLarsCV([fit_intercept, ...]) | Cross-validated Lasso, using the LARS algorithm |
linear_model.LogisticRegressionCV([Cs, ...]) | Logistic Regression CV (aka logit, MaxEnt) classifier. |
linear_model.MultiTaskElasticNetCV([...]) | Multi-task L1/L2 ElasticNet with built-in cross-validation. |
linear_model.MultiTaskLassoCV([eps, ...]) | Multi-task L1/L2 Lasso with built-in cross-validation. |
linear_model.OrthogonalMatchingPursuitCV([...]) | Cross-validated Orthogonal Matching Pursuit model (OMP) |
linear_model.RidgeCV([alphas, ...]) | Ridge regression with built-in cross-validation. |
linear_model.RidgeClassifierCV([alphas, ...]) | Ridge classifier with built-in cross-validation. |
3.2.4.2. Information Criterion
Some models can offer an information-theoretic closed-form formula of the optimal estimate of the regularization parameter by computing a single regularization path (instead of several when
using cross-validation).
Here is the list of models benefitting from the Aikike Information Criterion (AIC) or the Bayesian Information Criterion (BIC) for automated model selection:
linear_model.LassoLarsIC([criterion, ...]) | Lasso model fit with Lars using BIC or AIC for model selection |
3.2.4.3. Out of Bag Estimates
When using ensemble methods base upon bagging, i.e. generating new training sets using sampling with replacement, part of the training set remains unused. For each classifier in the ensemble,
a different part of the training set is left out.
This left out portion can be used to estimate the generalization error without having to rely on a separate validation set. This estimate comes “for free” as no additional data is needed and
can be used for model selection.
This is currently implemented in the following classes:
ensemble.RandomForestClassifier([...]) | A random forest classifier. |
ensemble.RandomForestRegressor([...]) | A random forest regressor. |
ensemble.ExtraTreesClassifier([...]) | An extra-trees classifier. |
ensemble.ExtraTreesRegressor([n_estimators, ...]) | An extra-trees regressor. |
ensemble.GradientBoostingClassifier([loss, ...]) | Gradient Boosting for classification. |
ensemble.GradientBoostingRegressor([loss, ...]) | Gradient Boosting for regression. |
最新文章
- Echart地图城市用json返回格式
- Linux关于vm虚拟机复制后无法启动网卡
- java读写Properties文件
- WP8 SqlCE和SqlLite数据存储性能比较
- 检测是否支持HTML5中的Video标签
- WPF中的DataTemplate
- sdut 2846 Remove Trees (二分 + 贪心)
- 对PostgreSQL中tablespace 与 database, table的理解
- 【打包成exe安装包文件发布你的程序】使用QT联系人管理系统的例子
- 【HDOJ】4612 Warm up
- 四句话表明JSON格式定义
- 各种类型Android源代码
- IC卡
- WPF 自定义 MessageBox (相对完善版)
- 【前端】js截取or分割字符串的常见方法
- npm install 报错(npm ERR! errno -4048,Error: EPERM: operation not permitted,)解决方法
- 061 hive中的三种join与数据倾斜
- PTA L2-002 链表去重
- SystemProperties.get/set property_get/set
- 安卓开发笔记——WebView组件
热门文章
- Java线程之Java内存模型(jmm)
- [agc016d]xor replace
- LightOJ-1138 Trailing Zeroes (III) 唯一分解定理 算n!的某个因数个数
- BZOJ 5394 [Ynoi2016]炸脖龙 (线段树+拓展欧拉定理)
- 脚本2,从1到99 ,添加用户user1,user2,。。。 user99
- java用freemarker实现导出word----包含图片
- HDU 1026 Ignatius and the Princess I(BFS+记录路径)
- [poj 2912] Rochambeau 解题报告 (带权并查集)
- 使用tinyxml2库解析xml
- Android项目实战(五十六):获取WebView加载的url的请求错误码