5. Model selection and evaluation

• 5.1. Cross-Validation: evaluating estimator performance
  • 5.1.1. Computing cross-validated metrics
  • 5.1.2. Cross validation iterators
    • 5.1.2.1. K-fold
    • 5.1.2.2. Stratified K-Fold
    • 5.1.2.3. Leave-One-Out - LOO
    • 5.1.2.4. Leave-P-Out - LPO
    • 5.1.2.5. Leave-One-Label-Out - LOLO
    • 5.1.2.6. Leave-P-Label-Out
    • 5.1.2.7. Random permutations cross-validation a.k.a. Shuffle & Split
    • 5.1.2.8. See also
    • 5.1.2.9. Bootstrapping cross-validation
  • 5.1.3. Cross validation and model selection
• 5.2. Grid Search: setting estimator parameters
  • 5.2.1. GridSearchCV
  • 5.2.2. Examples
  • 5.2.3. Alternatives to brute force grid search
    • 5.2.3.1. Model specific cross-validation
      • 8.17.25. sklearn.linear_model.RidgeCV
      • 8.17.24. sklearn.linear_model.RidgeClassifierCV
      • 8.17.6. sklearn.linear_model.LarsCV
      • 8.17.10. sklearn.linear_model.LassoLarsCV
      • 8.17.8. sklearn.linear_model.LassoCV
      • 8.17.4. sklearn.linear_model.ElasticNetCV
    • 5.2.3.2. Information Criterion
      • 8.17.11. sklearn.linear_model.LassoLarsIC
    • 5.2.3.3. Out of Bag Estimates
      • 8.7.1. sklearn.ensemble.RandomForestClassifier
      • 8.7.3. sklearn.ensemble.RandomForestRegressor
      • 8.7.4. sklearn.ensemble.ExtraTreesClassifier
      • 8.7.5. sklearn.ensemble.ExtraTreesRegressor
      • 8.7.6. sklearn.ensemble.GradientBoostingClassifier
      • 8.7.7. sklearn.ensemble.GradientBoostingRegressor
• 5.3. Pipeline: chaining estimators
  • 5.3.1. Usage
  • 5.3.2. Notes
• 5.4. FeatureUnion: Combining feature extractors
  • 5.4.1. Usage
• 5.5. Model evaluation
  • 5.5.1. Classification metrics
    • 5.5.1.1. Accuracy score
    • 5.5.1.2. Area under the curve (AUC)
    • 5.5.1.3. Average precision score
    • 5.5.1.4. Confusion matrix
    • 5.5.1.5. Classification report
    • 5.5.1.6. Precision, recall and F-measures
      • 5.5.1.6.1. Binary classification
      • 5.5.1.6.2. Multiclass and multilabels classification
    • 5.5.1.7. Hinge loss
    • 5.5.1.8. Matthews correlation coefficient
    • 5.5.1.9. Receiver operating characteristic (ROC)
    • 5.5.1.10. Zero one loss
  • 5.5.2. Regression metrics
    • 5.5.2.1. Explained variance score
    • 5.5.2.2. Mean absolute error
    • 5.5.2.3. Mean squared error
    • 5.5.2.4. R² score, the coefficient of determination
  • 5.5.3. Clustering metrics
  • 5.5.4. Dummy estimators