mvpa.algorithms.cvtranserror

Cross-validate a classifier on a dataset

The comprehensive API documentation for this module, including all technical details, is available in the Epydoc-generated API reference for mvpa.algorithms.cvtranserror (for developers).

CrossValidatedTransferError

class mvpa.algorithms.cvtranserror.CrossValidatedTransferError(transerror, splitter=None, combiner='mean', expose_testdataset=False, harvest_attribs=None, copy_attribs='copy', **kwargs)

Bases: mvpa.measures.base.DatasetMeasure, mvpa.misc.state.Harvestable

Classifier cross-validation.

This class provides a simple interface to cross-validate a classifier on datasets generated by a splitter from a single source dataset.

Arbitrary performance/error values can be computed by specifying an error function (used to compute an error value for each cross-validation fold) and a combiner function that aggregates all computed error values across cross-validation folds.

Note

Available state variables:

• confusion: Store total confusion matrix (if available)
• harvested: Store specified attributes of classifiers at each split
• null_prob+: State variable
• null_t: State variable
• raw_result: Computed results before applying any transformation algorithm
• results: Store individual results in the state
• samples_error: Per sample errors.
• splits: Store the actual splits of the data. Can be memory expensive
• training_confusion: Store total training confusion matrix (if available)
• transerrors: Store copies of transerrors at each step

(States enabled by default are listed with +)

See also

Please refer to the documentation of the base classes for more information:

DatasetMeasure, Harvestable

Parameters:

• transerror (TransferError instance) – Provides the classifier used for cross-validation.
• splitter (Splitter | None) – Used to split the dataset for cross-validation folds. By convention the first dataset in the tuple returned by the splitter is used to train the provided classifier. If the first element is ‘None’ no training is performed. The second dataset is used to generate predictions with the (trained) classifier. If None (default) an instance of NoneSplitter is used.
• combiner (Functor | ‘mean’) – Used to aggregate the error values of all cross-validation folds. If ‘mean’ (default) the grand mean of the transfer errors is computed.
• expose_testdataset (bool) – In the proper pipeline, classifier must not know anything about testing data, but in some cases it might lead only to marginal harm, thus migth wanted to be enabled (provide testdataset for RFE to determine stopping point).
• harvest_attribs (list of basestr) – What attributes of call to store and return within harvested state variable
• copy_attribs (None | basestr) – Force copying values of attributes on harvesting
• enable_states (None or list of basestring) – Names of the state variables which should be enabled additionally to default ones
• disable_states (None or list of basestring) – Names of the state variables which should be disabled

combiner

Access to the configured combiner.

splitter

Access to the Splitter instance.

transerror

Access to the TransferError instance.

See also

Derived classes might provide additional methods via their base classes. Please refer to the list of base classes (if it exists) at the begining of the CrossValidatedTransferError documentation.

Full API documentation of CrossValidatedTransferError in module mvpa.algorithms.cvtranserror.