`R/resample.R`

, `R/resample_convenience.R`

`resample.Rd`

The function `resample`

fits a model specified by Learner on a Task
and calculates predictions and performance measures for all training
and all test sets specified by a either a resampling description (ResampleDesc)
or resampling instance (ResampleInstance).

You are able to return all fitted models (parameter `models`

) or extract specific parts
of the models (parameter `extract`

) as returning all of them completely
might be memory intensive.

The remaining functions on this page are convenience wrappers for the various
existing resampling strategies. Note that if you need to work with precomputed training and
test splits (i.e., resampling instances), you have to stick with `resample`

.

resample(learner, task, resampling, measures, weights = NULL, models = FALSE, extract, keep.pred = TRUE, ..., show.info = getMlrOption("show.info")) crossval(learner, task, iters = 10L, stratify = FALSE, measures, models = FALSE, keep.pred = TRUE, ..., show.info = getMlrOption("show.info")) repcv(learner, task, folds = 10L, reps = 10L, stratify = FALSE, measures, models = FALSE, keep.pred = TRUE, ..., show.info = getMlrOption("show.info")) holdout(learner, task, split = 2/3, stratify = FALSE, measures, models = FALSE, keep.pred = TRUE, ..., show.info = getMlrOption("show.info")) subsample(learner, task, iters = 30, split = 2/3, stratify = FALSE, measures, models = FALSE, keep.pred = TRUE, ..., show.info = getMlrOption("show.info")) bootstrapOOB(learner, task, iters = 30, stratify = FALSE, measures, models = FALSE, keep.pred = TRUE, ..., show.info = getMlrOption("show.info")) bootstrapB632(learner, task, iters = 30, stratify = FALSE, measures, models = FALSE, keep.pred = TRUE, ..., show.info = getMlrOption("show.info")) bootstrapB632plus(learner, task, iters = 30, stratify = FALSE, measures, models = FALSE, keep.pred = TRUE, ..., show.info = getMlrOption("show.info")) growingcv(learner, task, horizon = 1, initial.window = 0.5, skip = 0, measures, models = FALSE, keep.pred = TRUE, ..., show.info = getMlrOption("show.info")) fixedcv(learner, task, horizon = 1L, initial.window = 0.5, skip = 0, measures, models = FALSE, keep.pred = TRUE, ..., show.info = getMlrOption("show.info"))

learner | (Learner | |
---|---|

task | (Task) |

resampling | (ResampleDesc or ResampleInstance) |

measures | (Measure | list of Measure) |

weights | (numeric) |

models | ( |

extract | ( |

keep.pred | ( |

... | (any) |

show.info | ( |

iters | ( |

stratify | ( |

folds | ( |

reps | ( |

split | ( |

horizon | ( |

initial.window | ( |

skip | ( |

If you would like to include results from the training data set, make sure to appropriately adjust the resampling strategy and the aggregation for the measure. See example code below.

Other resample: `ResamplePrediction`

,
`ResampleResult`

, `addRRMeasure`

,
`getRRPredictionList`

,
`getRRPredictions`

,
`getRRTaskDescription`

,
`getRRTaskDesc`

,
`makeResampleDesc`

,
`makeResampleInstance`

task = makeClassifTask(data = iris, target = "Species") rdesc = makeResampleDesc("CV", iters = 2) r = resample(makeLearner("classif.qda"), task, rdesc)#>#>#>#>#>#>#>#> mmce.test.mean #> 0.04666667#> iter mmce #> 1 1 0.02666667 #> 2 2 0.06666667#> Resampled Prediction for: #> Resample description: cross-validation with 2 iterations. #> Predict: test #> Stratification: FALSE #> predict.type: response #> threshold: #> time (mean): 0.00 #> id truth response iter set #> 1 2 setosa setosa 1 test #> 2 4 setosa setosa 1 test #> 3 5 setosa setosa 1 test #> 4 6 setosa setosa 1 test #> 5 9 setosa setosa 1 test #> 6 15 setosa setosa 1 test #> ... (#rows: 150, #cols: 5)# include the training set performance as well rdesc = makeResampleDesc("CV", iters = 2, predict = "both") r = resample(makeLearner("classif.qda"), task, rdesc, measures = list(mmce, setAggregation(mmce, train.mean)))#>#>#>#>#>#>#>#> mmce.test.mean mmce.train.mean #> 0.026666667 0.006666667