Create control object for hyperparameter tuning with Irace.
Source:R/TuneControlIrace.R
makeTuneControlIrace.Rd
Tuning with iterated F-Racing with method irace::irace. All
kinds of parameter types can be handled. We return the best of the final
elite candidates found by irace in the last race. Its estimated performance
is the mean of all evaluations ever done for that candidate. More information
on irace can be found in package vignette: vignette("irace-package", package = "irace")
For resampling you have to pass a ResampleDesc, not a ResampleInstance.
The resampling strategy is randomly instantiated n.instances
times and
these are the instances in the sense of irace (instances
element of
tunerConfig
in irace::irace). Also note that irace will always store its
tuning results in a file on disk, see the package documentation for details
on this and how to change the file path.
Usage
makeTuneControlIrace(
impute.val = NULL,
n.instances = 100L,
show.irace.output = FALSE,
tune.threshold = FALSE,
tune.threshold.args = list(),
log.fun = "default",
final.dw.perc = NULL,
budget = NULL,
...
)
Arguments
- impute.val
(numeric)
If something goes wrong during optimization (e.g. the learner crashes), this value is fed back to the tuner, so the tuning algorithm does not abort. Imputation is only active ifon.learner.error
is configured not to stop in configureMlr. It is not stored in the optimization path, an NA and a corresponding error message are logged instead. Note that this value is later multiplied by -1 for maximization measures internally, so you need to enter a larger positive value for maximization here as well. Default is the worst obtainable value of the performance measure you optimize for when you aggregate by mean value, orInf
instead. For multi-criteria optimization pass a vector of imputation values, one for each of your measures, in the same order as your measures.- n.instances
(
integer(1)
)
Number of random resampling instances for irace, see details. Default is 100.- show.irace.output
(
logical(1)
)
Show console output of irace while tuning? Default isFALSE
.- tune.threshold
(
logical(1)
)
Should the threshold be tuned for the measure at hand, after each hyperparameter evaluation, via tuneThreshold? Only works for classification if the predict type is “prob”. Default isFALSE
.- tune.threshold.args
(list)
Further arguments for threshold tuning that are passed down to tuneThreshold. Default is none.- log.fun
(
function
|character(1)
)
Function used for logging. If set to “default” (the default), the evaluated design points, the resulting performances, and the runtime will be reported. If set to “memory” the memory usage for each evaluation will also be displayed, withcharacter(1)
small increase in run time. Otherwisecharacter(1)
function with argumentslearner
,resampling
,measures
,par.set
,control
,opt.path
,dob
,x
,y
,remove.nas
,stage
andprev.stage
is expected. The default displays the performance measures, the time needed for evaluating, the currently used memory and the max memory ever used before (the latter two both taken from gc). See the implementation for details.- final.dw.perc
(
boolean
)
If a Learner wrapped by a makeDownsampleWrapper is used, you can define the value ofdw.perc
which is used to train the Learner with the final parameter setting found by the tuning. Default isNULL
which will not change anything.- budget
(
integer(1)
)
Maximum budget for tuning. This value restricts the number of function evaluations. It is passed tomaxExperiments
.- ...
(any)
Further control parameters passed to thecontrol
arguments of cmaes::cma_es or GenSA::GenSA, as well as towards thetunerConfig
argument of irace::irace.
See also
Other tune:
TuneControl
,
getNestedTuneResultsOptPathDf()
,
getNestedTuneResultsX()
,
getResamplingIndices()
,
getTuneResult()
,
makeModelMultiplexerParamSet()
,
makeModelMultiplexer()
,
makeTuneControlCMAES()
,
makeTuneControlDesign()
,
makeTuneControlGenSA()
,
makeTuneControlGrid()
,
makeTuneControlMBO()
,
makeTuneControlRandom()
,
makeTuneWrapper()
,
tuneParams()
,
tuneThreshold()