Construct your own resampled performance measure.

Construct your own performance measure, used after resampling. Note that individual training / test set performance values will be set to NA, you only calculate an aggregated value. If you can define a function that makes sense for every single training / test set, implement your own Measure.

Usage

makeCustomResampledMeasure(
  measure.id,
  aggregation.id,
  minimize = TRUE,
  properties = character(0L),
  fun,
  extra.args = list(),
  best = NULL,
  worst = NULL,
  measure.name = measure.id,
  aggregation.name = aggregation.id,
  note = ""
)

Arguments

measure.id

(character(1))
Short name of measure.

aggregation.id

(character(1))
Short name of aggregation.

minimize

(logical(1))
Should the measure be minimized? Default is TRUE.

properties

(character)
Set of measure properties. For a list of values see Measure. Default is character(0).

fun

(function(task, group, pred, extra.args))
Calculates performance value from ResamplePrediction object. For rare cases you can also use the task, the grouping or the extra arguments extra.args. - task (Task)
The task. - group (factor)
Grouping of resampling iterations. This encodes whether specific iterations 'belong together' (e.g. repeated CV). - pred (Prediction)
Prediction object. - extra.args (list)
See below.

extra.args

(list)
List of extra arguments which will always be passed to fun. Default is empty list.

best

(numeric(1))
Best obtainable value for measure. Default is -Inf or Inf, depending on minimize.

worst

(numeric(1))
Worst obtainable value for measure. Default is Inf or -Inf, depending on minimize.

measure.name

(character(1))
Long name of measure. Default is measure.id.

aggregation.name

(character(1))
Long name of the aggregation. Default is aggregation.id.

note

(character)
Description and additional notes for the measure. Default is “”.

Value

Measure.

See also

Other performance: ConfusionMatrix, calculateConfusionMatrix(), calculateROCMeasures(), estimateRelativeOverfitting(), makeCostMeasure(), makeMeasure(), measures, performance(), setAggregation(), setMeasurePars()