Calculates the standard errors of a given statistic using bootstrap
se_boot(Xk, dif = "simple", comparisons = "all.vs.all", boot.R = 999, ...)list object where each position contains a vector of observations of algorithm k on a given problem instance.
name of the difference for which the SEs are desired. Accepts "perc" (for percent differences) or "simple" (for simple differences)
standard errors to be calculated. Accepts "all.vs.first"
(in which cases the first object in algorithms is considered to be
the reference algorithm) or "all.vs.all" (if there is no reference
and all pairwise SEs are desired).
(optional) number of bootstrap resamples
(if method == "boot")
other parameters (used only for compatibility with calls to
se_param(), unused in this function)
Data frame containing, for each pair of interest, the estimated difference (column "Phi") and the sample standard error (column "SE")
A.C. Davison, D.V. Hinkley: Bootstrap methods and their application. Cambridge University Press (1997)
F. Campelo, F. Takahashi: Sample size estimation for power and accuracy in the experimental comparison of algorithms. Journal of Heuristics 25(2):305-338, 2019.
# three vectors of normally distributed observations
set.seed(1234)
Xk <- list(rnorm(10, 5, 1), # mean = 5, sd = 1,
rnorm(20, 10, 2), # mean = 10, sd = 2,
rnorm(20, 15, 5)) # mean = 15, sd = 3
se_boot(Xk, dif = "simple", comparisons = "all.vs.all")
#> Alg1 Alg2 N1 N2 Phi SE r ropt
#> 1 1 2 10 20 -4.886459 0.4743732 0.5 0.5991083
#> 2 1 3 10 20 -6.975376 0.9595352 0.5 0.2486304
#> 3 2 3 20 20 -2.041556 0.9921207 1.0 0.4610079
se_boot(Xk, dif = "perc", comparisons = "all.vs.first")
#> Alg1 Alg2 N1 N2 Phi SE r ropt
#> 1 1 2 10 20 -1.052635 0.1553476 0.5 1.2321237
#> 2 1 3 10 20 -1.500993 0.2637684 0.5 0.6294316
se_boot(Xk, dif = "perc", comparisons = "all.vs.all")
#> Alg1 Alg2 N1 N2 Phi SE r ropt
#> 1 1 2 10 20 -0.5708437 0.05924336 0.5 0.5923848
#> 2 1 3 10 20 -0.8092254 0.08902371 0.5 0.2523093
#> 3 2 3 20 20 -0.2448666 0.10754158 1.0 0.4466126