Skip to contents

Bootstrapped Standardized Mean Difference (SMD) Calculation

boot_smd_calc.Rd

[Maturing]

Calculates standardized mean differences (SMDs) with bootstrap confidence intervals, with optional hypothesis testing.

Usage

boot_smd_calc(
  x,
  ...,
  paired = FALSE,
  var.equal = FALSE,
  alpha = 0.05,
  bias_correction = TRUE,
  rm_correction = FALSE,
  glass = NULL,
  denom = c("auto", "z", "rm", "pooled", "avg", "glass1", "glass2"),
  boot_ci = c("stud", "basic", "perc", "bca"),
  R = 1999,
  output = c("htest", "data.frame"),
  null.value = 0,
  alternative = c("none", "two.sided", "less", "greater", "equivalence",
    "minimal.effect"),
  tr = 0
)

# Default S3 method
boot_smd_calc(
  x,
  y = NULL,
  paired = FALSE,
  var.equal = FALSE,
  alpha = 0.05,
  mu = 0,
  bias_correction = TRUE,
  rm_correction = FALSE,
  glass = NULL,
  denom = c("auto", "z", "rm", "pooled", "avg", "glass1", "glass2"),
  boot_ci = c("stud", "basic", "perc", "bca"),
  R = 1999,
  output = c("htest", "data.frame"),
  null.value = 0,
  alternative = c("none", "two.sided", "less", "greater", "equivalence",
    "minimal.effect"),
  tr = 0,
  ...
)

# S3 method for class 'formula'
boot_smd_calc(formula, data, subset, na.action, ...)

Arguments

x

a (non-empty) numeric vector of data values.

...

further arguments to be passed to or from methods.

paired

a logical indicating whether you want a paired t-test. Cannot be used with the formula method; use x and y vectors instead for paired tests.

var.equal

a logical variable indicating whether to treat the two variances as being equal. If TRUE then the pooled variance is used to estimate the variance otherwise the Welch (or Satterthwaite) approximation to the degrees of freedom is used.

alpha

alpha level (default = 0.05)

bias_correction

Apply Hedges' correction for bias (default is TRUE).

rm_correction

Repeated measures correction to make standardized mean difference Cohen's d(rm). This only applies to repeated/paired samples. Default is FALSE.

glass

Option to calculate Glass's delta instead of Cohen's d style SMD ('glass1' uses first group's SD, 'glass2' uses second group's SD).

denom

a character string specifying the denominator for standardization: - "auto": (default) Uses the standard denominator based on design and other arguments (glass, rm_correction, var.equal). - "z": SD of differences (Cohen's d_z). Valid for paired and one-sample designs. - "rm": Repeated-measures corrected (Cohen's d_rm). Valid for paired designs only. - "pooled": Pooled SD (Cohen's d_s). Valid for independent samples only. - "avg": Root-mean-square SD (Cohen's d_av). Valid for independent samples only. - "glass1": First group's (x) SD (Glass's delta). Valid for paired and independent designs. - "glass2": Second group's (y) SD (Glass's delta). Valid for paired and independent designs.

When set to any value other than "auto", this overrides the glass, rm_correction,
and var.equal arguments. The bias_correction argument is not affected.

boot_ci

method for bootstrap confidence interval calculation: "stud" (studentized, default), "basic" (basic bootstrap), "bca" (bias-corrected and accelerated), or "perc" (percentile bootstrap).

R

number of bootstrap replications (default = 1999).

output

a character string specifying the output format: - "htest": (default) Returns an object of class "htest" compatible with standard R output. - "data.frame": Returns a data frame for backward compatibility.

null.value

a number or vector specifying the null hypothesis value(s) on the SMD scale: - For standard alternatives: a single value (default = 0) - For equivalence/minimal.effect: two values representing the lower and upper bounds

alternative

a character string specifying the alternative hypothesis: - "none": (default) No hypothesis test is performed; only effect size and CI are returned. - "two.sided": Test whether SMD differs from null.value - "less": Test whether SMD is less than null.value - "greater": Test whether SMD is greater than null.value - "equivalence": Test whether SMD is between specified bounds - "minimal.effect": Test whether SMD is outside specified bounds

tr

a numeric value specifying the proportion of observations to trim from each tail when computing trimmed means and Winsorized variances (default = 0, no trimming). Must be in the range [0, 0.5). Common choices are 0.1 (10\ (20\ and the rescaled Winsorized standard deviation for the denominator, following Algina, Keselman, and Penfield (2005). The rescaling ensures the robust effect size equals Cohen's delta when data are normally distributed. Note: tr > 0 is not compatible with denom = "rm".

y

an optional (non-empty) numeric vector of data values.

mu

null value to adjust the calculation. If non-zero, the function calculates x-y-mu (default = 0).

formula

a formula of the form lhs ~ rhs where lhs is a numeric variable giving the data values and rhs either 1 for a one-sample test or a factor with two levels giving the corresponding groups. For paired tests, use the default method with x and y vectors instead of the formula method.

data

an optional matrix or data frame (or similar: see model.frame) containing the variables in the formula formula. By default the variables are taken from environment(formula).

subset

an optional vector specifying a subset of observations to be used.

na.action

a function indicating what should happen when the data contain NAs.

Value

If output = "htest" (default), returns a list with class "htest" containing:

  • estimate: The SMD estimate (Cohen's d, Hedges' g, or Glass's delta)

  • stderr: Standard error estimated from the bootstrap distribution

  • conf.int: Bootstrap confidence interval with conf.level attribute

  • alternative: A character string describing the alternative hypothesis

  • method: A character string indicating what type of test was performed

  • note: A character string describing the bootstrap CI method used

  • boot: The bootstrap distribution of SMD estimates

  • data.name: A character string giving the name(s) of the data

  • call: The matched call

  • statistic: z-statistic (only if alternative != "none")

  • p.value: Bootstrap p-value (only if alternative != "none")

  • null.value: The specified hypothesized value(s) (only if alternative != "none")

If output = "data.frame", returns a data frame containing:

  • estimate: The SMD calculated from the original data

  • bias: Estimated bias (difference between original estimate and median of bootstrap estimates)

  • SE: Standard error estimated from the bootstrap distribution

  • lower.ci: Lower bound of the bootstrap confidence interval

  • upper.ci: Upper bound of the bootstrap confidence interval

  • conf.level: Confidence level (1-alpha)

  • boot_ci: The bootstrap confidence interval method used

Details

This function calculates bootstrapped confidence intervals for standardized mean differences. It is an extension of the smd_calc() function that uses resampling to provide more robust confidence intervals, especially for small sample sizes or when data violate assumptions of parametric methods.

The function implements the following bootstrap approach:

  • Calculate the raw SMD and its standard error using the original data

  • Create R bootstrap samples by resampling with replacement from the original data

  • Calculate the SMD and its standard error for each bootstrap sample

  • Calculate confidence intervals using the specified method

Four bootstrap confidence interval methods are available via the boot_ci argument:

  • Studentized bootstrap ("stud"): Uses the bootstrap distribution of pivotal t-statistics to account for variability in standard error estimates. Usually provides the most accurate coverage probability and is set as the default.

  • Basic bootstrap ("basic"): Reflects the bootstrap distribution of estimates around the observed value. Simple approach that works well for symmetric distributions.

  • Percentile bootstrap ("perc"): Uses percentiles of the bootstrap distribution directly. More robust to skewness in the bootstrap distribution.

  • Bias-corrected and accelerated ("bca"): Corrects for both bias and skewness in the bootstrap distribution using jackknife-based acceleration. Most accurate when the bootstrap distribution is skewed, but computationally more expensive.

When hypothesis testing is requested (i.e., alternative is not "none"), the p-value is computed using the method that matches the selected boot_ci, ensuring that p < alpha if and only if the corresponding confidence interval excludes the null value (CI inversion principle). Previously, all bootstrap CI methods used the studentized (pivot) p-value, which could produce p-values inconsistent with non-studentized CIs.

The function supports various SMD variants:

  • Classic standardized mean difference (bias_correction = FALSE)

  • Bias-corrected version (bias_correction = TRUE)

  • Glass's delta: Uses only one group's standard deviation as the denominator (glass = "glass1" or "glass2")

  • Repeated measures d: Accounts for correlation in paired designs (rm_correction = TRUE)

The function supports three study designs:

  • One-sample design: Standardizes the difference between the sample mean and zero (or other specified value)

  • Two-sample independent design: Standardizes the difference between two group means

  • Paired samples design: Standardizes the mean difference between paired observations

The denom parameter provides a direct way to select the standardization denominator. When denom is not "auto", it takes precedence over the glass, rm_correction, and var.equal arguments, which are overridden as needed. A message is emitted if any explicitly provided arguments are overridden. The bias_correction argument is always respected regardless of denom.

For detailed information on calculation methods, see vignette("SMD_calcs").

Purpose

Use this function when:

  • You need more robust confidence intervals for standardized mean differences

  • You want to account for non-normality or heterogeneity in your effect size estimates

  • Sample sizes are small or standard error approximations may be unreliable

  • You prefer resampling-based confidence intervals over parametric approximations

  • You need to quantify uncertainty in SMD estimates more accurately

  • You want to test hypotheses about effect size magnitudes using bootstrap methods

References

Algina, J., Keselman, H. J., & Penfield, R. D. (2005). An alternative to Cohen's standardized mean difference effect size: A robust parameter and confidence interval in the two independent groups case. Psychological Methods, 10(3), 317-328.

See also

Other effect sizes: boot_ses_calc(), rank_diff(), ses_calc(), smd_calc(), trans_rank_prob()

Examples

# Example 1: Independent groups comparison with studentized bootstrap CI
set.seed(123)
group1 <- rnorm(30, mean = 100, sd = 15)
group2 <- rnorm(30, mean = 110, sd = 18)

# Use fewer bootstrap replicates for a quick example
result <- boot_smd_calc(x = group1, y = group2,
                      boot_ci = "stud",
                      R = 999)

# Example 2: Using formula notation with basic bootstrap and Hedges' g
df <- data.frame(
  value = c(group1, group2),
  group = factor(rep(c("A", "B"), each = 30))
)
result <- boot_smd_calc(formula = value ~ group,
                      data = df,
                      boot_ci = "basic",
                      bias_correction = TRUE,
                      R = 999)

# Example 3: Paired samples with percentile bootstrap
set.seed(456)
before <- rnorm(30)
after <- rnorm(30)
result <- boot_smd_calc(x = before,
                      y = after,
                      paired = TRUE,
                      boot_ci = "perc",
                      R = 999)

# Example 4: Glass's delta with homogeneous variances
set.seed(456)
control <- rnorm(25, mean = 50, sd = 10)
treatment <- rnorm(25, mean = 60, sd = 10)
result <- boot_smd_calc(x = control,
                      y = treatment,
                      glass = "glass1",
                      boot_ci = "stud",
                      R = 999)

# Example 5: Two-sided hypothesis test
result <- boot_smd_calc(x = group1, y = group2,
                      alternative = "two.sided",
                      null.value = 0,
                      R = 999)

# Example 6: Equivalence test with bootstrap
result <- boot_smd_calc(x = group1, y = group2,
                      alternative = "equivalence",
                      null.value = c(-0.5, 0.5),
                      R = 999)

# Example 7: Legacy data.frame output
result <- boot_smd_calc(x = group1, y = group2,
                      output = "data.frame",
                      R = 999)