Skip to contents

TOST with t-tests from Summary Statistics

tsum_TOST.Rd

[Stable]

Performs equivalence testing using the Two One-Sided Tests (TOST) procedure with t-tests based on summary statistics rather than raw data. This function allows TOST analysis when only descriptive statistics are available from published studies or reports.

Usage

tsum_TOST(
  m1,
  sd1,
  n1,
  m2 = NULL,
  sd2 = NULL,
  n2 = NULL,
  r12 = NULL,
  hypothesis = c("EQU", "MET"),
  paired = FALSE,
  var.equal = FALSE,
  eqb,
  low_eqbound,
  high_eqbound,
  mu = 0,
  eqbound_type = c("raw", "SMD"),
  alpha = 0.05,
  bias_correction = TRUE,
  rm_correction = FALSE,
  glass = NULL,
  smd_ci = c("nct", "goulet", "t", "z")
)

Arguments

m1

mean of group 1.

sd1

standard deviation of group 1.

n1

sample size in group 1.

m2

mean of group 2 (not required for one-sample tests).

sd2

standard deviation of group 2 (not required for one-sample tests).

n2

sample size in group 2 (not required for one-sample tests).

r12

correlation between measurements for paired designs. Required when paired = TRUE.

hypothesis

'EQU' for equivalence (default), or 'MET' for minimal effects test.

paired

a logical indicating whether you want a paired t-test.

var.equal

a logical variable indicating whether to treat the two variances as being equal. If TRUE then the pooled variance is used to estimate the variance otherwise the Welch (or Satterthwaite) approximation to the degrees of freedom is used.

eqb

Equivalence bound. Can provide 1 value (symmetric bound, negative value is taken as the lower bound) or 2 specific values that represent the upper and lower equivalence bounds.

low_eqbound

lower equivalence bounds (deprecated, use eqb instead).

high_eqbound

upper equivalence bounds (deprecated, use eqb instead).

mu

a number indicating the true value of the mean for the two-tailed test (or difference in means if you are performing a two sample test).

eqbound_type

Type of equivalence bound. Can be 'SMD' for standardized mean difference (i.e., Cohen's d) or 'raw' for the mean difference. Default is 'raw'. Raw is strongly recommended as SMD bounds will produce biased results.

alpha

alpha level (default = 0.05)

bias_correction

Apply Hedges' correction for bias (default is TRUE).

rm_correction

Repeated measures correction to make standardized mean difference Cohen's d(rm). This only applies to repeated/paired samples. Default is FALSE.

glass

An option to calculate Glass's delta as an alternative to Cohen's d type SMD. Default is NULL to not calculate Glass's delta, 'glass1' will use the first group's SD as the denominator whereas 'glass2' will use the 2nd group's SD.

smd_ci

Method for calculating SMD confidence intervals. Methods include 'goulet', 'noncentral t' (nct), 'central t' (t), and 'normal method' (z).

Value

An S3 object of class "TOSTt" is returned containing the following slots:

  • TOST: A table of class "data.frame" containing two-tailed t-test and both one-tailed results.

  • eqb: A table of class "data.frame" containing equivalence bound settings.

  • effsize: Table of class "data.frame" containing effect size estimates.

  • hypothesis: String stating the hypothesis being tested.

  • smd: List containing the results of the standardized mean difference calculations (e.g., Cohen's d).

    • Items include: d (estimate), dlow (lower CI bound), dhigh (upper CI bound), d_df (degrees of freedom for SMD), d_sigma (SE), d_lambda (non-centrality), J (bias correction), smd_label (type of SMD), d_denom (denominator calculation).

  • alpha: Alpha level set for the analysis.

  • method: Type of t-test.

  • decision: List included text regarding the decisions for statistical inference.

Details

This function performs TOST equivalence testing using summary statistics instead of raw data. It is particularly useful when analyzing published results or conducting meta-analyses where only summary statistics are available.

The function supports three types of tests:

  • One-sample test: Provide m1, sd1, and n1 only

  • Two-sample independent test: Provide all parameters except r12, with paired = FALSE

  • Paired samples test: Provide all parameters including r12, with paired = TRUE

For two-sample tests, the test is of \(m1 - m2\) (mean of group 1 minus mean of group 2). For paired samples, the test is of the difference scores, wherein \(z = m1 - m2\), and the test is of \(\bar{z}\) (mean of the difference scores). For one-sample tests, the test is of \(\bar{m1}\) (mean of group 1).

The function calculates both raw mean differences and standardized effect sizes (Cohen's d or Hedges' g), along with their confidence intervals.

For details on the calculations in this function see vignette("IntroTOSTt") & vignette("SMD_calcs").

Purpose

Use this function when:

  • You only have access to summary statistics (means, standard deviations, sample sizes)

  • You want to perform meta-analyses using published results

  • You're conducting power analyses based on previous studies

  • You need to reanalyze published results within an equivalence testing framework

See also

Other TOST: boot_log_TOST(), boot_t_TOST(), simple_htest(), t_TOST(), wilcox_TOST()

Examples

# Example 1: One-sample test
# Testing if a sample with mean 0.55 and SD 4 (n=18) is equivalent to zero within ±2 units
tsum_TOST(m1 = 0.55, n1 = 18, sd1 = 4, eqb = 2)
#> 
#> One-sample t-test
#> 
#> The equivalence test was non-significant, t(17) = -1.538, p = 7.12e-02
#> The null hypothesis test was non-significant, t(17) = 0.583, p = 5.67e-01
#> NHST: don't reject null significance hypothesis that the effect is equal to zero 
#> TOST: don't reject null equivalence hypothesis
#> 
#> TOST Results 
#>                  t df p.value
#> t-test      0.5834 17   0.567
#> TOST Lower  2.7047 17   0.008
#> TOST Upper -1.5380 17   0.071
#> 
#> Effect Sizes 
#>            Estimate     SE              C.I. Conf. Level
#> Raw          0.5500 0.9428 [-1.0901, 2.1901]         0.9
#> Hedges's g   0.1313 0.2367 [-0.2427, 0.5015]         0.9
#> Note: SMD confidence intervals are an approximation. See vignette("SMD_calcs").

# Example 2: Two-sample independent test
# Testing if two groups with different means are equivalent within ±3 units
tsum_TOST(m1 = 15.2, sd1 = 5.3, n1 = 30,
         m2 = 13.8, sd2 = 4.9, n2 = 28,
         eqb = 3)
#> 
#> Welch Two Sample t-test
#> 
#> The equivalence test was non-significant, t(56) = -1.195, p = 1.19e-01
#> The null hypothesis test was non-significant, t(56) = 1.045, p = 3e-01
#> NHST: don't reject null significance hypothesis that the effect is equal to zero 
#> TOST: don't reject null equivalence hypothesis
#> 
#> TOST Results 
#>                 t df p.value
#> t-test      1.045 56     0.3
#> TOST Lower  3.285 56 < 0.001
#> TOST Upper -1.195 56   0.119
#> 
#> Effect Sizes 
#>                Estimate     SE              C.I. Conf. Level
#> Raw              1.4000 1.3393 [-0.8401, 3.6401]         0.9
#> Hedges's g(av)   0.2706 0.2683 [-0.1585, 0.6973]         0.9
#> Note: SMD confidence intervals are an approximation. See vignette("SMD_calcs").

# Example 3: Paired samples test
# Testing if pre-post difference is equivalent to zero within ±2.5 units
# with correlation between measurements of 0.7
tsum_TOST(m1 = 24.5, sd1 = 6.2, n1 = 25,
         m2 = 26.1, sd2 = 5.8, n2 = 25,
         r12 = 0.7, paired = TRUE,
         eqb = 2.5)
#> 
#> Paired t-test
#> 
#> The equivalence test was non-significant, t(24) = 0.965, p = 1.72e-01
#> The null hypothesis test was non-significant, t(24) = -1.716, p = 9.91e-02
#> NHST: don't reject null significance hypothesis that the effect is equal to zero 
#> TOST: don't reject null equivalence hypothesis
#> 
#> TOST Results 
#>                  t df p.value
#> t-test     -1.7159 24   0.099
#> TOST Lower  0.9652 24   0.172
#> TOST Upper -4.3971 24 < 0.001
#> 
#> Effect Sizes 
#>               Estimate     SE               C.I. Conf. Level
#> Raw            -1.6000 0.9324 [-3.1953, -0.0047]         0.9
#> Hedges's g(z)  -0.3323 0.2061  [-0.6571, -9e-04]         0.9
#> Note: SMD confidence intervals are an approximation. See vignette("SMD_calcs").

# Example 4: Two-sample test using standardized effect size bounds
# Testing if the standardized mean difference is within ±0.5 SD
tsum_TOST(m1 = 100, sd1 = 15, n1 = 40,
         m2 = 104, sd2 = 16, n2 = 42,
         eqb = 0.5, eqbound_type = "SMD")
#> Warning: setting bound type to SMD produces biased results!
#> 
#> Welch Two Sample t-test
#> 
#> The equivalence test was non-significant, t(79.98) = 1.097, p = 1.38e-01
#> The null hypothesis test was non-significant, t(79.98) = -1.168, p = 2.46e-01
#> NHST: don't reject null significance hypothesis that the effect is equal to zero 
#> TOST: don't reject null equivalence hypothesis
#> 
#> TOST Results 
#>                 t    df p.value
#> t-test     -1.168 79.98   0.246
#> TOST Lower  1.097 79.98   0.138
#> TOST Upper -3.433 79.98 < 0.001
#> 
#> Effect Sizes 
#>                Estimate     SE              C.I. Conf. Level
#> Raw             -4.0000 3.4235 [-9.6971, 1.6971]         0.9
#> Hedges's g(av)  -0.2555 0.2244 [-0.6159, 0.1065]         0.9
#> Note: SMD confidence intervals are an approximation. See vignette("SMD_calcs").