Skip to contents

Equivalence Test using an F-test

equ_ftest.Rd

[Stable]

Performs equivalence or minimal effect testing on the partial eta-squared (pes) value using an F-test. This function provides a low-level interface that works directly with F statistics rather than ANOVA objects.

Usage

equ_ftest(Fstat, df1, df2, eqbound = NULL, eqb, MET = FALSE, alpha = 0.05)

Arguments

Fstat

The F-statistic from the F-test.

df1

Degrees of freedom for the numerator (effect degrees of freedom).

df2

Degrees of freedom for the denominator (error degrees of freedom).

eqbound

Equivalence bound for the partial eta-squared. This value represents the smallest effect size considered meaningful or practically significant.

eqb

Defunct argument for equivalence bound, use eqbound instead.

MET

Logical indicator to perform a minimal effect test rather than equivalence test (default is FALSE). When TRUE, the alternative hypothesis becomes that the effect is larger than the equivalence bound.

alpha

Alpha level used for the test (default = 0.05).

Value

Object of class "htest" containing the following components:

  • statistic: The value of the F-statistic with name "F".

  • parameter: The degrees of freedom for the F-statistic (df1 and df2).

  • p.value: The p-value for the equivalence or minimal effect test.

  • conf.int: A confidence interval for the partial eta-squared statistic.

  • estimate: Estimate of partial eta-squared.

  • null.value: The specified equivalence bound.

  • alternative: NULL (not used in this test).

  • method: A string indicating the type of test ("Equivalence Test from F-test" or "Minimal Effect Test from F-test").

  • data.name: A string indicating that this was calculated from summary statistics.

Details

This function tests whether an effect is practically equivalent to zero (when MET = FALSE) or meaningfully different from zero (when MET = TRUE) using the approach described by Campbell & Lakens (2021).

The function works by:

  1. Converting the F-statistic to a partial eta-squared value

  2. Converting the equivalence bound for partial eta-squared to a non-centrality parameter

  3. Computing the confidence interval for the partial eta-squared

  4. Performing an equivalence test or minimal effect test based on the non-central F distribution

For equivalence tests (MET = FALSE), a significant result (p < alpha) indicates that the effect is statistically equivalent to zero (smaller than the equivalence bound).

For minimal effect tests (MET = TRUE), a significant result (p < alpha) indicates that the effect is meaningfully different from zero (larger than the equivalence bound).

For details on the calculations in this function see vignette("the_ftestTOSTER").

Degrees of freedom and the non-centrality parameter

The non-centrality parameter is computed as \(\lambda = \frac{\Delta}{1 - \Delta} (df_1 + df_2 + 1)\). For a one-way ANOVA or a multivariable regression \(df_1 + df_2 + 1 = N\), so this reduces exactly to the expression given by Campbell & Lakens (2021). More generally \(df_1 + df_2 + 1\) is the effective sample size of the error stratum in which the effect is tested, which allows the same logic to be applied to factorial, within-subjects, and mixed designs. Supply the df1 and df2 belonging to the effect of interest and its own error term; in a mixed design a between-subjects effect will yield \(df_1 + df_2 + 1\) equal to the number of subjects rather than the number of observations, which is correct.

Note that eqbound is a bound on partial eta-squared. In within-subjects designs this excludes subject variance from the denominator and is therefore not a bound on the share of total variance in the data.

References

Campbell, H., & Lakens, D. (2021). Can we disregard the whole model? Omnibus non‐inferiority testing for R2 in multi‐variable linear regression and in ANOVA. British Journal of Mathematical and Statistical Psychology, 74(1), 64-89. doi: 10.1111/bmsp.12201

See also

Other f-test: equ_anova()

Examples

# Example 1: Equivalence test with a small effect
# F = 2.5, df1 = 2, df2 = 100, equivalence bound = 0.1
equ_ftest(Fstat = 2.5, df1 = 2, df2 = 100, eqbound = 0.1)
#> 
#> 	Equivalence Test from F-test
#> 
#> data:  Summary Statistics
#> F = 2.5, df1 = 2, df2 = 100, p-value = 0.09337
#> 95 percent confidence interval:
#>  0.0000000 0.1373114
#> sample estimates:
#> [1] 0.04761905
#> 

# Example 2: Minimal effect test with a large effect
# F = 12, df1 = 3, df2 = 80, equivalence bound = 0.1
equ_ftest(Fstat = 12, df1 = 3, df2 = 80, eqbound = 0.1, MET = TRUE)
#> 
#> 	Minimal Effect Test from F-test
#> 
#> data:  Summary Statistics
#> F = 12, df1 = 3, df2 = 80, p-value = 0.007548
#> 95 percent confidence interval:
#>  0.1331756 0.4332325
#> sample estimates:
#> [1] 0.3103448
#> 

# Example 3: Equivalence test with a very small effect
# F = 0.8, df1 = 1, df2 = 50, equivalence bound = 0.05
equ_ftest(Fstat = 0.8, df1 = 1, df2 = 50, eqbound = 0.05)
#> 
#> 	Equivalence Test from F-test
#> 
#> data:  Summary Statistics
#> F = 0.8, df1 = 1, df2 = 50, p-value = 0.2176
#> 95 percent confidence interval:
#>  0.0000000 0.1356009
#> sample estimates:
#> [1] 0.01574803
#>