Brunner-Munzel Test

This is a generic function that performs a generalized asymptotic Brunner-Munzel test in a fashion similar to t.test.

Usage

# Default S3 method
brunner_munzel(
  x,
  y,
  paired = FALSE,
  alternative = c("two.sided", "less", "greater", "equivalence", "minimal.effect"),
  mu = 0.5,
  alpha = 0.05,
  test_method = c("t", "logit", "perm"),
  R = 10000,
  p_method = c("plusone", "original"),
  perm = "deprecated",
  max_n_perm = "deprecated",
  ...
)

# S3 method for class 'formula'
brunner_munzel(formula, data, subset, na.action, ...)

Arguments

x

a (non-empty) numeric vector of data values.

y

an optional (non-empty) numeric vector of data values.

paired

a logical indicating whether you want a paired test.

alternative

a character string specifying the alternative hypothesis, must be one of:

"two.sided" (default): true relative effect is not equal to mu
"less": true relative effect is less than mu
"greater": true relative effect is greater than mu
"equivalence": true relative effect is between the lower and upper bounds specified in mu
"minimal.effect": true relative effect is less than the lower bound or greater than the upper bound specified in mu

mu

a number or vector specifying the null hypothesis value(s):

For standard alternatives ("two.sided", "less", "greater"): a single value representing the hypothesized relative effect (default = 0.5, i.e., stochastic equality).
For "equivalence" or "minimal.effect": two values representing the lower and upper bounds for the relative effect. Values must be between 0 and 1.

alpha

alpha level (default = 0.05)

test_method

a character string specifying the test method to use:

"t" (default): approximate t-distribution with Satterthwaite-Welch degrees of freedom
"logit": logit transformation for range-preserving confidence intervals
"perm": studentized permutation test (recommended when sample size per condition is less than 15)

R

the number of permutations for the permutation test (default is 10000). Only used when test_method = "perm".

p_method

a character string specifying the method for computing permutation p-values. Only used when test_method = "perm":

"plusone" (default): Uses the (b+1)/(R+1) formula from Phipson & Smyth (2010), which provides exact p-values when permutations are sampled without replacement.
"original": Uses the traditional b/R formula with a minimum of 1/R.

perm

Use test_method = "perm" instead.

max_n_perm

Use R instead.

...

further arguments to be passed to or from methods.

formula

a formula of the form lhs ~ rhs where lhs is a numeric variable giving the data values and rhs either 1 for a one-sample test or a factor with two levels giving the corresponding groups. For paired tests, use the default method with x and y vectors instead of the formula method.

data

an optional matrix or data frame (or similar: see model.frame) containing the variables in the formula formula. By default the variables are taken from environment(formula).

subset

an optional vector specifying a subset of observations to be used.

na.action

a function which indicates what should happen when the data contain NAs. Defaults to getOption("na.action").

Value

A list with class "htest" containing the following components:

"statistic": the value of the test statistic.
"parameter": the degrees of freedom for the test statistic.
"p.value": the p-value for the test.
"conf.int": a confidence interval for the relative effect appropriate to the specified alternative hypothesis.
"estimate": the estimated relative effect.
"null.value": the specified hypothesized value of the relative effect.
"stderr": the standard error of the relative effect.
"alternative": a character string describing the alternative hypothesis.
"method": a character string indicating what type of test was performed.
"data.name": a character string giving the name(s) of the data.

Details

This function is made to provide a test of stochastic equality between two samples (paired or independent), and is referred to as the Brunner-Munzel test.

This tests the hypothesis that the relative effect, discussed below, is equal to the null value (default is mu = 0.5).

The estimate of the relative effect, which can be considered as value similar to the probability of superiority, refers to the following:

$$\hat p = P(X>Y) + \frac{1}{2} \cdot P(X=Y)$$

Note, for paired samples, this does not refer to the probability of an increase/decrease in paired sample but rather the probability that a randomly sampled value of X is greater than a randomly sampled value of Y. This is also referred to as the "relative" effect in the literature. Therefore, the results will differ from the concordance probability provided by the ses_calc function.

The brunner_munzel function is based on the npar.t.test and npar.t.test.paired functions within the nparcomp package (Konietschke et al. 2015).

Test Methods

Three test methods are available:

"t": The default method uses a t-distribution approximation with Satterthwaite-Welch degrees of freedom. This is appropriate for moderate to large sample sizes.
"logit": Uses a logit transformation to produce range-preserving confidence intervals that are guaranteed to stay within [0, 1]. This method is recommended when the estimated relative effect is close to 0 or 1.
"perm": A studentized permutation test following Neubert & Brunner (2007). This method is highly recommended when sample sizes are small (< 15 per group) as it provides better control of Type I error rates in these situations.

Hypothesis Testing

For the standard alternatives, the null hypothesis is that the relative effect equals mu:

"two.sided": H0: p = mu vs H1: p ≠ mu
"less": H0: p ≥ mu vs H1: p < mu
"greater": H0: p ≤ mu vs H1: p > mu

For equivalence and minimal effect testing using the two one-sided tests (TOST) procedure:

"equivalence": H0: p ≤ mu[1] OR p ≥ mu[2] vs H1: mu[1] < p < mu[2]

Tests whether the relative effect falls within the specified bounds. The p-value is the maximum of the two one-sided p-values.
"minimal.effect": H0: mu[1] < p < mu[2] vs H1: p ≤ mu[1] OR p ≥ mu[2]

Tests whether the relative effect falls outside the specified bounds. The p-value is the minimum of the two one-sided p-values.

Test Statistic and P-value Calculation

The test statistic is calculated as:

$$t = \sqrt{N} \cdot \frac{\hat{p} - p_0}{s}$$

where $N$ is the total sample size (or $n$ for paired samples), $\hat{p}$ is the estimated relative effect, $p_0$ is the null hypothesis value, and $s$ is the rank-based standard error.

For equivalence testing, two test statistics are computed:

$$t_{low} = \sqrt{N} \cdot \frac{\hat{p} - p_{low}}{s}$$ $$t_{high} = \sqrt{N} \cdot \frac{\hat{p} - p_{high}}{s}$$

where $p_{low}$ and $p_{high}$ are the lower and upper equivalence bounds. The one-sided p-values are:

$p_1$: p-value for H1: p > $p_{low}$ (from the lower bound test)
$p_2$: p-value for H1: p < $p_{high}$ (from the upper bound test)

For equivalence: $p_{TOST} = \max(p_1, p_2)$

For minimal effect: $p_{MET} = \min(1 - p_1, 1 - p_2)$

Confidence Intervals for Equivalence Testing

When alternative = "equivalence" or alternative = "minimal.effect", the confidence interval is computed at the $1 - 2\alpha$ level (default: 90% CI when $\alpha = 0.05$). This follows the standard TOST procedure where the $(1 - 2\alpha) \times 100\%$ CI corresponds to two one-sided tests at level $\alpha$.

Permutation Tests with Non-0.5 Null Values

When test_method = "perm" and mu != 0.5, the permutation distribution is constructed by centering the permuted test statistics at 0.5 (the value implied by exchangeability), while the observed test statistic is centered at the hypothesized null value. This approach is valid because the studentized permutation distribution converges to the same limit regardless of the centering, following the asymptotic theory of Janssen (1997) and Neubert & Brunner (2007).

References

Brunner, E., Munzel, U. (2000). The Nonparametric Behrens-Fisher Problem: Asymptotic Theory and a Small Sample Approximation. Biometrical Journal 42, 17 -25.

Neubert, K., Brunner, E., (2006). A Studentized Permutation Test for the Nonparametric Behrens-Fisher Problem. Computational Statistics and Data Analysis.

Munzel, U., Brunner, E. (2002). An Exact Paired Rank Test. Biometrical Journal 44, 584-593.

Munzel, U., Hauschke, D. (2003). A nonparametric test for proving noninferiority in clinical trials with ordered categorical data. Pharmaceutical Statistics, 2, 31-37.

Konietschke, F., Placzek, M., Schaarschmidt, F., & Hothorn, L. A. (2015). nparcomp: an R software package for nonparametric multiple comparisons and simultaneous confidence intervals. Journal of Statistical Software 64 (2015), Nr. 9, 64(9), 1-17. http://www.jstatsoft.org/v64/i09/

Janssen, A. (1997). Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens-Fisher problem. Statistics & Probability Letters, 36(1), 9-21.

Phipson, B., & Smyth, G. K. (2010). Permutation P-values should never be zero: calculating exact P-values when permutations are randomly drawn. Statistical Applications in Genetics and Molecular Biology, 9(1), Article 39.

Examples

data(mtcars)
# Standard test of stochastic equality
brunner_munzel(mpg ~ am, data = mtcars)
#> Sample size in at least one group is small. Permutation test (test_method = 'perm') is highly recommended.
#> 
#> 	Two-sample Brunner-Munzel test
#> 
#> data:  mpg by am
#> t = -4.2653, df = 20.893, p-value = 0.0003479
#> alternative hypothesis: true relative effect is not equal to 0.5
#> 95 percent confidence interval:
#>  0.009114802 0.330966169
#> sample estimates:
#> P(0>1) + .5*P(0=1) 
#>          0.1700405 
#> 

# Test using logit transformation for range-preserving CIs
brunner_munzel(mpg ~ am, data = mtcars, test_method = "logit")
#> Sample size in at least one group is small. Permutation test (test_method = 'perm') is highly recommended.
#> 
#> 	Two-sample Brunner-Munzel test (logit)
#> 
#> data:  mpg by am
#> t = -2.8922, df = 20.893, p-value = 0.008748
#> alternative hypothesis: true relative effect is not equal to 0.5
#> 95 percent confidence interval:
#>  0.06147768 0.39053873
#> sample estimates:
#> P(0>1) + .5*P(0=1) 
#>          0.1700405 
#> 

# Test against a specific null value
brunner_munzel(mpg ~ am, data = mtcars, mu = 0.3)
#> Sample size in at least one group is small. Permutation test (test_method = 'perm') is highly recommended.
#> 
#> 	Two-sample Brunner-Munzel test
#> 
#> data:  mpg by am
#> t = -1.68, df = 20.893, p-value = 0.1079
#> alternative hypothesis: true relative effect is not equal to 0.3
#> 95 percent confidence interval:
#>  0.009114802 0.330966169
#> sample estimates:
#> P(0>1) + .5*P(0=1) 
#>          0.1700405 
#> 

# Equivalence test: is the relative effect between 0.35 and 0.65?
brunner_munzel(mpg ~ am, data = mtcars,
               alternative = "equivalence",
               mu = c(0.35, 0.65))
#> Sample size in at least one group is small. Permutation test (test_method = 'perm') is highly recommended.
#> 
#> 	Two-sample Brunner-Munzel test
#> 
#> data:  mpg by am
#> t = -2.3263, df = 20.893, p-value = 0.9849
#> alternative hypothesis: equivalence
#> null values:
#> lower bound upper bound 
#>        0.35        0.65 
#> 90 percent confidence interval:
#>  0.03689521 0.30318577
#> sample estimates:
#> P(0>1) + .5*P(0=1) 
#>          0.1700405 
#> 

# Minimal effect test: is the relative effect outside 0.4 to 0.6?
brunner_munzel(mpg ~ am, data = mtcars,
               alternative = "minimal.effect",
               mu = c(0.4, 0.6))
#> Sample size in at least one group is small. Permutation test (test_method = 'perm') is highly recommended.
#> 
#> 	Two-sample Brunner-Munzel test
#> 
#> data:  mpg by am
#> t = -2.9727, df = 20.893, p-value = 0.003644
#> alternative hypothesis: minimal.effect
#> null values:
#> lower bound upper bound 
#>         0.4         0.6 
#> 90 percent confidence interval:
#>  0.03689521 0.30318577
#> sample estimates:
#> P(0>1) + .5*P(0=1) 
#>          0.1700405 
#> 

# Permutation-based equivalence test
brunner_munzel(mpg ~ am, data = mtcars,
               alternative = "equivalence",
               mu = c(0.35, 0.65),
               test_method = "perm")
#> NOTE: Permutation-based TOST for equivalence/minimal.effect testing.
#> 
#> 	Two-sample Brunner-Munzel permutation test
#> 
#> data:  mpg by am
#> t-observed = -2.3263, N-permutations = 10000, p-value = 0.9807
#> alternative hypothesis: equivalence
#> null values:
#> lower bound upper bound 
#>        0.35        0.65 
#> 90 percent confidence interval:
#>  0.03478663 0.30529434
#> sample estimates:
#> P(0>1) + .5*P(0=1) 
#>          0.1700405 
#>