This is a generic function that performs a generalized asymptotic Brunner-Munzel test in a fashion similar to t.test.

  paired = FALSE,
  alternative = c("two.sided", "less", "greater"),
  mu = 0.5,
  alpha = 0.05,
  perm = FALSE,
  max_n_perm = 10000

# S3 method for default
  paired = FALSE,
  alternative = c("two.sided", "less", "greater"),
  mu = 0.5,
  alpha = 0.05,
  perm = FALSE,
  max_n_perm = 10000,

# S3 method for formula
brunner_munzel(formula, data, subset, na.action, ...)



a (non-empty) numeric vector of data values.


further arguments to be passed to or from methods.


a logical indicating whether you want a paired test.


a character string specifying the alternative hypothesis, must be one of "two.sided" (default), "greater" or "less". You can specify just the initial letter.


a number specifying an optional parameter used to form the null hypothesis (Default = 0.5). This can be thought of as the null in terms of the relative effect, p = P (X < Y ) + 0.5 * P (X = Y); See ‘Details’.


alpha level (default = 0.05)


a logical indicating whether or not to perform a permutation test over approximate t-distribution based test (default is FALSE). Highly recommend to set perm = TRUE when sample size per condition is less than 15.


the maximum number of permutations (default is 10000).


an optional (non-empty) numeric vector of data values.


a formula of the form lhs ~ rhs where lhs is a numeric variable giving the data values and rhs either 1 for a one-sample or paired test or a factor with two levels giving the corresponding groups. If lhs is of class "Pair" and rhs is 1, a paired test is done.


an optional matrix or data frame (or similar: see model.frame) containing the variables in the formula formula. By default the variables are taken from environment(formula).


an optional vector specifying a subset of observations to be used.


a function which indicates what should happen when the data contain NAs. Defaults to getOption("na.action").


A list with class "htest" containing the following components:

  • "statistic": the value of the test statistic.

  • "parameter": the degrees of freedom for the test statistic.

  • "p.value": the p-value for the test.

  • "conf.int": a confidence interval for the relative effect appropriate to the specified alternative hypothesis.

  • "estimate": the estimated relative effect.

  • "null.value": the specified hypothesized value of the relative effect.

  • "stderr": the standard error of the relative effect.

  • "alternative": a character string describing the alternative hypothesis.

  • "method": a character string indicating what type of test was performed.

  • "data.name": a character string giving the name(s) of the data.


This function is made to provide a test of stochastic equality between two samples (paired or independent), and is referred to as the Brunner-Munzel test.

This tests the hypothesis that the relative effect, discussed below, is equal to the null value (default is mu = 0.5).

The estimate of the relative effect, which can be considered as value similar to the probability of superiority, refers to the following:

$$\hat p = p(X<Y) + \frac{1}{2} \cdot P(X=Y)$$

Note, for paired samples, this does not refer to the probability of an increase/decrease in paired sample but rather the probability that a randomly sampled value of X. This is also referred to as the "relative" effect in the literature. Therefore, the results will differ from the concordance probability provided by the ses_calc function.

The brunner_munzel function is based on the npar.t.test and npar.t.test.paired functions within the nparcomp package (Konietschke et al. 2015).


Brunner, E., Munzel, U. (2000). The Nonparametric Behrens-Fisher Problem: Asymptotic Theory and a Small Sample Approximation. Biometrical Journal 42, 17 -25.

Neubert, K., Brunner, E., (2006). A Studentized Permutation Test for the Nonparametric Behrens-Fisher Problem. Computational Statistics and Data Analysis.

Munzel, U., Brunner, E. (2002). An Exact Paired Rank Test. Biometrical Journal 44, 584-593.

Konietschke, F., Placzek, M., Schaarschmidt, F., & Hothorn, L. A. (2015). nparcomp: an R software package for nonparametric multiple comparisons and simultaneous confidence intervals. Journal of Statistical Software 64 (2015), Nr. 9, 64(9), 1-17. http://www.jstatsoft.org/v64/i09/

See also

Other Robust tests: boot_log_TOST(), boot_t_TOST(), boot_t_test(), log_TOST(), wilcox_TOST()


brunner_munzel(mpg ~ am, data = mtcars)
#> Sample size in at least one group is small. Permutation test (perm = TRUE) is highly recommended.
#> 	two-sample Brunner-Munzel test
#> data:  mpg by am
#> t = -4.2653, df = 20.893, p-value = 0.0003479
#> alternative hypothesis: true relative effect is not equal to 0.5
#> 95 percent confidence interval:
#>  0.009114802 0.330966169
#> sample estimates:
#> p(X<Y) + .5*P(X=Y) 
#>          0.1700405