Comparing Correlations Between Independent Studies with Bootstrapping

A function to compare correlation coefficients between independent studies using bootstrap methods. This function is intended to be used to compare the compatibility of original studies with replication studies (lower p-values indicating lower compatibility).

Usage

boot_compare_cor(
  x1,
  y1,
  x2,
  y2,
  alternative = c("two.sided", "less", "greater", "equivalence", "minimal.effect"),
  method = c("pearson", "kendall", "spearman", "winsorized", "bendpercent"),
  alpha = 0.05,
  null = 0,
  R = 1999,
  ...
)

Arguments

x1, y1

Numeric vectors of data values from study 1. x1 and y1 must have the same length.

x2, y2

Numeric vectors of data values from study 2. x2 and y2 must have the same length.

alternative

a character string specifying the alternative hypothesis:

"two.sided": correlation is not equal to null (default)
"greater": correlation is greater than null
"less": correlation is less than null
"equivalence": correlation is within the equivalence bounds (TOST)
"minimal.effect": correlation is outside the equivalence bounds (TOST)

You can specify just the initial letter.

method

a character string indicating which correlation coefficient to use:

"pearson": standard Pearson product-moment correlation
"kendall": Kendall's tau rank correlation
"spearman": Spearman's rho rank correlation
"winsorized": Winsorized correlation (robust to outliers)
"bendpercent": percentage bend correlation (robust to marginal outliers)

Can be abbreviated.

alpha

alpha level (default = 0.05)

null

a number or vector indicating the null hypothesis value(s):

For standard tests: a single value (default = 0)
For equivalence/minimal effect tests: either a single value (symmetric bounds ±value will be created) or a vector of two values representing the lower and upper bounds

R

number of bootstrap replications (default = 1999).

...

Additional arguments passed to the correlation functions.

Value

A list with class "htest" containing the following components:

p.value: The p-value for the test under the null hypothesis.
parameter: Sample sizes from each study.
conf.int: Bootstrap confidence interval for the difference in correlations.
estimate: Difference in correlations between studies.
stderr: Standard error of the difference (estimated from bootstrap distribution).
null.value: The specified hypothesized value(s) for the null hypothesis.
alternative: Character string indicating the alternative hypothesis.
method: Description of the correlation method used.
data.name: Names of the input data vectors.
boot_res: List containing the bootstrap samples for the difference and individual correlations.
call: The matched call.

Details

This function tests for differences between correlation coefficients from independent studies using bootstrap resampling methods. Unlike the compare_cor function, which uses Fisher's z transformation or the Kraatz method with summary statistics, this function works with raw data and uses bootstrapping to estimate confidence intervals and p-values.

It is particularly useful for:

Comparing correlations when assumptions for parametric tests may not be met
Obtaining robust confidence intervals for the difference between correlations
Comparing an original study with its replication using raw data
Testing if correlations from different samples are equivalent

The function supports multiple correlation methods:

Standard correlation coefficients (Pearson, Kendall, Spearman)
Robust correlation measures (Winsorized, percentage bend)

The function also supports both standard hypothesis testing and equivalence/minimal effect testing:

For standard tests (two.sided, less, greater), the function tests whether the difference between correlations differs from the null value (typically 0).
For equivalence testing ("equivalence"), it determines whether the difference falls within the specified bounds, which can be set asymmetrically.
For minimal effect testing ("minimal.effect"), it determines whether the difference falls outside the specified bounds.

When performing equivalence or minimal effect testing:

If a single value is provided for null, symmetric bounds ±value will be used
If two values are provided for null, they will be used as the lower and upper bounds

Examples

# Example 1: Comparing Pearson correlations (standard test)
set.seed(123)
x1 <- rnorm(30)
y1 <- x1 * 0.6 + rnorm(30, 0, 0.8)
x2 <- rnorm(25)
y2 <- x2 * 0.3 + rnorm(25, 0, 0.9)

# Two-sided test with Pearson correlation (use fewer bootstraps for example)
boot_compare_cor(x1, y1, x2, y2, method = "pearson",
                alternative = "two.sided", R = 500)
#> 
#> 	Bootstrapped difference in Pearson's correlation
#> 
#> data:  x1 and y1 vs. x2 and y2
#> n1 = 30, n2 = 25, p-value = 0.24
#> alternative hypothesis: true difference in correlation is not equal to 0
#> 95 percent confidence interval:
#>  -0.1512684  0.9824494
#> sample estimates:
#>       cor 
#> 0.3489526 
#> 

# Example 2: Testing for equivalence with Spearman correlation
# Testing if the difference in correlations is within ±0.2
boot_compare_cor(x1, y1, x2, y2, method = "spearman",
                alternative = "equivalence", null = 0.2, R = 500)
#> 
#> 	Bootstrapped difference in Spearman's rho
#> 
#> data:  x1 and y1 vs. x2 and y2
#> n1 = 30, n2 = 25, p-value = 0.596
#> alternative hypothesis: true difference in rho is  0.2
#> 90 percent confidence interval:
#>  -0.1505546  0.7454722
#> sample estimates:
#>      rho 
#> 0.280024 
#> 

# Example 3: Testing with robust correlation measure
# Using percentage bend correlation for non-normal data
boot_compare_cor(x1, y1, x2, y2, method = "bendpercent",
                alternative = "greater", R = 500)
#> 
#> 	Bootstrapped difference in percentage bend correlation pb
#> 
#> data:  x1 and y1 vs. x2 and y2
#> n1 = 30, n2 = 25, p-value = 0.112
#> alternative hypothesis: true difference in pb is greater than 0
#> 90 percent confidence interval:
#>  -0.1248026  0.7187866
#> sample estimates:
#>        pb 
#> 0.3301441 
#> 

# Example 4: Using asymmetric bounds for equivalence testing
boot_compare_cor(x1, y1, x2, y2, method = "pearson",
                alternative = "equivalence", null = c(-0.1, 0.3), R = 500)
#> 
#> 	Bootstrapped difference in Pearson's correlation
#> 
#> data:  x1 and y1 vs. x2 and y2
#> n1 = 30, n2 = 25, p-value = 0.536
#> alternative hypothesis: equivalence
#> null values:
#> difference in correlation                      <NA> 
#>                      -0.1                       0.3 
#> 90 percent confidence interval:
#>  -0.1458491  0.8562676
#> sample estimates:
#>       cor 
#> 0.3489526 
#>