Skip to contents

SimplyAgree R Package

Artwork courtesy of Chelsea Parlett Pelleriti

Please see the package’s website for updates, vignettes, and other details about the package.

Background

SimplyAgree is an R package, and jamovi module, designed to simplify agreement and reliability analyses for researchers. The package implements rigorous statistical methods for method comparison studies, providing both classical and modern approaches to assessing measurement agreement.

Core Functionality

The package provides two primary approaches for assessing agreement between measurement methods:

1. Limits of Agreement: agreement_limit()

The agreement_limit() function implements Bland-Altman style limits of agreement analysis. This approach:

  • Computes confidence intervals for the range of agreement between two methods
  • Supports both simple and nested/repeated measures designs
  • Implements exact procedures based on Shieh (2019) and Jan & Shieh
  • Handles design effects for clustered data
  • Provides multiple interval estimation methods (exact, MOVER, Zou’s method)

This is the preferred function when you want to describe what proportion of differences fall within specified bounds or when making inferences about the central region of the paired-difference distribution.

2. Tolerance Intervals: tolerance_limit()

The tolerance_limit() function creates statistical tolerance intervals that:

  • Construct intervals expected to contain a specified proportion of future observations
  • Implement exact equal-tailed tolerance intervals
  • Support confidence intervals for ranges of percentiles
  • Handle both simple and complex study designs
  • Provide sample size and power calculations

Tolerance intervals are most appropriate when making simultaneous inferences about pairs of percentile limits or when you need prediction-oriented intervals for future measurements.

Additional Features

Beyond the two core functions, SimplyAgree provides:

  • Reliability Analysis: reli_stats() and reli_aov() functions for comprehensive reliability assessment
  • Agreement Testing: Exact hypothesis tests for declaring measurement agreement (based on Shieh, 2019)
  • Power Analysis: power_exact_agreement() and related functions for sample size determination in agreement studies
  • Visualization: Built-in plotting capabilities for Bland-Altman plots and related visualizations
  • Flexible Design Support: Handles simple, nested, and repeated measures designs with appropriate variance adjustments

Installing SimplyAgree

You can install the most up-to-date version of SimplyAgree from GitHub with:

devtools::install_github("arcaldwell49/SimplyAgree")

Quick Start Example 

library(SimplyAgree)

# Load example data
data(temps)

# Limits of agreement analysis
agree_results <- agreement_limit(x = "method1", 
                                  y = "method2", 
                                  data = temps,
                                  agree.level = 0.95)

# Tolerance interval analysis  
tol_results <- tolerance_limit(x = "method1",
                                y = "method2", 
                                data = temps,
                                prop = 0.95)

# Reliability analysis
reli_results <- reli_stats(data = temps,
                           wide = TRUE)

Contributing

We are happy to receive bug reports, suggestions, questions, and (most of all) contributions to fix problems and add features. Pull Requests for contributions are encouraged.

Here are some simple ways in which you can contribute (in the increasing order of commitment):

  • Read and correct any inconsistencies in the documentation
  • Raise issues about bugs or wanted features
  • Review code
  • Add new functionality

Code of Conduct

Please note that the SimplyAgree project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms.

References

The functions in this package are largely based on the following works:

Bland, J. M., & Altman, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307-310. https://doi.org/10.1016/S0140-6736(86)90837-8

Bland, J. M., & Altman, D. G. (1999). Measuring agreement in method comparison studies. Statistical Methods in Medical Research, 8(2), 135-160. https://doi.org/10.1177/096228029900800204

Carrasco, JL, et al. (2013). Estimation of the concordance correlation coefficient for repeated measures using SAS and R. Computer Methods and Programs in Biomedicine, 109, 293-304. https://doi.org/10.1016/j.cmpb.2012.09.002

Francq, B. G., Berger, M., & Boachie, C. (2020). To tolerate or to agree: A tutorial on tolerance intervals in method comparison studies with BivRegBLS R Package. Statistics in Medicine, 39(28), 4334-4349. https://doi.org/10.1002/sim.8709

Francq, B. G., Lin, D., & Hoyer, W. (2019). Confidence, prediction, and tolerance in linear mixed models. Statistics in Medicine, 38(30), 5603-5622. https://doi.org/10.1002/sim.8386

Jan, S. L., & Shieh, G. (2018). The Bland-Altman range of agreement: Exact interval procedure and sample size determination. Computers in Biology and Medicine, 100, 247-252. https://doi.org/10.1016/j.compbiomed.2018.06.020

King, TS and Chinchilli, VM. (2001). A generalized concordance correlation coefficient for continuous and categorical data. Statistics in Medicine, 20, 2131-2147. https://doi.org/10.1002/sim.845

King, TS, Chinchilli, VM, and Carrasco, JL. (2007). A repeated measures concordance correlation coefficient. Statistics in Medicine, 26, 3095-3113. https://doi.org/10.1002/sim.2778

Lin, L. I. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 45(1), 255-268. https://doi.org/10.2307/2532051

Lu, M. J., et al. (2016). Sample Size for Assessing Agreement between Two Methods of Measurement by Bland-Altman Method. The International Journal of Biostatistics, 12(2). https://doi.org/10.1515/ijb-2015-0039

Parker, R. A., et al. (2016). Application of mixed effects limits of agreement in the presence of multiple sources of variability: exemplar from the comparison of several devices to measure respiratory rate in COPD patients. PloS One, 11(12), e0168321. https://doi.org/10.1371/journal.pone.0168321

Shieh, G. (2019). Assessing agreement between two methods of quantitative measurements: Exact test procedure and sample size calculation. Statistics in Biopharmaceutical Research, 12(3), 352-359. https://doi.org/10.1080/19466315.2019.1677495

Weir, J. P. (2005). Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. The Journal of Strength & Conditioning Research, 19(1), 231-240. https://doi.org/10.1519/15184.1

Zou, G. Y. (2013). Confidence interval estimation for the Bland-Altman limits of agreement with multiple observations per individual. Statistical Methods in Medical Research, 22(6), 630-642. https://doi.org/10.1177/0962280211402548