1 Introduction
A correlation coefficient is a summary statistic between \(-1\) and \(1\) that describes the magnitude and direction of the linear association between two variables. Commonly used benchmarks interpret correlations of 0.1 as small, 0.3 as medium and 0.5 as large (Cohen 1988), but these labels say nothing about what the relationship means for a decision or a given application. Clinicians frequently have to choose between patients when a resource is limited. For instance, a rehabilitation program has one place available, a scarce treatment can be offered to only one of two patients, or a waiting list must be ordered by expected benefit. Suppose that a prognostic score correlates 0.50 with recovery after a treatment and that two patients differ on the score. The clinician making the choice needs to know the probability that the patient with the higher score will recover better, and whether that probability is large enough to justify giving that patient priority.
Common language effect sizes address questions of this kind by expressing an effect as a probability. The original and most widely used version is the probability of superiority for two groups (McGraw and Wong 1992; Grissom 1994; Vargha and Delaney 2000), defined as the probability that a randomly sampled person from one group scores higher than a randomly sampled person from the other group. Let \(Y^{(1)}\) and \(Y^{(0)}\) denote the outcome of a randomly sampled person from the treatment and control group, respectively. The probability of superiority is \[ \theta = P\!\left(Y^{(1)} > Y^{(0)}\right). \tag{1}\] A value of 0.5 means that the two groups are indistinguishable, and a value of 0.75 means that the person from the treatment group has the higher score in three pairs out of four. When the outcome is normally distributed in both groups with a common standard deviation, the probability of superiority is a function of the standardized mean difference \(\delta = (\mu_1 - \mu_0)/\sigma\) (Grissom 1994; Ruscio 2008), \[ \theta = \Phi\!\left(\frac{\delta}{\sqrt{2}}\right), \tag{2}\] where \(\Phi\) is the cumulative distribution function of the standard normal distribution. The factor \(\sqrt{2}\) arises because the difference between two independent draws has twice the variance of a single draw.
Dunlap (1994) generalized this effect size to the relationship between two continuous variables \(X\) and \(Y\). Let \((X_1, Y_1)\) and \((X_2, Y_2)\) be the scores of two randomly sampled people. The bivariate probability of superiority is the probability that the person with the higher predictor score also has the higher outcome, \[ \theta_b = P\!\left(Y_1 > Y_2 \mid X_1 > X_2\right). \tag{3}\] This is the probability that a randomly sampled pair is concordant, and it is related to Kendall’s rank correlation by \(\tau = 2\theta_b - 1\) (Kendall 1938). If \(X\) and \(Y\) are bivariate normal with correlation \(\rho\), then \[ \theta_b = \frac{1}{2} + \frac{\arcsin(\rho)}{\pi}, \tag{4}\] so that a correlation of 0.50 corresponds to \(\theta_b = 0.67\) and a correlation of 0.8 to \(\theta_b = 0.80\) (Dunlap 1994). Under bivariate normality this is also the probability that a person who is above the mean on \(X\) is above the mean on \(Y\), which is the quantity that Li and Waisman (2019) estimate without the normality assumption.
Equation 3 is a useful summary, but it does not answer the question that arises in a decision between two specific people. In the clinical example, the probability that the higher scorer recovers better is not the same when the two patients differ by one point as when they differ by thirty points on the prognostic score, and the bivariate probability of superiority averages over all such differences. The quantity needed for the decision is the probability of superiority at the observed difference between the two patients.
This paper introduces the probability of outcome superiority curve (POSC), which is the conditional probability that the person with the higher predictor score has the higher outcome, expressed as a function of the difference in the predictor, and describes the R package posc (Jané 2026) that implements it. The package is available from the Comprehensive R Archive Network (CRAN) at https://CRAN.R-project.org/package=posc. Section 2 defines the curve, establishes properties that hold for any joint distribution, and derives the curve under bivariate normality for a single predictor, for a composite of several predictors and for the predictions of a linear model. Section 3 develops an empirical version that is estimated from raw data without a distributional assumption. Section 4 describes the design of the package, Section 5 illustrates it with clinical examples, and Section 6 concludes. Existing R software for common language effect sizes, for example the effectsize package (Ben-Shachar et al. 2020), implements the probability of superiority for group differences. To my knowledge, no software implements the curve for correlations.
2 Probability of outcome superiority curves
2.1 Definition and properties
Let \((X_1, Y_1)\) and \((X_2, Y_2)\) be the predictor and outcome scores of two people sampled independently from the same population, and let \(\Delta = X_1 - X_2\) be their difference on the predictor. The probability of outcome superiority curve is the conditional probability that the first person has the higher outcome, as a function of the difference in the predictor: \[ \theta(\Delta) = P\!\left(Y_1 > Y_2 \mid X_1 - X_2 = \Delta\right). \tag{5}\] For a positive relationship the curve equals 0.5 when the two people have the same predictor score and increases toward 1 as the difference increases. The curve answers the clinician’s question directly. If patient A scores \(\Delta\) points higher than patient B on a prognostic score, \(\theta(\Delta)\) is the probability that patient A will have the better outcome.
Three properties hold for every POSC, regardless of the joint distribution of \(X\) and \(Y\), provided that the outcome is continuous. First, \(\theta(0) = 1/2\), two people with the same predictor score are equally likely to have the higher outcome. Second, because the two people are exchangeable, \[ \theta(-\Delta) = 1 - \theta(\Delta). \tag{6}\] If the first person is \(\Delta\) points below the second, the probability that the first person has the higher outcome is one minus the probability for the pair with the roles reversed. The curve is therefore point-symmetric about \((0, 1/2)\), and when it is monotone it is a sigmoid with its inflection point at \(\Delta = 0\). The half of the curve over \(\Delta \geq 0\) contains all of the information, and it is the half that the package plots by default, but the full curve is sometimes more informative to display (Figure 2, right). Third, the bivariate probability of superiority is the POSC averaged over the distribution of the difference between two randomly sampled people, \[ \theta_b = E\!\left[\theta(\Delta) \mid \Delta > 0\right], \tag{7}\] which follows from Equation 3 by conditioning on \(\Delta\). That is, Dunlap’s single value is the average of the POSC, weighted by how often each difference occurs. Two populations can have the same \(\theta_b\) and very different curves, which is why the curve is the more informative summary.
2.2 Bivariate normality
Assume that \(X\) and \(Y\) are bivariate normal with means \(\mu_X\) and \(\mu_Y\), standard deviations \(\sigma_X\) and \(\sigma_Y\), and correlation \(\rho\). Because the two people are independent draws from the same bivariate normal distribution, their differences \((X_1 - X_2, Y_1 - Y_2)\) are also bivariate normal, with means of zero, variances \(2\sigma_X^2\) and \(2\sigma_Y^2\), and the same correlation \(\rho\). The conditional distribution of a bivariate normal variable given the other variable is normal, so that \[ Y_1 - Y_2 \mid \Delta \;\sim\; \mathcal{N}\!\left(\rho\,\frac{\sigma_Y}{\sigma_X}\,\Delta,\; 2\sigma_Y^2(1 - \rho^2)\right). \tag{8}\] Figure 1 shows this relationship for two people with predictor scores \(X_1\) and \(X_2\), for a small and a large difference \(\Delta\). The conditional distributions of the outcome are two normal distributions with the same variance, whose means are separated in proportion to \(\rho\) and to \(\Delta\). The POSC is the probability that the difference in Equation 8 is positive, which is the area of its distribution above zero, and this area increases with \(\Delta\): \[ \theta(\Delta) = \Phi\!\left(\frac{\rho\,\Delta}{\sigma_X\sqrt{2(1 - \rho^2)}}\right). \tag{9}\] The quantity \(\rho\Delta / (\sigma_X\sqrt{1 - \rho^2})\) is the standardized mean difference between the two conditional outcome distributions in Figure 1, so the POSC is the probability of superiority of Equation 2 applied to those two distributions. In other words, at each difference in the predictor, the correlation is converted into a comparison between two groups. Averaging Equation 9 over the half-normal distribution of \(|\Delta|/\sigma_X\) gives Equation 4, as Equation 7 requires.
Figure 2 (left) shows the curve for several values of \(\rho\) with the predictor in standard deviation units. A correlation of 0.3, conventionally described as medium, gives a person who is one standard deviation higher on the predictor only a 59% probability of the higher outcome. The curves for different correlations differ little near \(\Delta = 0\) and differ most at large differences, so the practical value of a stronger predictor lies mainly in the probabilities it provides for people who are far apart on the predictor. Inverting Equation 9 gives the difference required to reach a given probability, \[ \Delta(\theta) = \frac{\sigma_X\sqrt{2(1-\rho^2)}}{\rho}\,\Phi^{-1}(\theta), \tag{10}\] which answers questions such as how far apart two patients’ scores must be before the probability of correctly identifying the patient with the better outcome reaches 75%. For \(\rho = 0.5\) the required difference is 1.65 standard deviations.
2.2.1 Estimation and confidence bands
A sample estimate is obtained by replacing \(\rho\) and \(\sigma_X\) with the sample correlation \(r\) and the sample standard deviation \(s_X\), \[ \hat\theta(\Delta) = \Phi\!\left(\frac{r\,\Delta}{s_X\sqrt{2(1 - r^2)}}\right). \tag{11}\] Because Equation 9 is monotone in \(\rho\) at every fixed \(\Delta\), a confidence interval for \(\rho\) maps directly onto a confidence band for the curve. The package uses the Fisher transformation (Fisher 1921), \(z = \operatorname{artanh}(r)\), which has an approximate standard error of \(1/\sqrt{n - 3}\). A \(100(1 - \alpha)\)% confidence interval for \(\rho\) is \[ \left[\rho_L, \rho_U\right] = \tanh\!\left(\operatorname{artanh}(r) \pm z_{1 - \alpha/2}\,\frac{1}{\sqrt{n - 3}}\right), \tag{12}\] and substituting \(\rho_L\) and \(\rho_U\) for \(r\) in Equation 11 gives the lower and upper limits of the band at each \(\Delta\) (for negative \(\Delta\) the limits are exchanged, consistent with Equation 6). The band is pointwise, which means that it has the stated coverage at any single difference. This is the appropriate guarantee for a decision about one pair of people.
2.3 Composites of several predictors
Clinical prognosis rarely depends on a single predictor. When several predictors are combined using the regression weights that best predict the outcome, the composite correlates with the outcome at the multiple correlation \(R\). Let \(\mathbf{R}_{xx}\) be the correlation matrix of the predictors and \(\mathbf{r}_{xy}\) the vector of correlations between the predictors and the outcome. Then \[ R^2 = \mathbf{r}_{xy}^\top\,\mathbf{R}_{xx}^{-1}\,\mathbf{r}_{xy}. \tag{13}\] Under multivariate normality the composite and the outcome are bivariate normal, so the POSC for a difference of \(\Delta\) standard deviations on the composite is Equation 9 with \(\rho\) replaced by \(R\) and \(\sigma_X = 1\). A sample multiple correlation is biased upward because the weights are estimated from the same sample. The usual correction replaces \(R^2\) with the adjusted value \(1 - (1 - R^2)(n - 1)/(n - k - 1)\) for \(k\) predictors (Cohen et al. 2003), and the package uses the square root of the adjusted value by default. The sampling distribution of \(R\) is skewed and its variance depends on \(k\), so the Fisher band is only approximate in this case. The package applies Equation 12 with a standard error of \(1/\sqrt{n - k - 2}\) and truncates the interval at zero, and the resulting band should be treated as approximate. Figure 3 illustrates both points using the correlation matrix of Section 5.
2.4 Predictions of a linear model
The same reasoning applies to a fitted linear model. The fitted values \(\hat Y\) of the model are a composite of its predictors, and their correlation with the observed outcome is the multiple correlation of the model. Treating \(\hat Y\) as the predictor gives a POSC for two patients who differ by \(\Delta\) in their predicted outcomes, that is, the probability that the patient with the higher predicted outcome has the higher observed outcome. In this case \(\Delta\) is in the units of the outcome and \(\sigma_X\) is the standard deviation of the fitted values. This version of the curve describes how reliably the model orders two cases.
3 Empirical POSCs
3.1 Motivation
Equation 9 depends on the assumption of bivariate normality, and the assumption matters in practice. If the outcome is skewed, or the relationship is monotone but not linear, the Pearson correlation understates the rank association and the normal-theory curve is too low. If the relationship has a ceiling, the normal-theory curve continues to increase when the true curve becomes flat. If the relationship is nonmonotone, a correlation cannot describe it at all. A POSC is defined for any joint distribution, so the natural solution is to estimate the curve directly from the data.
3.2 Pairwise construction and model
Every pair of people in a sample is a realization of the definition in Equation 5. For people \(i\) and \(j\) with \(x_i \neq x_j\), we record the difference in the predictor \(\Delta_{ij} = x_i - x_j\) and an indicator of whether the person with the higher predictor score also has the higher outcome, \[ c_{ij} = \mathbf{1}\!\left[(x_i - x_j)(y_i - y_j) > 0\right] + \tfrac{1}{2}\,\mathbf{1}\!\left[y_i = y_j\right], \tag{14}\] where ties on the outcome count as one half. A sample of \(n\) people gives \(n(n-1)/2\) pairs, and the mean of \(c_{ij}\) over all pairs is the sample estimate of \(\theta_b\) in Equation 3. The POSC is the regression of \(c_{ij}\) on \(|\Delta_{ij}|\), that is, the proportion of concordant pairs among pairs that differ by a given amount on the predictor.
The regression must satisfy Equation 6, and the simplest way to ensure this is to build the symmetry into the model. The logit of the curve is modeled as an odd function of the difference, \[ \operatorname{logit}\,\theta(\Delta) = \operatorname{sign}(\Delta)\, g(|\Delta|), \qquad g(d) = \sum_{k=1}^{K} \beta_k B_k(d), \tag{15}\] where the \(B_k\) are the basis functions of a natural cubic spline in \(d = |\Delta|\) with no intercept term (Hastie et al. 2009). The basis is constructed so that every \(B_k(0) = 0\), which forces \(\theta(0) = 1/2\), and the factor \(\operatorname{sign}(\Delta)\) provides the symmetry. Because a natural spline has zero curvature at its boundary, the two halves of the curve join smoothly at \(\Delta = 0\). The spline is a flexible parametric smoother in the sense that it has a finite number of coefficients. The curve is empirical in the sense that no assumption is made about the joint distribution of \(X\) and \(Y\). The number of degrees of freedom \(K\) controls the flexibility of the curve, with interior knots placed at quantiles of the observed differences. The default is \(K = 3\), and \(K = 1\) gives a curve that is linear on the logit scale. The curve is not constrained to be monotone, for reasons given in Section 3.4. The coefficients \(\beta_k\) are estimated by maximizing the binomial likelihood of the concordance indicators, which is ordinary logistic regression. The responses are shrunk by a negligible amount toward one half so that a solution exists even when large differences order the outcome perfectly. For large samples a random subset of pairs is used.
3.3 Confidence bands and binned checks
The pairs are not independent, because every person appears in \(n - 1\) of them. Standard errors from the logistic regression treat the pairs as independent and are therefore much too small. The package instead uses a nonparametric bootstrap that resamples people (Efron 1979; Efron and Tibshirani 1993). In each resample, \(n\) people are sampled with replacement, the pairs are rebuilt (omitting pairs of a person with their own duplicate, which carry no information), Equation 15 is refitted with the knots held fixed, and the procedure is repeated. The pointwise band at each \(\Delta\) is formed from the percentiles of the bootstrap curves. One thousand resamples give a stable 95% band.
The fitted curve can also be compared with the observed proportions. The pairs are grouped into bins of \(|\Delta|\), the proportion of concordant pairs is computed in each bin, and the proportions are plotted as points on the curve. Because these points are interpreted as a set, they are given simultaneous rather than pointwise intervals. The same bootstrap provides the joint sampling distribution of the bin proportions, and a sup-\(t\) critical value (Montiel Olea and Plagborg-Møller 2019) on the empirical logit scale gives intervals that cover all of the true proportions with the stated probability and that are asymmetric when a proportion is near 0 or 1. If the curve passes through these intervals, the spline is flexible enough. Figure 4 illustrates the construction with simulated data and a skewed outcome.
3.4 Nonmonotone relationships
A POSC describes pairs, and it averages over the location of the pair in the distribution of the predictor. Consider an inverted-U relationship, \(Y = -(X - c)^2\), such as the benefit of a drug that is greatest at an optimal dose \(c\), ineffective below it and toxic above it. For two patients who differ by \(\Delta\), the one with the higher value of \(X\) has the higher outcome exactly when their midpoint is below the optimum. If \(X\) is normal with mean zero, the midpoint is independent of the difference, and the POSC is constant at \(\Phi(c\sqrt{2})\) for every \(\Delta > 0\). When the optimum is at the mean of the predictor (\(c = 0\)), the POSC equals 0.5 everywhere, even though the predictor determines the outcome exactly. A higher value of \(X\) is beneficial for pairs below the optimum and harmful for pairs above it, and the two cases cancel in the population. When the optimum is half a standard deviation above the mean, the curve increases immediately to about 76% and remains there, so a larger difference on the predictor gives no further increase in the probability. The empirical POSC recovers curves of this kind, whereas the normal-theory curve, which depends only on the linear correlation, does not, and this is why the empirical method does not impose monotonicity. Figure 5 shows the two curves for simulated data in which clinical benefit is an inverted-U function of dose, with the optimum half a standard deviation above the mean dose, plus random error. A more flexible spline (df = 4) is used so that the curve can become flat.
delta p lower upper
1 0.5 0.7260 0.6671 0.7737
2 1.0 0.8041 0.7379 0.8519
3 2.0 0.8572 0.7717 0.9215The Pearson correlation is 0.67, so the normal-theory curve increases steadily toward 1. The empirical curve increases rapidly over small differences, where the random error dominates, and then becomes flat near the theoretical value of 0.76. Beyond a moderate difference, a higher dose does not increase the probability of greater benefit, because for pairs above the optimum the higher dose is harmful. Neither a correlation nor the normal-theory curve can describe this relationship. The empirical POSC, together with its binned proportions, incorporates the shape of the relationship into the effect size.
4 Software
4.1 Design
The posc package is written entirely in R and depends only on ggplot2 (Wickham et al. 2025) in addition to the base and recommended packages. Four constructor functions, one for each type of curve, return an object of class posc:
posc_uni(r, n, sd_x)computes the normal-theory curve of Equation 11 from a correlation, a sample size and the standard deviation of the predictor (Section 2.2).posc_multi(r_mat, n, outcome)computes the curve for the optimally weighted composite implied by a correlation matrix (Section 2.3).posc_lm(model)computes the curve for the fitted values of a model estimated withlm()(Section 2.4).posc_emp(x, y)estimates the empirical curve of Section 3 from raw data, with its bootstrap band and binned proportions.
A posc object is a list. Its main components are curve, a data frame containing the curve and its confidence limits on a symmetric grid of differences, and the quantities r, n, level and sd_x from which the curve was computed. The component details holds information specific to the type of curve (for an empirical curve, the spline knots and coefficients, the number of pairs and bootstrap resamples, and the binned proportions), and a prediction function evaluates the curve, with its confidence interval at any level, at arbitrary differences. The constructors differ only in how they build the prediction function, so the methods for the class are shared:
print()describes the curve and reports the probability at a few differences.summary(object, delta)andpredict(object, delta, level)return the probability and its confidence limits at any differences as a data frame.as.data.frame()returns the whole curve.plot()andautoplot()draw it.
The plot methods return ggplot2 objects, so anything not covered by their arguments, such as the wording of an axis label, can be changed in the usual way, for example with labs(). The methods take arguments for the common variations. Setting full = TRUE draws the full curve over negative and positive differences instead of the default half. Setting mark = delta marks and labels selected differences on the curve, and mark_ci = TRUE adds their confidence intervals. Setting percent = TRUE labels the probability axis in percentages. For empirical curves, bins = TRUE adds the binned proportions with their simultaneous intervals, and reference = TRUE adds the normal-theory curve for the same Pearson correlation. Two further functions operate on several curves. posc_comp() overlays any number of curves and tests whether their correlations differ (a Fisher \(z\) test for two curves and a homogeneity test for more than two), returning an object of class "posc_comparison" with its own print and plot methods. posc_bins() returns the binned proportions of an empirical curve with their simultaneous intervals as a data frame. Figure 6 shows the main variations of the plot method for one curve.
posc_uni(r = 0.5, n = 100, sd_x = 15). Top left: the default, the curve over positive differences with its 95% confidence band. Top right: full = TRUE, the full sigmoid. Bottom left: mark = c(10, 20), mark_ci = TRUE and percent = TRUE, which label chosen differences with their probabilities and confidence intervals on a percentage axis. Bottom right: a comparison of two curves from posc_comp(), labelled along the curves.
4.2 Basic use
The package is loaded in the usual way, and a curve is created with one of the constructors. The printed object reports the method, the correlation with its confidence interval, and the probability at differences of 0.5, 1 and 2 standard deviations of the predictor.
Probability of Outcome Superiority Curve
Method: bivariate normal
Predictor: X Outcome: Y
r = 0.500, 95% CI [0.337, 0.634], n = 100
P(higher Y | X higher by delta):
delta p lower upper
0.500 0.581 0.550 0.614
1.000 0.658 0.600 0.719
2.000 0.793 0.693 0.877The probability at any difference, with its confidence interval, is obtained with predict(), and the full curve with as.data.frame().
delta p lower upper
1 0.5 0.5809 0.5503 0.6141
2 1.0 0.6585 0.5998 0.7190
3 2.0 0.7929 0.6934 0.8769 delta p lower upper
1 0.000 0.5000 0.5000 0.5000
2 0.015 0.5024 0.5015 0.5035
3 0.030 0.5049 0.5030 0.50695 Illustrations
5.1 Prioritizing patients with a prognostic score
A rehabilitation program has one place available and two eligible patients. At intake, patients complete a prognostic score (0 to 100, with higher values indicating a better prognosis) that is used to predict functional recovery at six months. Patient A scores 34 and patient B scores 22. A validation study of \(n = 100\) earlier patients found a correlation of \(r = 0.52\) between the score and recovery, and the standard deviation of the score among referred patients is 10. The two patients differ by \(\Delta = 12\) points, so by Equation 11, \[ \hat\theta(12) = \Phi\!\left(\frac{0.52 \times 12}{10\sqrt{2(1 - 0.52^2)}}\right) = \Phi(0.517) = 0.697. \] Patient A therefore has approximately a 70% probability of recovering better than patient B. In the package, the predictor and outcome can be named so that the output and plots are labeled:
delta p lower upper
1 12 0.6973 0.6285 0.766The 95% confidence interval extends from 0.628 to 0.766, so the data support a modest advantage for patient A. Figure 7 shows the full curve with the two patients marked, and Table 1, produced with summary(), gives the probabilities at other differences. A 30-point difference, three standard deviations, would increase the probability to 0.90. By Equation 10, a difference of 15.7 points is required before the higher scorer has a 75% probability of being the patient who recovers better.
delta p lower upper
1 5 0.5852 0.5543 0.6188
2 10 0.6666 0.6076 0.7274
3 12 0.6973 0.6285 0.7660
4 20 0.8054 0.7076 0.8868
5 30 0.9017 0.7938 0.9652| Difference in prognostic score | Probability | 95% CI |
|---|---|---|
| 5 | 0.585 | [0.554, 0.619] |
| 10 | 0.667 | [0.608, 0.727] |
| 12 | 0.697 | [0.628, 0.766] |
| 20 | 0.805 | [0.708, 0.887] |
| 30 | 0.902 | [0.794, 0.965] |
5.2 Comparing prognostic measures
Suppose that three measures taken at intake could be used to predict recovery, and that in validation data their correlations with recovery are 0.51 for a baseline functional assessment (BF), 0.38 for a clinician’s global rating (CR) and 0.26 for a patient self-rating (PSR). These values are illustrative. posc_comp() accepts any number of curves, named or unnamed, and overlays them, and Figure 8 (left) shows the curves for the three correlations. The difference between the functional assessment and the clinician’s rating appears modest as a difference in correlations, but it means that two patients one standard deviation apart are ordered correctly 66% of the time by the assessment and 61% of the time by the rating.
When sample sizes are supplied, posc_comp() also tests whether the correlations differ. For two curves from independent validation samples of 100 and 120 patients:
Comparison of 2 probability of outcome superiority curves
curve method r n
POSC 1 normal 0.510 100
POSC 2 normal 0.380 120
Test of equal correlations (independent samples):
z = 1.18, p = 0.2365.3 Composites and linear models
posc_multi() accepts a correlation matrix of the predictors and the outcome. With an illustrative matrix in which baseline function, the clinician’s rating and treatment adherence correlate 0.51, 0.38 and 0.31 with recovery and are modestly intercorrelated, the optimally weighted composite has a multiple correlation well above that of any single predictor:
Probability of Outcome Superiority Curve
Method: bivariate normal, optimally weighted composite
Predictor: predictor Outcome: recovery
r = 0.629, 95% CI [0.572, 0.679], n = 500
(r adjusted for the number of predictors; unadjusted r = 0.632)
P(higher recovery | predictor higher by delta):
delta p lower upper
0.500 0.612 0.597 0.628
1.000 0.716 0.689 0.743
2.000 0.874 0.838 0.905The printed correlation is the adjusted multiple correlation of Section 2.3, and the unadjusted value, 0.632, is reported with it. Figure 3 compares the curve for the composite with the curves for the single predictors. posc_lm() uses a fitted model instead of a correlation matrix. For a simulated data set of 200 patients in which recovery depends on baseline function and adherence:
delta p lower upper
1 0.5 0.6443 0.6207 0.6700
2 1.0 0.7704 0.7307 0.8106
3 2.0 0.9306 0.8906 0.9608Here the differences are in units of predicted recovery, and the curve (Figure 8, right) gives the probability that the patient with the higher predicted recovery has the higher observed recovery.
5.4 An empirical POSC: performance status and survival
The VA data set in the MASS package (Venables and Ripley 2002) comes from a Veterans Administration trial of treatments for advanced lung cancer. The Karnofsky performance status of each patient at entry, a clinician’s rating of functional ability from 0 to 100, is a widely used prognostic score, and survival time in days is strongly skewed, so the relationship between the two variables is monotone but far from bivariate normal. The patients who died during follow-up are used here, with survival time treated as a continuous outcome, and the few censored patients are omitted for the purpose of the illustration. posc_emp() takes the two variables, fits the spline model of Equation 15, runs the bootstrap (1000 resamples by default) and computes the binned proportions (eight bins here). The printed output reports the Pearson correlation for reference, the degrees of freedom of the spline, and the numbers of pairs and resamples.
Probability of Outcome Superiority Curve
Method: empirical (spline logistic, bootstrap CI)
Predictor: Karnofsky score Outcome: survival time
r = 0.409 (Pearson), n = 128
Spline df = 3, pairs used = 7051, bootstrap resamples = 1000
P(higher survival time | Karnofsky score higher by delta):
delta p lower upper
9.950 0.593 0.551 0.641
19.900 0.692 0.638 0.751
39.801 0.869 0.797 0.933The Pearson correlation is 0.41, whereas the Spearman correlation is 0.60, and the difference between them corresponds to the difference between the empirical and normal-theory curves in Figure 9. For two patients whose Karnofsky scores differ by 20 points (one standard deviation), the empirical curve gives a probability of 0.69 (95% CI [0.64, 0.75]) that the patient with the higher score survives longer, compared with 0.62 under normal theory. The binned proportions, returned by posc_bins(), follow the empirical curve closely and lie above the normal-theory curve over the whole range. Assuming normality here would substantially understate how reliably performance status orders survival. The figure combines the scatterplot and autoplot(emp_va, bins = TRUE, reference = TRUE), with the vertical axis relabeled, using the patchwork package (Pedersen 2024).
delta_low delta_high delta n_pairs p lower upper
1 0 10 9.729 2104 0.5917 0.5252 0.6549
2 10 20 19.757 1747 0.6940 0.5984 0.7753
3 20 30 29.797 1279 0.7776 0.6523 0.8666
4 30 40 39.771 940 0.8830 0.7409 0.9518
5 40 55 49.883 642 0.9120 0.7055 0.9778
6 Summary and discussion
The probability of outcome superiority curve converts a correlation into the quantity that a decision requires, namely the probability that the person with the higher predictor score has the higher outcome, at the observed difference in the predictor. The curve contains the existing common language effect size for correlations as its average (Equation 7), it reduces to the probability of superiority for group differences at each level of the predictor (Equation 9), and it can be estimated either from a correlation under bivariate normality or directly from data without a distributional assumption. The posc package provides both approaches through one class of object with a small set of methods, so that a curve computed from a published correlation and a curve estimated from raw data are handled, plotted and compared in the same way.
The curve is closely related to a statistic that is already widely used in clinical research. The concordance index of Harrell et al. (1982) is the probability that, of two patients, the one with the higher predicted risk has the worse outcome. It is the standard summary of the discrimination of a prognostic model, and for a continuous outcome it is equal to \(\theta_b\) in Equation 3 with the prediction of the model as the predictor. The POSC is the concordance index expressed as a function of the difference between the two patients on the predictor, and by Equation 7 the index is the average of the curve. A model with a concordance index of 0.7 may discriminate well between patients who are far apart on its score and poorly between patients who are close together, and the curve shows which is the case.
Although the examples in this paper are clinical, the curve applies wherever a continuous predictor is used to choose between people. Personnel selection is the most direct application. A validity coefficient relating a test score to job performance can be expressed as the probability that, of two applicants whose scores differ by a given amount, the higher scorer will perform better, which is the question the person making the hiring decision faces. Other examples include educational admissions and placement, and any setting in which a risk or prognostic score is used to allocate a limited resource.
Several limitations should be noted. The curve describes a pair of people sampled from the population in which it was estimated. If the patients to whom the curve is applied are a restricted range of that population (for example, patients referred to a specialist clinic), or if the predictor is measured with error, the correlation used in Equation 11 should be the one that applies to the decision at hand, corrected in the usual ways, because any bias in the correlation carries over to the curve. A curve evaluated at a difference of \(\Delta\) describes which of two people is more likely to have the better outcome and says nothing about the outcome of either individual. A 70% probability of identifying the patient who will do better still implies an error rate of 30%. The normal-theory confidence band is accurate for a single predictor but only approximate for composites and linear models. The empirical curve requires the raw data and, because of the bootstrap, more computation than a formula. It should be preferred whenever the data are available and the relationship may be nonlinear, skewed or nonmonotone, and the binned proportions should be inspected to confirm that the spline is flexible enough. When only a correlation is available, which is the usual situation when a published study is being interpreted, the parametric curve is the only option, and its assumptions should be stated.
7 Computational details
The results in this paper were obtained using R 4.5.1 (R Core Team 2026) with the posc 0.2.0, ggplot2 4.0.3, patchwork 1.3.2, geomtextpath 0.2.0 (Cameron and Brand 2025) and MASS 7.3.65 packages. The three bootstrap fits take about two minutes in total. R itself and all packages used are available from CRAN at https://CRAN.R-project.org/, and the source of this paper, with a replication script, is available at https://github.com/MatthewBJane/posc.
Acknowledgments
The author wrote the original manuscript and the core of the posc package. Claude, a large language model developed by Anthropic, was then used under the author’s direction to revise the text and figures and to assist with extending and testing the code. The author specified the methods, reviewed and verified all code, analyses and results, and takes full responsibility for the content.
References
Ben-Shachar, Mattan S., Daniel Lüdecke, and Dominique Makowski. 2020. “Effectsize: Estimation of Effect Size Indices and Standardized Parameters.” Journal of Open Source Software 5 (56): 2815. https://doi.org/10.21105/joss.02815.
Cameron, Allan, and Teun van den Brand. 2025. Geomtextpath: Curved Text in Ggplot2. https://CRAN.R-project.org/package=geomtextpath.
Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum.
Cohen, Jacob, Patricia Cohen, Stephen G. West, and Leona S. Aiken. 2003. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. 3rd ed. Lawrence Erlbaum.
Dunlap, William P. 1994. “Generalizing the Common Language Effect Size Indicator to Bivariate Normal Correlations.” Psychological Bulletin 116 (3): 509–11. https://doi.org/10.1037/0033-2909.116.3.509.
Efron, Bradley. 1979. “Bootstrap Methods: Another Look at the Jackknife.” The Annals of Statistics 7 (1): 1–26. https://doi.org/10.1214/aos/1176344552.
Efron, Bradley, and Robert J. Tibshirani. 1993. An Introduction to the Bootstrap. Chapman & Hall.
Fisher, Ronald A. 1921. “On the ‘Probable Error’ of a Coefficient of Correlation Deduced from a Small Sample.” Metron 1: 3–32.
Grissom, Robert J. 1994. “Probability of the Superior Outcome of One Treatment over Another.” Journal of Applied Psychology 79 (2): 314–16. https://doi.org/10.1037/0021-9010.79.2.314.
Harrell, Frank E., Robert M. Califf, David B. Pryor, Kerry L. Lee, and Robert A. Rosati. 1982. “Evaluating the Yield of Medical Tests.” JAMA 247 (18): 2543–46.
Hastie, Trevor, Robert Tibshirani, and Jerome Friedman. 2009. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. Springer.
Jané, Matthew B. 2026. Posc: Probability of Outcome Superiority Curves. https://github.com/MatthewBJane/posc.
Kendall, Maurice G. 1938. “A New Measure of Rank Correlation.” Biometrika 30 (1–2): 81–93. https://doi.org/10.1093/biomet/30.1-2.81.
Li, Johnson Ching-Hong, and Rory M. Waisman. 2019. “Probability of Bivariate Superiority: A Non-Parametric Common-Language Statistic for Detecting Bivariate Relationships.” Behavior Research Methods 51 (1): 258–79. https://doi.org/10.3758/s13428-018-1089-5.
McGraw, Kenneth O., and S. P. Wong. 1992. “A Common Language Effect Size Statistic.” Psychological Bulletin 111 (2): 361–65. https://doi.org/10.1037/0033-2909.111.2.361.
Montiel Olea, José Luis, and Mikkel Plagborg-Møller. 2019. “Simultaneous Confidence Bands: Theory, Implementation, and an Application to SVARs.” Journal of Applied Econometrics 34 (1): 1–17. https://doi.org/10.1002/jae.2656.
Pedersen, Thomas Lin. 2024. Patchwork: The Composer of Plots. https://CRAN.R-project.org/package=patchwork.
R Core Team. 2026. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. https://www.R-project.org/.
Ruscio, John. 2008. “A Probability-Based Measure of Effect Size: Robustness to Base Rates and Other Factors.” Psychological Methods 13 (1): 19–30. https://doi.org/10.1037/1082-989X.13.1.19.
Vargha, András, and Harold D. Delaney. 2000. “A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong.” Journal of Educational and Behavioral Statistics 25 (2): 101–32. https://doi.org/10.3102/10769986025002101.
Venables, William N., and Brian D. Ripley. 2002. Modern Applied Statistics with S. 4th ed. Springer.
Wickham, Hadley, Winston Chang, Lionel Henry, et al. 2025. Ggplot2: Create Elegant Data Visualisations Using the Grammar of Graphics. https://CRAN.R-project.org/package=ggplot2.