The use of best visual acuity over several encounters as an outcome variable: An analysis of systematic bias

PURPOSE. To investigate whether the use of the best of multiple measures of visual acuity as an endpoint introduces bias into study results. METHODS. Mathematical models and Monte Carlo simulations were used. A model was designed in which a hypothetical intervention did not influence the visual acuity. The best of one or more postintervention measures was used as the outcome variable and was compared to the baseline measure. Random test-retest variability was included in the model. RESULTS. When the better of two postintervention measures was used as the outcome variable with a sample size of 25, the model falsely rejected the null hypothesis 55% of the time. When the best of three measures was used, the false-positive rate increased to 90%. The probability of falsely rejecting the null hypothesis increased with increasing sample size, also increasing the number of measures used to select the outcome variable. CONCLUSIONS. Using the best of multiple measures as an outcome variable introduces a systematic bias resulting in false conclusions of improvement in that variable. The use of best of multiple measures of visual acuity as an outcome variable should be avoided.