TY - JOUR
T1 - The Effects of Unproctored Internet Testing on Applicant Pool Size and Diversity
T2 - Using Interrupted Time Series to Improve Causal Inference
AU - Auer, Elena M.
AU - Cavanaugh, Katelyn J.
AU - Petor, Jessica R.
AU - Kinney, Ted B.
AU - Landers, Richard N.
N1 - Publisher Copyright:
© 2022 The Author(s)
PY - 2022
Y1 - 2022
N2 - Although it is commonly assumed that implementing unproctored internet testing (UIT) in employee selection systems can result in increased applicant pool diversity, this assumption has not been explicitly tested. Thus, we analyzed the applicant pool composition of a major U.S.-based manufacturing organization across the span of over 8 years (N = 24,963) using an interrupted time series analytic approach. This allowed us to evaluate changes before and after the implementation of unproctored testing as well as changes following subsequent mobile device blocking. The results of this analysis suggested that although adding a UIT option appeared to increase the size of the applicant pool, the magnitude of this effect did not appear to differ between Black and White applicants. Furthermore, removing the option to apply on a mobile device dampened this general effect, but similarly, there were no differences in the magnitude applicant pool reduction between Black and White applicants. This evidence contradicts the common notion that UITs result in more diverse applicant pools, suggesting UIT’s primary value in this regard is increasing access to the application process across groups. Additionally, this study demonstrates the use of interrupted time series analysis as a powerful framework to understand longitudinal effects in real-world employee selection data.
AB - Although it is commonly assumed that implementing unproctored internet testing (UIT) in employee selection systems can result in increased applicant pool diversity, this assumption has not been explicitly tested. Thus, we analyzed the applicant pool composition of a major U.S.-based manufacturing organization across the span of over 8 years (N = 24,963) using an interrupted time series analytic approach. This allowed us to evaluate changes before and after the implementation of unproctored testing as well as changes following subsequent mobile device blocking. The results of this analysis suggested that although adding a UIT option appeared to increase the size of the applicant pool, the magnitude of this effect did not appear to differ between Black and White applicants. Furthermore, removing the option to apply on a mobile device dampened this general effect, but similarly, there were no differences in the magnitude applicant pool reduction between Black and White applicants. This evidence contradicts the common notion that UITs result in more diverse applicant pools, suggesting UIT’s primary value in this regard is increasing access to the application process across groups. Additionally, this study demonstrates the use of interrupted time series analysis as a powerful framework to understand longitudinal effects in real-world employee selection data.
KW - diversity
KW - employee selection
KW - interrupted time series
KW - longitudinal design
KW - unproctored internet testing
UR - http://www.scopus.com/inward/record.url?scp=85145855751&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85145855751&partnerID=8YFLogxK
U2 - 10.1037/tmb0000079
DO - 10.1037/tmb0000079
M3 - Article
AN - SCOPUS:85145855751
SN - 2689-0208
VL - 3
JO - Technology, Mind, and Behavior
JF - Technology, Mind, and Behavior
IS - 3
ER -