Validity Stability Across Entering College Cohorts: Exploring the temporal generalizability of local validity estimates

Christopher R. Huber; Nathan R. Kuncel; Paul R. Sackett; Adam S. Beatty

doi:10.1111/ijsa.12111

Validity Stability Across Entering College Cohorts: Exploring the temporal generalizability of local validity estimates

Christopher R. Huber, Nathan R. Kuncel, Paul R. Sackett, Adam S. Beatty

Psychology (Twin Cities)

Research output: Contribution to journal › Article › peer-review

3 Scopus citations

Abstract

Local validity studies rely on the assumption that validity estimates from one incumbent sample approximate validity for future applicant pools. We test this assumption using SAT scores and high school grades as predictors of first year college grade point average across multiple college applicant pools for over 100 schools. We present evidence for substantial absolute and rank order consistency in validity estimates. However, this consistency is far less than perfect, resulting in potentially meaningful utility differences over time. In addition, observed fluctuations are not fully explained by sampling error alone.

Original language	English (US)
Pages (from-to)	237-246
Number of pages	10
Journal	International Journal of Selection and Assessment
Volume	23
Issue number	3
DOIs	https://doi.org/10.1111/ijsa.12111
State	Published - Sep 1 2015

Bibliographical note

Publisher Copyright:
© 2015 John Wiley & Sons Ltd.

Access

10.1111/ijsa.12111

OpenUrl availability

Full text

Cite this

@article{de115c9ee97a44d6973def98ab346e82,

title = "Validity Stability Across Entering College Cohorts: Exploring the temporal generalizability of local validity estimates",

abstract = "Local validity studies rely on the assumption that validity estimates from one incumbent sample approximate validity for future applicant pools. We test this assumption using SAT scores and high school grades as predictors of first year college grade point average across multiple college applicant pools for over 100 schools. We present evidence for substantial absolute and rank order consistency in validity estimates. However, this consistency is far less than perfect, resulting in potentially meaningful utility differences over time. In addition, observed fluctuations are not fully explained by sampling error alone.",

author = "Huber, {Christopher R.} and Kuncel, {Nathan R.} and Sackett, {Paul R.} and Beatty, {Adam S.}",

note = "Publisher Copyright: {\textcopyright} 2015 John Wiley & Sons Ltd.",

year = "2015",

month = sep,

day = "1",

doi = "10.1111/ijsa.12111",

language = "English (US)",

volume = "23",

pages = "237--246",

journal = "International Journal of Selection and Assessment",

issn = "0965-075X",

publisher = "Wiley-Blackwell",

number = "3",

}

TY - JOUR

T1 - Validity Stability Across Entering College Cohorts

T2 - Exploring the temporal generalizability of local validity estimates

AU - Huber, Christopher R.

AU - Kuncel, Nathan R.

AU - Sackett, Paul R.

AU - Beatty, Adam S.

PY - 2015/9/1

Y1 - 2015/9/1

N2 - Local validity studies rely on the assumption that validity estimates from one incumbent sample approximate validity for future applicant pools. We test this assumption using SAT scores and high school grades as predictors of first year college grade point average across multiple college applicant pools for over 100 schools. We present evidence for substantial absolute and rank order consistency in validity estimates. However, this consistency is far less than perfect, resulting in potentially meaningful utility differences over time. In addition, observed fluctuations are not fully explained by sampling error alone.

AB - Local validity studies rely on the assumption that validity estimates from one incumbent sample approximate validity for future applicant pools. We test this assumption using SAT scores and high school grades as predictors of first year college grade point average across multiple college applicant pools for over 100 schools. We present evidence for substantial absolute and rank order consistency in validity estimates. However, this consistency is far less than perfect, resulting in potentially meaningful utility differences over time. In addition, observed fluctuations are not fully explained by sampling error alone.

UR - http://www.scopus.com/inward/record.url?scp=84939808635&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84939808635&partnerID=8YFLogxK

U2 - 10.1111/ijsa.12111

DO - 10.1111/ijsa.12111

M3 - Article

AN - SCOPUS:84939808635

SN - 0965-075X

VL - 23

SP - 237

EP - 246

JO - International Journal of Selection and Assessment

JF - International Journal of Selection and Assessment

IS - 3

ER -

Validity Stability Across Entering College Cohorts: Exploring the temporal generalizability of local validity estimates

Abstract

Bibliographical note

Access

OpenUrl availability

Other files and links

Fingerprint

Cite this