Another take on test scores

Recent list work: #9, #94

I've spent much of today catching up on recent issues of Nature and Science, as I've not been reading them as much this summer as I should. So, this is old news, but I just came across it today.

For those with access to Science, a few letters appear in the June 22nd issue regarding a study from February advancing a claim that higher standardized test scores are a predictor in graduate student success, as measured by things such as publications, etc. The first of these is the one I think is the more insightful.

As the letter's authors (Manuel Lerdau and Christopher Avery) point out, the initial finding hypothesized the twin causal links: 1) greater aptitude --> higher test scores, and 2) greater aptitude --> increased probability of success. That's a potentially valid causal chain. On the other hand, they suggest that the causal chain might be more along the lines of higher test scores --> greater support (both financial and mentoring) --> greater probability of success. This also strikes me as potentially valid.

As the letter writers state, these two different causal chains lead to very different policy views. I think it would be interesting to try to disentangle the two effects. One way of doing so off the top of my head would be to look at departments where students are given essentially the same financial support and teaching duties regardless of whether they have fellowships or not. (For instance, Stanford's Department of Biological Sciences). Another would be to look at students who have the very high test scores but don't have fellowships for other reasons--perhaps by being non-citizens and therefore disqualified from national fellowships such as those through the NSF. It would also be interesting to look at students who have fellowships and thus don't need to teach, yet have lower standardized test scores than peers in their departments. There could be some difficult things to control for in these examples--for instance, students with low test scores who have fellowships most likely have significant undergraduate research experience, which might let them hit the ground running--but I think the general idea of studying these things is worth exploring.