By Miranda McDaniel
Most of us likely remember the horror that was the experience of standardized testing. Armed with a sharpened #2 pencil, an interrupted too-few hours of sleep, and a healthy sense of self-doubt, we marched onto the intellectual battlefield ready to prove our entirely knowledge-based academic worth. Although standardized testing certainly reared its ugly head during our K-12 days, the most important of the exams appeared to bookend our undergraduate studies. The Scholastic Aptitude Test (SAT) or the American College Test (ACT) awarded us the perceived qualifications of being prepared for college and, even after earning our pre-doctoral degrees, the Graduate Record Examination (GRE) was necessary to measure and predict our ability to perform well in graduate school.
Now I don’t know about you, but the ability to correctly use the word “jejune” in a sentence or to determine which train will make it to the station first if one left at time W with speed X and the other at time Y with speed Z, doesn’t exactly represent my ability to troubleshoot a failed experiment or design a new one. As it turns out, more and more universities are also becoming skeptical about the validity of the exam for evaluating academic potential. Recently, several universities across the country have begun investigating the effectiveness of GRE scores for predicting successful completion of doctoral programs, performance on qualifying exams, time to degree, publication count, and more.
Relationships between GRE Quantitative quartile (Q) scores and PhD completion rates for men in four state flagship universities.
In one study, GRE scores and PhD completion data were collected from institutions in the Northeast Alliance for Graduate Education and the Professoriate (NEAGEP)(1). The multi-institutional study investigated the relationship between GRE scores and degree completion specifically for science, technology, engineering, and mathematics (STEM) doctoral programs. Their results showed that the GRE Verbal (GRE V) and the GRE Quantitative (GRE Q) scores were similar for women who completed STEM PhD degrees and those who did not, implying that the GRE score was not a good predictor of future graduate school success. Moreover, GRE scores were significantly higher for men who left their programs than those who completed their degrees. So, the exam is not only a poor predictor of degree completion, but at least for men, it appears to select for students who are less likely to complete their degree! Now, I’m not going to sit here and say “I told you so”, but I will say that I am feeling extremely validated by this finding.
Vanderbilt University and University of North Carolina at Chapel Hill similarly explored the relationship between GRE scores and graduate student success in biomedical research(2,3). Their results showed that the GRE did not prove to be a reliable indicator for predicting who will graduate with a PhD, pass the qualifying exam, earn their degree faster, or achieve a more notable research presence. In addition to providing evidence to show that the GRE is not sufficient to predict success, these universities also advocate for consideration of the more qualitative credentials that students use in their applications, including letters of recommendation or research experience, without relying too heavily on one specific component of the application. In this way, more recognition is granted to those of us who meticulously balanced classwork with research obligations during our undergraduate studies with the stringency usually reserved for checkbooks or Thanksgiving dinner preparation.
While GRE scores appear not to correlate with graduate school performance, they do trend with socioeconomic status, gender, and ethnicity. According to an article published in The Atlantic, research from the University of Florida, Stanford, New York University, and the University of Missouri showed that the GRE under-predicts the success of minority students(4). This message was echoed by an article featuring data from Educational Testing Service (ETS) in Princeton, New Jersey(5). The article explains that women, minorities, and those with a lower socioeconomic status perform lower on average, stating that “in simple terms, the GRE is a better indicator of sex and skin colour than of ability and ultimate success”. This relationship becomes increasingly important to consider when a minimum acceptable GRE score is used as a threshold to easily identify applicants who are “most qualified” to join a graduate program before the application reviewing process even begins. Such elimination criterion lends itself to a lack of diversity in the applicant pool and, ultimately, the graduate program itself.
So, if the GRE doesn’t predict future success and selects for limited diversity, why has it stuck around for more than 80 years? An article featured in Inside Higher Ed addressed this question, arguing for the GRE to be upheld as a requirement(6). The appeal of the GRE for admission committees is, of course, its quantitative and objective nature. The article points out that without such a measure, admission committees are challenged to eliminate subjective interpretation of application materials that could introduce bias into the decision-making process. This task is made more difficult by the inability for admission committees to reliably compare letters of recommendations or grade point averages across universities. The article states that, “the objective, comparative data that scores yield is especially helpful when comparisons are difficult to make, such as when evaluating applicants from unfamiliar undergraduate institutions or from countries with different educational and grading systems.”
While a single numerical score certainly aids in reducing biases and other comparative discrepancies, this value alone is insufficient to serve as an elimination criterion or predictor of success. So how should universities decide which students to accept into their graduate programs? An article published in Nature states that “the best way of predicting a scientist’s future success is for peers to evaluate scientific contributions and research depth”(7). As such, admission committees stand to benefit from diligent consideration of testimonies provided by recommendation letters. While the in-depth review of such material will take longer to process than a single numerical score, this approach would greatly increase the ability of graduate programs to select for the students that are most likely to find success in graduate school. If additional time spent reviewing recommendation letters has the potential to protect graduate programs from selecting students who will not follow through with their degrees, then this certainly seems like a worthwhile investment.
In response to these studies, there has been a trend among biological and biomedical graduate programs to no longer require the GRE. Universities have begun dropping the requirement altogether, shifting their focus toward the more accurate criteria for predicting graduate student success and a more holistic approach to evaluating academic acumen. In fact, seven of the eight programs in the Graduate Division of Biological and Biomedical Sciences at Emory University no longer require the GRE
By moving away from the inherently inequitable standardized test, graduate schools are able to expand their applicant pool to select for a diverse student population with the highest chances of success. This adjustment to the way admission decisions are made benefits both the programs and the students and can provide opportunities to students who might otherwise have been overlooked as a result of less-than-stellar GRE scores. Beyond just the weight of standardized testing being lifted from the shoulders of the already anxious college senior, more emphasis on undergraduate research experience is likely to encourage more students to start early on building a presence in the academic arena. Instead of undergraduate students putting all their metaphorical eggs in the GRE basket, students will be motivated to gain research experience, inspired to develop a wider skillset, and more likely to head into their graduate studies with a better chance of success.
References:
1. Petersen, S. L., Erenrich, E. S., Levine, D. L., Vigoreaux, J., & Gile, K. (2018). Multi-institutional study of GRE scores as predictors of STEM PhD degree completion: GRE gets a low mark. PloS one, 13(10), e0206570.
2. Moneta-Koehler, L., Brown, A. M., Petrie, K. A., Evans, B. J., & Chalkley, R. (2017). The limitations of the GRE in predicting success in biomedical graduate school. PloS one, 12(1), e0166742.
3. Hall, J. D., O’Connell, A. B., & Cook, J. G. (2017). Predictors of student productivity in biomedical graduate school applications. PloS one, 12(1), e0169121.
4. Clayton, V. (2016, Mar 1). The Problem with the GRE [Blog Post]. Retrieved from https://www.theatlantic.com/education/archive/2016/03/the-problem-with-the-gre/471633/
5. Miller, C., & Stassun, K. (2014). A test that fails. Nature, 510(7504), 303-304.
6. Payne, D. (2018, May 21). The Value of Testing in Graduate Admissions [Blog Post]. Retrieved from https://www.insidehighered.com/admissions/views/2018/05/21/standardized-testing-needed-graduate-school-admissions-opinion
7. Acuna, D. E., Allesina, S., & Kording, K. P. (2012). Future impact: Predicting scientific success. Nature, 489(7415), 201.