by Jane Willenbring, Scripps Institution of Oceanography, University of California San Diego
by Jane Willenbring, Scripps Institution of Oceanography, University of California San Diego
Frequently asked questions regarding GRE and Graduate Admissions
Like you, I’ve struggled with what to use for graduate admission criteria in the Geosciences. A little over a year ago, I intended to write about why we should use the GRE for admissions, but the more I read, the more I decided it was a flawed measure. Since writing my post, To GRE or not to GRE, I’ve received several questions that I’d like to respond to here, publicly. Given that COVID -19 has drastically changed testing, I also discuss that.
WHAT
What is the GRE?
The Graduate Record Examinations is a standardized test that is an admissions requirement for many graduate schools in the United States (and Canada). It is administered by the company ETS. Though technically a non-profit organization, ETS reports [1] net income of $66M in 2017 including their executive compensation of $17M.
[1] https://projects.propublica.org/nonprofits/organizations/210634479
What does the GRE measure?
The GRE exam has three components: quantitative reasoning, verbal reasoning and analytical writing and it is intended only to be a predictor of first-year graduate school GPA [2].
[2] https://news.ets.org/stories/gre-test-validity-putting-it-in-perspective/
Does the GRE predict ‘success’ in graduate school?
There’s not much data, but so far, the answer seems to be: no. One study found that GRE scores of male graduate students are actually negatively correlated with PhD completion rate [3]. In fact, none of the supposedly ‘objective’ credentials like GRE score (or GPA) predicted scientific productivity (i.e. first-author publications, conference presentations, fellowships or grants won, completing the Ph.D., passing the qualifying exam, or proceeding swiftly to dissertation defense or to the degree) [4]. One study did find a (barely: p=0.048) statistically significant correlation between PhD completion and the quantitative GRE in physics applicants but not for US Men or US Women groups [5].
[3] Multi-institutional study of GRE scores as predictors of STEM PhD degree completion: GRE gets a low mark https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0206570
[4] Posselt, J. 2016. Inside Graduate Admissions: Merit, Diversity, and Faculty Gatekeeping https://www.jstor.org/stable/j.ctvjghw8s
[5] Miller, C.W., Zwickl, B.M., Posselt, J.R., Silvestrini, R.T. and Hodapp, T., 2019. Typical physics Ph. D. admissions criteria limit access to underrepresented groups but fail to predict doctoral completion. Science advances, 5(1), p.eaat7550.
Aren’t these studies of the GRE not correlating with student success in a PhD program just conditioning on a collider?
The heart of this question is the analogous observation that height does not correlate with total point scoring for professional basketball players. A perfect GRE experiment would take a random sampling of a sufficiently large number of students graduating from university to establish a realistic range of GRE scores, make them take the GRE, admit them to graduate school and then see how GRE scores correlates with the PhD completion, papers published and other metrics one would use to assess ‘success’ in graduate school. That experiment will never happen and would also be inherently flawed. Fortunately, the most relevant experiment to this question was already done at Vanderbilt University, where GRE scores were not used for admissions decisions, but were still available as metadata. Though the sample size is low, the results are informative, and no threshold score emerges, despite there being a considerable range in the GRE score [6].
[6] Sealy, L., Saunders, C., Blume, J. and Chalkley, R., 2019. The GRE over the entire range of scores lacks predictive ability for PhD outcomes in the biomedical sciences. PloS one, 14(3), p.e0201634 https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0201634
What is GRExit?
GRExit is a clever term for the mass exodus of graduate programs requiring the GRE [7
[7] An (incomplete) list of Geo programs permanently abandoning the use of the GRE is here: https://docs.google.com/spreadsheets/d/1ZUZowZJ25DtqCgr3kK_Ief1C0YOnrZeADcm4K81bq08/htmlview and many departments and institutions are temporarily abandoning it in 2020 because of the extra burden to low income students (see below for details on the impact of COVID on GRE test-taking).
Should we really succumb to peer pressure and let GRExit influence our decision?
One argument in favor of eliminating the GRE is that with so many departments no longer requiring it, many students may decide it is not worth the hassle of taking it just to apply to the departments still requiring it. This point is particularly important when considering how a university or department might want to signal how they are trying to diversify the graduate student body.
We have used the GRE for years and have had great, successful students. Why mess with a system that works?
A large body of evidence shows that status quo bias, or an emotional preference for the current state of affairs, frequently affects human decision-making. Given that intelligence and ability are not genetically or racially determined [8], any admission measure that systematically excludes URM or low-income students is discriminatory and flawed, i.e. not ‘working’. As a whole, we are not improving diversity in the geosciences.
[8] Saini, A., 2019. Superior: the return of race science. Beacon Press.
EQUITY AND THE GRE
How does COVID19 impact students taking the GRE?
Because the GRE must be taken at home and in a specific way this year, taking the exam is not possible for some students. One personal account of what it is like *this year* to take the exam from home describes the structural barriers for low income students [9].
[9] https://threadreaderapp.com/thread/1271195100970799105.html
Why is the GRE considered biased?
GRE scores correlate strongly with race and sex [10][11] even though intelligence and ability are not determined by one’s race or sex [8][12]. Reasons why the GRE may be biased based on economic status are related to the fact that affluent test takers can access test prep that can be prohibitively expensive for some students. The GRE also includes unidentified sections of ‘trial questions’ that are unscored [13]. Some students whose first language is not English might spend more time on poorly worded questions that are later discarded by ETS. Although ultimately unscored, these questions might affect time allocation.
[10] Clayton, V. 2016. The Problem With the GRE. The Atlantic. https://www.theatlantic.com/education/archive/2016/03/the-problem-with-the-gre/471633/
[11] Miller, C.W., Zwickl, B.M., Posselt, J.R., Silvestrini, R.T. and Hodapp, T., 2019. Typical physics Ph.D. admissions criteria limit access to underrepresented groups but fail to predict doctoral completion. Science Advances, 5(1), p.eaat7550. DOI: 10.1126/sciadv.aat7550
[12] Saini, A., 2017. Inferior: How science got women wrong and the new research that’s rewriting the story. Beacon Press.
[13] https://www.ets.org/gre/revised_general/about/content/computer/
Doesn’t taking the GRE from home during COVID actually decrease barriers for students with disabilities?
No. Some students requiring special accommodations (including hearing- or sight-impaired students) are not allowed to take the exam from home and need to venture out to a few select group-testing locations during a pandemic.
What is the cost of taking the GRE?
This year, the GRE costs $205 to take and each additional school to which the report is sent costs $27 [8].
Why is the GRE considered unaffordable to low income students if they offer a fee waiver?
The GRE does allow a fee waiver for low income students to 50% of the original cost [14], but students of substantial financial means know they can retake the exam multiple times if they happen to perform badly, which reduces stress for financially stable students. The stakes are higher and stress higher for low income students. It is well known that stress decreases ability to perform.
[14] https://www.ets.org/s/gre/pdf/gre_fee_reduction_request_form.pdf
NOW WHAT?
Isn’t there bias or other problems with every quantitative measure used for graduate school admissions?
Yes, but using multiple biased tools does not solve the problem. NSF Graduate Research Fellowships stopped using GREs several years ago.
Isn’t there as much or more bias in qualitative measures used for graduate school admission?
Yes. For example, one study focusing on postdoctoral applications [15] showed that women (in Earth Science) are only about half as likely as men to receive letters containing language that describes them as excellent, rather than just good. One could imagine that this effect would be impacted by race and ethnic minority status as well.
[15] Dutt, K., Pfaff, D.L., Bernstein, A.F., Dillard, J.S. and Block, C.J., 2016. Gender differences in recommendation letters for postdoctoral fellowships in geoscience. Nature Geoscience, 9(11), pp.805-808. https://www.nature.com/articles/ngeo2819
Isn’t it better to use the GRE as an additional piece of information rather than ignoring it?
No. It is better to use other metrics rather than flawed, discriminatory metrics.
If we don’t use the GRE for admissions because it is biased, what do we do?
Experts suggest holistic admissions [16].
[16] Benderly, B.L., 2017. GREs don’t predict grad school success. What does? https://www.sciencemag.org/careers/2017/06/gres-dont-predict-grad-school-success-what-does
What is ‘holistic admissions’?
‘Holistic admission’ is a strategy that assesses an applicant’s unique experiences before or alongside traditional measures of academic achievement such as grades. It includes assessing each students’ application fully prior to ranking based on quantitative measures such as GPA. Read more about inclusive practices from the Equity in Graduate Education group [17].
[17] https://sites.google.com/igenetwork.org/equity-in-graduate-education/
Why should the GRE not be part of holistic admissions?
If the case is made that GRE results are discriminatory, then including them, even on an optional basis, will preferentially advantage affluent, white students.
Jane Willenbring is an Associate Professor at Scripps Institution of Oceanography, University of California San Diego, a GSA Fellow and an Op-Ed Project alum
###
Speaking of Geoscience 