June 03, 2004

GREs--Why?

According to this Brian Leiter post, GREs are taken very seriously by all top philosophy departments, up to 1/3 of the ranking at one school. I can't help but wonder--why? It seems to me that emphasizing GREs rewards standardized-test geeks such as myself over people who may have shown that they can do philosophy. Except for the Analytic part of the test, which tracks logic, why should a good performance on the GRE predict a good performance as a philosopher? Comments welcome.

Posted by Matt Weiner at June 3, 2004 04:59 PM
Comments

I don't think that the Analytic part of the GRE tracks logic well. At least it didn't in my case. My thought process tends to be careful and deliberate, but that's penalized on a timed exam.

Why do admissions committees use GRE scores? Because GRE scores are the only information on the application that's 'objective', at least in the sense that everyone takes the same test. It's more difficult to compare students by other factors, such as GPA, because different schools have different standards.

This is just an explanation for the use of GRE scores in admissions, not necessarily a justification.

Posted by: ABD at June 3, 2004 07:05 PM

It's telling that philosophy grad schools are interested in the general GRE rather than the philosophy subject test.

General GREs are good predictors of grad school grades. There's no mystery here. The predictive success of the GRE is a testament to the statistical muscle of the ETS. They start with a large sample of grad students with known GPAs. Then they pepper them with questions. If high GPAs are more likely to get a question right than low GPAs, the question stays. The content of the question doesn't really matter.

The correlation between GRE and GPA is imperfect. Besides, GPA in only one measure of academic success. Who knows if the general GRE is as good for predicting philosophy grades as it is for predicting grad school grades on average? Still, it's important to have one objective variable in admissions.

Posted by: Lindsay Beyerstein at June 3, 2004 08:20 PM

Standardized test-taking geek? Ah, yet another piece of the puzzle falls into place...you were one of those people whom I envied. How I wished that I could do well on the GRE, as well as friends who made it look effortless!

Posted by: Linda Fiorentino at June 3, 2004 09:19 PM

consider cornell and mit; both schools do not use the gre in their admission process. both programs are first-rate. i think this is telling about what the gre can and cannot say about *philosophical* ability.

Posted by: d at June 3, 2004 11:20 PM

I don't even think the analytic bit is part of the test nowadays. When I took it last year there was just quantitative, verbal and some dumb essay questions.

Posted by: Jonny at June 4, 2004 12:29 AM

I used to think that the world would be a better place if the GREs were relegated to the waste-basket. And maybe it would be. Unfortunately, I think that about pretty much every component of the admissions package (letters, writing sample, etc.). None of them strike me, either individually or jointly, as reasonable /comparitive/ measures of philosophical ability. (I suspect that letters of recommendation are the single best gauge, especially if little or no weight is placed on the name of the letter writer.) The problem is that there is just too much background variability in prospective students. Taken together, the admissions packet probably selects for some package of characteristics which includes philosophical ability but which also includes a host of other features like parental education level, parental income, parental values concerning education, parental ability, early student emotional maturity, undergrad advisor competence, etc., etc. ... The point is that selection for philosophical ability more or less in isolation from these other factors would be a massive (practically impossible) epistemological problem.

So it seems to me that worrying about the GRE specifically makes it appear as if there aren't similar issues involved in each of the other factors. The moral, I think, is that grad admissions as it stands does a pretty good job of successfully picking candidates that will succeed (relatively few false positives) at the cost of giving a reasonably high number of false negatives. And I suspect that the best way of dealing with the latter fact is systemic--allow MA and PhD programs to proliferate so that outliers can stay in the game if they so desire (assuming all the disclosure caveats about job success outside the top 20).

Posted by: marc at June 4, 2004 11:24 AM

I can confirm that the current GRE format no longer includes an analytic multiple guess section. The analytic writing section has replaced it.

I find ETS to be a total racket and I'm highly dubious as to what the GRE test will tell one about a candidate. To say that there is a correlation between GPA and GRE scores is misleading since GRE scores have been used as a screening tool for years. Further I'm not sure what portion of the GRE would be a good indicator of possible success in philosophy. At best is seems that the verbal section should be what is weighed the most since vocabulary and reading comprehension seem to be more important than the ability to memorize math formulas.

Letters also seem to be problematic. I’ve heard many reports that letter inflation is just as bad a grade inflation. Much like letters of recommendation for jobs the graduate school letters are “essentially devoid of cognitive content.”

Posted by: Matthew at June 4, 2004 01:00 PM

One of my professors once told me that she considers the GRE to be especially important for egalitarian reasons. Students who went to good schools, she said, are more likely to have received good help in preparing their writing samples, so there's no indication that a good paper is indicative of ability. The GRE, on the other hand, is taken alone. (And, as another poster said, everyone takes the same one (technically, everyone takes a different one, since they're computer-adaptive and weird).)

I don't agree with her completely -- it seems to me that having received good help in preparing a writing sample is likely to correlate with having received a quality philosophical education. But that's the argument I've heard.

Posted by: Jonathan at June 4, 2004 03:13 PM

Well, I believe I've betrayed my age--when I took the GRE, not only did it have a multiple-choice analytic section, it involved bubbling in answers with a No. 2 pencil. (I think I must have been the last person to do this--computerized GREs were available that year, but I started applying to grad school too late to sign up for them.)

Lindsay--Is there a philosophy subject test? I looked at the ETS web site and couldn't find one. Perhaps one concern is that the subject test would test breadth of education, which the grad schools would feel they could fix on their own.

Marc--As I've mentioned elsewhere (maybe not on this site), I'm opposed to proliferation of PhD programs, but strongly in favor of proliferation of MA programs--and in fact, making an MA pretty much a prerequisite for PhD programs. One of the advantages of this is that it would give PhD programs something substantial to admit based on. But--I am curious as to why you think writing samples are devoid of cognitive content; is it for the reasons Jonathan gives?

The problem I have with the GRE is that it doesn't seem as though it should correlate even weakly with philosophical ability.

As far as grade inflation goes, Jordan Ellenberg has an interesting argument that it doesn't much matter--across a college career, better students will wind up with higher grades. His argument gets weaker when you restrict it to grades in major, since that will be averaged over fewer courses, but it's still interesting. I take it that a lot of people on the thread would disagree with this statement of his:

Anyone who's read fellowship applications, or graduate school admission folders, knows that the best undergraduates aren't hard to pick out—they're the ones who excel in nearly every course, the ones with a healthy sprinkling of A+s, the ones whose recommendation letters read like mash notes.

(In fact, I have one argument already--you can't give an A+ at the University of Utah, so one of his factors can't help our undergrads.)

[Disclosure: My mom works for ETS, though not on the GRE, and Jordan was a friend in college.)

Posted by: Matt Weiner at June 5, 2004 01:08 PM

Perhaps the problem begins even earlier, back with undergraduate admissions, which is an even bigger crapshoot than graduate admissions. There are two issues:

(1) Your high school record (GPA, SAT scores, athletic achievement, etc.) and your parents' ability to pay are even worse indicators of philosophical potential than your undergraduate record.
(2) On average, you'll probably get a better undergraduate education at the more prestigious schools.

Thus, applicants to graduate school who come from the more prestigious schools have a double advantage: the very name of the schools impress the admissions committee, and the applicants from those schools are probably better prepared (though not necessarily better philosophers).

The same thing happens, to a lesser extent, with junior faculty hiring.

Basically, there are no really good criteria for predicting philosophical potential until the person actually achieves that potential. Mistakes will be made at every level, from undergraduate admissions through hiring. Evaluation gets better over time, however, so the best way to deal with mistakes is to have ample opportunities for correcting those mistakes. That's why I think that Matt's idea is a good one: require an MA for admission to a PhD program.

Posted by: ABD at June 7, 2004 07:45 PM

the question I have is why dont grad schools in the US do interviews?

The application package looks very nice but I think that an experienced professor is better at differnetiating a journeyman from a genius thatn the GRE.

Posted by: Giles at June 8, 2004 03:04 PM

Matt wrote: I'm ... strongly in favor of ... making an MA pretty much a prerequisite for PhD programs.

That, as it happens, is pretty much how graduate education works in Canada, the UK, Australia and New Zealand. While technically most Canadian PhD programs are supposed to admit students with honours degrees, such applicants rarely get admitted and even more rarely get funding, since they are competing against a host of applicants with honours degrees and MAs.

Posted by: Nicole Wyatt at June 9, 2004 05:04 PM