U.S. News & World Report (USN&WR) magazine did not invent college rankings, but its annual fall issue has dominated the market for the past two decades.  They continue to capture the public fancy despite evidence revealing their vapid content, serious methodological flaws, and limited influence on where the vast majority of undergraduates go to college. The major criticism is that the rankings are based primarily on what an institution has – its reputation and resources – and say nothing about what students do during college or how they benefit after they matriculate. Indeed, we developed the National Survey of Student Engagement a dozen years ago in part to emphasize that how students use an institution’s resources to educational advantage is far more important than the resources themselves.

Efforts to game some of the variables in the ranking algorithm also are well-documented, such as omitting entrance exam scores of selected groups of students to inflate the institutional average. Some institutions recalculated the number of alumni eligible to contribute to the annual fund (e.g., graduates for whom no current mailing address was available were removed) in order to shrink the denominator, thereby artificially boosting the percentage who do (appear) to support their alma mater.  Not only do some institutional leaders rate their school well above peer institutions on the reputation measure, thereby increasing the odds that their own school will fare better, Christopher Morphew and Barrett Taylor found that many colleges and universities modify their mission statement for posting on the USN&WR website.  Of one hundred colleges they checked, only six submitted their official mission statement, contrasted with more than half which submitted prose “entirely dissimilar to the official mission statement.”  This may be much ado about nothing in terms of influencing college choice decisions, inasmuch as the written espoused mission may or may not comport with the enacted mission, which my studies of high- performing institutions over the past twenty years show is far more important to student success.  The disturbing implication of the Morphew and Taylor discovery is that when schools tamper with their primary public declaration of what they stand for, how can we have confidence in anything such institutions say?

These revelations underscore a profoundly insidious influence of rankings–encouraging colleges and universities and those who support them to privilege and perpetuate status attainment as an institutional priority rather than student attainment. While such ends need not be mutually exclusive, they represent very different values and institutional goals that can compete for resources.

We value what we measure, whether we are measuring the right things or not.  And what we talk about and spend time on often falls back on the measures we have. College rankings are based on nothing consequential in terms of student achievement.  Participating in a commercial enterprise thinly cloaked as a public service signals to an institution’s constituents, prospective students, and others that what matters is looking good, not necessarily providing a good education.  At a point in history when more people than ever must acquire the knowledge, dispositions, skills, and competencies promised by a postsecondary education, we need our institutional leaders, governing boards, and policy makers squarely focused on making sure precious resources are directed to effective educational practices that will help students survive and thrive in college— not playing the rankings game.

George Kuh is Chancellor’s Professor of Higher Education and director of the Center for Postsecondary Research at Indiana University Bloomington.

Tags: college rankings, student attainment