U.S. News & World Report (USN&WR) magazine did not invent college rankings, but its annual fall issue has dominated the market for the past two decades.  They continue to capture the public fancy despite evidence revealing their vapid content, serious methodological flaws, and limited influence on where the vast majority of undergraduates go to college. The major criticism is that the rankings are based primarily on what an institution has – its reputation and resources – and say nothing about what students do during college or how they benefit after they matriculate. Indeed, we developed the National Survey of Student Engagement a dozen years ago in part to emphasize that how students use an institution’s resources to educational advantage is far more important than the resources themselves.

Efforts to game some of the variables in the ranking algorithm also are well-documented, such as omitting entrance exam scores of selected groups of students to inflate the institutional average. Some institutions recalculated the number of alumni eligible to contribute to the annual fund (e.g., graduates for whom no current mailing address was available were removed) in order to shrink the denominator, thereby artificially boosting the percentage who do (appear) to support their alma mater.  Not only do some institutional leaders rate their school well above peer institutions on the reputation measure, thereby increasing the odds that their own school will fare better, Christopher Morphew and Barrett Taylor found that many colleges and universities modify their mission statement for posting on the USN&WR website.  Of one hundred colleges they checked, only six submitted their official mission statement, contrasted with more than half which submitted prose “entirely dissimilar to the official mission statement.”  This may be much ado about nothing in terms of influencing college choice decisions, inasmuch as the written espoused mission may or may not comport with the enacted mission, which my studies of high- performing institutions over the past twenty years show is far more important to student success.  The disturbing implication of the Morphew and Taylor discovery is that when schools tamper with their primary public declaration of what they stand for, how can we have confidence in anything such institutions say? Read the rest of this entry »