Author Archive
“When you come to a fork in the road, take it” (Yogi Berra)
A few years ago, the National Survey of Student Engagement (NSSE) and USA Today arranged for schools to post their NSSE benchmark scores on the USA Today college website. While the Community College Survey of Student Engagement had championed public reporting since its inception, this NSSE-USA Today relationship marked the first time that several hundred four-year schools took the leap of faith and made their student engagement results available to a national audience. More recently, national associations developed templates for their member schools to use to report cost, NSSE or other student experience measures, and—in some instances—selected student learning outcomes. To my knowledge, no institution has closed or been otherwise adversely affected by making public these kinds of student or institutional performance measures.
It’s almost certain that in the future, colleges and universities will be expected to provide much more information about what and how much students learn during college. Institutions are not of one mind, of course, about whether and how to do this. There are justifiable worries about people drawing erroneous conclusions from data. Another risk is that making visible all our laundry—some clean and some not so clean—will have the adverse effect of stifling candid internal discussions about where improvements need to be made and will discourage efforts to address such shortcomings. These concerns are real and not trivial.
U.S. News & World Report (USN&WR) magazine did not invent college rankings, but its annual fall issue has dominated the market for the past two decades. They continue to capture the public fancy despite evidence revealing their vapid content, serious methodological flaws, and limited influence on where the vast majority of undergraduates go to college. The major criticism is that the rankings are based primarily on what an institution has – its reputation and resources – and say nothing about what students do during college or how they benefit after they matriculate. Indeed, we developed the National Survey of Student Engagement a dozen years ago in part to emphasize that how students use an institution’s resources to educational advantage is far more important than the resources themselves.
Efforts to game some of the variables in the ranking algorithm also are well-documented, such as omitting entrance exam scores of selected groups of students to inflate the institutional average. Some institutions recalculated the number of alumni eligible to contribute to the annual fund (e.g., graduates for whom no current mailing address was available were removed) in order to shrink the denominator, thereby artificially boosting the percentage who do (appear) to support their alma mater. Not only do some institutional leaders rate their school well above peer institutions on the reputation measure, thereby increasing the odds that their own school will fare better, Christopher Morphew and Barrett Taylor found that many colleges and universities modify their mission statement for posting on the USN&WR website. Of one hundred colleges they checked, only six submitted their official mission statement, contrasted with more than half which submitted prose “entirely dissimilar to the official mission statement.” This may be much ado about nothing in terms of influencing college choice decisions, inasmuch as the written espoused mission may or may not comport with the enacted mission, which my studies of high- performing institutions over the past twenty years show is far more important to student success. The disturbing implication of the Morphew and Taylor discovery is that when schools tamper with their primary public declaration of what they stand for, how can we have confidence in anything such institutions say? Read the rest of this entry »
