Posts Tagged ‘assessment’
By Susan Albertine, with Terry Rhodes, AAC&U vice president for Curriculum, Quality, and Assessment, and Nevin C. Brown, senior fellow, Siena Italian Studies and former senior fellow for post-secondary initiatives, Achieve Inc.
Spring is nearly here—and as we near this transition from one season to the next, it seems like a good time to address another key transition point in many people’s lives along the educational pathway from school to college—the transition between grades 12 and 13.
The current development of a set of Common Core Standards for K-12 education (adopted by 43 states and territories) presents those of us in higher education with an opportunity not to be missed. The Common Core is a step forward. It is aspirational, detailed, complex, and explicit. Its expectations for student performance extend beyond one-dimensional approaches to skills or content. It may not do all that many of us in post-secondary education would like to see (e.g. covering a fuller scope of outcomes in the sciences, humanities, or the arts). Yet it reaches for higher levels of proficiency than before for all students in English, language arts, communication, and quantitative reasoning. Unfortunately, the Common Core has yet to prompt widespread post-secondary minding and engagement. Can we change that pattern and bring about more connected work with schools on the ground?
“When you come to a fork in the road, take it” (Yogi Berra)
A few years ago, the National Survey of Student Engagement (NSSE) and USA Today arranged for schools to post their NSSE benchmark scores on the USA Today college website. While the Community College Survey of Student Engagement had championed public reporting since its inception, this NSSE-USA Today relationship marked the first time that several hundred four-year schools took the leap of faith and made their student engagement results available to a national audience. More recently, national associations developed templates for their member schools to use to report cost, NSSE or other student experience measures, and—in some instances—selected student learning outcomes. To my knowledge, no institution has closed or been otherwise adversely affected by making public these kinds of student or institutional performance measures.
It’s almost certain that in the future, colleges and universities will be expected to provide much more information about what and how much students learn during college. Institutions are not of one mind, of course, about whether and how to do this. There are justifiable worries about people drawing erroneous conclusions from data. Another risk is that making visible all our laundry—some clean and some not so clean—will have the adverse effect of stifling candid internal discussions about where improvements need to be made and will discourage efforts to address such shortcomings. These concerns are real and not trivial.
I know that many of you have probably been reading the copious news coverage of the new book Academically Adrift. The articles in the higher education press have already elicited hundreds of comments—some useful, some defensive, some derisive. All this attention, of course, is because the book presents some very compelling and disturbing data about learning in college. I know there are limitations to the data presented in this book, but I don’t think any reasonable person can simply dismiss the facts its authors present: that too many students are, indeed, not developing their analytic thinking and writing skills, especially in the first two years of college. These two aspects of college learning are incredibly important for success in today’s world. There are, however, other capabilities that we also need to develop in college students if we are to prepare them well for work, life, and citizenship in the twenty-first century.
For those of us who care about increasing students’ achievement of key liberal education outcomes—including students’ integrative and applied skills—there are also two other articles published in the past few days that shouldn’t be missed. Heather Wilson, a former member of the US House of Representatives, graduate of the US Air Force Academy, and former Rhodes Scholar, writes in the Washington Post about her experiences evaluating candidates for Rhodes scholarships. She has served for twenty years on the selection committees and has “become increasingly concerned in recent years—not about the talent of the applicants but about the education American universities are providing.” Rather than bemoan these top students’ lack of writing skills, however, she worries about another important capacity we would hope a quality liberal education would develop in students. In her experience, even “high-achieving students seem less able to grapple with issues that require them to think across disciplines or reflect on difficult questions about what matters and why.”
Readers of this blog may not have noticed the significant announcement last week by Secretary of Education Arne Duncan of the recipients of Race to the Top grants to support the development of a new generation of K-12 assessments that are more performance-based, technologically sophisticated, and inclusive of both formative and summative evaluations of students’ math and language arts skills. This development—if successfully implemented in even a fraction of the states involved—will, indeed, be a game-changer in K-12 education and have profound implications for higher education as well.
Whether these assessments really will measure students’ “college readiness” or not remains to be seen, however—at least in my judgment. Much will depend on how involved college and university faculty and assessment experts are in both the development of the tests and the implementation of the new common core standards they are designed to assess. To date, there has not been nearly enough serious involvement by college faculty either in the assessment consortia or in the development of new standards. Time will tell if that can be remedied going forward. The leaders of these consortia seem to be seriously committed to involving higher education faculty and leaders— a reason for optimism.
By: Susan Elrod
Friday morning of AAC&U’s Annual Meeting, a panel of experts in STEM (science, technology, engineering and mathematics) learning and assessment was convened to present different tools and resources for assessing learning in the STEM disciplines. The focus was on interdisciplinary learning, and the room was filled with an enthusiastic crowd of 150 people.
Charlie Blaich (Wabash Center of Inquiry in the Liberal Arts, Wabash College) emphasized that good assessment starts with knowing who your students are from the very start. In other words, what experiences, knowledge, and expectations do they bring to your institution as first-year students? Charlie presented results from a STEM-focused analysis of data from the Wabash National Study, which utilizes multiple metrics to analyze the critical factors influencing liberal arts education. Here are a few highlights:
By: Ross Miller
I recently took on the role of director of assessment at a proprietary business school, bringing my background as an aging white guy educated as a musician, experienced in both public school and college teaching, and employed for nine years by AAC&U. The session on Liberal Learning and Business Education (with William Sullivan and Anne Colby of the Carnegie Foundation for the Advancement of Teaching) was of interest to me as I ponder how to make general education and elective liberal arts study engaging, useful, and even life-changing for the students at my college.
With both associate’s and bachelor’s degree programs in business, my college is very successful at enrolling students attracted by our promise of small classes, friendly and attentive faculty, and an excellent job placement rate.
By: Katrina Carter-Tellison, Ph.D.
The saying goes, “you have to bet big to win big” and there’s no doubt about it –e-portfolios are a “big bet.” Whether it is the effort needed to engage students, the time required by faculty, or the commitment of institutional resources, e-portfolios are an enormous endeavor. However, what has emerged from The Search for VALUE Symposium, is that there can be no more effective way to assess student learning and no greater tool to make changes to that learning process than e-portfolios. The workshops focused on three key areas: the philosophical argument of “why” use e-portfolios; the nuts and bolts of using e-portfolios for assessment; and how to accurately evaluate institutional technology needs.
As educators, we find ourselves facing a difficult challenge. As Darren Cambridge stated in his session, “the outcomes we most value are often the most difficult to measure.” How do we measure empathy and personal and social responsibillity? Can there be a standardized test for such outcomes? E-portfolio methods allow us to examine these principles and concepts in a contextual way across time. E-portfolios allow students to engage in the all-important principles of Integrative learning. Students can make links and synthesize learning across courses, semesters, and disciplines over time. In fact, that time does not have to be restrained to the period during which the student is enrolled, but depending on the technology employed, the e-portfolio can be utilized indefintely throughout students’ lives.
By: Ross Miller
For me, e-portfolio use is all about expectations and Greater Expectations. Teaching all of our students, and teaching all students to higher levels was a dual challenge that (as Randy Bass mentioned) basically established a logical (and one hopes inevitable) drive to spread the practice of using portfolios for learning. (If you are thinking that I neglected to include assessment, then you may still be thinking of assessment as separate from the process of learning.)
The Greater Expectations project advocated for all of us to find intentional approaches to liberal education – working to develop intentional learners, teachers, and institutions. If we really intend to be intentional while also working within the limits of time, then we must focus on those practices that produce the greatest effects. Portfolios and their electronic progeny have long pointed toward a handful of powerful practices of which we should be more conscious and use much more often. Read the rest of this entry »
In an Inside Higher Ed essay, colleagues who developed the Collegiate Learning Assessment (CLA) attacked both the concept of e-portfolios as a strategy for assessing students’ learning gains and my own argument that it is high time to break the habit of treating standardized tests as a source of special and privileged insight into an individual’s potential and/or achievement across a course of study.
For the convenience of readers who are just learning about AAC&U’s approach to assessment, we’re providing links to three resources available to download.
1) The first is Our Students’ Best Work: A Framework for Accountability Worthy of Our Mission. Revised and reissued last year, this is an official Board of Directors statement. It describes ways of focusing assessments on students’ actual work, completed across the curriculum. The core idea is captured in the title. Assessments ought to motivate students to do their very best work, and higher education ought to make the production of such “best work” a focal point for the college curriculum. When students are producing “authentic work,” that work can be assessed using validated rubrics by faculty who have been trained to apply rubrics to samples of student work.
Recognizing the scalability challenge this approach to assessment presents, Our Students’ Best Work recommends that each academic program build into the regular curriculum abundant opportunities for students to practice and produce work that deploys important college outcomes, such as analysis, communication, problem solving, engagement with difference, and integrative learning. For purposes of institutional assessment and external reporting, a random sample of portfolios can be scored and reported using rubrics and multiple blind raters.
2) The second link takes you to the VALUE rubrics that have just been released through AAC&U’s federally funded national project, Rising to the Challenge. These rubrics are keyed to the essential learning outcomes that AAC&U has developed—in concert with the higher education community—through its ongoing initiative, Liberal Education and America’s Promise (LEAP). The LEAP VALUE rubrics feature “dimensions” of specific learning outcomes that faculty should take into account in determining a student’s growth in competence through his or her studies. The VALUE project studied hundreds of existing campus rubrics for specific learning outcomes that faculty had already developed to assess student work and progress. The rubrics were developed by faculty-led expert teams and have been tested multiple times against actual student work at many different institutions.
3) The third link takes you to my own essay, “The Proof is in the Portfolio,” which I published last year to express my dismay that higher education, in the wake of the Spellings furor, was now piloting the use of a single test to be taken by student volunteers that would supposedly provide external evidence about what students have learned over time. While I respect my CLA colleagues for their psychometric fervor, I stand firmly by my view that no institution should use a single test, taken by a set of student volunteers, to form or report judgments about the quality of student achievement across the entire family of programs and majors.
As I said in my essay, we are educators. As educators, we have a responsibility to help our constituents distinguish between good practice and bad practice. Using a single measure to capture the academic achievement of an entire college or university curriculum is bad practice.
Taken out of context—that is, considered within a contemporary American context—Alan Bennett’s play (and, later, movie) The History Boys can be read as satirizing the values of the standardized testing movement. Set in early-1980s England, the plot centers on a group of pupils who are preparing (or, perhaps more accurately, are being prepared) for the Oxbridge entrance exams. Their ambitious headmaster recruits a cynical and, as it turns out, fraudulently credentialed history teacher who dedicates himself to what we’d call “teaching to the test,” with all that that phrase implies. Meanwhile, Hector, an erudite and unorthodox English teacher, refuses the mandate to prepare his students for the all-important test, hoping instead to prepare them for life by exposing them to wisdom—and his own highly singular influence. “I count examinations, even for Oxford and Cambridge, as the enemy of education,” Hector explains.
As you’d expect, the test-happy headmaster is quick to recognize the threat Hector poses to his ambition. (Oxbridge scholarships would translate directly into increased prestige for the school.) “Shall I tell you what is wrong with Hector as a teacher?” the headmaster asks. “It isn’t that he doesn’t produce results. He does. But they are unpredictable and unquantifiable and in the current educational climate that is no use. He may well be doing his job, but there is no method that I know of that enables me to assess the job that he is doing. There is inspiration, certainly, but how do I quantify that?” Read the rest of this entry »
