Friday morning of AAC&U’s Annual Meeting, a panel of experts in STEM (science, technology, engineering and mathematics) learning and assessment was convened to present different tools and resources for assessing learning in the STEM disciplines. The focus was on interdisciplinary learning, and the room was filled with an enthusiastic crowd of 150 people.

Charlie Blaich (Wabash Center of Inquiry in the Liberal Arts, Wabash College) emphasized that good assessment starts with knowing who your students are from the very start. In other words, what experiences, knowledge, and expectations do they bring to your institution as first-year students? Charlie presented results from a STEM-focused analysis of data from the Wabash National Study, which utilizes multiple metrics to analyze the critical factors influencing liberal arts education. Here are a few highlights:

• Of 17,750 freshmen from a variety of institutions in the study, 23 percent indicated they were interested in contributing to science; however only 30 percent of these indicated that they plan to choose a major in science. This proportion does not change much by the end of the freshman year, suggesting that students’ decisions to enter a STEM major are made by the time they enter college. First-year experiences do not appear to increase students’ interest in majoring in science.

• Three important college experiences that contribute to improving students’ interest in science are: active and collaborative learning settings, faculty interactions, and cooperative learning experiences. These experiences also contribute to student improvement in other areas outside of science.

• The frequency of student-reported faculty interaction at both large universities and small colleges shows overlapping means and the same degree of wide variation, although the mean is slightly higher at smaller institutions. These data might be counterintuitive and suggest we might all benefit from such an analysis at our own institutions.

Jillian Kinzie (Indiana University Center for Postsecondary Research) presented information on how NSSE, the National Survey of Student Engagement, can be used to assess STEM courses and programs. Most campuses participate in NSSE, so the data are available. For example, a comparison of first-year (FY) STEM majors vs. non-STEM majors revealed that:

• STEM majors studied a little more than non-STEM majors – 20 percent STEM majors studied more than 20 hours per week, compared to 18 percent of non-STEM majors.

• Forty-nine percent worked outside class with peers on projects, compared to 40 percent of non-STEM

• 65 percent frequently worked on papers or projects that integrated ideas, compared to 75 percent of non-STEM.

NSSE also beta-tested some STEM-focused questions in 2009, which are available for use by other campuses (contact jikinzie@indiana.edu). A white paper (pdf) of a more detailed analysis of NSSE data regarding high impact practices reported by STEM and non-STEM majors was referenced.

Ashley Finley (AAC&U) presented the VALUE rubric project, which is a national project that has developed fifteen rubrics for measuring high level student learning outcomes. Rubrics that might be specifically relevant to the STEM disciplines are:

• Inquiry and Analysis

• Critical Thinking

• Quantitative Literacy

• Problem Solving

• Integrative Learning

STEM programs are encouraged to review and adapt these or other AAC&U rubrics to determine their value in assessing STEM-related learning outcomes.

Jim Swartz (Grinnell College) presented a multicampus collaborative effort to develop a counterpart to the CURE (Classroom Undergraduate Research Experiences), called RISC (Research on the Integrated Science Curriculum), which is designed to link faculty course goals and activities with student experiences relative to science interdisciplinary learning. Swartz reported that the development process has highlighted the importance of using language that both faculty and students understand. For example, students may not report interdisciplinary experiences if they are in an “interdisciplinary” major such as environmental studies (they may see it as disciplinary because that is their major). He also reported that developing and using the survey has helped him and other faculty members in the project to focus on and fine-tune student learning outcomes. The instrument has just been pilot- tested and the results are being analyzed.

A lively question-and-answer period was followed by a concise summary from Mike Kerchner (Washington College). Here are some highlights: faculty doing undergraduate STEM education work were encouraged to publish their results in the science education literature; challenges of IRB approval for educational research by STEM faculty were raised; a call for more resources and workshops on assessment was made. PKAL will be working to help faculty with these issues and others from this session. Stay tuned!

Materials from this session will be posted on the AAC&U Annual Meeting Web site, including a podcast of this session, in the next few weeks.

Susan Elrod is the director of Project Kaleidoscope at the Association of American Colleges and Universities in Washington, DC.

Here’s an e-mail I got from a colleague after this session:  “Brings tears to my eyes.  Truly amazing.”   I wasn’t quite crying, but this was an amazing set of stories.  Professor Barbara Clinton of Highline Community College has developed an Honors Program that, since its 2003 inception, has transformed hundreds of lives.  Her three copanelists were all alumni, with spectacular stories to tell.

Clinton described Highline as a college in a “poverty pocket” of King County, near Seattle, Washington.  It’s the most diverse community college in Washington state, and most of its students come in not knowing a lot about higher education — where it can take them, and how they can get it.  She was blunt about the raw material of the student body, and I was surprised her three panelists were, too.

Highline students volunteer for the Honors Program, usually their third quarter in; the absence of a teacher recommendation process is intended to make it inclusive for all kinds of learners, not just those who stand out in traditional classrooms.  A 3.5 GPA in any one course is enough to qualify.  The program’s student demographics match those of the college.  AAC&U has recognized it as a site of Inclusive Excellence.

In the first term of the Honors Program, students enroll in a two-unit “boot camp” to learn about writing personal statements, resumes, and college-level essays.  Grading is with rubrics, and by the end of the term, students use those to grade their own work — accurately.  Clinton pointed out this leverages the independent streak of the community college learner, often older than the traditionally aged, and used to taking responsibility in other spheres (home, work).  All these materials are available online.

From the boot camp, students move on to regular coursework.  Many Highline faculty have signed on to the program by offering their Honors students supplemental assignments on a pass/no-pass basis, usually an additional research project.  Completion doesn’t change the course grade, but adds an Honors designation to the transcript.  The course number stays the same, to preserve articulation.  A typical research project:  in a philosophy survey course, teach the class about a philosopher not on the syllabus.

Clinton’s three students told amazing life stories:  a woman from Ethiopia who learned writing and networking skills to harness her natural charisma.  A nightshift package handler at UPS whose resume exercise got him promoted to manager so he’d be less exhausted in class, and who’s now earning a law degree at George Washington University.  A former Marine security guard and computer programmer now studying international relations at Tufts, who learned to “translate” the strengths of his military background into civilian lingo (e.g., a security clearance is a kind of character reference).

In each case, Clinton used a practical knowledge of student affairs to support her classroom claims about what’s possible — for example, by meeting students’ concerns about money with specific advice about financial aid.

It was well after the session that I realized what the stories had in common:  Clinton had helped all three of these students find a way to see themselves candidly but differently, as university students and professionals. In the words of student Joseph Burnett, she “helped me restructure my self-image.”

Ken O’Donnell is associate dean of Academic Programs and Policy at the Office of the Chancellor, California State University.

As we enter a new decade, education is still plagued by one of the more misdirected assessment initiatives of the past decade:  No Child Left Behind. NCLB, unfortunately, arose from the faulty notion that simply by administering standardized tests, educational practices magically would be improved. Indeed, the most underperforming schools as measured by standardized test scores are punished and the children who need the resources the most are deprived of them.

Along the same mistaken line of accountability thinking, institutions of higher education were coerced into engaging in the misnamed Voluntary System of Accountability (VSA). The misguided idea behind VSA is that the primary focus of assessment in colleges and universities should be garnering some single standardized score by which an institution can compared to its peers. Although we know from the LEAP initiative that all institutions of higher education should share some common focus on liberal education as measured by the Essential Learning Outcomes, each institution must have its own unique mission and vision. No standardized test can come close to demonstrating the degree to which any institution is achieving its self-identified mission.

AAC&U’s VALUE rubric initiative stands as one of the few viable alternatives to VSA and NCLB. After a two-year FIPSE-funded national development process, AAC&U has published fifteen developmental rubrics that are paired with each of the Essential Learning Outcomes that arose from the LEAP (Liberal Education and America’s Promise) initiative begun in 2005 by AAC&U. After a full day pre-conference symposium on Wednesday, January 20, focused on VALUE and e-portfolios, Terry Rhodes and Wende Morgaine provided an overview of the VALUE rubric initiative on Friday, January 22, during AAC&U’s annual conference in Washington, DC.

I had some personal experience with the LEAP rubrics during their development process as a member of one of the VALUE leadership campuses, and I regularly use the rubrics for designing specific assignment rubrics. I can attest to their value as diagnostic and prescriptive assessment instruments whose purpose is not primarily oriented toward accountability, but more appropriately toward student learning and improvement. Because each rubric is constructed in the form of multiple developmental sequences, they enable both student and instructor to understand where their achievement falls along a developmental continuum, from whence they have improved and what they still need to do to continue to improve their learning. No standardized test has that power to guide learning for individuals or for institutions collectively.

I encourage readers to explore the usefulness of the VALUE rubrics for assessing individual learning artifacts, for program assessment, and for institutional evaluation. They are published in a new work, Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics, edited by Terrel (Terry) Rhodes, available from AAC&U. The rubrics also are available free of charge as .pdf files from the AAC&U Web site. Find the value of the VALUE rubrics.

Stephen Langendorfer is director of the BG Perspective (general education) program and professor of kinesiology at Bowling Green State University, Bowling Green, Ohio. As an associate member of AAC&U, he has participated in a Greater Expectations Institute and the VALUE rubric project.

Highline Community College’s Honors Scholar program demonstrates that combining wit and will can benefit students’ wallets.   Through the will of a dedicated faculty, the program was implemented in 2003 with thirteen students and three faculty and has grown to approximately 250 students who work with 100 faculty on research projects integrated in courses.  The students enroll in a two-credit “bootcamp” course to develop a personal statement, an academic resume, and explore transfer opportunities throughout the United States. A one-credit course serves as a capstone experience to prepare them to submit admission and scholarship applications to further their higher education.  Graduates of the program have transferred to public universities, elite private universities, and the military academies.

The three Highline honors graduate students at the session demonstrated the varied paths community college students take through higher education.  One student was a Marine who is now a student at Tufts; the second student, from Ethiopia, now is in a master’s program at the University of Washington; and the third student, a UPS employee, is now a George Washington University law student. Some five years later, each student recalled in detail his or her research topic and the results of their honors projects.  The three students represented very well the 85 percent of their fellow honors students who transferred to a university.

While the honors scholar program is operated on a small wallet, the financial benefits for students are substantial.  Faculty without financial compensation work with the students on their research projects, with these experiences helping students realize $3.2 million in financial aid from transfer institutions.

At the beginning of the AAC&U conference, attendees heard Under Secretary for the U.S. Department of Education Martha Kanter call upon the higher education community to educate the top 100 percent in order for the United States to reclaim its status as the leader in the world for proportion of citizens with a college degree.  She emphasized that to reach this goal, the higher education community must focus on student success.  Honors programs at community colleges provide additional pathways to students who are moving through the college pipeline at a pace that is perhaps slower than others, but moving nonetheless.  The opportunities and the expectations for success of the three Highline Community College graduates provides evidence that excellence can be inclusive.

Dwight Smith is the vice president of academic affairs at the County College of Morris.

For AAC&U members to succeed in advancing the values and issues important to us, it is crucial to understand not only our institutional constituents but also the public view of higher education. Accounting for the predominant attitudes toward higher education in society, we are better equipped to adapt our goals of liberal education for broader, public audiences. One strategy for analyzing our public audience is to take stock of the ways media institutions discuss higher education.

On Comedy Central’s The Colbert Report, Comedian Stephen Colbert bemoaned contemporary higher education after he met a college intern taking “whatever courses look interesting.” Colbert joked, “It turns out these days they let college kids do anything they want. They live in co-ed dorms, make friends with people from different backgrounds both in the real world and on ‘The MyFace.’ And they can even eat cereal for dinner. It is chaos and we need to address this crisis.” Colbert’s satire successfully reflects for his audience common public attitudes and discussions about higher education.

But we need not turn to comedy for our snapshots of cultural trends. News institutions offer strong indicators of public orientations toward higher education. David Glenn from the Chronicle of Higher Education, Doug Lederman from Inside Higher Ed, and Mary Beth Marklein from USA Today discussed some trends emerging from the media.

Given the economic downturn and the financial constraints at many institutions, concerns percolate more frequently over the continued quality of higher education. With the real threat of program cuts, the quality of general education may decline. And with increasing numbers of underprepared students entering postsecondary education, particularly due to the economy, many worry about the consequences of cutting support services necessary for high student success rates.

Race and class in higher education also remain much-discussed, hot-button issues that illicit strong reactions and emotions. In particular, Marklein noted, stories about immigration and undocumented students garner significant attention and response.

In addition to recognizing hot topics, we must listen carefully to the silences in the media. Glenn and Lederman explained that the major publications and news outlets report principally on elite institutions such as Ivy League schools and well-known research institutions. Smaller schools and community colleges receive little air time, their innovations and concerns ignored. Consequently, national discussions and attitudes toward higher education are often framed according to a narrow, incomplete portrait of higher education institutions.

These hot topics and silences represent only a sample of the lessons to be gleaned from the media. In order to encourage the public to embrace the liberal education that members of AAC&U champion, we must take cues from media voices. These cues may help us better adapt our goals, messages, and education to our contemporary cultural climate.

Jonathan Rossing is a Rhetoric and Public Culture Graduate Student at Indiana University. Rossing received the 2010 K. Patricia Cross Future Leaders Award during AAC&U’s 2010 Annual Meeting.

President Obama’s call for America to reclaim its place as having the highest proportion of college graduates in the world by 2020 presents several challenges and requires wit, will, and wallet by community colleges to meet this goal of increasing the number of students receiving a degree by approximately 150,000 annually.  The wit will require community colleges to embrace a “culture of completion” for our students and believe that students have the “right to succeed.”  Wit will be revealed in knowing our students, their hopes and aspirations, and engaging faculty in the use of high-impact practices throughout the college.

Will is determined by community colleges’ success in the political arena to advance the American Graduation Initiative.  With approximately six million students enrolled in community colleges, this sector of higher education provides the largest source of potential graduates to propel the United States to reclaim its position as world leader in educated citizens.  The response to the American Graduation Initiative has not been embraced enthusiastically by all sectors of higher education for a variety of reasons.  Community colleges will need to exert their political will with the help of their students, faculty, administrators, and the communities they serve if they are to realize the role that is called for in the American Graduation Initiative.

The wallet is a challenge to both the wit and the will.  Funding for each full-time student at a community college is the lowest of any sector of higher education, and with structural budget deficits projected in all states, faculty, staff, and administrators will be tested to meet the demands of students and communities.  This funding disparity and the financial aid rules that drain available funding for students with developmental education needs serve as rallying points for community colleges to use their wit and will to increase the size of their wallet.

The first LEAP Principle of Excellence calls for higher education to “Aim High and Make Excellence Inclusive.” Community colleges—through their wit, will, and wallet—will help higher education live this principle.

Dwight Smith is the vice president of academic affairs at the County College of Morris.

On January 21^st, a panel of leading experts in STEM (science, technology, engineering and mathematics) education set the stage for a conversation with academic leaders regarding the key leverage points and critical questions Project Kaleidoscope (PKAL) should be focusing on as we formulate an action agenda for the next five years. Dan Sullivan, president emeritus at St. Lawrence University and a major player in creating the PKAL-AAC&U alliance, introduced Peter Bruns (Howard Hughes Medical Institute), Jim Gentile (Research Corporation), Shirley Malcom (The American Association for the Advancement of Science), and Cora Marrett (National Science Foundation), who provided their perspectives on the current state of undergraduate STEM education reform.

Inspired by the panelists’ identification of issues ranging from providing authentic science learning experiences for all students at all types of institutions to rethinking faculty tenure and promotion systems, over 100 participants in small-group discussions identified three key areas for improving STEM education: 1) making professional development for K-12 teachers, graduate students and faculty more pervasive; 2) continuing to engage external organizations, such as scientific disciplinary societies, corporations, and foundations, to support and give prestige to reform efforts; and 3) tackling difficult institutional issues, such as changing the rigid policies and structures that impede interdisciplinary teaching and learning, and restructuring the first year science curriculum to be more integrative and problem-based.  A need for more widespread use of evidence-based strategies was also identified. Groups were then asked to generate the next generation of questions we should ask  to frame the future work of PKAL. These questions were collected and will inform the continuing planning process (more to come on the PKAL Web site and in subsequent blog postings).

Jay Labov (National Academies of Science) provided a brief synthesis of the discussion, highlighting the 2009 National Research Council report on the New Biology for the 21^st Century. This report calls for a New Biology to that will focus on solving major societal and global problems, such as health, environment, energy, and food. These problems require integrative thinking that involves not only the sciences but also the humanities. Which undergraduate classes prepare students, and not just majors, to address these challenges? How can we move more institutions to offer the kinds of authentic educational experiences that will prepare every student for the world we live in? Jay argued that we are not yet at the tipping point in our reform efforts, but at a place where a rising tide can lift all boats.

We all have a role to play in this work, and PKAL with AAC&U is poised to play its role in raising the level of dialogue and networking to accelerate the pace and reach of STEM reform. This was the first of many conversations we will have with key stakeholder groups around the country. As the new Project Kaleidoscope director, I am looking forward to hearing from these groups, as well as from you. Comments?

Susan Elrod is the director of Project Kaleidoscope at the Association of American Colleges and Universities in Washington, DC.

“I don’t know if you’ve heard this before,” John Pryor, director of the Cooperative Institutional Research Program at UCLA, wondered, “but finances are on the minds of a lot of people, not just presidents and boards, but also students who are coming to college.”

Thursday’s release of the 2009 CIRP data by the Higher Education Research Institute noted some intriguing principles and practices of today’s first-year college and university students. Fifty-five percent have some concerns about financing college, the highest percentage since 1971; more students are turning to loans to finance college; their fathers are unemployed at the highest percentage in the history of the survey (4.4 percent); and even though academic reputation is still the top reason students choose a particular school, other concerns, such as affordability or offers of financial assistance, are increasingly having an impact on a student’s choice of school.  Perhaps needless to say, colleges and universities will need to be able to deal with students who are increasingly anxious about financing their education.

And this is clearly the storyline the major news outlets picked up, as their headlines show: “Annual Poll of Freshmen Shows a Effect of Recession” (New York Times),  “UCLA Survey Finds More Freshmen Worried About Finances” (Los Angeles Times), and “Cost of College is a Big Worry of Freshmen in National Survey” (Chronicle of Higher Education).

Our students’ financial concerns certainly do matter, and within the context of endowment losses, discount rates, state funding cuts, and increasing costs, there may be no other story on higher education today more worth talking about.

But to only tell this story—the story of the wallet—can obscure the great possibilities of the wit and the will of both institutions and the students they serve.  Those institutions with missions that encompass values of personal and social responsibility, the CIRP results also suggest, should be able to take advantage of the existing high desirability among students for volunteering and should work to facilitate such opportunities.  This call was echoed in a late afternoon session on AAC&U’s initiative, Core Commitments: Educating Students for Personal and Social Responsibility.

We may not be able to do much to boost the nation out of the Great Recession, or boost our students’ pessimism about paying for college.  But we may be able to carve out a place within the economic anxiety to exercise our own institutional and community values, and in so doing, encourage our students to do the same.

Laura Behling is the associate provost of faculty affairs and interdisciplinary programs at Butler University.

Would it be an exaggeration to suggest that in its first five years the Liberal Education and America’s Promise (LEAP) initiative has become the single most influential program ever created by AAC&U? This claim may be debatable, but the accomplishments of LEAP at its mid-point mark are truly remarkable. LEAP has promoted multiple programs in campus action, public advocacy, and authentic evidence. The Campus Action Network and Partner States initiatives are bringing the existence and adoption of the Essential Learning Outcomes to the forefront on college campuses across the country. The periodic Hart Research Associates surveys conducted for AAC&U are documenting that employers indeed value the achievement of the Essential Learning Outcomes in those they hire. The newly published Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics are providing,  for the first time, a national basis for assessing the Essential Learning Outcomes and offer a realistic alternative to standardized testing.

In 2005, LEAP’s lofty goals—sparking public debate about essential learning outcomes for all students, promoting liberal education and its broad benefits; and documenting the degree to which students were achieving liberal education outcomes—must have seemed daunting at the very least to the AAC&U leadership and membership. Looking back over LEAP’s brief history, these goals, like the Essential Learning Outcomes they spawned, are well on the way toward achievement. As Carol Geary Schneider proudly pointed out in the opening plenary session of AAC&U’s 2010 annual conference, the chief academic officers at AAC&U member institutions report that 63 percent of their campuses have learning goals that address the essential learning outcome of integrative learning, while 89 percent of campuses address the essential learning outcome of writing skills.

But while LEAP has made remarkable advances, a number of AAC&U members gathered at the membership meeting  discussed ways to address the fact that an estimated 3 out of 5 undergraduate students (and perhaps faculty as well) do not know most of their institution’s learning outcomes. Members suggested a number of plausible ways that their institutions could raise awareness of their learning outcomes to students, faculty, and stakeholders. A few ideas included imbedding the learning outcomes intentionally within the first year and general education curricula, expanding the publicity of learning outcomes through brochures or on campus homepages, adding them to the use of assessment of the learning outcomes through e-portfolios, and including them within professional development efforts for faculty and student affairs staff. 

Will LEAP have as much impact during its second five years as it has had during its first five? The mark of a truly transformative initiative will come from its continued positive trajectory and the energy that is sustained across its full decade. If the focus on liberal education, student-centered learning, and authentic assessment of student learning outcomes has changed at least as much from 2010 to 2015 as it has since 2005, the answer will certainly be affirmative. 

Stephen Langendorfer is director of the BG Perspective (general education) program and professor of kinesiology at Bowling Green State University, Bowling Green, Ohio. As an Associate member of AAC&U, he has participated in a Greater Expectations Institute and the VALUE rubric project.

I recently took on the role of director of assessment at a proprietary business school, bringing my background as an aging white guy educated as a musician, experienced in both public school and college teaching, and employed for nine years by AAC&U.  The session on Liberal Learning and Business Education (with William Sullivan and Anne Colby of the Carnegie Foundation for the Advancement of Teaching) was of interest to me as I ponder  how to make general education and elective liberal arts study engaging, useful, and even life-changing for the students at my college.

With both associate’s and bachelor’s degree programs in business, my college is very successful at enrolling students attracted by our promise of small classes, friendly and attentive faculty, and an excellent job placement rate.

But far too many of our students leave before they finish even a year of school. It seems like there is some Maslovian need that interrupts students’ plans to complete a degree and venture forth to careers and lives as engaged citizens.

While I love brainstorming about creative integrative teaching and learning strategies, foremost on my agenda these days is finding ways to keep our students in college and engage them instantly in their learning.  How can my students get a really quick, nearly addictive learning experience their first day on campus to keep them coming back for all of the great projects and experiences the faculty have planned for them?

I know our students are not fragile, but in terms of college persistence, they seem to be very weakly attached to the idea of going to college.  I have speculated with colleagues that they need to be taught how to go to college. (Although that notion seems somehow presumptuous, I know that there are things college students should know and be willing to do to facilitate their success in learning).

So, I am left with questions.  How do you give students a really great start to learning in college – one that will hook them on the experience of learning and help them to believe that they are indeed capable of college-level work?  Can the same kind of integrative learning that will prepare them well for graduation also jump-start their initial attachment to college? Can this be done without requiring an expensive Outward Bound-style experience?

What do you do?

Ross Miller is the senior director of assessment at Berkeley College in New York and New Jersey.