A Kaleidoscope of Perspectives on Student Learning
Goals and Assessment in STEM
By: Susan Elrod
Friday morning of AAC&U’s Annual Meeting, a panel of experts in STEM (science, technology, engineering and mathematics) learning and assessment was convened to present different tools and resources for assessing learning in the STEM disciplines. The focus was on interdisciplinary learning, and the room was filled with an enthusiastic crowd of 150 people.
Charlie Blaich (Wabash Center of Inquiry in the Liberal Arts, Wabash College) emphasized that good assessment starts with knowing who your students are from the very start. In other words, what experiences, knowledge, and expectations do they bring to your institution as first-year students? Charlie presented results from a STEM-focused analysis of data from the Wabash National Study, which utilizes multiple metrics to analyze the critical factors influencing liberal arts education. Here are a few highlights:
• Of 17,750 freshmen from a variety of institutions in the study, 23 percent indicated they were interested in contributing to science; however only 30 percent of these indicated that they plan to choose a major in science. This proportion does not change much by the end of the freshman year, suggesting that students’ decisions to enter a STEM major are made by the time they enter college. First-year experiences do not appear to increase students’ interest in majoring in science.
• Three important college experiences that contribute to improving students’ interest in science are: active and collaborative learning settings, faculty interactions, and cooperative learning experiences. These experiences also contribute to student improvement in other areas outside of science.
• The frequency of student-reported faculty interaction at both large universities and small colleges shows overlapping means and the same degree of wide variation, although the mean is slightly higher at smaller institutions. These data might be counterintuitive and suggest we might all benefit from such an analysis at our own institutions.
Jillian Kinzie (Indiana University Center for Postsecondary Research) presented information on how NSSE, the National Survey of Student Engagement, can be used to assess STEM courses and programs. Most campuses participate in NSSE, so the data are available. For example, a comparison of first-year (FY) STEM majors vs. non-STEM majors revealed that:
• STEM majors studied a little more than non-STEM majors – 20 percent STEM majors studied more than 20 hours per week, compared to 18 percent of non-STEM majors.
• Forty-nine percent worked outside class with peers on projects, compared to 40 percent of non-STEM
• 65 percent frequently worked on papers or projects that integrated ideas, compared to 75 percent of non-STEM.
NSSE also beta-tested some STEM-focused questions in 2009, which are available for use by other campuses (contact firstname.lastname@example.org). A white paper (pdf) of a more detailed analysis of NSSE data regarding high impact practices reported by STEM and non-STEM majors was referenced.
Ashley Finley (AAC&U) presented the VALUE rubric project, which is a national project that has developed fifteen rubrics for measuring high level student learning outcomes. Rubrics that might be specifically relevant to the STEM disciplines are:
STEM programs are encouraged to review and adapt these or other AAC&U rubrics to determine their value in assessing STEM-related learning outcomes.
Jim Swartz (Grinnell College) presented a multicampus collaborative effort to develop a counterpart to the CURE (Classroom Undergraduate Research Experiences), called RISC (Research on the Integrated Science Curriculum), which is designed to link faculty course goals and activities with student experiences relative to science interdisciplinary learning. Swartz reported that the development process has highlighted the importance of using language that both faculty and students understand. For example, students may not report interdisciplinary experiences if they are in an “interdisciplinary” major such as environmental studies (they may see it as disciplinary because that is their major). He also reported that developing and using the survey has helped him and other faculty members in the project to focus on and fine-tune student learning outcomes. The instrument has just been pilot- tested and the results are being analyzed.
A lively question-and-answer period was followed by a concise summary from Mike Kerchner (Washington College). Here are some highlights: faculty doing undergraduate STEM education work were encouraged to publish their results in the science education literature; challenges of IRB approval for educational research by STEM faculty were raised; a call for more resources and workshops on assessment was made. PKAL will be working to help faculty with these issues and others from this session. Stay tuned!
Materials from this session will be posted on the AAC&U Annual Meeting Web site, including a podcast of this session, in the next few weeks.
Susan Elrod is the director of Project Kaleidoscope at the Association of American Colleges and Universities in Washington, DC.