Program Design-Assessment Integration
From 2006-2013, Michele V. Price, now retired Director of Study Abroad and International Exchanges and Dr. Victor Savicki, Professor of Psychology, Emeritus, engaged in an ongoing assessment and research effort aimed at increasing its effectiveness in enhancing the study abroad experience for its students. As a result of repeated assessments and more focused research endeavors, the required study abroad capstone project, student orientation, and re-entry interventions were revised and strengthened based on systematic data collection, analysis, and interpretation. This integration of assessment/research with program components led to a richer, more effective program for students that was evidence based. Each link for the years 2006-2013 illustrates the project direction and results for that year. The project was approved by WOU's Institutional Review Board.
The following timeline highlights specific assessment/research projects and their subsequent impact on study abroad program improvements.
2006 Selection of assessment instruments and integration of assessment and evaluation into the application and selection process for study abroad students. Pre and Post survey questions were incorporated into an online format available to students via web browsers at a time and place of their choosing. The pre- and post survey became a required component of the study abroad capstone project. A journal assignment, completed while abroad, also was added. (Evolution of a Study Abroad Capstone Project)
2007 First data analysis findings indicate marginal changes in hypothesized directions on a majority of measures, yet a need to focus on deeper reflection in on-site and re-entry requirements. Therefore, the structure for the final reflection paper or capstone assignment was revised, and a new grading rubric developed to guide students in a more focused way to successfully complete the assignment. (Re-entry Postdiction of Pre-Departure Readiness)
2008 Analysis of capstone papers using content analysis software indicated high degrees of insight, and inclusiveness, focus on positive emotions, and a linkage between self-reported study abroad goal attainment and critical thinking and systemic thinking. To further the reflective process, changes were made in orientation to help focus student attention on issues that may be topics for culturally based reflection. (Re-entry Integration, Pre-departure Readiness: Development and Research on Student Writing)
2009 Content analysis based on the Developmental Model of Intercultural Sensitivity indicated 57% of code-able units as ethnocentric and 43% ethno-relative. An unexpected result was discovery of patterns of how students avoid effective reflection by complete externalization or internalization rather than integration of the two. To encourage students to overcome these avoidance strategies, a mandatory re-entry session was added to the capstone project. (Student Readiness and Intercultural Sensitivity)
2010 Orientation, on-site, and re-entry activities continued to be revised, including the elimination of the reflective paper and the addition of a re-entry resume development assignment, focused on transferable skills based on study abroad experiences as a mechanism to enhance reflection The journal assignment, previously modeled on the “Strategies for Keeping a Journal” chapter of the University of Minnesota’s text, Maximizing Study Abroad, also was revised. The new assignment, focused on asking the right questions pre, while abroad, and post, was designed to encourage students to view their study abroad experience as an integrated whole and to be able to articulate why the experience was important. Chisolm’s text, Charting a Hero’s Journey, was used in developing the questions.
2011 Over the five years of this project, it has become more and more evident that students need guidance to unpack their experience abroad. The effectiveness of the redesigned capstone project in helping students to cultivate awareness, hone observational skills, and articulate expression will be assessed in the 2011/12 academic year. (Asking the Right Questions)
2011/12 Results of research project to date
Students who had completed both pre and post, online study abroad student assessment questionnaires (N= 136) were divided into two groups (median split) based on their report of how well they attained their personal goals during their study abroad sojourn. Overall goal attainment was based on the sum of the 13 item Study Abroad Goals Scale (Kitsantas, 2004) which included sub-scales on Cross-cultural competence, Subject interest and competence, and Social gathering goals.
Linkages between program design and assessment, evaluation, and research projects have enhanced, and will continue to enhance our students’ study abroad experience. Changes in program design have been evidence-based.
The effectiveness of the redesigned capstone project in helping students to cultivate awareness, hone observational skills, and articulate expression will be assessed in the coming year with particular attention:
1) to whether students actually take the time to reflect back in their responses;
2) to what degree they succeed or miss the mark in making connections about their experience as a whole. The process continues.
2012 These graphs indicate changes over time on several key variables that have been tracked by the WOU Study Abroad Office since 2006. Only changes that were statistically significant are reported. Graphs accompany each of the reported assessment results for ease of visualizing the outcomes.
2013 Reflection can be seen as a fundamental expression of humankind's "will to meaning:" the desire to make sense of life's events. Reflective thinking has been proposed as an important feature of study abroad learning; the challenge to students to construct a meaningful understanding of events during their encounter with a different culture. The current study examines how study abroad students use reflection and other cognitive processes through their writings, and how the process of reflection may be affected by different experiences that are reflected upon. Affective responses are examined along with cognitive processes.
return to top
The Office of Study Abroad phone: 503-838-8905 fax: 503-838-8089 | or e-mail: firstname.lastname@example.org