Parallel Session 3: Assessment of Intercultural Development in Mobility Programs  

Matthias Barth, Co-director of the Institute for Sustainable development and learning (ISDL) at Leuphana University, Germany, set the objectives for the discussions. The development of intercultural competencies has been widely held as a key value in global learning programs. At the same time, Barth noted that the development of assessment tools is not always regarded as an important element in the design of mobility programs. Understanding this context, Barth invited panelists to share their expertise in designing learning outcomes, developing tools to assess both the short and long-term impact on participants, and ultimately bringing more awareness to the value and need for assessment as part of the learning journey within international mobility programs.

Peter Wells, Chief of the Higher Education Section at UNESCO Headquarters in France, opened the discussions by interrogating the assumptions of international education. He questioned the possibilities of studying abroad and suggests that the value of studying abroad is a difficult question to answer. He thinks that international educational experiences and collaboration are important for students, faculty and researchers but international studies are not for everyone, and there should not be much pressure on some individuals who may not want to experience these abroad opportunities. Drawing from his experience, Wells questioned whether intercultural learning can be quantified or qualified and whether there is a real value to intercultural learning. In particular, he warned that physical mobility programs alone have a limited potential fostering intercultural development and awareness of new cultures. While the connection to intercultural learning experiences to the field of International Affairs is clear, Wells underlined the challenge in defending the value of the intercultural learning experiences and mobility programs to fields of study that are not necessarily international in nature. Finally, Wells acknowledged that physical mobility programs may not be suitable for all learning paths. As a result, avenues for intercultural development should be available at a wider institutional level.

Darla Deardorff, Executive Director of the Association of International Education Administrators, United States, expanded on the topic of developing models for intercultural competencies. For many years, the field of intercultural learning has been holding onto a myth that assumes that simply sending students abroad is sufficient to reach new quantitative and qualitative heights in internationalization. Instead, Deardorff recommends incorporating intercultural competencies within the design of their mobility programs. To this end, Deardorff addresses three key misconceptions in assessing intercultural development:

• Pre and Post Assessment Tools are sufficient to encourage intercultural development;
• Assessment of the mobility programs can be equated to assessment of intercultural learning;
• There exists one-single tool or practice that will address intercultural learning.

Deardorff explained the focus should not be on the assessment tool, as it should on the reasons and needs to assess intercultural development. In addition, it is important to look at the evidence in the literature as well as other perspectives in intercultural development allowing to change the paradigm in assessment. This Paradigmatic Change focuses on intercultural development as a key learning outcome for mobility programs moves beyond immediate results, towards a holistic lifelong journey. The foundation of this approach requires learners to be capable of articulating the meaning and value of intercultural learning. Through meaningful outcome assessments, the learner can become an agent in this learning process. To be effective, these assessments must be tailored to individual learners and must be informed by a holistic account of the personal development of students, emerging from a multiplicity of voices (including self, host families, instructors, peers, and even future employers). As argued in one of the latest UNESCO Reports “Global Citizenship Education: taking it local” (2018), perspectives from the Global South, such as the South African notion of Ubuntu, challenge and displace the dominance of those voices from the North. Thus, the question of assessment of intercultural competencies cannot merely be resolved by the adoption of one best assessment tool. Instead, Deardorff emphasizes that focus should be on qualifying the journey, rather than quantifying the destination.

Heather Williams, Work Integrated Learning: Equity, Diversity and Inclusion Project Manager, Simon Fraser University (SFU), Canada, offered similar questions and considerations to those expressed by Deardorff on the value of assessment and lifelong intercultural learning. Her work in designing and assessing intercultural learning in the English Language Learning curriculum for international students seeking employment in Canada is unique in its ability to intentionally weave Canada’s Truth and Reconciliation Committee’s Calls to Action into the curriculum and learning outcomes. Williams mentioned the use of Deardorff’s model of intercultural competence to help guide SFU’s learners throughout their ongoing development, as well as aligning the learning outcomes with the Association of American Colleges and Universities (AAC&U) rubric. Since the development of the curriculum, it has been shared with 14 institutions across Canada and has been adopted for professional development for the Canadian Bureau for International Education (CBIE) and most recently Colleges and Institute Canada (CICan).

Williams explained how the program has used assessments before and after the completion of the program to qualify the growth in learning and awareness of the key themes amongst the students. With the use of impact assessments, the program administrators were able to identify the top themes that people learned about including reconciliation and allyship. These assessments have served as self-reflexive exercises for students because they can reflect on their own learning. Results have also demonstrated a commitment to lifelong learning, reconciliation, mindfulness, and attention to the emotional aspects of intercultural interactions while building relationships across difference. Williams concluded with a reminder of the importance of constant reflection on the benefits and risks of the assessment practices, to examine potential areas of marginalization, exclusion, and gaps in current global learning programs.

Alex Rendon, Universidad San Francisco de Quito (USFQ), Ecuador, Dominique Scheffel-Dunand, York University, and David Huerta Harris, Tecnológico de Monterrey, Mexico reported on their experiences in Virtually Augmented Mobility (also known as Globally Network Learning or Collaborative Online International Learning) programs. These programs (in collaboration with peers across geographical and cultural borders) offer students an opportunity to build 21st Century skills by exploring challenging content such as human rights, diversity, and global forced migration. The presenters framed and situated their experiences within the discussion as a case study of the challenges and successes when operationalizing some of these emergent points and principles. The Virtually Augmented Mobility programs successfully brought together staff, faculty, and students at different levels of their educational trajectories, from three different post-secondary institutions. In forming their Virtually Augmented Mobility program, Rendon and Harris emphasized the experiences of staff and faculty’s preparation alongside those of students. There was a shared determination to understand the impact of participating in collaborative and intercultural networks, in reference to current practices of knowledge production and sharing as well as the emergent propensity for future cross-institutional and intercultural faculty collaborations. Faced with the need to invent new and shared learning outcomes and assessment tools, difficult questions such as “What is meant by intercultural competencies?”, “Who measures them?”, “According to which standards?” suddenly required immediate and tangible responses. Scheffel-Dunand explained how with more conventional methods of data collection such as surveys, focus groups, and sprints, assessment of the student, faculty and staff experience included sentiment analysis. The sentiments and vocabulary used by staff, faculty and students on social media to describe their virtual interactions or even comment on the tools used, were collected to be analyzed in the interest of future scalability.

Reflections from the Chair and Way Forward

Contributions in the panel clearly pointed out the complexity when it comes to assessing intercultural development in mobility programs. There is no easy answer how to design learning outcomes and how to assess impacts on the learner in a meaningful way and there are many pitfalls to be avoided. The panelists added nuances and insights to the discussion of the value and need for assessment within international mobility programs.

Three underlying themes became obvious in the session’s discussion and conclusion. First, there is a need to acknowledge individual learning experiences in which international and intercultural exposure may happen in many different ways and to different extents. Diversity in such pathways should be considered both in mobility programs and in ways to assess the impact interventions have on the learner. Second, this also holds true for assessment procedures. To acknowledge the diversity on the learners side means to consider different tools and assessment procedures. There is no one fits all solution. Third and finally, a strong impetus was given on the need for lifelong learning. While mobility programs can be strong triggers for intercultural development, they are just one building block out of multiple in the learning journey. Accordingly, support for intercultural development need to take the bigger picture into consideration and to avoid planning in silos.

Based on the discussion of these underlying themes, requirements for ways forward became obvious. To empower students to develop intercultural competencies and being able to positively contribute to the SDGs means to not only assess but to foster learning in a meaningful way. Multiple learning opportunities in mobility programs – but also complemented in regular studies – will increasingly support individual lifelong learning pathways. Additionally, such learning pathways do not only ask for assessment and thus feedback on efficacy and efficiency but much stronger for an evaluation of the learning process. Such an evaluation needs to open up spaces for self-reflection for the learner which in turns will support the learning process. Digitalization will play an important role in all these developments as it offers new ways to experience internationalization and by opening up new ways of evaluation and assessment.