Integrating community service learning into a master’s program

Purpose – In order to address the complex problems of society, the innovation research process should incorporate technical, social, economic and ethical factors, but also actively involve a diverse group of non-scientific actors. One way to prepare students for this type of research is to create “Citizen Scholars,” students who want to work for the betterment of society. Arvanitakis and Hornsby (2016) argue that we need to change how we teach and train students in specific proficiencies. The purpose of this paper is to assess how the pedagogical approach applied within the program contributes to building the proficiencies and attributes as described by Arvanitakis and Hornsby (2016). Design/methodology/approach – The authors conducted a total of 12 interviews with alumni who started their program in 2014, one focus group discussion with lecturers and 132 questionnaires with alumni to discuss to what extend the proficiencies are trained in the program and whether these are used in their current jobs. The authors also included data of an earlier study conducted in 2014. These data contain 26 interviews with students during the first year of the program. These students graduated in 2017 and are thus from the same cohort as the alumni. Findings – The results show that the pedagogical approach in the management policy analysis program trains all the attributes. Important elements in the program are: the inquiry-based approach intertwined with community service learning (CSL) throughout the program; gradually increased complexity of the real-world problems addressed; students working in teams; and gradually reducing support of the lecturer. Research limitations/implications – The authors conclude that our pedagogical approach applied in the program contributes to learning the proficiencies. The authors argue that for the training of interand transdisciplinary, the proficiency knowledge integration should be added. Practical implications – The result show that more inquiry-based approaches and CSL programs can stimulate the four clusters of proficiencies, which should hold a central place in universities if we want to create citizen scholars. Social implications – With the approach, students contribute to research issues of local communities. Originality/value – Despite the increasing interest of higher education to involve civic activities in the curricula, few pedagogical approaches are described. The research shows that theoretical insights in the adaption of a model to realize a citizen scholar.


Introduction
Society confronts a growing range of complex problems that have impact on health and well-being, such as the vicious circle of poverty and health problems, the refugee crisis, an aging population, insufficient access to healthy food and climate change.
In the past, we have seen how science and technology contributed to substantial economic progress in many countries and improved welfare in terms of health and living standards of millions of people.Addressing complex problems is more difficult, however, since these do not fit well into mono-disciplinary science, which still dominates the way in which universities are organized (Arvanitakis and Hornsby, 2016).Furthermore, the focus on technical, often mono-disciplinary, knowledge has resulted in solutions that have negative and unforeseen consequences and raise new ethical dilemmas when applied in practice (e.g.Rip and Schot, 1995).The challenge for science is to find a better link between addressing complex problems, research, and social, economic and ethical factors.
Various scholars have argued that the research and innovation process needs to be organized differently ( Jasanoff, 2003;Mitcham, 1999;Nowotny et al., 2001).It should incorporate not only technical, but also social, economic and ethical factors, and also actively involve a diverse group of non-scientific actors (e.g.Klein et al., 2001;Bunders et al., 2010).Thus, the research and innovation process has to become interdisciplinary, integrating multiple scientific domains and transdisciplinary, integrating non-scientific stakeholders.The rationale is that all these actors have relevant knowledge and experiences to provide different perspectives on and interest in the research and innovation process.Integration therefore leads to better informed and contextualized scientific knowledge and technologies (Bunders et al., 2010).But how can we prepare future scientists for this type of research, and train university students to involve actors, and especially non-scientific actors, in research?Arvanitakis and Hornsby (2016) argue that we need to change how we teach if we want students to become "citizen scholars," who work to improve society.They identify four clusters of proficiencies, which should hold a central place in universities in order to create citizen scholars and argue for innovative pedagogical approaches that promote them.We hypothesize that the attributes of citizen scholars, and their training, are also relevant in the education of inter-and transdisciplinary researchers.
This paper describes the lessons learnt from the two years' inter-and transdisciplinary Master's program management policy analysis (MPA) and entrepreneurship in health and life sciences, offered by the Athena institute of the VU University Amsterdam, the Netherlands.The program aims to prepare researchers who can contribute to solving complex social issues related to health and life sciences in collaboration with social actors.We assess how far this program contributes to building the proficiencies and attributes described by Arvanitakis and Hornsby (2016) and whether the proficiencies also contribute to the training of inter-and transdisciplinary researchers or that adaptions are required.We start with a description of the proficiencies and attributes defined by Arvanitakis and Hornsby (2016).We then present the development of pedagogical approaches that aim to train students to work together to change the world around them, followed by a description of the MPA, the methodology and the results.Finally, we discuss our findings in relation to the wider literature.
Proficiencies and attributes needed to create citizen scholars Arvanitakis and Hornsby (2016) identify four proficiencies and attributes to prepare students to become citizen scholars: creativity and innovation; resilience; working across teams and across experiences; and design thinking.These clusters are overlapping.
The first proficiency cluster -"creativity and innovation"consists of six attributes: "critical thinking," "problem solving," "reflexivity," "entrepreneurship," "being process driven" and "systems thinking."Arvanitakis and Hornsby (2016) argue that "creative and innovative" attributes and a "focus on process instead of just content" are of great importance in creating citizen scholars.The second proficiency cluster -"resilience"consists of two attributes, namely, "adaptability" and "mistakability."It means that students can learn from mistakes, which results in people who are flexible in the face of change and can anticipate innovations.
The third proficiency cluster, "working across teams and across experiences," comprises five attributes: "interdisciplinarity," "cross cultural understanding or cultural humility," "developing new literacies," "internationalization" and "inclusivity."This is critical as most employees now work in teams.University programs should emphasize "a culture where individual and collective success are synonymous" (p.17).The last cluster is "design thinking," which comprises the attributes of "people-centered thinking," "aesthetics" and "ethical leadership."Arvanitakis and Hornsby argue that we need to place people at the center of our decision making.In addressing problems, it is essential to take into account people's needs and aspirations as well as the relationships around them (p.18).They argue that is important to replace the hierarchical one-way relationship between lecturer and students with a dynamic exchange between the two.

Transformative education
The call for transformative education that trains students to work together to change the world around them, rather than to follow the logic of existing society, is not new.Long ago, Dewey (1916) and Freire (1970) argued that in order to address complex social problems, education needs to focus on critically reflecting on the underlying rules and assumptions in society and the involvement of those who are affected by problems.In response, various approaches have been developed that engage students in a process of "learning by doing."These inquiry-driven approaches to learning (e.g.Bruner, 1961;Dewey, 1933;Piaget, 1964;Vygotsky, 1978) have given rise to various approaches, such as problem-based learning (Barrows and Tamblyn, 1980;Savery, 2006), experiential learning (Kolb and Fry, 1975) and project-based learning (Barron and Darling-Hammond, 2008).In most cases, students have to discover and construct knowledge through their own activities (Prince and Felder, 2006).In universities, however, where mono-disciplinary knowledge production and expert knowledge dominate, transformative research and transformative education are not yet well established.
In the last decade, various schools and universities have involved civic activities in their curricula (Lake et al., 2015;Maloyed, 2016).Community service learning (CSL) characterizes a form of experiential learning and engagement in higher education, which combines academic learning and practical work experience (Deeley, 2010).From its inception (community) service learning focused mainly on nurturing students' sense of civic responsibility in response to their observed social disengagement (Billig and Welch, 2004).Currently, CSL is aimed at student learning and making a difference in the lives of recipients (the community) at the same time (Furco, 1996;Wang, 2013;Butin, 2015).In conducting CSL activities, students need to collaborate with the wider society and practice these skills during their studies.In CSL efforts, members of educational institutions and community organizations "work together toward outcomes that are mutually beneficial" (Hayes and King, 2006, p. 2).CSL can be an ideal way to focus on relevant knowledge, skills, work habits and character traits of students (i.e.twenty-first century skills) (Sabat et al., 2015).
In a review of the literature on CSL, Stewart and Wubbena (2015) found that its reported outcomes include intra-/interpersonal skills, academic knowledge and professional skills, and civic engagement and social responsibility.According to Hatcher et al. (2016, p. 11), service learning enables "broadening and deepening specific and general learning outcomes in higher education."Building on various authors, they suggest that there is accumulating evidence that service learning can contribute to the academic, civic and personal learning domains (see Bringle et al., 2016).Nevertheless, there is a need for more research on the contributions CSL makes at the level of institutions, faculties, students and communities (e.g.Hatcher et al., 2016), especially as CSL is generally just bolted on to existing curricula (Sheffield, 2015) and does not result in a transformation of the person or the communities (Sheffield, 2015).

The pedagogical approach in the MPA program
The inter-and transdisciplinary MPA program specifically focuses on conducting research at the interface of science and society, to address complex social problems in health and life sciences.There are four specializations: international public health, management and entrepreneurship, policy and science, and communication.Students with a BSc degree in various science disciplines are accepted on the program, which is based on an inquiry-based approach and CSL, and on "problem driven education," where students address actual problems in simulation and reality.For the CSL components, the program is linked to the "Athena Science Shop," which supports non-profit organizations by undertaking research projects with a potential social impact.MPA students conduct research in CSL components during the courses and CSL-based internships.In most cases, the research includes active participation with those who commissioned the research and various stakeholders, including communities.
The program provides a link between the knowledge and skills from students with a BSc in various disciplines, most notably science, technology and social studies, political sciences and management studies.In the MPA program, the following core competencies are developed: • analysis of complex social issues related to health and life sciences; • formulation and implementation of strategies to deal with complex social problems through inter-and/or transdisciplinary research; and • effective cooperation and communication with researchers from scientific disciplines other than health and life sciences and with social actors.

Year 1
The aim of the first year is to obtain the core competencies, comprising three compulsory courses, two elective courses and a five-month research internship.In all program elements, complex social problems play a central role.
The program starts with "Research methods for analysis of complex social problems," which introduces students to a transdisciplinary methodology in order to address complex social problems.The lecturers present various research paradigms and introduce the students to theories and quantitative and qualitative methods from several disciplines.In groups, students design research to address a complex social problem.
The second course, "Analysis of Governmental Policy" (AGP), contains a CSL component.Students conduct research that addresses a complex social problem.The research questions come from a diverse group of commissioning parties, e.g.non-government organizations (NGOs), patients' organizations, the National Institute for Health and municipalities.In teams of ten, students write a policy advisory report for the commissioning organization, based on data collected from stakeholder interviews and the literature.The research is supervised by a coach and a lecturer, who trains them in policy theories and knowledge translation.Reflection on the knowledge obtained is central, and on completing the course, the students present the research to the commissioning parties.
The third compulsory course is "Communication Organization and Management," which aims to provide students with deeper understanding of the management process of using scientific knowledge to address complex social problems.The lecturers provide facts and theories on organizational behavior.In groups, students select a trend in the health and life sciences and determine what actions a hospital management team needs to take in order to address its positive and negative consequences.
After the three compulsory courses students choose two elective courses from their respective specialization (international public health, management and entrepreneurship, policy and science communication).Most of these elective courses contain a CSL component.Next, students conduct a five-month individual internship.This is not an apprenticeship, but an interdisciplinary research project, usually based on research questions from NGOs or other (social) organizations.Most of the research conducted during the internships involves a diverse group of stakeholders.

JARHE 11,2
Year 2 The aim of the second year is to deepen the research skills in one of the four specializations identified above, and to prepare students to conduct independent research.Students start with the compulsory course "Managing Science and Technology in Society," which aims to understand the linkages between science, technology and society.In an individual assignment, students need to design an innovation to address a complex social question.
The other compulsory course is "Ethics in the Health and Life Sciences," in which the principles are illustrated with cases of technology ethics and medical ethics.A role-play exercise on an actual research protocol simulates how an ethical review committee on animal research works.Finally, as a group-training project, an ethical dialogue is prepared and discussed with another team.
Other components of the second year are another optional course (similar to year one) and a literature-based thesis.Again, most of the optional courses contain a CSL component, and the literature theses are often written in response to a research question posed by a social organization.Students end the program with a second internship in which they independently address a complex social problem presented by a commissioning party.In the second year, some students conduct their internship abroad.(For more information on the program see, http://masters.vu.nl/en/programmes/management-policy-analysisentrepreneurship-health-life-sciences/Index.aspx).

Methodology
This is a mixed-methods study consisting of both quantitative and qualitative data: interviews with 12 alumni who started their program in 2014 (2017); a questionnaire to alumni with 132 respondents (2017); focus group discussion (FGD) with four lecturers in the MPA program (2017).We also included data of an earlier study executed in 2014.These students graduated in 2017 and are thus from the same cohort of the alumni interviewed in 2017.These data consist of 26 interviews with first-year students (2014), and an interview with the course lecturer (2014).Below we discuss each method in more detail.
In 2017, we conducted in-depth, semi-structured interviews with 12 alumni (2014-2015 cohorts) a few months after they graduated.Six of these interviews were face to face, and the other six were by telephone.The alumni were selected on the basis of convenience sampling.The interview had a semi-structured design and consisted of two parts.First the alumni were asked to reflect on their masterprogramma.We asked which (optional) courses they had conducted.These were in case of the face-to-face interviews written down on cards.Next, we asked the student from which course they learnt most and why.The alumni could discuss the course (with the help of the cards) in the order they preferred.We also asked them about the internships, the location and the difference between the internship conducted in the first year of the program and the internship conducted in the second year of the program.In the second part of the interview, alumni were asked to reflect on the proficiency clusters of Arvanitakis and Hornsby (2016).All 16 specific attributes of the four proficiency clusters were printed on cards.The proficiencies were explained to the respondents.Next, the respondents were asked whether they understood the attributes and if they had obtained these attributes.Next, they reflected which aspects of the master contributed to the building these proficiencies.The alumni could discuss the attributes in the order they preferred.Additionally, they were asked whether there were other relevant attributes that were taught in the program and contributed to becoming a "citizen scholar." We compared and enriched these findings with a questionnaire that was administered to alumni to gain information about their career and perspectives on the relation between their current job and the MPA.In this questionnaire, six questions were related to the extent alumni apply the specific attributes of proficiency cluster 1 on a scale from 1 to 5 (see Box 1).This part of the questionnaire was used in this research and was completed by 132 respondents (74 percent women, 26 percent men).

Community service learning
Box 1. Six questions on the specific attributes of proficiency cluster 1 Addressing complex problems: to what extent is the formulation and implementation of strategies that deal with complex social problems part of your current job?
Interdisciplinary and social cooperation: do you cooperate and communicate with researchers from several scientific disciplines and social actors in your current job?
Critical thinking: to what extent do you need critical thinking, i.e. making clear and reasoned judgments by taking into account challenging perceptions and conceptions, in your current job?
Systems thinking: to what extent do you need systems thinking, i.e. the understanding of a system by examining the linkages and interactions between different elements and the whole system, in your current job?
Problem solving: to which extent is problem solving, i.e. finding solution to problems through innovative thinking, part of your current job?
Entrepreneurship: to which extent is entrepreneurship part of your current job?
In 2017, we conducted one FGD with four lecturers involved in a variety of course components, both compulsory and elective, workgroups and internships.Of the selected lecturers, two are involved in compulsory courses, one in an optional course and one in the supervision of workgroups.All lecturers are involved with supervising students during the internships.In the FGD, we reflected on the MPA program in relation to the four proficiency clusters with the specific attributes and the findings of the other interviews.
In the earlier study, conducted in 2014, we conducted in-depth, semi-structured, face-to-face semi-structured interviews with 26 students in groups of two at the end of the second compulsory course.As mentioned earlier, this cohort graduated in 2017 and therefore made up the group of alumni interviewed in 2017.Interviews focused on various aspectsaims of the course, new insights acquired, benefits from the course, difficulties encountered and differences with previous teaching experiences.Specific questions which are interesting with respect to the study of 2017 are: What parts of the course did you experience as challenging?What is different from the way of teaching if you compare it to your bachelor?These interviews of 2014 provide some insights in how the alumni experienced the learning during the program when they were students.In 2014, we also conducted an interview with the course lecturer, focusing on the lecturer's perspectives on the highlights of the course, the difficulties the students experienced and the challenges of open-inquiry teaching.
All interviews were recorded and transcribed verbatim and analyzed for this purpose by the first and second author.By comparing these results, we gained more insight into which proficiencies and attributes most of the learning takes place.Analysis of all qualitative data was done deductively, based on the proficiencies and attributes framework.Inductive, thematic coding was performed to understand to what extent the pedagogical approach used in the program contributes to building these proficiencies in relation to educating inter-and/or transdisciplinary research.Quantitative data were analyzed using descriptive statistics.To ensure validity, we conducted data triangulation and checked for data saturation.

Results
In the following paragraphs, we reflect upon the extent to which the MPA achieves learning in the four proficiency clusters.
Cluster 1: creativity and innovation Alumni and lecturers in the FGDs indicate that the development of all attributes in this cluster is embedded throughout the program.Critical thinkingespecially "the ability to challenge 240 JARHE 11,2 perceptions and conceptions"is "a typical MPA skill," said one alumnus during the interview [Alumni 9].For example, alumni state that applying and integrating conceptual frameworks from different disciplines, and incorporating perspectives of different social actors made "critical thinking" a much-needed skill, as can be seen in the following quote: Critical thinking, making linkages between concepts, operationalization of the selected conceptual models, are recurring themes in the program.(Alumni 8) Proficiency cluster 1 skills were introduced during the first (RMCP) and second course (AGP), and alumni interviewed especially linked "problem solving skills" to these courses.They appreciated these courses, since they are very good preparation for the internships.In interviews conducted with students in 2014, students were (already) very enthusiastic about it.The assignment of a real commissioning party improved their ability to understand and address a complex problem, although they also faced several challenges.
Teachers, alumni and students (2014) indicated that one of the challenges is getting used to working within various research paradigmsa prerequisite for inter-and transdisciplinary researchas most of them have a mono-disciplinary bachelor background.Alumni indicated that this results in difficulties in analyzing complex problems, developing conceptual frameworks and selecting methodologies.This was confirmed by the interviews conducted with students in 2014.As can be seen from this quote: I have a background in a bachelor program in which we did not assess social aspects, and thatin this programit is obvious that in addition to the numbers and lab results, it is important to add qualitative data, even though for me this was difficult to comprehend.(Student 11.1)In addition, it is difficult to deal with different sources of information (e.g.scientific literature and stakeholder perspectives).To integrate knowledge, reconcile conflicting opinions and perspectives and translate these into policy advice or action was the most challenging.During the interviews, alumni indicated that the emphasis on iterative processes of problem solving, based on data collection, inclusivity and reflection, improved their capacity to integrate and translate perspectives.
In the FGDs, lecturers mentioned that changing their roles during the learning process contributes to proficiencies in all clusters.Whereas the lecturers provide much structure during the first course, they gradually reduce this in subsequent courses.Lecturers indicated that this is difficult both for teaching staff as well as for students.In the interviews conducted in 2014, students said they found the lack of guidelines and affirmation very stressful.Being trained in a mono-disciplinary program, students are used to addressing assignments in one particular way.This is seen in the following quote of a student in 2014: […] we got feedback, which was great, but she never said that it was right or wrong.Instead she said you could do it like this, that way you have no support.(Student 12.1) As a result, students will ask the lecturers to provide guidance, which the lecturers may be inclined to do.Since the assignments are from real commissioning parties, lecturers tend to focus more on the report than on the process.It is interesting to see how these perspectives change during the program.In hindsight during the interviews, alumni stated that it was very positive that lecturers refrained from giving too many guidelines.This stimulated a more creative, entrepreneurial process in which students took ownership of the process of problem solving, as can be seen in the following quote: It is important to swim […] courses of which I think, yes, I learnt most, these were the moments, in which you have to swim and in which you have to make choices yourself […].(Alumni 9) The online survey provided more insights into whether the alumni applied the attributes of cluster 1 in their current job.The items were scored on a scale from 1 to 5, where 1 implied

Community service learning
"not at all" and 5 "to a large extent."Both critical thinking and systems thinking were generally part of their jobs, respectively, 81 and 71 percent scored either 4 or 5 on these items.Similarly, 81 percent of alumni indicated that problem solving was part of their job (a total of scale 4 and 5), 51 percent that they addressed complex societal problems (32 percent scored 4 and 18 percent scored 5) and 53 percent (29 percent scored 4 and 27 percent scored 5) that they worked in cooperation and communication with researchers from several scientific disciplines and social actors (see Figure 1).However, only 27 percent said that entrepreneurship was part of their job.In the interviews with alumni, it became apparent that entrepreneurship was related to starting up and running a business, and not to the broader conceptualization used by Arvanitakis and Hornsby (2016).
Cluster 2: resilience This cluster consists of two attributes "adaptability" and "mistakability."As discussed in cluster 1 in relation to iterative processes of problem solving, alumni and lecturers agreed that adaptability is a much-needed and well-addressed attribute throughout the program.In course assignments and the internships, students are encouraged to work iteratively and adapt (research) activities to emerging insights.
A more remarkable point is "mistakability."During the FGD, the lecturers indicated that learning from mistakes is stimulated, for example, through regular feedback on the content of outputs and the working process and through simulations (e.g.role-plays, FGD simulations).Interviews with alumni, however, showed that they experience this differently and were hesitant to take risks.This was mainly in response to the assessment structure.Since the students have to pass assignments and examinations and are under pressure of time, there is limited room for making and learning from mistakes during the process.This is illustrated by the quote of an alumnus:  Cluster 3: working across teams and across experiences Both alumni in interviews and the lecturers in the FGD said that working across teams and across experiences is essential in the program.Students work in different teams of varying size and participants in most courses of the program.In interviews, alumni said they were comfortable with team work and that this was both a means and an end to facilitate learning throughout the program.Not all students appreciated team work initially.In interviews with students (in 2014), students suggested that at first they preferred to work individually, but that after the second course, they realized the value of working in a team.With respect to the specific attributes, in interviews, the alumni consistently commented that interdisciplinarity is at the heart of the program and that they felt comfortable with facilitating processes of inter-and transdisciplinary knowledge production.This is enabled by the emphasis throughout courses on the relevance of different scientific paradigms and by working in teams with students from various backgrounds: MPA stands out because of all the different disciplinary backgrounds of students, therefore you learn to work with different persons with different expertise.(Alumni.2) In interviews, the alumni linked interdisciplinarity to cross-cultural understanding and inclusivity.To them, understanding different frames of reference (e.g.different ethnic, epistemological and/or organizational cultures) and acceptance of these perspectives is a prerequisite for conducting inter-or transdisciplinary problem solving.Alumni indicate that the optional courses and compulsory courses later in the program are required to obtain expertise in these attributes.
In interviews, alumni, however, also indicated that although they are proficient in oral and written presentation of scientific data, the program could be more specific in developing new literacies to better articulate information, especially for specific audiences (e.g. policy makers, health professionals).The attribute of literacy in itself was not well understood by the alumni.
Cluster 4: design thinking The people-centered thinking of cluster 4 is at the core of the MPA and is understood in two ways: putting the interests of people who are affected first when addressing problems; and putting the perspectives of individuals at the center of methodologies.In interviews, alumni linked the first mainly to the problems addressed in the assignments and internships, which are based on real-world problems.The second is part of the training in participatory methods and interview methodology, where the young researchers learn how to genuinely listen to the experiences and perspectives of participants and let them be involved in decision-making processes, as is seen in this quote from an alumni: […] it was the first time, that we were going to interview people and would ask about their experiences, what is important to people, and in this way we started thinking from people's perspective.(Alumni.1)Aesthetics was, according to alumni in the interviews, not a prominent attribute in the course.They recalled that tutors emphasized aesthetics when presenting data, but always from the perspective of functionality.Ethical leadership is recognized as a concept being taught in a variety of courses (such as ethics) and in the behavior of teaching staff, who generally base decisions on processes of deliberation and acceptance of inputs, as can be seen from this quote: I have always had the experience that with all [teachers] I worked with I could have an input, everyone could have input, and it was always taken very seriously.(Alumni.10)However, alumni said that they were unsure whether they had sufficiently mastered the practice of ethical leadership during the program.

Reflections on the attributes and proficiencies
We identified three recurrent themes in relation to incorporating the attributes and proficiencies in the Master's program.First, lecturers underlined that critical to the pedagogical approach in the MPA program is the (real-world) problem-driven approach and loop-learning.Loop-learning referred to rounds of feedback on the attributes trained in and between courses and the loops between theory and practice of addressing real-world problems.According to the lecturers in the FGD, this culminates in the final internship: The real "real-life-problem" is in real life, the internship.(FGD) Second, alumni in the interviews and lecturers in the FGD stressed the importance of integrating knowledge to address social issues.This cluster was not mentioned in the original model of Arvanitakis and Hornsby (2016).Alumni and lecturers argued that in facilitating change processes, it is important to have a basic understanding of the relevant content, concepts, context and the methodologies in order to address the problem in a scientific manner.In the FGDs, lecturers reiterated the importance of knowledgerelated activities, and that MPA students must be able to integrate and co-create knowledge for the purpose of problem solving.Knowledge on related matters increases the ability to relate, reflect and advise on problems and solutions in relation to complex problems.This new cluster comprises the attributes of knowledge regarding "content," "context," "concepts and methodologies" (see Box 2 for a description of this proficiency cluster).
Second, alumni indicated in the interviews that, although nearly all attributes and proficiencies were part of the program, these were barely assessed in the examinations.Most assessments focus on the content of courses or the written assignments.Thus, the students are not directly rewarded for mastering the proficiencies and attributes.During the FGD, the lecturers indicated, that in response to the findings of the 2014 interviews, they introduced a reflection report as an assessment tool, in which they focus on the process rather than the final report.
Box 2. The knowledge integration proficiency cluster Knowledge integration cluster Attributes and proficiencies related to "Knowledge integration" focus on obtaining and integrating knowledge to facilitate social change.This requires having knowledge, collecting knowledge and integrating knowledge in collaboration and the attitude of life-long learning, to match he demands of society/social change.
Context knowledge is understood as sufficient knowledge about dominant cultures, structures and processes that determine actors' actions.
Content knowledge is understood as having sufficient knowledge related to the topic of the change process in order to have meaningful discussions with stakeholders involved, including specialists.
Concept-related knowledge/skills are understood as sufficient knowledge about relevant concepts and theories of various disciplines to be able to conceptualize problems and solutions deductively and inductively link findings to relevant concepts and theories.This is critical in order to understand mechanisms that also hold true in a variety of contexts.
Methodological-related knowledge/skills are understood as sufficient knowledge and skills to design and implement methodologies that contribute to addressing complex social issues.In addition to quantitative and qualitative methodologies this includes transdisciplinary methodologies to allow for true inclusivity and participation, in particular the ability to facilitate and analyze group sessions.

Conclusion/discussion
We recognize that the MPA program trains the proficiencies clusters of Arvanitakis and Hornsby (2016) and thus contributes to the training of citizen scholars.Most proficiencies and attributes were directly recognized by alumni as being part of the program and our quantitative analysis indicates that attributes of proficiency cluster 1 are also practiced in their current jobs.In this discussion, we highlight aspects of the program that contribute to developing these attributes and proficiencies.Next, we discuss how we believe the model can be adapted to fit the requirements of inter-and transdisciplinary education.
The pedagogical approach and the "Proficiencies and Attributes" The MPA program's pedagogical approach helps to develop the attributes in the proficiency clusters.We believe that certain elements particularly contribute to its success.First, the adoption of an inquiry-based approach combined with CSL throughout the program creates an ideal setting in which to acquire attributes from the various clusters.Similar to other inquiry-based approaches, complex real-world problems are central (see Woods, 2014 for an overview of learning approaches that start with a problem).Also, in more than half of the program the students address "real" real-world problems based on questions from social organizations, meaning that they conduct research in and with society, in other words, CSL (e.g.Deeley, 2010).
Second, the complexity of social problems increases throughout the program, promoting attributes such as systems and critical thinking, inclusivity and interdisciplinarity.Increased complexity is not related to the content of the problem as such, but to the wider contextual aspects, scientific perspectives and methodologies that are taken into account in addressing it.
Third, support provided to the students is gradually reduced during the program and students become more independent.This is different from most inquiry-based approaches which have strong support at one end of the spectrum (e.g.Oriented Guided Inquiry Learning (POGIL) (Farrell et al., 1999) and no support at the other (e.g.discovery learning).Like Banchi and Bell (2008), we recognize that there are various levels of inquiry and that, by gradually reducing support, students can be prepared for independent scientific investigations.In our approach, every course has a different level of support: assignments have just enough guidelines to provide direction, but not so many as to impede independence, which allows the students to master the attributes (see also Zweekhorst et al., 2015).All of is supported through continuous loops of learning.
Fourth, working in teams of various sizes contributes to learning.Group work, similar to Deeley (2010), allows students to engage intellectually with the course work and give each other feedback.The variety of disciplinary backgrounds enhanced their capacity to work in an interdisciplinary manner.In addition, it had a social benefit and promoted inter-and intrapersonal skills (Zweekhorst et al., 2015).
On a more critical note, the way we assess students may inhibit their learning potential, in particular in the areas of learning from mistakes and entrepreneurial activity.Lecturers and former students agree that it is not the attributes defined by Arvanitakis and Hornsby (2016), but rather the final product and examinations are the main form of assessment.Over the years, various taxonomies have been developed, for example, Bloom (1956), Krathwohl (2002), Romiszowski (2013); the Solo taxonomy (Biggs and Collins, 1982); or Miller's pyramid (Miller, 1990), which are well suited to the assessment of understanding, applying and reflecting on knowledge of content, concepts and methods, but are less so for assessing attributes.Moreover, the assessment of graduation products and examinations has negative side-effects as students prefer to focus on passing the examination rather than on mastering the attributes.As elegantly described by Machemer and Crawford (2007), "any activity, be it active, cooperative or traditional, that directly relates to improving exam performance is valued by students."

Community service learning
The model adapted Knowledge production is at the heart of transdisciplinary research.Therefore, in order to make it more applicable to inter-and transdisciplinary education, we suggest adding a new cluster, "knowledge integration," to the model of Arvanitakis and Hornsby (2016) (see Figure 2).This cluster comprises the attributes knowledge regarding "content," "context," "concepts and methodologies."These involve skills and competences such as "how to operationalize concepts," "rapidly find and assess and apply information" and "select and implement methods."This cluster shares similarities with Bringle et al. (2016), the academic learning domain that stresses deeper knowledge of theory and concepts to contribute to community service and personal learning.The explanation of the attributes and proficiencies is given in Box 2. We propose the a wider application of the perspectives of the adapted model of Avanitakis and Hornsby in the evaluation and development of CSL programs or other educational programs aiming to train inter-or transdisciplinary researchers.

Strengths and limitations
Data triangulation between students, alumni, lecturers and data saturation were obtained in this study.Although we applied convenience sampling, we argue that results are generalizable to other former and current students based on the great extent of data saturation.However, our data do not provide much insight into the long-term effect of our program.We therefore suggest in-depth interviews with alumni five years after graduation.Furthermore, some of the attributes were difficult to operationalize during interviews (e.g.entrepreneurship and literacy), or students found were too overlapping (e.g.inclusivity, interdisciplinary, cross-cultural understanding and internationalization).In this research, we assume that by training students in inter-and transdisciplinary research, the graduates will be (more) inclined to conduct this type of research in the future, but we did not assess this.Nor did we assess the impact of the students' research on the communities.Finally, it must be noted that we (the authors of this paper) are involved in the program and that despite our genuine efforts to remain objective and practice triangulation, this might have influenced our analysis.

Concluding remark and further research
Overall, this study shows that the problem-driven Master's program MPA, with its combination of inquiry-based learning and CSL, provides a good grounding for learning the attributes.From a more global perspective, this study supports and adds to the work of Arvanitakis and Hornsby (2016) in relation to proficiencies and attributes of citizen scholars.
It also shows that more inquiry-based approaches and CSL programs can stimulate the four clusters of proficiencies, which should hold a central place in universities if we want to create citizen scholars.Our research raises two questions.First, what is the exact impact in the field and the added value to the local communities.Although we have some indications that there is a positive contribution, we do not know the real impact, and how this impact can be improved given that research finds that there is much room for improvement in community partnerships (e.g.Tryon and Stoecker, 2008).Moreover, it would be interesting to assess the long-term effects of the program.We also invite other scholars to implement and assess our problem-driven pedagogical approach for training transdisciplinary researchers.Finally, we argue that the proficiency and attributes are highly suitable to train inter-and transdisciplinary researchers, but for this purpose, we suggest adding a new proficiency cluster, "knowledge integration." Community service learning

I
think you can learn a lot from mistakes, […], however, I think it is also complex since in the end you have to obtain your grades.(Alumni 5.1) Students were interviewed in 2014 did not experience many opportunities to learn from mistakes It was remarked that the time-consuming assignments and the self-imposed pressure on "doing things right" made this difficult as shows this quote from a student: You cannot just relax, because you are constantly working on a new deadline.This makes it difficult to reflect and gain insights.(Student3.1) Figure 1.The attributes of proficiency cluster 1 applied in current jobs of MPA alumni Figure 2. The proficiencies and attributes of the citizen scholar of Arvanitakis and Hornsby (2016) enriched with knowledge integration