Feminist Approaches to Monitoring, Evaluation and Learning

Sport, Gender and Development

ISBN: 978-1-83867-866-1, eISBN: 978-1-83867-863-0

Publication date: 10 December 2021


Ahmad, N. (2021), "Feminist Approaches to Monitoring, Evaluation and Learning", Sport, Gender and Development (Emerald Studies in Sport and Gender), Emerald Publishing Limited, Leeds, pp. 191-209. https://doi.org/10.1108/978-1-83867-863-020211008



Emerald Publishing Limited

Copyright © 2021 Nida Ahmad. Published by Emerald Publishing Limited. This work is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this work (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


This work is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this work (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

The interest in SDP has been accompanied by calls for robust systems to monitor (or measure) and evaluate (M&E) if and how change occurs through sport. These calls for M&E have emerged both from stakeholders within the SDP sector and those outside of it. Stakeholders working within the sector can include funders who wish to measure value for money of programming through social return on investment (SROI) analyses; practitioners who seek to improve their project's effectiveness and impact; or external evaluators hired by SDP projects to meet donor requirements for evaluation (Langer, 2015). Those outside of the sector include critics of the claims made by SDP projects (Ireland-Piper, 2013). We – the authorship team (AT) who engage with both domains – also prove vested in this topic, and we believe that critical feminist research on MEL (Monitoring, Evaluation, and Learning) can both build the capacity of SDP projects and advance important discussions on the purpose and place of MEL within an organization or project.

In this chapter, we apply feminist perspectives and praxis to the topic of MEL within SDP and explore subsequent possibilities within sport, gender and development (SGD). Feminist critiques of MEL and feminist approaches to rethinking MEL are especially suited for SGD initiatives that seek to prioritize girls' and women's voices and ways of knowing in local contexts. Additionally, MEL systems – when designed with feminist principles – can reveal the existence of gender imbalances within any SDP project, ultimately allowing projects to achieve greater gender equity. The insights from this chapter demonstrate the possibility of embedding bespoke MEL systems that meet donor expectations, facilitate organizational learning, and align with an organization's philosophy. Perhaps most importantly, this chapter demonstrates how and why one should integrate feminist principles into the MEL process.

Well-designed MEL systems can help SDP and SGD projects assess their programming and demonstrate the unique impact of sport. However, many organizations develop MEL systems “guided solely toward upwards accountability” to donors and not for their own institutional learning or improvement (Azevedo, Garwood, & Pretari, 2019, p. 492). In this chapter, we consider the broad pressures faced by Skateistan-South Africa (S-SA) to respond to the dominant expectations around MEL in SDP. The chapter then builds upon the overview of ASDP presented in Chapter 5 to examine the possibility of developing an MEL system and building organizational capacity in MEL. This process unfolds between the international staff (IS) of Skateistan, the local staff of S-SA, and the AT.

Background and Context

While Skateistan's work in Afghanistan has garnered the most attention from media and supporters within the SDP community, in this chapter, we focus on a 2016 training held at their Johannesburg (South Africa) facility. While Skateistan had been running skateboarding sessions in Johannesburg for several years, it was not until 2016 – and the opening of the newly built Skate School facility – that they could focus on building their educational programs and developing more systematic approaches to MEL. 1 During the time of our collaboration, S-SA sought assistance on designing and integrating an MEL system that facilitated their reporting to external entities, aligned with their program's sensibility and design, and was transferable to their projects in Afghanistan and Cambodia. As feminist academics, we saw this as a unique opportunity to explore and re-imagine the way that MEL is understood, taught, and represented within SDP contexts. In this chapter, we utilize feminist decolonial theory in combination with an innovative narrative approach to encourage locally specific approaches to MEL that disrupt global SDP/SGD knowledge hierarchies and engage S-SA staff (SSA).

Drawing upon broad feminist principles and a decolonial approach, we encouraged a (MEL) workshop approach that allowed for the co-construction of knowledge about MEL practices and avoided a “top-down” mentality that positioned us (as academics) as sources of knowledge and the SSA as recipients. Such approaches are useful for reversing the paradigm by changing the sequence of typical training methods. This includes involving the intended beneficiaries and NGO staff (Mayoux & Chambers, 2005) in early discussions, decisions which are suggestive of moving toward a more participatory monitoring and evaluation strategy (Holte-McKenzie, Forde, & Theobald, 2006). In the case study presented in this chapter, we document our attempt to apply insights from feminist decolonial theory to both re-imagine the MEL workshop and to represent the findings/knowledge of this endeavor. In so doing, we hope this chapter makes a unique contribution to the growing body of literature on MEL in SDP, particularly in outlining how one might utilize insights from feminist decolonial theory to develop meaningful MEL trainings and practices. This chapter establishes insights into the MEL process that derive from the direct experiences of practitioners and academics and reveals the importance of considering feminist approaches to working in local contexts and with local communities, despite growing pressures from stakeholders and other key groups to “quantify” impact in particular ways.

Feminist Evaluation: Who Counts?

Feminist scholars have long questioned the dominance of particular forms of knowledge and challenged assumptions underpinning what “data” are and what ways of knowing count. They have often noted the exclusion of women's voices and knowledge, and this extends to the realm of evaluation as well. In the introduction of their special volume in New Directions for Evaluation, Sielbeck-Bowen, Brisolara, Seigart, Tischler, & Whitmore (2002) offer a corrective to this issue and outline six key understandings of feminist evaluation:

  1. Feminist evaluation has as a central focus the gender inequities that lead to social injustice.

  2. Discrimination or inequality based on gender is systemic and structural.

  3. Evaluation is a political activity; the contexts in which evaluation operates are politicized; and the personal experiences, perspectives, and characteristics evaluators bring to evaluations (and with which we interact) lead to a particular political stance.

  4. Knowledge is a powerful resource that serves an explicit or implicit purpose. Knowledge should be a resource of and for the people who create, hold, and share it. Consequently, the evaluation or research process can lead to significant negative or positive effects on the people involved in the evaluation/research.

  5. Knowledge and values are culturally, socially, and temporally contingent. Knowledge is also filtered through the knower.

  6. There are multiple ways of knowing; some ways are privileged over others (pp. 3–4).

This concise presentation has allowed for practitioners and scholars to apply feminist principles in their evaluation work although many readily acknowledge the challenges in doing this. For instance, Azevedo et al. (2019), a group of feminists working for Oxfam GB, reflect on their efforts to embed these values into their MEL work. In addition to considering the “successes and challenges” faced while utilizing MEL for gender equality, they also recognized the necessity to explore the “structural changes needed to bring about social changes through MEL and research more broadly” (Azevedo et al., 2019, p. 485).

Acknowledging these concerns, some feminist development scholars are working to advance approaches to MEL that can help highlight gender inequities. Writing in a special issue focused on gender and MEL, Bowman and Sweetman (2014) state:

MEL provides feminists with the means to explore the gendered impact of programmes and projects on the women and girls, and men and boys, whose lives are affected by it …. MEL can render development policymakers, practitioners and researchers accountable to the individuals and groups they aim to support, as well as accountable to the funders and supporters of that work.

(p. 201)

Yet how such data are gathered, what knowledge is prioritized, and what is done with evidence are deeply political issues imbued with gendered power relations. Such inquiries and issues are also highly relevant to MEL in SDP.

Though not adopting a focused gender lens, Nicholls, Giles, and Sethna (2011) raised comparable questions and concerns for the SDP sector, arguing that “the increasing call for positivist scientific evidence limits our ability to incorporate knowledge from grassroots practitioners into the policy development process” (p. 250). Conducting interviews with 17 SDP practitioners, they reveal how policy within SDP, as well as development discourses more broadly, tend to prioritize voices and expertise from the Global North (Nicholls et al., 2011). Moreover, they argue that one of the challenges to incorporating and amplifying voices from the Global South is a discourse of partnership which fails to consider the power dynamic embedded within donor-recipient dyads as well as colonial legacies around knowledge, voice, and expertise (Nicholls et al., 2011). This is one issue that could be mitigated by more attention to feminist principles in MEL, and examples do exist within SDP wherein partnerships appear to have carefully considered the power dynamics within the “donor-recipient-researcher” arrangement.

In their work with Go Sisters, an SDP project in Zambia focused on girls' empowerment, Kay, Mansfield, and Jeanes (2016) sought to create a space for reciprocal learning to proceed within their research project. They worked with the Go Sisters staff to help establish an M&E system that aligned with the requirements of their key donor and helped to develop the research capacity of the organization. Within the collaboration, they sought to imbue knowledge and insights from Go Sisters staff throughout the research and evaluation process (Museke, Namukanga, & Palmer-Felgate, 2016). One critical component of this was the need for qualitative approaches in order to monitor and evaluate the program's success and effectiveness. As Kay et al. (2016) note, critiques of qualitative methods in SDP within M&E “do not always distinguish between ‘stories’ compiled for promotional purposes and bona fide research-based accounts obtained through rigorous qualitative enquiry” (p. 224). They point to two examples wherein qualitative data and implicit challenges to the notion of objective and detached research afforded key learning opportunities for Go Sisters.

Over the past few years, some scholars have turned their attention to SDP practitioners to better understand their experiences and perspectives on M&E (Harris, 2018; Jeanes & Lindsey, 2014; Kaufman, Rosenbauer, & Moore, 2013). Kaufman et al. (2013) sought to understand how SFD practitioners in the Caribbean felt about M&E in order to assist others who seek to create effective M&E systems. Through interviews with five program coordinators, the following themes emerged which provided insight into the challenges faced by those tasked with M&E responsibilities: the need for capacity building; lack of sufficient time; difficult geographies; (the need for) technology and online tools; and passion for results. These concerns likely prove common to those working in the field, but as Harris and Adams (2016) suggest, we may not know them because (with noted exceptions) practitioners' perspectives on M&E are seldom sought. Moreover, Harris and Adams (2016) argue that

Practitioners have a limited voice in the discursive formation of evidence and to date have been unable to challenge dominant approaches to evidence. To what extent practitioners are docile bodies and represent dominant or alternative discourses are open to debate.

(p. 104)

In their estimation, SFD scholarship would be well served to learn from, and understand, practitioners' “relationship and view of evidence” (Harris & Adams, 2016, p. 104). Harris (2018) answers this call and further makes the case that practitioners are best suited to offer valuable insights into the M&E process because they are “entirely contextualized within their projects and are well placed to make sense of and reflect upon the extent to which any change is manifesting itself” (p. 3). To sum, much of this literature presents M&E as a critical element of SDP programming, especially for practitioners to report outcomes and demonstrate the effectiveness of SDP. It further shows how M&E “patterns of work” are increasingly becoming critical to the sustainability of the SDP field (McSweeney et al., forthcoming, p. 12).

In addition to the calls for more engagement with practitioners, scholars in both SDP and international development are considering broader questions of evidence and M&E in increasingly complex ways. For instance, Henne's (2017) recent work on the power of “indicator culture” implores SDP scholars to move away from an exclusive focus on human actors as the sole shapers of M&E to consider the ways that technologies hold sway in the MEL realm. This research encourages us to consider how indicators are agentic and become both “descriptive and instructive” when it comes to SDP projects and outcomes (Henne, 2017). Eyben, Guijt, Roche, and Shutt's (2015) work on The Big Push Forward (conference and initiatives) within the international development sector raised meaningful questions about the “results and evidence agenda.” Commencing in September 2010, the “Big Pushback” initiative consisted of an informal network of development practitioners interested in “constructive ways to advance conceptually and methodologically … beyond the narrow bureaucratic protocols that assume guaranteed predictable outcomes” (The Big Push Forward, n.d., “About,” para. 2). For Eyben, Guijt, Roche, and Shutt (2015), the results agenda is designed “to improve and manage development aid through protocols, procedures, and mechanisms for reporting, tracking, disbursing, appraising, and evaluating its effectiveness and impact” (p. 1). In a similar vein, other development scholars critique indicator culture in development writ large, suggesting that there is an urgent need to question “indicator culture” that is mostly driven by Global North actors in wealthy, industrialized nations, ensuring that “indicators emerge through social processes shaped by power relations, expertise, and techniques of measurement” (Merry, 2016, p. 25). Thus, Eyben, Guijt, Roche, and Shutt (2015) and Merry (2016) seek to move scholars and practitioners beyond a straightforward belief in the value of evidence. They work to identify “collective efforts to make development work for social justice” as opposed to top-down approaches to M&E (The Big Push Forward, n.d., “What We Are,” para. 6). They inspire new lines of questioning about MEL that are significant for work in SDP, such as: Who is asking for evidence? How is power associated with knowledge? Who is seen as a credible source or reporter of evidence? This chapter seeks to explore such questions and bridges the divide between academic and practitioner insights on M&E, in part, by foregrounding how local Skateistan practitioners and IS respond to various calls within the sector for evidence, results, and robust M&E systems.

Toward a Feminist Decolonial Approach to MEL: Framing the Case Study

As feminist researchers, we are generally highly critical of top-down approaches to training and education and thus approached the S-SA training with specific understandings about what this training could and should look like. Mainly, we sought to find ways to collaborate and work with international and local staff to ensure that the methods being developed consider local knowledge and are meaningful to the local program. In so doing, our theoretical orientation and methodological approach were guided by feminist decolonial theory and crystallization, which are complementary but distinct ways of unsettling notions and representations of knowledge and power. Feminist approaches to the decolonization of SDP also underpin our thinking about the politics, power relations, and potential of MEL (see Chapter 2 for more on decolonization in SDP). In an applied sense, we used insights from decolonial thinkers to inform how we constructed and delivered the workshop.

Our desire to engage in a more collaborative process of MEL training aligns with recent calls from academic writing in the SDP field (e.g., Sherry, Schulenkorf, Seal, Nicholson, & Hoye, 2017; Spaaij, Schulenkorf, Jeanes, & Oxford, 2018). While we do not claim that our approach to MEL training was participatory action research (PAR) in its purest sense, we included many of the typical sensibilities of PAR, which includes careful considerations of “participation, power, and reflexivity” (Spaaij et al., 2018, p. 27). As academics traveling to South Africa from the United States and Aotearoa New Zealand, we were especially mindful of embracing a “decolonial research consciousness that [was] … attentive to the productions, circuits, policing(s), and geopolitics of knowledge” throughout the entirety of this project (Murrey, 2017, p. 80). Outlining such sensibilities, Murrey (2017) suggests that this type of approach “refrains from claims to authority and challenges the positivist notions of objective knowledge …. [It] is an ethos that is questioning, humble, and grounded in the respectful turn and return to the voices and stories of people” (p. 80). Murrey's (2017) representation of their research in Cameroon included “de-privileged knowledge expression [by] including poetry, joke-telling, and narrative” to highlight how knowledge is co-created through exchanges as opposed to emanating from one (or more) particular academic experts (p. 80). We attempt to access this sensibility in the narrative “write-up” of this chapter.

Additionally, taking feminist critiques of evaluation and MEL to heart, we did not approach the workshop process as “experts” seeking to “empower” local staff with “our” knowledge of MEL. Instead, we sought to break down power relations and create space for dialogue and mutual learning between the authors and the staff, before, during, and after the workshop. We did this when we sought regular feedback on the training schedule, workbook materials, and the training itself. We intentionally co-designed a program that valued the multiple voices and perspectives of those involved in this process, even (and especially when) those voices diverged, revealed uncomfortable tensions, and offered insights that challenged our perceptions and understandings.

These practical approaches reflect the tenets of decolonial theory and its utility in SDP (Mendoza, 2015; also see Chapter 2). Mendoza (2015) offers a comprehensive overview of decolonial theory and its history. For our purposes, the central tenet of decolonial theory that we sought to utilize was the ways in which knowledge is (re)conceptualized. According to Mendoza (2015), colonialism and coloniality (the present-day manifestation of colonialism) purport that:

European knowledge production was accredited as the only valid knowledge, [and] indigenous epistemologies were relegated to the status of primitive superstition or destroyed. Eurocentrism locked intersubjective relations between the European and the non-European in a temporal frame that always positioned the European as more advanced.

(Decolonial theory, para. 6)

This matters within our project as we recognized the potential for (mis)understandings of who could and would be knowledge producers within this workshop and within this research. That is, as researchers, we traveled from outside South Africa with academic and geopolitical “credentials,” and we were likely perceived as bringing knowledge to the local staff. 2 Aware of potential knowledge hierarchies, we tried to complicate such perceptions, as evidenced in both the structure of the workshop and the responses provided in the case study narrative section below.

We also draw insights from decolonial feminist scholar Lugones (2010) whose work affords us the opportunity to consider how “categorical, dichotomous, hierarchical logic” is vital to “modern, colonial, capitalist thinking about race, gender, and sexuality” (p. 741). In this case, we considered how this logic manifests in the establishment of categories, dichotomies, and hierarchies present in most SDP MEL as it relies on logic frameworks (logframes) to set its course. We did not necessarily seek to challenge the notion or viability of MEL through the course of the workshop, though this certainly could be a part of the application of decolonial theory. In this project we sought to build on Lugones' (2010) understanding that “feminism does not just provide an account of the oppression of women … it enable[s] women to understand their situation without succumbing to it” (p. 747). In a similar vein, within the workshop we sought to acknowledge the tensions around MEL that exist for the organization and the sector. Alongside this, we co-created space to identify resources for making the most of the industry expectations around M&E.


As both SDP research and approaches to M&E have been long dominated by scholars working in the Global North (Schulenkorf, Sherry, & Rowe, 2016; also see Chapter 2), the acute recognition of the politics of knowledge underscores our decision to represent our research findings through an innovative narrative approach that challenges our positions as author/experts. In our representational approach, we draw inspiration from Ellingson's (2009) application and explication of crystallization. Ellingson (2009) builds upon foundational work by Richardson (1997) and Lather and Smithies (1997) who sought to push boundaries of accepted representational strategies within qualitative research. The metaphor of the crystal is often juxtaposed with the triangle (of triangulation) to underscore a key difference in conceptions of knowledge:

Crystallization enables qualitative researchers to generate less naïve representations. We can continue to engage in and learn from systematic knowledge production without simply perpetuating the remnants of positivism in our writing. Crystallization allows more freedom to portray accounts that reflect current sensibilities about the slippery nature of claiming knowledge without forcing us to give up systematic research methods.

(Ellingson, 2009, p. 15)

In this case, crystallization allows for multiple truths to be shared without disavowing the research process entirely. In practical terms, the use of crystallization enabled us the space to deliver a training that included “traditional” research methods and emphasized the importance of systematically collecting data, while also presenting our findings from this training in a creative and dynamic manner. We represent the findings/learnings from this training and larger experience via an “imagined” dialogue that transcends space, time, and medium. While imagined is used in the previous sentence to acknowledge the constructed nature of the findings, it also indicates that the interactions and creation of the M&E training took place across multiple spaces (i.e., office at Skateistan's International Headquarters; Skateistan Johannesburg offices and skatepark; personal accommodations), mediums (i.e., face-to-face, email, Skype), and countries (i.e., Cambodia, Germany, Aotearoa New Zealand, South Africa, United States).

In the sections below, we present important contextual information on the MEL process as well as key moments from the research process to move toward additional understanding about MEL for both S-SA and the realm of SDP. The deliberate co-existence of voices – from academics, SSA, and IS – is done to avoid any sort of hierarchical ordering of voices, perspectives, and knowledge. While some excerpts below have been edited slightly to protect confidential information, the edits do not compromise the sentiments expressed. This co-mingling of voices is an important element of decolonizing the expected “outcome” of research such as this as well as the presumed dynamic in much SDP work.

MEL in Skateistan-South Africa: A Narrative Approach

The following paragraphs offer important background information about the MEL workshop and the context of the case study that is the focus of the remainder of this chapter. In early 2016, Megan and Holly were involved in a meeting with the international Skateistan team in their headquarters in Berlin. This meeting emerged from Holly's research with Skateistan. The meeting considered the potential for Skateistan to develop MEL approaches that could meet the organization's needs and were consistent with their philosophies, as opposed to being steered by their funders (Win, 2004). Following this meeting, Megan engaged in regular virtual meetings with an IS member involved in program development across the three countries and another IS member who was recently appointed the role of managing M&E across the organization. Based on her experience of consulting with other organizations on these topics, she offered feedback on Skateistan's Performance Management Plan, theory of change, and other key organization documents. This pro bono consulting role included regular Skype meetings, written feedback, and productive questions on the aforementioned documents. These exchanges built the foundation for the eventual training workshop at S-SA.

Through the course of these encounters and the eventual workshop with S-SA, our assistance was sought to help Skateistan move from a focus on M&E to an approach that encouraged monitoring, evaluation, and organizational learning (MEL). The organization sought to make M&E more useful and meaningful for the organization instead of just fulfilling a requirement for donors; they hoped to learn and grow from this process. Skateistan staff wanted to embed organizational learning in the M&E process through collaboration with academics (the AT) as well as through considered involvement of IS and local practitioners. The vision for the workshop at S-SA was that staff would learn about M&E and be encouraged to think through their M&E options. Then, they would be equipped to make informed decisions about their MEL systems and design an approach that worked with and for them. To facilitate this, Megan and Holly were to acquaint SSA with a range of methods available for evaluating their work with youth. Importantly, we were not expected to design or implement M&E systems for the organization. We wanted to introduce a variety of ideas and options and to let those closest to the work decide on a path forward for the organization.

The Workshop

It was from both Megan and Holly's relationships with Skateistan that the opportunity to plan and deliver S-SA M&E training/workshop came about. They worked closely with international Skateistan staff to plan an eight-day workshop that would meet the needs of the organization and local staff. The preparations included the creation of a workbook that was used daily and that could be used for future training sessions at other locations. Closer to the date of the training, Thorpe was unable to travel to Johannesburg due to health reasons, one of her (then) PhD students (Nida Ahmad) stepped in to help facilitate the workshop. Following a series of research team meetings, in July 2016, Chawansky and Ahmad traveled to Johannesburg to facilitate the workshop. Their travel costs, accommodation, and a modest per diem were paid for by S-SA through grant funding that was allocated for M&E development.

The eight-day workshop included the following topics: interviews, observations, visual and creative methods, life history methods, curriculum-based methods, and focus groups. Each day of the workshop differed due to the on-the-ground demands of local staff, but most days included a lecture/discussion on a method, an applied activity that allowed practitioners to “try out” the methods and reflections on the viability of the method for S-SA. The reflections focused on cultural considerations and necessary modifications within the South African context. For most days during the eight-day program (see Table 1), four local staff members attended the training sessions, as did a member of the IS. On occasion, part-time or volunteer staff would participate in portions of the training, and we welcomed a dynamic, open space where people and ideas could flow in and out freely.

Table 1.

Skateistan Eight-Day M&E Workshop Agenda.

Day 1 Introductions: Conducted Interviews on MEL Knowledge with staff
Day 2 Morning: Key concepts in research and M&E and Interviews
Afternoon: Applied session
Day 3 Morning: Systematic observations
Afternoon: Applied session
Day 4 Morning: Visual and creative methods
Afternoon: Applied session
Day 5 Morning: Life history method (testimonials)
Afternoon: Child-centered research and ethics
Day 6 Morning: Discussion: Curriculum-based methods
Afternoon: Case study/writing up results
Day 7 Morning: Focus groups
Afternoon: Analysis of data/write-up
Day 8 Morning: Final questions/evaluation of training

Chawansky and Ahmad worked closely throughout the workshop to prepare and then to reflect on the sessions. They submitted daily summaries of learnings and reflections to Thorpe to ensure she could remain engaged in the process and offer feedback. These summaries were also shared via email with IS to keep them informed of the workshop progress. Throughout the entire process, reflections from the authors occurred in person and via email or Skype with IS. Interviews and focus groups involving staff members occurred at the international headquarters of Skateistan as well as the S-SA offices. During the training, we collected data and reflections on the experiences for both the research projects and also used these moments as teaching tools. At the time of our workshop with S-SA, Skateistan was in the early stages of developing organizational MEL approaches and strategies, and the workshop and subsequent MEL efforts were part of this broader work. Since our first training session, the organization has continued to invest considerable time and energies in their MEL approaches, and in March 2017, they joined forces with a consortium of eight international NGOs to form the “Action Impact Network” – a group that is working together to “develop a series of indicators and data collection tools to measure the impact of youth action sport for development programs” (Skatesian, 2017, p. 36). This involvement reflects Skateistan's continued interest and investment in this area.

Crystallization of the Voices of MEL

This section summarizes the key learnings that emerged from our attempt to co-construct an M&E workshop and assist in the development of an organization's MEL practices while engaging with feminist decolonial perspectives. As noted above, our practical application of this theoretical orientation valued collaboration, local and organizational knowledge, and sought to break down expected knowledge hierarchies related to both the workshop and the embedded academic/practitioner dynamics. To organize this section, we offer creative and re-created conversations in response to a series of common questions about MEL within SDP. In our attempt to bring multiple voices into conversation, we offer (partial) answers to these questions by drawing upon interviews, email correspondence, daily summaries of the training sessions, and post-training reflection notes.

The voices of IS, SSA, or the AT are presented in a manner that complicates a singular and linear narrative about MEL. The IS includes those Skateistan staff members who worked across projects/locations on MEL while the SSA staff included the four participants who worked for the organization and attended the 2016 training. As the AT, we seek to avoid speaking over or for our collaborators or to write ourselves in as the experts. The lack of clear attribution to one named individual is intentional. The narratives that follow are intentionally provocative to add complexity and nuance to the discussion rather than seeking to prescribe a correct way to “do” MEL workshops and systems. Decolonization is ongoing, and in this understanding, it requires embracing the incompleteness and messiness of the process, and subsequently, of the learnings that have continued long after the workshop came to a close.

Who Needs Evidence?

Skateistan-South Africa Staff (SSA): For us on the ground, we recognize the importance of producing all of these data. I also understand that it's very difficult to meet the interests of the funders … without ending up overexaggerating what's really happening. You do need all these reports to be able to give back so the funding can keep on happening, but how do we stay true to what's on the ground and expectations which are met elsewhere and not evidenced in the data and reports? Finding that balance can be tricky.

International Staff (IS): I think until now a lot of our M&E has been really donor-driven so it has really been this reactive thing. Now we've gotten to the point that we have so many questions internally, but we've set a goal for ourselves to become leaders in M&E for action sports for development. So now we want to go over and above that and do some stuff for ourselves and try out new things.

Authorship Team (AT): Well, I think this is a wonderful place to be in. It's great to hear that you're interested in moving beyond the need to simply evidence the effectiveness of your programs. I understand that quantifying impact is important, and you need to do this for reporting to funders, etc., but you are an organization that has always prided yourselves on thinking outside of the box. I suggest there is a lot of potential for you to explore more qualitative M&E methods, more creative approaches to M&E, and finding ways to really draw upon the skills and strengths of your local staff to design approaches that are culturally specific and meaningful to them, and not just funders.

IS: Yes, we want to be one step ahead of any requirements of our donors and we also want to dictate the way that it's happening. But because we have programs in three countries now, we do need to be consistent across the sites, so that we're not doing it one way in one location and another way in another location. We need to do it to a certain standard that we can then convince people that that's enough, and that's all that they need us to do. That will make it easier on our side if we can have it harmonized globally.

AT: Okay, so as I understand it, a challenge moving forward is finding ways of doing M&E that are consistent across your programs. This makes sense. But I would also encourage you to continue to look for ways to modify or adapt some of your M&E approaches so that they are locally specific, culturally meaningful, and culturally appropriate in each location.

What Kind of Evidence Is Useful?

IS: For the first few years, I think we've been influenced by what we've seen other NGOs doing with M&E, as well as expectations from funders. We have been focused on outputs. We've also been very curious about what the outcomes are, but we haven't been very consistent about how we're measuring those. We have tried different surveys. We wanted to do a giant baseline survey covering all of our theory of change outcomes at one point, but our imagination was sort of limited to surveys and media. We've also used blog posts and photos and stuff, as well as student testimonials that were mainly being collected for media or for donor reports specifically. Mostly we've been focusing on trying to quantify our outputs, but I think we could be open to exploring other approaches.

IS: I agree, sometimes it has been hard for us to understand the potential of things like a student interview or a student quote, and to organize those things according to particular outcomes, or to have these conversations about M&E that dig a little bit deeper. We've tried to do focus groups, but there's been a lack of training at all levels on how to do a focus group.

IS: Yes, this is a good opportunity to stop and think about how we're doing M&E and why we're doing it that way. I think potentially a lot of organizations are falling into a trap of following what someone else has done and then trying to make that fit into what they're doing. I think we've also fallen into that trap a tiny little bit as well. Like, we're trying to move away from surveys but at the moment it's still the easiest and most familiar for us.

AT: It's exciting to see that within your organization you're starting to question some of the pressures and assumptions underpinning dominant M&E models. There are actually quite a few researchers and practitioners out there who are also questioning some of the assumptions and expectations of M&E and advocating alternative approaches. We have experience using quantitative methods, but much more experience using qualitative methods in our research and we would love to work with you to explore some alternative approaches to how M&E is typically thought to be done.

How Will Evidence Work for Us as an Organization?

IS: Something we're really keen to move toward is data-driven decision-making, but we're also really excited about developing methods that are maybe more consistent with our organization's philosophy. One thing we do need to consider is how we can develop M&E approaches that aren't too heavy, aren't too onerous on our local staff, because they are already really busy with everything they're doing on a day-to-day basis.

AT: It's great that you're considering how the implementation of M&E will impact upon your local staff workloads. I think it's also important to consider how local staff might perceive the processes and methods of M&E: Are they perceived as valuable? Do they feel they have had input in developing or modifying the method? Do they know where that information is going and how it's being used by whom and for what? Perhaps if we can provide opportunities for local staff to be involved in developing a method they might have better buy-in and commitment to using it on a regular basis. Of course, this doesn't need to be all of the methods used, but maybe one method, or something where there is some space for co-construction to help with that sense of value and understanding of why it's used.

AT: So perhaps this is something we can explore. What's the potential there for working with local staff so that they understand the purpose of this, and it doesn't feel like some onerous task being imposed upon them from above, but actually that they are part of its process? We will certainly try to design the training program with these aims in mind.

How Will We Co-construct Knowledge about M&E within the Workshop?

AT: We are really excited about this opportunity. We are working through some of our preliminary plans and ideas, and one thing we know for certain is that we really want this to be a collaborative, hands-on training, and that we will not adopt a “top-down” approach. I sense that we all agree that any methods or expertise we can bring must have buy in and be relevant to local staff, so we plan to create a training that allows for not only the presentation of information on different MEL methods (i.e., interviews, focus groups, systematic observations, curriculum-based methods) but also piloting various methods and reflection on these methods as they relate to the local context. We wanted to raise this with you now to make sure we are all on the “same page” and to make sure our expectations are the same.

IS: Thank you for sharing your initial plans with us. We have talked through your proposed training program at the international office, and we're all really excited for what this will offer our local staff, and also for future possibilities to roll out similar training across our sites.

IS: Yes, I agree, your proposed approach is very interesting. I think it's important to keep in mind though, that data needs to be collected systematically and consistently for it to have value. So, for me, that's where the main focus of the training should really be, and then once the data is there, it's not that you're just collecting it for nothing, we are going to be able to do a certain amount of analysis in-house. But it will also be really valuable, because a lot of sport development organizations and a lot of projects are only for six months or for a year or a workshop or they've got such small data points in time, the timeframes are always quite small. If we've got high quality data collected over five or 10 years, this might be the first time that there are those data sets over such a long period of time. That could then be interesting to people from lots of different backgrounds, and they'll be able to use that data in different ways. So I think it's more about focusing on the collection than the analysis.

AT: Okay, this is good feedback as we were planning to include some analysis in the training. But if you would like us to focus more on data collection methods, then we can certainly do this. This makes good sense and will give us a bit more time to work through some of the challenges of gathering data – systematically and consistently – in local contexts. For example, after discussing different approaches to doing systematic observations, we could go out and practice doing systematic observations in a busy skatepark during a lesson and then head back inside to discuss some of the challenges of doing this method in this space at that time.

SSA: That will be really great if we can get some hands-on experience taking these methods into the spaces that we actually run our classes. The skatepark is so busy that it's hard to imagine how to collect good quality data there. The local staff have talked about the proposed program and like the opportunities that you're suggesting we will have to contribute to some of the methods. I think the one area that we should reflect more on as an organization, because we have the various sites, but also because South Africa is so culturally diverse, is the cultural basis. How do we use or develop methods that consider the different cultural knowledge we have here? How can we use a particular method but still learn from our participants or staff experiences without trying to make them fit into our definition of confidence or Western ideas of whatever leadership is, or whatever these concepts are we're looking at? So, how can we try to broaden our definitions to include lots of worldviews in different M&E methods?

AT: These are really insightful questions that we will need to discuss at the training, and it's highly likely that you will have some of the answers to these questions because you know your context better than anyone! Ultimately, the training is only going to be useful if the approaches can be used by SSA staff in ways that make sense for your site.

What Are Your Hopes and Fears about Participating in an M&E Training?

AT: Finally, we're here with eight days of M&E training ahead of us. Thanks so much to everyone for their input on the program thus far and for being here to share this journey. Before we get started, we'd like to open it up to hear some of your thoughts and feelings about the planned training.

SSA: I won't lie, the MEL that we're doing now, it's the first time for me. It is a lot of responsibility being the point person, but so far I've done as much as I can where I've tried to understand what it is that is required of our team for this particular situation with [name of funder] where they want me to report back on very specific things or create these tools for very specific things. I guess we're stressed because we don't know exactly how it's going to be done, but I think it's exciting for us because we always want to know, like what's the outcome.

S-SA: My concern is more about your motivation. What are you trying to achieve in coming all the way to South Africa? What are your expectations? Or at least your preconceived expectations before anything happens here?

AT: I hope I can learn a lot from this experience and see what's going on here on the ground in S-SA. We're also hoping that we can make a little contribution to make your life in M&E easier. We know that people have to do this [M&E] if they want to stay alive and get funding, and sometimes it's not a nice process all the time. But I think we can help make it a better experience, that's what we want to help do. At the end of the day, we're coming here to train you all in M&E, but I think we will learn from each other. I think it's more likely that we'll learn from each other. We might be coming in with knowledge about M&E, but you are bringing all of your knowledge and expertise about this local context. Hopefully together we can help you and Skateistan move a little further down the road toward effective and efficient M&E processes.

What Are Attributes of an Effective Day of M&E Training?

AT: Overall, today was a good day. The space we've tried to create to try out some of the methods we've introduced seems to be working. The hard part is fitting in some of what we planned despite the fact that the local staff still have their outreach and other duties to complete throughout the day. One thing we do need to prioritize is more time for reflection and conversation about whether some of these M&E ideas are viable. This has come up as something that the staff would like more time to process, so I think we need to revisit some of our daily plans, change things around a little bit, to ensure there is more time and space for reflection and discussion.

IS: I'm really happy with how the training went today. I think that it felt really good. It was fun. A lot of balance, at least half of the time was very hands-on, there was very little lectures – some info shared then space it out with some activities. I think making it hands-on made it really successful for everybody, they could quickly imagine integrating it into what they do on a day-to-day basis. So, I think we are doing something good. I was really grateful that you came super-prepared and had that whole guidebook and whole agenda made. Your organization was really appreciated.

SSA: What I learned today after doing the discussion about observations and then trying to do them in the skate lesson, is that we had these grand plans and we realized that okay, these were just plans in our heads. When we actually did the tool itself we came back with nothing and literally had to rework it.

SSA: Something that came up for me is thinking about the kids. I really think that kids are very sensitive, and they can easily see a change in some of the ways we have been interacting with them. But now, it seems like it is going to have much more of a structure, so the question is: does it take away from our sessions now that we know what the agenda is?

AT: That is an interesting question. How can you make sure that the steps you are taking to collect data do not interfere with how you are delivering your sessions, or with how the children respond to you/the session? Some of this might be out of your control, but it is good to be aware of this possibility and to think about ways to work through this issue.

Did the Workshop Change Staff Perceptions of M&E?

IS: I think we've built up a lot of capacity to actually administer M&E, and I think there's more buy-in and more understanding of what it entails and more willingness to do it. I think we still have quite a bit of learning to do, but we're on the right track.

SSA: After the training, I now feel that I can go to corporations now and do a proper M&E and be proud. We've done it in the past, but we didn't know methods. We knew about monitoring and evaluation, but at some point, remember, we were like “what does MEL stand for?” And you said Monitoring, Evaluation, and the Learning. At that early stage in the training, we knew, but we didn't know. I feel like now we have a proper understanding. We can do it more professionally, which gives us confidence to be like, “Hey, we're teaching kids stuff, and yes, they're learning stuff.”

IS: Going forward, I think it's going to make our job much easier in terms of structure. Because we need to know what we are doing and why we are doing it, like which boxes we need to tick. That's really important, because it's very easy to just waste time with some of our agendas, and some of our own personal agendas and think that they are part of what we are doing here at Skateistan … I think this training has helped us, equipping us with more skills to order all this chaos. So, I think, just giving something very precise and cohesive, and it has got much more of a solid direction. It's much clearer. I think that's what it is going to help with. That's what I'm taking out of this.

IS: Now that I've witnessed your approach and been involved, I think now we should be able to replicate that, adapt it very efficiently, I would say, fairly rapidly to a whole new context. I know you guys put in a ton of labor and made that PDF and stuff, but I think that it’s not just for this year, because it's going to support us going forward to do one of these in Afghanistan in the near future. This training has given me a lot of confidence that I can teach this stuff and that we can find ways to teach it ourselves. I think it's great to have you here too, just for moral support and encouragement, as well as to share your knowledge.


Life without MEL [training] is a bit weird because now we have all the cool and useful information and knowledge, but we don't know where to start. I won't lie, it's a bit nice not learning the whole day for 10 days [sic], but it was really nice having you guys coming from so far to share your expertise with us.


We conclude this chapter with an unsolicited message offered by one SSA staff member to the AT. This email excerpt came to the AT after we had returned to our respective homes and captures some of the complex feelings we have about the workshop. The staff member suggests that getting back to daily life after an intense training on MEL can be an adjustment. Throughout the duration of our training we were conscious of the double duty that SSA managed. They worked hard to stay attentive and engaged during the MEL training, and we often adjusted our plans because of their other commitments and daily responsibilities. We changed plans on the fly to integrate unique learning opportunities and learned to sit back and watch when no adjustments were possible. We tried hard to challenge traditional knowledge hierarchies within the workshop through co-creating the workshop plan and the teaching style we imparted, but in the end, we were still lauded for “sharing our expertise” with the local staff. This post-training reflection perhaps indicates some of the limitations of our desired decolonial approach and of the structures we were up against.

Despite this, we remain committed to both further developing a decolonial approach and to assisting the organization in their MEL efforts. We have also maintained contact with Skateistan since completing the MEL workshop with S-SA. Thorpe continued her larger research project with the organization, and in June 2017, Chawansky observed the international Skateistan staff lead a MEL training in Cambodia. She served as a “critical friend” and provided constructive feedback to the staff running the sessions for their first time. The training attended to the local staff needs and differed in significant ways from the S-SA workshop, but stayed true to the spirit of trying to build MEL culture from within the organization. While Skateistan seeks some common tools and indicators across program locations, they are flexible enough to adjust and manage their expectations around MEL as it relates to local staff, language, experience, and cultural context. We believe that we took important steps in the S-SA workshop to help Skateistan in their efforts toward rethinking MEL. 3

Through this example we shared why and how one might decolonize elements of SDP, especially as it relates to transnational partnerships and relationships. We see this chapter as extending Oxford and Spaaij's (2019) use of decolonial feminist analysis insofar as it illustrates our attempt to actively decolonize the process of teaching, learning, and subsequently designing MEL for one organization. Our narrative approach sought to bring to life our quest to co-create knowledge and amplify local voices, especially when it concerns MEL. This topic (MEL) proves especially fruitful for this task as so many of the conversations we had with SSA alluded to the imposition of M&E from outside (funders). Though the voices and perspectives of funders were not included in an explicit sense in this research, their perspectives and voices pervaded the conversations we had before and during the workshop.

As a counter to this, our final section did not privilege our analysis or voices as another external imposition telling readers what or how to interpret the information presented. Taking inspiration from feminist critiques of evaluation and MEL, we attempted to challenge conventions of transnational knowledge production with our representational choice. In this way, we follow Mignolo (2009) who argues that “it is not enough to change the content of the conversation … it is of the essence to change the terms of the conversation” (italics included, p. 162). We believe this is one of the most valuable contributions of this chapter, and we hope it inspires further considerations of the politics of knowledge and representation in MEL and SDP. Using feminist decolonial theory as a guide to inform our workshop allowed us to co-construct knowledge in unique and unanticipated ways. In this instance, we suggest it helped us to recognize the opportunities and challenges that come with continuous reflection on power, privilege, and knowledge within MEL and the SDP sector. Using this approach does not guarantee success or specific outcomes; it is an ongoing project and process.

While decolonization is always an ongoing project, the quest to rethink how, where, and from whom knowledge originates is the cornerstone of this chapter. In so doing, this chapter offers a specific example of some of the key issues raised in Chapter 2. We hope the insights we share via our case study are useful to those working in MEL or researching MEL or SGD. The insights intend to encourage a rethinking of how one can “do” MEL and of what feminist principles can bring to the practice of MEL. The case study also extends recent critiques of how and why we evaluate our programs, what we “do” with data, and how to make it work for organizations. The long-standing feminist concerns around what “counts” as knowledge, whose voices are prioritized, and how such knowledge is represented (in reports or research) are equally valid in SGD. We believe this chapter offers a model and encouragement for those working in the sector to pay attention to the power relations and ethical considerations in such knowledge production processes. We also hope that our efforts to offer an alternative representational style – a narrative style that privileges multiple voices across time and space – encourage others to rethink the ways we write in our academic outputs as well as organizational reports. There is much potential in future feminist-inspired SGD work to explore alternative modes of representation that better reflect the seismic shifts still required to destabilize the patriarchal, neocolonialist, and Global North/South power relations that remain deeply entrenched in the field of SDP scholarship and practice.


In 2017, the Skate School had a total of 291 active students, 40% of whom are girls, with 94% of the students coming from low-income backgrounds (Skateistan, 2017, p. 33).


Although we did not collect biographical information on the local staff, to our knowledge and recollection, none of the staff had relocated to South Africa to work for the organization, and they could all be considered “local” staff. These are our interpretations and not necessarily that of the organization or the participants.


For more information on Skateistan's current MEL efforts, please see https://www.skateistan.org/our-impact.