Rewiring the STEM pipeline - a C-B-E framework to female retention

Abstract

Purpose

Females are underrepresented in Science, Technology, Engineering and Mathematics (STEM), both in the workforce and in universities. Low self-efficacy and limited access to role models are key factors preventing retention of female STEM students enrolled in university degrees. This paper aims to report on one social marketing pilot programme that was co-designed to increase self-efficacy in females currently enrolled in STEM programmes.

Design/methodology/approach

The Co-create, Build and Engage (C-B-E) framework was applied. Process and outcome evaluations were conducted using a repeated measure design to assess pilot programme effectiveness.

Findings

A significant increase in self-efficacy and high satisfaction rates were observed for STEM students that attended the bias literacy workshop. Social advertisements raised awareness for available STEM specific university services.

Originality/value

This paper outlines the application of the C-B-E framework. To the best of the authors’ knowledge, this study delivers the first scientific paper reporting an outcome evaluation for a social marketing programme seeking to retain women enrolled in university STEM degrees.

Citation

Roemer, C., Rundle-Thiele, S., Pang, B., David, P., Kim, J., Durl, J., Dietrich, T. and Carins, J. (2020), "Rewiring the STEM pipeline - a C-B-E framework to female retention", Journal of Social Marketing, Vol. 10 No. 4, pp. 427-446. https://doi.org/10.1108/JSOCM-10-2019-0152

Introduction

Science, Technology, Engineering and Mathematics (STEM) industries have been central to most great innovations in the modern world. This is reflected in economic terms and the numerous innovations improving peoples’ quality of life (Corbett and Hill, 2015; National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2010; Riegle-Crumb and King, 2010; Suresh, 2007; Uncles, 2018). The importance of STEM and the need to grow STEM is referred to as the “fourth industrial revolution” (Moavenzadeh, 2015; Schwab et al., 2015). The STEM workforce is demanding, and nations compete to recruit and retain the best minds (Cover et al., 2011; Espinosa, 2011; National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2010; Padró, 2012; Sax et al., 2017). With constant demand for innovation and efficiency, recruitment of a strong STEM workforce must remain a national priority (House, 2010; Tan, 2016; Uncles, 2018). As outlined in the Commonwealth of Australia Advancing Women in STEM agenda (2019, p. 3):

[…] the causes of poor attraction and retention of girls and women in STEM begin from an early age and compound as progression to more senior careers is made. Often referred to as a “leaky pipeline”, the result is a system with low representation of women in STEM education, workplaces, and senior level leaders, and a society that undervalues the opportunities and contributions a career in STEM can provide for girls and women.

Females comprised less than 15% of domestic students completing engineering and related technology degrees in Australia in 2016 (Commonwealth of Australia, 2019). Recent statistics indicate that almost one-third of females in Australia employed in STEM occupations expect to leave their job within the next five years (Professional Australia, 2016). Issues that contribute to a failure to retain women in STEM employment are many and varied with income inequality rates as high as 20.2% reported in the Information, Communication and Technology industry as one example of the many contributing factors (Commonwealth of Australia, 2019). Given the systemic nature of the problem, efforts are needed at all levels to extend understanding of how women can be attracted to, and how they can be retained within STEM.

Examination of the evidence base indicates that numerous barriers have been identified, including underestimation of female STEM capability, sexism, negative biases and discouraging classroom experiences (Blackburn, 2017). Previous studies have identified that low self-efficacy and limited access to role models were key factors preventing females’ from persisting with STEM degrees (Fuesting and Diekman, 2016; Herrmann et al., 2016). Substantial efforts have been put into reversing this trend. Multiple factors driving female underrepresentation in STEM require appropriately focussed solutions through collaboration of knowledge from various disciplines (DeCoito, 2014; McDermott et al., 2005). The National Science Statement was released in 2017 emphasising the importance of STEM, making science a commitment and priority for the Australia Government, including advancing the talent pool of STEM fields in the country, by reducing the gender inequality gap (Commonwealth of Australia, 2019). A substantial pool of literature testifies to the significant work undertaken in STEM areas addressing the underrepresentation of females globally (Crawford et al., 2018; Drew et al., 2015; Tupper et al., 2010). A notable example can be seen in the MemphiSTEP programme conducted in the USA, which effectively increased female STEM retention from 18 to 86% of enrolled students (Windsor et al., 2015) through the methods of scholarship and mentoring programmes. The AIMS programme, delivered in the USA, aimed to graduate underrepresented minorities in STEM by offering a four-year undergraduate programme. The AIMS programme was reported to have achieved a higher graduation rate than control cohorts in a follow-up study (Gilmer, 2006).

While progress has been made in the field, ongoing attention is still needed to attract and retain females in STEM. To date, no application of social marketing to retain females in STEM in university higher education has been reported. Previously, social science scholars have opted to advance conceptual frameworks such as recommending the use of undergraduate socialisation models, input-environment-outcome models and offering transformative research perspectives to examine the institutional environment for females in STEM (Johnson, 2012). The role of social cognitive career theory to assess intentions to graduate from within the STEM discipline has also been considered (Soldner et al., 2012).

The field of social marketing is capable of addressing complex social problems. Moreover, application of social marketing to combat complex social problems (e.g. human trafficking and gender equality) has been called for (Truong, 2014) and observed (e.g. environmental conservation, sustainable consumption) (Truong et al., 2018). Application of social marketing within the STEM setting remains limited to understanding females’ preferences (Friedmann, 2018) with no evidence reporting the effectiveness of social marketing programme application. Application of the Co-create – Build – Engage (C-B-E) framework is outlined. The social marketing C-B-E process emphasises a female STEM student-centred approach. C-B-E ensures design, implementation and evaluation is centred on the people targeted for change starting with co-creation; in contrast to other frameworks which continue to promote expert-led design approaches (Dietrich et al., 2017). This is important given differences that are identified between user-designed programmes when contrasted to expert designed approaches (Rundle-Thiele et al., 2019 for one example highlighting group differences). This paper extends the evidence base aiming to demonstrate the capacity of social marketing to be applied to attract, retain and graduate females in STEM.

The contributions of this paper are threefold. First, this paper details stepwise application of the C-B-E framework. Through detailing the process, an actionable step-by-step guide that social marketing researchers and practitioners can apply to co-design, implement and evaluate social marketing programmes is offered. Second, insights from this study can inform future STEM programmes. Insights generated in this study can be used to guide future STEM retention programme designs to enhance effectiveness. Lastly, this paper adds to the limited peer reviewed literature focussed on attainment of measurable outcomes known to contribute to female retention in higher education institutions in the Australian context. This study delivers the first scientific paper reporting an outcome evaluation for a social marketing STEM retention programme conducted in a higher education institution in Australia.

Social marketing and the C-B-E process

Evidence indicates behaviour change is more likely when more social marketing principles are applied (Carins and Rundle-Thiele, 2014; Kubacki et al., 2015). Developed by Rundle-Thiele and colleagues (paper currently under review), the C-B-E framework offers a three-step process that ensures programme design occurs with key programme stakeholders and users (see Figure 1). A clear distinction between C-B-E and other marketing frameworks (Lee and Kotler, 2019; Mckenzie‐Mohr, 2000) is inclusion of co-design. Co-design ensures that programmes are built with (and not for) people (Trischler et al., 2019). C-B-E delivers an action framework that outlines the sequence in which the eight benchmark criteria outlined by the National Social Marketing Centre (NSMC) are applied when a programme is co-created and trialled. Ensuring core marketing principles such as exchange and utilisation of the full marketing mix, the C-B-E framework can deliver lasting change through the provision of programme, products and/or services that people value and are willing to pay for. By focussing effort on co-designing innovative solutions with people targeted for change (e.g. females currently enrolled in STEM programmes) user needs are identified and where possible met. Stakeholder views are gained to inform programme design and planning ensuring user-designed strategies are practical and feasible. Programmes are designed to engage the target audience from the outset, moving and motivating them to achieve the programme goals (e.g. stay enrolled in the STEM degree). The three C-B-E stages are briefly described next.

Co-creation is the first C-B-E step. Co-creation works with end-users of a product/service and sees programme users as co-developers who spend time creating ideas (Chan et al., 2010; Durugbo and Pawar, 2014; Yim et al., 2012). Previous campaign strategies seeking to retain female students in STEM programmes include female-only mentoring, conferences and seminars (Carver et al., 2017; Crawford et al., 2018). A literature review highlights that female students were not included in programme design (Carver et al., 2017; Doerschuk et al., 2016; Yoder and Murphy, 2012). By applying the co-creation process, people who are to be targeted (e.g. females enrolled in STEM degrees) can express their opinions and thoughts to inform programme design. Participants are empowered and co-creation ensures that females are not mere recipients, rather they are viewed as designers, programme architects and experts who can map their preferences to inform strategies to be implemented (Khalid and Helander, 2003). Consistent with other social marketing frameworks (Lee and Kotler, 2019; Mckenzie‐Mohr, 2000; NSMC) other formative research approaches are applied to gain insights to assist programme build. Formative research techniques used can include interviews with key stakeholders, literature reviews, surveys and more to gain knowledge of what has worked in the past, and what can practically be implemented (Rundle-Thiele et al., 2019 for full description of programme co-creation).

The next step in the C-B-E framework is Build. The Build step involves converting insights into a programme, product or service that can be implemented. The Build stage involves moving from strategy to implementation and covers the schedule of work needed to construct a programme for the first time or update programme components in light of strategic insights gained during co-creation. The culmination of the build step is a programme that is market ready and can deliver intended outcomes. Programmes built aim to deliver value to the target audience (e.g. females enrolled in STEM degrees) and are designed so that females voluntarily choose to engage with the programme built. Key social marketing principles that can be used to strategically guide the second step of the C-B-E framework is the marketing mix and exchange. In its most traditional form the marketing mix can be defined by four fundamental marketing strategies: product, price, place and promotion (Kotler et al., 1990). We note a large variety of marketing mix frameworks exist, with debate continuing in scholarly literature (Lahtinen et al., 2020). Full application of the marketing mix encourages build of priced programmes that people can willingly pay for. Many behavioural change programmes are criticised for being manipulative, and through delivery of volitional choice, social marketing is ideally placed to overcome weaknesses associated with other change approaches. By offering fee for service programmes, social marketing can clearly deliver solutions that people value, which in turn can achieve behavioural changes sought by funding bodies (see David et al., 2019 for one example).

The third step in the C-B-E framework is Engage. This final step consists of delivering the programme that has been built in market and continuously monitoring the programme to optimise outcomes. During the engagement phase, the key focus is on ensuring the target audience is aware of the programme and that tactics are in place to encourage trial and ideally repeated programme participation. During engagement, programme managers must carefully monitor progress and changes to implementation must be made rapidly to ensure that people know about and are trying the programme. During the engagement phase, programme team effort is centred on ensuring the programme is delivered in a fun and engaging way. Within the Engage phase, promotion plays a central role. Evaluation is considered as the final stage of the engagement phase to continue the C-B-E cycle.

Methodology

This section outlines how the mixed method study was applied within the social marketing C-B-E framework to design and evaluate the pilot programme. This mixed method study used a systematic literature review and a co-design study to co-create and build a pilot programme. A repeated measure survey was used to evaluate programme performance. For a complete overview of the C-B-E process with its respective mixed method research elements, see Figure 2. Methodological detail is first provided followed by a detailed explanation of C-B-E in the results section.

Co-creating the pilot programme

Systematic literature review: identifying strategies for inclusion in co-design (Co-create, Step 1)

A systematic literature review was conducted as the first step of the Co-create stage to draw from past experience in STEM retention programmes to learn what works. The systematic literature review sourced studies, reported in peer review literature; that had been implemented in a higher education context that aimed to recruit and retain women in STEM degrees. A key focus for the systematic literature review was identification of past programmes that had been implemented to gather sensitisation materials to be used in the co-design sessions. Sensitisation encourages creative thinking, and it nurtures idea generation (Hurley et al., 2018). The systematic literature review followed PRISMA guidelines (Moher et al., 2009) and widely used procedures in social marketing (Carins and Rundle-Thiele, 2014; Kim et al., 2019; Kubacki et al., 2015).

A total of six databases (EBSCO All Databases, Emerald, ProQuest All Databases, Ovid All Databases, Science Direct and Web of Science) were searched to locate studies, using keywords including, ‘STEM, female, gender*, Intervention, programme, recruit and retain. The search yielded 1,200 unique articles after duplicates (n = 398) were removed. Non-journal articles, formative and review studies, unrelated studies (i.e. not STEM, undergraduate or female focussed) and studies beyond the scope of the review (e.g. STEM-cell research rather than STEM discipline studies) were excluded (n = 1187). A backward and forward search revealed four (n = 4) more unique records, delivering a total of 17 eligible studies for the review. Four reviewers undertook screening of articles, data extraction and quality control, identifying programme strategies, assessing study quality and effectiveness of programmes in delivering increases in recruitment, retention and graduation likelihood for female STEM students in the higher education context.

Co-designing solutions with females currently enrolled in STEM 2D degrees (Co-create, Step 2)

Co-design was Phase 2 of the Co-create stage. Programme user insights were gained following the six-step co-design process which involves resourcing, planning, recruiting, sensitising, facilitating and evaluation (Dietrich et al., 2017). The co-design sessions incorporated stimuli identified in the systematic literature review. A total of seven co-design sessions were conducted across two university campuses of one Australian higher education provider using a convenience sample. A total of 47 female undergraduate and postgraduate students who were currently enrolled in a STEM degree participated in the seven co-design sessions. Each session had a minimum of two and a maximum of 11 participants. Undergraduate students were the majority (n = 40). For programme design, participants were randomly assigned into smaller groups allowing more intimate discussions to take place and reducing facilitator interaction during idea generation (Hurley et al., 2018). Each co-design session ran for 90 min and was audio recorded. The co-design sessions were subject to thematic analysis to identify new ideas for implementation. Following guidelines (Braun and Clarke, 2006; Patton, 1990) data analysis was undertaken at a semantic level. To increase data validity, this study followed five steps of thematic analysis (Braun and Clarke, 2006) and three researchers cross-checked data to increase reliability of analytical findings (Fereday and Muir-Cochrane, 2006).

Pilot programme evaluation

Evaluation of the Pilot Programme, guided by the RE-AIM framework, commenced prior to the Engage stage and concluded following delivery of the pilot programme. The RE-AIM framework consists of five dimensions, namely, Reach, Efficacy, Adoption, Implementation and Maintenance (Glasgow et al., 2001). Process evaluation included measures to examine participants’ overall satisfaction levels with the bias literacy workshops (David et al., 2019). Outcome evaluation consisted of a repeated-measure research design, ensuring that the same items and measures were asked at two different time points (pre- and post-pilot implementation) through online surveys. A convenience sampling method was used to recruit participants in the online baseline survey in November 2018 prior to implementation of the pilot programme. The post-survey was conducted in March 2019, after the programme was delivered. It was a prerequisite for workshop attendees to have completed the pre-survey prior to workshop commencement and post surveys were completed in person immediately following completion of their bias literacy workshop.

A bi-polar measure was used for self-efficacy. Items such as “I am confident I can continue studying in my chosen field through to graduation”, where −3 was “Strongly Disagree”, and 3 was “Strongly Agree”, captured self-efficacy. Self-efficacy items were sourced from (Rhodes and Courneya, 2003). The measurement scales were later converted to 1–7 in SPSS. The Cronbach’s alpha for the self-efficacy measurements were high (Alpha_pre=0.927 and Alpha_post=0.929). Demographic variables measured gender and age. One recall question was added to the post-survey questionnaire to measure advertising reach and awareness. In the recall question, respondents were asked to recall where they saw the pilot programme. Process evaluation questions measured the level of satisfaction with the bias literacy workshops and how likely participants were to attend and/or recommend the workshop to others (David et al., 2019). Incentives (win one of ten gift vouchers valued at AUD$50 or one of two iPads valued at AUD$383) were offered to increase survey participation. Survey participation was voluntary and anonymous, and participants could withdraw from the survey at any time. A total of 150 participants were matched from both pre and post surveys.

Results

Two studies were conducted during the co-creation stage, namely, a systematic literature review (Step 1), and co-design sessions (Step 2). Nine broad strategies functioning as inputs for the co-design sessions were identified in the systematic literature review. Strategies included Mentoring (n = 11), Financial Incentives aimed at females (n = 10), Preparatory Academic Service Support (n = 10), Seminars and Workshops (n = 9), Learning Communities (n = 9), Preparatory Courses (n = 8), Conferences (n = 6), Internships and Work Placement Opportunities (n = 4) and female exclusive STEM student social clubs (n = 3). Studies located ran from as little as four months to as much as 15 years, and featured sample sizes of n = 8 to as many as n = 1000. While the majority of interventions targeted female only (n = 10), many targeted Underrepresented Minorities. The majority (76 %) (n = 13) compared pre and post data between one or more cohort groups. All 17 studies identified reported positive outcomes for one or more identified outcome variables. Eight studies reported positive outcome effects for recruitment, three reported positive effects for retention, and six studies reported positive effects on graduation likelihood. All studies were developed and delivered in the USA. Idea templates depicting the nine broad strategies identified in the systematic literature review were created for the co-design workshops (see Figure 3).

Four key insights were gained to guide programme design. These were “stop stigmatising women”, “build my confidence”, “dysfunctional communication” and “role models”. Each insight is discussed in turn.

Stop stigmatising women

While gender equality is an issue, the participants revealed inclusion of both genders in any programme strategy is highly desired to avoid females from further standing out. Participants noted that all STEM programmes were female focussed, and they indicated this only served to stigmatise them further, and extend rather than close the gender gap. Participants enrolled in STEM degrees at university felt that excluding males creates barriers that in turn prevents females from entering and staying employed in industry. Participants viewed inclusion of both genders as an opportunity to broaden their network, collaborative partnerships and learning. However, the participants acknowledged gender disparity continues to be prevalent, and therefore, efforts centred on increasing awareness of gender biases must be a key priority for all:

If you’re trying to get females into science it doesn’t do good if you separate females from males because it’s just going to create that barrier in the future, as well (Session 6, response 80).

Build my confidence

A lack of confidence was identified in co-design workshops. Participants identified a lack of confidence as an issue that needed to be addressed through university support services:

(…) I know that confidence can be an issue with certain things and sometimes it can be tricky to speak to your tutors or teachers and things like this (Session 7, response 40).

Further, participants reported currently or previously experiencing a lack of confidence in themselves and in others, that led to difficulties in maintaining academic performance:

(…) crisis of confidence: “I don’t think I’m good enough.” If your academic ability, as a relation to these factors, isn’t that good, you might think, “I’m actually not that good.” There’s no relatable aspect to it (Session 7, response 276).

Dysfunctional communication

Dysfunctional communication was another theme prevalent in the co-design sessions. Communications refers to the distribution of messages, timing and message framing. Taken together these affect the message outcome resulting in lack of awareness and engagement in the various strategies offered by the university. The co-design session identified that while strategies were already in place, many of the participants missed out on these opportunities due to lack of communication:

(…) there are so many scholarships, and mentoring, and network opportunities that don’t ever seem to really get communicated down to students. It is more just an information channel (Session 5, response 168).

Role models

The importance of appropriate role models was identified as a key area for action in co-design workshops. At various times, participants noted their own preferences and experiences when considering female experts/industry leaders within STEM disciplines:

What I have done, and I think it’s actually been more of a conscious decision rather than a subconscious decision, but I’ve always found myself being drawn towards female scientists that are at the head of their game (Session 1, response 66).

Participants stated that creating a campus environment to better support and motivate female STEM undergraduates would not only facilitate more active peer-to-peer engagement within their STEM field but equally would also assist to deliver additional role models.

Building the pilot programme

Key insights arising from co-design informed pilot programme build. The programme, which was to be implemented over a one-semester period, focussed on delivering something of value for female STEM students who were currently enrolled in degrees in one Australian university drawing on currently available offerings where possible. Specifically, insights gained in co-design indicated that increasing female students’ self-efficacy would be most highly valued. Co-creation insights revealed an opportunity to increase the confidence of the target audience through increased awareness and visibility of female role models in the STEM industry and to ensure increased awareness of support services currently provided by the university. The programme to be built consisted of two major components: delivery of bias literacy workshops, and a communication campaign designed to increase awareness of currently available university services. The programme is detailed next using the 4P marketing mix framework.

Product – bias workshops (NEW) and existing university services

Bias Literacy Workshops were carefully designed drawing from insights gained from the co-create phase. Workshops aimed to increase female undergraduate students’ self-efficacy to continue studying in their STEM degrees and expose female students to a STEM role model while addressing gender bias. This was undertaken with the careful selection of the speaker. A collaborative process followed ensuring the speaker developed and delivered the bias literacy workshops in line with feedback gained from co-design sessions. The pilot programme was open to all STEM students. There were no restrictions to participation (for example, gender, Grade Point Average (GPA)) and the workshop tone was casual.

Existing university services to support STEM students in the university were identified by the project team for promotion. Websites and supporting services identified included Scholarships, Peer Assisted Study Sessions, Work Integrated Learning Programmes and Services Offering opportunities to further enrich studies (e.g. improve employability), Mentoring, Bridging and Short Courses, International Student Support and Social Clubs. Raising awareness for currently available services formed the second component of the pilot programme, given low or no awareness for the broad range of support services that were available to support female STEM students.

People

Relevant potential speakers for the Bias Literacy Workshops were identified and shortlisted by the project team. Selection criteria included being a successful female and an aspirational professional from the STEM area that could easily be seen as a role model. The speaker offered one example of what the students had the potential to achieve in their chosen field of research. The speaker was selected based on her substantial track record of experience in technology, her continuing passion of engaging individuals in STEM fields (particularly females), and her desire to change industry structures and beliefs through illuminating prevalent biases.

Place

Four Bias Literacy Workshops were delivered at two main university campuses on different days and at different times to make it convenient for students to attend. The workshops were delivered in the mornings and in the afternoons during one semester to further accommodate students’ schedules. Additionally, to ensure a good attendance rate, engagement and to maximise reach, the workshops were offered for a three-hour duration. This ensured that the students felt they could attend the workshop without compromising their other study commitments.

Promotion

Social advertisements were delivered via electronic direct email to STEM university students during the engage phase. The artwork and content were strategically designed to subtly target female students, while not excluding other students (e.g. males). The advertisements included tailored messages focussing on self-efficacy and graphic designs presented a pink brain on a dark burgundy background featuring embedded hyperlinks (see Figure 4). The hyperlink led to a website with information about the Bias Literacy Workshop and the support services currently offered by the university for STEM students.

Engage

The pilot programme engaged the target audience through multiple communication channels including in person promotion, electronic direct mail, posters, flyers and social media.

Bias Literacy Workshop

The Bias Literacy Workshops were promoted via online and face-to-face channels. Outdoor posters, flyers and an email campaign were used. Emails promoting the workshops were sent to a mailing list of 9,438 STEM students who were currently enrolled in the university. The workshops were also promoted on campus via printed posters, and during market days featuring a stall, where STEM students volunteered to promote. The content of the posters emphasised the aim of the bias literacy workshops, the practicality of the topic to students, and highlighted the workshop speaker’s expertise. Words like “bias literacy” were deliberately omitted in promotional material to avoid negative connotations for some individuals, which in turn could impact workshop registrations. Workshop registrations were offered in two modes, namely, online and offline. A QR code and website link was incorporated into printed materials and a hyperlink was featured in online materials. To allow close interaction between the bias literacy workshop speaker and participants, spaces were provided for 30 participants per workshop. Across the four workshops, 127 participants registered. A total of 47 STEM students participated in the workshops.

The social advertising communication campaign aimed to raise awareness of existing university support services (see Figure 4). As previously outlined, advertisements used self-efficacy messaging, and embedded hyperlinks to websites ensuring students could locate key support services.

Social advertisements were sent to STEM students via the provided email list (n = 9,438) at two points in time across a two-week interval. We compared the social advertising group with the Bias Literacy Workshop group. Repeated exposure aimed to increase the exposure/open rate of the emails and message recall, without overwhelming students email inboxes.

Results

For the Bias Literacy Workshop evaluation, a total of 47 participates enrolled and participated in the evaluation. Approximately 60% of the participants were females and participants were typically aged between 18 and 25 years old. The process evaluation indicated that 42 participants (89%) of the participants were satisfied with workshop, 44 participants (94%) demonstrated their willingness to attend a similar workshop in the future, and 38 participants (81%) stated that they are likely to recommend the workshop to their peers. Evaluation of the social advertisement component of the pilot programme indicated that a total of 6,008 students opened the email. Approximately 70% of the participants identified as Caucasian, followed by 20% as Asian. When analysing the website traffic for all communication campaign emails, Social Clubs was the most commonly opened link (n = 605) followed by the STEM Project Website (n = 443), Scholarships (n = 193), the University Sciences Work Integrated Learning Programme (n = 162) and the Bridging Short Courses (n = 122). Many participants returned to emails with multiple opens, with some participants clicking on available links up to 13 times.

A total of 150 participants completed both the pre and the post surveys (see Figure 5) – control group – no programme advertisement, (n = 33); social advertisement exposure only, (n = 72) and Bias Literacy Workshop and social advertisement exposure, (n = 45).

The paired sample t-tests indicated the control group did not significantly change their self-efficacy (Mean difference = 0.16, p = 0.256), whereas participants attending the Bias Literacy Workshop reported significant increases in self-efficacy (Mean difference = 0.29, p < 0.01). Participants who were exposed to the social advertisements only did not report changes in self-efficacy (Mean difference = 0.04; p = 0.606), see Tables 1 and 2. Partial eta squared indicated a small effect for participants who were exposed to social advertising and who participated in a bias literacy workshop (F = 2.462, partial eta squared = 0.17).

An overview of the outcomes of the pilot programme using the RE-AIM framework is provided (see Table 3). The aim of this social marketing pilot programme was to increase self-efficacy (level of confidence to continue studying their chosen field) and to deliver additional social support through active workshop participation to reinforce the intentions of female STEM undergraduates to remain enrolled through to graduation. Bias Literacy Workshop was effective in improving self-efficacy and while the social advertisement campaign successfully increased awareness of existing support services with unique open rates (n = 6,000 and more than n = 1,000 click throughs) results indicate that active participation is needed to deliver increases in self-efficacy.

Discussion

This paper provides three contributions. First, this paper details application of the C-B-E framework. Second, this paper provides insights that can inform future STEM programmes worldwide. Finally, this study delivers the first scientific outcome evaluation for a pilot Australian STEM programme.

Applying co-create, build and engage

Few social marketing frameworks offer clear practical guidance demonstrating procedurally how programmes can be co-designed. This paper applies the novel three step C-B-E process co-creating a gender-neutral pilot programme that engaged STEM students, both males and females. C-B-E provides a clear, actionable step-by-step guide that can be followed to co-design, implement and evaluate social marketing programmes. C-B-E features stakeholder orientation ensuring unique insights (e.g. gender neutral) are gained and programmes that the target audience value are built (e.g. >90% of workshop participants were satisfied and would attend again). The C-B-E process invited the target audience to take part in the design process and it drew from past programme experience, which is consistent with calls to share responsibility to address wicked problems (Buyucek et al., 2016). The build stage incorporated insights from the co-creation stage (e.g. provide clear role models and avoid a female only emphasis) and used the marketing mix to actualise the programme. The pilot programme (product) was specifically designed for female students’ needs, while appealing to both genders (e.g. 40% of programme attendees were male). More than 60% of the student STEM population were reached and programme evaluation gauged the extent that interaction with social advertising and workshop participation delivered desired programme outcomes.

Retaining females in Science, Technology, Engineering and Mathematics degrees

Given the systemic nature of the problem, efforts are needed at all levels to extend understanding of how females can be attracted to, and how they can be retained within STEM. Insights gained in this study deliver unique contributions to STEM retention literature. For example, females do not wish to be singled out in attempts to attract and retain them. Both males and females seek to understand what they can do to reduce the gender imbalance. In the current study, males indicated they seek male role models to provide clear examples of the set of actions that can be taken to increase female participation in STEM. Females indicated they did not wish to be singled out, and they sought opportunities to learn how to navigate male dominated environments rather than female only events featuring female speakers. Additionally, this study provided further evidence to support the importance of providing access to role models and for the role self-efficacy in retaining females in STEM university programmes (Fuesting and Diekman, 2016; Herrmann et al., 2016).

Following insights gained during the co-creation phase, the pilot programme aimed to increase self-efficacy. Conducted in one Australian higher education setting, this is the first scientifically reported Australian study aiming to retain women in STEM degrees. Key learnings were participation in the Bias Literacy Workshop increased female students’ self-efficacy to continue studying in their enrolled STEM degree. Confidence is a key factor that has been associated with females continuing studying and working in their STEM field (Fuesting and Diekman, 2016; Herrmann et al., 2016). As efficacy is constructed at an individual level, most identified factors are internal drivers. However, when it comes to female STEM undergraduates’ retention, external factors including stereotypes underestimating female capability, biases and sexism, pessimistic campus culture, discouraging classroom experiences and lived experiences have a crucial role in determining perceived self-efficacy (Blackburn, 2017). Examination of click throughs to advertised support services indicates students’ interest in the support services that are already offered by the university. The perceptions of institutional climate have been found to have an effect of performance ability, often referred to as a “chilly” climate (Settles et al., 2016) referring to individuals not being treated equally within an environment (Blackburn, 2017). Significant correlations have been found to impact university climate and sense of belonging for underrepresented groups (Johnson, 2012). Awareness of the range of services available is a necessary pre-condition for service access that in turn can deliver the climate warmth needed for STEM retention.

Management implications

The pilot programme was delivered in a short time frame with a small budget. Important learnings were gained that can inform future projects. First, while considerable reach was achieved, the capacity for programme elements (e.g. communication and participation in one workshop) to achieve large effects were limited owing to the nature and duration of pilot testing (e.g. exposure to one message and/or participation in one workshop). The workshop was well received and feedback from participants indicated that workshops with other themes should be delivered. Feedback indicated that workshops delivered by successful female professionals from STEM fields should be implemented to further extend role models. Other workshops could be presented by males who can outline the role they have played in increasing female participation in STEM. As the issue around gender inequality requires a holistic and long-term approach to increase students’ confidence to complete the degree, continued availability and promotion of support services is recommended. Student testimonials are recommended to outline the role services have provided in delivering the support sought to create warm, and not chilly climates, for female STEM students.

Limitations and future research

This pilot programme was limited to an individual approach (e.g. students enrolled in STEM degrees), which overlooks social and other external factors. The discipline of social marketing has been calling for systems applications (Domegan et al., 2016). Given that the issue of female underrepresentation in STEM is multi-faceted (Blackburn, 2017) to assure delivery of lasting behavioural outcomes, programmes can benefit from inclusion of a broader stakeholder cohort (e.g. Faculty members, policy makers) in programme design. The pilot project was designed and implemented in a short timeframe of 12 weeks, limiting the lead time available to promote and recruit participants for the bias literacy workshops and this reduced programme performance. Similarly, the timing of the pilot coincided with the peak summer University holiday period for students, potentially further limiting pilot programme reach. Moving forward, inclusion of bias literacy in core courses is recommended to enhance self-efficacy for the female student cohort. As the issue around gender inequality requires a holistic and long-term approach to increase students’ confidence to complete the degree, continued availability and promotion of support services (including the piloted Bias Literacy Workshops) is recommended. Student testimonials are recommended to outline the role that currently available university support services have provided for students. Additionally, research to examine the customer experience for students using the support services is recommended to ensure that word of mouth is activated following positive experiences with the support services aiming to retain females in STEM degrees.

Further limitations arise in this study. The timeframe for this project (three months) was too short to permit measurement of actual drop-out rates, and this represents an opportunity for future research. Moving forward, a four- to five-year project is recommended to track students from first enrolment to completion delivering a comprehensive programme over the course of the degree. A convenience sample informed the repeated measure outcome evaluation and the research team was not blinded to the trial data. In future, research that randomly selects students for the evaluation and incentivises completion by students is recommended to improve attrition rates and reduce any biases that occur with convenience samples. The co-design sessions were conducted with female STEM students only, which potentially limited the comprehensiveness of the insights generated due to sampling biases. Including other samples in the formative research stage such as male STEM students and female students who have previously dropped out of STEM should be considered in future research to overcome this limitation. Similarly, study participants freely chose to participate in the pilot programme. Pre-survey results showed a high self-efficacy indicating their commitment to remain in their degrees through to graduation. These results indicate that considerable self-selection bias exists within the pilot study. A consequent limitation is this pilot programme may be preaching to the converted (e.g. females seeking to stay enrolled) and potentially this pilot programme may not have appealed to females who are at risk of dropping out. This is an important area for future research to determine if programme appeal varied for different female cohorts. A further limitation of this study may be survivorship bias, and this should also be considered in future research.

Figures

Figure 1.

The C-B-E process

Figure 2.

The CBE process applied

Figure 3.

Co-design workshop idea template

Figure 4.

Social advertisement example

Figure 5.

Overview of data collection

Table 1.

Mean of self-efficacy

Groups		Mean	Std. deviation	Std. error mean
Control group (N = 33)	PRE	6.27	0.71	0.12
	POST	6.42	0.71	0.12
Exposed to social advertising only (N = 72)	PRE	6.18	1.13	0.13
	POST	6.22	1.03	0.12
Bias literacy workshop participants (N = 45)	PRE	6.32	0.72	0.11
	POST	6.61	0.53	0.08

Table 2.

T-tests

Groups	Paired differences					t	df	Sig. (2-tailed)
	Mean	SD	Std. error mean	95% confidence interval of the difference
				Lower	Upper
Control group	−0.16	0.79	0.14	−0.44	0.12	−1.157	32	0.256
Exposed to social advertising only	−0.04	0.68	0.08	−0.20	0.12	−0.519	71	0.606
Bias literacy workshop participants	−0.29	0.58	0.09	−0.46	−0.11	−3.348	44	0.002

Table 3.

RE-AIM dimension

RE-AIM dimension	Outcomes
Reach	127 Bias Literacy Workshop programme registrations More than 200 double-sided flyers distributed promoting the Bias Literacy Workshops 3,561 people reached via email marketing for the Bias Literacy Workshop promotion 6,008 unique open rates for the “Communication Campaign” providing awareness of services currently available to support STEM students More than 1,000 click throughs to one of four existing support services webpages
Effectiveness	The Bias Literacy Workshops were effective in increasing self-efficacy and perceived university climate The social advertisements increased awareness of existing support services
Adoption	7 social clubs and 8 lecturers voluntarily assisted promotion of the Bias Literacy Workshops Partnered speaker showed high intention to participate again in the future 47 students participated in 4 Bias Literacy Workshops The students particularly sought social clubs and scholarship offerings
Implementation	Over 90% of the Bias Literacy Workshop participants were satisfied with the workshop 80% of the Bias Literacy Workshop participants stated that they are likely to recommend the workshop to university peers
Maintenance	93% of the participants reported willingness to attend a similar workshop in the future after Bias Literacy Workshop