Abstract
Purpose
Involvement of parents/carers may increase effectiveness of primary school-based alcohol-misuse prevention projects through strengthening family-based protective factors, but rates of parental engagement are typically low. This paper reports findings from an exploratory trial of a school-based prevention intervention – Kids, Adults Together (KAT), based on the Social Development Model, which aimed to promote pro-social family communication in order to prevent alcohol misuse, and incorporated strategies to engage parents/carers. The purpose of this paper is to assess the feasibility and value of conducting an effectiveness trial of KAT.
Design/methodology/approach
The study was a parallel-group cluster randomised exploratory trial with an embedded process evaluation. The study took place in south Wales, UK, and involved nine primary schools, 367 pupils in Years 5/6 (aged 9-11 years) and their parents/carers and teachers. Questionnaires were completed by pupils at baseline and four month follow-up, and by parents at six month follow-up.
Findings
Overall KAT was delivered with good fidelity, but two of five intervention schools withdrew from the study without completing implementation. In total, 50 per cent of eligible parents participated in the intervention, and KAT had good acceptability among pupils, parents and teachers. However, a number of “progression to effectiveness trial” criteria were not met. Intermediate outcomes on family communication (hypothesised to prevent alcohol misuse) showed insufficient evidence of an intervention effect. Difficulties were encountered in identifying age appropriate outcome measures for primary school-age children, particularly in relation to family communication processes. The study was unable to find comprehensive methodological guidance on exploratory trials.
Research limitations/implications
It would not be appropriate to conduct an effectiveness trial as key progression criteria relating to intervention and trial feasibility were not met. There is a need for new measures of family communication which are suitable for primary school-age children, and more guidance on the design and conduct of exploratory/feasibility trials.
Originality/value
KAT achieved high rates of parental involvement, and its theoretical framework and processes could be adapted by other interventions which experience difficulties with recruitment of parents/carers.
Keywords
Citation
Segrott, J., Rothwell, H., Pignatelli, I., Playle, R., Hewitt, G., Huang, C., Murphy, S., Hickman, M., Reed, H. and Moore, L. (2016), "Exploratory trial of a school-based alcohol prevention intervention with a family component: Implications for implementation", Health Education, Vol. 116 No. 4, pp. 410-431. https://doi.org/10.1108/HE-01-2015-0005
Publisher
:Emerald Group Publishing Limited
Copyright © 2016, Authors. Published by Emerald Group Publishing Limited. This work is published under the Creative Commons Attribution (CC BY 3.0) Licence. Anyone may reproduce, distribute, translate and create derivative works of the article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licenses/by/3.0/legalcode .
Introduction
Alcohol misuse has high personal, social and economic costs and widens health inequalities (Rehm et al., 2009). Misuse of alcohol by children and young people raises particular concerns about the number who initiate alcohol consumption at a young age, and high levels of regular and harmful alcohol use (Bellis et al., 2007; Smith and Foxcroft, 2009). It can have multiple health and social impacts, including: disorderly and violent behaviour; risky sexual behaviour (Arria et al., 2008); accidental injury; and poor school attendance and achievement (The Alcohol Harm Reduction Group, 2003; Wynn et al., 1997). Early initiation of alcohol consumption increases the risk of alcohol-related problems in later life (Hawkins et al., 1997; Moffitt, 1993; Ward and Snow, 2008), therefore developing effective primary prevention interventions is a public health priority.
Schools are important settings for delivering prevention interventions. They have high reach among the target population (Kirby, 2002; Storr et al., 2002), a longstanding role in health education (Young, 2005) and an expanding function as health promoting institutions (Macnab et al., 2014; Stewart et al., 2004). When school-based programmes involve parents/carers they have the potential to strengthen dimensions of family functioning such as parenting which are protective factors for later alcohol misuse by young people (Petrie et al., 2007; Velleman et al., 2005). Consequently, children and young people’s experiences outside school may become more consistent with what they learn in the classroom (Flay, 2000), for example in the rules parents set around drinking (Rowland et al., 2013).
There is a clear theoretical case for addressing “early antecedents” of substance misuse to prevent or delay its commencement (Storr et al., 2002) – including parenting protective factors, during pre- and early adolescence whilst parents are a key point of socialisation (Catalano and Hawkins, 1996; Spoth et al., 2005). However, there is insufficient empirical evidence to inform the design of early prevention interventions, particularly in countries outside the USA (Foxcroft and Tsertsvadze, 2012). While there is evidence that several programme components are needed and that components work together to increase effectiveness (e.g. Nation et al., 2003), the large number and variety of programmes, and the rarity of long-term outcome evaluation and systematic adaptation, complicate efforts to understand cause and effect (Foxcroft and Tsertsvadze, 2012). Designing primary school-based studies is particularly challenging because relatively few interventions have been based in primary schools (Lloyd et al., 2000; Spoth et al., 2008). Another challenge is the recruitment and retention of parents (Axford et al., 2012; Garvey et al., 2006; Garcia-Dominic et al., 2010), even when interventions are modified to increase levels of parental involvement (Stead et al., 2007; Kaminski et al., 2002), and more evidence is needed on effective strategies.
Methodological guidance on evaluating complex health interventions (e.g. Craig et al., 2008) emphasises the importance of an exploratory/feasibility evaluation phase to inform decisions about whether to invest the significant resources needed for a randomised controlled trial (RCT) or other study types designed to assess effectiveness. Exploratory/feasibility trials provide information on key parameters of trial design, including: feasibility of intervention implementation; likely rates of recruitment and retention of schools and individuals; potential effect sizes; sample size requirements, and their achievability; and the cost, acceptability and workability of data collection methods. However, there is little guidance on how to design and conduct exploratory trials, including how to develop and assess criteria for progression to an effectiveness trial.
In this paper we report the findings from an exploratory trial of a primary school-based alcohol misuse prevention intervention with a family component – Kids, Adults Together (KAT). A non-controlled study (Rothwell and Segrott, 2011) found that KAT achieved high rates of family engagement and acceptability which could be explained by the Social Development Model (SDM) (Catalano and Hawkins, 1996), and that there was the potential to evaluate KAT’s effectiveness using an RCT design. The aim of the current study was to assess the value and feasibility of conducting a cluster RCT of KAT. Its objectives were to: first, assess intervention feasibility and acceptability across school contexts; second, establish intervention participation rates and reach, including equality of engagement across socio-economic groups and localities; third, refine the intervention theory; fourth, assess trial recruitment and retention rates; fifth, determine the feasibility of methods for measurement of primary and secondary outcomes; sixth, identify potential effect sizes and an appropriate sample size needed to detect these; and seventh, identify optimal delivery structures and systems for the KAT programme.
Methods
The intervention
KAT was originally developed by a consortium of health/education agencies led by the Gwent Police Force in Southeast Wales, UK, who wished to reduce alcohol misuse by young people, and identified that parents’ drinking and parenting practices were important influences on children’s behaviours. The programme – adapted from the Australian Parents, Adults and Kids Together Programme, was modified to focus specifically on alcohol (rather than multiple substances), and a Digital Versatile Disc (DVD) component was added (Rothwell and Segrott, 2011). In line with the SDM (Catalano and Hawkins, 1996), KAT aimed to prevent alcohol misuse by strengthening attachments between children and pro-social others (particularly family members). This was hypothesised to occur through creating perceived and actual opportunities for pro-social family communication, rewarding and reinforcing children’s participation in the programme, and cultivating communication and other skills needed to give “skilled performances”, such as how to work on interactive tasks in class (Catalano and Hawkins, 1996). KAT’s logic model is displayed in Figure 1.
KAT had three components. First, interactive classroom work was delivered by teachers on health aspects of alcohol consumption, and displays and other materials were prepared for a family event. Most of the work was done through pair/group work to strengthen pupils’ communication skills. Teachers were asked to create opportunities for pupils to take some of the classroom work home to generate interest among family members. Second, the family event was delivered in school. This incorporated a presentation by pupils on the work they had done in class, displays of their work, and opportunities for children and their parents to work together (e.g. during quizzes). Schools were asked to ensure that all pupils were involved in delivering the family evening. Third, at the end of the family event pupils were given a “goody” bag to take home, containing fun items, information leaflets and an educational DVD for families to watch together.
KAT required approximately one week’s total classroom contact time (20 hours), with flexibility to spread the work over a longer period. Staff training was provided by a health education consultant and a teachers’ handbook, which set out the programme’s aims, learning outcomes and links to the curriculum for each lesson/activity and included supporting information such as a timetable for the family event. Staff were also given: a PowerPoint presentation for use at the family event; appropriate information from websites, downloaded and photocopied; and Tacade’s “Keys to Alcohol” resource (Advisory Council on Alcohol and Drug Education (TACADE), 2010) The four schools randomised to the control group did not receive KAT, but continued with their usual activities, including classroom work/school activities on alcohol.
Research design
The study comprised an exploratory cluster RCT, with data collected from pupils via written questionnaire and from parents/carers via telephone interviews. A process evaluation was embedded within the trial and utilised a mixed methods approach. Methods used were: observation of programme delivery; focus groups with participating pupils; and interviews with teachers and parents/carers.
Recruitment
All English language-medium primary schools (n=39) in the local authority area chosen for the trial (a city in Southeast Wales, UK) with Year 5/6 classes were invited by letter to participate. Phone calls were then made to each school until eight schools agreed to participate. One intervention school withdrew from the study before baseline data collection. Because this was an exploratory trial (concerned with testing feasibility), we wrote again to all eligible schools not already participating in the trial (n=29) inviting them to participate as a replacement intervention school. One school responded and agreed to participate, but subsequently withdrew from the trial after baseline data collection had taken place, shortly after programme delivery had commenced.
Consent from head teachers for school participation was obtained before randomisation, and consent from children and parent participants sought after randomisation. The study tested the feasibility and acceptability to schools of “opt-out” parental consent for children’s involvement in the trial. No school raised objections, and this method was therefore used in all participating schools. Ethical approval was given by the Cardiff School of Social Sciences Research Ethics Committee (reference SREC/697). The trial protocol is available at www.nets.nihr.ac.uk/projects/phr/10300203 (ISRCTN80672127).
Data protection regulations precluded research team access to names or contact details of parents/carers via schools, so a letter and information sheet from the principal investigator were posted by schools to parents/carers of pupils in the classes participating in the trial. The letter asked parents/carers to return a reply slip to the research team if they were interested in participating in the research, and to contact the school if they did not wish their child(ren) to participate.
Randomisation
Schools were stratified by size and free school meal (FSM) entitlement – which was used as a proxy for levels of social deprivation. These variables were used to balance the randomisation. Optimal allocation was used to determine the randomisation sequence. Here a balance algorithm was used to provide a pre-defined sequence and all schools were randomised jointly (Raab and Butcher, 2001; Carter and Hood, 2008). The method was implemented in R statistical software in the South East Wales Trials Unit (SEWTU) and the allocation was concealed until after recruitment and the start of the intervention. An independent statistician within SEWTU assigned schools to the intervention arm.
Measures
Details of measures used and the rationale for their inclusion are summarised in Table I. Measures for children were piloted. Potential primary outcomes for a future effectiveness trial (all collected from children) were included in questionnaires administered to trial participants to test their feasibility and acceptability. These comprised drinking initiation using a modified question from the Survey of Smoking, Drinking and Drug Use Among Schoolchildren in England in 2008 (Fuller, 2009), and measures of past month alcohol consumption and drunkenness frequency based on the 2009 Health Behaviour in School-Aged Children (HBSC) survey (Griebler et al., 2010) (completed by 11-15 year olds), with some modifications made to the wording and response categories at follow-up to optimise comprehensibility for 9-11 year old participants. One question on “ever” drunkenness – also adapted from the HBSC survey – was included. It was not possible (due to resource constraints) or appropriate (given the study’s focus on feasibility rather than measuring effectiveness) to include long-term follow-up of pupils. Data on past month frequency of drinking and drunkenness from the 2009 HBSC study in Wales were used to estimate prevalence of drinking in the 11-13 age group as a basis for estimation of the sample size required for a potential future effectiveness trial, which would measure long-term behavioural outcomes.
Hypothesised intermediate outcomes of KAT were assessed to determine their feasibility and acceptability to participants, and identify potential effect sizes within any future effectiveness trial. Paucity of suitable measures of family communication for children aged 9-11 years in the UK necessitated adaptation of measures originally designed for older age groups or other countries. Measures used were: the KIDSCREEN-52 subscale on parent relations and home life (The KIDSCREEN Group Europe, 2006; Ravens-Sieberer et al., 2005); the family activity scale (Griebler et al., 2010); and the family communication scale (Olson and Barnes, 2012). The targeted parent-child communication about alcohol scale (Miller-Day and Kam, 2010) was modified and used at baseline to examine alcohol-related communication. However, participants found the questions difficult to understand, and it was replaced at follow-up with the parent-child communication scale (Ennett et al., 2001). Measures of gender, age, ethnicity, socio-economic status and for parents, of qualifications and employment were used to assess their acceptability to participants and comparability between intervention and control groups. For children, the Family Affluence Scale was used as a measure of socio-economic status (Boyce et al., 2006; Currie et al., 2008).
Data collection
Written questionnaires were completed at baseline and four month follow-up by children at school under the supervision of the research team. Researchers assisted any participants who had difficulty in understanding questions. Pupils who were absent from school or did not want to complete the questionnaire at baseline were allowed to complete follow-up questionnaires. Parents/carers participated in one telephone interview approximately six months post-baseline. Data from questionnaires/interviews were encrypted and stored in anonymised form, using participant identification numbers. The follow-up interval for pupils was shortened (originally six months) so that follow-up data collection could take place before Year 6 pupils moved to secondary schools. A short follow-up interval was sufficient to provide an initial estimate of likely effects on proximal outcomes, and to test the feasibility of measures and data collection procedures.
Criteria for progression to an effectiveness trial were developed, with input from a stakeholder group of policymakers and health/education agencies, and approved by the trial’s Independent Steering Committee. Based on the study’s objectives, these criteria concerned: the feasibility and implementation of KAT; acceptability of the intervention to parents/carers, pupils and teachers; rates of intervention participation and reach across socio-economic groups; intervention theory/processes; trial recruitment/retention rates; feasibility of potential primary and secondary outcome measures; the sample size needed for an effectiveness trial; and the existence of delivery structures and systems for the KAT programme as part of an effectiveness trial. The progression criteria (and the means of their assessment) are detailed in Tables II and III.
Assessment of reliability of questionnaire measures was undertaken using Cronbach’s α for internal consistency and factor analysis for uni-dimensionality. Summed scores were created for each outcome scale. With previously validated scales, the outcome was used as directed in the manuals or scale references – either categorical (using validated cut-offs) or as a continuous score. Where no guidance was given, the continuous data were used in the primary analysis. Secondary outcome measures adapted for KAT were validated using baseline or follow-up data, depending on the timepoints at which they were administered.
Analysis
All analyses were on an intention to treat basis using all randomised participants in the groups to which they were randomised, regardless of the intervention received, using complete data for baseline and follow-up. However missing data were not imputed so all analyses were carried out on complete cases. In intervention schools all trial participants’ data were included in the analyses, regardless of the extent of their engagement with the KAT programme. Outcomes were analysed using two-level generalised logistic models. Responses from pupils were nested within schools using models adjusting for baseline data where appropriate. The distribution of frequency of drinking and drunkenness in the last month was examined and it was determined that these ordinal categorical data were most appropriately collapsed into binary outcomes due to sparse data in higher categories. Covariates included in the models were those that were used to balance the randomisation (school size and FSM entitlement). As well as any differences between trial arms, estimates of intra-cluster correlations at school level were reported. Rates of missing data were examined to determine if they were related to children’s age or the stage of data collection.
Process evaluation
A process evaluation assessed intervention fidelity, reach and receipt by participants (Steckler and Linnan, 2004). A researcher observed KAT classroom delivery and family events using a structured schedule (Spradley, 1980) to describe activities, pupil engagement and contextual factors (e.g. the classroom environment, and numbers of pupils/staff present). A total of 11 hours of observation took place in school 4 (S4) (over eight observations), seven hours in S3 (four observations) and six hours in S6 (three observations), accounting for approximately 10 per cent of the total delivery hours. Four focus groups with children explored receipt, acceptability and intervention processes (two focus groups with four children in each group at S3, and one focus group with six children each at S4 and S6). Pupils had participated in KAT as part of their class group through interactive activities, and a key rationale for the use of a focus group approach was that it would enable us to explore the interactions between, and shared experiences, of pupils (Kitzinger, 1994). Participants were pupils who expressed an interest in taking part in a focus group at initial recruitment into the study. Interviews with 17 parents (seven at S4, three of whom were fathers, four at S3 and six at S6) examined receipt, acceptability and intervention processes. Interviews with nine members of teaching staff (including three head teachers) explored their experiences of delivering KAT, school contextual influences, acceptability and implementation. Interviews with head teachers in control schools asked about usual practice in delivering education about alcohol. Interviews adopted a semi-structured approach whereby the interviewer developed a framework with key themes or issues to be covered, but within which the ordering of the questions could be changed to suit the direction of the interview, and additional questions could emerge (Corbetta, 2003). Thematic data analysis (Bowling, 1994; Braun and Clarke, 2006) used the study objectives and data collection schedules as an analytical framework, which was modified as new themes were identified in the data. Interview, focus group and observation data were compared to identify thematic similarities and differences.
Findings
Intervention feasibility
Table IV provides details of the schools which took part in the trial, including numbers of pupil participants. Three intervention schools completed programme delivery. Observation data analysis indicated that the majority of programme activities were implemented with good fidelity in all three schools. Teachers leading the classwork followed the handbook closely, including suggestions for assessing children’s knowledge, and the use of group/pair work, and all classes prepared work for display/performance at the family event, including posters. However other classwork guidance was less well implemented. Personal invitations for parents to attend the family event were produced at only one school and school policies or existing practices contributed to low fidelity to guidance to set homework. Observation of family events at all three schools, and data from interviews and focus groups confirmed that, in accordance with guidance, they demonstrated a non-judgmental style and promoted a healthy approach to alcohol use. There was less success in involving all children in delivering the event, with two of the schools not appearing to have followed the handbook guidance.
Training was provided for teachers in intervention schools but some in school 6 did not attend training sessions due to pressure of other work, or because it was impossible to bring together relevant staff at one time. Consequently the Year 6 teacher and Year 5 deputy head did not appear to be familiar with the contents of the handbook or other information they would have received through training, and said in interviews that they wanted more support and detail regarding the family evening (school 6 Year 4/5 teacher; school 6 Year 6 teacher; school 6 deputy head). Observation data indicated that although school 6 staff arranged the date and venue for the family event and contacted parents, they did not get the hall ready or take charge of the activities. Teachers in the other two schools were on the whole satisfied that the KAT handbook provided “a good basis to work from” (Year 6 teacher and head, school 4). Some teachers said the resources provided “took off the pressure” (school 4 head teacher; school 3 head teacher).
Acceptability of the KAT intervention
Initially there were a small number of objections to KAT from parents/carers (schools 3 and 4) and these were withdrawn after discussions with school staff. In all three schools, parents/carers and teachers who took part in process evaluation interviews were positive about the topic of alcohol being addressed at primary school, with most believing that KAT’s developmental timing (for 9-11 year olds) was appropriate. All parents interviewed across all schools said they had enjoyed the family event. This was confirmed by observation and children’s focus groups which reported positive feedback from pupils’ families. Teachers at all three schools also reported positive feedback from parents about the event, directly and/or through children. Observation, focus group and interview data suggested that the majority of pupils enjoyed KAT and were fully engaged in it. Teachers were pleased with the way KAT fitted into the curriculum and the way KAT facilitated parent-school contact, although there was some disquiet amongst teachers at the amount of preparatory work required for the family event at school 6. Some heads were keen to use the KAT format for other topics:
[…] it’s made us think now, “Okay, we want to get parents in. We then have to make sure that we have some kind of activity where they can see what the children would be doing in their classroom […] perhaps make it a bit more […] that they come in to see the children’s work in displays.” That’s one way really to get them in […] I thought it [KAT] [was] the best response that we’ve had from parents and parents communicating with us and their children (Head, School 3).
In the school which withdrew from the study after baseline (school 8), one teacher said he did not think KAT would benefit the school or the children in his class and had expressed concern to colleagues from the outset about delivering KAT to children from mostly Muslim families. This teacher was absent for most of the training session delivered at school 8 and did not raise these issues with the trainer.
Intervention participation rates and reach
The majority (70.9 per cent) of intervention school pupils who completed follow-up questionnaires reported covering KAT classroom work in full, and most of the remaining pupils (22.8 per cent) had done part of it. In school 4 a greater proportion of pupils had completed all the work (86 per cent) compared with schools 3 and 6 (64.7 and 63 per cent).
An average of 50 per cent of pupils reported attending the family events with their parents/carers. The proportions of pupils whose families were represented by at least one adult member were 45.1 per cent (school 4), 46.6 per cent (school 6) and 65.7 per cent (school 3). The school with the highest FSM score (school 3) also had the highest proportion of families represented by at least one adult. FAS scores (Table IV) indicated that in the intervention arm as a whole, pupils with high scores were more likely to have an adult member of their family present at the family event, compared with pupils who had low/medium scores (56.6 vs 41.4 per cent). At schools 3 and 4, staff reported that attendance by parents was higher than usual, with one teacher (school 3) “shocked” when they saw the queue waiting to get in. At school 6 parental attendance was “on a par” with the turnout for class assemblies, which appeared to be a positive assessment because the Year 5 deputy head said that the school had been making a special effort to get parents involved with their children in events at the school.
Process evaluation data highlighted three main reasons why parents/carers attended the family events: to support their children; to support the school; and to satisfy children who put pressure on them to go:
[My daughter] said, well, that some of her friends were going to be going along and she’d put quite a lot of work in and her friend was also going to be doing some talking there as well, so I thought it would be good to go along and support the event. The school had put a lot of effort in so I wanted to make it worthwhile (Father, School 4).
Also mentioned frequently by parents and children was that parents wanted to see their children’s work; some had a general rule of attending school events (school 4, parents (P) 5, 6, 7; school 6, P3). At school 3, free tea and coffee may also have been influential (Year 4 focus group, Year 6 teacher, school 3).
Intervention theory and processes
In each school, both parents and children reported that children told their parents about what they were doing in class during the classroom component. Topics included: posters and other activities; the physical effects and dangers of alcohol; safe drinking; parental alcohol use; the nature of the KAT project; reasons why people drink; and units of alcohol. There were a few reports that children had not talked about it (pupil focus group participants A and C, school 4, P5, school 4, P1 school 6) and two children withheld information so that their parents would have a surprise at the family event (focus group participant, school 3, P6 school 6). No homework was set in school 6 but at school 4 and school 3 two teachers set homework with the specific aim of stimulating conversation between parents and children (second Year 6 teacher school 4, Year 6 teacher school 3). In both schools there were more reports that parents helped with homework than not. Topics covered included support groups (P1 school 4), physical harm (participant, Year 4, school 3) and consequences of misuse (P3 school 4).
Parents at schools 4 and 3 enjoyed talking to other parents at the family events (P5 and P7 school 4, P1 school 3). Most parents maintained that they were “quite open” about alcohol, that they talked about it at home anyway, and that attending the family event had not made any difference to that. However, two said their child would “chip in” more when alcohol was discussed (P1 school 4) and:
[…] he knows about it. We talk about it if he sees somebody on the telly. I’ll say to him “Well you don’t have to drink like that. It just makes you feel bad”. So he knew all about that anyway. This has been more since the evening. We are not big drinkers. We don’t really drink in the house so it doesn’t arise but as he’s getting older it’s the same old conversation you have with drink and drugs. I’m glad he’s doing it in school as he’ll get more information in school (P6 School 6).
Another parent had been shocked at some of the information and had become more aware of the implications of her own drinking behaviour (P7 school 4). P4 (school 4) seemed to reflect that the KAT project as a whole had led to more conversation, and while P5 (school 4) said there possibly had been more conversation after the family event, this was just repeating what had been said at home before. A significant number of children at each school said they had discussed alcohol with their families after the event, though it was not clear (except for boy 4, Year 6, school 3) whether families had discussed it more than usual. At all schools, some parents and children had told other members of their family and friends about the KAT activities and what they had learnt. Teachers at all intervention schools said that KAT provided an opening for discussions, or more serious discussions, about alcohol, and drew on evidence of children having conversations not only with their parents but also with friends, classmates, siblings and other relatives.
Trial recruitment and retention
Of the 39 eligible schools, nine were recruited, with FSM rates ranging from 1 to 37.2 per cent (Table IV). Six were above the county median, and three below. School A (intervention arm) withdrew from the study before data collection due to parents’ concern about the research topic. School 8 withdrew (also intervention arm) shortly after data collection, citing unfavourable programme timing and implementation workload issues, and the need for multiple members of the study and programme support team to contact the school.
At baseline 418 pupils were eligible to participate in the trial across the eight schools (Figure 2). Questionnaire completion rates at baseline were 81 per cent (control), and 74 per cent (intervention arm). Follow-up rates were 74 and 68 per cent, respectively (85 per cent if school 8 – from which no follow-up data were collected, is excluded). Absence from school or the classroom were the most likely reasons for non-completion, and rates of parental refusal for children’s participation were low (4 per cent).
Of the 363 pupils, 358 provided demographic information. Gender, year of birth, nationality and family affluence were well balanced across trial arms. However there were more Year 5 pupils in the control group (70.4 per cent) compared to intervention (31.7 per cent) and slightly more Caucasian pupils in the intervention group (76.3/85.7 per cent). Pupils in the intervention group were generally older, with almost twice as many in Year 6 (10-11 years) than in control (28.4/52.9 per cent). More boys than girls had missing follow-up data (58.2 vs 40.3 per cent of the total number of pupils who only provided data at baseline). Year 5 pupils were also more likely than Year 6 pupils to have missing follow-up data (59.7 vs 32.8 per cent).
Using an estimate that each eligible pupil would have only one parent/caregiver, the number of eligible parents was 418, of whom 52 (12 per cent) volunteered to take part. More parents from intervention schools expressed interest in the research than from control schools. However, the larger drop-out rate from the intervention arm meant that approximately equal proportions of parents from each group provided data. Demographic data supplied by the 41 respondents were examined in case they were all alike in one or more respects but this was not the case. As the dataset was so small, no further analyses were conducted.
Feasibility of primary outcome measures
Rates of missing data for the alcohol-related questions were low, ranging from 0.7 to 3.4 per cent (Table V). At baseline and follow-up there were some inconsistencies in the data. For example, at baseline 45 participants said that they had had a drink at some point in their lives, but 61 (16 more) indicated that they had consumed alcohol during the last month.
Feasibility of secondary outcome measures
Rates of missing data ranged from 10.9 to 34 per cent (Table VI). During data collection pupils needed more help with understanding the secondary outcome measures compared with questions on alcohol consumption. The two measures of alcohol-related communication (targeted parent-child communication about alcohol and parental-child communication scales) presented most difficulties. There was no evidence of effect in any of the secondary outcomes between the control and intervention groups. Table VI shows that confidence intervals were wide, crossing zero in every case.
Estimation of sample size needed for an effectiveness trial of KAT
Sample size calculations for an effectiveness trial were based on detecting differences in past month drinking (as a binary outcome) between intervention and control two years post-baseline (aged 11-13), using the prevalence rate of 24.5 per cent (11-13 year olds), derived from the 2009 HBSC survey (Griebler et al., 2010)[1]. KAT was a brief intervention, which was delivered at relatively low cost, and achieved high levels of reach. These factors and the lack of evidence of effect on hypothesised mediators in the exploratory phase, suggested that it would be worthwhile for a future effectiveness trial to detect a small difference in drinking, and that a 2.5 per cent reduction would be an appropriate estimate. Sample size calculations estimated that 263 schools would be needed for an effectiveness trial using 80 per cent power to detect a 2.5 per cent reduction in past month drinking rates.
Delivery structures and systems for the KAT programme as part of an effectiveness trial
Discussions with the project’s stakeholder group indicated that although one agency was willing to contribute staff resources to KAT, no organisation was able to fund its implementation as part of an effectiveness trial.
Discussion
Summary of findings
Overall KAT was delivered with good fidelity in terms of form (most activities were completed) and function (key programme messages were conveyed and embedded within the activities) (Hawe et al., 2004). In schools which completed delivery of KAT the programme had high levels of acceptability among parents, teachers and pupils in relation to content, and developmental timing. The withdrawal of two intervention schools was partly linked to concerns about the acceptability of the alcohol topic to parents (school 8) and staff perception of its suitability for a largely Muslim population – among whom alcohol consumption (and therefore decisions about the age at which it might be appropriate for young people to drink), may not have been part of everyday life (school A). School A also had concerns regarding the amount of staff input needed to deliver KAT. These issues, and the difficulties encountered in organising training, and achieving ownership of the programme in school 6, indicated some variation in implementation feasibility and the importance of support from schools for successful delivery. Overall, the programme met trial progression criteria relating to feasibility and acceptability.
One of KAT’s strengths was its ability to engage large numbers of parents/carers in all of the schools in which it was implemented, with an average participation rate of 50 per cent, exceeding the trial progression criterion of > 25 per cent. It compares favourably with similar interventions which have found recruiting parents extremely challenging (e.g. Heinrichs et al., 2005; Stead et al., 2007). The three schools which completed KAT had varying FSM levels, suggesting that its engagement processes operated in similar ways across contrasting school contexts – meeting the trial progression criterion of intervention feasibility across socio-economic groups and localities. The programme’s engagement mechanisms are well explained by the SDM (trial progression criterion 10). Interactive activities during the classroom work and family event created perceived and actual opportunities for involvement in KAT which had value and “personal interest” (Catalano and Hawkins, 1996) for pupils and parents. The programme harnessed children’s desire to impress their parents, and the interest which parents had in supporting their children’s work at school. Perceived and actual opportunities for pro-social involvement are key hypothesised mediators in the SDM of positive family attachment and avoidance of risk behaviours (Catalano and Hawkins, 1996). KAT also promoted the other two forms of socialisation which the SDM hypothesises as important to later attachment and pro-social behaviour. Classroom activities helped pupils to develop new skills and knowledge, which were then utilised during the family event. Involvement in the KAT programme (a form of pro-social behaviour) was reinforced and rewarded at the family event through the attendance of pupils’ parents, and opportunities to discuss their work.
While these engagement processes operated as intended and participants reported family conversations about KAT during the period of the classroom work, there was no evidence of effect on hypothesised intermediate outcomes at four month follow-up, and process evaluation data concerning family communication following the family event was also equivocal. Trial progression criterion 14 – which required promising effect sizes on intermediate outcomes was therefore not met. In an exploratory trial such as this one, effect sizes provide only a broad estimation of what might be expected in an effectiveness trial. There are therefore several possible explanations for our results. One is that KAT is ineffective, and the results accurately reflect a lack of intervention effect. A second possibility is that KAT does, or may have, short-term effects, but our study has not detected them. This could be because the small sample size produced an imprecise estimate, the unreliability (and therefore lack of precision) of the measures used, or a combination of these factors. A third possibility is that our programme logic model is incorrect or incomplete, and KAT operates through processes which we did not measure. Lastly, it is possible that the current programme theory is right but strengthening communication and attachment requires a more sustained intervention than KAT, which focused on a “one off” family-based event, rather than a longer period of activities embedded within schools’ existing activities and goals.
Although nine schools were recruited, two intervention schools withdrew. Sample size calculations indicated that the number of schools needed for an effectiveness trial to assess the long-term effectiveness of KAT would be prohibitive in terms of cost and feasibility (trial progression criterion 12), especially given the need to design a trial to detect a small effect size. Recruitment and retention rates for eligible pupils were high, but we were unable to engage sufficient numbers of parents to participate in the research component of the study. For pupil questionnaires rates of missing data were low but there were some inconsistencies in reporting of alcohol behaviours, suggesting that although the questions were acceptable to participants, their meaning or interpretation were not always clear, raising questions about feasibility (trial progression criterion 13). We faced considerable difficulty in identifying suitable measures of family communication for children aged 9-11. From the very small number of candidate measures identified, most were aimed at older age groups (and outside the UK), and were not always easily understood by participants in our study, even after piloting/adaptation by us. Another weakness of many of the measures was their assumption that children lived with one or two parents at home, and they were difficult to answer for some pupils who lived with different non-cohabiting parents or other adult figures.
Study limitations
Our data on the acceptability and feasibility of KAT needs to be viewed in the context of the withdrawal of two schools. For key aspects of programme implementation, we have data from the three intervention schools which remained in the study. The balance or content of the themes present in the data might have been different had these two schools remained in the study, or if we had been able to collect more data subsequent to their withdrawal. Although the study found high rates of acceptability in relation to the KAT family event, we were unable to conduct interviews with parents/carers who decided not to attend, and their views may have identified important barriers to engagement. Numbers of participants in some of the pupil focus groups were small, and they may not have fully captured the range of views about the KAT intervention among the relevant class groups. The study achieved high rates of pupil participation completion at baseline and follow-up in primary schools, but we were not able to test the feasibility of long-term follow-up of alcohol-related behavioural outcomes in secondary schools, including likely response rates. Use of parental “opt-out” consent at baseline could make secondary schools less willing to facilitate data collection due to lack of evidence of consent from parents.
Implications for research
It would not be appropriate to proceed to an effectiveness trial of KAT, as a number of progression criteria were not met. Given the low costs of implementation, and its high levels of reach, KAT may be a worthwhile and cost effective programme, but the size of effect is likely to be so small that a very large sample would be needed to estimate a statistically significant effect, and any trial would therefore be prohibitively expensive. Difficulties in accurately measuring intermediate outcomes on communication processes, and uncertainty concerning the retention of trial participants at long-term follow-up would need to be resolved. However, the study has shown that KAT has valuable assets. It provided a theoretically informed and effective mechanism for engaging parents/carers in a school-based prevention programme by creating pro-social involvement and interaction activities which pupils, parents and teachers perceived as relevant to their goals. These mechanisms could be incorporated in other school- and family-based prevention programmes.
Our study highlights the need for more detailed guidance on the design and conduct of exploratory/feasibility trials, including the formulation and assessment of progression criteria. This is imperative because exploratory/feasibility trials generate important theoretical and methodological data to inform decisions about which effectiveness trials to invest scarce resources in, and how to optimise their design and implementation. There is also a need to develop more age appropriate measures of family communication for pre-teenage children, and such measures need to make sense to children from different family structures. Interventions delivered in primary schools have the potential to reach large numbers of children and parents during a period when a family-based approach is developmentally well timed. It is important that researchers have valid and feasible measures to assess the effects of such programmes on family relationships from the perspective of children themselves.
Note
The data collected on alcohol consumption was from 9-11 year olds, and could not be used to estimate long-term effects of KAT. Its main purpose was to test feasibility/acceptability.
Corresponding author
Jeremy Segrott can be contacted at: segrottj@cardiff.ac.uk
References
Advisory Council on Alcohol and Drug Education (TACADE) (2010), “Keys to alcohol for children aged 7 to 11 years old”, TACADE, Manchester.
Arria, A. , Kuhn, V. , Caldeira, K. , O’Grady, K. , Vincent, K. and Wish, E. (2008), “High school drinking mediates the relationship between parental monitoring and college drinking: a longitudinal analysis”, Substance Abuse Treatment, Prevention, and Policy , Vol. 3 No. 6.
Axford, N. , Lehtonen, M. , Kaoukji, D. , Tobin, K. and Berry, V. (2012), “Engaging parents in parenting programs: lessons from research and practice”, Children and Youth Services Review , Vol. 34 No. 10, pp. 2061-2071.
Bellis, M. , Hughes, K. , Morleo, M. , Tocque, K. , Hughes, S. , Allen, T. , Harrison, D. and Fe-Rodriguez, E. (2007), “Predictors of risky alcohol consumption in school children and their implications for preventing alcohol-related harm”, Substance Abuse Treatment, Prevention, and Policy , Vol. 2 No. 15.
Bowling, A. (1994), Research Methods in Health: Investigating Health and Health Services , Open University Press, Buckingham.
Boyce, W. , Torsheim, T. , Currie, C. and Zambon, A. (2006), “The family affluence scale as a measure of national wealth: validation of an adolescent self-report measure”, Social Indicators Research , Vol. 78 No. 3, pp. 473-487.
Braun, V. and Clarke, V. (2006), “Using thematic analysis in psychology”, Qualitative Research in Psychology , Vol. 3 No. 2, pp. 77-101.
Carter, B. and Hood, K. (2008), “Balance algorithm for cluster randomized trials”, BMC Medical Research Methodology , Vol. 8 No. 65.
Catalano, R.F. and Hawkins, J.D. (1996), “The social development model: a theory of antisocial behavior”, in Hawkins, J.D. (Ed.), Delinquency and Crime: Current Theories , Cambridge University Press, Cambridge, pp. 149-197.
Corbetta, P. (2003), Social Research Theory, Methods and Techniques , Sage Publications, London.
Craig, P. , Dieppe, P. , Macintyre, S. , Michie, S. , Nazareth, I. and Petticrew, M. (2008), Developing and Evaluating Complex Interventions: New Guidance , Medical Research Council, London.
Currie, C. , Molcho, M. , Boyce, W. , Holstein, B. , Torsheim, T. and Richter, M. (2008), “Researching health inequalities in adolescents: the development of the health behaviour in school-aged children (HBSC) family affluence scale”, Social Science & Medicine , Vol. 66 No. 6, pp. 1429-1436.
Ennett, S.T. , Bauman, K.E. , Foshee, V.A. , Pemberton, M. and Hicks, K.A. (2001), “Parent-child communication about adolescent tobacco and alcohol use: what do parents say and does it affect youth behavior?”, Journal of Marriage and Family , Vol. 63 No. 1, pp. 48-62.
Flay, B.R. (2000), “Approaches to substance use prevention utilizing school curriculum plus social environment change”, Addictive Behaviors , Vol. 25 No. 6, pp. 861-885.
Foxcroft, D.R. and Tsertsvadze, A. (2012), “Universal alcohol misuse prevention programmes for children and adolescents: cochrane systematic reviews”, Perspectives in Public Health , Vol. 132 No. 3, pp. 128-134.
Fuller, E. (Ed.) (2009), Smoking, Drinking and Drug Use Among Young People in England in 2008 , NHS Information Centre for Health and Social Care, London.
Garcia-Dominic, O. , Wray, L.A. , Treviño, R.P. , Hernandez, A.E. , Yin, Z. and Ulbrecht, J.S. (2010), “Identifying barriers that hinder onsite parental involvement in a school-based health promotion program”, Health Promotion Practice , Vol. 11 No. 5, pp. 703-713.
Garvey, C. , Julion, W. , Fogg, L. , Kratovil, A. and Gross, D. (2006), “Measuring participation in a prevention trial with parents of young children”, Research in Nursing & Health , Vol. 29 No. 3, pp. 212-222.
Griebler, R. , Molcho, M. , Samdal, O. , Inchley, J. , Dür, W. and Currie, C. (2010), “Health behaviour in school-aged children: a world health organization cross-national study. Research protocol for the 2009/2010 study”, Child and Adolescent Health Research Unit, St Andrews, Child and Adolescent Health Research Unit, Vienna, Ludwig Boltzmann Institute of Health Promotion Research, Vienna.
Hawe, P. , Shiell, A. and Riley, T. (2004), “Complex interventions: how ‘out of control’ can a randomised controlled trial be?”, British Medical Journal , Vol. 328, pp. 1561-1563.
Hawkins, J.D. , Graham, J.W. , Maguin, E. , Abbott, R. , Hill, K.G. and Catalano, R.F. (1997), “Exploring the effects of age of alcohol use initiation and psychosocial risk factors on subsequent alcohol misuse”, Journal of Studies on Alcohol and Drugs , Vol. 58 No. 3, pp. 280-290.
Heinrichs, N. , Bertram, H. , Kuschel, A. and Hahlweg, K. (2005), “Parent recruitment and retention in a universal prevention program for child behavior and emotional problems: barriers to research and program participation”, Prevention Science , Vol. 6 No. 4, pp. 275-286.
Kaminski, R.A. , Stormshak, E.A. , Good, R.H. III and Goodman, M.R. (2002), “Prevention of substance abuse with rural head start children and families: results of project STAR”, Psychology of Addictive Behaviors , Vol. 16 No. 4S, pp. S11-S26.
Kirby, D. (2002), “The impact of schools and school programs upon adolescent sexual behavior”, The Journal of Sex Research , Vol. 39 No. 1, pp. 27-33.
Kitzinger, J. (1994), “The methodology of focus groups: the importance of interaction between research participants”, Sociology of Health and Illness , Vol. 16 No. 1, pp. 103-121.
Lloyd, C. , Joyce, R. , Hurry, J. and Ashton, M. (2000), “The effectiveness of primary school drug education”, Drugs: Education, Prevention and Policy , Vol. 7 No. 2, pp. 109-126.
Macnab, A.J. , Gagnon, F.A. and Stewart, D. (2014), “Health promoting schools: consensus, strategies, and potential”, Health Education , Vol. 114 No. 3, pp. 170-185.
Miller-Day, M. and Kam, J.A. (2010), “More than just openness: developing and validating a measure of targeted parent-child communication about alcohol”, Health Communication , Vol. 25 No. 4, pp. 293-302.
Moffitt, T.E. (1993), “Adolescence-limited and life-course-persistent antisocial behavior: a developmental taxonomy”, Psychological Review , Vol. 100 No. 4, pp. 674-701.
Nation, N. , Crusto, C. , Wandersman, A. , Kumpfer, K.L. , Seybolt, D. , Morrissey-Kane, E. and Davino, K. (2003), “What works in prevention: principles of effective prevention programs”, American Psychologist , Vol. 58 Nos 6-7, pp. 449-456.
Olson, D.H. and Barnes, H. (2012), “Family communication”, available at: www.facesiv.com/pdf/4.communication.pdf (accessed 6 August 2015).
Petrie, J. , Bunn, F. and Byrne, G. (2007), “Parenting programmes for preventing tobacco, alcohol or drugs misuse in children < 18: a systematic review”, Health Education Research , Vol. 22 No. 2, pp. 177-191.
Raab, G.M. and Butcher, I. (2001), “Balance in cluster randomized trials”, Statistics in Medicine , Vol. 20 No. 3, pp. 351-365.
Ravens-Sieberer, U. , Gosch, A. , Rajmil, L. , Erhart, M. , Bruil, J. , Duer, W. , Auquier, P. , Power, M. , Abel, T. , Czemy, L. , Mazur, J. , Czimbalmos, A. , Tountas, Y. , Hagquist, C. and Kilroe, J. , the European KIDSCREEN Group (2005), “KIDSCREEN-52 quality of life measure for children and adolescents”, Expert Review of Pharmacoeconomics and Outcomes Research , Vol. 5 No. 3, pp. 353-364.
Rehm, J. , Mathers, C. , Popova, S. , Thavorncharoensap, M. , Teerawattananon, Y. and Patra, J. (2009), “Global burden of disease and injury and economic cost attributable to alcohol use and alcohol-use disorders”, The Lancet , Vol. 373 No. 9682, pp. 2223-2233.
Rothwell, H. and Segrott, J. (2011), “Preventing alcohol misuse in young people aged 9-11 years through promoting family communication: an exploratory evaluation of the Kids, Adults Together (KAT) programme”, BMC Public Health , Vol. 11 No. 810.
Rowland, B. , Toumbourou, J.W. , Osborn, A. , Smith, R. , Hall, J.K. , Kremer, P. , Kelly, A.B. , Williams, J. and Leslie, E. (2013), “A clustered randomised trial examining the effect of social marketing and community mobilisation on the age of uptake and levels of alcohol consumption by Australian adolescents”, BMJ Open , Vol. 3 No. 1, e002423.
Smith, L. and Foxcroft, D.R. (2009), Drinking in the UK: an Exploration of Trends , Joseph Rowntree Foundation, York.
Spoth, R. , Greenberg, M. and Turrisi, R. (2008), “Preventive interventions addressing underage drinking: state of the evidence and steps towards public health impact”, Pediatrics , Vol. 121 No. S4, pp. S311-S336.
Spoth, R. , Randall, G.K. , Shin, C. and Redmond, C. (2005), “Randomized study of combined universal family and school preventive interventions: patterns of long-term effects on initiation, regular use, and weekly drunkenness”, Psychology of Addictive Behaviors , Vol. 19 No. 4, pp. 372-381.
Spradley, J.P. (1980), Participant Observation , Holt, Rinehart & Winston, New York, NY.
Stead, M. , Stradling, B. , MacKintosh, A.M. , Morag, M. , Minty, S. , Eadie, D. and the Blueprint Evaluation Team (2007), Delivery of the Blueprint Programme , Institute for Social Marketing, Stirling.
Steckler, A. and Linnan, L. (Eds) (2004), Process Evaluation for Public Health Interventions and Research , Jossey-Bass, San Francisco, CA.
Stewart, D. , Sun, J. , Patterson, C. , Lemerle, K. and Hardie, M. (2004), “Promoting and building resilience in primary school communities: evidence from a comprehensive ‘health promoting school approach’”, International Journal of Mental Health Promotion , Vol. 6 No. 3, pp. 26-33.
Storr, C.L. , Ialongo, N.S. , Kellam, S.G. and Anthony, J.C. (2002), “A randomized controlled trial of two primary school intervention strategies to prevent early onset tobacco smoking”, Drug and Alcohol Dependence , Vol. 66 No. 1, pp. 51-60.
The Alcohol Harm Reduction Group (2003), “No half-measures: a report on the impact of alcohol misuse on the work of emergency service and emergency healthcare workers”, Alcohol Concern, London.
The KIDSCREEN Group Europe (2006), The KIDSCREEN Questionnaires – Quality of Life Questionnaires for Children and Adolescents – Handbook , Pabst Science Publishers, Lengerich.
Velleman, R.D.B. , Templeton, L.J. and Copello, A.G. (2005), “The role of the family in preventing and intervening with substance use and misuse: a comprehensive review of family interventions, with a focus on young people”, Drug and Alcohol Review , Vol. 24 No. 2, pp. 93-109.
Ward, B. and Snow, P. (2008), “The role of families in preventing alcohol-related harm among young people”, Prevention Research Quarterly , Issues Paper No. 5, pp. 1-14.
Wynn, S.R. , Schulenberg, J. , Kloska, D.D. and Laetz, V.B. (1997), “The mediating influence of refusal skills in preventing adolescent alcohol misuse”, Journal of School Health , Vol. 67 No. 9, pp. 390-395.
Young, I. (2005), “Health promotion in schools – a historical perspective”, IUHPE – Promotion and Education , Vol. 12 Nos 3-4, pp. 112-117.
Acknowledgements
© Segrott, Jeremy; Rothwell, Heather; Pignatelli, Ilaria; Playle, Rebecca; Hewitt, Gillian; Huang, Chao; Murphy, Simon; Hickman, Matthew; Reed, Hayley; Moore, Laurence. Published by Emerald Group Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 3.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/3.0/legalcode
This project was funded by the National Institute of Health Research Public Health Research Programme (project number 10/3002/03).
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Public Health Research Programme, NIHR, NHS or the Department of Health. The work was undertaken with the support of The Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), a UKCRC Public Health Research Centre of Excellence. Joint funding (MR/KO232331/1) from the British Heart Foundation, Cancer Research UK, Economic and Social Research Council, Medical Research Council, the Welsh Government and the Wellcome Trust, under the auspices of the UK Clinical Research Collaboration, is gratefully acknowledged. Laurence Moore was funded by the Medical Research Council (MC_UU_12017/14) and Chief Scientist Office at the Scottish Government Health Directorates (SPHSU14). The authors are grateful to the staff, children and parents/carers in the schools who took part in the study. The project benefited from funding provided for implementation of KAT by the Welsh Government, and from the expertise of Ana Koleva-Thompson and Maggie Gregory and their team at the Participant Resource Centre, Cochrane Institute of Primary Care and Public Health. The authors would also like to thank Dr Rebecca Cannings-John, Dr Sara Jones, Cheryl Briscombe, Julie Hayward, Elen Jones and Anna Flicker for their help and input during the study.