Identifying occupational therapy outcome measures supportive of recovery-orientated mental health services in Ireland


Purpose
No occupational therapy outcome measures have been designed specifically for recovery-orientated services.This paper aims to identify occupational therapy outcome measures relevant to mental health practice and assess them against recovery principles adopted by Irish Mental Health Services.


Design/methodology/approach
A narrative review methodology was used to appraise outcome measures against CHIME recovery principles.


Findings
A systematic search across 13 databases identified eight well-established outcome measures commonly used within occupational therapy mental health literature. The included outcome measures were appraised using a recovery alignment tool.


Practical implications
All outcome measures connected to some recovery processes. Those using semi-structured interview formats and notably the Canadian Occupational Performance Measure (COPM) had the strongest alignment to recovery processes.


Originality/value
This is the first known review which provides some validation that the included outcome measures support recovery processes, yet the measures rely heavily on therapist’s skills for processes to be facilitated. It recommends that ways to better support the process of partnership in occupational therapy mental health outcome measures be explored and further research be undertaken.



Introduction
Recovery ideals focus on the individual's process and lived experiences; it involves the development of meaningful goals, personal growth and engagement in a meaningful life (Kelly et al., 2010). The concept of a recovery approach to mental health has grown to be a prominent influencer of policy and practice internationally (Field and Reed, 2016). It emerged following the progressive shift away from institutional settings and deficitfocused models (Slade et al., 2014;Field and Reed, 2016). While there is no set model of recovery, it is generally described as a unique, non-sequential journey that involves personal growth towards the attainment of meaning in life (Slade et al., 2014). A systematic narrative synthesis of 97 studies (Leamy et al., 2011) included experiences of over 1,100 participants living with a mental health condition, the results of which established five recovery processes (CHIME): 1 connectedness; 2 hope; 3 identity; 4 meaning; and 5 empowerment.
The findings demonstrated that these processes emerge at different stages in the recovery journey. Observing this, Leamy et al. (2011) mapped processes unique to each stage of applying Prochaska and Di Clemente's (1982) transtheoretical model of behaviour change as shown in Table 1.
Although recovery offers a structured guidance for practice, policy and research meaningful to stakeholders (Slade et al., 2014;Leamy et al., 2011), critiques exist regarding the limited evidencebase on how best to facilitate a recovery supportive service (Slade et al., 2014). Additionally, vague definitions within the recovery approach may mask inconsistencies between recovery-oriented policy recommendations and subsequent changes in service delivery (Slade et al., 2014). Despite these critiques, the recovery approach established a strong footing in mental health services, given its focus upon strengths and personal meaning (Field and Reed, 2016). In line with recovery, this paper uses the term person rather than the conventional occupational therapy term client to acknowledge expertise by experience (Leamy et al., 2011).

Recovery-orientated practice in Ireland
The Vision for Change in 2006, was the first policy framework which saw the recovery approach placed as a fundamental part of Irish Mental Health Services [Department of Health (DH), 2020]. It outlined the Government's commitment to developing a recovery-orientated approach and the commencement of restructuring and evaluating services (DH, 2020). It draws from the work of Leamy et al. and emphasises the need for partnerships between services and the person (and/or family members). This is a significant shift in Irish service delivery.
In 2020, Sharing the Vision policy document was released. The continuous drive towards facilitating recovery-orientated services remains a central principle of the policy. It asserts a commitment that the future directions of the Irish Health Service will be designed in partnership with service users, families and carers (DH, 2020).

Recovery and occupational therapy
There is strong alignment between occupational therapy and the recovery approach (Merryman and Riegel, 2007). Both place the person and their story as the primary focus of practice [American Occupational Therapy Association (AOTA), 2008; Best et al., 2017]. Occupational therapy's value and knowledge on the intricate processes of a person's narrative supports the profession to also appreciate the intricate and non-linear process of recovery (Reed et al, 2020). Qualitative studies exploring the perspectives of occupational therapists emphasise these shared ideals (Hurley and McKay, 2009;Synovec, 2015). A qualitative study involving occupational therapists (n = 8) working in acute adult mental health units in Ireland highlighted some approaches used to support recovery processes (Hurley and McKay, 2009). The findings indicated that therapists had a good understanding of recovery but concluded that further study is needed to examine occupational therapy practice. Gibson et al. (2011) completed a systematic review of 52 studies examining the effectiveness of occupational therapy interventions in recovery-orientated services. They found evidence to support some occupational interventions but emphasised the need for further research and use of wellestablished outcome measures (Gibson et al., 2011).
Occupational therapy and recovery include the promotion of hopefulness and place focus on quality of life rather than clinical recovery (AOTA, 2008;Merryman and Riegel, 2007). The profession's focus on strengths, well-being and quality of life illustrate additional connections (Gruhl, 2005;AOTA, 2008). The recovery journey is well aligned with occupational participation processes and the constructs of doing, being, belonging and becoming (Merryman and Riegel, 2007;Doroud et al., 2015). Qualitative and narrative studies exploring core factors that supported participant's recovery corresponded with themes of meaningful occupational engagement (Kelly et al., 2010;Reed et al., 2020).
The shared principles between occupational therapy and recovery gives the profession a pivotal opportunity to facilitate recovery and to guide recovery-orientated practice (Gruhl, 2005;Kelly et al., 2010;Reed et al., 2020). Despite this, there is limited evidence to guide best occupational therapy practice within recovery-orientated settings (Doroud et al., 2015;Gibson et al., 2011).

Statement of problem
Recovery is the central approach underpinning Irish Mental Health Services (DH, 2020). Occupational therapy's founding philosophies afford it with a unique scope for enhancing the advancement of recovery processes (Gruhl, 2005;Merryman and Riegel, 2007). In recent years, a number of evidence-based and recovery focused outcome measures have emerged (Brown et al., 2019); however, these measures are not discipline specific. To empower and support people to engage in meaningful occupations, therapists require sensitive assessments that draw from occupational therapy models capable of detecting changes in occupational performance (Brown et al., 2019;AOTA, 2008).
Literature searched for this review yielded no study which explored whether occupational therapy outcome measures are supportive of the recovery approach. Given that Irish Mental Health Services have committed to a recovery-orientated approach this raises a pertinent issue for occupational therapists in the sector. This study explores the extent to which Developing feelings of hope, awareness and establishing supportive connection 3 Preparation Shift in self-perception, reengagement, planning and setting goals 4 Action Active engagement, enhanced understanding of self, progression and sense of empowerment 5 Maintenance and growth Attainment and management of a meaningful life and personal perception of well-being Source: Prochaska and Di Clemente (1982), Leamy et al. (2011) occupational therapy specific outcome measures incorporate core recovery processes. Thus, the aim of this review is twofold: 1 to identify commonly used occupational therapy outcome measures used in peer-reviewed publications related to Irish Mental Health Services; and 2 to determine the extent to which the identified outcome measures align with the CHIME recovery processes.

Methodology
The first aim of this review was achieved by completing a systematic search strategy. To meet the second aim, a narrative review methodology (Hawker et al., 2002) was selected. This allowed insights and conclusions to be drawn regarding the alignment between recovery processes and the identified outcome measures (Baumeister and Leary, 1997;Green et al., 2006) using a purpose-designed tool. Searches, data extraction and the recovery alignment appraisal process were crosschecked by a peer and discussed in supervision. An audit trail and key decisions were documented within research supervision records.

Search strategy
The search strategy inclusion criteria were defined using the ECLIPSE model (Wildridge and Bell, 2002): The Expectation was to identify occupational therapy outcome measures used in mental health practice via searching for peer-reviewed studies published in English which used occupational therapy outcome measures.
The Client group was adults aged 18-65 years with a mental health diagnosis.
The Location of studies was from inpatient, outpatient and community settings.
The Impact criteria included quantitative studies that used occupational therapy outcome measures to capture change.
Professionals involved were occupational therapists.
The types of Service were mental health settings.
The MeSH headings, database specific subject headings and the list of databases searched to capture relevant studies have been stored by the authors. An expert librarian guided search strategy refinement. Thirteen databases were systematically searched between July 2017 and September 2019 complemented by hand searching key occupational therapy journals and harvesting references from included studies. The Preferred Reporting Items for Systematic Reviews and Metaanalysis (PRISMA) diagram ( Figure 1) demonstrates the results of data collection and screening processes (Moher et al., 2009). PRISMA reporting guidelines (Green et al., 2006;Moher et al., 2009) were followed.
The systematic search yielded 84 included studies, data was extracted and categorised applying the Population, Intervention, Control, Outcome (PICO) format (Cooke et al., 2012). Most studies were cross-sectional (n = 41) or cohort study designs (n = 22). Studies were classified by level of evidence according to the work of Sackett, 1989. Most of the studies were level III (n = 54) and level IV (n = 14). Settings were relatively evenly distributed between community and inpatient services.
Studies were published in 23 journals, most were published in the past 20 years (n = 72). Thirteen countries represented:

Figure 1. Flow chart illustrating data collection process
Sweden was the most frequent (n = 31), followed by the USA (n = 15) and the UK (n = 11).
Identifying outcome measures was the priority for the search. Due to time constraints and to maintain focus on this paper's aims, the included studies were not critically appraised.
The identified outcome measures Twenty outcome measures were identified and reviewed against inclusion criteria set out in Table 2. Twelve were excluded due to the inclusion criteria, leaving eight measures to review. The included outcome measures were presented to an external advisor to explore their relevance to occupational therapy practice. The advisor was a Senior Occupational Therapist practicing in the Irish Adult Mental Health Service.
Designing the recovery appraisal tool To systematically determine the extent to which the eight outcome measures aligned with the recovery framework, a tool was developed by the first and second authors. This involved modifying a tool created by Evans et al. (2000) to embed core recovery processes (Leamy et al., 2011;DH, 2020). Research supervisors (second and third authors), the external advisor (fourth author) and peers contributed to the final version of the alignment tool. In the alignment tool the CHIME processes are grouped and represented using three headings: personcenteredness and empowerment, hope and partnership and collaboration.

Person-centeredness and empowerment
Person-centeredness was chosen to collectively describe the placement of the person, with their unique journey with individual meaning, at the centre, and the growth and nurturing of a person's self-identity (Leamy et al., 2011;DH, 2020). Empowerment relates to the acknowledgment of the person's lived experience and their strengths or insights arising through their personal journey (DH, 2020).
The recovery alignment tool focuses on how well the outcome measure is suited to each stage of recovery and to what extent personal experience is integrated. This was accomplished by reviewing clinical utility in manuals then comparing the content to descriptions of recovery stages outlined in Table 1 (Leamy et al., 2011). Each outcome measure was categorised as generally suitable, maybe unsuitable or unsuitable for each recovery stage based on instructions from manuals. To enhance trustworthiness during this process, authors of each outcome measure were contacted with the preliminary category assigned. Input from the six authors who replied enabled the research team to refine the suitability rating process. The author's feedback was generally supportive of the categorisation, with five authors concurring with the allocation. A fruitful discussion with one author enabled an agreement to be reached regarding the classification.

Hope
The second recovery process, hope, involves having ambitions and an optimistic view of one's future (Leamy et al., 2011). For services, this entails that a person is actively involved in their care planning and personal goal setting (DH, 2020). Personal-centred goals carry more meaning and can support a sense of self-determination; increasing motivation towards goal attainment (Synovec, 2015). Assessments and interventions which harness a person's intrinsic, environmental and social supports promote selfbelief and self-efficacy (Synovec, 2015); the core supporting aspects of hope (Leamy et al., 2011).
This question centres on the outcome measures means of obtaining a personally determined goal rather than a clinical one. In the recovery alignment tool, the extent to which the person is involved in identifying strengths and in goal-setting signifies hope. The component was explored by observing if outcome measures contained: a personal goal-setting element and a means of regarding or documenting a person's strengths and supports.

Partnership and collaboration
The importance of promoting people's active roles in therapy and service design is embedded throughout the themes of meaning, identity connectedness and empowerment (Leamy et al., 2011). Partnership and collaboration are incorporated into the recovery alignment tool by reflecting on how the outcome measure is administered and scored.
This component aimed to consider the placement of the power balance. The questions asked to reflect this included the format of administration and looked at whether the person has any input with regards to sharing their reflections on their performance within the scoring and analysis.
Forty percent of adults in Europe have difficulties comprehending information related to health [HSE and National Adult Literacy Agency (NALA), 2018]. Therefore, the accessibility of the language used in the measures is a key consideration in people's active involvement in therapy. The degree to which the manuals of outcome measures adhered to Plain English guidelines was determined by asking 22 applicable questions drawn from the guidelines (HSE and NALA, 2018). The measures were then graded as adhering, generally adhering or somewhat adhering to Plain English guidelines (HSE and NALA, 2018). To ensure relevance to contemporary practice, outcome measures cited more than once or cited once within the last 10 years Outcome measures cited once in the included 84 studies over 10 years ago

Results
The systematic search identified twenty occupational therapybased outcome measures from the included 84 articles; of these eight met the inclusion criteria. The most frequently cited measures were the Canadian Occupational Performance Measure (COPM) (n = 16) and the Assessment of Motor and Process Skills (AMPS) (n = 12). In relation to the underlying theory, five of the eight outcome measures were derived from the Model of Human Occupation (MOHO). The Kohlman Evaluation of Living Skills (KELS) was the only outcome measure specifically designed for use with people experiencing mental health problems. The other outcome measures were not diagnostically bound. The AMPS was the only measure requiring software for scoring; other outcome measures include this as an option. Table 3 provides information regarding the eight included outcome measures. The systematic search process also identified three emerging outcome measures which did not meet the inclusion criteria but are of note given their population focus within mental health and the level to which they were cited in the last ten years. The first was the Satisfaction with Daily Occupations (SDO) (n = 11) it measures a person's perceived occupational performance and level of activity in occupational domains via interview (Eklund, 2009). The second was the Profile of Occupational Engagement in People with Severe Mental Illness (POES) and the alternate version, the Profile of Occupational Engagement in people with Schizophrenia (n = 10 combined). Both versions measure people's time use to assess occupational patterns of engagement in ADLs and occupations (Bejerholm et al., 2006). The third measure was the Engagement in Meaningful Activates Survey (EMAS) (n = 4) The EMAS uses a Likert Scale to measure the extent to which people participate in meaningful activities (Goldberg et al., 2002).

The recovery alignment appraisal
To meet the second aim of the study the recovery alignment tool was used to assess the eight included outcome measures. The results (Figure 2) are detailed under the processes: personcenteredness and empowerment; hope; and partnership and collaboration. Each of these processes will be considered in turn. Recovery stages are represented in Figure 2 as corresponding numbers from one to five.

Person-centeredness and empowerment
Recovery stages are central to a personalised approach. The alignment of each tool specific recovery stages is illustrated in Figure 2. No outcome measures were found to be unsuitable in a recovery stage; therefore, Figure 2 only includes the generally suitable and may be unsuitable categories. Two of the four observation-based formats were suitable across all recovery stages (ACIS and MOHOST). All outcome measures were found to be generally suitable at stages four and five. Fisher and Jones (2007) do not recommend the AMPS for use when people are acutely unwell nor for those who are not presently engaging in ADLs; thus, AMPS may be unsuitable with some persons in recovery stage one.
Interview-style measures require a person to discuss life circumstances and the therapist explores occupational performance. As a consequence, interview-style formats may be unsuitable for some people in stages one and two of recovery. For example, the OPHI-II was assessed as being may be unsuitable at stages one to three given that an in-depth personal interview of life history may be distressing to some people in the early stages of recovery . This also pertained to the KELS as it contains both interview and observational components.
The personal experience of the recovery journey was integrated into the COPM, OPHI-II and OCAIRS. These outcome measures use a semi-structured interview to attain personal narratives Forsyth et al., 2005;Law et al., 2005). Manuals of the other outcome measures emphasised the importance of personal narratives and rapport surrounding administration. Hence, the remaining six outcome measures informally integrate personal experience.

Hope
Personal goal-setting, documenting personal strengths and supports were factors linked to the process of hope in the recovery alignment tool. Goal-setting elements were found within the AMPS, COPM, OCAIRS OPHI-II and OSA. In contrast, the ACIS, KELS and MOHOST did not include goal-setting elements. Although not integral to administration of these measures, the manuals specified that results from the measures could be used to guide goal-setting via offering insights relating to performance in meaningful activities (Kohlman Thompson, 1992;Forsyth et al., 1998;Parkinson et al., 2004).
All outcome measures which used an interview format (COPM, OCAIRS, OPHI-II and OSA) provided a means of identifying strengths and supports. Observational formats do not directly address hope; however, the AMPS suggests that personal strengths and supports should be incorporated in the assessment and intervention process (Fisher and Jones, 2007). While the ACIS, KELS and MOHOST manuals do not explicitly address hope, it is embedded within the theory the measures draw upon (Kohlman Thompson, 1992;Forsyth et al., 1998;Parkinson et al., 2004).

Partnership and connectedness
All outcome measure manuals encourage interaction between the person and therapist, thus laying the foundation for a recovery partnership. Overall, the administration formats are mixed, four (the ACIS, AMPS, KELS and MOHOST) are observational formats and four are interview-styles (the COPM, OCAIRS, OPHI-II and OSA). The flexibility within semi-structured interview formats (COPM, OCAIRS and OPHI-II) present a stronger alignment with this process; offering a more balanced two-way interaction to capture valuable information compared with being observed performing a task.
Scoring and the interpretation of outcome measures rely upon clinical expertise, yet the OSA and COPM illustrate capacity to obtain the person's perception of their performance via self-rating scales. The COPM and the OSA presented with the strongest alignment to partnership and connectedness given that they include the person's input. The other six measures are solely scored by the occupational therapist.
Six of the outcome measures contained instructions for therapists to then explain, two of which (AMPS and COPM) adhered to Plain English guidelines (HSE and NALA, 2018) and the other four measures generally adhered (ACIS, OCAIRS, OPHI-II and MOHOST). Two measures contained instructions for the person. Of these three, one adhered (KELS) and one somewhat adhered to the guidelines (OSA).

Discussion
Both aims of this review were achieved. Eight occupational therapy outcome measures relevant to mental health practice were identified then appraised against recovery processes. This review provides some validation that the appraised outcome measures reflect the CHIME recovery principles to different degrees and in different ways. The COPM performed the strongest overall, the two other semi-structured interview formats (OCAIRS and OPHI-II) also rated well in terms of recovery alignment. A rank-ordering is not suitable given the diverse approaches and focus of measures. The extent to which the included outcome measures are best fit for purpose in Irish recovery-orientated services will now be explored.

Person-centeredness and empowerment
Recovery has shifted concentration away from just acute needs, observing that needs and wellness planning span across all stages (Best et al., 2017). The recovery stage suitability scale identified no gap preventing tailored support across any stage (Best et al., 2017) each format aligned with different recovery processes. Observational and interview-style formats present different qualities under which they support recovery stages. Observational-style measures offer flexibility through their general suitability across recovery stages while interview-style measures centre on capturing lived experience (Leamy et al., 2011).
The accessibility of language and terms used in therapy is a key consideration concerning recovery partnerships. To explore accessibility each measure was critiqued. The measures, with exception to the OSA, scored as adhering or generally adhering to Plain English guidelines (HSE and NALA, 2018); however, information about the scoring and analysis of the outcome measures is dependent on the therapist's skills to explain.

Hope
Hope is evidenced by employing a strengths-based approach and by attending to the person's own definition of well-being (Best et al., 2017). All the measures reflected elements of hope. Interview-style formats appear to reflect hope more by  Explanation of scores therapist dependent.

Yes
Semi-structured interview Therapist Instructions for therapist generally adhere to plain English guidelines.
Explanation of scores and concepts are therapist dependent.
Yes providing a higher level of input from the person, capturing their narrative, motivation and sense of self-efficacy (Synovec, 2015). The relationship between the person and the therapist is essential for leveraging strengths (Gruhl, 2005;Synovec, 2015). It is important to note, that the integration of a person's strengths and possibly promote hope relies on the therapist's skills to incorporate them (Nugent et al., 2017). Despite the profession being focused on a person-centred, strength-based approach (Gruhl, 2005), qualitative findings indicate that this is not always perceived by persons receiving occupational therapy input (Maitra and Erway, 2006;Palmadottir, 2006). Some of the participants described their therapists as having a deficit focus and discussed the damaging impact this had on their sense of hope. Incorporating interview style formats to the assessment process along with attention to one's therapeutic approach appear to be factors that may promote the development of hope within therapy. This raises a key topic of ongoing reflection in practice.

Partnership and connectedness
Partnership is a strong element in recovery, it endorses a culture of shared power and co-designing services (Best et al., 2017).
Overall, the measures have supportive qualities yet there appears to be a potential area for enhancement. Interview-style formats may present as being more collaborative but the questions and areas being discussed are directed by the therapist. In the case of observational formats, the therapist is the only scorer hence the extent to which these outcome measures promote partnership is dependent upon the therapist's ability to embody recovery principles (DH, 2020). These points present a potential area of qualitative research to explore the person's perception partnership during the administration process of the reviewed measures.
In spite of the focus on collaboration throughout the occupational therapy process (AOTA, 2008) the outcome measures showed a poorer alignment to partnership within the recovery appraisal. The above mentioned potentially emerging outcome measures (POES, SDO and EMAS) include the person's perceptions and active collaboration in the administration (Eklund, 2009;Bejerholm et al., 2006;Goldberg et al., 2002). These measures could enter into mainstream use and consequently have an influence on partnership within occupational therapy practice.

Implications for practice
This review contributes insights regarding the types of outcome measures being used in mental health practice. It generates some discussion supplementing the lack of guiding literature regarding occupational therapy practice in the context of recovery (Gibson et al., 2011).
Observational and interview style formats present different qualities under which they support person-centeredness and empowerment. The components of hope reflected the strongest authentication of the outcome measures shared ideals with recovery; in particular outcome measures that use an interview style format. The promotion of partnership and connectedness highlighted some positive traits and potential areas for change, mainly, by considering the level to which an outcome measure allows for collaboration.
The recovery alignment appraisal highlighted that recovery themes strongly rely on the therapist's skills; particularly in relation to the promotion of partnership. This implies that regardless of the strength of alignment with recovery processes, ultimately, it comes down to the individual practitioner. This dependency on the therapist outlines there is a need to consider how outcome measures are administered to ensure it is recovery focused. A reflection point could include whether an assessment or report could be written in a more recovery focused way.
Studies are needed to explore the concept of partnership involving therapists and persons with lived experience; this could include persons appraising therapists' recovery focus. Such research could allow for partnership to be better supported in the administration of the appraised outcome measures. Outside of research, workshops involving peersupport workers and occupational therapists could explore ways to strengthen partnership and other recovery processes when administering and presenting information from outcome measures.

Limitations
This paper was written from a clinician perspective, this presents a potential risk of bias. Studies involving persons with lived experience appraising occupational therapy outcome measures against recovery principles would provide richer insights regarding the recovery qualities of the included measures.
The recovery alignment tool was designed for the purpose of this study. Its reliability and validity to assess for recovery alignment has not been established. Exploring the validity of the tool could be the focus of future research.
Thirteen countries were represented in the 84 studies; however, articles not published in English were excluded. This possibly skewed representation of studies.