Preventing unnecessary referrals into specialist mental health services: an interrupted time-series evaluation of a new primary care-linked mental health service

Abstract

Purpose

Many patients referred by their GP for an assessment by secondary mental health services are unlikely to ever meet eligibility thresholds for specialist treatment and support. A new service was developed to support people in primary care. “the authors evaluate” whether the phased introduction of the Lambeth Living Well Network (LWN) Hub to a population in south London led to: a reduction in the overall volume of patients referred from primary care for a secondary mental health care assessment; and an increase in the proportion of patients referred who met specialist service eligibility criteria, as indicated by the likelihood of being accepted in secondary care.

Design/methodology/approach

The evaluation applied a quasi-experimental interrupted time series design using electronic patient records data for a National Health Service (NHS) provider of secondary mental health services in south London.

Findings

Scale-up of the Hub to the whole of the population of Lambeth led to an average of 98 fewer secondary care assessments per month (95% CI −118 to −78) compared to an average of 203 assessments per month estimated in the absence of the Hub; and an absolute incremental increase in the probability of acceptance for specialist intervention of 0.20 (95% CI; 0.14 to 0.27) above an average probability of acceptance of 0.57 in the absence of the Hub.

Research limitations/implications

Mental health outcomes for people using the service and system wide-service impacts were not evaluated preventing a more holistic evaluation of the effectiveness and cost-effectiveness of the LWN Hub.

Practical implications

Providing general practitioners with access to service infrastructure designed to help people whose needs cannot be managed within specialist mental health services can prevent unnecessary referrals into secondary care assessment teams.

Social implications

Reducing unnecessary referrals through provision of a primary-care linked mental health service will reduce delay in access to professional support that can address specific mental-health related needs that could not be offered within the secondary care services and could prevent the escalation of problems.

Originality/value

The authors use NHS data to facilitate the novel application of a quasi-experimental methodology to deliver new evidence on whether an innovative primary care linked mental health service was effective in delivering on one of its key aims.

Citation

Healey, A., Melaugh, A., Demetriou, L., Power, T., Sevdalis, N., Pritchard, M. and Goulding, L. (2021), "Preventing unnecessary referrals into specialist mental health services: an interrupted time-series evaluation of a new primary care-linked mental health service", Mental Health Review Journal, Vol. 26 No. 4, pp. 317-327. https://doi.org/10.1108/MHRJ-10-2020-0071

Introduction

There is a longstanding concern that too many patients who reach secondary mental health services for assessment via a primary care referral are unlikely to meet eligibility criteria for specialist support and treatment.(Harrison et al., 1997; Slade et al., 2008) This can create preventable resource pressures on front-end assessment teams in the secondary care system, an increase in the time taken for eligible cases to be assessed and the risk that people with needs that do not meet secondary care acceptance criteria are repeatedly “recycled” around the primary care and secondary care systems while problems escalate and remain unaddressed.

The Living Well UK (LW UK) (Innovation Unit, 2019) is a new national program that aims to address this. It builds directly on an innovative proto-type service introduced to the population of Lambeth in south London – the Lambeth Living Well Network (LWN) Hub. The Hub gives primary care-registered patients access to specialist mental health support with the aim of:

• facilitating early identification of mental health issues to prevent escalation into more serious problems;

• providing access to support during acute periods of distress and offering practical advice and guidance on a range of issues that cause and that are impacted on by poor mental health – housing, employment and social isolation being examples; and

• reducing the frequency that GPs inappropriately refer patients for secondary care assessment, specifically in instances where patients seek help for mental health-related issues that GPs neither have the skills or resources to address and that are unlikely to meet the clinical threshold for acceptance into the secondary care system.

After consultation with their GP a patient may be referred directly to the Hub to be assessed as to whether they are suitable for receiving support from Hub staff. People can also self-refer themselves to the Hub or be referred from other agencies (e.g. referral by the police or a social worker).

The LWN Hub constitutes a new model of primary care-linked mental health care consistent with recent national policy developments in England: the National Health Service Community Mental Health Framework for Adults and Older Adults includes a focus on delivering care to people with enduring mental health problems too severe for management using existing psychological programmes available to primary care but not severe and complex enough to meet thresholds set by specialist services (NHS England, 2019). This development reflects a wider concern that opportunities are being missed within primary care settings to intervene with people regularly presenting with more complex problems to general practitioners (Das et al., 2016). Nationally, in the UK, there are now increasing examples of other service models outside of Lambeth that are responding to this service gap. An evaluation of the Primary Care Psychotherapy Consultation Service in east London, a service targeted at people with complex mental health needs and with broadly similar aims to the Lambeth service, has reported evidence of improved service user outcomes and prevention of potentially unnecessary contacts with the wider mental health system (including accident and emergency care and psychiatric admissions) (Parsonage, 2014). Positive outcomes, including a reduction in unnecessary referrals to secondary care teams, have also been reported for the Primary Care Service for Mental Health (PRiSM), another comparable and innovative service model operating in Cambridgeshire and Peterborough (Elliot, 2018). International examples of similar service developments in other health care systems have also been reported (Malachowski et al., 2019; Reiss-Brennan et al., 2010).

Study questions

This paper reports an evaluation that addressed the third aim identified above. It applied a quasi-experimental methodology to data extracted from NHS electronic patient records to assess whether:

• The introduction of the Hub reduced the overall frequency of referrals made by GPs to secondary mental health services, as indicated by the recorded volume of patients received for assessment by secondary mental health teams.

• There was complementary evidence that the Hub also led to an increase in the likelihood of secondary care referrals received by assessment teams meeting specialist service thresholds.

LWN hub: development and service model

The LWN Hub evolved from a co-productive process (Hyde and Davies, 2004; Tummers et al., 2016) initiated by the local NHS commissioning agency for mental health services. This included a series of consultation events with people in contact with local services and carers, secondary mental health providers, the voluntary sector and local public health officials to identify current gaps in service provision. The Hub was subsequently introduced to Lambeth GP practices in two phases. In the first phase (November 2013–June 2015) the service was restricted to a single team serving patients registered with Lambeth GPs located in the north of Lambeth only. From July 2015 onwards, the service was scaled-up to include two additional teams with coverage subsequently extended to the entire Lambeth population registered with a GP.

On introduction to the Hub, a person is assessed for their suitability for continued access to professional advice and support tailored to individual needs. Those deemed to be unsuitable for further Hub involvement at this stage can be “signposted” on to other local statutory or voluntary sector services. For those receiving a full “intervention” from the Hub, access is provided to a multidisciplinary team of staff comprising clinicians, nursing staff, occupational therapists, social workers, people with lived experience of mental health problems and other support staff from the voluntary sector.

The total number of contacts with Hub staff is variable, ranging from just a single contact with Hub staff to 12 or more contacts. On “closure” of an episode of advice and support, a person may then be referred on to other services as appropriate. While primarily a GP-linked service, in practice referrals are also received from other agencies including, for example, the police, social services as well as self-referral. In its first year the LWN Hub received an average of 392 referrals per month increasing to 473 per month in year two. Further background details regarding the co-productive processes underpinning the development of the Hub, the service model and the clinical and the socio-demographic make-up of people who attend can be found in a publicly available evaluation report (King's Improvement Science, 2017).

Methods

Evaluation framework

This study, a quasi-experimental impact evaluation, formed part of a wider multi-methods “participatory evaluation” (Fawcett et al., 2003; Zukoski and Luluquisen, 2002) of the LWN Hub (King's Improvement Science, 2017). This approach involves academic research teams working in direct collaboration with stakeholders (e.g. service staff) to deliver qualitive and/or quantitative evidence relating to a health service or programme. It differs to more standard models of evaluation (e.g. clinical trials) and has the disadvantage of involving some sacrifice of independence between researchers and those involved directly with service delivery. However, in this instance it offered a consistency of approach with the co-productive philosophy underpinning the development of the Hub and also facilitated timely feedback of evidence on whether the Hub had met its key aims and evidence relevant to on-going improvements in service quality. The evaluation was overseen by an evaluation board comprised of local service commissioners. Beyond approval of the evaluation work-programme itself, the board acted in an advisory capacity and facilitated feedback of findings from the evaluation to wider project stakeholders.

Outcome measures

The impact evaluation focussed on two outcome metrics agreed in advance between the academic team, LWN Hub staff and the evaluation board:

1. The monthly volume of secondary care assessments undertaken for Lambeth-resident patients by clinical teams within the local NHS secondary mental healthcare system.

2. Recorded “acceptance” or “rejection” on assessment for further intervention from secondary care services (used to estimate the impact of the Hub on the likelihood of acceptance for those referred for assessment).

Analytical approach

Data.

Study data were extracted from the Clinical Record Interactive Search (CRIS) database (Perera et al., 2016). CRIS provides authorised researchers with access to anonymised data on all episodes of secondary mental health care recorded on electronic patient records held by the South London and Maudsley (SLaM) NHS Foundation Trust. The use of patient data held on CRIS for research purposes has been approved by Oxford Research Ethics Committee.

Data were extracted from CRIS on all recorded contacts with secondary care assessment teams relating to patients registered with Lambeth primary care services between January 2013 and March 2018 (our period of study – 75 months in total). To evaluate the impact of the LWN Hub on the volume of secondary care assessments patient contact data were collapsed and aggregated to create a 75-month time series of the total volume of assessments recorded in each study month. Patient-level assessment data were retained for analysis of the likelihood of patient acceptance into secondary care. A binary outcome variable was created for each assessment contact (1 = “accepted” for further treatment/management by secondary mental health services, 0 = “rejected”). An “acceptance” outcome was defined as any assessment contact where an acceptance date was recorded and there was no corresponding entry of a date of rejection. Correspondingly, “rejections” were identified in cases where there was a recorded rejection date and no acceptance date. Data on a range of patient socio-demographic variables and an assessment team identifier were also included in the data extract and used to create covariates for use in the main analysis (described later in the text).

Analytical approach.

Outcomes data for the study period were divided into three segments: a “baseline” period covering the 22 months prior to the introduction of the LWN Hub (January 2012 to October 2013); a 20 month period of partial population coverage (when the Hub was available to north Lambeth residents only) beginning in November 2013 and ending in June 2015; and full Lambeth population coverage for a period of 33 months from July 2015 to the end of the study period (March 2018).

Using this segmentation of the period of study an interrupted time series design (ITS) (Bernal et al., 2016) was used to evaluate the effect of the Hub on each service outcome. This approach uses regression analysis to identify the expected difference in outcome with and without the scale-up of Hub activity, with the relevant outcome trend extrapolated from the baseline period acting as an approximation for outcomes that would have been observed had the Hub not been introduced (the “counterfactual”). The ITS approach was selected because, in the absence of the ability to randomise populations to the Hub, it can offer a pragmatic and robust quasi-experimental methodology for identifying causal programme or policy effects, specifically when there is an extended time series of outcome data and where there exists a clearly identifiable temporal point when the intervention in question becomes active (Bernal et al., 2016).

The ITS design involved fitting the following statistical model to the outcomes data for Lambeth patients:

Yt= β0+ β1time +β2partial + β3full+et

where Y_t is the outcome measured at month t, β₀ is the model intercept (outcome level in the first month of study), “time” is the time (in months) from the start of the study period, “full” and “partial” are dummy variables identifying outcomes during the periods of partial and full Lambeth population coverage respectively. The coefficients β₂ and β₃ identify the intervention effect on each outcome and e_t is the unexplained variation in Y_t. In this study we assume that the intervention effect would translate into an immediate shift in the outcome trend at each point of population coverage scale-up. This “level-change” specification was partly informed by data supplied by the Hub showing, at the point of scale-up to both partial and full population coverage, there were large and immediate step changes in the number of people being received into the Hub, rather than a gradual growth in Hub activity. For example, between the final month of partial population coverage (June 2015) to the first month of full population (July 2015), there was an immediate 3.7-fold increase in total volume of referrals received into the Hub (from 156 over one month to 573). Therefore, if the Hub provided a substitute for GP referrals into secondary care, this would imply that any subsequent changes in the level of GP referrals to secondary care would also be largely in the form of an immediate step change.

Statistical methods.

For the ITS analysis a Poisson generalised linear model (GLM) (McCullagh and Nelder, 1983) with an identity link function was used to estimate the effect of the LWN Hub on monthly volume of secondary care assessments. Newey-West standard errors (Newey and West, 1987) were used to allow for any heteroskedastic and auto-correlated modelling errors. The Poisson model was fitted to n = 75 months of data (equating to the period of study). Intervention impact on volume of secondary care assessments is estimated as the difference in total monthly assessments relating to each population coverage period compared to the predicted baseline period trend (the counterfactual). Given that an expansion in scale of coverage to all of Lambeth would be expected to have a larger effect on the volume of secondary care assessments carried out compared to partial population coverage, an additional check on the internal validity of estimated effects was carried out using a chi-square test of the null hypothesis of an equal effect on volume of assessments for partial- and full-population coverage.

The likelihood of acceptance into secondary care was modelled as a binary outcome (1 = “accepted” into secondary care, 0 = “rejected”) using logit regression applied to patient-level assessment data for which there was complete data for identification of secondary care acceptance or rejection. The estimation sample included N = 9,062 unique patient assessment episodes. Predictions were obtained from the logit modelling to evaluate the change in the probability of acceptance into secondary care resulting from both partial and full population coverage from the LWN Hub. As the predicted effects on probability are non-linear with respect to the study period, predictions are anchored to the same time segment (the partial population coverage period). They are, therefore, used to effectively describe the impact on the probability of acceptance of full compared to partial coverage had full coverage been introduced immediately (with both impacts evaluating the change from the probability of acceptance estimated in the absence of the LWN Hub). All standard errors in the logit modelling were adjusted for patient clustering (the same patient could be assessed multiple times over the period of study). For all analyses (linear regression and logit models) 95% confidence intervals are reported to convey uncertainty in the estimated effect of the Hub on outcomes.

Analyses were carried out using Stata (Version 15) (StataCorp, 2017).

Sensitivity analysis.

ITS models were subjected to a series of sensitivity tests including: use of an alternative specification of seasonal effects (monthly instead of quarterly effects); different time trend specifications (linear versus non-linear/quadratic); and use of chained equation multiple imputations (Azur et al., 2011) for missing data on patient covariates used in the logit model of assessment outcome. Missing data principally affected recorded deprivation scores and ethnicity and lead to a 6% loss of observations when analysis was restricted to observations with complete data.

Results

Descriptive statistics for each outcome and patient-level co-variates used in the logit model along with further detailed regression outputs are presented in the supplementary appendix.

Volume of secondary care assessments

Figure 1 is a graphical description of the observed monthly trend in the volume of secondary care assessments undertaken for Lambeth-residing patients relative to the dates when there was a scale-up in Hub activity. The hollow circles identify the actual number of assessments undertaken over the study period.

The dotted curved line in the figure is a plot of the predicted number of assessments obtained from the ITS statistical model assuming that the pre-Hub trend in volume of assessments (i.e. the “counterfactual” trend) would have continued to the end of the study period. It, therefore, provides a graphical sense of the impact of the Hub on volume of assessments undertaken. Table 1 presents the numerical equivalent of what can be observed graphically in Figure 1, with adjustment for seasonal variation and underlying time trend in monthly volume of assessments.

Partial population coverage is estimated to have reduced secondary care assessment volume by 54 assessments per month (95% CI; −67 to −41) compared to an expected mean of 203 per month had the Hub not been introduced. Scale-up to full population coverage was associated with 98 fewer assessments per month (95% CI; −118 to −78): 201 assessments would have been expected over the same period. A chi-square test of the null hypothesis of an equality of effect between partial and full population coverage was rejected at the 0.1% level of statistical significance (χ²(1 df) = 85.67, p < 0.001)).

Likelihood of secondary care acceptance

Figure 2 is a plot (with 95% confidence intervals) of the predicted effect of scale-up to partial and then to full population coverage on the probability of acceptance into secondary care for Lambeth patients assessed by secondary care teams (these predicted effects are also shown in Table 1). The solid dots show the expected increase in the likelihood of acceptance into secondary (based on the ITS analysis) had the roll-out of the Hub been restricted to the north of Lambeth only. The hollow dots show what the likelihood of acceptance would have been predicted to be had the Hub instead been introduced to the entire of population of Lambeth over that same period. The vertical lines around both sets of dots are 95% confidence intervals.

The probability of acceptance was estimated to increase in absolute terms by 0.12 in each month during partial population coverage (95% CI; 0.07–0.16) from an average probability of acceptance of 0.57 estimated from the baseline period trend. Had the availability of the Hub been scaled-up to full Lambeth population coverage during the same period the probability of acceptance would have increased by 0.20 based on model predictions (95% CI; 0.14–0.27).

Sensitivity analysis

The model predictions from ITS models were insensitive to alternative specifications for seasonal variation (using monthly rather than quarterly seasonal indicators) and to use of imputed values for missing data on patient-level co-variates used in the logit model of likelihood of acceptance. The ITS model for likelihood of acceptance was also insensitive to inclusion of a non-linear rather than a linear time trend. However, for the ITS model of monthly volume of assessments, inclusion of a linear trend reduced the predicted effect of the Hub compared to the main ITS (quadratic trend) by 57% (partial population coverage) and 36% (full coverage).

Discussion

Using an interrupted time-series design the introduction of the Lambeth Living Well Network Hub to a GP-registered south London population was found to be associated with a fall in the volume of patients received by secondary care teams for initial assessment. The estimated effect of scale-up of the Hub to the entire Lambeth population was, as expected, larger and statistically significantly different to that observed when the Hub was restricted to the population of north Lambeth only. The availability of the Hub to primary care services in Lambeth was also found to increase the likelihood of those assessed being accepted into the secondary care system. In this paper, we have focussed on secondary care impacts that were the primary concern for the commissioners of the Hub evaluation in terms of what the service model was primarily expected to achieve with respect to wider service outcomes. However, in secondary analysis (not reported here) we also found some evidence of a reduction in contact with psychiatric services embedded within hospital emergency departments, though these effects were estimated with comparative statistical imprecision and, therefore, carry more uncertainty (King's Improvement Science, 2019). Reductions in accident and emergency visits have also been reported as part of an evaluation of a comparable primary care-linked service (also London-based) that had similar aims to the Lambeth Hub (Parsonage, 2014).

These are important findings because they suggest that the Hub as implemented was delivering on one its key intended aims: it reduced both the quantity of GP referrals to assessment teams while at the same time altering the case composition of secondary care referrals given that people who were still being referred by their GP were now significantly more likely to contact assessment teams with mental health needs more aligned to the requirements for acceptance into specialist services. These impacts were observed alongside a large concurrent increase in Hub activity, with the implication that GPs were using the Hub as an appropriate substitute to secondary care referral for people who would be unlikely to gain ever gain acceptance into the secondary care system.

We did not set out to conduct a full comparative economic evaluation of the LWN Hub. However, for purely benchmarking purposes analysis of service activity data and contacts made with service clients suggests that the direct costs of providing a service of this nature does not exceed and may be in fact be lower, for other mental health programmes aimed at different population groups embedded within primary care. The average cost for the most expensive 25% of cases managed through the Hub during the initial years of its introduction was £79 (2018/2019 prices). The average cost for initial assessment of a “low severity” service user within the Improving Access to Psychological Therapies (IAPT) programme is £116, with the most expensive 25% of such cases being over £124 (King’s Improvement Science, 2019). The Hub can, therefore, be viewed as service model that can be delivered at a cost not entirely divergent from the resourcing required to sustain other more established primary care-base programmes.

Study strengths and limitations

A key strength of this study was our ability to exploit electronic patient records data over a sufficient period and with a degree of frequency to enable the novel application of a robust quasi-experimental evaluation design. The approach we adopted enabled further insight to be gained into whether the implementation of a new primary-care linked mental health service was delivering on one of its intended aims.

This study also has limitations. The evaluation is highly location- and context- specific and the secondary care impacts observed here would, therefore, not necessarily translate to a similar service model implemented in another location and mental health service setting. We were unable to track and compare mental health outcomes for cohorts of patients registered to GPs with and without access to the LWN Hub. While the Hub was set up to improve outcomes for people who might not otherwise gain timely access to needed support there was an inherent risk that in some instances the Hub may have either harmed outcomes or have had limited impact: in some cases GPs may have avoided making clinically appropriate referrals to secondary care because of the availability of the Hub; in other instances a referral to the Hub by a GP will not necessarily have guaranteed patient uptake and the opportunity to improve outcomes for those with identified needs. We were unable to evaluate these unintended consequences in this instance. We were also unable within the use of an ITS study design to rule out the possibility that concurrent area specific effects unique to Lambeth might partly explain the impacts observed on study outcomes. We could also not fully account for trends in service availability over time within the local mental health Trust which might partially have explained an increase in rates of service acceptance into secondary care. We do note, however, that over the period of study workforce levels within the Trust (as a very crude indicator of service capacity) remained broadly stable (South London and Maudsley NHS Foundation Trust, 2021). The period of study also coincided with wider financial pressures imposed on public services owing to central government “austerity” measures in public spending, so major investments in capacity in relevant services would have been unlikely. Finally, in this study we were concerned with testing whether the Hub was effective against meeting one of its key aims – a reduction in unnecessary referrals from primary care to specialist mental health assessment teams. However, this does not address the more holistic question of whether the LWN Hub offered an effective and overall cost-effective alternative to the service arrangements that were in place in its absence. This would require evidence that mental health outcomes for people using the Lambeth Hub were at least as favourable, if not better, than would have been expected had the Hub not been active. It would also require a system wide analysis of resource impacts and cost: it is plausible, for example, that the introduction of the Hub could lead to higher overall system costs if any resource “savings” from fewer secondary care assessments are offset by the cost of increased Hub activity or wider service activity generated by Hub “signposting” and onward referral. This would need a more complete system wide economic analysis that was not possible within the context of this study.

Conclusion

The introduction of a new innovative mental health service linked to primary care was effective in reducing the volume of people received from primary care for assessment by secondary mental health services. There was also evidence it increased the likelihood of acceptance for specialist intervention among those assessed, suggesting that more people who were referred for assessment by their GP were meeting secondary care eligibility criteria.

Figures

Figure 1

Lambeth secondary care assessments: counterfactual and observed

Figure 2

Predicted increase in probability of acceptance with 95% CIs

Table 1

Predicted impacts from ITS model

	Predicted impacts
Monthly volume of assessmentsa (95% CI)
Partial population coverage: average monthly change in outcome (95% CI)	−54 (−67 to −41)
Full population coverage: average monthly change in outcome (95% CI)	−98 (−118 to −78)
Likelihood of acceptanceb (95% CI)
Partial population coverage: marginal effect on probability of acceptance (95% CI)	0.12 (0.07 to 0.16)
Full population coverage: marginal effect on probability of acceptance (95% CI)	0.20 (0.14 to 0.27)

Notes:

^a Predicted impacts on monthly volume of secondary care assessments based on Poisson generalised linear model with identity link function. Confidence intervals (CI) generated from Newey-West standard errors.

^b Marginal effect on predicted probability of assessment derived from logit model of secondary care assessment outcome. Confidence intervals based on robust standard errors