BJPsych Advances
Ensuring the community cares: assessment and evaluation of social care needs in long-term mental illness
Sarah Davenport

Abstract

Clinicians have been divided in their views on whether systematic needs assessment leads to an improvement in the outcome of care for people with chronic mental illness. Policy makers assume that assessment and evaluation will improve the clinical and cost-effectiveness of mental health services. Many of these assumptions are already embedded within mental health policy and practice. A decade of research has not resolved this tension. However, clinicians need to know about the options available to guide systematic assessment and evaluation of care. They can then make informed decisions about their use to promote effective care planning or as a tool to direct socially inclusive service development.

Assessment and evaluation – essentially the translation of the results of assessment into quantifiable or numerical form – are becoming ubiquitous in healthcare. Mental health clinicians have long been assessing the health and social care needs of individual patients as a matter of course. However, over the past decade, the UK government has approached healthcare from the perspective of setting health targets, so assessment and evaluation on a population basis have become part of service performance monitoring and are used to inform effective service development.

Outcome measures and needs assessment

The Health of the Nation strategy set health outcome targets in five key areas, of which mental health was one (Department of Health, 1994). Of the three targets set in this area the first was to improve significantly the health and social functioning of people with mentally illnesses.

As a consequence the Royal College of Psychiatrists’ Research Unit was commissioned to develop the Health of the Nation Outcome Scales (HoNOS; Wing et al, 1996), which are now widely used in the UK and elsewhere. They have been incorporated into the minimum data-set that will become the common standard for mental health information in England and Wales (Glover, 2000).

The assessment and evaluation of social care needs is already embedded in practice and policy in England and Wales. The care programme approach (CPA) describes the process for ensuring that the health and social care needs of every person with a mental illness are assessed by a care coordinator and are addressed in each individual’s care plan. This policy has been explicit for over a decade, but still lacks an agreed and validated systematic approach to needs assessment, about which there is much debate. In a recent systematic review Gilbody et al(2003) examined whether providing feedback to clinicians on the results of outcome measures and needs assessments affected the subsequent management and outcome of people with schizophrenia and related disorders. They concluded that:

‘the use of outcomes measurement in the context of routine care of those with schizophrenia is unsupported by robust clinical evidence of the clinical and cost-effectiveness of this strategy’.

Whether needs assessment and evaluation are a necessary component of the care and recovery process remains a hotly contested issue among practitioners. However, policy makers continue to drive the collection of routine outcome measures for use in the planning of care for people with chronic mental illness. While the debate continues, clinicians require a working knowledge of needs assessment and evaluation to locate their practice within a contemporary context.

Regular assessment of the health and social care needs of people with severe mental illness is a statutory requirement under the Mental Health Act 1983, the National Health Service and Community Care Act 1990 and the Carers (Recognition and Services) Act 1995. However, the Mental Health Act Code of Practice (Department of Health, 1999a) gives no guidance on how clinicians should undertake this. Furthermore, there have been significant changes in the philosophy and practice of mental health services since the introduction of this legislation. The perspectives introduced by the widespread use of advocacy and by the recovery movement have been influential, and working in partnership with service users and their families is now expected in both assessment and service delivery. The traditional power imbalance between service users and mental health practitioners needs to be redressed. The National Service Framework for Mental Health (Department of Health, 1999b) has been helpful in supporting this shift in practice from ‘doing to’ towards a more collaborative process of ‘doing with’.

Guidelines published by the National Institute for Clinical Excellence (now the National Institute for Health and Clinical Excellence; NICE) on assessment of people with schizophrenia (National Institute for Clinical Excellence, 2002) offer a case in point. The Guideline Development Group included service user representatives, who argued from a recovery perspective and were instrumental in the inclusion of social criteria in the recommended audit tool. This specifies that assessment should improve the experience and outcomes of care for people with the disorder (Box 1). The predominantly social outcomes push the balance of assessment and evaluation towards quality of life issues and social inclusion.

Box 1

NICE guidelines on schizophrenia

‘The assessment of needs for health and social care for people with schizophrenia should . . . be comprehensive and address medical, social, psychological, occupational, economic, physical and cultural issues.’

(National Institute for Clinical Excellence, 2002: p. 6)

The proposed outcomes to be assessed include:

  • the degree of symptomatic recovery

  • quality of life

  • the degree of personal autonomy

  • ability and access to work

  • stability and quality of living accommodation

  • the degree and quality of social integration

  • the degree of financial independence

  • the experience and impact of side-effects

Definition of need

Clinical aspects of illness can be assessed using standardised needs assessment instruments that measure symptom-based outcomes. Assessment of the impact of an illness on an individual’s quality of life, social functioning, role functioning and service satisfaction requires patient-based measures. Box 2 shows formal definitions of need refined during the development of the Medical Research Council’s (MRC’s) Needs for Care Assessment (Brewin, 2001). These underpin the discussion on assessment schedules and their uses.

Box 2

MRC’s definition of need

  • Need is present when: (a) a patient’s functioning (social disablement) falls below or threatens to fall below some minimum specified level; and (b) this is owing to a remediable, or potentially remediable, cause.

  • A need (as defined above) is met when it has attracted some at least partly effective item of care, and when no other items of care of greater potential effectiveness exist.

  • A need (as defined above) is unmet when it has attracted only partly effective or no item of care and when other items of care of greater potential effectiveness exist.

(Brewin, 2001: p. 282)

What is the evidence for the use of systematic needs assessment?

Needs assessment schedules were developed to support the belief (rather than the evidence) that routine systematic monitoring of needs (followed by feedback and review) would be associated with improvements in care, at both the individual and the population level. The same belief underpinned the development of routine outcome measures in mental healthcare. The review cited above (Gilbody et al, 2003) does not confirm that systematic assessment reliably improves clinical outcome, even though some clinicians and researchers believe that it does.

Clinicians are reluctant to use standardised outcome schedules in routine clinical practice. In a national survey by Gilbody et al(2002) the majority of consultant psychiatrists did not use them in their day-to-day clinical work. The basic psychometric properties of validity, reliability and sensitivity to change of existing tools were seen as deficient.

Can assessment be useful?

People with chronic mental illness have complex needs and strengths across multiple health and social domains. Assessment of social care needs is a complicated task, involving the individual patient at the centre of the process, and their family, carers and multiprofessional care team. Assessment provides an opportunity to explore in a dialogue with the individual (and those who are important to them) how they see their life, their strengths and their difficulties. Putting the person at the centre of the process can promote collaboration, engagement and empowerment. Assessment should point to appropriate or evidence-based interventions to meet identified need.

Some of the functions of needs assessment are outlined in Box 3.

Box 3

The functions of needs assessment

  • To define health and social care needs at an individual level

  • To help care planning (need should predict intervention)

  • To monitor change in social care needs over time as a measure of the effectiveness of care planning for the individual

  • To define health and social care needs at a population level

  • To track changes in social care needs within services over time

  • To support research and evaluation

  • To guide service development and planning

What has changed to influence assessment of social care needs?

Over the past decade five influential themes have shaped the assessment of social care needs.

The endorsement of the biopsychosocial model of mental illness

The stress vulnerability model of Zubin & Spring (1977) is the most widely accepted integrative model of mental illness, positing an interaction between underlying vulnerability (biological) and social, psychological and environmental factors. Evidence for the effectiveness of this type of integrated approach to the care of mentally ill people comes mainly from Scandinavia, where the outcome of integrated treatment programmes for severe mental illness has been evaluated in three large multicentre outcome studies. Alanen (1997: pp. 139–190) has described the Finnish integrated model, the need-adapted approach to treatment of psychosis. Cullberg and colleagues in Sweden followed up programmes involving the integration of psychosocial treatments with low-dose antipsychotic medication (Cullberg et al, 2000). Johannessen et al(1999) carried out a multicentre study of early intervention that again relied on an integrated approach to treatment in which psychological and social interventions were combined with low-dose drug treatment. Thus, forms of psychosocial intervention were included in all three treatment programmes. If psychosocial interventions are necessary components of treatment, then social care needs assessment is essential in the development of an effective integrated care package.

Input from the user movement

Service users have argued strongly for an emphasis on assessment of social care needs. The influential Rethink survey of users’ views in the 1990s identified straightforward universal human needs such as money, accommodation, something to do and someone to talk to as central considerations for any assessment leading to the development of a care plan (Rethink, 2004).

The Sainsbury Report On Our Own Terms (Wallcraft et al, 2003) states:

‘There is no such thing as the ordinary service user. There [is] a wide range of personal experience of mental distress and use of services compounded by different racial and cultural backgrounds, gender, age, sexuality, disability and social class’ (p. 38).

This makes the case for a holistic social needs assessment, if the process is going to be relevant to the individual with chronic mental illness, their families and the services they use. Using schedules developed in partnership with service users may be more acceptable.

The recovery movement

There is a trend in practice and philosophy towards recovery-oriented services. This is most evident in rehabilitation services dedicated to providing care for people with chronic mental illness. The recovery approach involves building on the personal strength and resilience of individual service users, and on their cultural background, while recognising that there are many diverse routes to recovery. ‘Psychological wellness’ is defined not by the absence of symptoms of mental illness, but by the presence of valued roles and relationships and the satisfactory fulfilment of basic human needs. Two such needs are believed to be essential in pursuing healthy relationships among individuals and groups. The first is respect for diversity, which ensures that an individual’s unique identity can be affirmed by others, and the second is collaboration and democratic participation. These enable community members to have a voice in decisions that affect them, and promote both empowerment and social inclusion. The characteristics of those who have recovered include hope, empowerment, social connectedness and a subjective experience of having regained control over their own lives. So, assessments of social need should reflect community participation, opportunities for empowerment, support for diverse capabilities and freedom from discrimination.

Respect for diversity

In 2003 the National Institute for Mental Health in England published a report highlighting the need for change in the way in which Black people and those from minority ethnic communities gain access to mental health services and in the way in which services are delivered to them. It identified institutional racism and discrimination against Black people and those in minority ethnic groups. Keating et al(2002) highlighted the experience of Black people using mental health services, describing it as degrading and alienating, rather than empowering and inclusive. Footprints UK, a national user organisation working primarily with African–Caribbean service users, has raised significant concerns about care and treatment, including ‘the need for better assessment to promote more culturally acceptable interventions’ (Footprints UK, 2003).

The statement by Footprints UK points to the difficulty of developing nationally agreed assessment tools, as they must be both sensitive and specific for each ethnic or cultural group with which they are to be used. Moreover, they require specific agreement with local Black and minority ethnic community groups. Joint ownership of any assessment process is crucial to its utility and acceptability, but the need for specific measures in different communities will make a universal approach very difficult to attain.

Social inclusion

The overarching objective of social inclusion of people with mental health problems is to ensure that they do not face social barriers to achieving their individual goals or to participation in society. The promotion of social inclusion should be a guiding principle for local service development.

Multiple factors common to people with chronic mental illness hinder social inclusion: they tend to be poor, socially isolated, inadequately housed, unemployed and at risk of victimisation. People from minority ethnic groups are overrepresented. Service providers in the fields of housing, arts and leisure, and supported employment must engage with health and social care providers to ensure that meaningful social inclusion and community participation occur. Assessment of social care needs can be used to promote social inclusion.

Assessment tools

Surveys have shown high levels of unmet need in representative samples of people with severe mental illness, suggesting that many needs are overlooked by informal needs assessment. However, evidence to support systematic needs assessment (Gilbody et al, 2003) as the route to improved outcome is still lacking. Marshall et al(2004) used a cluster randomised controlled trial conducted in a routine care setting to investigate whether standardised assessment of need enhanced the effectiveness of care planning. Patients’ needs were assessed using the Cardinal Needs Schedule (Marshall et al, 1995). The findings were fed back to the care coordinators under the care programme approach. Nearly 80% of the group were followed up at 12 months. The only significant effect of the feedback to care coordinators was to improve patients’ satisfaction with services; it made no significant difference to clinical outcome. It has been argued that patient satisfaction may be taken as a proxy measure of quality of care. Service users may also find it a more acceptable outcome measure than reduction in symptoms or improvement in specific skills.

Most needs assessment schedules are based on the MRC Needs for Care Assessment (Brewin, 2001). This addresses the presence or absence of identified need and whether needs are met or not. Other schedules, such as the Cardinal Needs Schedule, go further and link an identified need to an evidence-based intervention.

The Camberwell Assessment of Need

The Camberwell Assessment of Need (CAN; Phelan et al, 1995) is a relatively short instrument intended for clinical and research purposes. It considers a range of health and social care needs across 22 domains. These domains are rated separately from interventions. The CAN includes ratings by both service users and staff and can be completed within 30 min. It can be used by a wide range of staff and can record the views of informal carers. It rates the amount and appropriateness of help currently received, so contributing to care planning.

The CAN has adequate psychometric properties in terms of validity and reliability. However, the degree of agreement between service users and staff on individual items is often poor.

A shortened version, the Camberwell Assessment of Need Short Appraisal (CANSAS), takes only about 10 min to complete and includes the 22 domains, rated on a 3-point scale (no need, met need or unmet need). It is suitable for assessing outcomes and for examining the profile of needs across a service and the case-loads of individual professionals.

The Cardinal Needs Schedule

This is a computerised modification of the MRC Needs for Care approach, providing a structured procedure for including service users’ and carers’ views (Marshall et al, 1995).

‘Cardinal problems’ are identified through the application of three criteria:

  • the cooperation criterion: the service user is willing to accept help for the problem

  • the carer stress criterion: the problem causes considerable anxiety or difficulty for the carer

  • the severity criterion: the problem endangers the health or safety of the service user or other people.

Information is collected using the Manchester Scale for Mental State Assessment, the Rehabilitation Evaluation of Hall & Baker (the REHAB Scale) for social function assessment, and two other schedules to collate the views of the service user (the Client Opinion Interview) and of a carer (the Carer Stress Interview). The interrater and test–retest reliability are generally good, as are the psychometric properties (Marshall et al, 1995).

Both the CAN and the Cardinal Needs Schedule have been influential in promoting a systematic approach to care planning at the level of the individual service user; if social need is identified it is linked with a relevant intervention.

The identification of unmet need is of central importance, both to the individual and for future service development. The use of aggregated data, particularly in areas of identified unmet need, can be a very powerful lever for service development and planning.

The Two-Way Communication Checklist

A different approach to systematic assessment has been taken by van Os and his team, who promoted patient-led needs assessment through the use of a checklist completed by the service user before consultation (van Os et al, 2002).

In a multinational randomised controlled trial, people with schizophrenia were allocated to standard care or to use of the Two-Way Communication Checklist, which they completed before seeing their clinician for routine follow-up. This checklist includes questions about health and social care needs (Box 4) and its purpose was to help patients to identify and discuss their needs during routine clinical care. This simple structured communication about perceived needs was related to an increased likelihood that a change in management would be recommended by the clinician. Although this checklist appears to have improved communication, there is no evidence that it improved clinical outcome.

Box 4

Examples of social need questions from the Two-Way Communication Checklist

  • Do you feel lonely?

  • Are you having problems getting on with your family or other people?

  • Do you have enough money for the things that you need?

  • Do you feel your life is boring?

  • Are you having problems with where you live?

  • Do you have problem findings things to do?

  • Is getting out or getting around a problem?

(van Os et al, 2004)

The Carers’ and Users’ Expectations of Services project

There are some alternative patient-led assessments and outcome measures. The Carers’ and Users’ Expectations of Services (CUES) project, part of the Department of Health’s collaborative Outcomes of Social Care for Adults initiative (Department of Health, 2001), has generated a set of useful assessment schedules that enable service users to inform practitioners about their social functioning and outcomes of care over time. The CUES assessment can be used to improve care management and care planning, and aggregated data provide information from the service users’ perspective on quality monitoring, evaluation and planning.

The service user version of the CUES covers 16 items, identified by service users as priorities (Box 5). Each item is introduced with a normative statement, and the service user then rates their experience and satisfaction, together with any comments.

Box 5

The 16 items on the CUES project (service user version)

  • Where you live

  • Money

  • Help with finances

  • How you spend your day

  • Family and friends

  • Social life

  • Information and advice

  • Access to mental health services

  • Choice of mental health services

  • Relationships with mental health workers

  • Consultation and control

  • Advocacy

  • Stigma and discrimination

  • Your medication/drug treatment

  • Access to physical health services

  • Relationships with physical health workers

(Royal College of Psychiatrists et al, 2002)

The Lancashire Quality of Life Profile

Quality of life schedules in which many of the assessed domains relate to social integration/inclusion and access to work and leisure can also be helpful in assessing (and evaluating) social needs. The Lancashire Quality of Life Profile (LQOLP; Oliver & Mohammed, 1992) is a structured self-report patient interview designed for administration by clinical staff in community settings (Box 6). It is based on Lehman’s original Quality of Life Interview (Lehmann, 1988), which has been culturally adapted for the UK setting and developed in response to the policy requirement that all community care programmes assess the impact of their services on their clients’ quality of life. In this sense it is an outcome or evaluation measure rather than a needs assessment.

Box 6

Lancashire Quality of Life Profile

Consists of 100 items and takes about 1 h to administer. It assesses objective quality of life in categories, and life satisfaction on a 7-point Likert scale in the following nine life domains:

  • Work/education

  • Leisure/participation

  • Religion

  • Finances

  • Living situation

  • Legal and safety

  • Family relations

  • Social relations

  • Health

It includes a measure of general well-being and self-concept. It has adequate psychometric properties (content, construct and criterion validity, and test–retest reliability). A brief version is also available.

(Oliver & Mohammed, 1992)

The practice of assessment

The process and practice of assessment (Box 7) must be culturally sensitive, pursued using a partnership model and informed by the perspectives of the recovery movement and social inclusion. It should begin in the service user’s space, both psychological and physical (their home, if this is acceptable to them). It must identify strengths as well as difficulties, so that care planning can be based on consolidating the former as a route to tackling the latter. It should involve as many carers or relevant informants as possible. Multiple sources increase the reliability of the information gained. It should be recorded in a manner that respects confidentiality and should be readily accessible to those who need to use it to produce responsive care plans. Assessments need to be regularly reviewed and updated, as do the effectiveness and acceptability of related care plans. The information gained through a systematic assessment should be shared with all members of the team involved in the delivery of care; a usual way of achieving this is through a multidisciplinary case formulation meeting that includes the service user and family where appropriate. All participants have the opportunity to reflect on the findings and develop a coherent understanding of the service user’s social care needs. They can then work together collaboratively on a consistent set of care plans.

Box 7

The process and practice of assessment

Assessment should involve active engagement of and collaboration with service users and their families

Assessments should be:

  • tailored to meet individual requirements of service users be user-centred

  • culturally sensitive

  • strengths-based rather than deficits-directed

  • multidisciplinary and reflective

  • comprehensive, drawing on multiple sources of information

  • continuous/ongoing and subject to multi-professional review

  • conducted where the service user feels most comfortable

  • systematic and repeatable

Evaluation

As mentioned above, evaluation is the translation of assessment into quantifiable or numerical form. All the tools mentioned above provide numerical data that can be used for a variety of purposes, for example to evaluate outcome for individual service users or to evaluate the effectiveness of services. The desired outcome for a service and for those who use it is, explicitly, recovery. Therefore a successful service might be described as one that brings about recovery that is more rapid than might have occurred without it.

The recovery movement places a strong requirement for basic human and social needs to be addressed. It may be argued that the evaluation of social care needs can lead to the delivery of effective recovery-oriented services. Evaluation of social needs can therefore be seen as criteria (or standards) for an effective recovery-oriented service (Box 8).

Box 8

Main uses of evaluation of social care needs

  • Tracking individual recovery

  • Keeping an overview of unmet need from the perspective of the individual and the service

  • Service evaluation

  • Promoting effective service development

Tracking the recovery of the individual

Sequential evaluation (or tracking) of a service user’s social care needs can yield information on whether the interventions provided are effective in reducing those needs. Regular evaluation should identify unmet needs, leading to an examination of how these might be addressed. Quantifying the unmet social care needs of individuals (such as the lack of supported employment services) can provide data to support appropriate service development.

Service evaluation

Services may be evaluated on the basis of outcomes of the people using them, but the outcomes chosen need to be sensitive and specific and should be measured using instruments agreed in partnership with all stakeholders. This is not usually feasible outside of a research setting. An alternative approach to service evaluation is the measurement of satisfaction with services.

The CUES questionnaire mentioned above provides useful feedback on service satisfaction, and has been used to audit some services. Quality of life assessments may also be used. Audit of social care needs across a particular service, using a systematic tool such as the CUES questionnaire, can be used to promote improvements in the quality of service delivery.

Effective service development

The ideal method for the development of comprehensive and appropriate local mental health services is to use a standardised assessment of need or standardised outcome measures with all those identified as mentally ill within the catchment area. Services can then be developed so that they fit the aggregated needs of the population as closely as possible. This is aspirational for most community mental health services. In practice, a number of proxy measures for need (or outcome) are commonly used to inform service planning and development. Local needs are often estimated from epidemiological studies that provide evidence of the prevalence of psychiatric disorders. Data about national service utilisation (e.g. bed usage, case-loads) may be extrapolated to predict local service provision. The latter may be compared with policy implementation guidance (Department of Health, 2001) for the particular type of service (e. g. assertive outreach teams) and then subjected to deprivation weighting according to the local mental health index. None of these proxy measures are likely to be particularly sensitive to the range of social care needs of individuals within the locality. Related service development is likely therefore to lack some specificity, and hence both clinical and cost-effectiveness.

Systematic evaluation of social care needs can be used to monitor the health of a population. It can also facilitate a focus on service development to meet some of the political targets set for mental health services (see below).

Evidence-based service development and social inclusion

The case for supported employment services

People with chronic mental illness have much higher than average levels of unemployment and are already socially excluded as a result of their mental health problems. However, 30–40% of these people are capable of holding down a job (Ekdawi & Conning, 1994). Furthermore, many wish to be in some form of employment, but few mental health regions have supported employment schemes with sufficient capacity or flexibility to help everyone.

Expansion of supported employment is backed by evidence. A systematic review (Crowther et al, 2001) of the ways of helping people with severe mental illness to obtain work confirmed that supported employment schemes were more likely to get people back into competitive employment than was pre-vocational training.

Being in work enhances quality of life (Hatfield et al, 1992; Hill et al, 1996) and work is a powerful route towards social inclusion. Helping people to gain and sustain employment should be considered a valid ‘treatment’ in its own right (Posner et al, 1996). It also achieves many of the social inclusion targets set for mental health services. This is one example of the way that systematic social needs evaluation that links need with evidence of effectiveness can be a powerful lever to direct service development.

Conclusions

There is evidence that social environment plays a part in the aetiology of mental illness such as schizophrenia, and strong evidence that it plays a determining role in the course and outcome of the disorder. Assessment of social care needs (Box 9) is likely to remain central to care planning and service development, despite the lack of robust evidence to date that it is linked with improved outcome. Although many assessment tools (e.g. the CAN) were developed before the major changes in both philosophy and policy within mental health services they retain considerable clinical utility and face validity in a contemporary context. Some user-developed instruments (such as the CUES) are now in use and have a contribution to make to care planning at the level of the individual, and to appropriate and responsive service development at the population level. Social needs assessment is a collaborative process pursued in partnership with people with chronic mental illness and their families; it should reflect changes in socio-political culture and embrace the recovery and social inclusion agenda. Social needs assessment and evaluation can then inform the development of modern socially inclusive and effective services for people with chronic mental illness.

Box 9

Useful schedules

For routine clinical practice

For research

For service evaluation

Declaration of interest

None.

MCQs

  1. The following make needs assessment a statutory requirement:

    1. the National Health Service and Community Care Act 1990

    2. the Mental Health Act 1983

    3. NICE Guidelines for Schizophrenia 2002

    4. the National Service Framework for Mental Health 1996

    5. Carers (Recognition and Services) Act 1995.

  2. There is good evidence that systematic needs assessment improves the following:

    1. clinical outcome

    2. patient satisfaction

    3. cost-effectiveness

    4. service development

    5. access to services.

  3. The following themes should inform needs assessment in a contemporary context:

    1. respect for diversity

    2. the recovery perspective

    3. carer burden

    4. patient participation

    5. social inclusion.

  4. The following are based on the MRC Needs for Care Assessment:

    1. the Two-Way Communication Checklist

    2. the CAN

    3. the CUES

    4. the Cardinal Needs Schedule

    5. the Lancashire Quality of Life Profile.

  5. Needs assessment can inform:

    1. service evaluation

    2. individual care planning

    3. the health of a population

    4. performance monitoring

    5. cost-effectiveness.

12345
aTaFaTaFaT
bTbTbTbTbT
cFcFcTcFcT
dFdFdTdTdT
eFeFeTeFeF

MCQ answers

References

View Abstract