Secondary uses of patient information

Paul Lelliott is Director of the Royal College of Psychiatrists’ Research Unit (6th Floor, 83 Victoria Street, London SWIH 0HW, UK).

	Abstract

Top Abstract Problems with information... Secondary uses of patient... Multiple choice questions References

 
The pace of development and implementation of good information management systems has been disappointingly slow and the potential benefits of using patient information for secondary purposes have not been fully realised. The secondary use of patient data might be at the level of individual practitioners or teams, or at a local-service, regional or national level. Using patient data for purposes other than direct patient care requires an understanding of the principles of case-mix, standards for recording data, the limits of accuracy of data collected in routine practice and of the principles of data security and confidentiality. The alternatives to secondary use of information are to manage the mental health service without information about the people who use it or to gather this information through channels that run in parallel to the process of clinical care.

This paper continues the series on health informatics. Previous papers have looked at information organisation and communication (Lewis, 2002), confidentiality (McClelland & Thomas, 2002), knowledge management (Sensky, 2002), clinical governance (Palmer, 2002) and telemedicine (McLaren, 2003). The next paper will be on the use of routinely collected data.

Patient information should be organised and managed in a way that makes it accessible to those who have a need to know, including the recipient of care. It should be available when, where and in the form that it is needed. At the same time, adequate steps should be taken to ensure that confidentiality is not breached, other than in exceptional circumstances (McLelland & Thomas, 2002). Information should be complete and accurate, and wasteful duplication of data collection should be avoided. The information collected should be standardised as far as is consistent with the diversity of problems and needs of patients. This standardisation should provide the basis for the data to be used for a number of important purposes secondary to the direct care needs of the individual patient.

	Problems with information management

Top Abstract Problems with information... Secondary uses of patient... Multiple choice questions References

 
Direct patient care must suffer when clinical data are managed ineffectively or inefficiently. The negative consequences range from the irritation that patients feel when they have to recount the same information repeatedly to different care staff in the same care team, to the catastrophic failures that can occur when essential information is not communicated (Sheppard, 1996). Poor management of clinical data also limits the extent to which patient information can be used for the many purposes which, although secondary to direct patient care, can indirectly improve the quality of service.

Progress towards realising the benefits arising from good information management has been disappointingly slow over the past decade. Many obstacles have yet to be overcome (Box 1).

Box 1 Obstacles to the development of good clinical information management systems for mental health services (based on Lelliott, 1995) The evolution of mental health services from hospital-based to community-oriented. Staff now work in many dispersed locations and much care is now delivered in people’s homes. This poses both a technical and an organisational challenge to integration of information management. The fragmentation of services compounds this problem. Statutory-sector health and social services are working more closely together but the rate of integration is slow. Integration with the independent sector, which is now a significant provider of mental health care, has not begun. The constant reorganisation of services, including the frequent merger and demerger of trusts which manage mental health services and the emergence of primary care trusts, makes the long-term development of an integrated information system for services to a specific population difficult. The service ‘culture’ means that systematic data collection is not considered a priority for hard-pressed front-line workers, who are not adequately trained in information management (NHS Executive, 1999). A lack of investment in the information technology infrastructure and the administrative supports that underpin good information management. Managers usually decide strategy and procurement, too often resulting in the development of information systems whose main purpose is to support management and administrative rather than clinical functions. Central data requirements are outdated (Glover, 1995): the Hospital Episode Statistics (http://www.doh.gov.uk/hes/) and Körner returns provide information of little relevance to the monitoring or management of modern mental health services. The data cannot be linked within a care episode to build up a longitudinal picture of patients’ progress through care or between hospital and community or between health and social services, to allow the totality of care provided to be described.

 

The problem over the past decade has been not a lack of rhetoric but a failure of implementation. As a result, very few mental health services operate computerised information systems which are used routinely by clinicians and care workers to give direct support to their work with patients. Electronic records for patients remain a distant dream. The proposed new mental health minimum clinical data-set (MHMDS) has still not been implemented, 6 years after its conception (Glover, 2000). Although chief executives of trusts have been instructed to make returns based on the MHMDS by March 2003, drawing from existing databases, this is a far cry from the vision of a data-set that follows the rhythm of community care for individual patients (Glover, 1995). This could not be achieved without very substantial investment in information technology and information management support staff and a culture change in the National Health Service (NHS) to one that emphasises the benefits of information management for the support of clinical care rather than for the purposes of performance management.

In reality, the complete health and social care record is still largely paper-based and information about a person with complex mental health care needs is often dispersed over several locations which might be managed by different agencies. Information is rarely available at short notice, for example to the nurse or psychiatrist making an assessment of a person with a long-term illness who presents in crisis to an accident and emergency department.

There is, however, some cause for optimism. The NHS Information Management Group (IMG) has been replaced by the NHS Information Authority, and an NHS information strategy, Information for Health has been published (NHS Executive, 1998). The principles that underpin this strategy are the right ones (Box 2). However, they are also identical to those espoused by the IMG a decade ago (NHS Management Executive, 1992).

Box 2 Key principles of the NHS information strategy (NHS Executive, 1998) Information will be person-based Systems will be integrated Management information will be derived from operational systems Information will be secure and confidential Information will be shared across the NHS

 

It is possible to introduce a system for routine data collection, including routine measurement of outcomes across a national mental health care system. The federal and state governments in Australia have mandated a set of measures, which includes the Health of the Nation Outcome Scales (HoNOS; Wing et al, 1996), across all public and private care providers. The two essential ingredients that have enabled this in Australia, and that have been lacking in England, are consistent, long-term planning and proper investment.

	Secondary uses of patient information

Top Abstract Problems with information... Secondary uses of patient... Multiple choice questions References

 
The principal purpose of clinical data is to meet the information needs of the care workers who collect them and of the users of services. This should be reflected in the implementation of mental health information systems. Such systems should minimise the burden of collecting, recording, retrieving and manipulating information for clinical purposes. The quality of clinical information collected will depend on the extent to which clinicians, who ideally should input the clinical data, find an information management system easy to use and useful to them in their clinical work. This is true whether the information system is underpinned by paper, for example through well-organised, multi-disciplinary case notes which are accessible to all involved in a patient’s care, or by a network of computers. This is important because the quality and completeness of clinical data are the factors that most determine the extent to which information derived from them can be used for purposes other than the direct support of clinical care.

The range of such secondary uses is listed in Box 3. These can be considered at a number of levels, ranging from the appraisal and supervision of individual practitioners to the health of the whole population or the activity and performance of the NHS. Many of these uses require the aggregation of patient data. Specific examples of some of these secondary uses, at each of the levels, are given below. However, a number of general principles and skills need to be applied if the information derived from these data is to be made meaningful and acceptable at any level of use.

General principles and skills
An understanding of the limits of completeness, reliability and validity of data collected in routine practice
Trained assessors who use validated diagnostic interview schedules have acceptable levels of interrater reliability for determining psychiatric diagnoses. The assignment of diagnosis in routine clinical settings is far from this ideal (Glover et al, 1997). The task may be left to very junior medical staff and is sometimes carried out by unqualified coding staff. The problem is compounded by the high proportion of diagnostic codings which are missing altogether from central returns (Elphick et al, 1997). The same limits apply to other data that require a clinical judgement during collection, for example data relating to symptoms, problems, needs or outcomes. It has been argued that even the use of brief and simple measurement instruments requires training of raters and ongoing quality control if data are to be aggregated and/or used for comparisons (Dunn, 1996).

The need to agree data-recording standards
The training of psychiatrists imposes some uniformity on how they elicit and record clinical information. However, this does not extend to the aggregation of data derived from free-text clinical histories and mental states. For example, there is no nationally accepted set of standards stating which information items should comprise a personal history or a past psychiatric history, or of the terms used to describe these items. The problem becomes even greater when it is considered that nurses, social workers, psychologists and occupational therapists also assess people as part of their contribution to the work of a community mental health team. However, the NHS Executive’s Clinical Headings Project has begun to address this problem by working towards an agreed set of headings for a clinical summary (Lewis, 2002).

Concepts related to case-mix
When data about groups of patients are compared, it is important to ensure that like is being compared with like or that the effect of differences between the patient groups is taken into account. This applies whether the comparison is made to balance case-loads across a community mental health team, or to compare the activities or outcomes of different teams or services. Both require an understanding of the effect of case-mix on service use.

Two major factors should determine the needs for, and outcomes of, care of people with a mental illness. The first is the nature of the problem and the second is its degree. The nature of a problem can be described in a number of ways, most obviously in terms of a diagnosis. The degree of a problem has a number of possible dimensions, including severity, risk and level of dysfunction. To complicate matters further, people with a severe and enduring mental illness often have a complex array of comorbid conditions (such as substance misuse or personality disorder), social disabilities and social handicaps.

The extent of the challenge facing those working on case-mix in mental health is illustrated by the difficulty of even reaching agreement on a reliable definition of a concept as broad and ‘high level’ as severe mental illness (Ruggeri et al, 2000). Despite this, early English work on case-mix showed that diagnosis, severity and complexity do predict use of resources as determined by use of hospitals beds or community services (Elphick et al, 1997). In that study, severity and complexity were determined by three simple measures: score on the HoNOS; whether the person was subject to a section of the Mental Health Act 1983; and whether he or she was a ‘new’ or an ‘old’ patient (with less than or more than 6 months’ contact).

More-recent work has tested ‘health benefit groups’ and ‘health care resource groups’ (Carthew & Page, 2000). Health benefit groups are ‘groups of people with similar healthcare needs who, if given appropriate treatment, have similar outcomes’. Draft health benefit groups are based on a combination of problem, severity and complexity (based on HoNOS scores) and other factors such as risk and physical disability. Health care resource groups ‘are groups of treatments that are more-or-less iso-resource and clinically homogeneous’. It is proposed that a health care resource group be composed of three parts: the setting, principally whether residential or not; the contact rate and type; and the intervention type.

The experience in both England and Australia has been that case-mix variables relating to the nature and severity of disorder are only weak predictors of resource use (Buckingham et al, 1998). It is probable that their effect is swamped by a third factor, the idiosyncratic behaviour of individual practitioners. It is likely that the first major contribution of case-mix analysis in mental health will be to elucidate this.

The need to ensure the security of clinical data
The new NHS information strategy (NHS Executive, 1998) insists that the ‘implementation of the policies and procedures necessary to protect the privacy of patients must become an overriding priority’ (p. 50). Complex issues of data security, confidentiality and ethics arise when patient information is put to secondary uses. This is particularly true when the information is in a form that allows the identification of patients and/or is shared with people from outside the immediate team providing direct patient care.

Health and social care workers are still waiting for clear guidance from the Department of Health and the professional bodies regarding the framework that will guide them through these complex issues. Until this is given, mental health services will have to rely on local protocols for data security. Those developing these protocols will need a good working knowledge of the Data Protection Acts 1984 and 1998 (the latter bringing the principles of the European Directive on Data Protection into national law), the Subject Access Amendment Order 1987; the Caldicott report (NHS Executive, 1997); the General Medical Council’s guidance on confidentiality (General Medical Council, 2000); and the Royal College of Psychiatrists’ good-practice guidelines on confidentiality (Royal College of Psychiatrists, 2000).

Examples of secondary use of information at different levels
These examples illustrate the secondary use of patient information at each of the levels shown in Box 3.

Box 3 Potential secondary uses of patient information By the individual practitioner or the clinical team: Education and training Clinical supervision Individual performance appraisal Workload monitoring, case-load management Monitoring of outcomes achieved by individual practitioners/teams Clinical audit Clinical research By local provider or commissioner organisations: Service planning and resource and workforce allocation Local performance management Contract monitoring Compilation of ‘registers’, e.g. of those at risk or who pose a risk to others Comparisons across localities Local investigations of adverse events Health services research At the regional or national level: Strategic planning To secure funding in the annual public spending round Resource allocation within health and social services To provide information as part of ‘open government’ Regional/national performance management Comparisons across services and as the basis for inspection Monitoring of national targets for health and social services Outcome indicators Monitoring of the health of the population or of sub-populations Regional/national investigations of adverse events Health services research/epidemiology

 

Individual practitioner or clinical team level
Clinical audit is perhaps the most common secondary use of patient information at this level. Unfortunately, the patchy quality and lack of standardisation of clinical information recorded in case notes limits the extent to which meaningful audit can be conducted using only data captured during routine clinical practice. Perhaps the major exception to this is information about drug treatments, because the rules governing the writing and dispensing of prescriptions impose the standardisation that is lacking elsewhere in the clinical record. This is particularly true for in-patients for whom drugs are dispensed by hospital pharmacists.

The potential value of prescription data is illustrated by the results of a multi-centre audit of prescribing of antipsychotic medication for in-patients (Harrington et al, 2002a). The information used in this large-scale project was collated by local staff using data that were already recorded in case notes and on prescription charts. The audit standards were derived from guidelines and consensus statements published by the professional bodies of a number of English-speaking countries. The standards were about the prescription of high doses of antipsychotic medication and the concurrent use of more than one antipsychotic drug (polypharmacy).

The main results were unequivocal: high-dose medication and polypharmacy are commonly prescribed and, when they are, practice often fails to adhere to the recommended standards. In particular, less than 10% of patients had been given an electrocardiograph before being prescribed a high dose and less than 20% of case notes recorded the fact that the patient had been informed if the dose of medication exceeded recommended limits.

Local provider or commissioner organisation level
Surprisingly little routine, secondary use of patient information is made at this level. Most data collected for routine monitoring and performance management purposes are obtained through separate channels parallel to those involving information used for clinical purposes. As a result, many mental health services, or even community mental health teams, are still unable to describe accurately the number of patients receiving care from their service and the intensity of that care, let alone its outcomes.

There are good examples, however, of ad hoc use of patient information that have influenced service development. One is the local review of serious incidents, including deaths of people in contact with services. When information about these events is pooled and discussed by those who have had contact with the patient, important lessons can be learned that lead to better care and services (Rose, 2000).

Regional or national level
The National Confidential Inquiry into Suicide and Homicide by People with Mental Illness extends and complements local reviews of serious incidents by using patient information held by local clinicians. Here the large number of incidents reviewed makes possible the identification and dissemination of important common themes (Appleby et al, 1999; Amos & Shaw, 2000). If it succeeds in establishing its national reporting system, the National Patient Safety Agency would extend this to other types of incident and to ‘near misses’.

The multi-centre audit of prescribing described above illustrates the potential for using patient information to make comparisons between services (Harrington et al, 2002b). Acute psychiatric wards in 44 NHS trusts participated in the audit. The variation between wards in the proportion of in-patients who were prescribed a high dose ranged from 0% to 50% and, in the proportion prescribed polypharmacy, from 12% to 71%. Simple measures of differences between the patient groups in the wards in different services could explain only a small percentage of this variance. Presumably, at least some of the remaining variance is explained by differences in the practice of clinicians. Such results provide the basis for discussion by local practitioners.

Another potential use of patient information at the national or regional level is in the derivation of indicators of the effectiveness of mental health services. The ultimate aim would be indicators that reflect meaningful outcomes for groups of patients and that could both comment on the performance of mental health services nationally and allow comparisons of the performance of individual services. A working group of the Department of Health has proposed a set of candidate outcome indicators for severe mental illness (Box 4; Charlwood et al, 1999). A number of these are based on information that is collected routinely in clinical practice.

Box 4 Proposed outcome indicators for severe mental illness (SMI) (Charlwood et al, 1999) Reduction or avoidance of risk of SMI Prevalence of SMI Detection of SMI In-patient admission on detection of SMI Maintenance of function and reduction of need for hospital admission HoNOS scores for a service-provider population of people with SMI Proportion of people with SMI lost to follow-up by specialist services Proportion of people with SMI discharged from follow-up by specialist services Proportion of people with SMI spending more than 90 days in a given year in in-patient care Restoration of function and reduction of relapse following hospital admission Longitudinal indicators of change in item, sub-scale and total HoNOS scores for people with SMI admitted to in-patient psychiatric care Hospital readmission frequencies for a resident population of people with SMI Promotion of independent living and well-being User-assessed, health-related quality of life for a service-provider population of people with SMI Prevalence of side-effects associated with maintenance neuroleptics within a service-provider population of people with SMI Paid employment status of people with SMI Financial status of people with SMI Accommodation status of people with SMI Sustaining a collaborative approach Summary of a measure of user satisfaction with specific mental health services, among people with SMI Percentage of Care Programme Approach care plans for people with SMI signed by the users Supporting carers Assessment of impact of SMI on carers Ensuring protection and good physical health of service user Mortality among people with SMI Use of non-psychiatric health care services by people with SMI Ensuring protection of carers, service providers and the public Number of homicides by people with SMI Incidence of serious physical injury resulting from assaults on staff and service users by people with SMI

 

The National Service Framework (Department of Health, 1999) is committed to identifying and applying such indicators, and strenuous efforts have been made to introduce performance indicators by March 2003.

The education agenda
Professional bodies have agreed an educational framework for health informatics (NHS Executive, 1999). Box 5 lists the key components of the module concerning secondary uses of clinical data and information. The Royal College of Ps ychiatrists’ curriculum for its membership examination is based on this module (Royal College of Psychiatrists, 2001). The real challenge, however, is the training needs of consultants, both as practitioners and as trainers. In this regard, a programme board has been established by the NHS Information Authority to oversee the implementation of ‘Working together with Health Information‘(NHS Executive Information Policy Unit, 1999), the education, training and development strategy linked to the ‘Information for Health’ strategy (NHS Executive, 1998).

Box 5 Expectations for learning about secondary uses of clinical data and information (NHS Executive, 1999) During the course of professional development health care professionals should: identify the information needs of different health care sectors explain the connection between data entry and data retrieval explain the concept of aggregation of clinical data and the reasons why it is done review the relationship between capture of data items and their subsequent use demonstrate the benefits and disbenefits, including fitness for purpose, of common aggregate data measures (e.g. contract minimum data-sets; groupings such as health resource groups, registers and clinical indicators) demonstrate what factors affect the analysis and interpretation of aggregated data (e.g. changes in case-mix, complexity of illness, models of care, lack of agreed standards for recording data) describe systems used to produce national statistics (e.g. hospital episode statistics) and their potential uses and misuses discuss the policies and procedures that need to be in place to support the secondary use of patient data for education evaluate a common educational tool that uses data derived from patient/health records (e.g. log books) use report generators and undertake standard searches by topic, frequency and ad hoc queries undertake a clinically relevant task using aggregated data (e.g. a needs assessment, a service plan or a clinical audit)

 

	Multiple choice questions

Top Abstract Problems with information... Secondary uses of patient... Multiple choice questions References

 

1. An information strategy for mental health services:
   
   1. should prioritise the collection of information for management purposes
      
   2. should ensure that information is accessible to those who provide care and have a need to know
      
   3. should support the provision of information to users of services
      
   4. depends principally on the introduction of a computerised system
      
   5. should minimise duplication of data collection.
      
   
   
2. The development of a good clinical information management system is helped by:
   
   1. the repeated reorganisation of mental health services
      
   2. the dispersed nature of modern mental health care
      
   3. system procurement and development being managed by the finance director
      
   4. the proposed introduction of a new minimum data-set reflecting the care process
      
   5. better integration between health and social services.
      
   
   
3. In the case-mix of groups of patients:
   
   1. both the severity and degree of the patients’ problems must be considered
      
   2. the presence of comorbid conditions must be taken into account
      
   3. diagnosis, severity and complexity are strong predictors of resource use
      
   4. health benefit groups are groups of treatments that are more or less iso-resource and clinically homogeneous
      
   5. health care resource groups are groups of people with similar health care needs who, if given appropriate treatment, have similar outcomes.
      
   
   
4. Optimal use of patient data for secondary purposes:
   
   1. requires agreed standards for recording data
      
   2. is helped by the quality of the method currently used to record diagnosis
      
   3. requires training of clinical staff in the collection of information
      
   4. is supported by current guidance from the centre about issues relating to data security and confidentiality
      
   5. depends on local protocols concerning data security and confidentiality.
      
   
   
5. As regards secondary uses of patient information:
   
   1. they can be considered at a number of levels
      
   2. they are well-developed at the level of local mental health services
      
   3. there are examples of service developments based on secondary use of patient information
      
   4. there are now reliable indicators of service performance based on patient information which is collected routinely
      
   5. there is potential for developing outcome indicators based on data which are collected routinely.
      
   
   

	References

Top Abstract Problems with information... Secondary uses of patient... Multiple choice questions References

Appleby, L., Shaw, J., Amos, T., et al (1999) Safer Services (Report of the National Confidential Inquiry into Suicide and Homicide by People with Mental Illness). London: Department of Health.

Buckingham, B., Burgess, P., Solomon, S., et al (1998) Developing a Casemix Classification for Mental Health Services. Canberra: Commonwealth Department of Health and Family Services.

Carthew, R. & Page, R. (2000) Mental Health and Learning Disability Groups: An Update. Winchester: NHS Information Authority.

Charlwood, P., Mason, A., Go1dacre, M., et al (1999) Health Outcome Indicators: Severe Mental Illness (Report of a Working Group to the Department of Health). Oxford: National Centre for Health Outcomes Development.

Department of Health (1999) National Service Framework for Mental Health. Modern Standards and Service Models. London: Department of Health.

Dunn, G. (1996) Statistical methods for measuring outcomes. In Mental Health Outcome Measures (eds G. Thornicroft & M. Tansella). Heidelberg: Springer-Verlag.

Elphick, M., Anthony, P., Lines, C., et al (1997) Mental Health Report: Casemix Outcome Resources Needs (CORN). Winchester: National Casemix Office.

General Medical Council (2000) Confidentiality: Protecting and Providing Information. London: General Medical Council.

Glover, G. S. (1995) Mental health informatics and the rhythm of community care. BMJ, 311, 1038–1039.[Free Full Text]

Glover, G. S. (2000) The minimum data-set. At last – information! Psychiatric Bulletin, 24, 163–164.

Glover, G. S., Knight, S., Melzer, D., et al (1997) The development of a new minimum data set for specialist mental health care. Health Trends, 29, 48–51.

Harrington, M., Lelliott, P., Paton, C., et al (2002a) The results of a multi-centre audit of the prescribing of antipsychotic drugs for in-patients in the UK. Psychiatric Bulletin, 26, 414–418.

Harrington, M., Lelliott, P., Paton, C., et al (2002b) Variation between services in polypharmacy and combined high dose of antipsychotic drugs prescribed for in-patients. Psychiatric Bulletin, 26, 418–420.

Lelliott, P. (1995) Mental health information systems: problems and opportunities. Advances in Psychiatric Treatment, 1, 216–222.

Lewis, A. (2002) Health informatics: information and communication. Advances in Psychiatric Treatment, 8, 165–171.[Free Full Text]

McLaren, P. (2003) Telemedicine and telecare: what can it offer mental health services? Advances in Psychiatric Treatment, 9, 54–61.[Abstract/Free Full Text]

McClelland, R. & Thomas, V. (2002) Confidentiality and security of clinical information in mental health practice. Advances in Psychiatric Treatment, 8, 291–296.[Free Full Text]

NHS Executive (1997) Report on the Review of Patient-Identifiable Information (The Caldicott Committee). London: Department of Health.

NHS Executive (1998) Information for Health: An Information Strategy for the Modern NHS 1998–2005. A National Strategy for Local Implementation. London: Department of Health.

NHS Executive (1999) Learning to Manage Health Information. Bristol: NHS Executive South & West.

NHS Executive Information Policy Unit (1999) Working Together with Health Information: A Partnership Strategy for Education. Leeds: NHS Executive.

NHS Management Executive (1992) An Information Management and Technology Strategy for the NHS in England: IM & T Strategy Overview. London: Stationery Office.

Palmer, C. (2002) Clinical governance: breathing new life into clinical audit. Advances in Psychiatric Treatment, 8, 470–476.[Free Full Text]

Rose, N. (2000) Six years’ experience in Oxford: Review of serious incidents. Psychiatric Bulletin, 24, 243–246.[Free Full Text]

Royal College of Psychiatrists (2000) Good Psychiatric Practice: Confidentiality (Council Report CR85). London: Royal College of Psychiatrists.

Royal College of Psychiatrists (2001) Curriculum for Basic Specialist Training and the MRCPsych Examination (Council Report CR95). London: Royal College of Psychiatrists.

Ruggeri, M., Leese, M., Thornicroft, G., et al (2000) Definition and prevalence of severe and persistent mental illness. British Journal of Psychiatry, 177, 149–155.[Abstract/Free Full Text]

Sensky, T. (2002) Knowledge management. Advances in Psychiatric Treatment, 8, 387–396.[Free Full Text]

Sheppard, D. (1996) Learning the Lessons: Mental Health Inquiry Reports Published in England and Wales between 1969–1994 and their Recommendations for Improving Practice (2nd edn). London: The Zito Trust.

Wing, J. K., Curtis, R. H. & Beevor, A. S. (1996) HoNOS: Health of the Nation Outcome Scales. Report on Research and Development, July 1993–December 1995. London: Royal College of Psychiatrists.

This article has been cited by other articles:

				R. Bayney Benchmarking in mental health: an introduction for psychiatrists Advan. Psychiatr. Treat., July 1, 2005; 11(4): 305 - 314. [Abstract] [Full Text] [PDF]

				P. Rowlands The NICE schizophrenia guidelines: the challenge of implementation Advan. Psychiatr. Treat., November 1, 2004; 10(6): 403 - 412. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Lelliott, P. Search for Related Content PubMed Articles by Lelliott, P.