Outcome evaluation

Chapter 36 OUTCOME EVALUATION




LARGE-SCALE OUTCOME MEASUREMENT


Continual improvement in the quality and effectiveness of the treatment of all people with a mental illness is a major objective of Australia’s National Mental Health Strategy. Routine outcome measurements are undertaken to monitor the impact of services on people with mental health problems and to develop casemix systems. The National Outcomes and Casemix Collection (NOCC) comprises clinician-rated, consumer-rated and carer-rated instruments. The instruments applicable to older persons’ mental health services (OPMHS) are:








People with mental health problems find benefit in routine outcome measurement and believe it leads to improvement in care (Guthrie et al 2008).



Health of the Nation Outcome Scales (HoNOS 65+)


The Health of the Nation Outcome Scales (HoNOS) (Wing et al 1996) are a British development designed to gather information from key areas of mental health and social function for the purpose of comparing mental health services and monitoring changes both within individuals and within services. Although the HoNOS were developed for the full age range, by 1999, arguments had been put forward that the scales needed to be modified to be more suitable for the assessment of older people with mental health problems. Modifications and revisions were made to capture greater rating accuracy for depression and dementia, while the essential structure of the HoNOS was maintained. This work produced the HoNOS 65+ (Burns et al 2004, HoNOS 65+ Implementation Group 2002).


HoNOS and various derivatives have been in routine use in many Australian mental health services since 2000. It appears that, in this country at least, the data are being collected, but they are only utilised in very general terms for service evaluation, rather than clinical management or service development (Andrews & Page 2005). This may be due to a number of problems with data collection, including:






Overall, as an outcome measure for inpatient and community services, it has been established that the HoNOS 65+ is acceptable to mental health workers, has satisfactory reliability, is sensitive to change, and has good concurrent validity and interrater reliability. However, it lacks content validity, construct validity, predictive validity and test–retest reliability (Cheung & Strachan 2007, Spear et al 2002, Turner 2004). The HoNOS 65+ is a global assessment tool; therefore, if some areas of assessment require other information, a more specific screening and assessment tool should be used.


The HoNOS 65+ assessment is based on clinical judgment, taking into account the most severe symptom or problem during the previous 2 weeks. Assessment information can be obtained from the older person themselves, from clinical notes, through observation and interview, or via other informants such as family carers. The scale has 12 items, with each item contributing equally to the total score. Each item is rated as: 0 = no problem; 1 = minor problem, which requires no intervention; 2 = mild problem, which would need intervention; 3 = moderate problem; and 4 = severe problem. It can be completed by any one of the mental health disciplines and administration time is between 5 and 15 minutes.


The items are as follows (HoNOS 65+ Implementation Group 2004):














There are a few inpatient/community studies that have reported on the use of the HoNOS 65+. McKay and McDonald (2008) described the experience of using the HoNOS 65+ in a small integrated OPMHS in Sydney, Australia. The HoNOS 65+ data had been collected for many years, but they were not being utilised at a local level. A service review recommended integrating outcome measures into routine clinical practice to enhance individual care, team and service operations. At the individual care level, a ‘key item’ score is derived by clinical team consensus regarding the older person’s overall functioning and the main clinical issue that involves the OPMHS. The ‘key item’ is the most prominent HoNOS 65+ item (e.g. if the admission is for aggressive behaviour it will be item 1).


The advantages of using the HoNOS 65+ were identified as focused care planning, comprehensive data collection and analysis on a regular schedule (admission, review and discharge), evaluation of progress and measuring change in mental health status. At the clinical team level, improvements were noted as improved understanding of the clinical issues faced by the team, factors that influenced outcomes with the team and how effectively the team was performing. In regard to service management, the HoNOS 65+ data were utilised to better characterise the service users, reasons for admission, length of stay and benchmarking activities with other OPMHS.


In another Australian study, Spear et al (2002) found the HoNOS 65+ is sensitive to change and that the community-based OPMHS was having a positive impact on treatment for mental health problems. A study conducted in the United Kingdom (MacDonald 2002) established that the HoNOS 65+ can determine:







Life Skills Profile 16 (LSP–16)


The Life Skills Profile 16 (LSP–16) is a shortened version of the LSP–39 (Rosen et al 1989). It measures the level of disability experienced by people with a mental health problem, particularly those living in the community. It is a 16-item scale administered by a mental health worker. The LSP–16 has four subscales, examining the domains of withdrawal, self-care, compliance and antisocial behaviour. The reference period is the previous 3 months. Information is obtained by direct observation or from other people who are in contact with the older person. It can be completed in 5 minutes. The scale has been rated well for clinical utility and feasibility (Pirkis et al 2005). Although recommended for use with older people in the NOCC (Pirkis et al 2005), it does not appear to have been tested specifically with OPMHS users.



Resource Utilisation Groups–Activities of Daily Living (RUG–ADL)


The Resource Utilisation Groups–Activities of Daily Living (RUG–ADL) is a component of the casemix tool RUG–III developed for use in North American RACFs (Fries et al 1994). It was conceptualised from the point of view that diagnosis is a poor predictor of resource use. Factors like objectives of care, functional deficits, rehabilitation problems and behavioural problems are better predictors. The RUG–ADL focuses on basic activities of daily living, such as toileting, bathing, eating and mobility. These four items are rated by a mental health worker and the reference period is ‘current status’. Psychometric testing has centred around the RUG–III as a casemix tool, rather than the RUG–ADL as an outcome measure. With that in mind, its content validity has been criticised for not being adequate for mental health problems such as dementia. It has good interrater reliability, but there has been limited or no other standard psychometric testing (Pirkis et al 2005).

Only gold members can continue reading. Log In or Register to continue

Stay updated, free articles. Join our Telegram channel

Aug 6, 2016 | Posted by in GERIATRICS | Comments Off on Outcome evaluation

Full access? Get Clinical Tree

Get Clinical Tree app for offline access