PURPOSE OF THE EVALUATION

The objective of this study was to conduct an evaluation of the “National Guidelines for Seniors' Mental Health:

From Evidence to Implementation” of the Canadian Coalition for Seniors’ Mental Health (CCSMH) as part of the 5^th Anniversary National Mental Health Awards of the Mental Health Commission of Canada (MHCC).

In 2005, CCSMH was funded by the Public Health Agency of Canada to create Canada’s first interdisciplinary evidence-based guidelines in seniors’ mental health. The four guidelines focused on key mental health issues (depression, delirium, mental health issues in long-term care homes, and suicide prevention). Since their release in 2006, the guidelines have been widely disseminated, piloted across seven sites, and translated into booklets, toolkits, and quick reference pocket cards. The initiative was selected as an honourable mention in the Partnership category of the Awards and as part of its Awards package it was given an evaluation opportunity.

The overall purpose of the evaluation is to determine the initiative’s impacts and success, to highlight its innovative approach and design, and to support CCSMH and the MHCC in sharing best practices related to the initiative.

B. METHOD OF STUDY

The methodology undertaken to conduct the evaluation involved a number of steps, including:

 Conducting a preliminary review of documents (e.g., Awards application form);

 Undertaking a project launch meeting with the Project Manager and Executive Director of CCSMH;

 Developing an evaluation methodology report including a list of evaluation issues and questions,⁴⁹ and an interview guide;

 Conducting a detailed review of relevant documents that provide information on the initiative’s objectives, activities, performance, achievements and other impacts of the initiative (e.g., “Lessons Learned: Sharing the Successes and Knowledge about Guideline Implementation”);⁵⁰

 Undertaking interviews with 5 key stakeholders including the current as well as former Project Manager and Executive Director of CCSMH, 1 pilot project coordinator, and 2 researchers and clinical professionals involved in the development of the guidelines;

 Undertaking an analysis of all the quantitative and qualitative data obtained from the different lines of evidence and summarizing the data in response to each evaluation question; and

 Preparing this evaluation report which summarizes all the major findings and conclusions.

C. STRUCTURE OF THE REPORT

This document contains four chapters, including this introduction (Chapter I). Chapter II provides a profile of the initiative. Chapter III describes the major findings of the evaluation with respect to the initiative’s impacts and success to date, design, and knowledge exchange and evaluation activities. Chapter IV presents the major conclusions arising from the evaluation.

49 The evaluation questions are presented in Appendix I.

50 The list of documents reviewed is presented in Appendix II.

FERENCE WEICKER & COMPANY

II. PROFILE OF THE INITIATIVE

This chapter provides an overview of the initiative in terms of its objectives, activities, target groups, and organizational structure.

A. OBJECTIVES

Throughout the many phases of this initiative there have been moving targets for goals and objectives. These include to:

 Facilitate a process of collaboration and partnership where key leaders and identified stakeholders create a set of recommendations and/or guidelines for identified areas within seniors’ mental health;

 Disseminate completed guidelines to health care professionals and stakeholders across the country;

 Support the transfer of knowledge and uptake of guideline recommendations through the facilitation of pilot projects focused on guideline implementation;

 Expand knowledge translation activities to focus on seniors and families by creating revised versions of the guidelines that can be used by the public for personal education and to empower older adults;

 Create a comprehensive bilingual suite of knowledge products to assist with the usability of the CCSMH guidelines for health care professionals based on the lessons learned in earlier phases of the project;

 Engage seniors and families across Canada and raising awareness of the issues of seniors’ mental health, increasing knowledge and decreasing stigma in seniors’ mental health; and

 Create a better understanding of CCSMH guideline utilization based on a thorough and multifaceted evaluation plan.⁵¹

Stakeholders that were interviewed as part of this evaluation were asked to provide their opinion on what they saw as the primary objectives of the initiative. Stakeholders provided similar responses, emphasizing the development, dissemination, and up-take of the guidelines. Stakeholders noted that the initial objective was to develop evidence-informed, interdisciplinary guidelines on four topics⁵² in seniors’ mental health and to fill the knowledge gap nationally. Another objective was to ensure the evidence moved into practice and policy by developing tools and projects to support dissemination of the guidelines (e.g., Baycrest pilot project to introduce the long-term care guidelines into the facility). Stakeholders also emphasized that the purpose was to raise awareness and understanding of best practices in seniors’ mental health care, and to improve health care for older adults living with mental health problems.

B. ACTIVITIES

In 2005, CCSMH was funded by the Public Health Agency of Canada (PHAC) to create Canada’s first interdisciplinary evidence based guidelines in seniors’ mental health. The four guidelines focused on key mental health issues (depression, delirium, mental health issues in long-term care homes, and suicide prevention). In 2006, the guidelines were released and extensively disseminated. The CCSMH continued to receive funding from PHAC to ensure that the guidelines moved from paper to practice. This included piloting implementation across seven sites, creating companion guidebooks to distill the information to seniors and

51 MHCC. 2012. National Mental Health Awards Application Form – Canadian Coalition for Seniors Mental Health, “National Guidelines for Seniors' Mental Health: From Evidence to Implementation,” p. 5.

52 The Assessment and Treatment of Delirium; The Assessment and Treatment of Depression; The Assessment of Suicide Risk and Prevention of Suicide; and The Assessment and Treatment of Mental Health Issues in Long Term Care Homes (Focus on Mood and Behaviour Symptoms).

FERENCE WEICKER & COMPANY

their families, creating quick reference pocket cards for clinicians, and developing a comprehensive case-based toolkit focused on suicide prevention in late life.⁵³

The process for the many phases of the project has been iterative and based on responding to the needs of many target audiences and partners. Spanning over five years, the CCSMH received just over one million dollars from the Public Health Agency of Canada to create the National Guidelines, create knowledge translation products, build partnerships, support implementation and evaluate the process. Grants ranged from

$94,000 (to create the companion seniors and family guides) to $226,000 (to create and publish all four guideline documents). Specific activities undertaken for the project are described below.

 Development and launch of the four guidelines. From 2005 to 2006, four guidelines were developed including the assessment and treatment of delirium, depression, suicide risk and prevention, and mental health issues in long term care homes (with a focus on mood and behavior symptoms). The guidelines were launched in 2006. Launch activities included dissemination online and in print, translation into French, presentations, and a special supplement on the guidelines was published in the Canadian Journal of Geriatrics.

 Implementation and knowledge translation. CCSMH engaged in seven separate pilot projects across Canada (ON, AB, BC, QC, NS) to provide new knowledge on implementation of the guidelines.

Multiple types of stakeholders were engaged by these projects including service providers (family medicine, nursing, social work, occupational therapy, etc.), researchers and academics, family and advocacy groups, among others. Knowledge translation tools were also developed and disseminated to encourage up-take including DVDs, pocket cards, laminates, family guides, and toolkits.

 Evaluation. Following the completion of the pilot projects and knowledge translation, evaluations were conducted of the activities. To share the experiences of the CCSMH guidelines pilot projects, a report was released entitled: Lessons Learned: Sharing the Successes and Knowledge about Guideline Implementation.” In addition, evaluation of the guideline development process and utilization was conducted.⁵⁴

Although funding officially ended in March 2010, the CCSMH has been committed to continuing the knowledge translation phase of the project. This includes ongoing dissemination, identifying and filling gaps in knowledge translation products, facilitating partnerships across sectors, ongoing evaluation, and embarking on an update of the guidelines to ensure the evidence base remains robust. The true success of this long-term initiative has been the collaborative and passionate partnerships with champions of seniors’ mental health to ensure the best evidence is used for clinical care, policy, and program development.⁵⁵

C. TARGET GROUPS

According to the summary on the guidelines in the Canadian Journal of Geriatrics (2006), there are multiple target audiences for these guidelines. They include multidisciplinary care teams, health care professionals, administrators, caregivers, and policy makers whose work focuses on the senior population. In addition, these guidelines may prove useful in the development and/or evaluation of health care service delivery models,

53 MHCC. 2012. National Mental Health Awards Application Form – Canadian Coalition for Seniors Mental Health, “National Guidelines for Seniors' Mental Health: From Evidence to Implementation,” p. 4.

54 CCSMH Annual Reports 2008, 2009, 2010; accessed from: http://www.ccsmh.ca/en/about/annualReports.cfm

55 MHCC. 2012. National Mental Health Awards Application Form – Canadian Coalition for Seniors Mental Health, “National Guidelines for Seniors' Mental Health: From Evidence to Implementation,” p. 4.

FERENCE WEICKER & COMPANY

human resource plans, accreditation standards, training and education requirements, research needs, and funding decisions.⁵⁶