Barbara E. Barnes, MD, MS (Associate Vice Chancellor for Continuing Education and Industry Relationships, University of Pittsburgh)

Synopsis

Assessing the overall impact of an academic institution’s CME program on competency, performance, and patient outcomes can be inherently challenging given the number and diversity of activities offered. This work, which builds on that of Moore and others, describes a framework for categorizing activities that supports the assessment of how well an institution’s CME program is meeting the needs of learners and the educational mission of the organization.

Purpose

The purpose of this work is to develop and implement a system for categorizing CME activities based on their intended outcomes in order to demonstrate the impact of the entire CME program.

Methods

Based on the work of Moore and others, various levels of outcomes, ranging from knowledge acquistion to changes in patient care and population health were defined. Individual activities from a very large academic CME program were then grouped within these categories, based on characteristics such as learning format, intent of the intervention, and thematic areas that encompassed sequential activities. Depending on the level of intended change, various assessment methodologies were used and outcomes determined.

Results

The categorization system was useful for aggregating outcomes data among activities with similar intent and characteristics, permitting benchmarking within each category. The system was also valuable in evaluating the scope and impact of the entire CME program and in improving assessment methods for individual activities.

Applications

This system is useful in helping large and diverse CME programs better understand what is being accomplished across the entire spectrum of their activities. In addition, it provides guidance in allocating resources and determining methodologies for outcomes assessement. It will also be beneficial in helping CME providers meet ACCME accreditation requirements related to overall program evaluation.

Future Directions

This tool will be further refined and piloted in other institutions.