Some CME Outcomes Measurement Challenges

July 26, 2010 at 5:53 pm Leave a comment

There are no shortages of long and short term measures we can use to assess the outcomes of our CME efforts.  We have to measure outcomes related to physician competence, or physician performance, or the health status of the patients our physician audiences care for. That is what the ACCME requires. What we measure depends on our CME mission and other expectations in the environments in which we work. But many CME operations struggle with measuring anything, especially the right things.

Outcomes from CME activities are inherently difficult to measure because they are often intangible and subjective. We frequently measure a number of short-term outcomes. Why? Because it is easier. Seldom do we measure long term outcomes.  While short terms measures are informative for a variety of reasons they may not be enough to meet our accreditation requirements. But measuring long term outcomes often leads to other problems, like the time involved in data collection, data overload, and the difficulty or failure to link cause and effect.

CME programs face a number of significant challenges when it comes to outcomes measurement.  I hear things like:

  • Outcomes are too difficult to measure.
  • Our outcomes measures don’t drive any improvement efforts, we do it just to meet ACCME  requirements
  • By the time we get the results of our measurements it is too late to do anything with them. We have already moved on to doing other CME activities.
  • Those of us in the CME operation can’t make meaningful decisions about the data. All we can do is report it to others. We are not positioned to effect change in our organization if the outcomes results suggest change is required.
  • We don’t have the money to do this. We’ll do as little as we can get away with to maintain our accreditation.
  • I really don’t know how to effectively measure outcomes.  I need help.

I wonder if these things are symptoms of deeper issues associated with where CME is often positioned in an organization. What do leaders in our parent organizations see as the function of CME? Do they recognize that CME is now required to be integrally involved in improvement initiatives?  Are there effective communications, systems, and structures to support this role for CME? Organizations that have CME as a function may need to spend more time defining roles, responsibilities, and decision rights for people in the CME unit as well as developing the capabilities of its CME people, the larger CME process, and technologies required for effective outcomes assessment and analysis. But is there a will to do that?

Some Things to Consider for Outcomes Measurement

  1. Keep it simple – Focus on a few critical measures.
  2. Measure the right things – Demonstrate outcomes that our physician customers and management care about.
  3. Engage Your Physician Customers – Increase participation of members of our target audience in planning the outcomes assessment. What do they think is reasonable to measure as a result of a specific CME activity?
  4. Everything must connect – Performance gaps, educational needs, instructional design and outcomes assessment must all align. This cannot be an afterthought. It has to be planned.
  5. Learn from Your Data– Use the outcomes you measures as a diagnostic tool. Learn how to ask the right questions of the data. Act on what you learn.
  6. Standardize some outcomes measurement throughout the CME program – Create a common structure for consistency in part of your outcomes assessment. This will make it much easier to query the data when the ACCME asks us to show the impact of our CME effort on achieving our CME mission.

It takes someone with the knowledge, skills, and practical experience to carry out outcomes measurement and to manage an effective outcomes measurement effort. Another skill set required of the effective CME professional.


Entry filed under: CME, CME Issues, Improvement, Physician Continuing Education. Tags: , , , .

Pharma Funding for CME? Why Bother? Selecting What Root Cause(s) to Address With Countermeasures

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


July 2010
« Jun   Aug »

Most Recent Posts

%d bloggers like this: