Pharmafile Logo

Using AI Responsibly in CME Outcomes Reporting

October 1, 2026 |  

- PMLiVE

ICYMI at #AIS2026: using AI in outcomes reporting is a lot like onboarding a research assistant. There’s real efficiency in having the help, and a significant amount of background teaching, modeling, and guardrail-setting needed before you can trust what comes back.

That comparison landed the way it did because of where the rest of the Summit went. Several sessions dealt with how we communicate the value of IME initiatives to our internal stakeholders, and another focused on aggregating data across grants to provide just-in-time insights. All of that asks for more evidence, faster, for people who weren’t part of the original analysis.

So our session, “From Slide Decks to Systems: Using AI to Generate and Consume CME Outcomes Responsibly,” turned out to be a very relevant place to land.

Some of our key takeaways:

Curiosity and potential are outpacing trust and confidence in the technology. Researchers spend years learning to do this work rigorously, and right now it’s about changing the script from seeing AI as cheating to seeing it as a form of rigorous methodology when it’s used appropriately. Appropriate use is the key phrase.

There are two different things people mean when they say “we used AI,” and they need different guardrails. Substitution is work someone already did by hand, like coding open comments, cleaning data, merging files, or building the deck. There’s a known answer to check against. New capability is work that wasn’t possible before, like testing whether a finding survives every reasonable analytic path, or pooling results across a whole portfolio. There’s nothing to check it against, so expert review has to be built into the process rather than added at the end.

Here’s the case we walked through. A real program, self-reported outcomes came back modest, renewal was close, and there was pressure to show something valuable. So the dataset went to a model with one instruction: find what’s significant. What came back was 168 comparisons. Seven looked significant, which is about what you’d expect from chance alone when you run that many. None of them held up once we corrected for the number of tests. And the finding we could have led with came down to two respondents labeled cardiologists who were actually nurses.

None of it was fabricated. Someone just would have had to choose which result to put on the slide.

Here’s the part I keep coming back to. Supporters want to use AI to help communicate value to their internal stakeholders across an entire portfolio of grant funding. MECs and consultants want to use it to increase value and personalize what we deliver. Both of those are reasonable and both are already happening.

The hard part is the handoff. The supporter’s AI is reading the report, not the dataset behind it. Whatever context didn’t make it onto the page is gone by the time that number gets pooled with fourteen other programs.

So the practical question isn’t whether we use AI. It’s what has to arrive with the report so the number still means something when it hits the supporter’s desk: how many people it’s based on, what was measured and when, which numbers can and can’t be combined, and which parts were AI-assisted. Now, more than ever before, does a supporter have the ability to take that into consideration in their summation of the insights and value from what they funded with AI tools. 

All of this points back to needing the human in the loop, with the expertise required to guide the work and evaluate what’s coming back.

Thank you to our panel and moderator, and to everyone who worked through what could be on the checklist with us.

Stay tuned for an Almanac article with the checklist you helped us build: companion guidelines for what providers should consistently do in outcomes reporting, and a list of things supporters can confidently ask for.

 

This content was provided by Medscape Education

Company Details

 Latest Content from  Medscape Education 

Medscape Oncology Lights the Night to Advance Blood Cancer Care

MERCER COUNTY, NJ — Recognizing that clinical excellence and patient experience are inseparable, Medscape Oncology served as the official Caregiver Central Sponsor in support of Blood Cancer United’s Mercer County…

Reaching New Heights in Liver Care: The Power of Strategic Advocacy in Liver Health

“The view from the top is always better with the people who helped us get there.”

Medscape Education Wins “Partner of Hope Award” for Scleroderma Advocacy

Medscape Education receives National Scleroderma Foundation’s Rocky Mountain Chapter’s “Partner of Hope Award” for empowering the patient voice and shortening the diagnostic journey. This recognition honors Medscape Education’s significant commitment…

Redefining Pediatric Epilepsy Care: Why We Must Move Beyond Symptom Suppression

For families navigating pediatric epilepsy, the condition dictates every moment of their lives. The constant, crippling “anticipatory anxiety” of when the next seizure might hit transforms daily activities like grocery…

What 90 Minutes of Live AI Case Practice Revealed About Physician Readiness

At EHA 2026 in Stockholm, 425 hematologists and oncologists attended an independent, 90-minute Medscape symposium on the practical and ethical use of AI. The format was case-based. The faculty presented…

Rethinking Prenatal Migraine Management: The Clinical Case for Neuromodulation

Disclaimer: Management of migraine in pregnancy typically combines lifestyle strategies, nonpharmacologic approaches, selected pharmacologic options when benefits outweigh risks, and, increasingly, noninvasive neuromodulation technologies.

Advancing the Frontier of Cancer Care: Medscape’s Mission at ESMO 2026

For more than 30 years, Medscape Education’s dedication has been centered on a single goal: building independent, accredited oncology education that improves patient outcomes. This year, at the European Society…

Using AI Responsibly in CME Outcomes Reporting

ICYMI at #AIS2026: using AI in outcomes reporting is a lot like onboarding a research assistant. There’s real efficiency in having the help, and a significant amount of background teaching,…

The Pulse of AI Innovation: From More Data to Better Care

At the European Society of Cardiology (ESC) Congress, two of cardiology’s most influential voices sat down to talk about artificial intelligence at Medscape’s ‘The Pulse of AI Innovation’ event: Professor…

Next-Level Clinical Practice, Real-World Impact: Evolving Patient Simulation with the Clinical Readiness Studio

Healthcare is changing rapidly, bringing unprecedented data and complex treatment pathways to the point of care. Clinical mastery requires active experience, the confidence to make decisions when real patient outcomes…