A large randomized trial found that an AI clinical support tool improved clinical documentation and treatment planning but did not significantly change short-term patient outcomes. An AI support tool has been shown to improve clinician decisions in a real-world primary care trial.
Let me show you something.
A pragmatic, cluster-randomized trial involving more than 9,600 patients across 16 primary care clinics in Kenya tested whether a generative AI support tool could improve patient-level outcomes in real-world clinical settings . The study, published in Nature Medicine on June 25, 2026, is one of the first randomized controlled trials worldwide to test whether generative AI can improve patient outcomes rather than just clinician performance in simulated cases .
The AI tool improved clinician decision-making quality—but the improvement did not translate into measurable short-term patient benefits. This finding is both reassuring and sobering for the future of AI in healthcare.
The AI support tool was tested in a randomized trial across 16 primary care clinics in Kenya.
This guide explores how the AI support tool improved clinical documentation and treatment planning.
Table of Contents
- The Trial Design and AI Tool
- Key Findings: What the Data Actually Shows
- Why Patient Outcomes Didn’t Improve
- What Did Improve: Documentation and Treatment Planning
- Safety and Patient Experience
- Global Relevance and Limitations
- FAQ
The AI Support Tool: Trial Design and Implementation
The AI support tool was embedded directly into the existing electronic medical record system.
The trial was conducted in 16 primary care clinics in Nairobi and Kiambu, Kenya, from April to July 2025 . A total of 9,691 patients were enrolled, overseen by 103 clinical officers — mid-level practitioners who deliver much of Kenya’s primary care .
Study design:
| Feature | Detail |
|---|---|
| Location | 16 primary care clinics, Kenya |
| Sample size | 9,691 patients |
| Clinicians | 103 clinical officers (52 AI-assisted, 51 control) |
| Duration | April – July 2025 |
| Journal | Nature Medicine |
The AI tool, known as “AI Consult,” was embedded directly into the existing electronic medical record system . During consultations, it worked in the background by:
- Analyzing information entered by the clinician into the medical record
- Generating context-specific diagnostic and treatment suggestions, aligned with Kenyan national clinical guidelines
- Flagging potential concerns using a simple color-coded alert system (green, yellow, or red)Â
Crucially, clinicians retained full autonomy. They were not required to follow the AI’s advice and retained responsibility for all diagnosis, prescribing, and referral decisions. The AI interface was not visible to patients, preserving normal patient-clinician interaction .
Key Findings: What the Data Actually Shows
Clinicians using the AI support tool showed significantly improved documentation quality. The AI support tool generated context-specific diagnostic and treatment suggestions for clinicians.
The trial’s primary outcome was treatment failure within 14 days of enrollment, adjudicated by an independent expert panel .
Primary outcome — treatment failure within 14 days:
| Group | Treatment Failure Rate |
|---|---|
| AI-supported care | 2.2% (102/4,693 patients) |
| Standard care | 2.0% (94/4,654 patients) |
Difference: Not statistically significant (adjusted odds ratio 0.77, 95% CI 0.55 to 1.08, P = 0.13) .
Senior author Professor Bilal Mateen, Honorary Professor of Machine Learning for Health at the University of Birmingham and Chief AI Officer at PATH, put it this way:
“What we found is reassuring but also sobering. The technology appears safe and clearly improves aspects of clinical decision-making, but translating those gains into measurable patient benefit is much more challenging, particularly in everyday primary care.”
Why Patient Outcomes Didn’t Improve
The AI support tool did not significantly change short-term patient outcomes.
The researchers identified a key reason for the finding: serious outcomes such as hospitalization or death are rare in primary care .
Professor Alastair Denniston, co-author and Professor of Regulatory Science and Innovation at the University of Birmingham, explained:
“A large part of primary care is to deal with common conditions, including those that are self-limiting, where many patients require low levels of healthcare intervention. In that context, even meaningful improvements in clinical reasoning may only result in small changes in patient outcomes that are very difficult to measure.”
The implication: Detecting modest patient-level effects would require trials involving more than 100,000 patients — a scale that is difficult to achieve in primary care settings where serious events are rare .
What Did Improve: Documentation and Treatment Planning
Documentation and treatment planning were the main areas where the AI support tool excelled. The AI support tool also reduced antibiotic-related costs through more cost-conscious prescribing.
Despite the null primary outcome, the AI tool delivered significant improvements in clinician-facing process measures .
Documentation and treatment planning improvements:
| Measure | Result |
|---|---|
| Appropriate diagnosis recording | ~70% higher in AI-supported group |
| Comprehensive notes | Significantly improved |
| Sound treatment plans | Significantly improved |
| Clinical guideline alignment | Better in AI-supported group |
An independent panel of experienced clinicians, blinded to whether AI had been used, found that AI-supported visits had significantly better clinical documentation and treatment planning .
Clinicians using the tool were about 70% more likely to log an appropriate diagnosis, a comprehensive note, and a sound treatment plan, with all three gains being statistically significant .
Antibiotic-related costs were also lower in the AI-supported group due to more cost-conscious prescribing choices, despite similar overall antibiotic prescribing rates .
Safety and Patient Experience
The AI support tool did not alter patient satisfaction or undermine trust in clinicians.
The trial found no evidence of harm from the AI tool.
Safety outcomes:
| Outcome | AI-supported | Standard care |
|---|---|---|
| Hospitalization rates | Similar | Similar |
| Death rates | Similar | Similar |
| Serious adverse events linked to AI | None | — |
Among 1,000 high-severity alerts generated by the tool, more than 90% were judged safe and appropriate on expert review, and no serious adverse event was linked to the tool .
Patient satisfaction was the same in both groups, suggesting the AI support did not alter patients’ experience of care or undermine patient trust .
Global Relevance and Limitations
Although the trial was conducted in Kenya, the researchers emphasize that the findings have global relevance, including for high-income health systems .
Key limitations:
| Limitation | Implication |
|---|---|
| Rare outcomes in primary care | Detecting modest effects requires >100,000 patients |
| Short follow-up (14 days) | Longer-term outcomes not measured |
| Single country | Generalizability to higher-income settings needs evaluation |
Professor Richard Riley, Professor of Biostatistics at the University of Birmingham, noted:
“Robust trials like this are so important to establish the real impact of using AI in practice. They help set realistic expectations of what AI can actually contribute within existing care pathways, and helps guide where future investment and research effort should be focused.”
FAQ
Q: Did the AI tool improve patient outcomes?
A: No. The trial found no statistically significant difference in treatment failure within 14 days between AI-supported care (2.2%) and standard care (2.0%) .
Q: What did the AI tool actually improve?
A: It significantly improved clinical documentation quality, treatment planning, and alignment with clinical guidelines. It also reduced antibiotic-related costs .
Q: Was the AI tool safe?
A: Yes. The trial found no evidence of harm, with similar hospitalization and death rates in both groups. No serious adverse events were linked to the tool .
Q: Did patients know they were receiving AI-assisted care?
A: No. The AI interface was not visible to patients, preserving normal patient-clinician interaction .
Q: Why didn’t improved clinician decisions lead to better patient outcomes?
A: Serious outcomes are rare in primary care. Detecting modest patient-level effects would require trials involving more than 100,000 patients .
Q: Does this study apply to healthcare settings outside Kenya?
A: The researchers suggest the findings have global relevance, but they caution that generalizability to higher-income settings needs to be evaluated .
Q: What did the AI support tool improve?
A: Documentation and treatment planning.
Final Thoughts
The AI Consult trial represents a rigorous test of generative AI in real-world primary care. The findings are clear: AI can improve clinician-facing process measures like documentation quality and treatment planning, but translating those improvements into measurable patient outcomes is far more difficult.
What’s clear:
- AI-assisted clinicians produced better documentation and treatment plans
- The tool was safe and did not harm patients
- Patient outcomes did not significantly improve in the short term
- Detecting modest patient-level effects will require much larger trials
The message from the researchers: AI can be integrated safely into real clinical workflows without undermining patient trust or clinician autonomy — which is a critical foundation for any future impact . The AI support tool trial represents a rigorous test of generative AI in real-world primary care. The AI support tool trial sets realistic expectations for what AI can contribute in primary care.
Related Posts on Pixelaizone
- Generative AI in Healthcare: MIRA and AMIE Outperform Physicians in 2026 Complete Guide
- How Medical AI Models Are Diagnosing Better Than Doctors in 2026
What do you think — should AI tools be integrated into primary care? Drop a comment below!
