Microsoft’s AI outperforms doctors in complex cases

Microsoft’s AI diagnostic system has been shown to correctly diagnose complex medical cases in a simulation more than twice as often as generalist physicians, according to a study published in Science. This system, known as MAI-DxO, works by coordinating several AI agents that act like a panel of doctors. They question a patient, order tests, and debate a diagnosis. In a simulation using complex cases from the New England Journal of Medicine, the system was correct 83% of the time, while primary care doctors and general internists, working individually with reference materials, were correct 36% of the time. The study’s own data suggests that much of the improvement comes from more powerful underlying AI models, with Microsoft’s orchestration system adding gains on top.

Microsoft tested MAI-DxO with models from several companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek, but its best results came with OpenAI’s GPT-5 and GPT-5.5. The study’s own data suggests that much of the improvement comes from more powerful underlying AI models, with Microsoft’s orchestration system adding gains on top. Microsoft measured DxO on cost as well as accuracy, meaning the estimated cost of running the AI on each case. On its own with a short, basic prompt, GPT-5 scored 78.9% on complex cases at about $8,400 per case. DxO cut that cost by about $3,000 but raised accuracy by only about four points.

DxO helps most on complex cases and with less capable AI models. Gemini 2.5 Flash, for example, scored 46.4% on its own but nearly 69% with DxO. Even so, the gap between the weaker and stronger models was larger than the boost DxO provided. Microsoft’s researchers said that accuracy gains naturally shrink as models approach the theoretical maximum.

Microsoft’s study in Science coincides with a manifesto of sorts around what it will take to build medical superintelligence, defined as ‘the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care.’ Microsoft hopes to move beyond benchmarking individual tasks, like answering medical exam questions and generating diagnoses, into integrating with clinicians and healthcare systems for better outcomes.

Through a new partnership with Beth Israel Deaconess Medical Center in Boston, Microsoft will evaluate the clinical reasoning capabilities of DxO across different hospital settings. This technology is ready for real-world evaluations, and you can imagine the complexity and diversity of issues that come from a Boston emergency department on a Friday evening. Google published a clinical study alongside Beth Israel in The Lancet this month to evaluate Google’s AMIE conversational diagnostic AI in real-world urgent care visits, while human physicians monitored the interactions in real time.