Microsoft AI outperforms generalists in complex-case simulation

A study published in Science found that Microsoft’s MAI-DxO correctly diagnosed complex medical cases more than twice as often as generalist physicians in a simulation. The system achieved 83% accuracy on cases from the New England Journal of Medicine, compared with 36% for primary care doctors and general internists.

MAI-DxO coordinates several AI agents that work like a panel of doctors: they question a patient, order tests and debate a diagnosis. The comparison gave the physicians reference materials, but they worked individually.

The findings come with an important limitation: cases of this complexity are usually handled by specialists or physician teams, and neither specialists nor teams were included in the comparison. The results therefore show how MAI-DxO performed against individual generalists in this particular simulation—not how it compares with clinicians who routinely manage such cases.

Dominic King, Microsoft AI’s vice president of Health, described the results as substantial, pointing to a growing mismatch between demand for healthcare and the supply of medical expertise. The study also places Microsoft among companies including Google and OpenAI that have reported AI performance comparable to or better than doctors on diagnostic tasks.

Underlying models drive much of the performance

Microsoft tested MAI-DxO with models from OpenAI, Google, Anthropic, Meta, and DeepSeek. Its strongest results came with OpenAI’s GPT-5 and GPT-5.5, suggesting that the underlying model accounts for much of the system’s performance, with DxO adding gains on top.

DxO’s impact varied by model and type of case. With GPT-5, it raised accuracy on complex cases by about four percentage points while cutting the estimated cost by roughly $3,000 per case. With GPT-5.5, it lifted accuracy from 72% to nearly 87% on complex cases, at a slightly lower cost.

The orchestration system also appeared to help less capable models more. Gemini 2.5 Flash, for example, scored 46.4% on its own and nearly 69% with DxO. Still, the gap between weaker and stronger models was larger than the boost DxO provided.

On emergency department cases, DxO offered little benefit: GPT-5 showed little improvement in either accuracy or cost, while GPT-5.5 scored about 85% with or without DxO at a similar cost. Microsoft’s researchers said accuracy gains naturally shrink as models approach the theoretical maximum.

Microsoft plans clinical evaluations and a healthcare model

Alongside its work on DxO, Microsoft is working with Mayo Clinic to train a healthcare foundation model, with release planned for next year, CEO Mustafa Suleyman said. He described the model as part of the effort to make major advances in healthcare, while DxO is among the foundational infrastructure Microsoft is building to support breakthroughs.

Microsoft is also moving DxO into clinical evaluation through a partnership with Beth Israel Deaconess Medical Center in Boston. The study will assess the system’s clinical reasoning across different hospital settings, including emergency and urgent care environments.

Patients in the study will receive standard treatment, from triage and specialist consultations to investigations, treatment, and decisions about whether discharge is safe. In parallel, patients who consent will engage with DxO. Microsoft and its partners will then compare the system’s recommendations with those of the human doctors.

Microsoft’s health vice president, Dominic King, said the study would examine the technology in varied real-world settings, including the complexity of a busy emergency department. He said Microsoft expects the study to provide evidence about whether systems like DxO could be deployed at greater scale in the future.

Microsoft’s vision for medical superintelligence

Microsoft defines medical superintelligence as the capacity to surpass the strongest human-led health systems in medical reasoning and coordination across the full arc of care. The ambition goes beyond benchmarks for isolated tasks, such as answering exam questions or generating diagnoses: Microsoft wants AI to work with clinicians and healthcare systems to improve outcomes.

The company already sells dictation and radiology tools to clinicians and sees MAI-DxO as an expansion of that work. CEO Mustafa Suleyman says Microsoft must build state-of-the-art clinical decision-support products over the next few years. He expects healthcare models to improve by an order of magnitude in capability within 24 to 36 months, with progress accelerating as models are deployed, evaluated in real-world use, and refined.

Suleyman has pursued AI and medicine since his time at DeepMind. After Google acquired DeepMind in 2014, he and Dominic King launched DeepMind Health the following year. Microsoft’s vision also comes amid Google’s work on AMIE: Google published a study evaluating its conversational diagnostic AI in urgent-care visits, with human physicians monitoring the interactions. Microsoft, meanwhile, is pursuing clinical evaluations of its own system across hospital settings.