Microsoft AI: Diagnostic system outperforms generalist physicians in complex case simulation

Microsoft AI has unveiled a new diagnostic system that demonstrated superior accuracy in handling complex medical scenarios compared to individual generalist physicians. In a study published in Science on Thursday, the company’s multi-agent framework, known as MAI-DxO, correctly identified diagnoses in difficult cases 83% of the time. By contrast, primary care doctors and general internists working independently with standard reference materials achieved a 36% success rate. Mustafa Suleyman, CEO of Microsoft AI, described the findings as a critical step toward achieving 'medical superintelligence.'

The system operates by coordinating multiple AI agents that function similarly to a medical panel. These agents interact with simulated patients by asking questions, ordering necessary tests, and debating potential diagnoses before reaching a conclusion. The study utilized complex cases derived from the New England Journal of Medicine to test the system's limits. It is important to note that these specific cases are typically managed by specialists or physician teams in real-world settings, rather than by the generalists used for the comparison in this simulation.

Dominic King, vice president of Health at Microsoft AI, emphasized the significance of the results in addressing current healthcare challenges. 'There is something very substantial and exciting here,' King stated. He noted that the primary issue in modern healthcare delivery is the widening gap between the demand for medical expertise and the available supply. By publishing these results, Microsoft joins peers such as Google and OpenAI in asserting that their AI models can match or exceed human performance in diagnostic tasks.

Analysis of the data indicates that much of the performance improvement stems from the underlying power of the AI models themselves, with the orchestration system providing additional gains. Microsoft tested the framework using models from several providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest performance was observed when the system utilized OpenAI’s GPT-5 and GPT-5.5 models. The company also evaluated the cost-efficiency of the system, measuring the estimated expense of running the AI for each case.

Cost and accuracy metrics revealed nuanced results. When using GPT-5 with a basic prompt, the system scored 78.9% on complex cases at a cost of approximately $8,400 per case. The addition of the DxO orchestration reduced this cost by about $3,000 while increasing accuracy by only four points. On emergency department cases, there was minimal difference in accuracy or cost between the standalone model and the orchestrated system. With the newer GPT-5.5 model, released in April 2026, accuracy on complex cases jumped from 72% to nearly 87% with the orchestration, though gains in emergency settings remained negligible.

The orchestration system proved most beneficial for complex cases and when paired with less capable models. For instance, Gemini 2.5 Flash improved from 46.4% accuracy to nearly 69% when integrated with the system. However, the performance gap between weaker and stronger models was larger than the specific boost provided by the orchestration. Researchers noted that accuracy gains naturally diminish as models approach theoretical maximums, with a spokesperson adding that the system helps optimize both accuracy and cost simultaneously.

Looking ahead, Suleyman stated that Microsoft is collaborating with the Mayo Clinic to train a healthcare-specific foundation model, with a release planned for next year. He expressed confidence that this will enable significant leaps in medical AI, predicting a 'coding style breakthrough' for medical records in 2027. This advancement would allow for the streaming of highly accurate predictions regarding how a patient's medical record might evolve, providing valuable foresight to clinicians.

Suleyman, who previously co-founded DeepMind and later joined Microsoft in 2024, believes that healthcare models will improve by an order of magnitude in capability within the next 24 to 36 months. He cited the 'flywheel' effect seen in coding and chat models, where production use identifies gaps that are then patched to improve performance. To validate these capabilities in real-world settings, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. King described this upcoming evaluation as the 'biggest study of its kind,' which will compare the AI's recommendations against human doctors in both emergency and urgent care environments.