Microsoft AI Touts Diagnostic Accuracy

Microsoft is advancing a system designed to assist in medical diagnoses, reporting that its artificial intelligence tool correctly identified complex cases in a simulation at a significantly higher rate than human doctors. The company is positioning this technology as a foundational step toward what it calls medical superintelligence, aiming to build tools capable of surpassing current human-led health systems in reasoning and coordination across the entire arc of care. While the technology is still in the testing phase, Microsoft is actively pursuing partnerships to evaluate its capabilities in real-world medical settings.

The tool, known as MAI-DxO, operates by coordinating multiple AI agents that function together like a panel of physicians. These agents question a patient, order necessary tests, and debate potential diagnoses to reach a conclusion. In a simulation using complex cases sourced from the New England Journal of Medicine, the system achieved an 83% accuracy rate. By comparison, primary care doctors and general internists working individually with reference materials were correct 36% of the time. It is important to note that cases of this complexity are typically handled by specialists or teams of physicians, who were not included in this specific comparison.

Microsoft’s researchers reported that the majority of the improvement stems from more powerful underlying AI models, with the orchestration system providing additional gains. However, the system's effectiveness varies depending on the specific model and scenario. For example, on emergency department cases, there was little improvement in either accuracy or cost when using the orchestration system compared to using a basic model like GPT-5 directly. Conversely, for complex cases involving less capable AI models, the orchestration system provided a significant boost. Microsoft stated that accuracy gains naturally diminish as models approach theoretical maximums, but the tool assists with the broader challenge of optimizing both accuracy and cost.

Mustafa Suleyman, CEO of Microsoft AI, is leading the company’s efforts to develop medical AI capabilities. He stated that the company is collaborating with the Mayo Clinic to train a foundation model for healthcare, with a release planned for next year. Suleyman expressed his belief that such technology could enable huge leaps in capability within the next 24 to 36 months. He added that while the use of generative AI in healthcare is still in its early stages, progress accelerates quickly once a flywheel effect is established, similar to what has been observed in coding and chat applications. He believes that starting next year, this process will begin for medical records and clinical decision support.

To evaluate these capabilities, Microsoft has entered a partnership with Beth Israel Deaconess Medical Center in Boston to assess the clinical reasoning of the tool across various hospital settings. The evaluation will involve patients receiving standard treatment while consenting participants also engage with the AI system in parallel. Microsoft and its partners will then compare the AI's recommendations against those of human doctors. Suleyman emphasized the importance of this work, stating that the company aims to be the provider of the absolute number one state-of-the-art medical clinical decision support products over the next few years.