New reporting guidelines will ensure early studies on AI in medicine will provide the relevant data needed.
Artificial intelligence (AI) in medicine is the use of machine learning models that process medical data and give important insights, improving health outcomes and patient experiences. AI in medicine has shown promising results in simulation studies; however, it is yet to be employed in-patient care.
To accelerate the use of artificial intelligence in patient care, researchers developed a new guideline, DECIDE-AI. The guideline aims to improve the reporting of research on AI in medicine when they are employed in real clinical settings.
The reporting guidelines were jointly published in Nature Medicine and the BMJ by University of Oxford researchers.
DECIDE-AI: guidelines for AI in medicine
Despite numerous early studies analysing AI in medicine, the reporting data is inadequate. To challenge this, a new guideline was developed by Oxford researchers. It provides a multi-stakeholder, consensus-based reporting guideline for the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence (DECIDE-AI). It was built on feedback from over 150 experts across 18 countries, including computer scientists and clinicians.
Baptiste Vasey of Oxford University’s Nuffield Department of Surgical Sciences said: “While AI has shown promising results for clinical application in simulations, very few AI systems have had any significant effect on patient care so far. A major problem has been a lack of robust scientific evidence to back widespread use in clinical practice.
“This study is the first to clearly define the minimum reporting standards for the early-stage evaluation of AI-based decision support systems in clinical settings.”
Evaluating the quality of AI-related research
The factors DECIDE-AI evaluates are:
- proof of clinical value in a live care environment,
- the AI system’s safety profile and potential risks for patients,
- human factors around the use of the system, such as how the AI system is integrated into the existing care pathway and how it influences clinicians’ behaviour,
- the steps needed to develop the larger-scale trials, which will be needed to provide conclusive evidence necessary for an AI system to be accepted into routine patient care.
Ben Hornsby, Project Coordinator for the IDEAL Collaboration at Oxford’s Nuffield Department of Medicine, said: “DECIDE-AI is only the second reporting guideline dedicated to AI systems research, and the first ever to focus on the development stage of this technology.
“We chose to focus on early-stage evaluation results to emphasise the need for researchers to report transparently about four key aspects of AI research, to build confidence in the technology and highlight the practical benefits and improved efficiencies the technology can bring to healthcare.”
