Researchers from the University of Liverpool have tested whether ChatGPT, an Artificial Intelligence-based chatbot, could be used for antibiotic prescribing.
ChatGPT is a chatbot developed by OpenAI and launched in November 2022. It uses an approach to transfer learning using both supervised and reinforcement learning techniques. It quickly became popular as it provides detailed responses and answers; however, its uneven factual accuracy has been identified. The technology is unique as it allows users to have human-like conversations with a chatbot which has begged the question of how it could be applied in healthcare.
The ongoing pressures and limited budgets within healthcare systems mean that researchers are looking for new technologies that can support medical practices. In a letter published in The Lancet, researchers from the Institute of Systems, Molecular and Integrative Biology have shown that Artificial Intelligence (AI) cannot replace GPs currently, but there is potential that this technology could play a key role in clinical practice.
Applying ChatGPT to eight doctor consultations
To understand the role ChatGPT plays in healthcare consultations, the researchers presented the AI software with eight hypothetical infection scenarios which people would commonly consult their doctor about.
They assessed the advice given by ChatGPT for its appropriateness, consistency, and impact on patient safety. The team found that the software can understand the scenarios and provided coherent answers, including disclaimers and signposting patients to further sources of advice. It also emerged that it could understand the need to only prescribe antibiotics when there was evidence of bacterial infection.
Unfortunately, the researchers found that ChatGPT provided unsafe advice in complex situations and where critical information was not provided correctly.
Does the software correctly prescribe antibiotics?
The AI appeared to focus on the type of antibiotic prescribed in each scenario rather than other factors, reflecting the assumptions often made by doctors during consultation.
Following the research, the Liverpool team have developed a checklist for standards that AI should meet to enable its use in clinical practice in the future.
Co-author of the letter Dr Alex Howard said: “It was fascinating to see the potential of artificial intelligence in healthcare demonstrated through this experiment testing ChatGPT’s ability to give antibiotic treatment advice.
“With the rise of antibiotic resistance poses a significant threat to global health, the ability of AI to provide accurate and safe treatment advice could revolutionise the way we approach patient care. We look forward to further exploration of this technology and its implications for the future of healthcare.”
