Generative AI, which can create and analyze images, text, audio, video, and more, is increasingly making its way into healthcare, driven by both Big Tech and startups.
Google Cloud, Google's cloud services and products division, is collaborating with Highmark Health, a Pittsburgh-based nonprofit healthcare company, on generative AI tools designed to personalize the patient admission experience. Amazon's AWS division said it is working with an anonymous customer on how to use generative AI to analyze “social determinants of health” in medical databases. Microsoft Azure also helped nonprofit healthcare network Providence build a generative AI system to automatically triage messages sent by patients to healthcare providers.
Prominent generative AI startups in the healthcare space include Ambience Healthcare, which is developing generative AI apps for clinicians. Nabla, an ambient AI assistant for practitioners. Abridge creates medical document analysis tools.
The widespread enthusiasm for generative AI is reflected in investment in generative AI efforts for healthcare. Generative AI in healthcare startups has raised tens of millions of dollars in venture capital to date, and the majority of healthcare investors say generative AI is having a significant impact on their investment strategy. Masu.
But both experts and patients are divided on whether healthcare-focused generative AI is ready for prime time.
Generative AI may not be what people want
A recent Deloitte survey found that only about half (53%) of U.S. consumers believe healthcare can be improved through generative AI. For example, it may be easier to access or reduce waiting times for appointments. Less than half of people say they expect generative AI to make healthcare more affordable.
Andrew Borkowski, chief AI officer at the VA Sunshine Healthcare Network, the U.S. Department of Veterans Affairs' largest health care system, doesn't think the cynicism is unwarranted. Borkowski warned that generative AI has “significant” limitations and concerns about its effectiveness that could make its introduction premature.
“One of the key issues with generative AI is that it cannot handle complex medical queries and emergencies,” he told TechCrunch. “The limited knowledge base, i.e. lack of up-to-date clinical information and lack of human expertise, makes it unsuitable for providing comprehensive medical advice and treatment recommendations. yeah.”
Some research suggests there is credence to these points.
According to a paper published in JAMA Pediatrics, OpenAI's generative AI chatbot ChatGPT, which has been piloted by some medical institutions for limited use, has an 83% success rate in diagnosing pediatric diseases. Turns out I'm making mistakes. And when they tested OpenAI's GPT-4 as a diagnostic assistant, doctors at Boston's Beth Israel Deaconess and Medical Center found that the model got him wrong nearly two out of three times. We observed that we ranked diagnosis as the top answer.
Today's generative AI also struggles with healthcare management tasks that are part of clinicians' daily workflows. In the MedAlign benchmark, which assesses how well generation AI can perform tasks such as summarizing a patient's health record or searching through entire notes, GPT-4 failed in 35% of his cases.
OpenAI and many other generative AI vendors have warned against relying on their models for medical advice. But Borkowski and others say more can be done. “Relying solely on generated AI in the medical field can lead to misdiagnosis, inappropriate treatment, and even life-threatening situations,” Borkowski said.
Borkowski's concerns are shared by Jan Egger, head of AI-guided therapy at the Institute for Medical AI at the University of Duisburg-Essen, which researches the application of emerging technologies to patient care. He believes that the only safe way to use generated AI in the medical field at this time is under the close supervision of a doctor.
“The results can be completely wrong, and it's becoming increasingly difficult to remain aware of this,” Egger said. “Certainly, generative AI can be used, for example, to pre-generate discharge notes. But the physician is responsible for reviewing it and making the final decision.”
Generative AI could perpetuate stereotypes
One particularly pernicious way that generative AI in healthcare can get things wrong is by perpetuating stereotypes.
In a 2023 study from Stanford University, a team of researchers tested ChatGPT and other generative AI-powered chatbots on questions about kidney function, lung capacity, and skin thickness. Not only were ChatGPT's answers often wrong, some of the answers also reinforced the long-held false belief that there are biological differences between blacks and whites. and co-authors discovered. This deception has been known to cause health care providers to misdiagnose health problems.
The irony is that the patients most likely to be discriminated against by medically generated AI are also the ones most likely to use it.
Deloitte research shows that people without health insurance, typically people of color, are more willing to try out generative AI for things like finding doctors and mental health support, according to KFF research. Became. Inequalities in treatment can be further exacerbated if AI recommendations are compromised by bias.
However, some experts argue that generative AI is improving in this regard.
In a Microsoft study published in late 2023, researchers announced they achieved 90.2% accuracy on four difficult medical benchmarks using GPT-4. Vanilla GPT-4 could not reach this score. But the researchers say that through prompt engineering (designing prompts for GPT-4 to produce specific outputs), they were able to increase the model's score by up to 16.2 percentage points. (It's worth noting that Microsoft is a major investor in OpenAI.)
Beyond chatbots
But generative AI is useful for more than just asking questions to chatbots. Some researchers say medical imaging could greatly benefit from the power of generative AI.
In July, a group of scientists announced a system called Complementarity-Driven Clinical Workflow Deferral (CoDoC) in a study published in Nature. The system is designed to help medical imaging professionals decide when to rely on AI rather than traditional techniques for diagnosis. According to the co-authors, CoDoC outperformed experts while reducing clinical workflow by 66%.
In November, a Chinese research team demonstrated Panda, an AI model used to detect potential pancreatic lesions in X-rays. The study showed that Panda highly accurately classified these lesions, which are often detected too late for surgical intervention.
In fact, Arun Thirunavukarasu, a clinical research fellow at the University of Oxford, says there is “nothing special” that prevents generative AI from being adopted in healthcare settings.
“More routine applications of generative AI technology are possible in the short to medium term,” he said, “including text correction, automated documentation of notes and letters, and applications for optimizing electronic patient records. This includes improvements to search functionality.” “If generative AI technology is effective, there’s no reason it can’t be deployed quickly in these types of roles.”
“Exact science”
But while generative AI holds promise in certain narrow areas of medicine, experts like Borkowski believe that technical and compliance challenges are needed before generative AI can become useful and trusted as an all-purpose medical assistance tool. points out that the obstacles need to be overcome.
“The use of generative AI in healthcare has significant privacy and security concerns,” Borkowski said. “The sensitivity of health data and the potential for misuse or unauthorized access poses significant risks to patient confidentiality and trust in the health care system. Additionally, the regulatory and legal landscape surrounding the use of generative AI in health care is still evolving, and issues around liability, data protection, and medical practice by non-human actors still need to be resolved.”
Even Thirunavukarasu is bullish about generative AI in healthcare, but says there needs to be “rigorous science” behind patient-facing tools.
“Pragmatic randomized controlled trials should be conducted to demonstrate the clinical benefits that justify the introduction of generative AI for patients, especially in the absence of direct clinician supervision,” he said. Ta. “Good governance going forward will be essential to prevent unintended damage after large-scale implementation.”
The World Health Organization recently published guidelines recommending this type of scientific and human oversight of generative AI in healthcare, as well as independent third-party auditing, transparency, and impact assessment of this AI. The WHO's stated goal in its guidelines is to encourage diverse participation in the development of medical generative AI, giving them the opportunity to raise concerns and provide input throughout the process.
Borkowski said, “Until concerns are properly addressed and appropriate safeguards are put in place, widespread adoption of medically generated AI…could be potentially harmful to patients and the healthcare industry as a whole.” said.