Should you trust it?
SINGAPORE (CNA) – Wondering if it’s a headache or a sinus infection? Curious about the sensation of a stress fracture? Concerned about chest pain? If you’re turning to Google for answers, you might find yourself face-to-face with artificial intelligence. Google’s latest feature, AI Overviews, harnesses generative AI to swiftly generate responses to your health queries by drawing on vast internet data.
However, since its recent launch, users have encountered a mixed bag of accuracy and oddities across various topics. When it comes to health inquiries, experts warn of heightened stakes. While the technology holds promise in directing individuals towards healthier habits or necessary medical attention, it also carries the risk of disseminating inaccurate information.
At times, the AI may fabricate facts, potentially steering users towards advice that contradicts medical guidance or poses health risks. As the answers may be influenced by sources lacking scientific grounding, it’s crucial to approach AI-generated health information with caution.
The system has already been shown to produce bad answers seemingly based on flawed sources. When asked “how many rocks should I eat,” for example, AI Overviews told some users to eat at least one rock a day for vitamins and minerals. (The advice was scraped from The Onion, a satirical site.)
“You can’t trust everything you read,” said Dr Karandeep Singh, chief health AI officer at UC San Diego Health. In health, he said, the source of your information is essential.
Hema Budaraju, a Google senior director of product management who helps to lead work on AI Overview, said that health searches had “additional guardrails,” but declined to describe those in detail. Searches that are deemed dangerous or explicit, or that indicate that someone is in a vulnerable situation, such as with self-harm, do not trigger AI summaries, she said.
While Google has remained tight-lipped about the specific websites supporting information in AI Overviews, the tool collaborates with the Google Knowledge Graph. This extensive system aggregates billions of facts from diverse sources, offering a foundation for the AI’s responses.
Although the new search feature does cite certain sources, such as the Mayo Clinic, WebMD, the World Health Organisation, and PubMed, it also draws from a variety of sources like Wikipedia, blogs, Reddit, and e-commerce platforms. However, it doesn’t delineate which facts originate from which sources.
Unlike standard search results, where users can discern reputable medical websites from others, AI Overviews amalgamate information from various sources into a single text block, potentially leading to confusion among users.
“And that’s if people are even looking at the source,” said Dr Seema Yasmin, the director of the Stanford Health Communication Initiative, adding, “I don’t know if people are looking, or if we’ve really taught them adequately to look.”
From her own research, she expressed skepticism on misinformation suggesting that the average user tends to opt for quick answers without delving deeper.
Meanwhile, cardiologist and Tufts University professor Dr Dariush Mozaffarian acknowledges that while the response regarding chocolate’s health benefits contains mostly accurate facts and summarises relevant research, it fails to differentiate between robust evidence from randomised trials and weaker evidence from observational studies. Additionally, it lacks any caveats regarding the evidence presented.
It’s true that chocolate contains antioxidants, Dr Mozaffarian said. But the claim that chocolate consumption could help prevent memory loss? That hasn’t been clearly proved, and “needs a lot of caveats,” he said. Having such claims listed next to one another gives the impression that some are better established than they really are.
The answers can also change as the AI itself evolves, even when the science behind a given answer hasn’t changed.
A Google spokesperson said in a statement that the company worked to show disclaimers on responses where they were needed, including notes that the information that shouldn’t be treated as medical advice.
It’s not clear how, exactly, AI Overviews evaluate the strength of evidence, or whether it takes into account contradictory research findings, like those on whether coffee is good for you. “Science isn’t a bunch of static facts,” Dr Yasmin said. She and other experts also questioned whether the tool would draw on older scientific findings that have since been disproved or don’t capture the latest understanding of an issue.
“Being able to make a critical decision – to discriminate between quality of sources – that’s what humans do all the time, what clinicians do,” said Dr Danielle Bitterman, a physician-scientist in artificial intelligence at Dana-Farber Cancer Institute and Brigham and Women’s Hospital. “They are parsing the evidence.”
If we want tools like AI Overviews to play that role, she said, “we need to better understand how they would navigate across different sources and how they apply a critical lens to arrive at a summary,” she said.
Those unknowns are concerning, experts said, given that the new system elevates the AI Overview response over individual links to reputable medical websites such as those for the Mayo Clinic and the Cleveland Clinic. Such sites have historically risen to the top of the results for many health searches.
A Google spokesperson said that AI Overviews will match or summarise the information that appears in the top results of searches, but isn’t designed to replace that content.
Rather, the spokesperson said, it’s designed to help people get a sense of the information available.
The Mayo Clinic declined to comment on the new responses. A representative from the Cleveland Clinic said that people seeking health information should “directly search known and trusted sources” and reach out to a health care provider if they’re experiencing any symptoms.
A representative from Scripps Health, a California-based health care system cited in some AI Overview summaries, said in a statement that “citations in Google’s AI generated responses could be helpful in that they establish Scripps Health as a reputable source of health information.”
However, the representative added, “we do have concerns that we cannot vouch for the content produced through AI in the same way we can for our own content, which is vetted by our medical professionals.”
For medical questions, it’s not just the accuracy of an answer that matters, but how it’s presented to users, experts said. Take the question “Am I having a heart attack?” The AI response had a useful synopsis of symptoms, said Dr Richard Gumina, the director of cardiovascular medicine at the Ohio State University Wexner Medical Centre.
But, he added, he had to read past a long list of symptoms before the text advised him to call 911. Dr Gumina also searched “Am I having a stroke?” to see whether the tool might produce a more urgent response – which it did, telling users in the first line to call 911. He said he would immediately advise patients experiencing symptoms of a heart attack or a stroke to call for help.
Experts encouraged people looking for health information to approach the new responses with caution. Essentially, they said, users should take note of the fine print under some AI Overviews answers: “This is for informational purposes only. For medical advice or diagnosis, consult a professional. Generative AI is experimental.”