Do you have a headache or a sinus infection? What does a stress fracture feel like? Should I be worried about my chest pain? If you Google these questions right now, the answers might be written by artificial intelligence.
This month, Google announced a new feature called “AI Overview,” which uses generative AI, a type of machine learning technology that is trained on information from across the internet to generate conversational answers to search questions in seconds.
In the weeks since the tool was released, users have encountered a variety of inaccurate or odd answers on a range of topics. But experts say there are particular risks when it comes to the way it answers questions about health. While the technology could guide people towards healthier habits or necessary medical care, it could also provide inaccurate information. AI could also falsify facts. And if the answers are shaped by websites that are not based on science, it could provide advice that contradicts medical guidance or poses dangers to a person's health.
The system has already been found to give incorrect answers that appear to be based on flawed sources: For example, when asked “How many rocks should I eat?”, AI Overviews advised some users to eat at least one rock a day for its vitamins and minerals (the advice was taken from satire site The Onion).
“You can't believe everything you read,” said Dr. Karandeep Singh, chief health AI officer at the University of California, San Diego Health Center. When it comes to health, he said, sources of information matter.
Hema Budaraju, a senior director of product management at Google who leads the AI ​​Overview effort, said there are “additional guardrails” for health searches but declined to elaborate. Searches that are deemed dangerous or explicit, or that indicate someone is in a vulnerable situation, such as self-harm, won't trigger an AI summary, Budaraju said.
Google declined to provide a detailed list of websites that support the information in AI Overviews, but said the tool works in conjunction with its existing information system, Google Knowledge Graph, which has extracted billions of facts from hundreds of sources.
The new search responses clearly state some sources. For health questions, these are often websites like the Mayo Clinic, WebMD, the World Health Organization and scientific research hub PubMed. But it's not an exhaustive list. The tool can also pull information from Wikipedia, blog posts, Reddit and e-commerce websites. It also doesn't tell users which facts come from which sources.
While standard search results allow many users to quickly distinguish between a trusted medical website and a candy company, a single block of text that combines information from multiple sources can cause confusion.
“That's if people are looking at the source,” said Dr. Seema Yasmin, director of the Stanford University Health Communications Initiative, adding, “I don't know if people are looking at the source, or if we've done a good job of teaching them to look at the source.” She said her research on misinformation makes her pessimistic about whether the average user is interested in looking at more than a quick answer.
Regarding the accuracy of the chocolate response, Dr. Dariush Mozaffarian, a cardiologist and professor of medicine at Tufts University, said the response was mostly accurate in its facts and summarized the research on chocolate's health benefits, but it did not distinguish between strong evidence from randomized trials and weak evidence from observational studies, nor did it offer any caveats about the evidence, he said.
It's true that chocolate contains antioxidants, Dr. Mozaffarian said. But what about the claim that eating chocolate can prevent memory decline? That hasn't been definitively proven and “requires a lot of caution,” he said. Listing claims like this gives the impression they're better established than they are.
Even if the science behind a given answer doesn’t change, the answer may change as the AI ​​itself evolves.
A Google spokesman said in a statement that the company makes efforts to include disclaimers when responding, noting that the information should not be treated as medical advice.
It's unclear how AI Overview will evaluate the strength of evidence, or whether it will take into account conflicting research findings, such as whether coffee is good for you. “Science is not a static collection of facts,” Dr. Yasmin says. She and other experts also question whether the tool will use older scientific findings that have since been disproven or don't capture the latest understanding of an issue.
“The ability to make important decisions, to distinguish the quality of information sources, is something that humans do all the time, and it's something that clinicians do,” said Dr. Daniel Bitterman, an artificial intelligence physician-scientist at Dana-Farber Cancer Institute and Brigham and Women's Hospital. “They're analyzing the evidence.”
If we want tools like AI Overviews to do their job, she said, “we need to better understand how they navigate different sources of information and how they apply a critical lens to arrive at a summary.”
Experts said the unknowns were concerning given that the new system prioritizes AI summary answers over individual links to reputable medical websites such as the Mayo Clinic and Cleveland Clinic, which have historically appeared at the top of search results for many health-related searches.
A Google spokesperson said that while AI Overviews match or summarize the information that appears at the top of search results, they're not designed to replace that content. Rather, they're designed to help people understand the information that's available to them, the spokesperson said.
The Mayo Clinic declined to comment on the new response. A Cleveland Clinic representative said people seeking health information should “directly search known, trusted sources” and contact their health care provider if they have any symptoms.
A representative for Scripps Health, a California-based health system quoted in several AI briefs, said in a statement that “citations in Google's AI-generated answers can help establish Scripps Health as a trusted medical source.”
However, the spokesperson added, “We do have concerns that we cannot guarantee content produced through AI in the same way that our own content is vetted by medical experts.”
When it comes to medical questions, experts say it's not just the accuracy of the answer that matters, but how it's presented to the user. Take the question, “Am I having a heart attack?” for example, and the AI's response provided a useful outline of symptoms, says Dr. Richard Gumina, chief of cardiovascular medicine at The Ohio State University Wexner Medical Center.
But he added that he had to read through a lengthy list of symptoms before the text advised him to call 911. Dr. Gumina also searched “am I having a stroke” to see if the tool would provide a more urgent response. And it did, directing users in the first line to call 911. He said he would advise patients experiencing symptoms of a heart attack or stroke to seek help immediately.
Experts are urging people looking for health information to be cautious about the new answers. Essentially, users should heed the fine print underneath the AI ​​summary answer, experts say: “This is for informational purposes only. For medical advice or diagnosis, consult a professional. Generative AI is experimental.”
Dani Blum contributed reporting.
