“Scissor running is an aerobic exercise that gets your heart rate up and requires focus and attention,” says Google's new AI search feature. “It's also said to improve pores and build strength.”
Google's AI feature got this answer from a website called Little Old Lady Comedy, which, as the name suggests, is a comedy blog. But the gaffe is so ridiculous that it's circulating on social media, along with a rundown of Google's other obviously flawed AI. In effect, regular users are red teaming these products on social media.
In the cybersecurity space, some companies hire “red teams,” or ethical hackers, to try to break into their products like bad guys. If the red teams find vulnerabilities, the company can fix them before the product ships. Google certainly uses red team investigations before releasing AI products for Google Search, which is estimated to process trillions of queries a day.
It's surprising, then, that a well-resourced company like Google continues to ship products with obvious flaws. This is why making fun of AI product failures has become a meme, especially in an era when AI is becoming more and more pervasive. We see this phenomenon often, from ChatGPT's spelling mistakes, to video generators not understanding how humans eat spaghetti, to X's Grok AI news summaries, which, like Google, don't understand satire. But these memes can actually provide useful feedback for companies developing and testing AI.
Despite the high profile of these flaws, tech companies often downplay their impact.
“The examples we saw were generally highly unusual queries and aren't representative of the majority of people's experiences,” Google told TechCrunch in an emailed statement. “We conducted extensive testing before launching this new experience, and we'll use these individual examples to refine our system overall.”
Not all users see the same AI results, and by the time a particularly bad AI suggestion becomes widespread, the problem has often already been fixed. In one recent high-profile case, if you were making pizza and the cheese wouldn't stick, Google suggested adding about an eighth of a cup of glue to the sauce to “make it stickier.” As it turns out, the AI was deriving this answer from an 11-year-old comment on Reddit by user “f––smith.”
A Google AI summary suggests adding glue to help cheese stick to pizza, but the source of that suggestion turns out to be an 11-year-old comment by Reddit user F*cksmith 😂 pic.twitter.com/uDPAbsAKeO
— Peter Gyang (@petergyang) May 23, 2024
Not only is this an incredible blunder, it also suggests that AI content deals may be overvalued: Google, for example, signed a $60 million deal with Reddit to license its content for training AI models. Reddit signed a similar deal with OpenAI last week, and Automattic-owned WordPress.org and Tumblr are rumored to be in talks to sell data to Midjourney and OpenAI.
To Google's credit, many of the errors circulating on social media stem from unconventional searches designed to confuse its AI. At the very least, hopefully no one is seriously searching for “health benefits of running with scissors.” But some of these gaffes are more serious. Science journalist Erin Ross posted on X that Google is spewing misinformation about what to do if you're bitten by a rattlesnake.
Ross's post, which has garnered more than 13,000 likes, describes how the AI recommends applying a tourniquet to the wound and cutting it to suck out the venom — all things that the U.S. Forest Service says you shouldn't do if you're bitten. Meanwhile, on Bluesky, author T. Kingfisher popularized a post showing Google's Gemini misidentifying a poisonous mushroom as a regular white button mushroom. A screenshot of the post has been circulated across other platforms as a cautionary tale.
Good old Google AI: Telling you what not to do when bitten by a rattlesnake.
From mushrooms to snake bites, AI content can be truly dangerous. pic.twitter.com/UZXgBjsre9
— Ehh. (@ErinEARoss) May 19, 2024
When an AI's poor answers go viral, the result can be new content on the topic, confusing the AI even more. On Wednesday, New York Times reporter Arik Toler posted a screenshot of a query to X asking if dogs have ever played in the NHL. The AI's answer was “yes.” For some reason, the AI called Calgary Flames player Martin Pospisil a dog. Now, if you run the same query, the AI will bring up a Daily Dot article about how Google's AI continues to think dogs play sports. The AI is being seeded with its own mistakes, making things even worse.
This is an inherent problem with training large-scale AI models on the internet: people on the internet sometimes lie. But just as there is no rule that stops dogs from playing basketball, unfortunately there is no rule that stops big tech companies from shipping bad AI products.
As the saying goes, “Garbage in, garbage out.”