ChatGPT Search, an AI-powered search engine that went live this month, can be tricked into producing a completely misleading summary, British newspaper The Guardian has discovered.
ChatGPT's search feature is intended to speed up your browsing by doing things like summarizing product reviews on web pages. But the Guardian found that by inserting hidden text into websites created by ChatGPT, it could be possible to ignore negative reviews and generate “completely positive” summaries. ChatGPT Search can also spit out malicious code using this method.
Hidden text attacks like this are a well-known risk to LLMs, but this appears to be the first time they have been demonstrated in an AI-powered live search product. Google, the search industry leader, has extensive experience dealing with similar issues, the Guardian noted.
When contacted by TechCrunch, OpenAI did not comment on this specific incident, but said it uses a variety of methods to block malicious websites and is continually improving them.