OpenAI has developed a tool that could potentially catch students cheating by asking ChatGPT to ghostwrite their assignments, but the company is debating whether to actually release it, according to The Wall Street Journal.
In a statement provided to TechCrunch, an OpenAI spokesperson confirmed that the company is researching the text watermarking technique described in the WSJ article, but said the company is taking a “cautious approach” to releasing anything publicly because of “the complexities involved and the potential impact to the broader ecosystem outside of OpenAI.”
“While the text watermarking methods we are developing show promise as a technology, they have important risks that we consider as we research alternatives, including the potential for circumvention by bad actors and the potential for disproportionate impacts on groups, such as non-English speakers,” the spokesperson said.
It's a different approach than previous efforts at AI-generated text detection that have been largely ineffective: OpenAI itself shut down its previous AI text detector last year, citing “low accuracy.”
With text watermarking, OpenAI plans to focus solely on detecting writings from ChatGPT, rather than other companies' models. They will do this by making a small change to the way ChatGPT selects words, essentially creating an invisible watermark in writings that can later be detected by another tool.
Following the publication of the journal article, OpenAI also updated a May blog post about its research into detecting AI-generated content. According to the update, text watermarking has proven to be “accurate and even effective against localized tampering such as paraphrasing,” but “less robust against global tampering, such as using a translation system, paraphrasing with a separate generative model, or asking the model to insert a special character between each word and then remove it.”
As a result, OpenAI wrote that the method is “easily circumvented by bad actors.” OpenAI's update also echoed a spokesperson's point about non-native English speakers, writing that text watermarking “can add stigma to the use of AI as a useful writing tool for non-native English speakers.”