An AI detector mislabeled nearly every essay written by a non-native English speaker as being written by a bot
Researchers found popular GPT-detectors flagged essays by non-native English speakers as AI-written.
One detection system marked almost 98% of their essays as AI-generated text.
The findings add to concerns on the effectiveness of AI-detection systems and systemic bias in AI.
Systems that detect AI-generated writing are flagging essays written by non-native English speakers as bot-generated, researchers from Stanford University said.
In the study published Monday, the researchers ran more than 100 essays written by non-native English speakers through seven popular GPT detectors. (GPT is the framework behind some of the most widely used chatbots today, including ChatGPT.) The essays were written for an English-proficiency exam.
The researchers also fed the detectors essays written by US eighth graders who speak English natively.
More than half of the essays written by non-native English speakers were marked as AI-generated by the detection systems, the Stanford researchers found. And one GPT-detection system flagged almost 98% of those essays as written by AI.
But when evaluating essays written by eighth-grade native English speakers, the detection systems performed much better, assessing that about 20% of their essays were produced by AI. Still mislabeling human-written essays as bot-written — but not at the same rate.
The researchers warned the discrepancy could cause considerable harm to non-native English speakers who could be falsely accused of submitting work produced by chatbots for school assignments, college applications, or for work they do.
AI-detection software is increasingly being marketed as a silver bullet to floods of misinformation being produced with tools like ChatGPT, Dall-E and MidJourney. But the study adds to the growing questions surrounding the effectiveness of these systems — and echoes longstanding calls to address biases hardwired into AI systems.
Chatbots like ChatGPT generate written text by guessing the most logical word to follow in a sentence. It makes these guesses based on underlying large language models, or LLMs, which are trained on datasets comprised of mounds of data scraped from all sorts of publicly available sources, like social media platforms and Wikipedia.
The researchers attributed the results of their study to a metric many GPT-detection systems use called "text perplexity." In an email to Insider, Stanford professor James Zou, who was the corresponding author of the study, said the metric "measures how surprising the word choices are in the text."
The text that ChatGPT and other chatbots spit out is generally marked as low text perplexity by detectors because they're programmed to use the most common words or responses when constructing sentences. As a result, Zou said, "these detectors are more likely to flag text with low perplexity as AI-generated content."
When the detection systems evaluated the essays written by non-native English speakers, it marked much of their writing as low perplexity because they used "a more limited range of linguistic expression," the study said. This resulted in a vast majority of their content being flagged as AI-generated text.
The GPT detectors tested in the study aren't the only defensive tools producing paltry results. In June, The New York Times evaluated five programs designed to spot AI-generated images. The newspaper fooled the sensors on several occasions, noting the detectors tested didn't use logical reasoning when assessing the authenticity of an image.
Meanwhile, upon ChatGPT's initial release, OpenAI warned users in a blog post that the chatbot would "sometimes respond to harmful instructions or exhibit biased behavior." The company assured then that it was moving to "block certain types of unsafe content," but that it did expect the chatbot "to have some false negatives and positives" for the time being.
OpenAI CEO Sam Altman acknowledged in a tweet earlier this year that ChatGPT has "shortcomings around bias" and vowed the company would improve them. "I'm optimistic that we will get to a world where these models can be a force to reduce bias in society, not reinforce it," Altman said in a May interview with Rest of World.
Read the original article on Business Insider
