The integration of artificial intelligence (AI) into our daily lives has certainly brought many benefits. As helpful as this technology has been, however, it has brought its fair share of problems, from political “deepfakes” to students automating their homework. AI detectors – themselves artificial intelligence programs – have become crucial tools. But how accurate are they, really? The Declaration of Independence, written more than two centuries before such technology existed, was recently flagged by one such program as 98% AI content.

AI Is Not Infallible

Before artificial intelligence, academics had to rely on personal knowledge to catch students trying to take the easy road with their papers and work. If something seemed suspicious, they’d research materials and even consult with colleagues to see if plagiarism was a factor. Today, many instructors rely on AI detectors to see if students have used tools to write their papers. But, according to Christopher Penn, chief data scientist at the Boston-based marketing analytics firm Trust Insights, “AI detectors are a joke.”

“What led me to the testing of AI detectors was seeing colleagues battling back and forth, arguing about whether a piece of content was AI-generated,” he told Decrypt. “I saw this on LinkedIn; some people were lobbing accusations against each other that so-and-so was being a lazy marketer, taking the easy way out, and just using AI.”

To prove his point, Penn tested the US Declaration of Independence and was appalled to discover that one detector claimed that 97.75% of the preamble was AI-generated. I decided to conduct my own experiment.

I pasted No. 1 of the Federalist Papers into Grammarly’s plagiarism and AI text check. The results said that 100% of the document matched external sources and that no AI-generated text was detected. That was encouraging. Next, I ran it through ZeroGTP’s detector and was told the text was AI/GPT generated: 93.74%. Considering Alexander Hamilton published this first of 85 essays on October 27, 1787 – 237 years ago, and long before AI existed – this was very discouraging.

AI programs can be used to rewrite content, so I continued my experiment to see if rewriting No. 1 would be flagged in the detectors. Using ChatGPT, I copied and pasted the document and told the tool to rewrite it. Using the AI-generated rewrite, I plugged it into Grammarly’s detector. Keep in mind that this is 100% AI-generated, but Grammarly found “no plagiarism detected” and “28% of your document appears to be AI-generated.” Even worse, ZeroGPT told me: “Your Text is Human written” and that only 13.61% was AI/GPT generated.

Will Artificial Intelligence Replace Human Writing?

One of the concerns about AI is that it will replace human writing, but is this a valid fear? UC Santa Barbara English professor Sown Park explained that the two have different motives and are not in competition with each other. “People write to make sense of things,” she said. “The need to create and make meaning through oneself is constant. AI writing can only be [considered] better if we invalidate the joy people get from writing.”

Writing connects us to ourselves, the world, and each other, she explained. “It is only by putting it into words that I make it whole,” Park said, quoting Virginia Woolf’s autobiographical work, Moments of Being. Furthermore, literature does not progress over time, and it is not a competition. “One cannot compare Woolf to Hemingway or Austen to Shakespeare – every writer’s job is to write what is true for them.”

To test this theory, I instructed ChatGPT to: “Write a short love poem.”  This is what it came up with:

In your eyes, the stars align,
A universe where hearts entwine.
Your touch, a breeze, soft and sweet,
The place where love and forever meet.

In every word, in every sigh,
You are my earth, my sea, my sky.
A love so pure, a light so true,
My world begins and ends with you.

Is it romantic and unique? I’ll let you be the judge.

Researchers at the University of Kansas developed a method they claim can identify AI-generated academic science writing with more than 99% accuracy. They published their findings in the scientific journal Cell Reports. “The need to discriminate human writing from AI is now both critical and urgent, particularly in domains like higher education and academic writing, where AI had not been a significant threat or contributor to authorship,” they wrote.

One of the problems is that detection tools focus on a broad spectrum of writing styles instead of focusing on the specific tone and purpose that characterizes academic science writing, according to the findings. The group compared 64 human-written pieces and 128 ChatGPT-generated articles on the same topics, and it claimed to have identified key markers of AI writing. As Decrypt explained, “For example, ChatGPT’s penchant for using the words ‘others’ and ‘researchers’ over words like ‘however,’ ‘but,’ and ‘although’ proved to be a dead giveaway. Also, humans’ predilection for complex paragraph structures, varied sentence lengths, and shifting word counts was another thumbprint missing in the AI writing.”

Programs such as ChatGPT can be useful in getting ideas and suggestions for writing topics and points of interest, but they should not be used to create full documents or job resumes. It’s not just the moral aspect of cheating; these tools are not infallible. In fact, ChatGPT has a disclaimer warning users, stating they should always check important facts. Still, more and more people are relying on AI to take some of the load off their busy schedules.

And just for giggles, I tested this article, which was 100% written by a human or otherwise cited. Lo and behold, both Grammarly and ZeroGPT agreed with me; no AI-generated text was used. At least they finally got something right.