A Cheating Scandal and Warning
A Palo Alto high school student was accused of using AI to cheat in an essay on “The Crucible” and his parents have filed a federal lawsuit for $150 million in damages. We mainly cover higher education in our work, but this story is an anecdotal example of a broader trend that spans educational institutions of all types. This story in particular brings all your favorites together. We have allegations of racism, faulty AI detection software, a competitive academic environment, and mixed messaging from the institution–the gang is all here.
I do not pretend to know the merits of this case or what happened, our focus is on what institutions need to do. ChatGPT was released in November of 2022. We are coming up on four-years of accessible generative AI. The time when we can excuse institutional misalignment is over. Schools and universities have to cohere around messaging and policies that are consistent for students.
Mixed-messaging confuses students. According to the San Francisco Standard,
The district’s public position on AI is that it “embraces the opportunities generative AI offers to enhance teaching and learning” and encourages “open dialogue between teachers and students about how generative AI tools may or may not be used in specific courses.” The school’s academic integrity policy lists using AI to complete essays as a form of plagiarism, on par with submitting work written by someone else. But the district has no specific published policy governing how AI-detection tools like Turnitin should be deployed, what score thresholds should trigger disciplinary action, or what process must follow a flag — leaving those decisions, apparently, to individual teachers.
This kind of ambiguity and misalignment is endemic across educational institutions and in some ways even worse at universities where policies may vary considerably class-to-class. Policy-making is hard, as we have argued previously, but we have to do better than saying all things simultaneously and just picking and choosing when confronted with a problem. It is time for institutions to stake out ground and stick to it because students and teachers deserve consistency.
AI detection tools are bad and we should stop using them. To the credit of TurnItIn, their own website outs their own tool as inaccurate, noting “Our AI writing detection model may not always be accurate (it may misidentify human-written, AI-generated, and AI-paraphrased text).” Deep into the FAQ some specific limitations emerge including the fact that their model is only trained to detect four leading AI models from the US. Meaning it is literally ignoring all of the incredibly popular Asian models like DeepSeek and Kimi along with second tier Western models like Mistral and Grok to say nothing of models customized and run locally. We have been humming this same tune for years so we will not belabor the point here, but these tools give instructors a false sense of security (on purpose in our opinion) about detecting AI and lead to the kinds of problems we are seeing here.
So much of education policy can be explained by efforts to minimize legal liability. If there is a silver lining in this story and similar ones it is that this will hopefully catalyze institutions into doing the hard work of getting stakeholders together and reaching agreements on AI disposition. Whether your institution is at the starting line or already taking appropriate steps, the time to sit down for the hard conversation is now. Waiting longer hurts students and exposes our institutions to lawsuits.