How do you solve a problem like ChatGPT?

LLMs have disrupted college teaching and not in a good way. But there is a solution, it just involves technology, enforcement, and sanctions--three things that make progressives deeply uncomfortable.

When ChatGPT exploded on the scene in 2022, like many other people, I was wowed by what it could do. To be fair, I had been hearing about the GPT model for a while before this; there were blog-posts and write-ups floating online where people had produced texts with GPT. But this was the first time anyone got to interact with the model and the idea of chatting with the model was just genius. Moreover, OpenAI had exploited a classic Silicon Valley strategy: rather than selling ChatGPT as a fully priced and fleshed out product, it had released it as a free web-based software. Many many people started using it for their various tasks. The world has never looked back.

Concerns—or rather, panic—that ChatGPT would invalidate most writing-heavy college assignments began almost immediately. But for the first two years at least, I thought I was okay. My classes usually have three papers that students write; the paper prompts tend to be very specific to the course concepts and the course readings and they often involved empirical work or asking students to solve a problem (like create a policy or a labeling scheme). I was pretty sure that if students were using LLMs to answer these papers, even if we weren’t necessarily catching them, they were at least getting bad grades.

But since last year, I have started to get far less complacent. The models had gotten better and newer features were being introduced. But more than that, what worried me was that students were using LLMs not so much for the big papers they had to write (we graded those thoroughly) but for the low-stakes assignments which were graded for participation. Assignments like: writing a weekly reading response or writing an outline of the paper using a specific rubric. The whole point of making these assignments low-stakes was because they helped students do work regularly (like the weekly readings) and were stepping stones towards writing their final papers (like the outline). But the quality of the reading responses—never very high in the first place, I should say—was now even shockingly lower. Students were essentially putting my questions into ChatGPT, including some text from the reading, and creating a response. It defeated the whole purpose of these assignments, which was to let the students practice their skills without worrying about grades, but it also gave them enough points that grossly distorted their grade.¹ There was at least a student or two in my evaluations who complained that the submissions to the low-stakes assignments from others were AI-generated.

As my friend David Singerman put it on Twitter, reading responses and other such assignments are “now toast.” It is now possible, using Google’s fantastic NotebookLLM, to query a series of documents; and what better a group of documents to use than class readings? Many of my colleagues now make students do in-class reading responses and have exams instead of papers. I am thinking of some similar things myself.

Sadly, though, and this is the real point of this post, the debate among college instructors about ChatGPT and LLMs has fallen into one of the classic debates of educational progressivism. And that debate—and our discomfort with technological fixes and policing—is stopping us, as an interest group with a clear interest in regulating LLM-usage amongst students, from trying to come up with a policy solution that both allows LLM adoption while also restricting student usage. What’s even worse is that a solution called “watermarking” exists. It is not a magic solution and would, indeed, require collective action from universities. But all things said, a world where students would not use LLMs for writing assignments (unless instructors specifically asked them to) is well within the realm of possibility.

The debate over LLMs in writing-heavy classes, in short

On the one hand, we have instructors who think LLMs are tools that can be useful to students and therefore encourage us to design assignments that incorporate LLMs. These educators tell us that what LLMs have exposed some of the problems with the assignments we have traditionally designed. Just as we try to design assignments that test for concepts rather than rote learning, so also, we need to design assignments that test for something concrete that an LLM can’t really do but which the students could use an LLM for.

I am actually fairly sympathetic to this view though I will say that it is really difficult to design such assignments, especially when one of the goals of a classroom is to teach students how to make an argument and how to write it. As an op-ed in Inside Higher Ed put it, “writing isn’t really something you learn or teach—it’s something you practice.”² It is also difficult because as a real-life instructor, I have to work with the students I have, not the students I wish I’d had. My students are often taking my class for a requirement and are not particularly interested in the topic; they have a hundred other classes they need to take and are starved of time; it’s no wonder that they strategize as much as they can to get the best grades with the least amount of work.

But, on the other hand, we also have educators who think that LLMs expose really what is rotten in higher education: the pursuit of profit over learning, the collusion between universities and technology companies, and pretty much everything that’s wrong with this world (racism, sexism, capitalism, colonialism, unchecked environmental degradation, the works). Many of the people in this camp don’t necessarily have a policy solution to the issue that college professors face on a day-to-day basis. What’s more, they would argue, critical theory-style, that focusing on the smaller issues means we actually end up legitimizing the system and it is the system that is rotten. We have to reject LLMs, they say, reject them completely along with the system that created them. I am not as sympathetic to this view partly because it offers no concrete solutions, neither to the problem of students using LLMs or to the problem of changing the rotten system (which, to be honest, I am not convinced it is).

One thing the proponents of both views do agree on is the futility, and perhaps even the unfairness, of enforcement. Some call “AI-detection software” as “unethical in itself” but others will point to the fact that this software is highly unreliable. I speak from personal experience having both tried a vendor-based software as well as my university’s official trial. And the WSJ tells me that even OpenAI’s algorithm to detect “text written by several AI models, including its own” only worked “26% of the time, and OpenAI pulled it seven months later.”

This unreliability means that using such software, or even using the heuristics that most instructors develop once they start to know their students, can be risky. We don’t want to wrongly punish students who are using AI for purposes like just improving their language. But we also want students to learn, and to learn how to write, rather than pass the course just by regurgitating AI assignments.

The solution: watermarking. The problem: standardization

Of course, in between these extremes, we do have a solution: watermarking. Not a perfect one, but definitely a pragmatic one, and yes, it’s about enforcement. It won’t work, however, unless universities act collectively.

But Let’s backtrack. In August 2024, Deepa Seetharaman and Matt Barnum wrote a detailed piece in The Wall Street Journal about a method that the researchers at OpenAI have been working on that can help detect whether a piece of text was produced by ChatGPT. No, this was not yet another classifier trained on AI-produced text that would identify whether some text was LLM produced or not. Rather, this technology would modify the output of the ChatGPT software itself. Here’s how the Journal describes it:

ChatGPT is powered by an AI system that predicts what word or word fragment, known as a token, should come next in a sentence. The anticheating tool under discussion at OpenAI would slightly change how the tokens are selected. Those changes would leave a pattern called a watermark.

The watermarks would be unnoticeable to the human eye but could be found with OpenAI’s detection technology. The detector provides a score of how likely the entire document or a portion of it was written by ChatGPT.

The watermarks are 99.9% effective when enough new text is created by ChatGPT, according to the internal documents.

In other words, this technology involves modifying AI-produced text such that there is a “fingerprint” that gets created along with a “detector” that can identify this fingerprint. A watermark, in other words, is like a trademark printed on an object in invisible ink or a serial number printed on a gun or even like the license plate of a car. It identifies the provenance that the text came from.³

Obviously, watermarks involve a burden on the AI company. OpenAI, in this case, needs to modify its LLM model since the watermark is literally hidden in the text produced by the model. There is a chance that this will degrade the quality of the output produced by ChatGPT.

But the Journal assures us that this was not the case according to OpenAI’s own internal surveys. “People familiar with the matter” told the Journal that “when OpenAI conducted a test,” they “found watermarking didn’t impair ChatGPT’s performance.” An internal document examined by the Journal states that “Our [OpenAI’s] ability to defend our lack of text watermarking is weak now that we know it doesn’t degrade outputs.”

So why has there been this huge delay in deploying this watermarking technology? The problem was that when OpenAI surveyed its users, they made it more than clear that should OpenAI implement watermarking, they would simply decamp to other LLM products. As the Journal puts it:

Nearly 30% said they would use ChatGPT less if it deployed watermarks and a rival didn’t.

In other words, it was the problem of competition. OpenAI risks decreasing its market share to its competitors if it implements this technology on its own.

This is a classic Prisoner’s Dilemma like problem of market standardization. A bunch of companies make a useful product that has a negative market externality. Company A has the technology X that will allow them to create a version of the product that will lessen this externality but this will come at a cost that can make company A less competitive. So company A will not implement this technology unless company B, its competitor, also does so. But company B feels the same way. And A and B fear that if they both do it, then a third company C will emerge that will indeed produce their product without X and displace both of them. So they all continue to make the product without the feature X, thus continuing with the negative externality.

The classic answer to the problem of negative externalities is regulation that forces ALL companies to take an action so that no one company suffers a competitive disadvantage from doing so. Take factories, that make stuff people like and use, but whose operation produces pollution. So governments force ALL factories to build air purifiers and fluid treatment centers before they emit their gaseous and liquid waste; the government also specifies standards for the kinds of emissions the factories are allowed. They may even make this purification technology broadly accessible by providing subsidies. And they punish companies that do not abide by the regulations with fines and perhaps even imprisonment.

In the case of AI, governments have been trying to regulate for negative externalities. As Kelsey Piper reported in Vox back in 2024, the state of California is trying to come up with some regulatory standards to enforce watermarking:

If every generative AI system is required to have watermarking, then it’s not a competitive disadvantage. This is the logic behind a bill introduced this year in the California state Assembly, known as the California Digital Content Provenance Standards, which would require generative AI providers to make their AI-generated content detectable, along with requiring providers to label generative AI and remove deceptive content. OpenAI is in favor of the bill — not surprisingly, as they’re the only generative AI provider known to have a system that does this. Their rivals are mostly opposed. (my emphases).

But as the extract above makes clear, there are some issues. OpenAI is in favor of the bill while its competitors are generally not. Piper thinks this is because OpenAI has developed the watermarking technology and if regulation makes this technology compulsory, OpenAI will have an advantage over its competitors.

This suggests that the watermarking technology itself should be widely shared and it would probably help if the regulation helped to subsidize this technology in the public interest. We shouldn’t forget that regulation, especially if it’s not well-designed, can actually help established companies by keeping their smaller competitors out because those competitors cannot bear the regulatory burden. The state has a compelling interest here in making sure both that the technology is implemented but also that it doesn’t confer any competitive power on any particular LLM product.

A shared regulation can also alleviate some of the other objections to watermarking such as ones that OpenAI brought up in its blogpost from 2024:

While [the watermarking method] has been highly accurate and even effective against localized tampering, such as paraphrasing, it is less robust against globalized tampering; like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character - making it trivial to circumvention by bad actors.

I do not have the best expertise on LLMs but I suspect that a watermarking standard that is used across ALL generative AI systems would also be robust to globalized tampering because it would be implemented in every model.

So, far from LLMs having completely disrupted education, the problem of students using LLMs to avoid learning is a tractable one that could be solved by cooperation—abetted by government enforcement—between LLM companies, companies that build learning management systems (LMSes, e.g. Canvas), and the universities themselves. All LLM companies build a watermark into every piece of text they produce and LMSes build tools to detect watermarks into their grading features. Instructors can then use the watermark detector to figure out whether students have used LLMs and take action.

This seems to me to be a system that can be reasonably accommodating of a variety of situations. For some assignments, instructors may encourage students to use LLMs; for other assignments, they may disallow them. There is a clear, transparent way of telling students what counts as academic integrity and how it will be tracked and evaluated. And with a few years, this problem could be just as tractable as the way college instructors have solved the problem of plagiarism (or at least, managed it to the extent that it does not seriously stop courses from functioning).

So why isn’t there more of a movement to get AI companies to implement watermarking?

This takes me to the real reason for this post. Over the last three years, I have participated in many discussions about LLMs with other instructors and witnessed many different arguments on mailing lists and blogs. These discussions stay within a particular kind of box. Some people oppose generative AI for political reasons (Silicon Valley, profit-seeking, environmental impacts, what have you). Others urge instructors to modify their assignments to make them AI proof: more specific, more project-like. Still others urge instructors to be careful when they punish students for using AI for their assignments; indeed, all agree that traditional “AI detection” is very, very weak and leads to both false negatives and false positives; instructors have also been cautioned that the punishment may fall disproportionately on those for whom English is a second language and who use the AI for grammar and polish.

But I have not seen yet a discussion of watermarking among instructors or among universities. This is despite the fact that watermarking seems to me to be the best kind of solution to the AI problem. It is agnostic to pedagogy (instructors get to decide whether to let students use AI in their assignments or not; instructors are free to create the kinds of assignments that work for their learning goals) and it also gives instructors significant amount of leeway even when it comes to enforcement and punishment.

The biggest problem, I suspect, is that it is too in-the-weeds and does not easily map on to issues of pedagogy or the university’s mission, issues that instructors often feel most comfortable discussing. It also does not map very clearly to politics; in fact, it is a politically agnostic solution that offers a clear path for action in the medium and long-term.⁴ Another reason could be that watermarking is a technology of policing and instructors, most of whom identify as pedagogical progressives, are uncomfortable with focusing on that aspect of their teaching.

To a few instructors, this solution also perhaps comes across as “technologically solutionist.” I feel like the term “solutionist” has now become diluted far beyond its initial usage when it was used to refer to projects like One Laptop Per Child or the idea that the internet would help citizens topple authoritarian regimes; it is often used as a pejorative for solutions one does not like.

But I would point out that watermarking is ABSOLUTELY NOT technologically solutionist. While it depends on a technological feature, it is ultimately about standardization, which is never just a “technical” matter but rather a question of collective action.

But collective action is the operative phrase here. Regulation in the US operates through an onerous process (unlike, say, China, which I am told, has already enacted some regulation about watermarking and plagiarism) and it is not clear this problem, academic misconduct, has the kind of newsworthiness or risk that would force governments to act. Governments and the broader public are more exercised by the thought of an election outcome changing because of a faked video made with AI — even though this is highly unlikely in practice — than by the thought of rampant academic misconduct.

But universities and instructors can act and if they band together, I like to think they can force AI companies to implement watermarking. For one thing, universities have the power of banning LLMs in their classrooms and it is clear from the last few days, as Google and OpenAI have offered cheap subscriptions to students, that students are a big part of the imagined customer base for AI products. AI companies, then, have an interest in working with universities to implement watermarking policies that allow for the enforcement of academic integrity. Universities also have power over the companies that make learning management systems or LMSes and can bring them into the room to discuss how they can build watermarking detectors into their software.

If watermarking standards can be worked out and implemented, this benefits not just universities and instructors but AI companies as well. Instructors, once convinced that they can safely and easily monitor whether students are using AI, are then free to construct assignments that encourage students to use AI. AI companies are free to innovate on building new products that can work in teaching contexts.

Watermarking is not a perfect solution but it seems to be a pretty good one for the problem at hand. When people worry about fake images and videos, they are often concerned about bad actors; these are determined, unscrupulous people who would go to great lengths to make a fake video. Students who use LLMs to take a short-cut on their writing assignments are not bad actors; they are mostly just trying to save time and cut some corners.

The bad actors we might want to be aware of are rogue AI companies that produce text outputs without watermarking but in a situation like this, established AI companies who do implement watermarking have an incentive to keep track of these rogue companies. And a rogue company can stay under the radar only if few students know of its existence; the minute a company becomes popular, its practices would come under scrutiny. Open-weight AI models might present another problem but again, students using open-weight models to write their assignments does not seem to me to be a scenario that instructors should be too worried about.

So, to sum up, the problem of academic integrity in college writing in an age of LLMs is not an intractable problem. It is not about the power of Silicon Valley’s disruptive innovation and it is not about our pedagogical styles. Ultimately, given that watermarking technology exists, it is a matter of creating technological standards and enforcement mechanisms. If universities act collectively as a group, they can almost certainly discourage students from using LLMs on assignments while also helping instructors create new kinds of assignments that require students to use LLMs.

1. I design my classes so that they are easy to pass and indeed, if students do ALL the formative assessments, they can pass the class with a D. But an A is difficult to get.

2. I don’t agree with much else in this op-ed, by the way. It’s symptomatic of the broader problem with “critical theory” where we express objections to a technology and its effects in sweeping language that actually prevents us from coming up with pragmatic policy solutions. But that’s a topic for another time.

3. A somewhat similar solution is to produce cryptographic metadata that gets attached to the particular image or text produced by an LLM. I suspect that this solution works much better for images which typically come in a certain format rather than text that can be copied and pasted.

4. Some might interpret it as pro-AI or pro-Silicon Valley although I think it is a pro-teacher and pro-university solution that has both costs and benefits for AI companies.