Even AI has to watch its diet

Imagine going to your doctor and have that doctor tell you how much they like a new health app — but when you ask them whether their family likes it, the doctor says “Oh, I don’t trust it enough to let them use it.” Or having your general contractor extol the benefits of robotic construction workers — while bluntly saying he’d never use them to work on his home.

Well, enter AI.

For the last nine months or so, there have been recurring stories about AI companies buying up huge stockpiles of rare and out-of-print books, and “harvesting” — and destroying — the physical books to train their AI models.

Plenty of people questioned the stories, and went to Snopes to see if there was any truth to the claim.

The rumour-busting, fact-finding website said it wasn’t clear if companies such as Anthropic were buying up books simply to destroy them. But it did find that there was clear evidence that books were being bought and then destructively processed — having their spines cut off to feed them through scanners.

Evidence of the plan was found in documents obtained during a breach of copyright lawsuit by authors against Anthropic.

An internal memo reads: “What is Project Panama? Project Panama is our effort to destructively scan all the books in the world…”

The memo continues, “Why use a codename? We use a ‘soft codename’ for it because we don’t want it to be known that we are working on this. This document is visible to all Anthropic employees, but you should avoid talking about it in public areas, and the fact that we are working on this should not be shared with anyone outside Anthropic.”

Now, if you love books, that would seem like a crime on its own. Even more so if the books being destroyed were rare. But more interesting, perhaps, was why it was being done.

And a good way to think about it is that AI is good enough for you, but not for its creators.

The reason is that AI companies want to train their models on “clean” content that doesn’t include — wait for it — AI generated copy.

Why? Because the literary equivalent of a feedback loop occurs when AI is trained using AI-produced material — the result is a “model collapse” of progressively degraded quality AI output.

In other words, perhaps, even for AI, you eventually are what you eat.

It also points to the concept that, while AI may quickly and easily handle the rudiments of your writing tasks, it can only be able to mimic, not create. As it digests its own leftovers — and as those leftovers make up more and more of its daily diet — the resulting product will become necessarily more and more careworn.

Garbage in, garbage out, as the saying goes. There is another corollary — if you’re trying to make your life easier by hiring another kid to do your homework for you, first make sure they actually know how to do the homework.

After all, AI models like Claude GPT are already churning out acres of click-bait, feel-good stories for social media sites: “My brother said our father’s inheritance was all his now, and he laughed. But I just smiled and asked a simple question…”

What happens when that kind of schlock is not only their product, but also the mainstay of their diet?

Where will AI find itself then?

And where will we all find ourselves, if we choose to exercise our own minds less and less?

Because, dear reader, we are also what we eat.