What happens when sites like SO become so polluted with AI generated text that the next generation of LLMs trained on the Internet is just AIs being trained on AIs?
Rapid degradation, like recording multiple generations of VHS tapes. The LLMs make the internet dumber, the LLMs get dumber by learning from it. Rinse and repeat.
Maybe, maybe not. The human brain trains on its own output without descending into chaos. It’s not hard to imagine a scheme where you use a model like GPT-4 to filter a dataset before training. Classification is easier than creation, so you’d expect performance to continue to improve regardless of how poisoned the unfiltered dataset becomes.
One of the more exciting scenarios would be if it turns out that performance can improve indefinitely with a generate -> filter -> train cycle. There are certainly parallels to how humans learn.
> The LLMs make the internet dumber, the LLMs get dumber by learning from it.
I see this said a lot, but in reality it just depends on how the network is trained and how it's prompted. For example, you don't get dumber because you read children's books, you just get better at understanding what makes a good children's book. It's only if reading children's books comes at the cost of reading other content that you might be dumber as a result.
Similarly, an AI doesn't automatically get dumber because it encounters dumb content. It's only if you're training exclusively on dumb content that it doesn't know what quality looks like that you'd have problems.
Broad training sets (ideally pruned of as much junk as possible) and RLHF in theory should condition the network to reproduce quality content and not simply the lowest common denominator of what's found on the internet.
And assuming all that fails there's nothing stopping researchers from just using past datasets with improved architecture going forward. I mean you'd have to wonder why on Earth OpenAI would even release GPT-5 if it's worse than GPT-4...
There's just no scenario here in which what you're saying would actually play out in reality. One way or another companies will ensure the next iteration of their LLMs are better than their previous.
> For example, you don't get dumber because you read children's books, you just get better at understanding what makes a good children's book.
I don't think this is an accurate analogy. It's not books for children -- well-written material at a lower educational/cognitive level -- it's more like books by children -- which is necessarily lacking skill and background context (connection to reality). Think about how children constantly pass around misleading, invented, and incorrect stories among themselves -- and they don't do it maliciously, they just don't know any better. Legends like "Candyman"/"Bloody Mary" for example.
They need an outside influence, an adult or a book or website, to nudge them out of that knowledge rut.
(Of course the same thing can happen with a (closed) group of adults, too, but it's more of a "natural state" with children because they simply haven't had time to encounter as much knowledge.)
Unrelated to this thread but how does Quora manage to be so bad yet so popular? It's a horrible interface and the answers never seem to be good. Often I'll click a top Google search result and it'll be a Quora "thread" where I can't even see an answer.
I'm willing to bet OpenAI knows how to detect OpenAI output, either via stenographic techniques or via keeping a database of all the text it's generated. Both, probably.
Which means future OpenAI models would be getting trained on the output of competitor models. Like Bard. Oof.
