AI model collapse is the gradual degradation that happens when a generative model is trained — directly or indirectly — on data produced by earlier AI models rather than by humans. Each generation drifts a little further from reality: rare facts, unusual phrasing, and minority perspectives quietly disappear, while outputs become more generic, repetitive, and occasionally nonsensical. It matters because a growing share of the text and images on the open web is now AI-generated, which means future models risk training, unknowingly, on the exhaust of the models that came before them.
What model collapse is
The term was formalized in a 2024 study by Ilia Shumailov and colleagues, published in Nature under the title “AI models collapse when trained on recursively generated data.” The researchers trained language and image models, then trained the next generation on that first generation’s output, then trained a third generation on the second’s output, and so on — mimicking what happens when AI-written text and AI-made images flood the same web that future models are scraped from. Within a handful of generations, quality collapsed. This is distinct from synthetic data use in general — labs deliberately generate synthetic data for many good reasons, such as filling gaps or protecting privacy. Model collapse is what happens when that process runs unchecked, generation after generation, without enough real human data to anchor it.
How it happens: two stages
Researchers describe collapse in two stages. In early collapse, a model starts losing information about the tails of the data distribution — the rare events, uncommon dialects, or unusual writing styles that appear only occasionally in real data. A model generating synthetic data tends to reproduce what it considers typical, so those rarer patterns get sampled less and less each generation, the way a photocopy of a photocopy loses fine detail first. In late collapse, these small losses compound: functional-approximation errors, sampling errors, and learning errors stack across generations until a model’s outputs converge into a narrow, repetitive range, sometimes blending unrelated concepts together or producing text that no longer reflects the real distribution of human knowledge.
Why it’s a growing risk
The internet is the default training ground for large AI models, and it is no longer a purely human archive — search results, forum posts, and stock images increasingly include AI-generated material. That creates a feedback loop: models trained partly on the open web absorb some AI-generated content without knowing it, and the next generation of models trains on an internet with even more of it. Not every researcher agrees on how severe the effect is in practice — a widely discussed follow-up analysis argued that the original results reflect a known statistical property of repeated resampling, and that collapse is far less severe when synthetic data accumulates alongside real human data rather than replacing it outright.
Can it be prevented?
The main defenses are not exotic. AI labs try to preserve a stable base of genuinely human-generated data as an “anchor” rather than letting synthetic data fully replace it across training generations. Increasingly, they also track data provenance — knowing which training examples are human-written versus machine-generated — instead of scraping the web indiscriminately. Content-authenticity standards such as C2PA, which attach tamper-evident labels to AI-generated media, give future data-collection pipelines a way to identify and filter synthetic content rather than absorbing it unknowingly. None of this fully solves the problem — reliably labeling every AI-generated file on the internet remains unsolved — but it is the direction the field is moving as AI-generated content keeps growing as a share of the web.
FAQ
Does model collapse mean AI models are getting worse over time? Not automatically. Leading labs actively guard against it by curating training data and preserving human-generated sources. The risk is highest for any system trained carelessly on unfiltered web data with no anchor of real human content.
Is model collapse the same as overfitting? No. Overfitting happens within a single training run, when a model memorizes quirks of its own dataset. Model collapse happens across generations of models, as each one trains — even partially — on the previous generation’s synthetic output.
Does it only affect chatbots and text models? No. The original study, and follow-up research, found the same degradation pattern in image-generating models, not just language models.
Sources: Shumailov, I. et al., “AI models collapse when trained on recursively generated data,” Nature (2024); Wikipedia: Model collapse.