Slowing Collapse in the AI Age

Researchers have a name for what happens when AI models are trained on the output of other AI models: collapse. Each generation loses a little of the variety and detail in the original data. Rare cases disappear first, then subtle ones, until the model reflects only a narrow, confident version of the world. Nothing breaks on any single day. The model simply becomes less true, one generation at a time.

The same pattern can happen inside organizations, and in society more broadly. It is far less visible, and it matters far more.

Collapse in data

As AI-generated text, images and figures fill documents, websites and databases, the share of original observation in the world’s records falls. Reports cite other reports that were themselves summaries. Forecasts are compared with other forecasts instead of with outcomes. Datasets are filled with plausible values where measurements were missing.

Inside an organization, this looks like a knowledge base that grows quickly and becomes less reliable, or a dashboard whose numbers nobody can trace back to a transaction. The defense is simple to state: keep the original records of what was ordered, measured, paid and delivered, and keep them separate from anything generated. Ground truth is an asset. Treat it like one.

Collapse in skills

A team hands a routine analysis to an AI tool. It works, so nobody checks the next one as closely. The people who knew why the numbers moved stop practicing that judgment. New staff learn the tool, not the work. A few years later, nobody in the room can tell when the tool is wrong.

This is not a new risk. Pilots who rely too heavily on autopilot lose manual skills; navigators who rely on GPS lose their sense of direction. What is new is the range of work AI can take over, from writing and analysis to planning and coding. Skills that took a career to build can fade in a few years of disuse.

Collapse in institutions

Institutions hold knowledge that is written nowhere: why a rule exists, which supplier to call when the usual one fails, how a case was decided ten years ago. That knowledge lives in people and in their relationships. When routine work moves to machines and experienced people leave, the institution keeps its procedures and loses its understanding.

In public institutions, the cost is higher still. A customs officer who no longer understands why a shipment was flagged cannot explain the decision to a trader, a court or a minister. An institution that cannot explain its decisions loses legitimacy, slowly, then suddenly.

Three things to protect

• Ground truth. Keep original records, and check every model, forecast and summary against them, not against another model’s output.
• Judgment. Keep people in the decisions that matter, with enough involvement that they can tell when an answer is wrong. A reviewer who only approves is not reviewing.
• Know-how. Write down why the organization does what it does, not only what. Tacit knowledge leaves with people, and AI cannot recover what was never recorded.

Measure against reality

The best protection is the simplest: score outputs against what actually happened, continuously. When accuracy drifts, you see it in the numbers before you feel it in the business. On our own platform, models earn their place on scored evidence and lose it the same way. Systems that are never scored decay without anyone knowing, and so do the people who depend on them.

What to do on Monday

• Label what is generated. In documents, datasets and reports, mark what a machine produced, so it is never mistaken for an observation.
• Keep a human practice. Have people do a sample of AI-assisted work by hand, regularly, so the skill stays alive and the comparison stays honest.
• Train for judgment, not tools. Teach new staff why the work is done, and how to tell a good answer from a fluent one.
• Capture the reasons. When an experienced person makes a hard call, record the reasoning as well as the decision.
• Score everything. Compare every forecast, classification and summary with what actually happened.

Slow is good news

Collapse in this sense is slow, so it can be slowed further, and reversed. The organizations that do well with AI will be the ones that use it to extend their people’s judgment, not to replace the practice of it. They will be more capable each year, not less, because they kept the original, the human and the true at the center of everything their machines learn from.