🔗 On cognitive debt

People are talking about cognitive debt, the idea that AI-generated codebases are at risk of falling into a state where nobody knows how they work. This threatens to leave them inextensible, unobservable, and hard to debug.

Here's Simon Willison citing Margaret-Anne Storey, as discussed by Martin Fowler.

"Cognitive debt" is a lovely term for this: it describes an important phenomenon that is much to be avoided. It is absolutely happening with AI-driven codebases--the links above have examples, and I've felt it in my own projects.

I suspect, however, that:

1. When you control for the size and scope of the project, cognitive debt tends to be at least as bad, and usually worse, in pre-AI codebases than in AI codebases;
2. This fact is obscured because so much of what is normalized as traditional engineering work is in fact either managing crippling cognitive debt or avoiding it at enormous cost;
3. The best users of AI are already pretty good at avoiding cognitive debt, and we’re only going to get better at it.