Fast learns, slow remembers
Stewart Brand has a model for how civilisations change. He calls it pace layering. The idea is that a society is made of layers, and each one changes at its own speed. Fashion moves fastest, then commerce, infrastructure, governance and culture, with nature slowest of all. Think of clothes that go out of style in a season, prices and shops that change in a few years, roads and laws that last decades, and language and customs that shift over generations. The fast layers try new things all the time. The slow layers keep only what worked, and they hold everything else steady. Brand's sharpest line is this one: "Fast gets all our attention, slow has all the power."
In 1997 Brian Foote and Joseph Yoder took the same idea into software. Their paper Big Ball of Mud has a pattern called Shearing Layers, borrowed from Brand's earlier book on buildings, How Buildings Learn. Parts of a system change at different rates. Data changes fastest, then code, then frameworks, then languages. Their advice fits in one sentence: "factor your system so that artifacts that change at similar rates are together."
I've been working on a very old system lately, and that sentence has been stuck in my head.
When the data becomes the slowest layer
In a young system, data is the fast layer. Records come and go every second. In an old system something flips. Years of records pile up in one shape, every screen and job depends on that shape, and the data model ends up as the slowest, heaviest layer of all.
There is supposed to be a layer between that model and the rest of the code. Call it a data-access layer, a DAO, a repository. Its job is to keep the two apart so each can move at its own speed. In old systems it leaks. Business rules end up inside it, and table or document shapes leak out into everything above it. The layers stop shearing and start sticking.
The example that made this concrete for me: two entities that can't be created without each other. To create A you need B's id, and to create B you need A's id. Neither can change, or even exist, alone. The layer has turned into glue.
Six patterns for moving the slow layer
None of these are mine. They come from people who have done this many times, and they work best in roughly this order.
- Read the layers first. Sort the parts of the system by how often they change, then look for the places where a fast part is stuck to a slow one. The work starts at that seam.
- Cordon off the mess. Foote and Yoder call this Sweeping It Under the Rug: "if you can't easily make a mess go away, at least cordon it off." Give the core a single door. Nothing reads or writes the data except through it.
- Put a stable contract in front, and pin it with tests. Once there is one door, you can swap what's behind it. Branch by Abstraction does exactly this: wrap the old code, build the new version behind the same interface, flip a switch, delete the old one. Tests at the contract record what the system does today, so you notice when that changes. A separate read path can do the same for queries, though Martin Fowler warns that splitting reads from writes "adds risky complexity" for most systems.
- Change the core in small, reversible steps. Parallel Change (also called expand and contract) has three moves: add the new shape next to the old one, move readers and writers over, remove the old shape. Teams skip the last move, and the article is blunt about it: skip it and "you might end up in a worse state than you started."
- Untangle mutual dependencies. Two things that can't exist without each other usually fall into one of two cases. Only one side owns the link, so the reference should point one way. Or they were always one thing and should be stored as one. Vaughn Vernon's advice on aggregates helps here: keep consistency boundaries small, and change one per transaction.
- Rewrite only as the exception. Big Ball of Mud keeps Reconstruction as an option for code "beyond repair, or even comprehension." Even then, the paper says decline usually comes from neglect, and the team's understanding carries over to the new system.
Accrued change
Brand has one more line I keep coming back to: "Fast and small instructs slow and big by accrued innovation and by occasional revolution." A rewrite is the revolution. Every pattern above is the accrued kind: small changes in the faster layers that add up until the slow one can follow.
My old system's core isn't slow because someone wrote bad code years ago. It's slow because everything leans on it. That changes the question I ask. Instead of "how do I fix the model?" it's "what can I loosen first so the model is free to move?"
Sources
- Brian Foote and Joseph Yoder, Big Ball of Mud (1997/1999)
- Stewart Brand, Pace Layering: How Complex Systems Learn and Keep Learning, Journal of Design and Science
- Pace layering, The Jolly Contrarian
- Branch by Abstraction, trunkbaseddevelopment.com
- Danilo Sato, Parallel Change (2014)
- Martin Fowler, CQRS (2011)
- Vaughn Vernon, Effective Aggregate Design, Part I (2011)