Designing Systems: Structuring for Change

This is Part 3 of the Designing Systems series.

• Part 1: How to Approach a System Design: Framing, requirements vs constraints vs tradeoffs, the hidden assumptions, and the design as a communication artifact.
• Part 2: Core Elements: Data, state, control flow, failure, and observability.
• Part 3: Structuring for Change: Boundaries, contracts, evolution, and ownership. How a system survives the changes that come after the first version ships.
• Part 4: Anti-Patterns and What Experience Teaches: The failure modes that show up repeatedly and what a good design actually feels like.

Part 1 covered framing the problem. Part 2 covered the five elements every system has to make decisions about. Part 3 picks up the next thread. Most systems are not killed by their original design. They are killed by what happens to them between version one and version five. New requirements arrive, scale changes by an order of magnitude, the team that built the system rotates out, and the constraints that justified the original decisions get quietly forgotten. A system that was designed only for its first version becomes brittle within two years. A system that was designed with change as a first-class input stays useful for a decade.

Part 3 covers the four levers that matter most for surviving change: boundaries, contracts, evolution, and ownership.

Table of Contents

• Boundaries
• Contracts
• Evolution
• Ownership
• Conclusion

Boundaries

A boundary in a system is any place where two components meet. Boundaries are where coupling shows up. The work of designing for change is mostly the work of putting the boundaries in the right places and making them honest about what crosses them.

A useful test: what changes can happen on one side of the boundary without forcing change on the other? If the answer is "almost nothing," the boundary is not real. If the answer is "anything internal to the component," the boundary is doing its job.

A few practical heuristics for placing boundaries well:

• Group things that change together; separate things that change for different reasons. This is the most common stated principle, and it is more useful when restated as: if two pieces of code are routinely modified in the same pull request, they probably belong in the same module. If they almost never are, they probably belong in different ones.
• Boundaries should reflect the domain, not the org chart. Conway's law is real; it should be a result of the design, not a substitute for it. Drawing the service boundaries around the current team structure is easy in the short run and creates expensive refactoring debt as the org changes.
• Smaller is not automatically better. Microservices became a default in many teams long after their preconditions stopped applying. A boundary that splits a tightly coupled responsibility across two services creates more operational complexity than the modularity is worth. A boundary that genuinely separates concerns earns its keep.

Principles:

• A good boundary makes one side of it cheap to change. If neither side is cheap to change, the boundary is not buying you anything.
• Visible coupling is better than hidden coupling. An explicit dependency you can see in a diagram is easier to manage than an implicit one that only shows up under load.
• Resist drawing boundaries you do not need yet. Boundaries have a cost. Pay it when the system actually needs the separation, not in anticipation.

Contracts

A contract is what one component promises to another at a boundary. APIs, schemas, message formats, file layouts, command-line interfaces - all of these are contracts. The quality of the contract design determines how much pain the boundary causes when things change.

A useful frame is to distinguish three levels of contract maturity:

• Implicit contracts are not written down anywhere. Whatever the producer happens to emit is what the consumer happens to parse. These work fine until something changes, at which point everything breaks at once.
• Documented contracts are written down but not enforced. The producer can emit something different from what the document says, and nothing catches it until a consumer fails.
• Enforced contracts are checked by tooling. Schema validation, typed clients, contract tests, integration suites that exercise the contract in CI. The producer cannot quietly break the contract without something turning red.

Most contracts in most systems live in the second category. The transition from "documented" to "enforced" is where most of the long-term value of contract discipline comes from.

A few specific habits:

• Versioning is part of the contract. The first time a contract changes in a way that is not backward compatible, the system needs a story for how producers and consumers move forward together. That story has to be designed before it is needed, because designing it after the first break is expensive.
• Backward compatibility is cheaper than retrofit. Adding optional fields, treating unknown fields as ignored, accepting multiple versions during a transition - these are small disciplines that compound into a system that can evolve without coordinated downtime.
• Deprecation has to be a process, not a wish. Sunsetting a contract means giving consumers time, communicating the timeline, providing the replacement, and following up. Systems that "deprecate" by leaving old endpoints around forever accumulate exactly that: forever.

Principles:

• Treat the contract as the most public thing about a service. Internal implementation is a private concern; the contract is the public face.
• Make breaking changes expensive at design time, so they become rare at runtime. A breaking change should require deliberate effort to ship.
• Optional and additive changes are nearly always safe. Required and subtractive changes are nearly always not.

Evolution

Even a perfectly designed system has to be changed in production. The work of evolution is the work of changing a live system without losing data, breaking consumers, or having to take coordinated downtime. Systems that handle evolution well are systems where the engineers have invested in the boring machinery that makes evolution cheap.

A few patterns recur across well-evolved systems:

• Expand-contract migrations. Add the new thing, dual-write to both old and new, validate that they agree, switch reads to the new thing, then remove the old thing. The pattern works for schemas, APIs, storage backends, and most refactorings of stateful systems. It is slower than a hard cutover, and far cheaper than a hard cutover that goes wrong.
• Feature flags and progressive rollouts. Change is shipped behind a flag, exercised for a small percentage of traffic, observed against control, and ramped up. The system is permanently capable of running two versions of the relevant behaviour at once, which is exactly what evolving safely requires.
• Backfills and reprocessing. Stateful systems frequently need to revisit historical data when the rules change. Designs that can replay or rebuild from a primary source are vastly easier to evolve than designs that cannot.
• Dual-writes and reconciliation. When the old and new are running in parallel, periodic checks that they agree are the only way to know the migration is healthy. Reconciliation tooling pays for itself within one migration.

Principles:

• The cost of a single change is small. The cost of accumulated small changes is what kills systems. Design for the latter.
• The tools you build to evolve a system safely are part of the system. Treat them as production code, not as scripts.
• A change that cannot be rolled back is a change that should not ship. If rollback is impossible by construction, the cost of getting the change wrong has to be priced into the decision to ship it.

The curve on the right side of this chart is what kills systems. Designs that invest in boundaries, contracts, evolution machinery, and clear ownership keep the curve closer to flat. Designs that defer those investments find that the cost of the next change grows on a compounding curve.

Ownership

The last lever is the human one. A system is maintained by the people who own it, and the boundaries of ownership shape how the system can evolve.

A few things tend to be true in practice:

• Every component should have an owner, named on a page that exists. "The platform team owns it" is fine if the platform team has actually accepted the responsibility. "Someone owns it, I think" is not.
• Ownership maps to change velocity. A component owned by one team can change quickly within that team's competence. A component owned by no one will change rarely, badly, or only under pressure. A component owned by three teams will change at the speed of the slowest agreement.
• Conway's law cuts both ways. The system tends to mirror the organisation. That can be used deliberately - design the components so that they map to teams who can own them - or it can happen by accident, in which case the organisation reshapes the system over time.
• Ownership has to be transferable. Teams reorganise. Engineers leave. A system that depends on one person's tacit knowledge will not survive their next career change. Documentation, runbooks, and shared on-call rotation are the mechanisms that make ownership survive turnover.

Principles:

• Make ownership visible. A service catalogue or equivalent, with named owners, is one of the highest-leverage pieces of organisational documentation an engineering org can produce.
• Pay attention to the unowned middle. The most fragile parts of most systems are the integrations between two clearly owned services, where neither owner feels responsible for the seam.
• The org and the system co-evolve. When the org changes, expect the system to want to change shortly after. Designing for that expectation is cheaper than fighting it.

Conclusion

A system survives change to the extent that its boundaries are honest, its contracts are enforced, its evolution machinery is built, and its ownership is visible. None of these four is a property of the original design alone - each one is an ongoing investment that determines whether the system can absorb the next round of requirements without breaking the previous round.

Part 4 closes the series at the failure side of the same question. What does a bad design look like, how do you recognise it before it ships, and what does a good one actually feel like to work in? That is the anti-patterns post.