Observability

Ask an agent why it decided something and it will tell you, fluently, and the answer is worthless as evidence. It describes a system in its current state, not the one that made the decision. What has to be captured at decision time instead, why corrections belong as new events rather than edits, and why the record is not a debugging convenience but the precondition for everything else in an agentic system.

Tests check whether code does what it was written to do. They have nothing to say about whether a judgement was any good. Scoring a non-deterministic system means comparing recorded predictions against observed outcomes, and most systems already emit both without recognising that they have the raw material for an eval harness. Why calibration matters more than accuracy, what bucketing is for, and the two problems - outcome lag and correlated error - that make this harder than it looks.

Closing the frontend track and the pillar. The client faces the same scale and failure the backend does, and has to stay trustworthy through it: graceful degradation, retry with backoff, offline and flaky-network handling. Then the cross-cut that ties the pillar together, auth that scales, client-side rate limiting, and frontend observability that closes the loop back to knowing your numbers. Plus the native reality of un-hot-fixable clients.

The infrastructure that carries an async system in production. Sharding the data layer past the single-primary ceiling, autoscaling that respects the real bottleneck, backpressure so overload degrades instead of collapsing, and the security and observability that make an asynchronous system operable at all. Closing the backend track.

The five things every system design has to make concrete decisions about: data, state, control flow, failure, and observability. Each one is a hinge point where good and bad designs diverge.

The auth feature from Part 4 ships. CI/CD agents run the deploy. Observability MCPs - Grafana, Sentry, incident.io - watch production. A few weeks in, an RBAC misconfiguration surfaces. An incident agent triages, fixes, drafts the postmortem, and updates the docs. Part 5 traces the back-arrow from production learning to next-iteration spec, and is honest about where the loop genuinely closes versus where the demos overstate it.