Agentic Systems: One Interface, Many Models

This is Part 3 of Agentic Systems: Bounds and Evidence, the second series in a pillar on building agentic systems.

• Part 1: Bounded Tools, Not Free-Form Access: constraining what an agent can express, not just checking what it produced.
• Part 2: The Ledger Is the Product: what gets recorded at decision time, and why reconstruction after the fact does not work.
• Part 3: One Interface, Many Models: the model layer underneath, and what swapping it does and does not give you.

Part 2 ended on a field in the decision record: the model and prompt version used for each call. That field is only interesting if those things change. This post is about the layer that lets them change, which is the least glamorous decision in an agentic system and one of the few that is genuinely expensive to reverse.

Table of Contents

• The Coupling You Do Not Notice Until You Need to Move
• What the Interface Should Hide
• What It Must Not Hide
• Per-Specialist Selection Beats One Default
• Swapping Is a Config Change; Trusting the Swap Is Not
• Fallback Paths Have to Be Exercised
• The Honest Bill
• When NOT to Abstract
• Conclusion

The Coupling You Do Not Notice Until You Need to Move

Every engineer with a few years behind them has an opinion about hardcoding a database vendor into business logic. The opinion is usually correct and usually learned expensively: it works fine until the day it does not, and by then the coupling is everywhere, because nothing ever forced it to be in one place.

Model providers are the same mistake at a higher layer, and it is new enough that the scar tissue has not formed. A direct provider SDK call inside a specialist's logic is convenient, works immediately, and spreads - one call becomes nine, each with slightly different parameters and error handling, and the provider's request and response shapes leak into the code around them.

The bill arrives in one of four ways, and at least one of them will happen:

• An outage. The provider has a bad hour. Without a fallback path, the entire panel is down, and a system that was carefully designed for one specialist degrading gracefully fails completely because all nine share a dependency.
• A price change. The economics that justified running nine specialists per decision shift under you, and the architecture becomes uneconomical without anything about it having changed.
• A better model. Something materially better ships, and capturing it costs a migration instead of a config edit.
• A constraint you did not have before. Data residency, a procurement rule, a customer who will not accept a particular provider. These arrive from outside engineering and are not negotiable.

None of these are hypothetical, and the response to all four is the same piece of architecture.

What the Interface Should Hide

The abstraction worth building is narrow: a specialist declares what it needs - a prompt, a context, an expected output shape - and something else decides which model serves it and how.

Hidden behind the boundary: request and response formats, authentication and endpoint configuration, retry and timeout handling, token accounting, and the mechanics of structured output. These differ across providers in ways that are purely incidental to what any specialist is trying to do, and letting them leak is what makes a migration expensive.

One workable shape leans on an off-the-shelf abstraction layer - the Vercel AI SDK and comparable libraries exist for exactly this - with the provider swappable per environment: cloud models for anything review-facing, where output quality is what a human is about to judge, and local models for the development loop, where iteration speed and cost dominate and the last few points of quality do not matter. A rule worth attaching to it: fall back to hand-rolled provider calls only where the abstraction actively fights the use case. That is the correct amount of commitment. An abstraction you are forbidden to escape is just a different kind of hardcoding, and the escape hatch being rare is what keeps it honest.

What It Must Not Hide

An abstraction designed to make every model look identical will also conceal the specific ways they are not, which is exactly the information you need when something degrades.

Three things must remain visible through the boundary:

Model identity, in the ledger. Part 2 put model and prompt version in the decision record for this reason. If the abstraction hides which model served a call, then a calibration drop is uninterpretable - it could be a prompt change, a data shift, or a silently rerouted model, and there is no way to tell them apart. An abstraction that makes model identity invisible destroys the ability to attribute behaviour to cause, and that is a much worse outcome than the coupling it was built to remove.

Capability differences. Providers differ in structured-output support, context limits, prompt caching, and extended reasoning modes. A lowest-common-denominator interface that hides all of this leaves real capability unused; an interface that exposes it as optional, with explicit degradation when a provider lacks it, keeps the choice visible at the call site where someone can reason about it.

Cost and latency per call. These are the inputs to the fan-out budgeting that the next series takes up, and they have to be observable per specialist. An abstraction that reports aggregate spend without attribution makes it impossible to find the specialist responsible for most of the bill.

Per-Specialist Selection Beats One Default

Once model choice is configuration rather than code, the obvious next question is whether every specialist needs the same model. Usually not, and treating them uniformly is the most common source of unnecessary cost in a panel.

The specialists in a panel differ enormously in what they actually require. A specialist scoring whether an indicator crossed a threshold, with a short prompt and a narrow enumerable output, is close to a deterministic computation with a language-model interpretation layer on top. A specialist doing genuine synthesis across ambiguous evidence is not. Giving both the largest available model is defensible only if nobody has measured the difference, and the difference is usually large in one direction and negligible in the other.

A signal-driven trading panel splits along a related axis: heavier reasoning work - sentiment analysis, thematic discovery, final synthesis - runs against a cloud model, with a local model available as fallback. The framing there is resilience as much as cost, which is the useful observation: the same swappable-provider interface serves cost control, latency management, and availability, depending on which constraint is binding at the time. Building it for one of those three gets the other two for free.

The selection should be per specialist and per environment, because the right answer differs: the development loop wants cheap and fast, CI wants deterministic and cheap, production wants whatever the specialist's calibration was measured against.

Swapping Is a Config Change; Trusting the Swap Is Not

This is the part that gets skipped, and it is where the abstraction becomes actively dangerous rather than merely incomplete.

Making a model swap syntactically trivial does not make it behaviourally safe. Prompts are tuned, deliberately or accidentally, against the behaviour of a specific model - its verbosity, its structured-output reliability, its tendency to hedge, how it handles an instruction placed at the end of a long context. Swap the model and those assumptions silently stop holding. The system still runs. The schema still validates. The specialist is simply worse in ways that will not surface until enough outcomes accumulate to move a calibration figure, which may be weeks.

So the gate on a model swap is the eval machinery from the first series, and this is precisely where the offline-versus-online distinction pays for itself:

• Offline golden sets are the right instrument before the swap. They answer whether the new model is better or worse on cases already judged, immediately, without waiting for production outcomes. A system with no golden set cannot evaluate a model change in advance at all - it can only ship and watch.
• Online calibration is the instrument after. It catches the degradation that the golden set's case selection missed, on the real distribution, and it is the reason model identity has to be in the ledger: so the drop is attributable to the swap rather than appearing as unexplained drift.

A swap should be treated as a change to the system's judgement, because that is what it is - evaluated, staged, and reversible. The abstraction makes the mechanics cheap. It does not make the verification optional, and the easier the swap becomes the more tempting it is to skip the part that was never automated.

Fallback Paths Have to Be Exercised

A fallback that has never run outside a demo is a hypothesis.

The failure modes are mundane and all of them are discovered at the worst time: expired credentials for the fallback provider, a prompt that exceeds the fallback model's context limit, a structured-output mode the fallback does not support, or a fallback so much slower that it breaches the latency budget and fails differently than the outage it was meant to absorb.

Exercise it deliberately. Route a small share of production traffic through the fallback continuously, or fail over on a schedule. This costs a little quality on a small slice of traffic and buys the knowledge that the path works - and, just as usefully, produces calibration data for the fallback model, so that when it is carrying real load there is some basis for knowing how much to trust it.

The Honest Bill

The development-divergence line is underrated. Running cheap local models in the loop and a frontier model in production means the thing being tested daily is not the thing that ships, and a prompt change that helps locally may do nothing or harm in production. The mitigation is to run the offline golden set against the production model configuration before merging, regardless of what the development loop uses.

When NOT to Abstract

• One model, no plausible reason to change. An internal tool on a provider the organisation has standardised on, with no residency or cost pressure, does not need an abstraction layer. Direct calls are honest, and the cost of adding the layer later is not catastrophic for a small surface.
• The provider's distinguishing capability is the reason you chose it. If the architecture depends on a specific feature - a particular caching behaviour, a specific reasoning mode, a tool-use format - then abstracting over it either leaks that dependency anyway or discards the advantage. Couple deliberately and write down why.
• Prototype stage. Find out what the system needs before designing the interface that serves it. The failure is not the direct call; it is never revisiting it.
• Only one specialist calls a model at all. A single call site is not a coupling problem. Abstraction earns its keep across many call sites.

Conclusion

A model provider hardcoded across a panel of specialists is a coupling that turns an outage into downtime, a price change into an architecture problem, and a better model into a migration. The fix is a narrow interface that hides transport, authentication, retries, and output mechanics, while keeping model identity, capability differences, and per-call cost and latency visible - because those are what make behaviour attributable. Select per specialist and per environment rather than globally, exercise the fallback path in production, and treat every swap as a change to the system's judgement requiring evaluation rather than a configuration edit that happens to be easy.

That closes this series. What an agent can do, what the system records about it, and what it runs on. The final series turns to running the thing: where state lives, what a panel actually costs per decision, and what happens when one specialist starts being quietly wrong.