Agentic Systems: Orchestrators and Specialists

This is Part 1 of Agentic Systems: The Decision Layer, the first series in a pillar on building agentic systems.

• Part 1: Orchestrators and Specialists: how many agents, what each owns, and the contract between them.
• Part 2: Autonomy You Have to Earn: routing a decision to a human or to execution, and why that gate has to be earned per decision.
• Part 3: The Eval Harness You Already Have: scoring specialists against outcomes, and feeding that score back into the system.

There is a difference between using agentic tools and building an agentic system. The first is about supervising something someone else built, which an earlier series on this blog covered at length. This pillar is about the other side: designing the thing. The first structural decision you make is how many agents there are and what each one owns, and it is the decision most systems get wrong in the same direction, for the same understandable reason.

Table of Contents

• The Single-Agent Default
• Why a Composite Cannot Be Evaluated
• The Split: A Thin Orchestrator Over Narrow Specialists
• Two Shapes: The Panel and the Pipeline
• The Contract Is the Whole Design
• The Orchestrator Must Stay Thin
• The Honest Bill
• When NOT to Split
• Conclusion

The Single-Agent Default

Take a concrete task: triaging an incoming production alert. Something useful has to happen in the next sixty seconds, and it requires four different kinds of judgment. How severe is this, read from the error rate and its trajectory? Is it related to the deploy that went out forty minutes ago? Which service actually owns the failing path? And given all that, does someone get paged at 3am or does this wait until morning?

The obvious build is one agent. Give it the alert payload, the recent deploy log, the service ownership map, and the on-call policy, and ask it to decide. It will work. It will produce a decision with a paragraph of reasoning that touches all four questions, and that paragraph will often be right.

This is the right way to start, and the demo is genuinely impressive. The problem arrives later, and it does not announce itself as an architecture problem. It shows up as an improvement problem.

Suppose this agent pages the wrong team on a Tuesday. The reasoning it emitted reads plausibly: it assessed severity as high, connected the failure to a recent deploy, attributed the path to the checkout service, and paged checkout's on-call. Somewhere in that chain is the error. Was the severity read wrong? Was the deploy correlation spurious? Was the ownership attribution stale because the service map changed last month? The output is one artifact in which four judgments have been woven together fluently enough that they cannot be pulled apart and checked individually.

Now try to improve it. Every available lever - a better prompt, more context, a stronger model - acts on all four competencies at once. Tighten the instructions around ownership attribution and severity assessment shifts too, because they were never separate things. There is no way to fix one judgment while holding the other three still, because in the artifact you are looking at, they are not three other things. They are the same paragraph.

Why a Composite Cannot Be Evaluated

The improvement problem has a precise cause, and it is worth stating exactly, because it is the argument for everything that follows.

Say the triage agent is correct on 80% of alerts. That number is nearly useless. It is an average over four unrelated competencies, and averages hide their distribution. A system at 95% on severity, 95% on deploy correlation, 95% on ownership, and 35% on the paging decision also lands near 80%. So does one that is mediocre at everything. These are completely different systems with completely different fixes - the first needs one competency rebuilt, the second needs the whole approach reconsidered - and the aggregate accuracy cannot distinguish between them.

This matters more than it first appears, because reliability is the currency everything else in an agentic system is bought with. Part 2 argues that autonomy should be granted per decision based on demonstrated reliability, and Part 3 is entirely about measuring that reliability and feeding it back. Both are impossible if the smallest unit you can score is a composite. A single agent doing four jobs has exactly one reliability number, which means it has exactly one autonomy setting available: all of it, or none of it.

The split into specialists, then, is not primarily about accuracy. A panel of narrow agents is not automatically more correct than one broad agent, and anyone promising that should be read skeptically. The split is about decomposability - creating units small enough that reliability means something specific, and therefore small enough to improve, to trust differentially, and to replace.

The Split: A Thin Orchestrator Over Narrow Specialists

The structure that follows from this has two kinds of component.

A specialist owns exactly one competency and nothing else. It sees only the inputs that competency requires, and it emits a structured, scored judgment about that competency alone. The triage system's severity specialist reads error rates and trajectories and emits a severity assessment with a confidence. It does not know the deploy log exists. It has no opinion about who gets paged, and cannot form one, because nothing about paging is in its inputs or its output schema.

An orchestrator owns no domain competency at all. It collects the specialists' judgments, weighs them, resolves the disagreement between them, and routes a decision. That is its entire job. It is deliberately thin, and the discipline of keeping it thin is most of what makes the pattern work.

This is not a new idea, and the framing that makes it click for most engineers is that it is the bounded-context argument applied one level up. The reason you do not build a service owning four unrelated responsibilities is that you lose the ability to reason about, test, deploy, and fix any one of them independently. The same reasoning holds when the unit of work is a judgment rather than a function call, and it holds for the same underlying reason: shared state and shared surface destroy independent evaluability. Anthropic's applied-engineering writing names roughly this shape, orchestrator-workers, among a small set of agentic patterns worth reaching for, alongside the broader advice to compose simple well-understood pieces rather than build one large autonomous agent.

Two Shapes: The Panel and the Pipeline

"Orchestrator plus specialists" describes two structurally different arrangements, and conflating them causes real design mistakes. The difference is whether the specialists are independent.

In a panel, several specialists examine the same decision concurrently and each emits an opinion. They do not see each other's output. The orchestrator aggregates the panel into one decision.

A signal-driven trading system is a natural panel. Nine researchers run continuously and independently - momentum, technicals, market regime, sentiment, risk, opportunity scanning, thematic discovery, event calendars, ETF decomposition - each publishing scored signals onto a shared bus without any knowledge of what the others are saying. An aggregation layer downstream collects those signals, weights each researcher by its own calibrated track record, and routes the result. The nine are genuinely independent: momentum's assessment does not depend on sentiment's, which is exactly why their errors are uncorrelated and their individual accuracies are separately meaningful.

In a pipeline, specialists run in sequence, and each one's output is the next one's input. There is no aggregation, because there is nothing concurrent to aggregate. The orchestrator's role shifts from weighing opinions to sequencing stages and deciding whether each stage's output is fit to hand onward.

A tender-response system, of the kind used in public-sector procurement, is a pipeline. A shredding stage breaks a tender pack into individual requirements, each traced back to its source clause. A matrix stage turns those requirements into a compliance matrix with mandatory items flagged as the disqualification risks they are. A drafting stage proposes first-pass responses with reasoning and source citations attached. You cannot run these concurrently: there are no requirements to structure until the shred has happened, and nothing to draft against until the matrix exists.

The distinction has direct consequences. A panel's cost is fan-out - N model calls for one decision, in parallel, which Part 2 of the operations series takes up in detail. A pipeline's cost is depth: total latency is the sum of its stages, and an error in stage one propagates into every stage after it, which a panel's independence specifically prevents. Panels need aggregation logic and disagreement handling. Pipelines need validation at each handoff. Reaching for the wrong shape means solving the wrong problem carefully.

The Contract Is the Whole Design

The interface between specialist and orchestrator is where this pattern is actually won or lost, and it comes down to one rule: a specialist hands upstream a structured, scored judgment, never prose.

A severity specialist that returns { severity: "high", confidence: 0.82, basis: "error_rate_trajectory" } has produced something an orchestrator can combine arithmetically, something a calibration job can score against what actually happened, and something a reviewer can check at a glance. A severity specialist that returns three fluent sentences about how the situation looks concerning has produced something the orchestrator must now interpret, which means the orchestrator has quietly acquired a natural-language understanding job nobody designed it to have.

Once that happens, the split has already failed. The orchestrator is no longer thin. The specialist's output cannot be scored without a human reading it. And the coupling between them is now semantic rather than structural, which means neither can be changed confidently without re-checking the other.

The practical test is whether you can write the specialist's output as a schema. If you can, the boundary is real. If the honest answer is "it returns an explanation," the boundary is a naming convention, not a design.

Note that "structured" does not mean the reasoning is thrown away. The tender system's drafting stage emits prose - it has to, the output is a tender response - but it emits prose bound to structure: pinned to the specific source clause and to the model and prompt version that produced it. The free text is the payload. The structure around it is what makes the payload checkable.

The Orchestrator Must Stay Thin

The most common way this architecture decays is not a dramatic failure. It is the orchestrator slowly acquiring domain judgment, one reasonable-sounding addition at a time.

It starts with a sanity check: the paging decision looks wrong when severity is high but the error rate is falling, so the orchestrator gets a rule about falling error rates. Then another: ownership attribution is unreliable for one particular service, so the orchestrator gets a special case. Each addition is individually defensible and locally correct. Cumulatively, the orchestrator is now re-deriving severity and ownership judgments that specialists were supposed to own, the specialists' scores no longer reflect the system's actual behavior, and the calibration machinery in Part 3 is measuring the wrong thing - because it scores the specialists, and the orchestrator is now quietly overriding them.

The useful heuristic: if the orchestrator needs information a specialist had but did not pass along, that is a signal to change the specialist's output schema, not to give the orchestrator access to the raw input. The moment the orchestrator reads primary data, it has become a specialist that also happens to route, and the thing being routed is its own opinion.

The Honest Bill

The split is not free, and several of its costs are paid immediately while its benefits accrue slowly. That asymmetry is exactly why the single-agent version keeps winning demos.

The heaviest line is the second. One decision that used to be one call is now several, and for a panel they are concurrent, so the cost multiplies immediately even though latency may not. The aggregation layer is the sneakiest: it is new code that did not exist in the single-agent design, it sits on the critical path of every decision, and it is easy to under-test precisely because it feels like plumbing rather than logic.

Disagreement deserves particular attention, because it is a genuinely new class of problem. When four specialists split two against two, something has to resolve it, and "the orchestrator weighs them by calibrated accuracy" is an answer that only works once Part 3's machinery exists. Before that, weighting is guesswork with a confident interface.

When NOT to Split

Because the bill is front-loaded, knowing when not to pay it matters as much as knowing the pattern.

• The task is genuinely one competency. Summarizing a document, classifying a ticket, extracting fields from a form - these are single judgments. Splitting them produces coordination overhead around work that was never decomposable, and the fragments are not independently meaningful.
• You cannot observe per-specialist outcomes. The split's main benefit is measurable per-competency reliability. If the only observable signal is whether the final decision was good, with no way to attribute that back to individual specialists, you have paid the full cost for a fraction of the benefit. Check that attribution is possible before designing the panel around it.
• The latency budget will not absorb fan-out. A panel's floor is its slowest specialist. For interactive, sub-second paths, that floor may already exceed the budget, and a single well-scoped agent may be the only shape that fits.
• You are splitting by pipeline stage rather than by competency. "Fetch the data" and "parse the data" are two stages of one job, not two specialists. Splitting there adds hops and contracts without creating anything independently evaluable. Split where the kinds of judgment differ, not where the processing steps do.
• The specialists would be substantially correlated. Two specialists scoring nearly the same thing from slightly different angles look like redundancy, which sounds safe. It is not: correlated errors mean neither is independently meaningful, and the aggregate is falsely confident precisely when both are wrong together.

The discipline is the same one that governs any decomposition: split where the seams actually are, not where the diagram looks tidiest.

Conclusion

A single agent handed a multi-competency task produces a composite judgment, and composites cannot be decomposed, measured, or improved in parts. The fix is a thin orchestrator over narrow specialists, arranged as a panel when the judgments are independent and as a pipeline when each stage feeds the next, held together by structured scored contracts rather than prose. The bill is real - more calls, more components, an aggregation layer, and disagreement to resolve - and it is worth paying only when the competencies are genuinely distinct and their outcomes are separately observable.

What the split buys, mostly, is the ability to say something precise about reliability. Part 2 spends that: once you can measure which specific decisions a system is reliable at, autonomy stops being a global switch and becomes something individual decisions earn, and can lose.

Sources and further reading:

• Anthropic - "Building Effective Agents" - orchestrator-workers among a small set of agentic workflow patterns, and the case for composing simple components over one autonomous agent.
• Sam Newman, Building Microservices - the bounded-context and independent-deployability arguments this post applies one level up.