The scheduler

Every page before this one has told you what an Agent is, how a pool is driven from above, and what a policy decides. This page is about the machine in between: the loop that, every few tens of milliseconds, decides which Agents sample, which results enter the cache, who is dropped, and how a dropped Agent still gets its findings out. It is the heart of an AgentPool, and it was designed to be understood whole.

The design has one idea in it. The decision is separated from the doing, and the decision is a value. Each tick the pool freezes what it knows into a value, hands that value to a pure function, and gets back a second value that says what this tick does. Only then does anything act.

You will not be asked to hold that idea abstractly. The page follows one brief from its first tick to its reports, and introduces each word at the moment the story needs it, so that by the end the whole machine is in your head in the order it runs. The brief is real: three research Agents, one 32k context, pages fetched from the web, the numbers taken from a trace.

It builds on Structured concurrency, Continuous Context and Adaptive compute. The policy page is about what an Agent should do next; this one is about the machine that asks.

1. Before the first tick

A harness has loaded one model and created one context for it. Three words describe what that context is made of, and everything later is accounting over them.

The brief arrives. The harness builds a spine: a branch holding the shared prefix every Agent will fork from, the system prompt and the task. Then it asks the pool for three Agents.

Three spawns are priced at a few hundred cells each. All three fit, all three get a sequence and a seat. The pool is ready to tick.

2. The tick

The pool is a loop, and one turn of the loop is a tick. Every tick has the same five steps in the same order, and the whole of the rest of this page is what happens inside them.

1. Observe. The prune pass reclaims every branch whose work ended last tick, returning its cells. Completed tool calls are taken in. Then the pool samples the cache once: cells in use, cells remaining. That one sample is the only pressure reading this tick will use.
2. Schedule. schedule(state) runs: a pure function over one value, TickState, returning one value, Schedule. It reads no store, calls no model, sends no event.
3. Apply the schedule. Drops are enacted and their recoveries booked; finished reports are parsed and announced; stall outcomes are announced. If the schedule says the pool is done, the loop ends here.
4. Execute. The executor runs the schedule's phases in one fixed order: halt cancelled tool calls, prefill the admitted items, dispatch tool calls, fork and activate admitted spawns, then sample one token from every Agent in the decode set and commit all of those tokens in one batched decode.
5. Apply what came back. An Agent that emitted a stop token has its turn parsed, and the policy says whether it made a tool call, returned, or idled. Calls and landed results become pending work for the next tick.

In the brief's first ticks all three Agents are active: each samples one token per tick, the three tokens are committed together, and the trace shows the pool ticking about twelve times a second. Nothing is pending. The schedule's only non-empty field is decode.

3. Where an Agent is

Three words and one latch say where an Agent is, and the scheduler reads nothing else about it to decide whether it participates.

• active: it samples this tick unless a verdict removes it.
• awaiting_tool: its call went out, or its result is waiting for admission. It samples nothing and decides nothing until admission re-activates it; its branch stays resident, so its cells are still in use.
• idle: its span is over. The next observe prunes its branch and returns the cells.
• disposed: the branch is gone.
• extracting, the latch: a one-way flag beside the status that says this Agent is writing its recovery report. We meet it in section 8.

Participation in the next tick is these rules and no others. An active Agent samples. An awaiting Agent returns only through admission. A dropped Agent's working turn is over; it samples again only for its recovery report, under the extracting latch. A recovering Agent is finished, never dropped. An Agent that became active during execute, from an admitted item or a spawn, samples from the next tick, after the scheduler has had its say on it.

4. The first tool call

Forty seconds in, the first Agent emits a stop token. Its turn is parsed, the policy's onProduced sees a tool call, and a dispatch request is booked. The Agent's status becomes awaiting_tool. Next tick, the executor dispatches the call. Which fiber runs it depends on one declaration on the tool.

Web search is a fan-out tool. The call goes to a child, the Agent waits, the other two keep sampling. Four seconds later the child completes and the loop's next observe takes the completion in. The frame interprets it: a returned value becomes a pending item, the result tokenized on the token rail or measured on the embedding rail, each item knowing its cost in cells; a transient failure becomes a parked retry with a wall-clock time it comes due; a hard failure is settled in the tool's place. The runner records; it does not interpret.

5. A result lands

The search result is a pending item of about 1,200 cells. Whether it enters the cache this tick is admission's decision, and admission is a ledger.

Headroom is 20,000 cells. The item is admitted, prefilled in the execute step, and the Agent is re-activated with its tool-call grammar. It samples again from the next tick, now with the result in its context. This is the rhythm of research: a turn, a call, a wait, a result, a turn.

6. The cache fills

Three Agents fetching pages fill a cache fast. Each page costs one to two and a half thousand cells, and by ninety seconds the three have used twelve thousand. Now the lines matter.

At 150 seconds the trace reads: remaining 8,132, soft line 8,192, headroom −60. All three Agents have just made one more call, nudged by the policy's time budget to wrap up with one last search, and all three results have landed at about 2,000 cells each. None fits. All three are deferred: carried in the pending record for a later tick.

7. Nothing can move

Deferred work is carried as long as something else can free room or make progress. The scheduler decides that with one predicate.

At 174 seconds every term of progress is false: all three Agents are awaiting, nothing is in flight, nothing has been admitted. The stall-break runs. The policy answers nudge for each; each nudge is bigger than −60 cells of headroom; all three Agents are dropped in one tick with pressure_settle_reject. The trace shows the three pool:agentDrop records within eight milliseconds of each other.

This is the moment the next word exists for.

8. Recovery

A drop ends an Agent's span, but not its obligation to report. Every drop carries a recovery, decided at the drop.

The recovery prompt is a pending item like any other, admitted at the cost of its prompt plus its whole share, against the room. On admission the Agent is re-activated with the extracting latch set and the report grammar alone, and it samples again. Its tokens are counted from that moment.

Here the story has two endings, and the difference between them is why the next section exists. In the trace this walkthrough is taken from, the three shares were sized from the cells above the hard line, which at 8,132 remaining against a hard line of 6,144 was a few hundred cells to divide three ways: 342 tokens each. Three reports were cut mid-sentence with six thousand cells unused. Sized from the room, the same tick grants each report the ceiling, and the three reports fit with room to spare.

9. Why a value

It would be easier to decide as you go: sample an Agent, notice the cache is tight, drop it, move on. Most loops are written that way, and they are hard to reason about for a reason that has nothing to do with their size. The decision is spread across time and places. Whether an Agent is dropped depends on who was visited before it, on a cache that moved under it, on an event that arrived in the middle. Nobody can hold the rule in their head because there is no single place the rule is.

Separating the value from the process puts the rule in one place. TickState is a snapshot; it does not change while it is read. Schedule is a description of what will happen; it can be printed, compared, and held against the state that produced it. The pressure sample is taken once, and every later number in the tick is derived from it by arithmetic, so two decisions in one tick cannot disagree about how much cache there is. Time is explicit: a tick is an epoch, and nothing decided in one epoch reaches the next except as the pending record and the Agents' own state.

That is also why the story above could be told from a trace and replayed in a test. The pool:tick record carries the pressure the schedule read; hand the same state to schedule() and it returns the same drops with the same shares. A production run is a sequence of values, and a value can be examined after the fact. The two endings of section 8 were found that way, and the second one was proven that way before it ran.

10. The rest of the loop

The brief's remaining machinery, each piece defined once, in the order the loop meets it:

• Verdicts. After admission, every active Agent receives one verdict in roster order: a wind-down drop if the run is winding down and the Agent is neither recovering nor already emitting the terminal tool; an exit if the policy's shouldExit says so, where a policy returning false vetoes and one that abstains defers to critical pressure; a finish for a recovering Agent at its share or out of room; a terminal cap salvage for an Agent writing a voluntary report past the configured cap; otherwise decode. An Agent mid-terminal-call is salvaged rather than re-prompted: it was already writing its report.
• Retries. A parked retry whose wall-clock time has come is re-dispatched. Under wind-down, the rest are abandoned: settled as honest failures so the drain can finish.
• Spine extensions. An orchestrator's extendSpine is a prefill onto the spine, admitted against headroom like everything else, because the spine is the one branch that cannot be pruned and replayed.
• Wind-down. A signal from the harness, a Stop or a wrap-up, that forces cohort recovery and drops every working Agent with wind_down.
• Cancel. A per-Agent signal. A cancel is decided always, even while paused; it halts the Agent's in-flight tool child and ends the span with no recovery.
• Pause. A hold, not a halt: a paused tick decides only cancels and carries everything else where it is. The run clock excludes paused spans, so time budgets are not charged for a hold.

• Close. The pool closes when the orchestrator is done, every Agent is idle or disposed, and nothing is pending or in flight. A drop decides its recovery at the drop, so an idle Agent is final; there is nothing left to sweep.

11. The shapes a run can take

The brief above took one shape: three Agents fetched pages, stalled together, were dropped in one tick and recovered under shares. A run can take many others, and a reader who cannot name the one in front of them concludes the machine is broken when it is doing exactly what this page describes. So this section names every shape. Each one is a transition the harness has to render. Most are announced on the bus a harness subscribes to; the few that are not, a nudge, a spawn waiting for room, a rerank, are named as such below, so the harness knows what it must infer. There are four families.

Stopped, and kept what it wrote

• It returned. The Agent called the terminal tool and emitted its stop token. The policy's onProduced answers return; the tool's onReturn, the harness's, then the frame's are asked whether the call may end the turn; any of them may reject, and otherwise the first accept stands. Accepted, the result is set and the Agent idles. The bus sees agent:tool_call naming the terminal tool, then agent:return carrying the result, then agent:done. The reader sees the report end on the Agent's own stop token and stand. A rejected return is a nudge instead: the reason goes into the cache in the call's place and the Agent writes again. The frame allows one rejection per Agent, and none once the Agent is out of turns or headroom, because a rejected return costs a turn the Agent must have left.
• It reported past the cap. Only when the harness sets recoveryBudget on the row; with none set a voluntary report runs to its stop token or the hard line. There is no default, because a limit the model was never told must not end a report the room can hold; a row that sets one should put the same number in the report instructions. At the cap the verdict is a drop with reason terminal_cap and a salvage: nothing more decodes, and what it wrote is parsed as its result. The trace records pool:agentDrop and pool:recoveryProduce with the raw text; the bus sees agent:done and at once agent:recovered. The reader sees the report stop at a fixed length and stand.
• It was killed mid-report at the hard line. The critical-pressure verdict, on an Agent already emitting the terminal tool. The policy protects a voluntary report from shouldExit only while pressure is not critical; at critical, protection ends, and the drop's recovery is a salvage rather than an extract, because the Agent was already writing its report and a second prompt would ask it to write it again. The reason is pressure_critical; the bus sees agent:done and at once agent:recovered. The reader sees the report stop before its own end, and the kept text stand. This is the one shape where kept text can be shorter than streamed text; the callout at the end of this section says how.
• Its recovery report ended. The finish of section 8, reached three ways: the Agent emitted its own stop token, its counter reached its share, or the room is gone. The bus sees agent:recovered; the reader, who saw agent:done at the drop, sees the write-up end.

Stopped, and now writing up

Every shape here is a drop whose recovery is extract: a recovery prompt is admitted, the Agent is re-activated under the extracting latch, and a report decodes in-loop. A drop that finds the Agent already writing its report is a salvage instead, and belongs to the family above. The bus sees agent:done at the drop, agent:produce while the report streams, then agent:recovered. What differs is the trigger and the reason on the trace.

• Out of time. The policy's shouldExit answers true because the Agent's run clock passed the time hard limit, and pressure is not critical. The drop reason is policy_exit. The reader sees a working Agent stop mid-search and begin its write-up.
• At the hard line, alone. The post-admission pressure is critical and shouldExit abstains or agrees; the default policy also staggers this to one Agent per tick. The drop reason is pressure_critical. One Agent is sacrificed so that its siblings regain headroom when its branch is pruned. The reader sees one Agent stop while the others continue.
• Together, at the stall. Section 7: every deferred item decided in one tick, reason pressure_settle_reject, each recovery sized by a share of the room. The reader sees every Agent stop within milliseconds of each other and every write-up stream at once. There is no trailing stop on this path.
• Wind-down. The harness sent a wrap-up. The pool halts the orchestrator first, so no new spawn can land against a draining pool, then raises the signal. The next schedule forces cohort shape, drops every working Agent with reason wind_down, and abandons every parked retry as an honest failure the model reads. An Agent already writing its report, voluntary or recovery, is left to finish. The bus sees run:windingDown, then one agent:done per working Agent. The reader sees every Agent begin its write-up together.
• Out of turns, or stopped without a call. The policy's onProduced answers idle because the turn cap was reached, because the policy's soft cut fired, or because the Agent produced prose before any tool call where free text is not accepted as an answer. The drop reason is maxTurns, pressure_softcut or none, and the recovery is decided like any other. The reader sees a working Agent stop and begin its write-up, or fail, as the next family describes.

Could not finish

Every shape here ends on the bus with agent:failed and a reason code, except the last, a spawn that never became an Agent, which ends as a refusal to the orchestrator and no event at all. The text behind a code lives on the trace, never on the bus.

• Too little to recover. At a drop, the policy's onRecovery answers skip: by default an Agent with fewer than 100 tokens or two tool calls behind it is not worth a recovery prompt. The bus sees agent:done, then agent:failed with reason recovery_skipped. The reader sees an Agent stop and fail at once. One harness-visible example is the planner's own turn: a single Agent with no tools writes its plan as prose, stops, is dropped with no reason, and skips recovery, while the tool that ran it reads the raw output and succeeds. The harness sees done then failed on an Agent whose work stood.
• No report in the write-up. The recovery report ended and its parse found no terminal call, or a terminal call with an empty result. The trace records pool:recoveryFailed with the first 200 characters of what the model wrote; the bus sees agent:failed with reason no_terminal_call or empty_terminal_result.
• A tool threw. The completion was an error the frame does not retry. The Agent is discarded, its branch pruned: the trace records tool:error and pool:settleFailed, the bus agent:failed with reason tool_error. A result that could not enter the cache fails the same way with tool_result_failed, and a page or image whose prefill the engine refused with media_prefill_failed.
• Cancelled. The harness cancelled this Agent. Cancels are decided first in every schedule, paused or not; the in-flight tool child is halted, and the span ends with no recovery. The bus sees agent:failed with reason user_cancel and no agent:done before it, so a reader goes straight to cancelled with no write-up in between.
• A spawn refused. The orchestrator's spawn is carried while something can still free room, and refused once nothing can: the trace records pool:spawnRefused with reason pressure_init or no_sequence, and the orchestrator's call rejects. Nothing is emitted on the bus; the Agent never existed.
• The cache ran out under a report. A commit failed with the cache full while recovery reports were decoding. Each in-flight report is announced failed with a reason beginning scope_error, and the pool closes partial.

Delayed, not stuck

The work the reader is waiting for goes on. In most of these the Agent is alive and will sample again. Two are different in kind: a waiting spawn has no Agent yet, and a heal ends one Agent so that a replacement carries its task from the same point. The reader needs to know only what is being waited on.

• Nudged. A call was refused by a gate, a result did not fit, or the policy wants the Agent to wrap up: an error in the call's place enters the cache and the Agent reads it next turn. The trace records pool:agentNudge with the message and the gate that decided; the bus sees nothing. The reader sees the Agent's next turn change course.
• Rate-limited. The tool threw a retryable error inside the frame's attempt budget. The call is parked with a wall-clock due time and re-dispatched when due, under the same call id, with no new agent:tool_call. The bus sees agent:tool_retry with retryAfterMs. A park outlasting the budget becomes a result the model reads, saying the tool is unavailable.
• Waiting for a seat, a sequence or room. A spawn beyond capacity, with no vacant sequence, or whose suffix does not fit the headroom is carried in request order. Nothing is emitted while it waits. The orchestrator's spawn is suspended until it is admitted or refused.
• Reranking a landed result. A result ranked on the main context before it is handed to its Agent shares the one context with the decode, so the loop's pulse drops for its length and every Agent's stream slows with it. The trace brackets it with rerank:start and rerank:end; the bus sees nothing.
• Healed. A page or image's prefill failed on the embedding rail in a way that poisons the branch. The original is discarded with media_prefill_failed. If this is its first poison and the backend is not suspect, the ladder records a lineage: the Agent's history up to its last completed transaction, with the poisoned turn's own text dropped so the replacement regenerates it. The loop forges a replacement spawn at the next observe, after the prune pass, because the replacement needs the sequence the original just gave back; it is priced whole, admitted by fit, and its lineage replayed onto a fresh fork. The trace records pool:agentHeal naming both; the bus sees agent:failed for the original, then agent:spawn for the replacement carrying the same key. The reader sees one Agent fail and a new one begin from the same point. A heal that cannot be forged or admitted stands down, and the original's failure stands alone.
• Paused. The harness paused the run. The loop holds at the tick boundary: no sample, no commit, no settle, no dispatch, cancels still decided. Branches stay resident. The run clock stops, so time budgets are not charged; retry parks stay on the wall clock. The bus sees run:paused, then run:resumed with the paused span.

12. Where structured concurrency sits

The pool is an Effection resource, and every fiber above is inside it. That is what makes the loop safe to reason about.

• The tick loop is one spawned task under the pool's scope, and it is the only fiber that writes to the cache. Fan-out children are spawned under the same scope with an ensure that releases their permit and clears their in-flight mark, whatever way they end.
• A cancel is a halt. Cancelling an Agent with a tool in flight halts that child. The halt unwinds through ensure, not catch, so a halted child pushes no completion and the loop never sees a ghost result.
• Nothing outlives the pool. When the orchestrator errors or the harness stops, the scope ends: the loop, every child, every branch. When the orchestrator merely finishes, the pool closes only once every Agent is final and nothing is waiting, as the close rule in section 10 says. The pool's own ensure releases unadmitted spawns and prunes every Agent. There is no teardown code to forget because nothing lives outside the scope.
• An Agent is not a Task. It is a branch owned by the pool; the pool is the Task. Stopping an Agent is a drop with a recovery; stopping the pool is a halt of the scope. Structured concurrency explains why that distinction holds and what goes wrong when it is ignored.

13. The invariants

Everything above, as the rules the suite locks:

• One pressure sample per tick, taken after the previous tick's effects landed; every decision in the tick reads it or an arithmetic derivation of it.
• The ledger only spends; refunds arrive with the next sample.
• One drop per Agent per schedule, and a drop decides its recovery at the drop.
• A working turn never samples below the hard line; a report never writes below the floor; the hard line never sits below the floor.
• A recovering Agent is finished, never dropped; finishing salvages what it wrote.
• Every stalled item is decided in the tick the stall is detected.
• Fan-out completions are interpreted on the loop fiber, never on the child.
• The cache has one writer: the loop fiber.
• The pool closes only when every Agent is final and nothing is waiting or in flight.

14. Read it in a trace

The trace names the phases, and because the decision was a value, the trace is enough to reproduce it. pool:tick carries the pressure the schedule read; pool:agentDrop carries the reason; pool:agentNudge records a nudge whether or not it fit; pool:recoveryProduce and pool:recoveryReturn carry a report's token count and length; branch:prefill carries the cells an admission cost; tool:dispatch, tool:result and tool:retry bracket each call. Line those up against the tick rate and the pressure curve and a run reads like a kernel log. Debug with traces shows the tooling.

The words, in the order you met them

Related

• Adaptive compute: the policy side of the same machine, the lines, and what an Agent should do at each.
• Continuous Context: why one cache, forks and prunes are the right primitive for many Agents.
• Structured concurrency: the ownership model the pool lives inside.
• Agents and orchestration: the shapes that drive the pool from above.