Graceful Shutdown Contracts Between Orchestrators and Coding Agents at Budget Thresholds

AI coding agent budget cap enforcement needs an orchestrator that sets aside spend before each call and turns down work that would cost more than the budget left. Graceful shutdown then requires a contract that stops new dispatches, resolves in-flight tool calls, checkpoints state, and emits a structured termination payload. A spending limit without that protocol can replace an unexpected API bill with corrupted work and human repair costs.

Why a budget threshold without a shutdown contract is not a safety control

A hard stop has a human remediation cost. Open transactions, half-merged branches, and partially written migrations can leave state that takes more effort to repair than the stopped execution saved. The agent has stopped spending, but the orchestrator now owns the unfinished work. A safety control must therefore govern both the spending boundary and the transition out of execution.

An alert addresses neither obligation on its own. In the Analyzer-Verifier loop incident, four LangChain agents ran for nearly two weeks and burned tens of thousands of dollars despite observability and alerts. There was no mechanism to stop execution before the next API call completed. Sending the full conversation history in each new request can push agentic cost scaling closer to O(n²) than O(n), so low costs in tests can hide how much a loop could spend. Monitoring reports that exposure, while enforcement prevents the next call and the shutdown contract determines what happens to work already underway.

What a shutdown contract actually specifies

The contract assigns obligations in a fixed order. First, the orchestrator fails readiness so that new work is no longer routed to the agent. Second, the agent stops taking new work but keeps its listener open long enough for draining. Third, work still in progress drains until none is left or the hard deadline fires. Fourth, resources close only after in-flight operations have committed or rolled back cleanly, with logs flushed, state checkpointed, and session credentials revoked before exit.

These stages separate dispatch authority from completion responsibility. The orchestrator controls whether another call may begin, while the agent must account for work already dispatched and preserve its results. The structured exit signal follows that preservation step, rather than merely announcing that termination has started. Otherwise, the orchestrator could receive a completion signal before the state it describes is durable.

The deadline is part of the protocol itself. For coding agents in Kubernetes CI pods, the application’s drain deadline must be strictly shorter than terminationGracePeriodSeconds. Kubernetes sends SIGKILL when that grace period ends, regardless of whether cleanup has finished.

How to enforce coding agent budget caps outside the context window

A system-prompt spending instruction asks the agent to obey a policy within the same reasoning process that is trying to finish the task. If the agent can modify the control that stops execution, the control depends on its decision to cooperate. Testing instruction-following does not remove that architectural dependency.

The budget enforcer must instead be a layer every call passes through. The orchestrator maintains the ledger and rejects an unaffordable call before dispatch, outside the agent’s context window. A BudgetExceededError can then reach the orchestrator without giving the agent an opportunity to place another call. The agent may observe its remaining allowance, but it cannot rewrite the spending rule or disable rejection.

A no-new-tool-calls flag checked at tool boundaries helps the agent enter the drain protocol, but it is not the sole barrier against dispatch. The external gate remains authoritative even if the agent ignores that flag. The control stack combines provider limits, cross-provider orchestration enforcement, and agent-side cleanup, so spending stays capped even if the agent does not help with cleanup.

The six budget dimensions the orchestrator must track before each call

One aggregate dollar cap detects several failures too late. The orchestrator needs six dimensions checked before dispatch, beginning with per-call and cumulative input/output token limits to contain prompt bloat and context-window failure. Model-call limits constrain total LLM invocations per run and per step, stopping repeated escalation when an answer remains uncertain. Tool-call limits must be specific to tool type because noisy web searches and risky database writes should not consume one undifferentiated allowance.

Retry budgets form a separate control dimension. They set limits by failure class, with some retries for transient errors and none for validation errors, and tool-specific caps on how often to try again. Dollar caps set a hard limit on spending and reconcile actual usage after each call. Time or wall-clock limits set a session deadline no matter how much has been spent, so repeated cheap calls cannot keep a low-value run alive forever.

Concurrency makes the accounting order decisive as well. A central budget manager backed by Redis or Postgres reserves spend before each call, then reconciles actual usage afterward. This reserve-then-reconcile pattern prevents two concurrent calls from both treating the same remaining funds as available. OpenAI’s per-project hard limit stops requests with a 429 when tracked spend reaches the limit, but it cannot account for Anthropic, a search API, or other paid tools. The orchestration ledger must close that cross-provider gap.

Staged degradation before the shutdown signal fires

A predeclared downgrade policy preserves opportunities to finish cheaply. As the budget falls, the contract can lower model cost, compress context, and limit optional tools before it moves from unrestricted operation to termination. The agent can choose work within the permitted state, but it cannot rewrite the policy when task pressure rises.

At roughly 70% of remaining budget, routine planning switches from a frontier model to a fallback. Past the halfway point, context compression applies. At a further reduced threshold, tool gating disables web search first and then non-critical retrieval while retaining required database reads. The thresholds belong to the workload’s configuration.

Progressive Interval Estimation estimates upper and lower bounds on remaining expenditure at each planning step, allowing a mode change before a spending threshold when completion appears infeasible. Retry circuit breakers add to that check with closed, open, and half-open states, stopping calls to a flaky tool once it crosses its failure threshold. The pi-subagents package shows where orchestration hooks can help, with child-to-parent token accounting, completion notifications at the parent’s next boundary, and explicit commands to steer or stop running agents.

What the shutdown signal must do to in-flight tool calls

The shutdown signal withdraws permission to start new work. Readiness must fail before draining begins, and the dispatch boundary must reject new work while existing calls remain accounted for. If that order is reversed, new calls can enter while the workload is meant to be draining.

A drain counter tracks tool calls as they start and finish. Exit is permitted when that counter reaches zero or the hard deadline expires. A sys.exit() after sleep(5) can appear to allow completion while actually raising SystemExit in the main thread and killing open transactions on worker threads.

Tool classification determines what the drain must preserve. Read-only file access, search, and retrieval can be classified as safe-to-abandon without state corruption. Database writes and file mutations require a must-complete-or-rollback classification, enforced by the orchestrator, because termination between BEGIN and COMMIT can undo work a downstream system already believes succeeded. Each tool’s drain deadline must be shorter than the session-level grace period, which must itself be shorter than terminationGracePeriodSeconds. These nested deadlines leave time for the inner protocol to finish or roll back work before the outer termination mechanism takes over.

Checkpointing state so the agent's work survives the shutdown

Without a checkpoint, the orchestrator can know that execution stopped cleanly yet lack the information needed to proceed. Durable execution therefore requires persistence of completed steps so that a crash or budget-triggered stop resumes from a checkpoint. For teams evaluating that capability, the test is to kill the process and examine recovery.

The shutdown checkpoint must describe the actual completion boundary. It captures completed steps, the last committed tool-call result, remaining budget, the next planned step, and any partial output a downstream system can use. Writing this record after draining updates the saved state from an earlier view of outstanding work to resolved tool outcomes. That distinction lets a reviewer decide what to accept and what needs another run.

Credentials minted for the session must be revoked when it ends, rather than remaining valid until someone resumes the task. Separately, attribution requires each tool call, diff, and token to be logged against a run_id and step_id that trace back to a configuration version committed by a named author. The agents-as-code pattern provides that lineage through repository-managed YAML. The checkpoint then has both usable state and an identifiable governing policy.

The structured termination payload the orchestrator needs to act on the shutdown

The exit payload must support a routing decision without manual log reconstruction. It contains status, reason code, consumed budget, remaining budget, completed steps, the blocked next step, saved checkpoint state, and a suggested next action. The action can be human review, a narrower rerun, or a higher-budget policy path.

Reason codes distinguish operationally different kinds of termination. A budget-threshold stop is not interchangeable with a tool failure, an open circuit breaker, a validation error, or a policy rejection. Downstream systems need that distinction to select the appropriate path. The suggested next action must also reflect the final feasibility estimate, explicitly stating when the remaining task requires more budget than any configured policy allows. Otherwise, a rerun can repeat the same infeasible attempt.

The payload also records why authorized execution ended. Combined with the checkpoint’s configuration lineage, it connects the enforced limit, the preserved work, and the proposed continuation to the policy governing the run. That connection gives a reviewer evidence beyond a process exit code. For compliance purposes, the termination payload becomes a governance artifact that explains the decision and its operational consequence.

Encoding the contract as versioned configuration before the agent runs

A runtime-only policy lacks a reviewable history of authorization. The shutdown contract must be declared in versioned configuration in the repository with the agent definition. YAML can define per-agent budget caps, maximum iteration limits, downgrade rules, tool-call limits by type, and alert thresholds. Review then applies to the policy diff before execution, rather than to an inferred description of deployed behavior.

Version control makes a run’s governing policy reconstructable. Prompt changes, guardrail updates, and policy revisions are recorded with a diff, author, approval signature, and deploy timestamp. The platform pulls the versioned configuration at deploy time, linking runtime behavior to an approved revision. This also creates a defined object to review or roll back when a threshold or drain rule proves unsuitable.

When control flow is Python or YAML that the team compiles, tests, and commits, moving the runtime to another container host can be a deployment change. When control flow exists only as a diagram in a vendor’s canvas, the policy resides in someone else’s database. Repository ownership gives the shutdown contract a form that can survive a platform migration.

Governance ultimately asks who authorized the policy, what version ran, and whether the shutdown decision can be reconstructed. A versioned contract connects those answers to the budget ledger, checkpoint, and termination payload. Graceful shutdown is therefore not a courtesy performed after enforcement, but part of the approved control that makes autonomous execution accountable.