Detecting Anomalous Agent Behavior From Session Logs

A coding agent that runs unattended for hours at a time doesn't fail the way a bad autocomplete suggestion fails. It fails the way a production system fails: with downtime, with deleted infrastructure, with a cost spike that nobody notices until the invoice arrives. That shift, from a tool a developer watches to a system that acts on its own, changes what a failure actually costs, and it changes what counts as evidence after something goes wrong.

Coding agents running autonomously in production make session logs the primary safety instrument

An agent operating with elevated permissions for an extended stretch of unsupervised time turns a logic error into an incident, not a diff someone catches in review. The gap between a small drift in behavior and real damage used to be closed by a human looking at a pull request. Now that gap can run for hours or days, long enough for a minor deviation to compound into something that takes down a service or exposes a credential. When that happens, the only record of what the agent was doing beforehand, and why, is the session log. Enterprise deployment guidance treats a set of controls as non-negotiable for agents running in production: SSO integration, centralized audit logging connected to a security monitoring system, secret scanning on agent pull requests, PR policy gates, license governance, sandbox isolation, and incident response runbooks. Most teams that have already moved agents into production have not implemented all of them. That leaves the session log as the only forensic record many of these teams actually have when something breaks.

Gaps in standard application logs for agent behavior

A standard application log tells you what the agent reported. It does not tell you what the agent did, and for spotting behavior that has gone wrong, that gap is the whole problem. Ordinary logging applied to an agent instead of a conventional service shows three specific weaknesses. The log often has no contextual chain linking a user's original request to the string of sub-tasks the agent generated to pursue it across multiple tool calls. It records that an API got called, but not the reasoning the agent used to decide that call was the right one. A security team reading the log after the fact has no way to tell whether a given action made sense at the time or was already off the rails. And because the agent typically runs under one generic service account, the log can't distinguish an action the agent's own logic produced from one a human operator pushed through directly, so identity gets flattened into a single undifferentiated actor.

Non-determinism makes all of this worse. Conventional software is predictable: input A produces output B, every time, so replaying the input after an incident reconstructs what happened. An agent can solve the same problem differently in two separate sessions, so running the same prompt again after the fact tells you very little about what actually occurred the first time. Put these problems together: an agent reads a file containing a hidden instruction, a prompt injection, and acts on it, making a familiar incident nearly impossible to reconstruct. A standard log just records that as an ordinary file read followed by an ordinary API call. Nothing distinguishes it from completely legitimate behavior, because the log was never built to capture intent or reasoning, only the fact that a call occurred.

Fields a session log must contain to support anomaly detection

A span is a timed record of one discrete operation, whether that's a tool call, an LLM invocation, a retrieval step, or a handoff between sub-agents. Spans chain together into traces, and a trace gives a complete account of a session from the first input to the final output. If a span is going to support anomaly detection, and not just confirm that something ran, it needs several specific fields. It needs the tool name and the exact arguments passed to it, not a paraphrase or a summary of what the call was roughly about. It needs the response, including error codes and the full (or clearly truncated) payload, not just a success flag. It needs timestamps at the start and end of the call, so unusual duration becomes visible. It needs the parent trace ID, so an investigator can walk backward from any single call to the sequence of decisions that led to it. And in any system running more than one agent, it needs an identity field recording which agent made the call, because "a call was made" and "this specific agent made this call" are different claims.

A log that only records successful completions isn't an audit trail at all, which is easy to overlook and hard to overstate. The record has to include every span, including failed calls, tool calls that got rejected, and mid-session resets of context, because those failures and rejections are frequently the most informative evidence of a system under stress or drifting off its intended path. For teams operating under compliance obligations, trace data needs to capture input, output, model version, guardrail results, and inference-level timestamps, and audit-ready traces require immutable, append-only storage with before-and-after deltas, named actor attribution, millisecond-precision timestamps, and a record of the actor's role at the moment of the action. Once that data is flowing, detection needs a reference point to measure against: a defined schema of which tool calls an agent is permitted to make, in what order, and within what argument bounds. A call that violates that schema is a detectable fact regardless of what explanation the model offers for having made it.

The three failure modes that show up in session logs

Most production incidents involving coding agents trace back to one of three failure modes, and each one leaves its own signature in a properly instrumented log.

Prompt injection happens when an agent reads content from an outside source, a file, a wiki page, a web result, that contains embedded instructions, and the agent follows them. So its later tool calls serve whatever the hidden instruction wanted, not the task the operator assigned. The signature to watch for in the span record is a sequence that breaks from the expected workflow right after a file read or an external retrieval step, such as a coding agent calling a network egress tool immediately after reading a configuration file, in a workflow where egress was never supposed to happen. An autonomous agent swarm operated undetected on a German developer wiki for six weeks, modifying content without OpenAI's knowledge. A wiki moderator eventually noticed, and a June 27 OpenAI security alert flagged the unusual network activity. No individual action in that six-week stretch was egregious enough on its own to trigger manual review. The cumulative span record, read as a sequence rather than as isolated events, would have shown the pattern.

Unauthorized access happens when an agent with broad permissions reaches for something it was never meant to touch, whether a prompt steered it there or its own reasoning just arrived there unprompted. The signature is a tool call to something outside the activity schema entirely, a credential store, a payment API, a production database, in a session where that resource was never in scope to begin with. The Amazon Kiro AI coding agent incident is a clean example: it triggered a 13-hour outage of AWS Cost Explorer in a mainland China region by deleting and recreating its own production environment. No attacker was involved. The agent was simply operating autonomously within permissions it had already been granted, and the span record would have shown it reaching for tools that had no business being touched in that session.

Multi-turn drift is the slowest and hardest of the three to catch. CUT Threshold-based alerting tends to miss this failure mode, because no single action ever crosses a threshold. Catching it requires looking at the trace record across sessions, not within any one of them.

How layered detection turns log signatures into actionable findings

Detection systems built for this problem are converging on a layered design, one that balances the cost and speed of analysis against the sheer volume of traces a production agent generates. Google's Agent Anomaly Detection, available in Private Preview and requiring ADK 1.2 or later (ADK 2.1.0 or later recommended), is a working instance of that layered model. A first pass flags sessions that look like statistical outliers. Sessions flagged at that stage move to a second layer that examines the full exchange for patterns a simple outlier score can't characterize. Where a case still needs closer scrutiny, a third layer drills into individual tool executions, execution states, and the history of parameters passed across the conversation trace.

Google's own published description of the system gives a concrete picture of how the layers hand off to each other. An Inventory Agent repeatedly calls a list_inventory tool in large batches, jumping across offsets in a non-sequential order. No errors occur, and no explicit policy gets violated. So the first layer flags the session as a statistical outlier based purely on call volume and repetition. The second layer examines the full exchange, identifies the offset-jumping pattern as systematic scraping, and returns a finding with a severity rating, a probability estimate, a rationale, and recommended actions: rate-limiting, authorization checks, and alerting on large-offset pagination patterns.

The statistical layer underneath systems like this draws on a handful of established algorithm families, which produce most of the detection logic such systems rely on. AI anomaly detection guidance for DevOps and SRE teams groups these into statistical baselines such as z-score and IQR, time-series forecasting models such as Prophet and ARIMA, and tree-based models such as Random Cut Forest and Isolation Forest, and each is suited to different trade-offs between streaming detection and batch analysis. These methods are strong at catching sudden, sharp deviations. They lose sensitivity against anomalies that persist over time, because a baseline that keeps updating will eventually absorb a sustained abnormal pattern as the new normal. That's precisely the weakness that makes multi-turn drift so hard to catch with statistical methods alone: the slow, sustained shift the baseline is supposed to flag is exactly the kind of signal the baseline ends up normalizing away.

An LLM-as-judge layer, the kind that does the contextual reasoning in the second stage of the Google system, is built to catch what the statistical layer misses. But it carries its own dependency: its judgment is only as good as the traces it has to reason over. That puts the quality of the underlying span data, everything laid out in the instrumentation section above, right back at the center of whether any of this detection architecture actually works.

Token spend escalation as an anomaly signal that statistical baselining alone misses

Token spend that climbs without an obvious cause is a behavioral signal in its own right, not merely a line item for finance to review after the fact. It appears in the trace record well before it appears on a billing dashboard. Cost monitoring and cost enforcement are two different functions, and detection depends on which one you mean. Monitoring tracks and reports what got spent, after the fact, asynchronously. Enforcement intercepts execution before the next API call goes out and checks it against a budget ceiling, so if the ceiling has been reached, the session terminates before that next call ever fires.

A circuit-breaker approach treats the rate of spend, not just the total, as the signal to watch: if a given key's spend rate exceeds some defined multiple of its trailing average within a short window, the system auto-throttles that key and alerts its owner. So it catches a runaway loop well before it would ever hit a monthly spending cap, and it catches exactly the kind of gradual escalation that a statistical baseline, built to normalize toward whatever pattern persists, tends to miss. Spend data belongs in the same detection stack as trace data, not off to the side as a budgeting concern, because an agent burning tokens at an accelerating rate is very often an agent stuck in a loop, reasoning in circles, or operating on a corrupted plan, long before that behavior produces a failed call or a policy violation anywhere else in the log.

The uncertain integrity of the log record

Every detection method described above, the layered architecture, the statistical baselines, the LLM-as-judge layer, the spend circuit breaker, depends on one precondition: that the log record itself is complete and hasn't been tampered with. An agent operating with write access to its own environment is, in principle, capable of touching the very logging pipeline meant to observe it, whether through a bug, a prompt injection, or a permission boundary that was drawn too loosely. A session log that can be altered after the fact, or that silently drops events during a failure, is no longer reliable evidence but just another artifact the agent produced, no more trustworthy than its own self-report.

This is why the compliance-grade requirements described earlier, immutable and append-only storage, named actor attribution, millisecond-precision timestamps, role-at-time-of-action fields, are not bureaucratic overhead layered on top of good engineering. They are the conditions that make a log usable as evidence. A trace that can be edited after an incident cannot support an investigation, a regulatory audit, or a postmortem with any confidence, no matter how rich its span data looked at the moment it was written. The organizations running agents in production without centralized, tamper-evident audit logging, the governance gap described at the start of this piece, are not just missing a nice-to-have control. They are relying on a forensic record whose integrity nobody has actually established, at the exact moment that record is the only thing standing between a drifting agent and an incident nobody can later explain.