Agent runs

An agent run is one execution of an agent, from the input it receives to its final output, with every AI call, tool call and subagent in between. The agent can be one of your product's own agents, or the user's own agent when it calls your MCP server or command-line tool. An AI call is one request that the agent makes to an AI model. A subagent is an agent that another agent starts to do part of its work.

Otis turns the spans of each agent run into its trajectory, which is the agent run's steps in order. This page explains how Otis finds agent runs, what a trajectory shows, and what each agent run result means, so that you can follow what an agent did when you debug it.

Purpose of agent runs

A task shows whether the user got what they wanted. When a task struggled, the reason often sits inside one agent run: a tool call that failed and was retried with the same input, a model that stopped at its output limit, or a subagent that returned nothing useful. A trajectory puts those calls in order, so you can see which AI call asked for each tool call, what came back, and where the work went wrong.

Agent run detection

Otis finds the agent runs in each session from its spans. An agent run usually has a span of its own that holds the rest, such as the span of a Vercel AI SDK generateText call, a ToolLoopAgent, a Claude Agent SDK query(), or an agent span from an OpenTelemetry GenAI instrumentation. Where your own code runs the agent's loop, the agent run is the nearest span above its tool calls, such as a traced function.

Calls to your MCP server or command-line tool have no such span, because the agent that makes them runs outside your product. Otis groups one agent's calls in the same chat and on the same surface into an agent run, and a long pause between calls starts a new one.

Otis records an agent run only when the agent called at least one tool. An AI call that answered without calling a tool appears among its task's own turns.

The agent run summary

The agent run summary is the record Otis keeps for each agent run. It holds counts and outcomes, and no text:

• Agent. The agent your app named, if it named one. A subagent belongs to the agent run that started it and has no agent name of its own.
• Kind. What the agent run was built from, listed below.
• Surface. The surface that the agent run took place on.
• Tool calls. How many tool calls the agent run made, how many failed, how many repeated an earlier call with the same input, and how many were repairs. A repair is a call that retries a failed call to the same tool with different input.
• Usage and cost. The tokens of every AI call in the agent run, its subagents' included, and their cost.
• Duration. How long the agent run took.
• Result. How the agent run ended, described under Agent run results.

Trajectories

A trajectory is an agent run's steps in order. A step is one entry of a trajectory, and each step is one of these kinds:

• User. A user step holds the input that started the agent run, or a new message from the user inside it.
• Model. A model step is one AI call, with its model, duration and tokens, the tool calls it asked for, and the text it wrote.
• Tool. A tool step is one tool call, with its input, its output, its duration and its status.
• Subagent. A subagent step is an agent that this agent started, and it holds steps of its own.
• Repeat. A repeat step holds calls to one tool with the same input, made one after another.
• Compaction. A compaction step marks where the agent's context was summarized to make room, when your framework records it.
• Other. A step of this kind is an embedding or retrieval call, or a failed span of another kind.
• End. The end step says how the agent run ended, with its last tool call and its reply.

Otis numbers the steps in the order they started: S1, S2 and so on. A subagent's steps are numbered under its own step, as S9.1 and S9.2. That number is the step reference, which you use to open a step or to read its full text.

A model step's text plays one of two roles. The reply is the text of the agent run's last AI call. Narration is the text of an earlier AI call, written beside the tool calls it asked for.

Spans that do no work of their own, such as middleware and HTTP spans, are left out of the steps unless they failed. When an AI call re-sends the earlier conversation, the trajectory takes only what is new in it, so a question appears once.

Trajectories of a task

A conversation task or a tool task reads as its agent runs in order, with the user's turns between them. Otis labels each agent run, and each stretch of the task's own turns between them, as A1, A2 and so on. A step inside one of them is written with that label in front, as A2:S3. An activity task holds product actions and no agent run, so it reads as its list of spans. Agent runs in a task covers which agent runs a task holds.

The compact agent run

The compact agent run is the text form of a trajectory that Otis writes for an AI model to read. The Otis MCP server returns it for an agent run or a task. It starts with a line of counts (steps, AI calls, tool calls and their errors, tokens, cost and duration) and how the agent run ended. Each step follows on its own line, with its text cut short to keep its start and end. A repeat or a subagent shows as one line until you open it. Text that your users or your AI wrote is marked as data, so a coding agent that reads it doesn't follow instructions inside it.

Agent run results

Otis records each agent run's result as one of four values, each with a more specific reason. It reads the result from what the agent's framework reports, such as the reason its last AI call stopped. Over the Otis MCP server, the result is the Terminal filter and the reason is the TerminalReason filter.

The reasons mean:

• completed. The model gave its final answer.
• handoff. The agent passed control to another agent. Otis reads a handoff from an OpenTelemetry GenAI agent, such as a Google ADK agent that transfers to another agent.
• max_turns. The agent stopped at its limit on rounds of AI calls.
• max_budget. The agent stopped at its spending limit.
• context_length. The conversation outgrew the model's context window.
• max_output_tokens. The model stopped at its limit on output tokens.
• refusal. The model declined to answer.
• content_filter. A content filter stopped the model.
• tool_error. The agent run failed because its last tool calls all failed, with no AI call after them to read the result.
• error. The agent run failed for another reason.
• user_abort. The user stopped the agent.
• no_result. A tool call never returned.
• unknown. Nothing in the agent run's spans says how it ended.

Reading agent runs

• A failed tool call doesn't make a failed agent run. An agent usually reads a tool's error and tries again. The result says how the agent run finished, and the tool steps say what failed on the way.
• Calls to your MCP server or command-line tool end as unknown. Your server sees the agent's calls and not its model, so nothing records how the agent finished.
• A call that never returned is seen only on the Claude Agent SDK. On other frameworks a tool call with no result leaves no span, so no_result doesn't appear there.
• Compare agent runs of one kind. An agent run built from calls to your MCP server holds only those calls, while one from your product's own agent holds its AI calls too.
• Repeats and repairs are counts. Otis counts them and doesn't judge whether either one was a mistake.
• An unpriced model hides the cost. An agent run with any call to a model that Otis has no price for shows no cost, as a task does. An agent run of calls to your MCP server has no AI calls, so it shows no cost.

Agent runs in Otis

• In the Otis app, a task's page lists the tool calls of its agent runs among its spans.
• Over the Otis MCP server, a coding agent lists agent runs with listEntities, filtered by agent, kind, surface, result, session, or their counts of failed tool calls, repeats and repairs. getEntity reads one agent run, or a task, as a compact agent run, and getSpanText reads the full text of one step. exportData writes agent runs to an ATIF or chat JSONL file for eval and training tools (Exporting rows to a file).
• In chat, you can ask Otis to compare agent runs across your project, such as how often each agent stops at its limit on rounds of AI calls.

Otis keeps each agent run summary for 90 days, as long as the spans it describes.

Related

• Tasks covers which task each agent run belongs to.
• Surfaces covers how a surface and an agent differ.
• MCP server covers listEntities, getEntity and getSpanText in full.
• Agent in the SDK guide covers how your app names an agent.
• OpenTelemetry covers spans from GenAI instrumentations.