Tasks

A task is one piece of work that a user set out to do, such as drafting an email, fixing a failing test, or exporting a report. Otis divides each session into tasks. For each task, it records what the user was trying to do and how the attempt turned out.

Otis also groups tasks that share a goal into intent clusters. This page explains where the boundaries of a task come from and what its outcome means, so that you can read task counts and outcomes correctly.

Purpose of tasks

A session covers one whole visit, and a visit usually includes several goals. A user might ask for a summary, rewrite a paragraph, and then export the result. One of those goals can go well while another goes badly, and a single outcome for the whole session hides the difference.

A single message or a single action is too small a unit, because on its own it rarely shows whether the user got what they wanted. A task is the smallest unit that has both a goal and a result, so it is the unit Otis scores.

Task detection

Otis uses three methods, depending on what a session contains. Each method produces its own kind of task.

Conversations with your AI

When a session contains messages between a user and your AI, Otis reads the user's messages in order. At each message, it decides whether the user is continuing the same goal or starting a new one. A new goal starts a new task.

Otis starts a new task only when the change of goal is clear. A message that could be read either way stays in the current task, because splitting too readily would break one piece of work into fragments.

Three rules affect where the boundaries fall:

• Separate conversations stay separate. If your app sends a chat ID, Otis divides each chat into tasks on its own. Two chats that are open at the same time never merge into one task.
• Only the user's own messages can start a task. Your AI often sends prompts of its own to a model while it works. Otis keeps those calls inside the task they belong to, and they never start a new one.
• A decision model call never starts a task. A decision model answers named questions about an input, such as a yes-or-no question or a score, instead of writing a reply. Otis counts its tokens and cost, and never reads what it classified as a message from the user. Decision models lists the calls that the SDK records this way.

Activity in your product

A session can also contain actions that involve no AI message, such as opening a record, saving a draft, or sharing a link. Your app records these with custom events and traced functions. Calls that your AI makes to its own tools aren't product actions, so they never form an activity task.

Otis groups these actions into bursts of activity. A burst ends when the user pauses for a few minutes, and each burst with more than a couple of actions becomes one task. Otis names the task after the action that occurs most often in it, such as prospect.view. If no single action occurs most often, the task is named mixed_activity.

Some events fire on a timer and don't show that the user did anything, such as a status check that runs every 20 seconds. Otis recognizes these background events, and leaves them out when it measures idle time and when it counts actions.

Tool calls from the user's own agent

You can instrument a Model Context Protocol (MCP) server or a command-line tool that the user's own agent calls. The user's own agent is an AI agent that the user directs, such as a coding agent. Otis sees the calls it makes to your tool. It doesn't see the conversation between the agent and the person directing it.

Each call may carry a short statement of what the agent is trying to do. Otis groups consecutive calls that share the same statement into one task, and a call with no statement stays in the task around it. A long pause between calls also starts a new task. When the agent gives no statement at all, those pauses are the only boundaries.

Sessions that mix conversation and activity

Many sessions contain both messages and product actions. Otis then produces both kinds of task for the same session. Conversation tasks are built from the messages, and activity tasks are built from the actions. The two kinds never overlap. A span is one recorded operation, such as an AI call, a traced function or a custom event. No span belongs to both kinds of task, so a message is never one of an activity task's spans and an action is never one of a conversation task's spans.

An action that a user takes in the middle of a conversation, in the same part of your product, isn't counted among any task's spans. If it fails, the failure counts toward the errors and the outcome of the conversation task that was under way. The same holds for a tool call that your AI makes, which carries no message and so isn't one of the conversation task's spans either.

Separation by surface

A surface is where and how a user, or the user's own agent, performs a task. Your web app, a Slack bot, an MCP server and a command-line tool are each a surface. If your app names its surfaces, Otis finds tasks within each surface separately, so a task stays on one surface. A conversation with a chat ID is the exception. Otis keeps a chat together as one task even if its messages carry different surface names.

Name the surface on all of the telemetry for a flow, or on none of it. If only part of a flow is labeled, Otis treats the labeled and unlabeled parts as different surfaces and splits the activity between them.

Surfaces covers how a surface is set and what else it changes.

Agents in a task

An agent is the entry point of a workflow that your product runs, such as a support assistant or a research workflow. If your app names its agents, each task lists the agents that took part in it. An agent is only a label on the task. It never affects where a task begins or ends, so a task in which a router passed the user's requests to two specialists lists both.

Two details affect the list:

• The message that starts a task counts for the new task. That message also ends the task before it, so its span appears in both tasks. Its agent answered the new request, so Otis lists that agent under the new task only.
• An empty list means not labeled. A task lists no agent when none of its spans named one, for example because the task was recorded before your app named its agents. It doesn't mean that no agent ran.

Task timing

Otis divides a session into tasks once the session has been idle for 30 minutes. The wait gives late telemetry time to arrive, and it keeps Otis from closing a task while the user is still working. If the user returns to the same session later, Otis divides the new activity in the same way.

Otis only divides sessions that are large enough to analyze. It skips a very short session, and one with only a few spans.

The task record

• Intent. A short summary of what the user was trying to do, and a category that names the kind of work. For a conversation task, Otis writes the summary from the messages. For an activity task, the summary lists the actions in order. For a tool task, the summary is the statement the user's own agent gave, or a list of the tools it called if it gave none. Intent covers all three.
• Outcome. One of four values, described in the next section.
• Signals. What the user's messages expressed, such as frustration, confusion or delight, along with any feedback your app sent. Signals lists them.
• Size. The number of user messages, the number of spans, and how long the task lasted.
• Errors. How many spans failed, and the causes. For a conversation task, the count includes the failed tool calls your AI made and the failed actions that the conversation set off.
• Cost. The token usage and cost of the AI calls in the task.
• Where it belongs. The user, session, chat, document and surfaces that the task belongs to, and the agents that took part, if your app names them.

Task outcomes

Every task has one of four outcomes:

Otis looks for evidence in the following order and uses the first kind it finds.

1. An outcome your product reports. This applies to activity tasks. An event that your funnel definition lists as a success stage, such as share, marks the task as a success. A failure stage, such as discard, marks it as struggled. If one task contains both, the failure counts. Funnels and artifacts describes how to send these stages.
2. What the user said or rated. In a conversation task, Otis reads each user message for signs of how the work is going. Delight, or a statement that the AI helped, marks the task as a success. So does an action that Otis recognizes as the user putting the result to use, such as saving, exporting or copying it. Frustration, confusion, or a request to reach a human marks it as struggled. Feedback that your app sends, such as a thumbs-up or a correction, counts in the same way. So does the feedback that the user's own agent files about a tool. If a task shows both satisfaction and frustration, Otis records it as a success.
3. How the work ended. If the last operation in the task failed, the task is struggled. If the user cancelled or interrupted it, the task is abandoned. If it returned a successful HTTP status, the task is a success. Otherwise, an earlier failure that the task never recovered from marks the task as struggled. In a conversation task, a failed tool call or a failed action that the conversation set off counts as such a failure, even when your AI's reply succeeded.
4. A very short conversation. A conversation task that ends almost at once and shows no other evidence is abandoned.

If none of these applies, the outcome is unknown.

Reading outcomes

• Unknown means no evidence. Otis doesn't guess an outcome. A task with no feedback, no outcome event and no error stays unknown, and some share of unknown tasks is normal. Sending feedback and artifact stages gives Otis more evidence and reduces that share.
• A struggled task may still have been finished. Struggled records that Otis saw a problem and saw no sign of success afterward. The user may have completed the work without saying so.
• Compare tasks of the same kind. A conversation task usually gets its outcome from what the user wrote. An activity task usually gets it from an event or an error. A success rate that combines the two kinds mostly reflects how many of each there are, so compare rates within one kind.

Agent runs in a task

An agent run is one execution of an agent, from the input it receives to its final output, with every AI call, tool call and subagent in between. A conversation task usually holds one agent run for each request in which your AI called a tool. A tool task holds the calls that the user's own agent made to your tool. Agent runs covers what Otis records about each one.

Otis gives each agent run to one task: the latest conversation task or tool task on the same surface that started at or before the agent run's first tool call. If there is none, the agent run goes to the task on its surface whose time span contains it. An agent run belongs to no task when no task on its surface qualifies.

Two details affect what a task shows:

• Otis records an agent run only when the agent called a tool. A task in which your AI only replied shows its messages and AI calls as the task's own turns.
• An agent run stays with the task it started in. If its work continues past the start of the next task, the next task shows a reference to it, and its steps stay with the first task.

On the task page in the Otis app, the tool calls of the task's agent runs are listed among its spans. The task's span count covers only its own spans, so the page can list more spans than the count shows. Over the Otis MCP server, getEntity returns a conversation task or a tool task as its agent runs in order, with the user's turns between them. Each agent run comes as its trajectory, which is the agent run's steps in order, numbered S1, S2 and so on.

Tasks in Otis

The data browser has a list of tasks that you can filter by type, category, outcome, surface, agent, signal, error count and duration. Opening a task shows its spans, with the tool calls of its agent runs among them. Each session page also lists that session's tasks, and a number in an insight can link to the tasks behind it.

A coding agent connected to the Otis MCP server lists tasks with listEntities and reads one with getEntity.

Related

• Sessions covers the unit that Otis divides into tasks, and Sessions in the SDK guide covers session and chat IDs.
• Signals and Feedback cover the evidence Otis reads from messages and from explicit reactions.
• Errors covers what Otis records when an operation fails, and which failures count against a task.
• Funnels and artifacts covers the stages your app can send to report how a piece of work ended.