Skip to main content

Tracing

A trace is a structured record of one full RunAsync invocation: every agent that fired, every LLM call, every tool, with timings, token counts, retry counts, and cost. AgentRunTrace collects it for you.

The fluent way

var ctx = new AgentContext().WithTracing(out var trace);
await admin.RunAsync(input);

Console.WriteLine($"Outcome : {trace.Outcome}");
Console.WriteLine($"Duration : {trace.Duration.TotalMilliseconds:0}ms");
Console.WriteLine($"Agent calls : {trace.TotalAgentCalls}");
Console.WriteLine($"LLM calls : {trace.TotalLlmCalls}");
Console.WriteLine($"Tool calls : {trace.TotalToolCalls}");
Console.WriteLine($"Retries : {trace.TotalRetries}");
Console.WriteLine($"Tokens : {trace.TotalTokenUsage.TotalTokens}");
Console.WriteLine($"Cost : ${trace.TotalCost.TotalCostUsd:0.0000}");

Shape

public sealed class AgentRunTrace
{
public AgentRunTrace(); // populates from first RunStartedEvent
public AgentRunTrace(string runId, string adminName, string task, DateTimeOffset startedAt);

public string RunId { get; }
public string AdminName { get; }
public string Task { get; }
public DateTimeOffset StartedAt { get; }
public DateTimeOffset? CompletedAt { get; }
public TimeSpan Duration { get; } // live while running; fixed after
public string Outcome { get; } // "running" | "completed" | "failed"
public string? ErrorMessage { get; }

public IList<AgentSpan> AgentSpans { get; }
public IList<LlmSpan> LlmSpans { get; }
public IList<BudgetEvent> BudgetEvents { get; }

public int TotalLlmCalls { get; }
public int TotalAgentCalls { get; }
public int TotalToolCalls { get; }
public int TotalRetries { get; }
public TokenUsage TotalTokenUsage { get; }
public CostEstimate TotalCost { get; }

public void AttachTo(IAgentEventBus bus); // for advanced use
}

Spans

Each span carries enough detail to reconstruct what the agent or LLM did.

AgentSpan

FieldWhat
AgentNameName from IAgent.Name.
InputThe string passed to RunAsync.
OutputThe agent's final response (post-validation).
StartedAt, CompletedAt, DurationWall-clock timing.
TokensUsedTokenUsage for this agent's LLM calls.
CostCostEstimate for the agent.
RetryAttemptsList of (attempt, rawResponse, error).
ToolCallsList of ToolCallSpan invocations.

LlmSpan

FieldWhat
CallerNameAgent name that issued the call.
ModelModel identifier (llama3.2, gpt-4o-mini, etc.).
MessageCountNumber of messages sent to the LLM.
ResponseRaw LLM response.
UsageTokenUsage.
StartedAt, DurationTiming.

ToolCallSpan

FieldWhat
ToolNameTool name.
ArgsJsonArguments JSON (as the LLM produced them).
ResultTool's return value.
Succeededtrue unless the tool threw.
DurationWall-clock.

BudgetEvent

FieldWhat
KindWarning or Exceeded.
CurrentCostUsdSpend at the moment of the event.
LimitUsdThe configured MaxBudgetUsd.
PercentUsed0..1.

Inspecting after a run

foreach (var span in trace.AgentSpans)
{
Console.WriteLine(
$"{span.AgentName}: {span.Duration.TotalMilliseconds:0}ms, " +
$"{span.TokensUsed.TotalTokens} tok, " +
$"${span.Cost.TotalCostUsd:0.0000}");

foreach (var tool in span.ToolCalls)
Console.WriteLine(
$" ↪ {tool.ToolName}({tool.ArgsJson}) -> {tool.Result}");

foreach (var retry in span.RetryAttempts)
Console.WriteLine(
$" ⤴ retry {retry.AttemptNumber}: {retry.Error}");
}

foreach (var llm in trace.LlmSpans)
Console.WriteLine(
$"LLM {llm.Model}: {llm.Usage.TotalTokens} tok, " +
$"{llm.Duration.TotalMilliseconds:0}ms");

Run-correlation patterns

Adopt the next run

var trace = new AgentRunTrace();
trace.AttachTo(bus); // populates from the next RunStartedEvent

await admin.RunAsync(input); // trace is fully populated here

This is what .WithTracing(out var trace) does under the hood.

Pre-set the run identity

When you already have an external request ID and want the trace populated up-front (e.g. you record the trace before the run starts):

var trace = new AgentRunTrace(
runId: HttpContext.TraceIdentifier,
adminName: "Editorial",
task: "summarise",
startedAt: DateTimeOffset.UtcNow);

trace.AttachTo(bus);

The trace will only attach to events whose RunId matches.

Shipping traces to your APM

A trace is plain data. Serialise after the run and forward however your infrastructure expects. Example — JSON to OpenTelemetry-style storage:

await admin.RunAsync(input);

var json = JsonSerializer.Serialize(trace, new JsonSerializerOptions
{
WriteIndented = false,
});
await myApm.SendTraceAsync(json);

For per-span shipping, iterate AgentSpans and LlmSpans and emit each as your APM's span format. The RunId field correlates them.

See also

  • Cost & budget — derive spend from trace.TotalCost, set MaxBudgetUsd to fail fast.
  • Events — what AgentRunTrace is subscribing to under the hood.