APM Tips

KQL · operations

End-to-end trace for one operation_Id

Every request, dependency call, log line, exception, page view and custom event recorded for one operation, in time order, across services.

Updated

requestsdependenciestracesexceptionspageViewscustomEvents operations

let lookback = 7d;
let operationId = "4bf92f3577b34da6a3ce929d0e0e4736";
union requests, dependencies, traces, exceptions, pageViews, customEvents
| where timestamp > ago(lookback)
| where operation_Id == operationId
| extend detail = coalesce(name, message, outerMessage)
| project
    timestamp,
    itemType,
    cloud_RoleName,
    detail,
    duration,
    success,
    resultCode,
    severityLevel,
    id,
    operation_ParentId
| order by timestamp asc

Returns: Every telemetry item recorded with the given operation_Id in the last 7 days, oldest first, with its type, service, name or message, duration, outcome, severity, and the IDs that link parent and child.

Pitfalls

  • Replace the example operation ID with a real one, from slowest requests, a failed request, or the portal's transaction view. The match is exact.
  • With sampling on, an operation is kept or dropped as a whole in the classic SDKs, but services that sample independently can each keep a different part of a distributed operation.
  • Each resource stores its own telemetry. If the services in the trace send to different Application Insights resources, this query only sees this resource's part; add the others with a cross-resource query.
  • Clocks differ between machines, so items from different services can appear a few milliseconds out of order. Use operation_ParentId and id for the true parent-child order.

How it works

The portal’s end-to-end transaction view draws one operation as a timeline. This query gives you the same items as a table you can filter, copy into an incident note, or extend with columns the view doesn’t show.

Line by line

  • let operationId = "..."; is the operation to trace. The value in the query is an example; paste a real operation_Id from the slowest requests, a failed request or the portal.
  • let lookback = 7d; bounds the search. Filtering on time first keeps the query fast; widen it if the operation is older.
  • union requests, dependencies, traces, exceptions, pageViews, customEvents reads every table an operation’s items can land in. The data model says every item carries operation_Id, set by the request or page view that started the operation, and Microsoft’s correlation example filters a union of tables on it in exactly this way.
  • detail = coalesce(name, message, outerMessage) gives every row one readable description: the name of a request, dependency, page view or event, the message of a trace, or the outer message of an exception. coalesce() skips empty strings.
  • project keeps the columns that tell the story. id and operation_ParentId are the links: a dependency’s id becomes the operation_ParentId of the request it caused in the next service.
  • order by timestamp asc lays the operation out in time order.

Reading the result

Read top to bottom. The first row is usually the incoming request or the page view that started everything. Gaps between a dependency’s end and the next item show time spent in your own code; a dependency that takes most of the request’s duration is the bottleneck. cloud_RoleName changing between rows shows the call crossing into another service, the same hop Application Map draws as an edge. The correlation and Application Map guide explains how trace context carries the operation ID across services and what breaks it.

If you expected another service’s items and they’re missing, it probably sends to a different resource. The cross-resource queries guide shows how to union them in.

Variations

  • Only the problems: add | where tostring(success) =~ "false" or itemType == "exception" or severityLevel >= 3 before project.
  • Workspace tables: union AppRequests, AppDependencies, AppTraces, AppExceptions, AppPageViews and AppEvents and filter on OperationId; the parent link is ParentId and the row type is Type.

Sources, checked

  1. Application Insights Classic API 2.x (telemetry correlation and the operation_Id example)learn.microsoft.com · checked
  2. Application Insights telemetry data model (operation_Id and operation_ParentId)learn.microsoft.com · checked
  3. union operator (Kusto)learn.microsoft.com · checked
  4. coalesce() (Kusto)learn.microsoft.com · checked
  5. Query across resources with Azure Monitorlearn.microsoft.com · checked