APM Tips

KQL · dependencies

Dependency calls per request, by operation

Average outgoing dependency calls per request for each operation, sampling-adjusted, to find chatty endpoints that make too many database or HTTP calls.

Updated

requestsdependencies dependencies

let lookback = 24h;
let calls = dependencies
    | where timestamp > ago(lookback)
    | summarize dependency_calls = sum(itemCount) by operation_Name;
requests
| where timestamp > ago(lookback)
| summarize requests = sum(itemCount) by operation_Name
| join kind=leftouter calls on operation_Name
| extend dependency_calls = coalesce(dependency_calls, 0)
| extend calls_per_request = round(todouble(dependency_calls) / requests, 1)
| project operation_Name, requests, dependency_calls, calls_per_request
| order by calls_per_request desc

Returns: One row per request operation with its request count, the dependency calls made under that operation name and the average calls per request, chattiest first.

Pitfalls

  • The Azure Monitor OpenTelemetry exporters have a known issue: operation name is missing from dependency telemetry. If your dependencies have an empty operation_Name, this join finds nothing; check first with the query below.
  • The ratio counts every dependency recorded under the operation name, including calls made by background work that inherits the name. Treat a surprising number as a lead, not a verdict.
  • Calls made by a downstream service are recorded by that service, normally under its own request's operation name, so they don't count toward the caller here.
  • Both sides use sum(itemCount), so sampling scales requests and calls alike and the ratio holds.

How it works

The classic performance bug is an endpoint that makes one database call per item in a list: fast in development with three items, slow in production with three hundred. Latency charts show the symptom; this query shows the cause, as a plain number of calls per request.

Line by line

  • let calls = ...; sums dependency calls per operation_Name, sampling-adjusted. The data model says every telemetry item inherits the operation name from the request or page view that created the operation, so a dependency recorded while handling GET /orders carries that name.
  • The main query sums requests per operation_Name the same way.
  • join kind=leftouter calls on operation_Name keeps every request operation, including those that made no calls at all.
  • coalesce(dependency_calls, 0) turns “no match” into zero, so those rows read 0 rather than empty.
  • todouble(...) / requests divides as real numbers; integer division would round 2.6 calls per request down to 2.
  • order by calls_per_request desc puts the chattiest endpoints first.

Check the operation name first

The join only works when dependencies carry an operation name. Microsoft lists a known issue for the OpenTelemetry exporters: the operation name is missing from dependency telemetry. Check before trusting a page of zeros:

let lookback = 24h;
dependencies
| where timestamp > ago(lookback)
| summarize calls = sum(itemCount), without_name = sumif(itemCount, isempty(operation_Name))

If most calls lack a name, group dependencies by operation_Id and join to requests on operation_Id instead, keeping cloud_RoleName in both keys so one service’s calls aren’t attributed to another.

Reading the result

Ten or more calls per request on a busy route is worth a look; hundreds is almost always a loop. Take the route into SQL dependency duration by name or outbound HTTP calls by target to see which dependency it hammers. The agent-based dependency tracking guide explains which calls are recorded at all, because an uninstrumented client makes a route look quieter than it is.

Variations

  • Workspace tables: use AppRequests and AppDependencies, TimeGenerated, OperationName and ItemCount.

Sources, checked

  1. join operator (Kusto)learn.microsoft.com · checked
  2. coalesce() (Kusto)learn.microsoft.com · checked
  3. Application Insights telemetry data model (operation_Name)learn.microsoft.com · checked
  4. Troubleshoot OpenTelemetry issues in .NET (known issues)learn.microsoft.com · checked
  5. Migrate an Application Insights classic resource to a workspace-based resource (table schemas)learn.microsoft.com · checked