Agent-based dependency tracking: what the Application Insights Agent sees and misses

Agent-based monitoring has always attracted the same complaint: requests show up, but the dependency calls behind them are thin, missing their SQL text, or missing entirely. Most of those gaps are by design, and knowing which collection technique produced a given row explains nearly all of them.

How dependencies get collected

Microsoft describes three techniques, and each one sees a different amount of detail:

• Bytecode instrumentation around selected methods, applied by the instrumentation engine, which Status Monitor, the Application Insights Agent and the App Service extension can switch on.
• EventSource callbacks from .NET libraries that emit structured events.
• DiagnosticSource callbacks from libraries that support distributed tracing, used by newer .NET SDKs.

Every dependency row records which one produced it, in the prefix of sdkVersion. Microsoft’s guidance for SQL is that rddp means the instrumentation engine collected the call, while rdddsd or rddf mean DiagnosticSource or EventSource did, in which case the full SQL text isn’t captured.

What the agent collects

The Application Insights Agent is the PowerShell module Az.ApplicationMonitor. It instruments ASP.NET and ASP.NET Core apps hosted in IIS, on-premises or on Azure VMs and scale sets (where it ships as a VM extension), and Microsoft states that it autocollects the same dependency signals as the SDK. Concretely, that’s the classic DependencyTrackingTelemetryModule list:

Java and Node.js apps on the same server need their own instrumentation; the agent covers .NET in IIS only. Microsoft’s current auto-instrumentation matrix no longer lists on-premises IIS at all, and the agent’s documentation now lives in the archived Classic API 2.x pages; its latest release on the PowerShell Gallery is 2.0.0, published in February 2023.

Classic SDK 2.x: getting SQL text

The server and database name are always recorded as the dependency’s name. The command text goes in the data field, and getting it there takes two things.

First, a way to see the text. ASP.NET apps on System.Data.SqlClient need the instrumentation engine; apps that use the Microsoft.Data.SqlClient package don’t. The agent installs the engine when you ask:

Enable the agent with the instrumentation engine on an IIS server (run as administrator, then restart IIS):

Enable-ApplicationInsightsMonitoring -ConnectionString 'InstrumentationKey=...;IngestionEndpoint=https://...' -EnableInstrumentationEngine -AcceptLicense
# Or, if monitoring is already on:
Enable-InstrumentationEngine -AcceptLicense
iisreset

Microsoft warns that the engine adds overhead and is off by default; besides SQL text, it records dependency result codes and HTTP verbs.

Second, an explicit opt-in. On .NET Framework it goes in ApplicationInsights.config, which the agent adds to each app it instruments:

<TelemetryModules>
  <Add Type="Microsoft.ApplicationInsights.DependencyCollector.DependencyTrackingTelemetryModule, Microsoft.AI.DependencyCollector">
    <EnableSqlCommandTextInstrumentation>true</EnableSqlCommandTextInstrumentation>
  </Add>
</TelemetryModules>

ASP.NET Core apps set the same property in code with services.ConfigureTelemetryModule<DependencyTrackingTelemetryModule>((m, o) => m.EnableSqlCommandTextInstrumentation = true). Command text can contain personal data in literals, so prefer parameterized queries before you switch it on.

What it misses, and how to fill the gaps

Clients it doesn’t know. Redis, custom TCP protocols, a vendor SDK that emits nothing: none of these become dependencies on their own. In 2.x, time the call and send it with TrackDependency, or wrap it in StartOperation<DependencyTelemetry> so it nests under the current request.

Timing a Redis call as a dependency with the 2.x SDK:

using (var op = telemetryClient.StartOperation<DependencyTelemetry>("GET cart:{id}"))
{
    op.Telemetry.Type = "Redis";
    op.Telemetry.Target = "cache-prod:6380";
    var value = await db.StringGetAsync(key);
    op.Telemetry.Success = true;
}

Time spent opening connections. For SQL, what’s measured is the command. The OpenTelemetry SqlClient instrumentation is explicit about it: the span starts when the command executes and ends when the result is known, so neither connection.Open() nor reading rows from a data reader is included. Microsoft’s classic docs don’t describe the agent’s boundaries in that much detail. If you suspect connection-pool exhaustion or slow logins, time the open yourself.

Processes outside IIS. Windows services, console jobs and scheduled tasks on the same server aren’t touched by the agent. They need the SDK or the Distro; see desktop and console apps.

Context. The agent can’t know which customer or tenant a call served. Custom properties need code: an initializer in 2.x or a processor with OpenTelemetry, as in telemetry initializers and processors.

Today: SDK 3.x and the Microsoft OpenTelemetry Distro

SDK 3.x has no DependencyCollector package; Microsoft lists it among the 2.x packages to remove, and dependency collection comes from OpenTelemetry instrumentation instead. The Microsoft OpenTelemetry Distro for .NET includes HttpClient, SqlClient and Azure SDK instrumentation, so those dependencies arrive without extra packages. The SqlClient instrumentation supports both Microsoft.Data.SqlClient and System.Data.SqlClient (though not Microsoft.Data.SqlClient 3.x), and its README lists db.query.text among the attributes it records. Microsoft’s Distro docs don’t spell out how that attribute maps to the data column, so check a few SQL rows after deploying.

Anything the bundled libraries don’t cover becomes a dependency when you wrap it in an Activity of kind Client, from an ActivitySource registered with AddSource(). Client, Producer and Internal activities land in dependencies.

Timing a SQL connection open as its own dependency with OpenTelemetry:

private static readonly ActivitySource Db = new("Contoso.Data");

using (var activity = Db.StartActivity("SQL open orders-db", ActivityKind.Client))
{
    activity?.SetTag("server.address", "orders-db.database.windows.net");
    await connection.OpenAsync();
}

Register the source once, for example with builder.Services.ConfigureOpenTelemetryTracerProvider((sp, t) => t.AddSource("Contoso.Data")). On IIS, the code-based path for ASP.NET on .NET Framework is SDK 3.x’s Microsoft.ApplicationInsights.Web, which requires .NET Framework 4.6.2 or later; ASP.NET Core apps can use the Distro or SDK 3.x. When you move an app to code, exclude it from the agent with an InstrumentationKeyMap rule that gives its site no key, so only one collector runs. The agent also backs off by itself when it finds the Application Insights SDK loaded, as the auto-instrumentation guide explains.

See what you’re collecting

Which technique produced each dependency type, and how many calls carry a data value (for SQL, the command text):

dependencies
| where timestamp > ago(1d)
| summarize calls = sum(itemCount),
            withData = sumif(itemCount, isnotempty(data))
  by type, collector = extract(@"^[a-z]+", 0, sdkVersion)
| order by calls desc

Then look for the slow ones with the slowest dependencies by target query.

Troubleshooting