APM Tips

KQL · exceptions

New exception types this week

Exception types recorded in the last 7 days that never appeared in the 28 days before, with count, first sighting and an example message.

Updated

exceptions exceptions 

let lookback = 7d;
let baseline = 28d;
let knownTypes = exceptions
    | where timestamp between (ago(lookback + baseline) .. ago(lookback))
    | distinct type;
exceptions
| where timestamp > ago(lookback)
| where type !in (knownTypes)
| summarize
    exceptions = sum(itemCount),
    first_seen = min(timestamp),
    operations = dcount(operation_Name),
    example_operation = take_any(operation_Name),
    example_message = take_any(outerMessage)
    by type
| order by first_seen asc

Returns: One row per exception type seen in the last 7 days but not in the 28 days before, with its count, the time it first appeared, how many operations threw it, and an example.

Pitfalls

  • Under heavy sampling a rare exception type can be missing from the baseline by chance and look new. Check its first_seen against your deployments before raising an alarm.
  • The baseline needs 35 days of data in total, within the default retention but not within a resource that is two weeks old.
  • New throw sites of an old type don't show here. Swap type for problemId to catch them; the result will be longer.
  • The in and !in operators read up to 1,000,000 distinct values from a tabular expression; a list of exception types is nowhere near that.

How it works

A release rarely adds a new bug with a familiar face. More often it introduces an exception your app has never thrown before: a NullReferenceException from a new code path, a serializer error from a changed contract. This query lists exactly those, so the weekly review starts with what is new rather than with the same noisy top ten.

Line by line

  • let lookback = 7d; is “this week” and let baseline = 28d; is how far back “never seen before” reaches. Together they read 35 days.
  • let knownTypes = ...; collects every distinct exception type seen between 35 and 7 days ago. between includes both ends.
  • where type !in (knownTypes) keeps this week’s exceptions whose type isn’t in that list. !in accepts a tabular expression and reads its first column.
  • sum(itemCount) counts them the sampling-aware way.
  • first_seen = min(timestamp) is when the type first appeared this week. Line it up with your deployment times.
  • operations counts how many operations threw it; example_operation and example_message give you somewhere to start.
  • order by first_seen asc lists types in the order they appeared, which is often the order of the releases that introduced them.

Reading the result

An empty result after a release is good news. A new type with many exceptions across several operations deserves attention today. A new type with a handful of exceptions in one operation is still worth a ticket: rare errors on new code paths tend to grow. Take its problem ID into exception details for a problem ID for messages and stack frames.

If your telemetry carries the build version, add application_Version to the summarize to see which version brought the new type in; the version and build information guide shows how to set it.

Variations

  • New throw sites: replace type with problemId in knownTypes, the filter and the by clause.
  • Per service: group by type, cloud_RoleName and build knownTypes from the same pair with distinct type, cloud_RoleName, then filter with a join kind=leftanti on both columns.
  • Workspace tables: use AppExceptions, TimeGenerated, ExceptionType, OuterMessage, OperationName and ItemCount.

Sources, checked

  1. The between operator (Kusto)learn.microsoft.com · checked
  2. The case-sensitive in string operator (Kusto)learn.microsoft.com · checked
  3. distinct operator (Kusto)learn.microsoft.com · checked
  4. Application Insights telemetry data model (exception type and problemId)learn.microsoft.com · checked
  5. Telemetry sampling in Application Insightslearn.microsoft.com · checked

  • Exceptions by type and operation

    exceptions exceptions

    Returns: Up to 50 rows, one per exception type and operation, with the sampling-adjusted count, the number of distinct problem IDs, one outer message and one innermost exception type.

  • Exception details for one problem ID

    exceptions exceptions

    Returns: One row for the problem ID with its sampling-adjusted count, first and last time seen, up to ten operations, roles and versions, the number of distinct users and up to five distinct messages.

  • Top failing operations, this week vs last week

    requests failures

    Returns: Up to 20 operations with the most failed requests in the last 7 days, each with last week's failures, the change in count and percent, and this week's total requests.