Glossary
Percentile (p50, p95, p99)
The value a given share of measurements fall at or below; a p95 latency of 800 ms means 95% of requests took 800 ms or less, and one in twenty took longer.
Averages hide the slow requests users notice. Ten requests at 100 ms and one at 5 seconds average about 545 ms, a duration no request actually had. Percentiles describe the spread instead: p50 is the median, p95 the point only one request in twenty exceeds, p99 one in a hundred. That is why latency dashboards and alerts are usually set on p95 or p99 rather than on the mean.
In KQL, percentiles() returns several in one pass. On sampled Application Insights data, use the weighted form, percentilesw(), with itemCount as the weight, so each kept row counts for the requests it stands for:
requests
| where timestamp > ago(1h)
| summarize requests = sum(itemCount), percentilesw(duration, itemCount, 50, 95, 99) by operation_Name
| order by percentile_duration_95 descKusto’s percentiles are estimates. The function documentation says they use the T-Digest algorithm, that accuracy depends on how dense the data is around the percentile, and that it is best at both ends of the range. Treat a p99 over a few hundred rows as a rough figure.
Percentiles don’t add up. The average of each instance’s p95 isn’t the service’s p95, and a daily p99 isn’t the mean of the hourly ones; work them out over the combined rows instead.