APM tips blog

Blog about application monitoring.

Client IP Address

| Comments

Client IP address is useful for some telemetry scenarios. You may discover very high latency from remote countries or the reason for a requests count spike in the night when countries across the ocean woke up.

Application Insights collects client IP address. Country, state and city information will be extracted from it and than the last octet of IP address will be set to 0 to make it non-identifiable. So Application Insights will never store an actual IP address by default.

There are two ways IP address got collected for the different scenarios.

Browser and devices

When telemetry is sent from browser by JavaScript SDK or from device - Application Insights endpoint will collect sender’s IP address. Endpoint doesn’t resolve as IPv6 so this IP address will always be IPv4.

Server applications

Client IP address for the server application will be collected by SDK. In .NET it is done by ClientIpHeaderTelemetryInitializer. This telemetry initializer will check X-Forwarded-For http header and if it is not set - use client IP.

Caveat here is that Application Insights only supports IPv4 at the moment of this writing. So if the clients of your application are using IPv6 – IP address will not be send to Application Insights.

Now when Application Insights receives an event without IP address set - it will assume that this event came from the device and will store the server’s IP address. This is why you may find some fake Brazilian clients when your application was deployed in Azure.

You may also end up getting the firewall/load balancer IP address for all your clients if this firewall sets an original IP address into a different http header. Popular one is X-Originating-IP.

It is easy to override the default logic of ClientIpHeaderTelemetryInitializer using configuration file. You can set a list of header names to check, separators to split IP addresses and whether to use first or last IP address. Here is how to override default settings:

1
2
3
4
5
6
7
<Add Type="Microsoft.ApplicationInsights.Web.ClientIpHeaderTelemetryInitializer, Microsoft.AI.Web">
  <HeaderNames>
    <Add>X-Originating-IP</Add>
  </HeaderNames>
  <HeaderValueSeparators>;</HeaderValueSeparators>
  <UseFirstIp>false</UseFirstIp>
</Add>

Now, when your application will receive the header X-Originating-IP: 8.8.8.1;8.8.8.2 telemetry will be sent with the following context property: "ai.location.ip":"8.8.8.2".

If you want to keep the full IP address with your telemetry and storing client’s PII information is not a concern - you can implement a telemetry initializer:

1
2
3
4
5
6
7
8
9
10
public class CopyIPTelemetryInitializer : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
        if (!string.IsNullOrEmpty(telemetry.Context.Location.Ip))
        {
            telemetry.Context.Properties["client-ip"] = telemetry.Context.Location.Ip;
        }
    }
}

This telemetry initializer will store IP address in the custom property and it’s last octet will not be set to zero. Make sure to add it after ClientIpHeaderTelemetryInitializer.

Another tip - C# SDK do not allow to sent IPv6 addresses to Application Insights. Using custom properties is a good alternative for sending it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
public class IPv6AddressTelemetryInitializer : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
        if (HttpContext.Current != null)
        {
            IPAddress ip;
            if (IPAddress.TryParse(HttpContext.Current.Request.UserHostAddress, out ip))
            {
                telemetry.Context.Properties["client-ip"] = ip.ToString();
            }
        }
    }
}

Where is my map?

Once IP addresses collected properly - the next step is to map them. There is no map in Azure portal. But you can easily visualize your telemetry on the map using Power BI integration.

Application Insights for MVC and Web API

| Comments

Application Insights Web SDK today has a very basic support for MVC and Web API applications. In this post I collected some links on how to make monitoring of your applications better. This is not a definitive guide on how to monitor those types of applications correctly. This is just a dump of accumulated knowledge of some problems and their solutions.

Telemetry correlation

This is how NuGet server implemented OWIN middleware to track telemetry. It uses custom CallContext and time tracking.

This blog post suggests much better approach for telemetry correlation using the newest features of Application Insights SDK. Author also suggests the workaround for this issue that looks very promising.

Use Action filters

Action filters can also be used for telemetry correlation. CallContext will not be lost when using action filters. However, timing of request will not be as accurate as when middleware being used.

Exceptions

Exceptions are handled by MVC infrastracture and sometimes will not reach http module. Here is a documentation on how to configure exception handling in different versions of MVC.

There is also an ApplicationInsights.Helpers NuGet package that implements exception handling for MVC.

Exceptions thrown by middlewares should also be collected. Typically, you’d need a separate middleware to catch exceptions. You can find an explanation why there should be two middlewares at Step 3 in this blog post.

Request names

I mentioned before that for attribute-based routing operation names will have identifier in them and will not be groupable. I advice to set Route template as a request name:

1
this.RequestContext.RouteData.Route.RouteTemplate

Configuration file

In order to collect performance counters and dependencies as well as some other good features - you’d want to install Windows Server nuget. This nuget will create and populate ApplicationInsights.config file. So most of monitoring configuration will be defined in that configuration file. However if you’d like to use code-based configuration approach and dependency injection - you’ll need to remove this file after every nuget update.

Singleton and DI

When using dependency injection to inject TelemetryConfiguration and TelemetryClient classes - singleton TelemetryConfiguration.Active will not be initialized. Or even worse - will be initialized with some unexpected values. Thus the code from documentation will not send event:

1
2
var client = new TelemetryClient();
client.TrackEvent("purchase completed");

This is how we solved it in ASP.NET core. We take configuration from Application Insights’s singleton and use it as DI singleton.

Another issue with the dependency injection is that identity provider can only be obtained using DI. It is not that easy to create a TelemetryInitializer that will be instantiated on every request by DI. Application Insights has it’s own mechanism of initializing and do not have a native mechanism to create a list of telemetry initializers per request.

Bugs

There are some bugs I cannot explain today. There are reports that sometimes request name has a mystery [id] in it. For instance, request to this controller will be reported as POST MyController [id]:

1
2
3
public class MyController : ApiController
{
    public async Task<HttpResponseMessage> Post(HttpRequestMessage request)

Looking at how request name generated I cannot explain it.

Have You Been Hacked?

| Comments

I’ve returned from the long parental leave. Checking telemetry for this blog I noticed the new dependency call

Here is how it looks on Application map:

It scared me as there are no AJAXes in the blog. So my first thought - my blog was hacked.

Diving deeper - it is a call to http://api4.adsrun.net/post. Looks even scarier. Quick internet search showed that this is a malware installed on visitor’s computer. This malware injects malicious code into jquery script that browser loads. So apmtips.com is safe, it’s visitor’s computer has a security problem.

Looking into All available telemetry for this user session I found some details of the visitor. So if you are visiting from the city of Sangli in India using Firefox - check your computer for malware.

Validate Event Json

| Comments

Application Insights endpoint now supports events validation. It’s very easy to use - just post your telemetry item json to this endpoint and if response code is not 200 response text will contain the error message.

In C# code will look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
static async Task<bool> IsEventCorrect(string json)
{
    json = json.Replace("duration", "durationMs");

    HttpClient client = new HttpClient();
    var result = await client.PostAsync(
        "https://dc.services.visualstudio.com/v2/validate",
        new ByteArrayContent(Encoding.UTF8.GetBytes(json)));

    Console.WriteLine(result.StatusCode);

    if (result.StatusCode != HttpStatusCode.OK)
    {
        var response = await result.Content.ReadAsStringAsync();
        Console.WriteLine(response);
        return await Task.FromResult(false);
    }
    return await Task.FromResult(true);
}

Result of execution will be something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
BadRequest
{
  "itemsReceived": 1,
  "itemsAccepted": 0,
  "errors": [
    {
      "index": 0,
      "statusCode": 400,
      "message": "106: Field 'duration' on type 'RequestData' is of 
                    incorrect type. Expected: string, Actual: undefined"
    }
  ]
}

Validation that this endpoint provides is not 100% strict. It guarantees that event is well-formed and has all the required fields. So it will be accepted by the /track endpoint. However, today it will allow sending some extra fields in json payload that will never be saved in backend.

With the .NET SDK it’s easy to generate test JSON. You’ll need to construct telemetry type of interest. In this example - RequestTelemetry. Then use JsonSerializer class to get byte array:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
RequestTelemetry rt = new RequestTelemetry();

rt.Context.InstrumentationKey = "c92059c3-9428-43e7-9b85-a96fb7c9488f";

rt.Name = "RequestName";
rt.StartTime = DateTimeOffset.Now;
rt.Duration = TimeSpan.FromSeconds(10);

string json = Encoding.Default.GetString(
    JsonSerializer.Serialize(new List<ITelemetry>() { rt }, false));

json = json.Replace("duration", "durationMs");

var t = IsEventCorrect(json);

You can also fill out all the properties:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
// Host context
rt.Context.Cloud.RoleName = "Role Name";
rt.Context.Cloud.RoleInstance = "Role Instance";

// Application context
rt.Context.Component.Version = "Application Version";

// Custom properties - limit 200 per application
rt.Context.Properties["DeploymentUnit"] = "SouthUS";

// Application user context
rt.Context.Location.Ip = "127.0.0.1";
rt.Context.Operation.SyntheticSource = "Test in production";

// Session context
rt.Context.User.Id = "Anonymous User Id";
rt.Context.Session.Id = "Anonymous Session Id";

rt.Context.User.AccountId = "Account Id";
rt.Context.User.AuthenticatedUserId = "Authenticated user id";

// Operation context
rt.Context.Operation.Id = "Root operatioin id";
rt.Context.Operation.ParentId = "Parent Operation Id";
rt.Context.Operation.Name = "Operation name";

Using C# JSON serialization and validation endpoint it is easier to understand why json formed by any other SDK is not being accepted by Application Insights endpoint.

Telemetry Channels Update

| Comments

This blog post was written by Anastasia Baranchenkova

From the time Sergey wrote his blog post there are a few changes.

Persistence channel was removed. There will be no new versions of Application Insights SDK for devices. The suggested solution is to use HockeyApp that Microsoft acquired last year.

Server channel became open source.

Also it was mentioned in Sergey’s blog post that Server telemetry channel stores events on disk if they cannot be sent. The channel would use either current user’s local app data folder or current process temp folder. I saw several cases when there was a problems with that.

In the first case process did not have access neither to local app data nor to the temp folder. The workaround was to make sure it has it.

Second one was more interesting. There was an application that worked perfectly fine locally but when deployed to a VM it crashed. It appeared that local app data was pointing to an unmapped drive. Unfortunately ApplicationInsights was not catching this type of exception, so it was just crashing without even trying to check temp folder. This bug was fixed in 2.0.0-beta4.

Additionally while fixing this issue we added an ability to specify custom folder either though configuration or in code (if it is not provided or inaccessible old logic is used).

In configuration file you would do it this way (make sure you have correct xml closing tags):

1
2
3
<TelemetryChannel Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel, Microsoft.AI.ServerTelemetryChannel">
    <StorageFolder>D:\NewTestFolder</StorageFolder>
</TelemetryChannel>

In code you would do it this way:

  • Remove ServerTelemetryChannel from configuration file
  • Add this piece of code to the place where application gets initialized:
1
2
3
4
5
ServerTelemetryChannel channel = new ServerTelemetryChannel();
channel.StorageFolder = @"D:\NewTestFolder";
channel.Initialize(TelemetryConfiguration.Active);

TelemetryConfiguration.Active.TelemetryChannel = channel;

An interesting side note: configuration loading is very flexible. If you see a public property on an object that can be defined in the configuration file (e.g. TelemetryInitializer, TelemetryProcessor or Channel) this property can be set via configuration file. We did not have to change configuration loading logic for this update, we just added a public property to the ServerTelemetryChannel.

WebApi Tracing Powered by ApplicationInsights in Visual Studio 2015 Update 1

| Comments

This blog post was written by Anastasia Baranchenkova

In Visual Studio 2015 Update 1 Application Insights telemetry is available right from the Visual Studio. And you can even get it without providing a valid instrumentation key.

I created a sample application that I describe in details later on. In this application I forward WebApi framework traces to Application Insights (it is a similar thing that SergeyK described in one of his previous posts but it is done differently: by implementing ITraceWriter interface).

Full sample is here.

So if you start debugging web hosted WebApi application even if you do not provide instrumentation key you can see all the traces rights in your Diagnostics Tools window. You can search there, filter and see actual json file that would have been sent if you provided a valid instrumentation key:

For self-hosted WebApi applications you would need to configure ApplicationInsights providing a valid instrumentation key because Diagnostics Tools hub does not currently support this type of application. But you still can see all the telemetry from the VS itself. For that you need to open: View ->Other Windows->Application Insights Search. From there you connect to your Azure Subscription and get back all the telemetry. You can select different time intervals, filter by type or property values and see each telemetry item details:

And now I want to describe in details how you can forward WebApi traces to ApplicationInsigts to get all this beauty.

WebApi web-hosted application

  1. Create WebApi web hosted application. If you have Azure subscription check “Add ApplicationInsights to project”.
  2. If you did not add ApplicationInsights on application creation then add the latest Application Insights for Web nuget through nugget package manager. (In this case you do not have instrumentation key in the ApplicationInsights configuration file and no data will be sent to the portal but you can add it that later and for debugging purposes you have all you need)
  3. Add Microsoft.AspNet.WebApi.Tracing nuget package.
  4. Add ApplicationInsightsTraceWriter.cs

ApplicationInsightsTraceWriter implements System.Web.Http.Tracing.ITraceWriter. Method Trace gets trace message parameters and converts it to ApplicationInsights trace or exception.

1
2
3
4
5
6
7
8
9
internal sealed class ApplicationInsightsTraceWriter : ITraceWriter
{
  public void Trace(HttpRequestMessage request, string category, TraceLevel level, Action<TraceRecord> traceAction)
    {
      message = this.systemDiagnosticsTraceWriter.Format(traceRecord);
      this.client.TrackTrace(message, GetSeverityLevel(level));
  }
}
  1. Add CompositTraceWriter.cs

CompositTraceWriter also implements ITraceWriter and is used in case if some other ITraceWriter is already registered by the application so that ApplicationInsightsTraceWriter does not replace existing but rather is added to the list of trace writers.

  1. Add Extensions.cs

This class adds HttpConfiguration.EnableApplicationInsights extension method that registers ApplicationInsightsTraceWriter.

1
2
writer = new ApplicationInsightsTraceWriter();
configuration.Services.Replace(typeof(ITraceWriter), writer);   
  1. In Global.asax add GlobalConfiguration.Configuration.EnableApplicationInsights();

WebApi self-hosted application

For a self-hosted application you will do almost the same but

  1. Instead of Web nuget package add ApplicationInsights API nuget
  2. Create application insights resource manually through the portal and set instrumentation key programmatically
1
TelemetryConfiguration.Active.InstrumentationKey = “MyKey”;
  1. Call config.EnableApplicationInsights() from Startup.Configuration where config would be an instance of HttpConfiguration that you create in this method for other registration purposes.

I would highly encourage to also read this blog post that explains how and when to use WebApi tracing. The example above demonstrates basic integration points while in the article above you can find detailed information on the best practices of how to use WebApi tracing.

Deploy to Azure With StyleCop

| Comments

I was working on enabling continues deployment from GitHub to Azure WebApp for the sample Glimpse and Application Insights integration application. It is easy to implement this integration. Simplest way is to create “Deploy to Azure” button in your GitHub repository like I explained in this blog post. You can also do it manually:

  1. Open Settings blade of your web app
  2. Click on Continuous deployment
  3. Select External Repository and set your repository URL

Once enabled - KuDu will pull sources from repository, compile it and deploy web application.

This time it didn’t work smoothly for me. I got an error:

1
2
3
4
5
6
7
8
9
10
MSBUILD : error : SA0001 : CoreParser : An exception occurred while parsing the 
file: System.IO.IOException, The specified registry key does not exist.. [D:\home
\site\repository\Glimpse.ApplicationInsights\Glimpse.ApplicationInsights.csproj]
Failed exitCode=1, command="D:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe" 
"D:\home\site\repository\Glimpse.ApplicationInsights.Sample\
Glimpse.ApplicationInsights.Sample.csproj" /nologo /verbosity:m /t:Build 
/t:pipelinePreDeployCopyAllFilesToOneFolder /p:_PackageTempDir="D:\local\Temp\
8d2f93a7a699626";AutoParameterizationWebConfigConnectionStrings=false;
Configuration=Release;UseSharedCompilation=false 
/p:SolutionDir="D:\home\site\repository\.\\"

My web project doesn’t have StyleCop enabled so the error was quite misleading. The good thing - this error message had original msbuild command. So I opened KuDu debug console using URL: https://<webAppName>.scm.azurewebsites.net/DebugConsole and typed the following command:

1
2
3
4
5
6
7
8
9
10
11
cd LogFiles

"D:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe"
        "D:\home\site\repository\Glimpse.ApplicationInsights.Sample\Glimpse.ApplicationInsights.Sample.csproj"
        /nologo
        /verbosity:detailed
        /t:Build
        /t:pipelinePreDeployCopyAllFilesToOneFolder
        /p:_PackageTempDir="D:\local\Temp\8d2f93a7a699626";AutoParameterizationWebConfigConnectionStrings=false;Configuration=Release;UseSharedCompilation=false
        /p:SolutionDir="D:\home\site\repository\.\\"
> buildLog.txt

Note, the command text is different from the original. Specifically, I changed verbosity /verbosity:detailed and redirected output to the file > buildLog.txt. Resulting error message was much easier to troubleshoot:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Using "StyleCopTask" task from assembly "D:\home\site\packages\StyleCop.MSBuild.4.7.49.1\build\..\tools\StyleCop.dll".
Task "StyleCopTask"
MSBUILD : error : SA0001 : CoreParser : An exception occurred while parsing the file:
System.IO.IOException, The specified registry key does not exist..
[D:\home\site\repository\Glimpse.ApplicationInsights\Glimpse.ApplicationInsights.csproj]
  1 violations encountered.

Done executing task "StyleCopTask" -- FAILED.
Done building target "StyleCop" in project "Glimpse.ApplicationInsights.csproj" -- FAILED.
Done Building Project "D:\home\site\repository\Glimpse.ApplicationInsights\Glimpse.ApplicationInsights.csproj"
        (default targets) -- FAILED.
Done executing task "MSBuild" -- FAILED.
Done building target "ResolveProjectReferences" in project
        "Glimpse.ApplicationInsights.Sample.csproj" -- FAILED.

Project that my web application referencing has StyleCop enabled. Also it seems that StyleCop fail to run.

At this point I decided that I don’t need StyleCop when publishing to azure. So I’ve added extra condition into target import AND ('$(_PackageTempDir)' == ''). This condition is the best I come up with to distinguish the build for deployment from regular compilation. Here is corresponding commit and full import statement:

1
2
3
4
5
<Import
        Project="..\..\packages\StyleCop.MSBuild.4.7.49.1\build\StyleCop.MSBuild.Targets"
        Condition="Exists('..\..\packages\StyleCop.MSBuild.4.7.49.1\build\StyleCop.MSBuild.Targets') 
                AND ('$(_PackageTempDir)' == '')"
/>

So I unblocked myself, but this solutions seems hacky. I was thinking of more robust solution with setting special parameter for build using application setting SCM_BUILD_ARGS that will disable StyleCop. See KuDu wiki for details. However I wanted to get to the root cause of why StyleCop needs registry access. So I decided to troubleshoot it further.

I know there is an exception happening in StyleCop and I need its callstack. So I decided to use remote debugger to get it. First, I found github project that will run StyleCop from command line. I found one called StyleCopCmd. I downloaded and compiled it with the proper version of StyleCop.dll. I also added an extra Console.Read in the beggining of it so I’ll have time to attach debugger. This is how I’ll run it from KuDu console:

1
.\StyleCopCmd.exe -s D:\home\site\repository\Glimpse.ApplicationInsights.sln -c -w

I followed instructions to attach remote debugger to StyleCopCmd.exe process. There are caveats:

  1. At first I used wrong credentials and was getting error like this:
  ---------------------------
  Microsoft Visual Studio
  ---------------------------
  Unable to connect to the Microsoft Visual Studio Remote Debugging Monitor named 
  'sitename.scm.azurewebsites.net'. The Microsoft Visual Studio Remote Debugging 
  Monitor (MSVSMON.EXE) does not appear to be running on the remote computer. 
  This may be because a firewall is preventing communication to the remote computer. 
  Please see Help for assistance on configuring remote debugging.
  ---------------------------
  OK   Help   
  ---------------------------

The reason was - I used user name $glimpse-play-ai-2 instead of fully-qualified glimpse-play-ai-2\$glimpse-play-ai-2. You may have the same message with the completely wrong credentials.

  1. When I run Attach to the process from Visual Studio I haven’t seen the process StyleCopCmd.exe. The reason is that this process is SCM (KuDu) process and I needed to specify “sitename.scm.azurewebsites.net” as a Qulifier, not “sitename.azurewebsites.net” in Attach to the Process dialog.
  2. When debugging external code in Visual Studio - make sure to uncheck the box “Just My Code” in debugger options.

I set Visual Studio to stop on every CLR exception and let StyleCopCmd.exe run. It paused on excpetion with the following call stack:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
mscorlib.dll!Microsoft.Win32.RegistryKey.Win32Error(int errorCode, string str) Line 1694        C#
mscorlib.dll!Microsoft.Win32.RegistryKey.CreateSubKeyInternal(string subkey, Microsoft.Win32.RegistryKeyPermissionCheck permissionCheck, object registrySecurityObj, Microsoft.Win32.RegistryOptions registryOptions) Line 409        C#
mscorlib.dll!Microsoft.Win32.RegistryKey.CreateSubKey(string subkey, Microsoft.Win32.RegistryKeyPermissionCheck permissionCheck) Line 297        C#
mscorlib.dll!Microsoft.Win32.RegistryKey.CreateSubKey(string subkey) Line 289        C#
StyleCop.dll!StyleCop.RegistryUtils.CurrentUserRoot.get()        Unknown
StyleCop.dll!StyleCop.RegistryUtils.CUGetValue(string name)        Unknown
StyleCop.dll!StyleCop.StyleCopCore.GetLastUpdateCheckDate()        Unknown
StyleCop.dll!StyleCop.StyleCopCore.CheckForStyleCopUpdate(StyleCop.CodeProject project)        Unknown
StyleCop.dll!StyleCop.StyleCopCore.InitializeProjectForAnalysis(StyleCop.CodeProject project, StyleCop.StyleCopThread.Data data, StyleCop.ResultsCache cache)        Unknown
StyleCop.dll!StyleCop.StyleCopCore.Analyze(System.Collections.Generic.IList<StyleCop.CodeProject> projects, bool ignoreCache, string settingsPath)        Unknown
StyleCop.dll!StyleCop.StyleCopCore.FullAnalyze(System.Collections.Generic.IList<StyleCop.CodeProject> projects)        Unknown
StyleCop.dll!StyleCop.StyleCopConsole.Start(System.Collections.Generic.IList<StyleCop.CodeProject> projects, bool fullAnalyze)        Unknown
StyleCopCmd.exe!StyleCopCmd.Core.StyleCopExecutor.Run() Line 70        C#
StyleCopCmd.exe!StyleCopCmd.Program.Main(string[] args) Line 61        C#

Using disassembler I found that StyleCop tries to set registry key when checks for the latest version:

1
 Registry.CurrentUser.CreateSubKey("Software\\CodePlex\\StyleCop")

So I’ve added this flag to StyleCop settings file to disable this version check. Here is the commit:

1
<BooleanProperty Name="AutoCheckForUpdate">False</BooleanProperty>

And it solved the issue.

Azure Web App infrastructure gives you great flexibility and power deploying and running your web applications. Even though it’s infrastructure has some limitations - it is really easy to troubleshoot issues with all the tools it provides.

You Need This nuget.config

| Comments

Suddenly many projects on my computer stoped compiling - they cannot find NuGet dependencies. After short digging I found that my default NuGet.config file %APPDATA%\NuGet\NuGet.Config was changed. Now it sets packages folder to be ..\..\..\Users\sergkanz\AppData\Roaming\packages. To be absolutely precise the packages folder is set to %APPDATA%\NuGet\..\packages, but Visual Studio will expand the path.

I’m not sure when and how I changed this file. Maybe it is a feature of new NuGet version? Anyway it turns out to be a very good thing. I see two benefits here:

  • For the test projects I share packages and never even copy them from cache. I just point my test projects directly to the cache folder
  • Any project that I share with other people now requires it’s own NuGet.config file. And I forced to create it - no way anybody will have the folder ..\..\..\Users\sergkanz\AppData\Roaming\packages.

Having personalized NuGet.config for the shared project is a good idea. First, you can configure the same packages folder for multiple solutions in one repo. Second, you also set the list of extra package sources you are using and some other settings.

Here is a small NuGet.config that I copy now to the test projects that I want to share with others. Just place it to the solution folder and it will replicate the default $(SolutionDir)\packages behavior:

1
2
3
4
5
6
7
8
9
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <config>
    <add key="repositoryPath" value=".\packages" />
  </config>
  <packageSources>
    <add key="nuget.org" value="https://www.nuget.org/api/v2/" />
  </packageSources>
</configuration>

Sometimes you need to restart Visual Studio so solution will pick up this configuration.

Application Insights Self-diagnostics

| Comments

This blog post was written by Anastasia Baranchenkova

Each application insights component writes diagnostic information using EventSource.

If you see no data and you already checked that correct instrumentation key is specified you can try to use PerfView to see if there are problems with sending telemetry out (More about collecting Application Insights traces here.

But it may be so that Application Insights is partially functional: channel is working and sending telemetry out but not all events are delivered. For example, you configured custom counter but you do not see it. If counter is configured incorrectly ETW event will be logged and you will actually be able to find Application Insights trace telemetry it in your Search explorer:

Why did you get this even as trace telemetry?

Application Insights Web nuget package adds Diagnostics module in the configuration file by default. This module subscribes to Application Insights diagnostic error events and sends them out to the same Application Insights resource that is used by your application. Diagnostics messages will be sent as trace telemetry and will have “AI:” or “AI Internal:” prefixes.

You can send Application Insights diagnostic messages to a different resource if you provide different instrumentation key for Diagnostics Module in your application insights configuration file:

1
2
3
<Add Type="Microsoft.ApplicationInsights.Extensibility.Implementation.Tracing.DiagnosticsTelemetryModule, Microsoft.ApplicationInsights" >
  <DiagnosticsInstrumentationKey>YOUR_KEY_2</DiagnosticsInstrumentationKey>
</Add>

UPDATE 1/31/16: Recently I’ve been asked several times about other override property that this module has: Severity. Updating this propety can force the diagnostics module to send all the telemetry to the portal and not only errors. I was not thinking that it is very useful for the end user but apparently it is e.g. when you cannot install Fiddler or PerfView on the production box. With this override you can collect verbose traces just by updaing the applicationinsights.config and restarting the application. Important! You do not want this to be enabled by default all the time! It decreases application performance, produces a lot of noice messages that you do not understand and for which you potentially pay money. If you still want to see all the traces in the portal you would add this to your config:

1
2
3
4
<Add Type="Microsoft.ApplicationInsights.Extensibility.Implementation.Tracing.DiagnosticsTelemetryModule, Microsoft.ApplicationInsights" >
  <DiagnosticsInstrumentationKey>YOUR_KEY_2</DiagnosticsInstrumentationKey>
  <Severity>Verbose</Severity>
</Add>

Performance Counters in Non-web Applications

| Comments

This post shows how to collect Performance Counters for the desktop application and features the answer to the questions:

There is no more “TelemetryModules“ collection in “TelemetryConfiguration“ class. where should I store my telemetry modules?

Step 1

Install NuGet package.

1
Install-Package Microsoft.ApplicationInsights.PerfCounterCollector

Step 2

NuGet will create an ApplicationInsights.config file. If you don’t use it (and you probably don’t use it for desktop applications) - remove this file.

Step 3

Define global variable that will live for a lifetime of an application, instantiate it, populate the list of counters and call Initialize method:

1
2
3
4
5
6
7
8
9
10
11
12
13
class Program
{
    private PerformanceCollectorModule perfCollectorModule;

    static void Main(string[] args)
    {
        TelemetryConfiguration.Active.InstrumentationKey = "Foo";

        this.perfCounterCollectorModule = new PerformanceCollectorModule();
        perfCounterCollectorModule.Counters.Add(
          new PerformanceCounterCollectionRequest(
              @"\.NET CLR Memory(??APP_CLR_PROC??)\# GC Handles", "GC Handles"));
        perfCounterCollectorModule.Initialize(TelemetryConfiguration.Active);

Step 4

In order to collect counters for the current process - you should use ??APP_CLR_PROC?? for CLR counters and ??APP_WIN32_PROC?? for windows counters. Typically counter instances will be named after process name. However in case of multiple instances of the process running you will have names like w3wp#3 representing third instance of the w3wp.exe process.

This indeces in instance names will change over time. For example, when process w3wp#2 will finish, w3wp#3 will become w3wp#2. Moreover, instance name for CLR counters is different than for windows counter as CLR counters only count processes that runs .NET code inside.

So PerfCounterCollector will regularly check the mapping between the instance name and process ID using counters: \.NET CLR Memory(*)\Process ID for managed counters and Process(*)\ID Process for windows counters is you are using keywords ??APP_CLR_PROC?? and ??APP_WIN32_PROC?? as instance names.

Step 5

You are all set. Counter will be sent to the portal every minute.

Custom counters will be sent as metrics:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
{
  "name": "Microsoft.ApplicationInsights.foo.Metric",
  "time": "2015-10-07T08:22:20.6783162-07:00",
  "iKey": "Foo",
  "tags": { "ai.internal.sdkVersion": "1.2.0.5639" },
  "data": {
    "baseType": "MetricData",
    "baseData": {
      "ver": 2,
      "metrics": [
        {
          "name": "GC Handles",
          "kind": "Measurement",
          "value": 123
        }
      ],
      "properties": {
        "CounterInstanceName": "TestPerfCounters.vshost",
        "CustomPerfCounter": "true"
      }
    }
  }
}

Standard perofrmance counters will be sent as performance counters:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
  "name": "Microsoft.ApplicationInsights.foo.PerformanceCounter",
  "time": "2015-10-07T08:22:20.6464765-07:00",
  "iKey": "Foo",
  "tags": { "ai.internal.sdkVersion": "1.2.0.5639" },
  "data": {
    "baseType": "PerformanceCounterData",
    "baseData": {
      "ver": 2,
      "categoryName": "Process",
      "counterName": "% Processor Time",
      "instanceName": "TestPerfCounters.vshost",
      "value": 20.0318031311035
    }
  }
}

For web applications you can configure the list of performance counters to monitor using configuration file.

Note, that for SDK to collect counters identity application is running under should be part of the group Performance Monitor Users.