APM tips blog

Blog about application monitoring.

How Application Insights Status Monitor DOES NOT Monitor Dependencies

| Comments

In this article I’ll address the common misperception that Status Monitor collects telemetry. I’ll show how it helps to collect (but not collects itself) application dependencies information.

Application Insights collects information about application dependencies. Most of the time you don’t need to do anything special to collect all outbound HTTP and SQL calls with the basic informaiton like URL and stored procedure name. Application Insights SDK will use the EventSource traces that .NET framework 4.6 emits.

However if you need more information about the dependencies like raw SQL statement we recommend to install Status Monitor.

I created a small demo application that shows how Status Monitor helps to collect dependencies. In this demo you can learn how to collect SmtpMail.Send method as an external dependency call.

The demo application doesn’t use Status Monitor. Instead it downloads binaries distributed by Status Monitor as NuGet packages:

1
2
<package id="Microsoft.ApplicationInsights.Agent_x64" version="2.0.5" />
<package id="Microsoft.ApplicationInsights.Agent_x86" version="2.0.5" />

Status Monitor will install the binaries (called Runtime Instrumentation Agent) from those NuGet packages into the folder %ProgramFiles%\Microsoft Application Insights\Runtime Instrumentation Agent.

If you run this demo application from Visual Studio you’ll get the message: Please run this application using Startup.cmd script..

Running the script will instruct .NET Framework to enable Runtime Instrumentation Agent for the current process. When you run the application using Startup.cmd script - you’ll see the message Application started with the Runtime Instrumentation Agent enabled. Press any key to continue.... Looking into the code you may notice that the method SmtpClient.Send was already called by that moment. However as Status Monitor do NOT monitor dependencies - Runtime Instrumentation Agent did nothing when this call was made.

We want to report every call of the method SmtpClient.Send as a dependency call. We know what information to collect with that dependency call - mail subject, smtp host, etc.

This is how to configure the monitoring. First - import another NuGet package:

1
<package id="Microsoft.ApplicationInsights.Agent.Intercept" version="2.0.5" />

Second - call the method Decorate and pass the following method information - assembly name, module name and full method name.

1
2
3
Decorator.InitializeExtension();
Functions.Decorate("System", "System.dll", "System.Net.Mail.SmtpClient.Send",
    OnBegin, OnEnd, OnException, false);

You also need to pass three callbacks:

1
2
3
public static object OnBegin(object thisObj, object arg1)
public static object OnEnd(object context, object returnValue, object thisObj, object arg1)
public static void OnException(object context, object exception, object thisObj, object arg1)

The call to Decorate will do the magic. It finds the method you specified and using the Runtime Instrumentation Agent inserts those callbacks into the beggining, end and in the global try{}catch statement of that method. This magic only allowed when Runtime Instrumentation Agent is enabled for the process.

In the callbacks implementation I start the operation in OnBegin callback:

1
2
3
4
5
6
7
8
9
10
11
12
13
public static object OnBegin(object thisObj, object arg1)
{
    // start the operation
    var operation = new TelemetryClient().StartOperation<DependencyTelemetry>("Send");
    operation.Telemetry.Type = "Smtp";
    operation.Telemetry.Target = ((SmtpClient)thisObj).Host;
    if (arg1 != null)
    {
        operation.Telemetry.Data = ((MailMessage)arg1).Subject;
    }
    // save the operation in the local context
    return operation;
}

And stop operaiton in OnEnd and OnException callbacks:

1
2
3
4
5
6
7
8
public static void OnException(object context, object exception, object thisObj, object arg1)
{
    // mark operation as failed and stop it. Getting the operation from the context
    var operation = (IOperationHolder<DependencyTelemetry>)context;
    operation.Telemetry.Success = false;
    operation.Telemetry.ResultCode = exception.GetType().Name;
    new TelemetryClient().StopOperation(operation);
}

Notice the runtime arguments passed to the original method are used in those callbacks to collect information. Argument called thisObj is an instance of SmtpClient that made a call and arg1 is a MailMessage that was passed as an argument.

So all the data collection logic is implemented in the application itself. Runtime Instrumentation Agent just provided a way to inject the callbacks into the methods of interest.

This is how Status Monitor helps to collect dependencies. During installation Status Monitor enables Runtime Instrumentation Agent for all IIS-based applications. Agent does nothing if application does not use Application Insights SDK. It has zero impact in the runtime. Only when Application Insights SDK is initialized - it can use Runtime Instrumentation Agent to monitor any methods it chooses. Status Monitor doesn’t have information on what needs to be instrumented, what data should be collected and where this information have to be send. It all defined in code by Application Insights SDK.

This approach allows to version Application Insights SDK data collection logic without the need to re-install the agent. It also guarantees that the update of an agent will not change the way telemetry is being collected for your application.

I’ll describe other functions of Status Monitor in the next blog posts.

BTW, the same Runtime Instrumentation Agent is installed by Microsoft Monitoring Agent and Application Insights Azure Web App extension.

Sync Channel

| Comments

Application Insights API library provides a basic methods to track telemetry in the application. Like TrackTrace and TrackMetric. It also implements the basic channel to send this data to the Application Insights.

If you are using Application Insights API in short running tasks you may hit a problem that some telemetry wasn’t send. Basic channel do not provide control over data delivery. Method Flush makes an effort to flush telemetry left in buffers, but do not guarantee delivery either.

This is a very simple sync channel you can use to overcome the issues above. It has two features:

  1. No need to Flush. When Track method complete - event is delivered
  2. Sending is synchronous. No new threads/tasks will be started
  3. When delivery fails - method Track will throw an exception and you can re-try.

Here is the code with the usage example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
using System;
using System.Collections.Generic;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.ApplicationInsights.Extensibility.Implementation;

namespace DemoSyncChannel
{
    class Program
    {
        class SyncTelemetryChannel : ITelemetryChannel
        {
            private Uri endpoint = new Uri("https://dc.services.visualstudio.com/v2/track");

            public bool? DeveloperMode { get; set; }

            public string EndpointAddress { get; set; }

            public void Dispose() { }

            public void Flush() { }

            public void Send(ITelemetry item)
            {
                byte[] json = JsonSerializer.Serialize(new List<ITelemetry>() { item }, true);
                Transmission transimission = new Transmission(endpoint, json, "application/x-json-stream", JsonSerializer.CompressionType);
                var t = transimission.SendAsync();
                t.Wait();
            }
        }

        static void Main(string[] args)
        {
            TelemetryConfiguration.Active.TelemetryChannel = new SyncTelemetryChannel();

            TelemetryConfiguration.Active.InstrumentationKey = "c92059c3-9428-43e7-9b85-a96fb7c9488f";
            new TelemetryClient().TrackTrace("Sync trace");

            // this will throw exception
            TelemetryConfiguration.Active.InstrumentationKey = "invalid instrumentation key";
            new TelemetryClient().TrackTrace("Sync trace");
        }
    }
}

Friday Post - Application Insights Misspellings

| Comments

Here is the list of popular misspellings of Application Insights. Welcome to my blog if you found this post by searching one of those. Here are some Application Insights links for you.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
lication insights         application inishgt      application inssights
application insight       application inishgts     application instight
spplication insight       application inisight     application instights
application insights      application insghits     application insugfhts
aapalication insight      application insght       application isights
aoolication insight       application insghts      application isnights
aoolication insights      application insgiht      application isnisghts
aaplication insight       application insgihts     application iunsights
pplication insights       application insgith      application lisights
aaplication insights      apllication insights     application sinsights
aapplication insights     application insgiths     application sisights
appliaction insights      application insgsights   apoplication insights
appliation insight        application inshgts      applicationb insights
appliation insights       application insi         applicationinisghts
applicaino insights       application insifhts     applicationinsight
applicaion insights       application insigghts    applicationinsights
applicaition insights     application insigh       applicationinsigts
applicaiton inisght       application insighes     applicationm insights
applicaiton inisghts      application insighets    applicationn insights
applicaiton insigh        application insighhts    applicatioon insights
applicaiton insight       application insighrs     applicatipn insights
applicaiton insights      application insighs      applicatno insights
applicaiton isnights      application insight      applicatoin insights
applicaitoninsights       application insightas    applicatoininsights
aplicaiton insgights      application insightd     applicaton insight
aplicatationm insisghts   application insighte     applicaton insights
applicarion inaights      application insightes    applicatuon insights
applicarion insights      application insightrs    appliciation insight
applicartion insights     application insights     applicsation insights
aplication insight        application insightsa    applictaion insight
aplication insights       application insightsd    appliction insight
aplication insites        application insightss    appliction insights
applicat5io insigh        application insighys     appicaiotn insight
applicataion insight      application insigight    appication insghts
applicated insight        application insignts     appication insight
applicatin insights       application insigt       appication insights
applicatino insighs       application insigth      appliocation insights
applicatino insights      application insigths     applkication insghts
applicatio insight        application insigts      appllication insights
applicatio insights       application insiguts     appilcation insights
applicatio insigths       application insihts      appilication insight
applicatioin insight      application insiights    applucation insights
applicatiom insights      application insinghts    appolication insights
application hinsights     application insioghts    appplication insight
application in sights     application insisghts    appplication insights
application inaighta      application insites      applation insights
application ingishts      application insitghs     applcation insights
application inights       application insitghts    applciaiton insights
application inisghts      application insoghts     applciation insights

SSL Expiration Monitoring

| Comments

Now this blog is available via https. Thanks for the courtesy of Let’s Encrypt and great step by step instruction how to install it on Azure Web App.

SSL certificate from Let’s Encrypt expires in 3 month. Instruction above configures a web job to update certificate before it expire. However, you may want to set up an extra reminder for the certificate expiration.

Application Insights web test will fail when certificate is invalid. It will be a little bit late as certificate is already expired.

So I created a little tool that will return certificate information for a given domain name.

When you call http://webteststools.azurewebsites.net/certificate/apmtips.com/ - it will return a JSON with certificate information like this:

1
2
3
4
5
6
7
{
    "ExpirationDate":"10/18/2016 6:30:00 AM",
    "ExpiresIn10Days":false,
    "IssuerName":"CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US",
    "Subject":"CN=apmtips.com",
    "Error":false
}

So you can set up an Application Insights web test that will call that url and validate response:

Now when "ExpiresIn10Days":false will turn into "ExpiresIn10Days":true - alert will fire and there will be 10 more days to fix a certificate.

There is now a new point of failure - this new tool. If it is down - you will get a false alarm. Considering Azure Web Apps SLA and the fact that certificates do not expire too often - it may be a good compromise.

Tracking Users in API Apps

| Comments

Triaging and diagnosing issues you need to correlate telemetry with the user. You may want to understand the impact of a failing dependency (like a percent of affected users) or look up all pages visited by the user before the failure occurred.

Application Insights has multiple fields to associate telemetry event with the user - anonymous user id, authenticated user name and account id. When you are using the combination of JavaScript and C# SDK - anonymous user id will be collected automatically. JavaScript SDK sets the cookies that will be sent with every request.

However when the application you are running is REST endpoint it does not have it’s own front end. So there is no JavaScript to set a cookie and track the user.

So you need to set user cookie ai_user yourself. The code will look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
string userId = null;

var ctx = HttpContext.Current;

if (ctx != null)
{
  if (ctx.Request.Cookies["ai_user"] == null)
  {
    userId = Guid.NewGuid().ToString();
    var c = new HttpCookie("ai_user", userId + "|" + DateTime.Now.ToString("G"));
    c.Expires = DateTime.MaxValue;
    c.Path = "/";

    ctx.Response.Cookies.Set(c);
  }
}

Now with every next request, telemetry initializer Microsoft.ApplicationInsights.Web.UserTelemetryInitializer will associate the telemetry item with the user id taken from the cookie.

End of story.

Except a little detail. Initial request will not be associated with the user id. There is a good thing in it. Not every request is coming from the browser. So if the cookie will not be saved by caller - this initial request will not generate a new user id. So metrics like user count would not be affected.

If you are quite sure that the request is coming from an actual browser - it is quite easy to associate newly generated user id with the current request telemetry item.

Just get request telemetry from the current HttpContext using the extension method GetRequestTelemetry and set Context.User.Id. All other telemetry items like traces, dependencies and exceptions tracked after that will be automatically correlated with this user id. The code will look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
if (string.IsNullOrEmpty(userId))
{
  var ctx = HttpContext.Current;

  if (ctx != null)
  {
      var requestTelemetry = ctx.GetRequestTelemetry();
      if (requestTelemetry != null))
      {
          requestTelemetry.Context.User.Id = userId;
      }
  }
}

There are definitely edge cases. For instance, if the landing page is cached and it initiates many ajax calls on loading - each of them will generate a new random user id. So you may end up with the different user id for the same user. But if it is the case - you probably would know how and when to set cookies for your application the best.

Client IP Address

| Comments

Client IP address is useful for some telemetry scenarios. You may discover very high latency from remote countries or the reason for a requests count spike in the night when countries across the ocean woke up.

Application Insights collects client IP address. Country, state and city information will be extracted from it and than the last octet of IP address will be set to 0 to make it non-identifiable. So Application Insights will never store an actual IP address by default.

There are two ways IP address got collected for the different scenarios.

Browser and devices

When telemetry is sent from browser by JavaScript SDK or from device - Application Insights endpoint will collect sender’s IP address. Endpoint doesn’t resolve as IPv6 so this IP address will always be IPv4.

Server applications

Client IP address for the server application will be collected by SDK. In .NET it is done by ClientIpHeaderTelemetryInitializer. This telemetry initializer will check X-Forwarded-For http header and if it is not set - use client IP.

Caveat here is that Application Insights only supports IPv4 at the moment of this writing. So if the clients of your application are using IPv6 – IP address will not be send to Application Insights.

Now when Application Insights receives an event without IP address set - it will assume that this event came from the device and will store the server’s IP address. This is why you may find some fake Brazilian clients when your application was deployed in Azure.

You may also end up getting the firewall/load balancer IP address for all your clients if this firewall sets an original IP address into a different http header. Popular one is X-Originating-IP.

It is easy to override the default logic of ClientIpHeaderTelemetryInitializer using configuration file. You can set a list of header names to check, separators to split IP addresses and whether to use first or last IP address. Here is how to override default settings:

1
2
3
4
5
6
7
<Add Type="Microsoft.ApplicationInsights.Web.ClientIpHeaderTelemetryInitializer, Microsoft.AI.Web">
  <HeaderNames>
    <Add>X-Originating-IP</Add>
  </HeaderNames>
  <HeaderValueSeparators>;</HeaderValueSeparators>
  <UseFirstIp>false</UseFirstIp>
</Add>

Now, when your application will receive the header X-Originating-IP: 8.8.8.1;8.8.8.2 telemetry will be sent with the following context property: "ai.location.ip":"8.8.8.2".

If you want to keep the full IP address with your telemetry and storing client’s PII information is not a concern - you can implement a telemetry initializer:

1
2
3
4
5
6
7
8
9
10
public class CopyIPTelemetryInitializer : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
        if (!string.IsNullOrEmpty(telemetry.Context.Location.Ip))
        {
            telemetry.Context.Properties["client-ip"] = telemetry.Context.Location.Ip;
        }
    }
}

This telemetry initializer will store IP address in the custom property and it’s last octet will not be set to zero. Make sure to add it after ClientIpHeaderTelemetryInitializer.

Another tip - C# SDK do not allow to sent IPv6 addresses to Application Insights. Using custom properties is a good alternative for sending it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
public class IPv6AddressTelemetryInitializer : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
        if (HttpContext.Current != null)
        {
            IPAddress ip;
            if (IPAddress.TryParse(HttpContext.Current.Request.UserHostAddress, out ip))
            {
                telemetry.Context.Properties["client-ip"] = ip.ToString();
            }
        }
    }
}

Where is my map?

Once IP addresses collected properly - the next step is to map them. There is no map in Azure portal. But you can easily visualize your telemetry on the map using Power BI integration.

Application Insights for MVC and Web API

| Comments

Application Insights Web SDK today has a very basic support for MVC and Web API applications. In this post I collected some links on how to make monitoring of your applications better. This is not a definitive guide on how to monitor those types of applications correctly. This is just a dump of accumulated knowledge of some problems and their solutions.

Telemetry correlation

This is how NuGet server implemented OWIN middleware to track telemetry. It uses custom CallContext and time tracking.

This blog post suggests much better approach for telemetry correlation using the newest features of Application Insights SDK. Author also suggests the workaround for this issue that looks very promising.

Use Action filters

Action filters can also be used for telemetry correlation. CallContext will not be lost when using action filters. However, timing of request will not be as accurate as when middleware being used.

Exceptions

Exceptions are handled by MVC infrastracture and sometimes will not reach http module. Here is a documentation on how to configure exception handling in different versions of MVC.

There is also an ApplicationInsights.Helpers NuGet package that implements exception handling for MVC.

Exceptions thrown by middlewares should also be collected. Typically, you’d need a separate middleware to catch exceptions. You can find an explanation why there should be two middlewares at Step 3 in this blog post.

Request names

I mentioned before that for attribute-based routing operation names will have identifier in them and will not be groupable. I advice to set Route template as a request name:

1
this.RequestContext.RouteData.Route.RouteTemplate

Configuration file

In order to collect performance counters and dependencies as well as some other good features - you’d want to install Windows Server nuget. This nuget will create and populate ApplicationInsights.config file. So most of monitoring configuration will be defined in that configuration file. However if you’d like to use code-based configuration approach and dependency injection - you’ll need to remove this file after every nuget update.

Singleton and DI

When using dependency injection to inject TelemetryConfiguration and TelemetryClient classes - singleton TelemetryConfiguration.Active will not be initialized. Or even worse - will be initialized with some unexpected values. Thus the code from documentation will not send event:

1
2
var client = new TelemetryClient();
client.TrackEvent("purchase completed");

This is how we solved it in ASP.NET core. We take configuration from Application Insights’s singleton and use it as DI singleton.

Another issue with the dependency injection is that identity provider can only be obtained using DI. It is not that easy to create a TelemetryInitializer that will be instantiated on every request by DI. Application Insights has it’s own mechanism of initializing and do not have a native mechanism to create a list of telemetry initializers per request.

Bugs

There are some bugs I cannot explain today. There are reports that sometimes request name has a mystery [id] in it. For instance, request to this controller will be reported as POST MyController [id]:

1
2
3
public class MyController : ApiController
{
    public async Task<HttpResponseMessage> Post(HttpRequestMessage request)

Looking at how request name generated I cannot explain it.

Have You Been Hacked?

| Comments

I’ve returned from the long parental leave. Checking telemetry for this blog I noticed the new dependency call

Here is how it looks on Application map:

It scared me as there are no AJAXes in the blog. So my first thought - my blog was hacked.

Diving deeper - it is a call to http://api4.adsrun.net/post. Looks even scarier. Quick internet search showed that this is a malware installed on visitor’s computer. This malware injects malicious code into jquery script that browser loads. So apmtips.com is safe, it’s visitor’s computer has a security problem.

Looking into All available telemetry for this user session I found some details of the visitor. So if you are visiting from the city of Sangli in India using Firefox - check your computer for malware.

Validate Event Json

| Comments

Application Insights endpoint now supports events validation. It’s very easy to use - just post your telemetry item json to this endpoint and if response code is not 200 response text will contain the error message.

In C# code will look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
static async Task<bool> IsEventCorrect(string json)
{
    json = json.Replace("duration", "durationMs");

    HttpClient client = new HttpClient();
    var result = await client.PostAsync(
        "https://dc.services.visualstudio.com/v2/validate",
        new ByteArrayContent(Encoding.UTF8.GetBytes(json)));

    Console.WriteLine(result.StatusCode);

    if (result.StatusCode != HttpStatusCode.OK)
    {
        var response = await result.Content.ReadAsStringAsync();
        Console.WriteLine(response);
        return await Task.FromResult(false);
    }
    return await Task.FromResult(true);
}

Result of execution will be something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
BadRequest
{
  "itemsReceived": 1,
  "itemsAccepted": 0,
  "errors": [
    {
      "index": 0,
      "statusCode": 400,
      "message": "106: Field 'duration' on type 'RequestData' is of 
                    incorrect type. Expected: string, Actual: undefined"
    }
  ]
}

Validation that this endpoint provides is not 100% strict. It guarantees that event is well-formed and has all the required fields. So it will be accepted by the /track endpoint. However, today it will allow sending some extra fields in json payload that will never be saved in backend.

With the .NET SDK it’s easy to generate test JSON. You’ll need to construct telemetry type of interest. In this example - RequestTelemetry. Then use JsonSerializer class to get byte array:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
RequestTelemetry rt = new RequestTelemetry();

rt.Context.InstrumentationKey = "c92059c3-9428-43e7-9b85-a96fb7c9488f";

rt.Name = "RequestName";
rt.StartTime = DateTimeOffset.Now;
rt.Duration = TimeSpan.FromSeconds(10);

string json = Encoding.Default.GetString(
    JsonSerializer.Serialize(new List<ITelemetry>() { rt }, false));

json = json.Replace("duration", "durationMs");

var t = IsEventCorrect(json);

You can also fill out all the properties:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
// Host context
rt.Context.Cloud.RoleName = "Role Name";
rt.Context.Cloud.RoleInstance = "Role Instance";

// Application context
rt.Context.Component.Version = "Application Version";

// Custom properties - limit 200 per application
rt.Context.Properties["DeploymentUnit"] = "SouthUS";

// Application user context
rt.Context.Location.Ip = "127.0.0.1";
rt.Context.Operation.SyntheticSource = "Test in production";

// Session context
rt.Context.User.Id = "Anonymous User Id";
rt.Context.Session.Id = "Anonymous Session Id";

rt.Context.User.AccountId = "Account Id";
rt.Context.User.AuthenticatedUserId = "Authenticated user id";

// Operation context
rt.Context.Operation.Id = "Root operatioin id";
rt.Context.Operation.ParentId = "Parent Operation Id";
rt.Context.Operation.Name = "Operation name";

Using C# JSON serialization and validation endpoint it is easier to understand why json formed by any other SDK is not being accepted by Application Insights endpoint.

Telemetry Channels Update

| Comments

This blog post was written by Anastasia Baranchenkova

From the time Sergey wrote his blog post there are a few changes.

Persistence channel was removed. There will be no new versions of Application Insights SDK for devices. The suggested solution is to use HockeyApp that Microsoft acquired last year.

Server channel became open source.

Also it was mentioned in Sergey’s blog post that Server telemetry channel stores events on disk if they cannot be sent. The channel would use either current user’s local app data folder or current process temp folder. I saw several cases when there was a problems with that.

In the first case process did not have access neither to local app data nor to the temp folder. The workaround was to make sure it has it.

Second one was more interesting. There was an application that worked perfectly fine locally but when deployed to a VM it crashed. It appeared that local app data was pointing to an unmapped drive. Unfortunately ApplicationInsights was not catching this type of exception, so it was just crashing without even trying to check temp folder. This bug was fixed in 2.0.0-beta4.

Additionally while fixing this issue we added an ability to specify custom folder either though configuration or in code (if it is not provided or inaccessible old logic is used).

In configuration file you would do it this way (make sure you have correct xml closing tags):

1
2
3
<TelemetryChannel Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel, Microsoft.AI.ServerTelemetryChannel">
    <StorageFolder>D:\NewTestFolder</StorageFolder>
</TelemetryChannel>

In code you would do it this way:

  • Remove ServerTelemetryChannel from configuration file
  • Add this piece of code to the place where application gets initialized:
1
2
3
4
5
ServerTelemetryChannel channel = new ServerTelemetryChannel();
channel.StorageFolder = @"D:\NewTestFolder";
channel.Initialize(TelemetryConfiguration.Active);

TelemetryConfiguration.Active.TelemetryChannel = channel;

An interesting side note: configuration loading is very flexible. If you see a public property on an object that can be defined in the configuration file (e.g. TelemetryInitializer, TelemetryProcessor or Channel) this property can be set via configuration file. We did not have to change configuration loading logic for this update, we just added a public property to the ServerTelemetryChannel.