Use Application Insights for Worker Service applications in .NET Core console apps, containers, and any non-HTTP apps

Relying on Application Insights to provide great data has always been a core component of anything I create.

Recently, I upgraded my projects to .NET Core 3.1, and in few cases I also upgraded to .NET 5 (Preview). A lot of my code is executed in containers, or other background processes that may not have a native Application Insight integration.

With the upgrade to .NET Core 3.1, we noticed after a while that no logs were persisted to the cloud anymore. I could see telemetry being created when debugging, and from that perspective all the unit tests were green. However, they never seemed to leave the machine and reach the cloud. Something was amiss.

I have been using Microsoft.ApplicationInsights and other related packages for a long time. However, Microsoft recently announced a new set of NuGet packages that can, and should, be used with background workers of any kind - anything that is a non-HTTP application, really. Of course, this got my attention given I recently noticed that my upgraded projects didn't push telemetry anymore.

The new package to download and replace the previous one (for console/non-HTTP apps) is:

• Microsoft.ApplicationInsights.WorkerService (NuGet).

Reading the updated Docs on Microsoft, it says:

It is highly recommended to use the Microsoft.ApplicationInsights.WorkerService package and associated instructions from here for any Console Applications. This package targets NetStandard2.0, and hence can be used in .NET Core 2.1 or higher, and .NET Framework 4.7.2 or higher.
- https://docs.microsoft.com/en-us/azure/azure-monitor/app/console

With this new information, it was all about modifying the existing code and make use of the new SDK. Here are the main changes I had to do, which are all low-effort.

Changes in the code

Upgrading to the new WorkerService SDK was easy, and only some initialization code had to change. Here's what I did.

Install Microsoft.ApplicationInsights.WorkerService

In my projects, I uninstalled any existing version of Application Insights NuGets, unless the projects are ASP.NET Core, or other http-centric applications. Mine are container-based workloads, console apps and other background tasks. Therefore, the WorkerService is my new best buddy for telemetry purposes.

• Get the Microsoft.ApplicationInsights.WorkerService (NuGet)

Initialization code

My previous initialization code for my service looked something like this:

private readonly TelemetryClient _telemetryClient;
public ApplicationInsightsService(string instrumentationKey)
{
    // NOTE: OLD CODE
    using (TelemetryConfiguration configuration = TelemetryConfiguration.CreateDefault())
    {
        configuration.InstrumentationKey = instrumentationKey;
        configuration.TelemetryInitializers.Add(new HttpDependenciesParsingTelemetryInitializer());

        using (DependencyTrackingTelemetryModule depModule = new DependencyTrackingTelemetryModule())
        {
            depModule.Initialize(configuration);
        }

        _telemetryClient = new TelemetryClient(configuration);
    }
}

The updated version of the code, using the WorkerService SDK, looks closer to this:

private readonly TelemetryClient _telemetryClient;
public ApplicationInsightsService(string instrumentationKey)
{
    // NOTE: NEW CODE
    IServiceCollection services = new ServiceCollection();
    services.AddApplicationInsightsTelemetryWorkerService(instrumentationKey);
    IServiceProvider serviceProvider = services.BuildServiceProvider();
    _telemetryClient = serviceProvider.GetRequiredService<TelemetryClient>();
}

If your code relies on injecting specific ApplicationInsightsServiceOption configurations, you can do that easily, too. Here's how to disable the adaptive sampling, for example:

IServiceCollection services = new ServiceCollection();

services.AddApplicationInsightsTelemetryWorkerService(new ApplicationInsightsServiceOptions
    {
        InstrumentationKey = instrumentationKey,
        EnableAdaptiveSampling = enableAdaptiveSampling
    });

IServiceProvider serviceProvider = services.BuildServiceProvider();
_telemetryClient = serviceProvider.GetRequiredService<TelemetryClient>();

Different scenarios will dictate different services you want to inject into the service containers, and alternate ways to initialize your AI service. See the links in the bottom of this post for full examples on Microsoft Docs.

Code to send telemetry

There are a ton of different ways you can craft the logic to send the telemetry and traces. In the remaining parts of my code, I didn't have to make any changes - I am posting a small snippet of the code here for clarity - none of the below code was changed.

public void TrackException(Exception exceptionTelemetry, Dictionary<string, string> properties = null, Dictionary<string, double> metrics = null)
{
    _telemetryClient.TrackException(exceptionTelemetry, properties, metrics);
    _telemetryClient.Flush();
}

public void TrackTrace(string message, Dictionary<string, string> properties = null)
{
    if (properties == null)
        _telemetryClient.TrackTrace(message);
    else
        _telemetryClient.TrackTrace(message, properties);

    _telemetryClient.Flush();
}

You can see that I am explicitly flushing my calls. You don't have to do this on every client send, you can also ensure that you do it before the application exists.

It is important to explicitly flush the TelemetryClient in console and background applications, to ensure that the data is actually sent to Application Insights before the application exists. Since the Flush call isn't synchronous, we need to add a Task.Delay or Thread.Sleep before our application exists - this is good to keep in mind if you see intermittent interruptions in your data, or no data at all.

Summary and links

I hope this can help someone else who are struggling to see their logs fly through to Application Insights. It wasn't obvious what the issue was, and the previous code have worked for a long time - and there are no apparent errors with it either, except that the cloud never sees the actual telemetry we send.

Switching to the new NuGet was easy, and I only had to change my initial wiring-code, and most of the rest remains the same. Low-effort, but important upgrade.

Read more.

• Application Insights for Worker Service apps (non-HTTP apps) (Microsoft)
• Application Insights for .NET Apps (GitHub)