Retrieve logs from Application Insights programmatically with .NET Core (C#)

When working with Azure's Application Insights, there's some times where I would've wanted to quickly and programmatically export specific events, search the logs or otherwise pull some data out based on dynamic metrics of applications or monitoring solutions I've set up.

In this post we'll take a look at how easy it is to use the Microsoft.Azure.ApplicationInsights NuGet package to utilize .NET Core to retrieve data programmatically from Application Insights.

For example, in the Azure Portal I can easily see my Application Insights data on demand and search and filter my logs in the intuitive and simplified UI:

However, you don't always want to use this view, or Log Analytics or Azure Monitor for the parsing and retreival of data. When you've got any type of requirement to do this programmatically and work with the data in other ways, there's good news...

In the following section, we'll talk about how we can programmatically expose information from App Insights (Exceptions, Custom Events, ...).

Here's an example of listing all exceptions in the last 12 hours and just printing the messages out. For demo purposes, it should be clear how the code works by the end of this post:

Follow along to learn step by step how to programmatically fetch information from your Application Insights service.

Don't forget to leave a comment in the bottom!

Step 1. Setup a new Azure AD Application

For this to work, we need to prepare a few things:

• A new Azure AD Application with a new Secret (ClientId, ClientSecret).
• Assign the required permissions of the new app to your Application Insights service.

This can be done using the Azure CLI, using the Azure Portal or any of the management API's. I'm using the portal in this context - hence the following steps are all relative to your Azure Portal.

1.1. Create an Azure AD Application

Go to Azure Portal - Azure Active Directory - App Registrations and then "+ New registration":

Make note of the App Id (ClientId) from your new application's "Overview" page. You'll need this in the code later:

Click "Certificates & Secrets" and "+ New client secret" in order to create a new secret for this application. Make a note of the secret, as you'll need it in the code later:

Great, we're now prepared for configuring the access rights of our application and allow it to read/contribute/whatever to our Application Insights service.

1.2. Assign Role Based Access Control to App Insights for your Azure AD Application

In order for our new AAD app to access Application Insights, we need to assign the desired role so it can access and read data from App Insights.

Go to your App Insights resource and then "Access control (IAM)" and click "Add" in the "Add a role assignment" box.

1.3. Grab the Application Insights API Identifier

For us to access the App Insight from the API, we need to grab the Application ID from "API Access" - "Application ID":

When this is done, we have:

• A new Azure AD application
• Created and copied a secret for our app, and copied the ClientId (App Id)
• Configured RBAC so it can access App Insights
• Copied the App Insights API Application ID

We are now prepared to programmatically use this app to access App Insights, and can use the SDK's to fetch data.

Step 2. Programmatically reading logs from Application Insights with .NET Core

Okay, we're done with the configuration and preparation phase. We've set up our new Azure AD application and ensured that it has access specifically to the App Insights that I want it to have access to using IAM.

In the demo code below, I haven't used protected secrets - please ensure you take care of your credentials if you use any code below; Don't use plain-text credentials ;)

My demo application is just a Console Application, so following along should be easy.

2.1. Get the required NuGet packages

There's two nuget packages we want in order to reach a working code sample.

• Microsoft.Azure.ApplicationInsights
• Microsoft.Rest.ClientRuntime.Azure.Authentication

2.2. Grab the code!

Since the code itself is very basic, you can just grab it in its demo-shape from below, and start using it as-is and then modify it to fit your own needs.

using System;
using Microsoft.Azure.ApplicationInsights;
using Microsoft.Rest.Azure.Authentication;

namespace Zimmergren.Azure.AppInsightLogFetcher
{
    public class Program
    {
        private static ApplicationInsightsDataClient _applicationInsightsDataClient;
        static void Main(string[] args)
        {
            WireUp();

            WriteExceptionsDemo();
            WriteCustomEventsDemo();
        }

        static void WireUp()
        {
            var activeDirectoryServiceSettings = new ActiveDirectoryServiceSettings
            {
                AuthenticationEndpoint = new Uri("https://login.microsoftonline.com"),
                TokenAudience = new Uri("https://api.applicationinsights.io/"),
                ValidateAuthority = true
            };

            var serviceCredentials = ApplicationTokenProvider.LoginSilentAsync(
                    domain: "<Your Azure AD domain name, or tenant id>",
                    clientId: "<Your Azure AD App Client Id>",
                    secret: "<Your Azure AD App Client Secret>",
                    settings: activeDirectoryServiceSettings)
                .GetAwaiter()
                .GetResult();

            _applicationInsightsDataClient = new ApplicationInsightsDataClient(serviceCredentials)
            {
                AppId = "<Your Application Insights Application ID>"
            };
        }

        private static void WriteCustomEventsDemo()
        {
            var events = _applicationInsightsDataClient.GetCustomEvents();
            foreach (var e in events.Value)
            {
                var name = e.CustomEvent.Name;
                var time = e.Timestamp?.ToString("s") ?? "";
                Console.WriteLine($"{time}: {name}");
            }
        }

        private static void WriteExceptionsDemo()
        {
            var exceptions = _applicationInsightsDataClient.GetExceptionEvents(TimeSpan.FromHours(24));
            
            foreach (var e in exceptions.Value)
            {
                // Just for demo purposes...
                var time = e.Timestamp?.ToString("s") ?? "";
                var exceptionMessage = e.Exception.Message;
                if (string.IsNullOrEmpty(exceptionMessage))
                    exceptionMessage = e.Exception.InnermostMessage;
                if (string.IsNullOrEmpty(exceptionMessage))
                    exceptionMessage = e.Exception.OuterMessage;

                Console.WriteLine($"{time}: {exceptionMessage}");
            }
        }
    }
}

That's it. If you've configured your Azure AD application and App Insight correctly, you can simply grab this code and replace the <...> strings with the values of your own environments.

Security awareness: As always, consider your application configuration and security. Don't put any credentials and secrets in plain text in your source code. Code snippet above is provided as a PoC only and shouldn't be used in this shape if you put it to use in your own software and systems.

Enjoy!