Programmatically create Azure Container Instances in an existing Virtual Network

In recent years I've worked extensively with various approaches to create ACI's, also known as Azure Container Instances.

I have these standard approaches for various scenarios:

• Define a YAML file, and create ACI from the command line.
• Define an ARM template and create ACI using a Resource Deployment.
• Programmatically create ACI using the Azure Fluent SDK.

In various use cases in my daily work, I have to rely on some of these approaches for spinning up new workloads, for short- or long-term tasks.

Use case

I have Azure Functions and Azure App Services for everyday background tasks and web front-end. It is not always that the Functions can, or should, handle the long-running operations we do with our containers - for that reason, I can programmatically spin new container groups up and let them do the asynchronous work as needed.

There are many use cases for programmatically creating containers on the fly. A typical situation when building products that sometimes need to burst and scale-out with long-running intensive processes, is to shoot out a new set of ACI groups and let them do the processing, and this can help me with a few things:

• Offload the Azure App Service Plans from spikes and long-running CPU and Memory intensive operations.
• Keep a full data- and process isolation per Azure Container Instance group and entirely separate all workloads logically and legally (if data processing requirements dictate this for the given case).
• Ensure I don't impact any other aspect of my applications running in the cloud.
• With a confident smile, I know that I am embracing the microservice architecture we want to achieve.
• Make bug fixes, repairs, restarts, and enhancements on these services running in the containers without any impact on other teams or resources.
• Allow for an asynchronous and decoupled workload.

Programmatically creating an ACI

There are many examples of how to create Azure Container Instances programmatically. I am using the Azure Fluent SDK.

It is a good idea to get acquainted with:

• Microsoft.Azure.Management.ContainerInstance.Fluent (NuGet)
• The IAzure object, where you'll get authenticated and get your connections wired up.

I use Managed Identity to ensure my Function App can connect and acquire the access, and then I'll map this to an IAzure object. This requires your Azure Function (or wherever you are running your code from) to be bound to a managed identity and have permission to create new container groups.

// Get service credentials through Managed Identity
AzureServiceTokenProvider azureServiceTokenProvider = new AzureServiceTokenProvider();
var serviceCreds = new TokenCredentials(await azureServiceTokenProvider.GetAccessTokenAsync("https://management.azure.com/"));

// Define the AzureCredentials object, and point to the public cloud.
AzureCredentials azureCredentials = new AzureCredentials(
    serviceCreds,
    serviceCreds,
    serviceCreds.TenantId,
    AzureEnvironment.AzureGlobalCloud);

// Define the RestClient, using public cloud, loglevel and the credentials we created above.
RestClient client = RestClient
    .Configure()
    .WithEnvironment(AzureEnvironment.AzureGlobalCloud)
    .WithLogLevel(HttpLoggingDelegatingHandler.Level.BodyAndHeaders)
    .WithCredentials(azureCredentials)
    .Build();

// Authenticate to Azure using the credentials, and target your subscription.
IAzure azure = Microsoft.Azure.Management.Fluent.Azure
    .Authenticate(client, serviceCreds.TenantId)
    .WithSubscription("YOUR_SUBSCRIPTION_GUID");

The code above will:

• Use Managed Identity of the executing host to acquire an access token.
• Define a RestClient and any options you want as part of that.
• Authenticate to Azure using the RestClient, and target your subscription.

With that out of the way, we can now start working with the rest of the Fluent SDK. For example, to spin up the new ACI, I need to know the Network Profile Name to which to connect.

Get a Network Profile Id using the Azure Fluent SDK

The tricky part when writing this was to grab the Network Profile Id. Perhaps there are better ways to do it now. However, when I wrote this, the following worked well for my workloads.

Here are some examples of values we can grab if we have a Virtual Network object, including subnets, network profile id, and network profile name. I have used this in various extents in the past, but you could also use a more direct approach to get only the profile name if you need it. Below are some examples of what you can get, including the Network Profile Name.

// Get Virtual Network
var vnet = await azure.Networks.GetByIdAsync("vnet-magicalunicorns");

// Get subnet
var containerSubnetValue = vnet.Subnets.First(s => s.Value.Name.Contains("snet-magicalunicorns-containers")).Value;
           
// We expect a profile with the Id from deployment: /subscriptions/<GUID>/resourceGroups/<RGNAME>/providers/Microsoft.Network/networkProfiles/<CONTAINERCONFIG>/
var ipConfigProfile = containerSubnetValue.Inner.IpConfigurationProfiles.First(p => p.Id.Contains("ipconfigprofileunicorns")).Id;

// Profile Id: The profile doesn't need to contain anything after /containerNetworkInterfaceConfigurations
var networkProfile = ipConfigProfile.Remove(ipConfigProfile.LastIndexOf("/containerNetworkInterfaceConfigurations"));

// Profile Name: The last part of the string is the name we need for our ACIs.
var networkProfileName = networkProfile.Split("/").Last();

Key points:

• networkProfileName now contains the name of the network profile, which is required in the IContainerGroup creation later.
• If you've found a smoother way to do this by now, I'd be happy to update the code snippets to reflect that.

Create an IContainerGroup and deploy into a Virtual Network

Now I want to spin up an ACI inside of the Virtual Network that I already have.

Deploying the container into the virtual network provides me with secure networking for all resources I have, and they can securely connect while disallowing all external requests.

For me, the case is that I have a User Assigned Managed Identity that can access my Azure Storage accounts and my Azure Key Vault. A specific Virtual Network connects these services, and only allow traffic from within the virtual network to access them - and on top of that, the Key Vault requires an access policy to grant you access to the Secrets, which is currently only delegated to a Managed Identity or service principal.

Sidenote: As of this writing 2020-07-01, the Azure Container Instances does NOT enable us to deploy an ACI to a Virtual Network AND a Managed Identity. If you want both Managed Identity AND a network connected, you're out of luck right now. See the section in the bottom for links and more info about this.

Remember that we have two things already when we go into the code below:

• The IIdentity object has been obtained as per the previous sections.
• The networkProfileName string has been obtained as per the previous sections.

Create a new IContainerGroup, and pass in whatever variables and configurations you need. Specifically in my current scenario, I can now get it into the Virtual Network I need, and at the same time assign my existing User Assigned Managed Identity.

IContainerGroup containerInstance = azure.ContainerGroups.Define(containerGroupName)
    .WithRegion(Region.EuropeWest)
    .WithExistingResourceGroup(this.ResourceGroupName)
    .WithLinux()
    .WithPrivateImageRegistry(acrServer, acrUser, acrPass)
    .WithoutVolume()
    .DefineContainerInstance(containerGroupName)
        .WithImage(containerImage)
        .WithCpuCoreCount(2.0)
        .WithMemorySizeInGB(4)                    
        .WithEnvironmentVariableWithSecuredValue("MySecretKey1", "secret value")
        .Attach()
    .WithRestartPolicy(ContainerGroupRestartPolicy.Always)
    .WithNetworkProfileId(this.SubscriptionId, this.ResourceGroupName, networkProfileName)
    .WithTag("ResourceArea", "tobias-demo")
    .Create();

Key line:

• WithNetworkProfileId

The method takes the subscription, resource group and the name of the network profile as parameters, and then ensures that the Azure Container Instance runs within the correct network.

Verify that the creation works correctly

Seeing it in action is a thing of beauty. If we go to the Azure Container Instance, we can ensure that it has been properly connected to the vnet.

Verify the Azure Container Instance properties

With the running containers, we can  see that it's connected to the Virtual Network now - we see this from the ACI resource's "Properties" page.

We can also see the private IP on the overview page. I already know that this IP range matches the one of my vnet, so things are looking great.

Verify the resource ARM template

If we go to "Export Template" to investigate the ARM template json, we can see that the network profile is tied to the Azure Container Instance from there:

"networkProfiles_nwprofile_REDACTED": {
    "defaultValue": "/subscriptions/REDACTED/resourceGroups/rg-weu-REDACTED/providers/Microsoft.Network/networkProfiles/nwprofile-REDACTED",
    "type": "String"
}

// ...

"networkProfile": {
    "id": "[parameters('networkProfiles_nwprofile_REDACTED')]"
}

Summary

Running Azure Container Instances are amazingly smooth, and programmatically creating them is straight forward. With connecting the ACI to an existing Virtual Network, we can ensure that the containers can access resources that are privately accessible only from within that network.

Examples of services I have in my virtual networks include:

• Azure Key Vault
• Azure Storage Accounts
• Azure App Services (Web Apps, Function Apps)
• Azure Container Instances
• ...

Connecting these all to the same virtual network, across various subnets, I can delegate access to resources inside the virtual network only, and no external requests can reach any of the resources. This adds an extra layer of control and security on top of resource access with keys and authentication.

Essential reading to understand the implications and benefits of networking with Azure Container Instances:

• Virtual network scenarios and resources (Microsoft Docs)
• Deploy container instances into an Azure virtual network (Microsoft Docs)

I hope this post can shed some light on your journeys with containers in the Azure cloud, and how to drop your new ACI's into an existing network.

Links

As a refresher, here are some interesting previous posts about Azure Container Instances and related topics:

• ACI - Using Managed Identity to access Key Vault secrets with C#
• ACI and Secrets - Creating secret volumes and consume secrets using C# .NET Core
• ACI and Secrets - Using Secure Environment Variables
• ACI and ACR - C# .NET Core apps running as containers in your private Azure Container Registry

Posts about Azure Container Registry:

• Best Practices for security in Azure Container Registry
• How Tokens and Scope Maps for Azure Container Registry introduces great repository-level access restrictions
• Thoughts on Bring Your Own Key, or BYOK, to Azure Container Registry
• Protecting your Azure Container Registry by denying all requests except from allowed IP addresses