Secure secrets management in AKS

Overview

In this article, we integrate Azure Key Vault with AKS to ensure proper secret management through Terraform.
You can find the link to the GitHub repository: aks-keyvault

Prerequisites

• Azure subscription
• AZ CLI
• Terraform

You can clone the repository and run the Terraform scripts to provision the infrastructure.

Low Level Design

Let's take a closer look at the key components:
Virtual Networks (VNets):
We have two different VNets in our configuration:

• One for AKS (where our Kubernetes cluster is located).
• One for Key Vault (where all our secrets are stored).

These VNets are interconnected, which means they can communicate securely with each other via the Azure private network.
Two key elements facilitate and enable communication:

1. Peering: Peering between the AKS VNet and the Key Vault VNet enables direct and seamless communication. This means that resources in both VNets can interact without going through the internet, ensuring low latency and better performance.
2. Private Endpoint: A Private Endpoint has been created for Azure Key Vault. This plays a crucial role in enabling secure access to Key Vault via a private IP address belonging to the same VNET. This ensures that requests sent from the AKS VNet to Key Vault remain within the Azure backbone network, protecting sensitive data from external threats. In addition, pods/applications in the AKS VNet can use managed identities to authenticate, which simplifies access management.

Inside the AKS VNet:
In this network, we deployed our Kubernetes cluster.
Now, let's assume that your pod is your application that needs a certain secret.
Each pod runs using a service account (SA). This SA is important because it determines who (or what) has permission to access resources such as secrets belonging to Azure Key Vault.

We then use the Secret Store CSI driver to allow our application to retrieve secrets from Key Vault and mount them as volumes in the pods. This allows the application to use these secrets as regular files or even as environment variables.

How Do Pods Access Secrets?

This is where security comes into play:
Each SA is linked to a User Assigned Managed Identity (UAMI), which is like a "special ID card" assigned by Azure to our pods.
This identity is recognized by Azure Key Vault and grants the pod access to secrets based on its permissions defined in the UAMI.

Integration of Federated Identity and UAMI into AKS and Azure Key Vault

To secure access to Azure services via AKS, it is essential to understand federated identity and user-assigned managed identity (UAMI). Here is a summary of these concepts and their role in application security.

What is a Managed Identity?

A Managed Identity allows applications to authenticate with other Azure services without directly managing credentials. There are two types:

• System-Assigned Managed Identity: Created automatically for a specific resource and deleted along with it.
• User-Assigned Managed Identity (UAMI): Created independently and can be shared across multiple resources, offering greater flexibility.

Why Use UAMI?
UAMI simplifies access to Azure resources for multiple applications, enabling centralized permission management and avoiding the need to embed sensitive information in code.

What is Federated Identity?
Federated identity establishes a trust relationship between AKS workloads and Azure Active Directory, enabling secure authentication with Azure services.

Role of Federated Identity

• Establishing Trust: Allows pods to access Azure services without hard-coding credentials.
• Simplified Authentication: The AKS service account can use UAMI to access Azure resources.
• Enhanced Security: Prevents sensitive information from being stored in the cluster, thereby protecting applications.

In the Terraform configuration below, we created a secret on Azure Key Vault named MYSQL-DB-PASSWORDD, which we will try to retrieve later in the pod on AKS. Then we created a UAMI to allow our AKS cluster to access Azure Key Vault. A role assignment grants it permissions, and a federated identity links the AKS service account to the UAMI, facilitating secure authentication without integrating sensitive credentials.

This approach uses RBAC (Role-Based Access Control) to define access permissions to Key Vault. With RBAC, we can assign specific roles to users and services, ensuring granular access control and better resource security. This makes it easier to manage access over time by aligning permissions with service roles and responsibilities, rather than managing individual access through access policies that will soon be deprecated.
This Terraform code summarizes the concepts described above:

In this example, the Key Vault Administrator role has been assigned to our UAMI, but it is possible to replace it with the Key Vault Reader role for a more restrictive approach.

And be sure to specify the correct name of the service account and namespace that will be created.

Before creating our SA, obtain the clientId using the following command and ensure you replace it in sa.yml inside azure.workload.identity/client-id: "" : 

Then obtain the tenantId and replace it in secretprovider.yml.

Ensure you use the correct objectName, which is MYSQL-DB-PASSWORDD, as we have already created it with Terraform. You can also reference as many secrets as necessary.

Here is an example configuration for referencing the secret in your YAML file:

Types of ObjectType

The objectType attribute can mainly have three possible values:

• secret: Represents sensitive information, such as passwords, API keys, or tokens, that must remain confidential.
• key: Generally refers to cryptographic keys used to encrypt or sign data.
• cert: Refers to certificates used to authenticate identities on networks (such as SSL/TLS certificates).

Creating a Pod to Access Secrets Mounted from Azure Key Vault

Now that we have configured our SecretProviderClass and linked it to Azure Key Vault, it's time to create a pod that will use the secrets stored in Key Vault. This is where the magic happens, as we can leverage our Service Account to securely access these secrets.

Pod configuration
Here is the configuration for our pod:
kubectl apply -f pod-poc-vault.yaml

And finally, we can see our very strong password 😉:

You have successfully created an AKS cluster, configured virtual networks, and integrated Azure Key Vault for secure secret management. Your applications can now securely access secrets using the Secret Store CSI driver, making your Kubernetes deployment more secure and efficient.

Sofiene GHARBI
Cyber Engineer Consultant