Securing AI with Zero-Trust Networking in 2026

April 28, 2026April 28, 2026 techhadoop ai, azure ai, artificial-intelligence, azure, cloud, technology

For an enterprise-grade AI Assistant in 2026, networking and security are the “make-or-break” components. If you are using Terraform and Databricks, you must move away from standard public access and embrace Zero-Trust Networking.

Here is the blueprint for networking and security setup.

1. The Network Backbone: Hub-and-Spoke

To keep your data safe, do not deploy everything into one VNet. Use the Hub-and-Spoke model.

The Hub: Contains shared services like Azure Firewall, VPN Gateway (for on-prem access), and Centralized DNS Zones.
The AI Spoke: This is where your Databricks workspace, AI Search, and OpenAI live.
The Connection: All communication between your spoke and the internet must pass through the Hub’s firewall.

2. Private Link & Managed Identities (No Keys!)

In 2026, API keys are a legacy risk. Your architecture should be “Keyless.”

Private Endpoints: Disable all public network access for ADLS Gen2, AI Search, and OpenAI. Assign each a Private Endpoint within your Spoke VNet. This ensures your data never touches the public internet.
Managed Identities (System-Assigned):
- Give your Databricks Cluster a Managed Identity with Storage Blob Data Contributor on ADLS.
- Give your Azure OpenAI resource a Managed Identity to read from AI Search.
- The Result: No secrets to rotate in your Terraform code or Key Vault.

3. Databricks-Specific Security (The Terraform Focus)

The blog post you mentioned focuses on Terraform for Databricks. For high security, your Terraform must include:

VNet Injection: Do not use the “default” Databricks VNet. Inject Databricks into your own managed VNet with two subnets (public and private).
No Public IP (NPIP): Enable the “Secure Cluster Connectivity” feature. This ensures your Databricks worker nodes have zero public IP addresses, making them invisible to the internet.
Unity Catalog + Private Link: Ensure Unity Catalog is configured to use a Private Access Connector. This allows Databricks to talk to your Metadata store without leaving the Azure backbone.

4. Advanced Protection for RAG

Since this assistant handles sensitive internal data, add these two “2026-standard” layers:

Microsoft Purview Integration: Link your AI Search and OpenAI to Microsoft Purview. This allows you to apply Sensitivity Labels (e.g., “Highly Confidential”). If a document is tagged as such, the AI will refuse to summarize it for a user who doesn’t have that specific clearance.
AI Content Safety: Place an Azure AI Content Safety layer in front of OpenAI. This detects “Prompt Injection” attacks where a user might try to trick the AI into revealing system prompts or unauthorized data.

Summary Checklist for your Terraform Modules

Resource	Security Requirement
ADLS Gen2	Firewall enabled; Allow only “Selected Networks” (your VNet).
Databricks	`enable_no_public_ip = true` and VNet Injection enabled.
AI Search	`public_network_access_enabled = false`; Private Endpoint active.
OpenAI	Managed Identity enabled; `local_auth_enabled = false` (forces Entra ID).
DNS	Private DNS Zones for `privatelink.openai.azure.com` and `privatelink.blob.core.windows.net`.

Pro-Tip: In your Terraform, use the azapi provider if the standard azurerm provider doesn’t yet support the latest 2026 AI Search security features. This allows you to call the Azure Resource Manager API directly for cutting-edge settings.

Automate Velero AKS Backups with Terraform and Ansible

April 25, 2026April 25, 2026 techhadoop AKS, azure, Uncategorized ai, azure, cloud, kubernetes, technology

Automating Velero AKS Backup with Ansible & Terraform

Option 1: Terraform

Project Structure

			
velero-terraform/
├── main.tf
├── variables.tf
├── outputs.tf
├── providers.tf
└── modules/
    ├── storage/
    │   ├── main.tf
    │   └── variables.tf
    ├── identity/
    │   ├── main.tf
    │   └── variables.tf
    └── velero/
        ├── main.tf
        └── variables.tf

		

providers.tf

			
terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.0"
    }
    helm = {
      source  = "hashicorp/helm"
      version = "~> 2.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.0"
    }
  }
}
provider "azurerm" {
  features {}
}
provider "helm" {
  kubernetes {
    host                   = azurerm_kubernetes_cluster.aks.kube_config.0.host
    client_certificate     = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_certificate)
    client_key             = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_key)
    cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.cluster_ca_certificate)
  }
}
provider "kubernetes" {
  host                   = azurerm_kubernetes_cluster.aks.kube_config.0.host
  client_certificate     = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_certificate)
  client_key             = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_key)
  cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.cluster_ca_certificate)
}

		

variables.tf

			
variable "resource_group_name" {
  description = "Resource group name"
  type        = string
  default     = "myResourceGroup"
}
variable "location" {
  description = "Azure region"
  type        = string
  default     = "eastus"
}
variable "aks_cluster_name" {
  description = "AKS cluster name"
  type        = string
  default     = "myAKSCluster"
}
variable "storage_account_name" {
  description = "Storage account for Velero backups"
  type        = string
  default     = "velerobackupstorage"
}
variable "blob_container_name" {
  description = "Blob container for Velero backups"
  type        = string
  default     = "velero-backups"
}
variable "velero_namespace" {
  description = "Kubernetes namespace for Velero"
  type        = string
  default     = "velero"
}
variable "backup_retention_hours" {
  description = "Backup TTL in hours"
  type        = number
  default     = 720  # 30 days
}
variable "backup_schedule" {
  description = "Cron schedule for backups"
  type        = string
  default     = "0 2 * * *"  # Daily at 2am
}

		

main.tf

			
# Resource Group
resource "azurerm_resource_group" "rg" {
  name     = var.resource_group_name
  location = var.location
}
# ── STORAGE ──────────────────────────────────────────────
resource "azurerm_storage_account" "velero" {
  name                     = var.storage_account_name
  resource_group_name      = azurerm_resource_group.rg.name
  location                 = azurerm_resource_group.rg.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
  blob_properties {
    delete_retention_policy {
      days = 30
    }
  }
  tags = {
    purpose = "velero-backup"
  }
}
resource "azurerm_storage_container" "velero" {
  name                  = var.blob_container_name
  storage_account_name  = azurerm_storage_account.velero.name
  container_access_type = "private"
}
# ── IDENTITY ─────────────────────────────────────────────
resource "azurerm_user_assigned_identity" "velero" {
  name                = "velero-identity"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
}
# Assign Contributor role to storage account
resource "azurerm_role_assignment" "velero_storage" {
  scope                = azurerm_storage_account.velero.id
  role_definition_name = "Storage Blob Data Contributor"
  principal_id         = azurerm_user_assigned_identity.velero.principal_id
}
# Assign Contributor role to resource group (for disk snapshots)
resource "azurerm_role_assignment" "velero_rg" {
  scope                = azurerm_resource_group.rg.id
  role_definition_name = "Contributor"
  principal_id         = azurerm_user_assigned_identity.velero.principal_id
}
# ── AKS CLUSTER ──────────────────────────────────────────
resource "azurerm_kubernetes_cluster" "aks" {
  name                = var.aks_cluster_name
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  dns_prefix          = var.aks_cluster_name
  default_node_pool {
    name       = "default"
    node_count = 2
    vm_size    = "Standard_D2_v2"
  }
  identity {
    type = "SystemAssigned"
  }
  tags = {
    environment = "production"
  }
}
# ── VELERO NAMESPACE ─────────────────────────────────────
resource "kubernetes_namespace" "velero" {
  metadata {
    name = var.velero_namespace
  }
  depends_on = [azurerm_kubernetes_cluster.aks]
}
# ── VELERO CREDENTIALS SECRET ────────────────────────────
resource "kubernetes_secret" "velero_credentials" {
  metadata {
    name      = "velero-credentials"
    namespace = kubernetes_namespace.velero.metadata[0].name
  }
  data = {
    "cloud" = <<EOF
AZURE_SUBSCRIPTION_ID=${data.azurerm_subscription.current.subscription_id}
AZURE_TENANT_ID=${data.azurerm_subscription.current.tenant_id}
AZURE_CLIENT_ID=${azurerm_user_assigned_identity.velero.client_id}
AZURE_RESOURCE_GROUP=${azurerm_resource_group.rg.name}
AZURE_CLOUD_NAME=AzurePublicCloud
EOF
  }
}
# Current subscription data
data "azurerm_subscription" "current" {}
# ── VELERO HELM RELEASE ──────────────────────────────────
resource "helm_release" "velero" {
  name       = "velero"
  repository = "https://vmware-tanzu.github.io/helm-charts"
  chart      = "velero"
  namespace  = kubernetes_namespace.velero.metadata[0].name
  version    = "5.0.0"
  values = [
    yamlencode({
      configuration = {
        provider = "azure"
        backupStorageLocation = {
          name     = "default"
          provider = "velero.io/azure"
          bucket   = azurerm_storage_container.velero.name
          config = {
            resourceGroup      = azurerm_resource_group.rg.name
            storageAccount     = azurerm_storage_account.velero.name
            subscriptionId     = data.azurerm_subscription.current.subscription_id
          }
        }
        volumeSnapshotLocation = {
          name     = "default"
          provider = "velero.io/azure"
          config = {
            resourceGroup  = azurerm_resource_group.rg.name
            subscriptionId = data.azurerm_subscription.current.subscription_id
          }
        }
      }
      credentials = {
        existingSecret = kubernetes_secret.velero_credentials.metadata[0].name
      }
      initContainers = [
        {
          name            = "velero-plugin-for-azure"
          image           = "velero/velero-plugin-for-microsoft-azure:v1.8.0"
          imagePullPolicy = "IfNotPresent"
          volumeMounts = [
            {
              mountPath = "/target"
              name      = "plugins"
            }
          ]
        }
      ]
      schedules = {
        daily-backup = {
          schedule = var.backup_schedule
          template = {
            ttl                      = "${var.backup_retention_hours}h0m0s"
            includeClusterResources  = true
            excludedNamespaces       = ["kube-system", "velero"]
          }
        }
      }
    })
  ]
  depends_on = [
    kubernetes_secret.velero_credentials,
    azurerm_role_assignment.velero_storage,
    azurerm_role_assignment.velero_rg
  ]
}

		

outputs.tf

			
output "storage_account_name" {
  value = azurerm_storage_account.velero.name
}
output "blob_container_name" {
  value = azurerm_storage_container.velero.name
}
output "velero_identity_client_id" {
  value = azurerm_user_assigned_identity.velero.client_id
}
output "aks_cluster_name" {
  value = azurerm_kubernetes_cluster.aks.name
}
output "kube_config" {
  value     = azurerm_kubernetes_cluster.aks.kube_config_raw
  sensitive = true
}

		

Deploy with Terraform

			
# Initialize
terraform init
# Preview changes
terraform plan -out=tfplan
# Apply
terraform apply tfplan
# Get kubeconfig
terraform output -raw kube_config > ~/.kube/config
# Verify Velero
kubectl get pods -n velero
velero backup-location get

		

Option 2: Ansible

Project Structure

			
velero-ansible/
├── inventory/
│   └── hosts.yml
├── group_vars/
│   └── all.yml
├── roles/
│   ├── azure_storage/
│   │   └── tasks/
│   │       └── main.yml
│   ├── azure_identity/
│   │   └── tasks/
│   │       └── main.yml
│   └── velero/
│       ├── tasks/
│       │   └── main.yml
│       └── templates/
│           ├── credentials.j2
│           └── backup-schedule.yml.j2
└── site.yml

		

group_vars/all.yml

			
# Azure Settings
resource_group: "myResourceGroup"
location: "eastus"
aks_cluster_name: "myAKSCluster"
# Storage Settings
storage_account_name: "velerobackupstorage"
blob_container_name: "velero-backups"
# Velero Settings
velero_namespace: "velero"
velero_version: "v1.12.0"
velero_azure_plugin_version: "v1.8.0"
velero_chart_version: "5.0.0"
# Backup Settings
backup_schedule: "0 2 * * *"
backup_ttl: "720h"
backup_name: "daily-backup"
# Namespaces to exclude
excluded_namespaces:
  - kube-system
  - velero

		

roles/azure_storage/tasks/main.yml

			
---
- name: Create Resource Group
  azure.azcollection.azure_rm_resourcegroup:
    name: "{{ resource_group }}"
    location: "{{ location }}"
    state: present
- name: Create Storage Account
  azure.azcollection.azure_rm_storageaccount:
    resource_group: "{{ resource_group }}"
    name: "{{ storage_account_name }}"
    type: Standard_LRS
    kind: StorageV2
    state: present
  register: storage_account_result
- name: Create Blob Container
  azure.azcollection.azure_rm_storageblob:
    resource_group: "{{ resource_group }}"
    storage_account_name: "{{ storage_account_name }}"
    container: "{{ blob_container_name }}"
    state: present
- name: Get Storage Account Keys
  azure.azcollection.azure_rm_storageaccount_info:
    resource_group: "{{ resource_group }}"
    name: "{{ storage_account_name }}"
  register: storage_info
- name: Set Storage Key Fact
  set_fact:
    storage_account_key: "{{ storage_info.storageaccounts[0].primary_endpoints.key }}"

		

roles/azure_identity/tasks/main.yml

			
---
- name: Get Azure Subscription Info
  azure.azcollection.azure_rm_subscription_info:
  register: subscription_info
- name: Set Subscription Facts
  set_fact:
    subscription_id: "{{ subscription_info.subscriptions[0].subscription_id }}"
    tenant_id: "{{ subscription_info.subscriptions[0].tenant_id }}"
- name: Create Service Principal for Velero
  azure.azcollection.azure_rm_adserviceprincipal:
    app_id: "velero-sp"
    state: present
  register: sp_result
- name: Assign Contributor Role to Service Principal
  azure.azcollection.azure_rm_roleassignment:
    scope: "/subscriptions/{{ subscription_id }}/resourceGroups/{{ resource_group }}"
    assignee_object_id: "{{ sp_result.object_id }}"
    role_definition_name: Contributor
    state: present
- name: Assign Storage Blob Contributor Role
  azure.azcollection.azure_rm_roleassignment:
    scope: "/subscriptions/{{ subscription_id }}/resourceGroups/{{ resource_group }}/providers/Microsoft.Storage/storageAccounts/{{ storage_account_name }}"
    assignee_object_id: "{{ sp_result.object_id }}"
    role_definition_name: "Storage Blob Data Contributor"
    state: present

		

roles/velero/templates/credentials.j2

			
AZURE_SUBSCRIPTION_ID={{ subscription_id }}
AZURE_TENANT_ID={{ tenant_id }}
AZURE_CLIENT_ID={{ client_id }}
AZURE_CLIENT_SECRET={{ client_secret }}
AZURE_RESOURCE_GROUP={{ resource_group }}
AZURE_CLOUD_NAME=AzurePublicCloud

		

roles/velero/templates/backup-schedule.yml.j2

			
apiVersion: velero.io/v1
kind: Schedule
metadata:
  name: {{ backup_name }}
  namespace: {{ velero_namespace }}
spec:
  schedule: "{{ backup_schedule }}"
  template:
    ttl: "{{ backup_ttl }}"
    includeClusterResources: true
    excludedNamespaces:
{% for ns in excluded_namespaces %}
      - {{ ns }}
{% endfor %}

		

roles/velero/tasks/main.yml

			
---
- name: Create Velero Namespace
  kubernetes.core.k8s:
    name: "{{ velero_namespace }}"
    api_version: v1
    kind: Namespace
    state: present
- name: Create Velero Credentials File
  template:
    src: credentials.j2
    dest: /tmp/credentials-velero
    mode: '0600'
- name: Create Kubernetes Secret for Velero Credentials
  kubernetes.core.k8s:
    state: present
    definition:
      apiVersion: v1
      kind: Secret
      metadata:
        name: velero-credentials
        namespace: "{{ velero_namespace }}"
      stringData:
        cloud: |
          AZURE_SUBSCRIPTION_ID={{ subscription_id }}
          AZURE_TENANT_ID={{ tenant_id }}
          AZURE_CLIENT_ID={{ client_id }}
          AZURE_CLIENT_SECRET={{ client_secret }}
          AZURE_RESOURCE_GROUP={{ resource_group }}
          AZURE_CLOUD_NAME=AzurePublicCloud
- name: Add Velero Helm Repository
  kubernetes.core.helm_repository:
    name: vmware-tanzu
    repo_url: "https://vmware-tanzu.github.io/helm-charts"
    state: present
- name: Install Velero via Helm
  kubernetes.core.helm:
    name: velero
    chart_ref: vmware-tanzu/velero
    chart_version: "{{ velero_chart_version }}"
    namespace: "{{ velero_namespace }}"
    state: present
    values:
      configuration:
        provider: azure
        backupStorageLocation:
          name: default
          provider: velero.io/azure
          bucket: "{{ blob_container_name }}"
          config:
            resourceGroup: "{{ resource_group }}"
            storageAccount: "{{ storage_account_name }}"
            subscriptionId: "{{ subscription_id }}"
        volumeSnapshotLocation:
          name: default
          provider: velero.io/azure
          config:
            resourceGroup: "{{ resource_group }}"
            subscriptionId: "{{ subscription_id }}"
      credentials:
        existingSecret: velero-credentials
      initContainers:
        - name: velero-plugin-for-azure
          image: "velero/velero-plugin-for-microsoft-azure:{{ velero_azure_plugin_version }}"
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - mountPath: /target
              name: plugins
- name: Wait for Velero Pod to be Ready
  kubernetes.core.k8s_info:
    kind: Pod
    namespace: "{{ velero_namespace }}"
    label_selectors:
      - app.kubernetes.io/name=velero
  register: velero_pod
  until: velero_pod.resources[0].status.phase == "Running"
  retries: 10
  delay: 15
- name: Apply Backup Schedule
  kubernetes.core.k8s:
    state: present
    template: backup-schedule.yml.j2
- name: Verify Backup Location
  command: velero backup-location get
  register: backup_location_status
  changed_when: false
- name: Display Backup Location Status
  debug:
    msg: "{{ backup_location_status.stdout }}"
- name: Clean Up Credentials File
  file:
    path: /tmp/credentials-velero
    state: absent

		

site.yml (Main Playbook)

			
---
- name: Setup Velero Backup for AKS
  hosts: localhost
  connection: local
  gather_facts: false
  pre_tasks:
    - name: Verify required tools are installed
      command: "{{ item }} --version"
      loop:
        - az
        - kubectl
        - helm
      register: tool_check
      changed_when: false
    - name: Verify AKS context
      command: kubectl cluster-info
      register: cluster_info
      changed_when: false
    - name: Display cluster info
      debug:
        msg: "{{ cluster_info.stdout_lines[0] }}"
  roles:
    - azure_storage
    - azure_identity
    - velero
  post_tasks:
    - name: Trigger initial manual backup
      command: >
        velero backup create initial-backup
        --include-cluster-resources=true
        --wait
      register: initial_backup
      changed_when: true
    - name: Display backup result
      debug:
        msg: "{{ initial_backup.stdout }}"

		

Run the Ansible Playbook

			
# Install required collections
ansible-galaxy collection install azure.azcollection
ansible-galaxy collection install kubernetes.core
# Install Python dependencies
pip install ansible[azure] kubernetes
# Run the playbook
ansible-playbook site.yml -v
# Run specific role only
ansible-playbook site.yml --tags "velero" -v
# Dry run
ansible-playbook site.yml --check -v

		

Comparison: Terraform vs Ansible

Feature	Terraform	Ansible
Best For	Infrastructure provisioning	Configuration & app deployment
State Management	Yes (tfstate file)	No native state
Idempotency	Built-in	Task-level
Azure Resources	Excellent	Good
Kubernetes Resources	Good	Excellent
Learning Curve	Medium	Low
Rollback	Via state	Manual
Recommended Use	Create AKS + Storage	Install & configure Velero

Best Practice: Combine Both

			
Terraform  →  Creates Azure infrastructure (AKS, Storage, Identity)
    ↓
Ansible    →  Installs and configures Velero on the cluster
    ↓
Velero     →  Runs scheduled backups automatically

		

This gives you the best of both worlds — Terraform for infrastructure and Ansible for application configuration.

How to Backup AKS with Velero: A Step-by-Step Guide

April 25, 2026April 25, 2026 techhadoop AKS, azure, Uncategorized ai, azure, cloud, kubernetes, technology

Backing Up AKS with Velero

Prerequisites

AKS cluster running
Azure CLI installed
kubectl configured
Helm installed

Step 1: Create Azure Storage Account & Blob Container

			
# Set variables
RESOURCE_GROUP="myResourceGroup"
LOCATION="eastus"
STORAGE_ACCOUNT="velerobackupstorage"
BLOB_CONTAINER="velero-backups"
AKS_CLUSTER="myAKSCluster"
# Create resource group (if not exists)
az group create --name $RESOURCE_GROUP --location $LOCATION
# Create storage account
az storage account create \
  --name $STORAGE_ACCOUNT \
  --resource-group $RESOURCE_GROUP \
  --location $LOCATION \
  --sku Standard_LRS \
  --encryption-services blob
# Create blob container
az storage container create \
  --name $BLOB_CONTAINER \
  --account-name $STORAGE_ACCOUNT

		

Step 2: Create Service Principal for Velero

			
# Create service principal
AZURE_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
AZURE_TENANT_ID=$(az account show --query tenantId -o tsv)
# Create SP and assign contributor role
AZURE_CLIENT_SECRET=$(az ad sp create-for-rbac \
  --name velero-sp \
  --role Contributor \
  --scopes /subscriptions/$AZURE_SUBSCRIPTION_ID \
  --query password \
  -o tsv)
AZURE_CLIENT_ID=$(az ad sp list \
  --display-name velero-sp \
  --query '[0].appId' \
  -o tsv)
echo "Client ID: $AZURE_CLIENT_ID"
echo "Client Secret: $AZURE_CLIENT_SECRET"

		

Step 3: Create Velero Credentials File

			
# Create credentials file
cat << EOF > ./credentials-velero
AZURE_SUBSCRIPTION_ID=${AZURE_SUBSCRIPTION_ID}
AZURE_TENANT_ID=${AZURE_TENANT_ID}
AZURE_CLIENT_ID=${AZURE_CLIENT_ID}
AZURE_CLIENT_SECRET=${AZURE_CLIENT_SECRET}
AZURE_RESOURCE_GROUP=${RESOURCE_GROUP}
AZURE_CLOUD_NAME=AzurePublicCloud
EOF

		

Step 4: Install Velero on AKS

			
# Add Velero Helm repo
helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts
helm repo update
# Get storage account access key
STORAGE_ACCOUNT_KEY=$(az storage account keys list \
  --account-name $STORAGE_ACCOUNT \
  --resource-group $RESOURCE_GROUP \
  --query '[0].value' \
  -o tsv)
# Install Velero with Azure plugin
velero install \
  --provider azure \
  --plugins velero/velero-plugin-for-microsoft-azure:v1.8.0 \
  --bucket $BLOB_CONTAINER \
  --secret-file ./credentials-velero \
  --backup-location-config \
    resourceGroup=$RESOURCE_GROUP,\
    storageAccount=$STORAGE_ACCOUNT,\
    storageAccountKeyEnvVar=AZURE_STORAGE_ACCOUNT_ACCESS_KEY \
  --snapshot-location-config \
    resourceGroup=$RESOURCE_GROUP,\
    apiTimeout=5m \
  --use-volume-snapshots=true \
  --wait

		

Step 5: Verify Velero Installation

			
# Check Velero pods are running
kubectl get pods -n velero
# Check backup storage location is available
velero backup-location get
# Expected output:
# NAME      PROVIDER   BUCKET/PREFIX     PHASE       LAST VALIDATED
# default   azure      velero-backups    Available   ...

		

Step 6: Create Backups

Manual Backup (Full Cluster)

			
# Backup entire cluster
velero backup create full-cluster-backup \
  --include-cluster-resources=true \
  --wait
# Check backup status
velero backup describe full-cluster-backup
velero backup logs full-cluster-backup

		

Backup Specific Namespace

			
# Backup a single namespace
velero backup create my-app-backup \
  --include-namespaces my-app-namespace \
  --wait
# Backup multiple namespaces
velero backup create multi-ns-backup \
  --include-namespaces namespace1,namespace2 \
  --wait

		

Backup with Labels

			
# Backup resources matching a label
velero backup create label-backup \
  --selector app=my-application \
  --wait

Exclude Specific Resources

			
# Exclude secrets and specific namespaces
velero backup create selective-backup \
  --exclude-namespaces kube-system,velero \
  --exclude-resources secrets \
  --wait

		

Step 7: Schedule Automated Backups

			
# Daily backup at 2am UTC (keep 30 days)
velero schedule create daily-backup \
  --schedule="0 2 * * *" \
  --ttl 720h \
  --include-cluster-resources=true
# Weekly backup every Sunday at midnight (keep 90 days)
velero schedule create weekly-backup \
  --schedule="0 0 * * 0" \
  --ttl 2160h
# Namespace-specific daily backup
velero schedule create daily-app-backup \
  --schedule="0 2 * * *" \
  --include-namespaces production \
  --ttl 168h
# List all schedules
velero schedule get

		

Step 8: Restore from Backup

Full Restore

			
# List available backups
velero backup get
# Restore full cluster backup
velero restore create \
  --from-backup full-cluster-backup \
  --wait
# Check restore status
velero restore get
velero restore describe <restore-name>

		

Restore Specific Namespace

			
# Restore a specific namespace
velero restore create \
  --from-backup my-app-backup \
  --include-namespaces my-app-namespace \
  --wait

		

Restore to Different Namespace

			
# Restore namespace-A into namespace-B
velero restore create \
  --from-backup my-app-backup \
  --namespace-mappings old-namespace:new-namespace \
  --wait

		

Restore Specific Resources Only

			
# Restore only deployments and services
velero restore create \
  --from-backup full-cluster-backup \
  --include-resources deployments,services \
  --wait

		

Step 9: Backup Hooks (Pre/Post Actions)

Useful for database consistency before backup:

			
# deployment-with-backup-hooks.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
  annotations:
    pre.hook.backup.velero.io/command: '["/bin/bash", "-c", "mysqldump -u root -p$MYSQL_ROOT_PASSWORD mydb > /backup/dump.sql"]'
    pre.hook.backup.velero.io/timeout: "60s"
    post.hook.backup.velero.io/command: '["/bin/bash", "-c", "rm -f /backup/dump.sql"]'

		

kubectl apply -f deployment-with-backup-hooks.yaml

Step 10: Monitor & Troubleshoot

			
# View backup details
velero backup describe my-backup --details
# View backup logs
velero backup logs my-backup
# View restore logs
velero restore logs my-restore
# Check Velero server logs
kubectl logs deployment/velero -n velero
# List all backups with status
velero backup get
# Delete old backup manually
velero backup delete old-backup-name

		

Summary Cheat Sheet

Task	Command
Full backup	`velero backup create <name> --include-cluster-resources=true`
Namespace backup	`velero backup create <name> --include-namespaces <ns>`
Schedule daily	`velero schedule create <name> --schedule="0 2 * * *"`
List backups	`velero backup get`
Restore backup	`velero restore create --from-backup <name>`
Check status	`velero backup describe <name>`
View logs	`velero backup logs <name>`

Best Practices

Test restores regularly – don’t wait for disaster to find out your backup is broken
Use TTL to automatically expire old backups and save storage costs
Backup before upgrades – always take a snapshot before AKS version upgrades
Store credentials securely – use Azure Key Vault instead of plain credentials files
Use hooks for databases to ensure data consistency
Cross-region storage – store backups in a different Azure region for disaster recovery
Combine with GitOps – use Velero for stateful data + Git for manifests

Top Scenarios to Ace Your AKS Interviews

April 24, 2026May 13, 2026 techhadoop AKS, azure ai, azure, cloud, kubernetes, technology

These are the kinds of enterprise failure scenarios where interviewers test depth, not just commands. A real interview: scenario → pressure → how you respond → what impresses.

Scenario 1: Entire AKS Cluster Becomes Unreachable

Situation

Apps are down
kubectl not responding
API server unreachable

What’s actually happening?

In Azure Kubernetes Service, the control plane is managed by Azure—so this is usually:

Network isolation issue (private cluster)
DNS issue
Azure-side outage (rare)

How to respond (structured)

1. Validate scope

Is it just you or everyone?
Can CI/CD still deploy?

2. Check cluster type

Private cluster?
API server behind Private Endpoint?

3. DNS resolution

Does API server FQDN resolve?

4. Network path

VPN / ExpressRoute working?
NSG blocking?

5. Azure health

Region outage?

Strong answer

“Since AKS control plane is managed, I’d immediately suspect network or DNS issues—especially in private clusters. I’d validate API server resolution, connectivity path, and Azure service health.”

Scenario 2: Production Outage After Deployment

Situation

New release deployed
All pods running
App returning 500 errors

Key Insight

This is NOT infrastructure—it’s application or config.

Approach

1. Rollback immediately

kubectl rollout undo deployment <app>

2. Compare versions

Env vars changed?
Secrets updated?

3. Check logs

App-level errors

4. Validate dependencies

DB reachable?
API endpoints correct?

Strong answer

“If pods are healthy but app fails, I treat it as an application issue. I’d rollback first to restore service, then investigate config drift or dependency failures.”

Scenario 3: Intermittent Failures Across Services

Situation

Random timeouts
Some requests succeed, others fail

Think: networking or scaling

Likely Causes

SNAT port exhaustion
DNS latency
Pod autoscaling delays
Node pressure

What you check

1. Node metrics

CPU/memory spikes

2. Pod distribution

Are pods unevenly spread?

3. Networking limits

Outbound connections?

Strong answer

“Intermittent failures usually point to resource contention or networking limits like SNAT exhaustion. I’d correlate metrics with traffic patterns.”

Scenario 4: AKS Can’t Pull Images from ACR

Situation

Pods stuck in ImagePullBackOff

Root cause area

Identity / permissions

Common causes

AKS not authorized to Azure Container Registry
Managed identity missing role

Fix

Assign AcrPull role to AKS identity

Strong answer

“This is typically a managed identity RBAC issue. I’d verify the cluster identity has AcrPull access to the registry.”

Scenario 5: Traffic Not Routing in Private AKS

Situation

Internal services work
External users can’t reach app

Think enterprise networking

Check:

1. Ingress controller

Running?

2. Azure Load Balancer / App Gateway

3. NSG rules

Ports open?

4. DNS

Internal vs external resolution

Strong answer

“In private AKS, exposure depends on controlled ingress. I’d trace traffic from DNS to ingress to service, checking NSGs and routing.”

Scenario 6: Cluster Autoscaler Not Working

Situation

Pods stuck in Pending
Nodes not scaling

Key checks

Autoscaler enabled?
Max node limit reached?
Subnet IP exhausted?

Strong answer

“Autoscaling failures often tie back to limits—either max node count or subnet capacity in Azure CNI setups.”

Scenario 7: Security Breach Suspicion

Situation

Unexpected outbound traffic
Suspicious container behavior

What matters

Containment + investigation

Actions

1. Isolate

Scale down or cordon node

2. Inspect

Container logs
Image source

3. Check runtime security

Alerts from Microsoft Defender for Cloud

Strong answer

“I’d isolate affected workloads immediately, then investigate logs and image provenance while leveraging Defender for alerts.”

Scenario 8: Multi-Region Failover Fails

Situation

Primary region down
Traffic not failing over

Root causes

DNS not switching
Traffic manager misconfigured
Backend unhealthy

Strong answer

“I’d validate DNS failover mechanism (Traffic Manager/Front Door), then confirm secondary cluster health and readiness.”

ENTERPRISE TROUBLESHOOTING MINDSET

Always think in layers:

1. Application

Logs, config, dependencies

2. Kubernetes

Pods, services, scheduling

3. Node / Compute

VMSS health

4. Networking

VNet, NSG, DNS

5. Azure Platform

Identity, RBAC, outages

What Interviewers REALLY Want

They’re looking for:

Structured thinking
Fast isolation of failure domain
Awareness of Azure-specific constraints
Calm, rollback-first mindset

If you want to go even deeper:

Feature	Kubenet	Azure CNI
IP assignment	NAT	Real VNet IP
Scalability	Better	Limited by subnet
Complexity	Lower	Higher
Use case	Small clusters	Enterprise

Feature	Disk	File
Access	Single pod	Multiple pods
Performance	High	Moderate

Unified Observability for AKS in 2026

April 24, 2026April 24, 2026 techhadoop AKS, azure ai, artificial-intelligence, azure, cloud, technology

Monitoring AKS in 2026 has moved beyond simple “health checks.” It now centers on a Unified Observability strategy that combines infrastructure metrics, high-cardinality logs, and application traces.

To provide top-tier support, you should propose a two-pronged approach: Azure Native for the platform and Managed Prometheus/Grafana for the microservices.

1. The 2026 Monitoring Stack (The “Big Three”)

Tool	Purpose	What to Propose
Container Insights	Platform Health. Focuses on Inventory, Node CPU/Memory, and K8s Events.	“We’ll use this for the ‘Golden Signals’ of the cluster infrastructure.”
Managed Prometheus	Workload Metrics. High-resolution scraping of your microservices (Pod-level).	“This gives us deep visibility into app-specific metrics like ‘Orders Processed’.”
Managed Grafana	Visualization. The single pane of glass for both Azure and Prometheus data.	“I’ll build executive dashboards to show uptime and developer dashboards for debugging.”

2. Advanced Log Management (Cost & Performance)

Log ingestion is usually the most expensive part of AKS support. As of 2026, Microsoft has introduced “Basic Logs” to help.

Log Analytics (ContainerLogV2): Propose migrating to the ContainerLogV2 schema. It is faster and supports Basic Logs, which can reduce costs by up to 50-70% for high-volume logs you don’t query often.
Diagnostic Settings: Enable these to capture Control Plane logs (API Server, Scheduler). Without these, you are blind to why a cluster upgrade failed or who deleted a namespace.

3. Network Observability (The 2026 Add-on)

Microsoft recently standardized the Network Observability add-on. This is a great “upsell” for security-conscious clients.

What it does: It tracks “East-West” traffic (pod-to-pod).
The Pitch: “If Service A can’t talk to Service B, I can tell you in 30 seconds if it’s a network drop, a DNS failure, or a security policy block.”

4. Proactive Alerting Strategy

Don’t just alert on “CPU > 80%.” That leads to alert fatigue. Instead, propose Service Level Objective (SLO) based alerting:

Latency Alerts: “Alert if 5% of requests take longer than 2 seconds.”
OOMKill Detection: “Alert if any pod in the production namespace is killed due to memory limits.”
Disk Pressure: “Alert if node local storage is at 85% to prevent the ‘DiskPressure’ taint from evicting pods.”

The “SRE” Proposal Snippet

If you want to present this to your company, try this:

“To ensure 99.9% availability for our microservices, I propose implementing the Azure Managed Observability Stack. By moving our logs to the V2 schema and implementing Managed Prometheus, we can reduce our monitoring costs by roughly 30% while gaining the ability to trace transactions across our entire microservice mesh. This transforms our support from ‘fixing breaks’ to ‘preventing downtime’.”

If you are only looking at logs, you are seeing what happened (the “post-mortem”), but you are missing where the bottleneck is and how the services interact.

To take your support to the next level, you should propose moving toward Distributed Tracing. This is the “holy grail” of microservices support.

1. The Missing Piece: Distributed Tracing

When a user says “the app is slow,” logs usually show a bunch of successful 200 OK messages across five different services. You have no way of knowing which of those five services added the 3-second delay.

The Solution: OpenTelemetry (OTel)

By implementing OpenTelemetry (the industry standard in 2026), you give every request a “Trace ID.”

Trace: The entire journey of a request from the user’s click to the database and back.
Span: The time spent inside a single microservice or database call.

2. Azure Application Insights (The Easy Win)

Since you are on AKS, the fastest way to get tracing is Application Insights.

Application Map: This is a live, auto-generated visual of your entire microservice architecture. It shows which services are talking to which, and highlights red links where errors or high latency are occurring.
No-Code Instrumentation: For many Linux/Docker apps (especially .NET, Java, and Python), you can enable tracing without changing a single line of code by using the Azure Monitor OpenTelemetry Distro or a “Sidecar” container.

3. How to Propose “Distributed Tracing” as a Service

This is a major value-add. Frame it as “Reducing Mean Time to Recovery (MTTR).”

The Pitch:

“Currently, when a performance issue occurs, we have to manually comb through logs across multiple containers to find the root cause. I propose implementing Distributed Tracing. This will give us a visual ‘Application Map’ of our microservices, allowing us to pinpoint exactly which service or database query is slowing down the system in seconds rather than hours.”

4. Practical Implementation Plan

If they say “Yes,” here is your 3-step rollout:

Infrastructure: Add the Application Insights resource via your Terraform code.
Instrumentation: Add an OpenTelemetry “Sidecar” to your Docker deployments (or use the Azure Monitor AKS add-on).
Dashboarding: Create a “Latency Heatmap” in Azure Managed Grafana that pulls data from App Insights.

The “SRE” Comparison

To show your client you know your stuff, use this comparison:

Logs: Tell you the “What” (e.g., “Database connection failed”).
Metrics: Tell you the “When” (e.g., “CPU spiked at 3 PM”).
Tracing: Tells you the “Where” (e.g., “The delay is happening specifically in the Authentication Service’s call to the User DB”).

To move from basic logs to full distributed tracing, we need to add Application Insights and a Log Analytics Workspace to your Terraform configuration.

By 2026, the standard is to use Workspace-based Application Insights, which stores all its data in a centralized Log Analytics workspace. This makes it easier to query both your system logs and your application traces in one place.

1. The Terraform Code (Infrastructure)

Add this to your Terraform files. This creates the monitoring “bucket” and the tracing “engine.”

Terraform

			
# 1. Create the Log Analytics Workspace (The storage)
resource "azurerm_log_analytics_workspace" "aks_monitor" {
  name                = "law-aks-prod-01"
  location            = azurerm_resource_group.aks_rg.location
  resource_group_name = azurerm_resource_group.aks_rg.name
  sku                 = "PerGB2018"
  retention_in_days   = 30
}
# 2. Create Application Insights (The tracing engine)
resource "azurerm_application_insights" "aks_app_insights" {
  name                = "ai-microservices-prod"
  location            = azurerm_resource_group.aks_rg.location
  resource_group_name = azurerm_resource_group.aks_rg.name
  workspace_id        = azurerm_log_analytics_workspace.aks_monitor.id
  application_type    = "web"
}
# 3. Output the Connection String (Your apps need this to send data)
output "app_insights_connection_string" {
  value     = azurerm_application_insights.aks_app_insights.connection_string
  sensitive = true
}

		

2. The “Last Mile”: Connecting your Docker Apps

Once the infrastructure is built, your microservices need to know where to send their traces. In 2026, we use Autoinstrumentation so you don’t have to change your code.

Step A: Enable the AKS Monitoring Add-on

In your azurerm_kubernetes_cluster resource, add this block:

Terraform

  oms_agent {
    log_analytics_workspace_id      = azurerm_log_analytics_workspace.aks_monitor.id
    msi_auth_for_monitoring_enabled = true
  }

Step B: Inject the Connection String into your Pods

In your Kubernetes deployment YAML, add the Connection String as an environment variable (best practice: pull this from the Azure Key Vault we set up earlier):

YAML

			
env:
  - name: APPLICATIONINSIGHTS_CONNECTION_STRING
    value: "InstrumentationKey=xxxx-xxxx-xxxx;IngestionEndpoint=..."

3. What this looks like for the Client

Once this is running, you can show the client two powerful views in the Azure Portal:

The Application Map: A live, visual diagram showing how Service A calls Service B. If a service turns red, it means it’s failing. If the line is thick/slow, it’s a bottleneck.
End-to-End Transaction Details: You can click on a single failed request and see exactly where it died—whether it was a code error in the container or a slow query in the database.

The Proposal Pitch

“Currently, we have ‘blind spots’ between our services. By implementing this Terraform-backed tracing, we can move from reactive log-hunting to Visual Troubleshooting. We’ll be able to see exactly how requests flow through our Docker containers, allowing us to fix performance issues before they impact the end-user.”

Understanding Azure RBAC vs Kubernetes RBAC

April 24, 2026April 24, 2026 techhadoop AKS, azure azure, cloud, devops, kubernetes, technology

When explaining this to a client, it is helpful to describe it as the difference between who can touch the physical server (the building) versus who can edit the files on the computer inside (the office).

In 2026, the industry standard is to use Azure RBAC for both, but they still operate on two distinct “control planes.”

1. The Two Control Planes

In AKS, access is split into two layers:

The Azure Control Plane (Azure RBAC): This governs the “outside” of the cluster. It’s about the Kubernetes resource itself as it exists in your Azure portal.
The Kubernetes Control Plane (Kubernetes RBAC): This governs the “inside” of the cluster. It’s about the pods, namespaces, and deployments running on the nodes.

2. Side-by-Side Comparison

Feature	Azure RBAC	Kubernetes RBAC
Scope	Subscription / Resource Group / AKS Resource	Cluster / Namespace / Specific Pods
Managed Via	Azure Portal, CLI, Terraform	`kubectl`, YAML manifests, Helm
Typical Actions	Scaling nodes, Upgrading K8s version, Deleting the cluster.	Creating a Pod, Editing a Service, Viewing logs in a Namespace.
Identity Source	Microsoft Entra ID (Azure AD)	Service Accounts (or Entra ID via integration)

3. The “Hybrid” Option (Azure RBAC for K8s Authorization)

This is the most confusing part for beginners, but the most important for you to propose to your client.

You can now use Azure RBAC to manage internal Kubernetes permissions. Instead of writing complex RoleBinding YAML files for every user, you assign them a built-in Azure role that Kubernetes understands.

Key Built-in Roles (2026 Standards):

AKS RBAC Viewer: Can see resources in a namespace but can’t see secrets or change anything.
AKS RBAC Writer: Can deploy apps and edit resources.
AKS RBAC Admin: Full control over a namespace.
AKS RBAC Cluster Admin: The “God Mode” for the entire cluster.

4. How to Explain the Workflow to Your Manager

“Think of it like a bank:

Azure RBAC is the security guard at the front door. He checks your ID (Entra ID) and decides if you’re even allowed in the building. He also decides who can add more teller windows (Scale nodes) or renovate the lobby (Upgrade cluster).
Kubernetes RBAC is the permissions on the safe. Once you’re inside, it decides if you can open Drawer A (Namespace ‘Dev’) or Drawer B (Namespace ‘Prod’).”

Pro-Tip: Recommendation

If you want to provide “Gold Standard” support, propose disabling local accounts and moving entirely to Azure RBAC for Kubernetes Authorization. > Why? Because when an employee leaves the company and their Entra ID (Azure AD) is deleted, their access to the Kubernetes cluster is instantly revoked. No orphaned RoleBindings to worry about.

Understanding how a developer goes from their laptop to a running container in a secured AKS environment is the best way to prove the value of your setup.

Here is the step-by-step lifecycle of a developer’s request in a Zero-Trust AKS environment.

The Access Lifecycle (Step-by-Step)

1. Authentication (The Gatekeeper)

The developer doesn’t have a “Kubernetes password.” Instead, they run:

Bash

			
az login
az aks get-credentials --resource-group rg-prod --name aks-01

At this moment, Azure RBAC checks if their Entra ID account has permission to even download the cluster configuration.

2. Authorization (The Office Door)

The developer tries to deploy a new microservice:

Bash

kubectl apply -f my-app.yaml

The AKS API Server intercepts this. Since we are using Azure RBAC for Kubernetes Authorization, it asks Entra ID: “Does this user have the ‘AKS RBAC Writer’ role for the ‘Production’ namespace?” * If Yes: The request proceeds.

If No: The request is blocked with a 403 Forbidden error.

3. Policy Validation (The Safety Inspector)

Before the pod is actually scheduled, Azure Policy (the Admission Controller) scans the my-app.yaml.

It checks: “Is this container trying to run as root? Does it have CPU limits?” * If the YAML is “lazy” (insecure), Azure Policy rejects it immediately, even though the developer has “Writer” permissions.

4. Identity & Secrets (The Secure Handshake)

Once the pod starts, it needs to talk to the database.

The pod presents its Workload Identity (a managed identity) to the Azure Key Vault.
Key Vault verifies the pod’s identity and hands over the database string via the CSI Driver.
The password is never stored in a file or an environment variable where a human could see it.

Summary Table for Your Proposal

To wrap this up for your client, you can present this “Success Path” to show them exactly what they are paying for:

Stage	Security Layer	Purpose
Login	Entra ID	Ensures only active employees can connect.
Action	Azure RBAC	Limits what a developer can do (e.g., Read vs. Write).
Deploy	Azure Policy	Forces best practices (No root, resource limits).
Connect	Workload Identity	Eliminates hardcoded passwords in the code.

Pro-Tip: The “Audit” Hook

Tell your client: “With this setup, we can generate a report at any time showing exactly who accessed the production cluster and what they changed. This makes SOC2 or ISO27001 audits a breeze.”

Understanding Azure RBAC vs Kubernetes RBAC

April 24, 2026April 24, 2026 techhadoop AKS, azure azure, cloud, devops, kubernetes, technology

In 2026, the industry standard is to use Azure RBAC for both, but they still operate on two distinct “control planes.”

1. The Two Control Planes

In AKS, access is split into two layers:

The Azure Control Plane (Azure RBAC): This governs the “outside” of the cluster. It’s about the Kubernetes resource itself as it exists in your Azure portal.
The Kubernetes Control Plane (Kubernetes RBAC): This governs the “inside” of the cluster. It’s about the pods, namespaces, and deployments running on the nodes.

2. Side-by-Side Comparison

Feature	Azure RBAC	Kubernetes RBAC
Scope	Subscription / Resource Group / AKS Resource	Cluster / Namespace / Specific Pods
Managed Via	Azure Portal, CLI, Terraform	`kubectl`, YAML manifests, Helm
Typical Actions	Scaling nodes, Upgrading K8s version, Deleting the cluster.	Creating a Pod, Editing a Service, Viewing logs in a Namespace.
Identity Source	Microsoft Entra ID (Azure AD)	Service Accounts (or Entra ID via integration)

3. The “Hybrid” Option (Azure RBAC for K8s Authorization)

This is the most confusing part for beginners, but the most important for you to propose to your client.

Key Built-in Roles (2026 Standards):

AKS RBAC Viewer: Can see resources in a namespace but can’t see secrets or change anything.
AKS RBAC Writer: Can deploy apps and edit resources.
AKS RBAC Admin: Full control over a namespace.
AKS RBAC Cluster Admin: The “God Mode” for the entire cluster.

4. How to Explain the Workflow to Your Manager

“Think of it like a bank:

Azure RBAC is the security guard at the front door. He checks your ID (Entra ID) and decides if you’re even allowed in the building. He also decides who can add more teller windows (Scale nodes) or renovate the lobby (Upgrade cluster).
Kubernetes RBAC is the permissions on the safe. Once you’re inside, it decides if you can open Drawer A (Namespace ‘Dev’) or Drawer B (Namespace ‘Prod’).”

Pro-Tip: Recommendation

Understanding how a developer goes from their laptop to a running container in a secured AKS environment is the best way to prove the value of your setup.

Here is the step-by-step lifecycle of a developer’s request in a Zero-Trust AKS environment.

The Access Lifecycle (Step-by-Step)

1. Authentication (The Gatekeeper)

The developer doesn’t have a “Kubernetes password.” Instead, they run:

Bash

			
az login
az aks get-credentials --resource-group rg-prod --name aks-01

At this moment, Azure RBAC checks if their Entra ID account has permission to even download the cluster configuration.

2. Authorization (The Office Door)

The developer tries to deploy a new microservice:

Bash

kubectl apply -f my-app.yaml

If No: The request is blocked with a 403 Forbidden error.

3. Policy Validation (The Safety Inspector)

Before the pod is actually scheduled, Azure Policy (the Admission Controller) scans the my-app.yaml.

It checks: “Is this container trying to run as root? Does it have CPU limits?” * If the YAML is “lazy” (insecure), Azure Policy rejects it immediately, even though the developer has “Writer” permissions.

4. Identity & Secrets (The Secure Handshake)

Once the pod starts, it needs to talk to the database.

The pod presents its Workload Identity (a managed identity) to the Azure Key Vault.
Key Vault verifies the pod’s identity and hands over the database string via the CSI Driver.
The password is never stored in a file or an environment variable where a human could see it.

Summary Table for Your Proposal

To wrap this up for your client, you can present this “Success Path” to show them exactly what they are paying for:

Stage	Security Layer	Purpose
Login	Entra ID	Ensures only active employees can connect.
Action	Azure RBAC	Limits what a developer can do (e.g., Read vs. Write).
Deploy	Azure Policy	Forces best practices (No root, resource limits).
Connect	Workload Identity	Eliminates hardcoded passwords in the code.

Pro-Tip: The “Audit” Hook

Tell your client: “With this setup, we can generate a report at any time showing exactly who accessed the production cluster and what they changed. This makes SOC2 or ISO27001 audits a breeze.”

Deploy AKS Clusters with Terraform: Best Practices

April 24, 2026April 24, 2026 techhadoop AKS, azure ai, azure, cloud, devops, technology

To deploy a production-ready AKS cluster using Terraform, it is best practice to separate your Network (VNet/Subnet) from the AKS Cluster resource. This ensures that if you ever need to destroy the cluster, your networking infrastructure remains intact.

Here is a clean, modular example using the AzureRM provider.

1. The Provider Configuration

First, create a main.tf to define your requirements.

Terraform

			
terraform {
  required_version = ">= 1.5.0"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.0" # Or 4.x if using the latest 2026 releases
    }
  }
}
provider "azurerm" {
  features {}
}

		

2. Networking Resources

AKS needs a dedicated subnet. We’ll use Azure CNI (Advanced Networking) as it’s the standard for enterprise security.

Terraform

			
resource "azurerm_resource_group" "aks_rg" {
  name     = "rg-production-aks"
  location = "East US"
}
resource "azurerm_virtual_network" "aks_vnet" {
  name                = "vnet-aks-prod"
  location            = azurerm_resource_group.aks_rg.location
  resource_group_name = azurerm_resource_group.aks_rg.name
  address_space       = ["10.0.0.0/16"]
}
resource "azurerm_subnet" "aks_subnet" {
  name                 = "snet-aks-nodes"
  resource_group_name  = azurerm_resource_group.aks_rg.name
  virtual_network_name = azurerm_virtual_network.aks_vnet.name
  address_prefixes     = ["10.0.1.0/24"]
}

		

3. The AKS Cluster Resource

This block includes the security features we discussed: System Assigned Identity, Azure RBAC, and Azure Linux as the OS.

Terraform

			
resource "azurerm_kubernetes_cluster" "aks" {
  name                = "aks-prod-01"
  location            = azurerm_resource_group.aks_rg.location
  resource_group_name = azurerm_resource_group.aks_rg.name
  dns_prefix          = "aksprod"
  
  # Enable Azure RBAC for Kubernetes
  azure_policy_enabled = true
  local_account_disabled = true
  default_node_pool {
    name       = "systempool"
    node_count = 3
    vm_size    = "Standard_DS2_v2"
    vnet_subnet_id = azurerm_subnet.aks_subnet.id
    
    # Use Azure Linux for better security/performance
    os_sku = "AzureLinux" 
    
    # Enable auto-scaling for production
    enable_auto_scaling = true
    min_count           = 3
    max_count           = 5
  }
  identity {
    type = "SystemAssigned"
  }
  network_profile {
    network_plugin    = "azure"
    load_balancer_sku = "standard"
    network_policy    = "azure" # Enables Kubernetes Network Policies
  }
  tags = {
    Environment = "Production"
    ManagedBy   = "Terraform"
  }
}

		

4. Essential Outputs

You’ll need the cluster configuration to connect via kubectl.

Terraform

			
output "client_certificate" {
  value     = azurerm_kubernetes_cluster.aks.kube_config.0.client_certificate
  sensitive = true
}
output "kube_config" {
  value     = azurerm_kubernetes_cluster.aks.kube_config_raw
  sensitive = true
}

		

Key Implementation Steps

Initialize: Run terraform init to download the Azure provider.
Plan: Run terraform plan -out=main.tfplan to preview the 4 resources being created.
Apply: Run terraform apply "main.tfplan".
Connect: Once finished, use the Azure CLI to get your credentials:Bashaz aks get-credentials --resource-group rg-production-aks --name aks-prod-01

Why this is a “Support Pro” Move

By delivering this in Terraform, you are telling the company: “I don’t just click buttons in the portal. I provide Infrastructure as Code that is version-controlled, repeatable, and documented.” This makes it much easier to propose a “Disaster Recovery” service later on.

Integrating the Azure Key Vault (AKV) Secrets Store CSI Driver into your Terraform code is the final step in removing sensitive data (like database passwords or API keys) from your Kubernetes manifests.

Here is the additional code to enable the driver and set up the necessary permissions.

1. Enable the CSI Driver in AKS

In your azurerm_kubernetes_cluster resource block (from the previous code), you need to add the key_vault_secrets_provider block:

Terraform

			
resource "azurerm_kubernetes_cluster" "aks" {
  # ... existing config ...
  key_vault_secrets_provider {
    secret_rotation_enabled = true
    secret_rotation_interval = "2m"
  }
}

		

2. Create the Key Vault

You need a vault to actually store the secrets.

Terraform

			
resource "azurerm_key_vault" "kv" {
  name                        = "kv-prod-aks-01"
  location                    = azurerm_resource_group.aks_rg.location
  resource_group_name         = azurerm_resource_group.aks_rg.name
  enabled_for_disk_encryption = true
  tenant_id                   = data.azurerm_client_config.current.tenant_id
  sku_name                    = "standard"
  # Best practice: Don't use access policies, use RBAC
  enable_rbac_authorization = true 
}
data "azurerm_client_config" "current" {}

		

3. Link AKS to Key Vault (The “Magic” Link)

When you enable the CSI driver, AKS creates a “Secret Provider Class” identity. You must give that identity permission to read from the Key Vault.

Terraform

			
# Identify the Managed Identity created by the AKS CSI Driver
resource "azurerm_role_assignment" "aks_kv_reader" {
  scope                = azurerm_key_vault.kv.id
  role_definition_name = "Key Vault Secrets User"
  principal_id         = azurerm_kubernetes_cluster.aks.key_vault_secrets_provider[0].secret_identity[0].object_id
}

		

4. Usage: The SecretProviderClass (K8s Manifest)

Terraform sets up the infrastructure, but you still need a small Kubernetes object to tell the pod which secrets to pull. You can apply this via kubectl or a Terraform kubernetes_manifest resource:

YAML

			
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: azure-kv-provider
  namespace: production
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "<AKS_CSI_CLIENT_ID>" # Output this from Terraform
    keyvaultName: "kv-prod-aks-01"
    objects:  |
      array:
        - |
          objectName: db-password
          objectType: secret
    tenantId: "<YOUR_TENANT_ID>"

		

Why this is a “Gold Standard” Setup

By using this approach, your Linux servers and Docker microservices become significantly more secure:

No “Cleartext” Secrets: Developers never see the production password.
Auto-Rotation: If you change the password in the Azure Portal/Key Vault, the CSI driver automatically updates the file inside the running Docker container within 2 minutes.
Audit Trail: Every time a pod accesses a secret, it’s logged in Azure Monitor.

Pro-Tip for your Proposal

When talking to the client, use this phrasing:

“I am implementing a Zero-Trust Secret Architecture. This ensures that sensitive credentials never touch our code repository or container images, and they are rotated automatically to prevent long-term credential leakage.”

Securing Azure Kubernetes Service: A 2026 Guide

April 23, 2026April 24, 2026 techhadoop AKS, azure, Uncategorized ai, azure, cloud, kubernetes, technology

Securing Azure Kubernetes Service (AKS) involves a Shared Responsibility Model: Microsoft manages the control plane (API server, etcd), while you are responsible for securing the worker nodes, networking, and the workloads themselves.

As of 2026, here are the non-negotiable security best practices for AKS.

1. Identity and Access Management (IAM)

Authentication is your first line of defense.

Microsoft Entra ID (Azure AD) Integration: Never use local Kubernetes accounts. Integrate AKS with Entra ID to manage cluster access via existing corporate identities and groups.
Azure RBAC for Kubernetes: Use Azure Role-Based Access Control to provide granular permissions (e.g., Azure Kubernetes Service RBAC Reader).
Workload Identity: Avoid using “Secret” objects for Azure credentials. Use Entra Workload ID, which allows pods to authenticate to Azure services (like Key Vault or Storage) using managed identities instead of long-lived passwords.

2. Network Security

Don’t let your cluster be a “sitting duck” on the public internet.

Private Clusters: Deploy AKS as a Private Cluster. This ensures that the API server is only accessible via a private IP within your Virtual Network (VNet), completely removing it from the public internet.
Authorized IP Ranges: If you must have a public API server, strictly limit access to specific CIDR ranges (e.g., your office or VPN IP).
Default-Deny Network Policies: By default, all pods in Kubernetes can talk to each other. Implement Azure Network Policies or Calico to enforce a “Default Deny” posture, only allowing explicitly permitted traffic.

3. Host and Node Security

The underlying VMs (nodes) are often the weakest link.

Azure Linux (CBL-Mariner): Use Azure Linux as your Node OS. It is a lightweight, security-hardened distribution maintained by Microsoft specifically for AKS.Critical Update: Support for Azure Linux 2.0 ended in late 2025. Ensure you are on Azure Linux 3.0 or higher to receive security patches.
Automatic Upgrades: Enable the Auto-upgrade channel (e.g., stable or node-image) to ensure your nodes automatically receive OS security patches and Kubernetes version updates.
Disable Public IPs for Nodes: Ensure worker nodes do not have public IP addresses; use an Azure Load Balancer or NAT Gateway for egress.

4. Workload and Secret Management

How you run your code determines your “blast radius” during an attack.

Secrets Store CSI Driver: Do not store secrets in standard Kubernetes YAML files. Use the Azure Key Vault Provider for Secrets Store CSI Driver to mount secrets directly from Key Vault into your pods as volumes.
Pod Security Standards: Use Azure Policy for Kubernetes to enforce the “Restricted” pod security standard. This prevents:
- Containers running as root.
- Privileged escalation.
- Writing to the root filesystem (use read-only filesystems instead).
Resource Limits: Always define CPU and Memory limits. This prevents a single compromised or “runaway” container from causing a Denial of Service (DoS) for the entire node.

5. Continuous Monitoring and Defense

You cannot protect what you cannot see.

Microsoft Defender for Containers: Enable this for real-time threat detection. It scans images for vulnerabilities in the registry and monitors running containers for suspicious behavior (e.g., unexpected shell execution).
Image Scanning in CI/CD: Use tools like Trivy or Microsoft Defender to scan images before they are pushed to the Azure Container Registry (ACR).
Audit Logging: Stream Kubernetes Audit Logs to a Log Analytics Workspace. This provides a paper trail of who did what in your cluster.

AKS Security Checklist (Quick Reference)

Category	High Priority Task
Access	Disable local accounts; use Entra ID integration.
API Server	Use a Private Cluster or Authorized IP ranges.
OS	Migrate to Azure Linux 3.0; enable auto-patches.
Secrets	Use Azure Key Vault via CSI Driver (no YAML secrets).
Traffic	Implement “Default Deny” Network Policies.
Policy	Apply Azure Policy to block non-compliant deployments.

To get you started, I’ve provided a Network Policy to secure traffic and an Azure Policy to enforce security rules at the cluster level.

1. Default Deny Network Policy

By default, Kubernetes allows all pods to talk to each other. This is a major security risk. The following policy creates a “Default Deny” posture for a specific namespace, meaning you must explicitly “whitelist” any traffic you want to allow.

YAML

			
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: default-deny-all
  namespace: production # Apply to your specific namespace
spec:
  podSelector: {} # Selects all pods in the namespace
  policyTypes:
  - Ingress
  - Egress

		

2. Azure Policy for Kubernetes (Enforcement)

While Network Policies handle traffic, Azure Policy ensures that developers don’t deploy “insecure” containers in the first place. You can apply the “Kubernetes cluster pod security restricted standards for Linux-based workloads” initiative directly from the Azure Portal.

Once assigned, it will automatically block any deployment that:

Tries to run as the Root user.
Attempts to mount the host path (which could allow a container to escape to the node).
Uses Privileged mode.

How to apply it via CLI:

Bash

			
# Get the ID of the 'Restricted' policy initiative
policyId=$(az policy set-definition list --query "[?displayName=='Kubernetes cluster pod security restricted standards for Linux-based workloads'].id" -o tsv)
# Assign it to your Resource Group or Subscription
az policy assignment create --name 'aks-restricted-policy' --policy-set-definition $policyId --scope /subscriptions/<your-subscription-id>/resourceGroups/<your-rg-name>

3. Securing the Flow

To visualize how these layers interact, imagine the lifecycle of a request:

Identity: The user authenticates via Entra ID.
Admission: Azure Policy checks if the YAML is “safe.”
Network: Network Policies ensure the pod only talks to its specific database.
Secrets: The pod pulls credentials from Azure Key Vault at runtime.

Pro-Tip: The “Audit” Mode

If you are worried that applying these policies will break the current app, you can set the Azure Policy effect to Audit instead of Deny. This allows you to see a list of non-compliant pods in the Azure Portal without actually taking them offline. Once you’ve fixed the configurations, you can flip the switch to Deny.

To pass the “Restricted” policy you just applied, your pod deployments can no longer be “lazy.” You must explicitly tell Kubernetes that the container is not allowed to do anything dangerous.

Below is a compliant deployment.yaml for a microservice.

The “Secure” Pod Template

YAML

			
apiVersion: apps/v1
kind: Deployment
metadata:
  name: secure-microservice
  namespace: production
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-api
  template:
    metadata:
      labels:
        app: web-api
    spec:
      # 1. Don't run as Root!
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        fsGroup: 2000
      containers:
      - name: api-container
        image: myacr.azurecr.io/web-api:v1.2
        # 2. Tighten the container's capabilities
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          capabilities:
            drop:
              - ALL
        # 3. Resource limits prevent DoS attacks
        resources:
          limits:
            cpu: "500m"
            memory: "256Mi"
          requests:
            cpu: "100m"
            memory: "128Mi"
        ports:
        - containerPort: 8080

		

Why this works (The “Why” for your client)

If you are proposing this to your client, explain the benefits of these specific lines:

runAsNonRoot: true: If a hacker exploits a bug in the app code, they only get “user” access. They can’t install software, change kernel settings, or break out of the container to the host server.
readOnlyRootFilesystem: true: This is a massive security win. It prevents malware from being downloaded and saved into the container. If the app needs to write temporary data, you should mount an emptyDir volume specifically for that.
capabilities: drop: - ALL: Linux containers usually come with “capabilities” they don’t need (like changing the system clock). Dropping all of them follows the Principle of Least Privilege.

How to handle “Storage” in a Read-Only Filesystem

Since the filesystem is now read-only, your app might crash if it tries to write to /tmp. You can solve this by adding a small “scratch space” in memory:

YAML

        volumeMounts:
        - name: tmp-volume
          mountPath: /tmp
      volumes:
      - name: tmp-volume
        emptyDir: {} # This creates a temporary writable folder in RAM

Proposing this as a “Service”

You can frame this to your company as a “Hardening Sprint.” > “I want to audit our existing 15 microservices to ensure they meet the 2026 AKS Restricted Security Standards. This will reduce our attack surface by 80% and ensure we pass our next compliance audit.”