Microsoft Sentinel: Automating Threat Response in Azure

Azure Sentinel (Microsoft Sentinel)

Microsoft Sentinel is Azure’s cloud-native SIEM (Security Information and Event Management) and SOAR (Security Orchestration, Automation and Response) platform — a single service that collects security data from across your entire estate, detects threats using AI and analytics, investigates incidents, and automates responses.


What Sentinel Actually Is — SIEM + SOAR Combined

SIEM (Security Information and Event Management)
→ Collects logs from everything
→ Correlates events across sources
→ Detects threats using rules + AI
→ Surfaces alerts and incidents
SOAR (Security Orchestration, Automation and Response)
→ Automates response to detected threats
→ Runs playbooks (Logic Apps) automatically
→ Integrates with ticketing, ITSM, and remediation tools
→ Reduces mean time to respond (MTTR)
Sentinel = both in one service, built on Log Analytics

The Four Pillars

Pillar 1 — Collect

Data flows into Sentinel through data connectors — pre-built integrations that normalise log formats and write to Log Analytics tables:

Azure native connectors (free ingestion):

  • Microsoft Defender for Cloud
  • Entra ID sign-in and audit logs
  • Azure Activity logs (ARM operations)
  • Azure Firewall logs
  • NSG flow logs
  • Key Vault audit logs
  • Azure Kubernetes Service (AKS/ARO)

Microsoft 365 connectors:

  • Microsoft 365 Defender (XDR)
  • Office 365 (Exchange, SharePoint, Teams)
  • Microsoft Defender for Endpoint
  • Microsoft Defender for Identity
  • Microsoft Defender for Cloud Apps

Third-party connectors:

  • Palo Alto, Fortinet, Check Point firewalls
  • Cisco ASA, Umbrella, Meraki
  • Okta, CrowdStrike, SentinelOne
  • AWS CloudTrail, S3 access logs
  • GCP audit logs

On-premises via agents:

Windows VMs → Log Analytics Agent → SecurityEvent table
Linux VMs → Syslog → Syslog table
Network devices → CEF → AMA agent → CommonSecurityLog table

Pillar 2 — Detect

Sentinel detects threats through five types of analytics rules:

Scheduled rules — KQL queries on a timer
// Detect impossible travel — same user, two countries, <1 hour apart
let threshold_minutes = 60;
SigninLogs
| where TimeGenerated > ago(1d)
| where ResultType == 0 // successful sign-in
| project TimeGenerated, UserPrincipalName,
Location, IPAddress,
Latitude = toreal(LocationDetails.geoCoordinates.latitude),
Longitude = toreal(LocationDetails.geoCoordinates.longitude)
| sort by UserPrincipalName, TimeGenerated asc
| extend PrevLocation = prev(Location, 1),
PrevTime = prev(TimeGenerated, 1),
PrevUser = prev(UserPrincipalName, 1)
| where UserPrincipalName == PrevUser
| extend TimeDiff = datetime_diff('minute', TimeGenerated, PrevTime)
| where TimeDiff < threshold_minutes
| where Location != PrevLocation
| project UserPrincipalName, Location, PrevLocation,
TimeDiff, IPAddress, TimeGenerated
Near Real-Time (NRT) rules — sub-1-minute detection
// Detect Azure Firewall blocking connections to known malicious IPs
AzureDiagnostics
| where Category == "AzureFirewallNetworkRule"
| where msg_s has "Deny"
| parse msg_s with * "from " SourceIP ":" SourcePort
" to " DestIP ":" DestPort ". Action: " Action
| join kind=inner (
ThreatIntelligenceIndicator
| where Active == true
| project NetworkIP, ThreatType, ConfidenceScore
) on $left.DestIP == $right.NetworkIP
| project TimeGenerated, SourceIP, DestIP, ThreatType, ConfidenceScore
Microsoft Security rules — auto-create incidents from Defender alerts

These automatically promote Defender for Cloud, Defender for Endpoint, and Defender for Identity alerts into Sentinel incidents with no KQL needed.

Fusion rules — ML-based multi-stage attack detection

Fusion uses machine learning to correlate low-severity signals across multiple products that individually look benign but together indicate an attack:

Signal 1: Entra ID — suspicious sign-in from anonymising proxy
Signal 2: Office 365 — mass email forwarding rule created
Signal 3: Azure — new service principal with owner role
Individual signals: low severity, easy to miss
Fusion correlation: HIGH severity — likely BEC (Business Email Compromise) attack
Anomaly rules — baseline + deviation detection

Sentinel builds behavioural baselines and alerts on deviations:

  • Unusual volume of data downloaded by a user
  • Login at an unusual time of day for this account
  • Process execution pattern not seen before on this host

Pillar 3 — Investigate

Incidents

Every triggered analytics rule creates an alert. Sentinel groups related alerts into incidents — the unit of work for a SOC analyst:

Incident: Possible BEC attack — john.smith@contoso.com
Severity: High
Status: New
Assigned: SOC Analyst 2
Alerts:
├── Impossible travel detected (Entra ID)
├── Mass forwarding rule created (Office 365)
└── New privileged service principal (Azure Activity)
Entities:
├── User: john.smith@contoso.com
├── IP: 185.220.101.45 (Tor exit node)
└── Host: LAPTOP-JSmith
MITRE ATT&CK:
├── T1078 — Valid accounts
├── T1114 — Email collection
└── T1098 — Account manipulation
Investigation graph

A visual relationship map automatically built from incident entities — shows how a user, IP, host, and mailbox are connected without manual correlation:

185.220.101.45 (Tor IP)
↓ signed in as
john.smith@contoso.com (user)
↓ created
Forward-all-mail rule (Office 365)
↓ same session created
sp-finance-automation (service principal)
↓ granted
Owner role on subscription

Entity pages

Every entity (user, IP, host, app) gets a timeline page showing all activity across all data sources — 90 days of context assembled automatically:

User: john.smith@contoso.com
Last 90 days:
├── Sign-ins: 847 (normal pattern: Mon-Fri 8am-6pm EST)
├── Anomalous sign-ins: 3 (Tor, Russia, Ukraine)
├── Files accessed: 12,847
├── Emails sent: 2,341
├── Azure resource operations: 156
└── Risk score: 94/100 (UEBA)

Pillar 4 — Respond (SOAR)

Playbooks are Azure Logic Apps triggered automatically when an incident is created or updated. They automate the first-response actions that would otherwise require a human:

Playbook 1 — Block compromised user automatically
Trigger: Sentinel incident created
Condition: Severity == High AND Entity type == User
Actions:
1. Get user details from Entra ID
2. Disable user account in Entra ID
3. Revoke all active sessions (MFA re-auth required)
4. Send Teams message to SOC channel:
"User john.smith auto-disabled — incident #1234"
5. Create ServiceNow ticket with incident details
6. Add comment to Sentinel incident:
"User account disabled at 14:32 UTC by playbook"
Playbook 2 — Isolate compromised VM
Trigger: Sentinel incident created
Condition: Severity == High AND Entity type == Host
Actions:
1. Get VM resource ID from entity
2. Apply isolation NSG (deny all inbound + outbound except Bastion)
az network nsg rule create --name ISOLATE --priority 100
--access Deny --direction Inbound --source-address-prefix *
3. Take VM disk snapshot (forensic preservation)
4. Tag VM: {"Status": "Isolated", "IncidentId": "1234"}
5. Notify SOC team via email + Teams
6. Create Jira ticket for IR team
Playbook 3 — Enrich IP with threat intelligence
Trigger: Sentinel alert contains IP entity
Actions:
1. Query VirusTotal API for IP reputation
2. Query Shodan for open ports and services
3. Query AbuseIPDB for abuse reports
4. Add enrichment comment to incident:
"IP 185.220.101.45:
VirusTotal: 47/92 vendors flagged malicious
AbuseIPDB: 847 reports, 100% confidence malicious
Shodan: Tor exit node — AS16276 OVH"
5. If malicious score > 80:
→ Add IP to Azure Firewall deny list automatically

KQL — The Query Language of Sentinel

Everything in Sentinel is queried with KQL (Kusto Query Language):

// Find all failed logins followed by success from same IP
// (credential stuffing pattern)
let failed_logins = SigninLogs
| where TimeGenerated > ago(1h)
| where ResultType != 0 // failed
| summarize FailCount = count() by IPAddress, UserPrincipalName
| where FailCount > 10;
let successful_logins = SigninLogs
| where TimeGenerated > ago(1h)
| where ResultType == 0 // success
| project IPAddress, UserPrincipalName, SuccessTime = TimeGenerated;
successful_logins
| join kind=inner failed_logins on IPAddress
| project IPAddress, UserPrincipalName,
FailCount, SuccessTime
| order by FailCount desc
// Detect Azure privilege escalation — new owner role assignment
AzureActivity
| where TimeGenerated > ago(1d)
| where OperationNameValue == "MICROSOFT.AUTHORIZATION/ROLEASSIGNMENTS/WRITE"
| where ActivityStatusValue == "Success"
| extend RoleDefinitionId = tostring(
parse_json(Properties).requestbody.properties.roleDefinitionId)
| where RoleDefinitionId contains "8e3af657-a8ff-443c-a75c-2fe8c4bcb635" // Owner
| project TimeGenerated, Caller, ResourceGroup,
SubscriptionId, RoleDefinitionId
// Hunt for lateral movement via PsExec or WMI
SecurityEvent
| where TimeGenerated > ago(7d)
| where EventID in (4688, 4624) // process create + logon
| where ProcessName has_any ("psexec", "wmic", "winrm")
or CommandLine has_any ("\\\\", "invoke-wmimethod", "wmiexec")
| summarize count() by Computer, Account, ProcessName, CommandLine
| order by count_ desc

MITRE ATT&CK Integration

Sentinel maps every analytics rule to MITRE ATT&CK tactics and techniques — giving you a visual coverage matrix:

TacticExample TechniqueSentinel Detection
Initial AccessT1078 Valid AccountsImpossible travel rule
PersistenceT1098 Account ManipulationNew owner role assignment
Privilege EscalationT1134 Token ImpersonationService principal abuse
Defence EvasionT1562 Impair DefencesDiagnostic setting deleted
Credential AccessT1110 Brute ForceFailed login threshold
Lateral MovementT1021 Remote ServicesPsExec / WMI detection
ExfiltrationT1048 Exfil over Alt ProtocolLarge blob download
ImpactT1486 Data EncryptedRansomware file extension

Sentinel in Hub and Spoke Context

In an enterprise hub and spoke topology, Sentinel sits at the subscription/tenant level — above the network, collecting from everything:

Microsoft Sentinel (Log Analytics Workspace)
│ data connectors
┌────┴──────────────────────────────────┐
│ │
Hub VNet Spoke VNets
Azure Firewall logs AKS/ARO audit logs
VPN Gateway logs VM security events
Bastion session logs NSG flow logs
DNS resolver logs App Gateway WAF logs
On-premises (via MMA/AMA agent)
Windows Security Events
Linux Syslog
Network device CEF

Sentinel vs Defender for Cloud

Microsoft SentinelDefender for Cloud
TypeSIEM + SOARCSPM + CWPP
FocusThreat detection + responsePosture management + workload protection
ScopeCross-tenant, multi-cloudAzure resources + connected clouds
DataAll log sourcesAzure resource configuration + telemetry
OutputIncidents + playbooksRecommendations + alerts
Use togetherDefender feeds alerts into SentinelSentinel adds SOAR response to Defender alerts

They are designed to work together — Defender for Cloud detects threats at the resource level and feeds high-fidelity alerts into Sentinel, which correlates them with signals from every other source and automates the response.


Pricing Model

Sentinel pricing has two components:

Log Analytics ingestion — pay per GB ingested:

  • Pay-as-you-go: ~$2.76/GB
  • Commitment tiers: 100 GB/day → 500 GB/day → lower per-GB rate

Sentinel capacity reservation — flat daily rate above the free Log Analytics tier:

  • First 10 GB/day per workspace: free
  • Above 10 GB/day: ~$100–$400/day depending on tier

Free data sources — no ingestion charge for:

  • Microsoft Defender alerts
  • Entra ID audit + sign-in logs (Basic SKU)
  • Azure Activity logs
  • Office 365 management activity

Key Takeaway

Microsoft Sentinel is the security brain of your Azure estate — it ingests logs from every corner of your infrastructure (Azure, Microsoft 365, on-premises, third-party), correlates signals using AI and KQL-based rules, groups related alerts into actionable incidents mapped to MITRE ATT&CK, and automates first-response actions through Logic App playbooks. In a hub and spoke network, it sits above the topology collecting from every layer — firewall, gateway, Bastion, ARO, VMs, and on-premises — giving your SOC a single pane of glass across the entire estate.

Understanding ARO’s Private DNS Zones Setup

Private DNS Zones Created by ARO

When you deploy a private ARO cluster, Azure automatically creates two private DNS zones in the ARO-managed resource group — one for the API server and one for application ingress. You own neither; they are managed by the ARO service, but you are responsible for linking them to every VNet that needs to resolve them.


The Two Zones ARO Creates

ARO creates both zones inside the ARO managed resource group — the resource group whose name starts with aro- that Azure creates automatically alongside your cluster. You cannot modify or delete these zones without breaking the cluster.

Zone 1 — API Server

Zone name: cluster.<unique-id>.<region>.aroapp.io
Example: cluster.a1b2c3d4.eastus.aroapp.io
A records:
api → 10.1.0.8 (private endpoint NIC IP)
Full FQDN: api.cluster.a1b2c3d4.eastus.aroapp.io:6443

Zone 2 — Application Ingress

Zone name:   cluster.<unique-id>.<region>.aroapp.io
             (same parent zone, different record)

A records:
  *.apps   →   10.1.1.100   (internal load balancer frontend IP)

Example resolutions:
  my-app.apps.cluster.a1b2c3d4.eastus.aroapp.io  →  10.1.1.100
  console.apps.cluster.a1b2c3d4.eastus.aroapp.io →  10.1.1.100
  grafana.apps.cluster.a1b2c3d4.eastus.aroapp.io →  10.1.1.100


Inspecting the Zones After Deployment

# Get the ARO managed resource group
MANAGED_RG=$(az aro show \
  --resource-group rg-aro \
  --name aro-prod \
  --query clusterProfile.resourceGroupId \
  --output tsv | xargs basename)

# List all private DNS zones ARO created
az network private-dns zone list \
  --resource-group $MANAGED_RG \
  --query "[].{Zone:name, Records:numberOfRecordSets}" \
  --output table

# Output:
# Zone                                          Records
# ─────────────────────────────────────────── ─────────
# cluster.a1b2c3d4.eastus.aroapp.io           4

# List all A records in the zone
az network private-dns record-set a list \
  --resource-group $MANAGED_RG \
  --zone-name cluster.a1b2c3d4.eastus.aroapp.io \
  --output table

# Output:
# Name     TTL    ARecords
# ──────   ────   ──────────────────────
# api      300    [{'ipv4Address': '10.1.0.8'}]
# *.apps   300    [{'ipv4Address': '10.1.1.100'}]

# List VNet links on the zone
az network private-dns link vnet list \
  --resource-group $MANAGED_RG \
  --zone-name cluster.a1b2c3d4.eastus.aroapp.io \
  --output table

# Output:
# Name                    VirtualNetwork          RegistrationEnabled
# ─────────────────────── ──────────────────────  ───────────────────
# aro-spoke-vnet-link     aro-spoke-vnet          false   ← auto-created


The VNet Linking Problem

This is the most common post-deployment mistake. ARO automatically links the private DNS zone to only one VNet — the ARO spoke VNet. Every other VNet that needs to resolve the API server or app routes must be manually linked:

ZONE_NAME="cluster.a1b2c3d4.eastus.aroapp.io"

# Link to hub VNet (required for jump host, Bastion, CI/CD runners)
az network private-dns link vnet create \
  --resource-group $MANAGED_RG \
  --zone-name $ZONE_NAME \
  --name "link-to-hub-vnet" \
  --virtual-network $(az network vnet show \
      --resource-group rg-hub \
      --name hub-vnet \
      --query id -o tsv) \
  --registration-enabled false

# Link to other spoke VNets if they need to call ARO-hosted APIs
az network private-dns link vnet create \
  --resource-group $MANAGED_RG \
  --zone-name $ZONE_NAME \
  --name "link-to-spoke2-vnet" \
  --virtual-network $(az network vnet show \
      --resource-group rg-spoke2 \
      --name spoke2-vnet \
      --query id -o tsv) \
  --registration-enabled false

registration-enabled false is correct here — you are linking for resolution only, not to auto-register VM hostnames into the zone.


On-Premises DNS Conditional Forwarding

On-premises DNS servers cannot be linked to Azure private DNS zones directly — they resolve through the DNS Private Resolver inbound endpoint using conditional forwarding:

On-premises Windows DNS Server:
Conditional Forwarder:
Domain: aroapp.io
Forward to: 10.0.5.4 (DNS Private Resolver inbound endpoint)
On-premises BIND (Linux):
zone "aroapp.io" {
type forward;
forwarders { 10.0.5.4; };
};

With this in place, an on-premises developer running oc login gets the full resolution chain:

1. oc login https://api.cluster.a1b2c3d4.eastus.aroapp.io:6443
2. Laptop DNS → corp DNS server
3. Corp DNS: aroapp.io → forward to 10.0.5.4
4. DNS Private Resolver checks linked private DNS zones
5. Finds: api.cluster.a1b2c3d4.eastus.aroapp.io → 10.1.0.8
6. Returns 10.1.0.8 to laptop
7. oc connects to 10.1.0.8:6443 via ExpressRoute / VPN tunnel
8. Login succeeds ✅

OpenShift Console DNS

The OpenShift web console runs as an application on the cluster, so it resolves through the *.apps wildcard record:

# Get console URL
az aro show \
  --resource-group rg-aro \
  --name aro-prod \
  --query consoleProfile.url \
  --output tsv

# Output:
# https://console-openshift-console.apps.cluster.a1b2c3d4.eastus.aroapp.io

# DNS resolution:
# console-openshift-console.apps.cluster.a1b2c3d4.eastus.aroapp.io
#   → matched by *.apps wildcard A record
#   → returns 10.1.1.100 (internal LB)
#   → browser connects via VPN/ER or Bastion proxy


Custom Domain — Replacing aroapp.io

If you want your own domain (e.g. openshift.contoso.com) instead of aroapp.io, you create a custom private DNS zone and manage the records yourself:

# Create your own private DNS zone
az network private-dns zone create \
  --resource-group rg-aro-network \
  --name openshift.contoso.com

# Add API server A record
az network private-dns record-set a add-record \
  --resource-group rg-aro-network \
  --zone-name openshift.contoso.com \
  --record-set-name api \
  --ipv4-address 10.1.0.8

# Add wildcard apps A record
az network private-dns record-set a add-record \
  --resource-group rg-aro-network \
  --zone-name openshift.contoso.com \
  --record-set-name "*.apps" \
  --ipv4-address 10.1.1.100

# Link to all VNets
az network private-dns link vnet create \
  --resource-group rg-aro-network \
  --zone-name openshift.contoso.com \
  --name link-hub \
  --virtual-network /subscriptions/.../hub-vnet \
  --registration-enabled false

Then update the ARO cluster to use the custom domain during deployment:

az aro create \
  --resource-group rg-aro \
  --name aro-prod \
  --vnet aro-spoke-vnet \
  --master-subnet master-subnet \
  --worker-subnet worker-subnet \
  --apiserver-visibility Private \
  --ingress-visibility Private \
  --domain openshift.contoso.com \    # ← custom domain
  --pull-secret @pull-secret.txt

With a custom domain the API server becomes api.openshift.contoso.com and apps become *.apps.openshift.contoso.com — owned and managed entirely by you, with no dependency on aroapp.io.


Key Takeaway

ARO automatically creates a private DNS zone under aroapp.io with two critical records — api pointing to the API server private endpoint IP and *.apps pointing to the internal load balancer frontend IP. The zone is auto-linked only to the ARO spoke VNet — you must manually link it to the hub VNet, any other spoke VNets, and configure on-premises DNS conditional forwarding to the DNS Private Resolver for the complete name resolution chain to work end-to-end across your hub and spoke estate.

Understanding ARO API Server Private Endpoint IPs

ARO API Server Private Endpoint IP

The ARO API server private endpoint IP is a private IP address automatically assigned by Azure from your master subnet’s address space when the cluster is created — it becomes the sole network entry point for all Kubernetes API traffic in a private cluster.


How the IP Is Assigned — the Full Mechanism

Azure subnet IP allocation order

Every Azure subnet reserves the first 5 IPs unconditionally:

10.1.0.0 — Network address (unusable)
10.1.0.1 — Default gateway
10.1.0.2 — Azure DNS
10.1.0.3 — Azure future use
10.1.0.4 — Broadcast address
─────────────────────────────────
10.1.0.5 → First assignable IP

After cluster provisioning, the master subnet fills up in this order:

10.1.0.5 → Master node 1 VM NIC (AZ1)
10.1.0.6 → Master node 2 VM NIC (AZ2)
10.1.0.7 → Master node 3 VM NIC (AZ3)
10.1.0.8 → API server private endpoint NIC ← auto-assigned
10.1.0.9 → Internal LB health probe IP
10.1.0.10+ → Future ARO platform components

The exact IP depends on provisioning order — Azure assigns the next available IP dynamically. You cannot pre-specify it, but once assigned it is static for the lifetime of the cluster.


The Private Endpoint NIC in Detail

The private endpoint is an Azure resource called a Private Endpoint — distinct from the VM NICs of the master nodes. You can inspect it:

# Find the ARO managed resource group (contains cluster infrastructure)
MANAGED_RG=$(az aro show \
--resource-group rg-aro \
--name aro-prod \
--query clusterProfile.resourceGroupId -o tsv)
# List private endpoints in the managed resource group
az network private-endpoint list \
--resource-group $MANAGED_RG \
--output table
# Output:
# Name ResourceGroup Location
# ───────────────────────────── ──────────────────── ─────────
# aro-prod-pe-apiserver aro-prod-cluster-rg eastus
# Get the private IP
az network private-endpoint show \
--resource-group $MANAGED_RG \
--name aro-prod-pe-apiserver \
--query 'customDnsConfigs[0].ipAddresses[0]' \
--output tsv
# Output: 10.1.0.8

What the Private Endpoint NIC Actually Is

The private endpoint is not a VM — it is a read-only synthetic NIC injected into your subnet by Azure’s network fabric. It has no OS, no compute, no management plane — it is purely a network construct:

Private Endpoint Resource
├── Name: aro-prod-pe-apiserver
├── Type: Microsoft.Network/privateEndpoints
├── NIC IP: 10.1.0.8 (from master subnet)
├── Target: ARO API server internal load balancer
│ (in Microsoft-managed ARO infrastructure)
├── Protocol: TCP
├── Port: 6443
├── Managed by: Microsoft / Red Hat (not customer)
└── Deletable: No — deleting breaks the cluster

Traffic arriving at 10.1.0.8:6443 is forwarded over Azure’s private backbone to the actual API server processes running on the master nodes — the customer never sees or touches the internal path.


How DNS Wires the IP to the FQDN

ARO automatically creates a Private DNS Zone and inserts an A record pointing the API server FQDN to the private endpoint IP:

# Find the private DNS zone
az network private-dns zone list \
--resource-group $MANAGED_RG \
--query "[].name" -o tsv
# Output:
# cluster.eastus.aroapp.io
# Inspect the A records
az network private-dns record-set a list \
--resource-group $MANAGED_RG \
--zone-name cluster.eastus.aroapp.io \
--output table
# Output:
# Name TTL Records
# ───── ──── ──────────
# api 300 10.1.0.8
# *.apps 300 10.1.1.100

The DNS zone is linked to the ARO spoke VNet automatically. You must manually link it to any other VNet (hub VNet, other spokes) that needs to resolve it:

# Link private DNS zone to hub VNet
az network private-dns link vnet create \
--resource-group $MANAGED_RG \
--zone-name cluster.eastus.aroapp.io \
--name link-hub-vnet \
--virtual-network /subscriptions/.../resourceGroups/rg-hub/providers/
Microsoft.Network/virtualNetworks/hub-vnet \
--registration-enabled false
# Verify resolution from jump host
nslookup api.cluster.eastus.aroapp.io 10.0.5.4
# Server: 10.0.5.4 (DNS Private Resolver)
# Address: 10.1.0.8 ← private endpoint IP returned ✅

Getting the API Server URL and IP Programmatically

# Get the full API server URL
API_URL=$(az aro show \
--resource-group rg-aro \
--name aro-prod \
--query apiserverProfile.url \
--output tsv)
echo $API_URL
# https://api.cluster.eastus.aroapp.io:6443
# Extract just the hostname
API_HOST=$(echo $API_URL | sed 's|https://||' | sed 's|:6443||')
echo $API_HOST
# api.cluster.eastus.aroapp.io
# Resolve to private IP (from inside VNet or connected network)
dig +short $API_HOST
# 10.1.0.8
# Verify TCP reachability on port 6443
nc -zv $API_HOST 6443
# Connection to api.cluster.eastus.aroapp.io 6443 port [tcp] succeeded!
# Login using oc
CREDS=$(az aro list-credentials \
--resource-group rg-aro \
--name aro-prod)
oc login $API_URL \
--username $(echo $CREDS | jq -r .kubeadminUsername) \
--password $(echo $CREDS | jq -r .kubeadminPassword) \
--insecure-skip-tls-verify=false

What Happens If You Try to Reach It From the Internet

The private endpoint IP (10.1.0.8) is a RFC 1918 private address — it is not routable on the public internet. From outside Azure, two things happen:

Scenario 1 — Public DNS lookup:
nslookup api.cluster.eastus.aroapp.io (from internet)
→ Returns NXDOMAIN or no answer
→ Public DNS has no record (zone is private only)
Scenario 2 — Direct TCP to port 6443:
curl https://api.cluster.eastus.aroapp.io:6443 (from internet)
→ DNS fails → connection never established
→ Even if DNS were somehow resolved, 10.1.0.8 is unreachable
from internet — packets dropped at Azure network edge

There is no public IP, no public DNS record, and no network path from the internet to the private endpoint — the attack surface is zero.


IP Stability — Does It Ever Change?

Once assigned at cluster creation time, the private endpoint IP is permanent for the cluster lifetime:

Cluster created: 10.1.0.8 assigned to API private endpoint
Cluster running: 10.1.0.8 (unchanged — days, months, years)
Master node reboot: 10.1.0.8 (private endpoint NIC is independent of master VMs)
ARO version upgrade: 10.1.0.8 (control plane upgrades don't change the PE IP)
Cluster deleted: 10.1.0.8 released back to subnet pool

This stability is intentional — UDRs, NSG rules, firewall rules, and DNS records all reference this IP. If it changed, every network policy referencing it would break. Azure guarantees it for the cluster lifetime without any reservation or static IP configuration needed on your part.


Key Takeaway

The ARO API server private endpoint IP is the next available IP after the master node NICs in your master subnet — automatically assigned by Azure during cluster provisioning, registered in a private DNS zone under aroapp.io, and permanently stable for the cluster lifetime. It is a synthetic NIC with no compute behind it — just a network fabric construct that forwards TCP port 6443 traffic over Azure’s private backbone to the actual API server processes on the master nodes. From the public internet it is completely invisible — no DNS record, no routable IP, no open port.

Benefits of a Private ARO Cluster in Azure

Private ARO Cluster

A private ARO cluster removes all public IP addresses from both the Kubernetes API server and the ingress router — making the cluster completely unreachable from the internet. Every connection to the cluster must travel over Azure’s private network backbone via VNet peering, ExpressRoute, or VPN.


Public vs Private ARO — What Changes

ComponentPublic clusterPrivate cluster
API server endpointPublic IP + DNSPrivate endpoint IP only
Ingress routerPublic load balancerInternal load balancer
Worker node IPsPrivate (always)Private (always)
Master node IPsPrivate (always)Private (always)
Access methodAny internet browserVPN / ER / Bastion only
DNS resolutionPublic DNSPrivate DNS zone
Attack surfaceAPI port 6443 exposedZero public exposure

How the API Server Is Hidden — The Mechanism

When you deploy a private ARO cluster, Azure does three things automatically:

1. API Server gets a Private Endpoint NIC

Instead of a public load balancer frontend, the API server is exposed exclusively through a private endpoint — a NIC in your VNet subnet with a private IP:

Public cluster:
  api.cluster.eastus.aroapp.io → 20.x.x.x (public IP)
  Anyone on internet can reach :6443

Private cluster:
  api.cluster.eastus.aroapp.io → 10.1.3.4 (private IP in your VNet)
  Only reachable from within the VNet or peered networks

The private endpoint is deployed into your ARO master subnet automatically during cluster creation. No public IP is allocated.

2. A Private DNS Zone Is Created Automatically

ARO creates a private DNS zone linked to your VNet so the API server FQDN resolves to the private endpoint IP:

Private DNS zone: cluster.eastus.aroapp.io
  A record: api → 10.1.3.4
  A record: *.apps → 10.1.4.8   (ingress internal LB)

Linked to: ARO spoke VNet + hub VNet

This means any VM in a peered VNet can resolve api.cluster.eastus.aroapp.io and get 10.1.3.4 — no public DNS lookup ever occurs.

3. Ingress Router Gets an Internal Load Balancer

The OpenShift ingress router (which handles *.apps.cluster.aroapp.io routes) is fronted by an Azure Internal Load Balancer with a private frontend IP:

Public cluster:   *.apps → Azure Public LB → 20.x.x.x
Private cluster:  *.apps → Azure Internal LB → 10.1.4.8

Applications running on the cluster are only reachable from inside the VNet or connected networks.


Deploying a Private ARO Cluster

# 1. Create resource group and VNet
az group create --name rg-aro --location eastus

az network vnet create \
  --resource-group rg-aro \
  --name aro-spoke-vnet \
  --address-prefixes 10.1.0.0/16

# 2. Create master subnet — disable private endpoint network policies
az network vnet subnet create \
  --resource-group rg-aro \
  --vnet-name aro-spoke-vnet \
  --name master-subnet \
  --address-prefixes 10.1.0.0/24 \
  --disable-private-link-service-network-policies true  # ← required for ARO

# 3. Create worker subnet
az network vnet subnet create \
  --resource-group rg-aro \
  --vnet-name aro-spoke-vnet \
  --name worker-subnet \
  --address-prefixes 10.1.1.0/23

# 4. Deploy private ARO cluster
az aro create \
  --resource-group rg-aro \
  --name aro-prod \
  --vnet aro-spoke-vnet \
  --master-subnet master-subnet \
  --worker-subnet worker-subnet \
  --apiserver-visibility Private \     # ← API server private
  --ingress-visibility Private \       # ← ingress private
  --master-vm-size Standard_D8s_v3 \
  --worker-vm-size Standard_D16s_v3 \
  --worker-count 3 \
  --pull-secret @pull-secret.txt

# Takes ~35 minutes to complete

The Three Access Paths

Path 1 — Azure Bastion + Jump Host (most common)

The simplest pattern — a small Linux VM in the hub VNet with oc and kubectl installed, accessed securely via Bastion:

# 1. Admin opens Azure portal → connects via Bastion to jump-host-vm
# 2. On jump host — get cluster credentials
az aro list-credentials \
  --resource-group rg-aro \
  --name aro-prod

# Output:
# kubeadminPassword: "XXXXX-XXXXX-XXXXX-XXXXX"
# kubeadminUsername: "kubeadmin"

# 3. Get API server URL
API_URL=$(az aro show \
  --resource-group rg-aro \
  --name aro-prod \
  --query apiserverProfile.url -o tsv)

# 4. Login — works because jump host is in peered VNet
oc login $API_URL \
  --username kubeadmin \
  --password XXXXX-XXXXX-XXXXX-XXXXX

# 5. Verify
oc get nodes
oc get clusterversion

Path 2 — ExpressRoute / VPN from on-premises

On-premises developers access the private API server directly from their workstations — but DNS must be configured to resolve the ARO private DNS zone:

On-premises developer workstation
Corporate DNS server: api.cluster.eastus.aroapp.io
↓ conditional forward to Azure DNS Private Resolver (10.0.5.4)
Azure DNS Private Resolver
↓ linked private DNS zone: aroapp.io → 10.1.3.4
Returns: 10.1.3.4
Developer runs: oc login https://api.cluster.eastus.aroapp.io:6443
Traffic travels: workstation → MPLS → ER Gateway → hub VNet peering → ARO spoke → API server

On-premises DNS server conditional forwarder:

Zone: aroapp.io
Forward to: 10.0.5.4 (DNS Private Resolver inbound endpoint)

Path 3 — CI/CD Pipeline (GitHub Actions / Azure DevOps)

For automated deployments, pipelines must also reach the private API server. Use a self-hosted runner inside the VNet:

# GitHub Actions — self-hosted runner in hub VNet
name: Deploy to ARO
on: [push]
jobs:
  deploy:
    runs-on: self-hosted    # ← runner VM inside Azure VNet
    steps:
      - uses: actions/checkout@v4

      - name: Login to ARO
        run: |
          oc login ${{ secrets.ARO_API_URL }} \
            --token ${{ secrets.ARO_SERVICE_ACCOUNT_TOKEN }}

      - name: Deploy application
        run: |
          oc apply -f k8s/
          oc rollout status deployment/my-app

The self-hosted runner is a VM in the hub VNet — it can resolve the private API server DNS and reach port 6443 over VNet peering.


Private DNS — The Critical Detail

After cluster creation, ARO automatically creates a private DNS zone. You must link this zone to every VNet that needs to resolve the API server — including the hub VNet where your jump host and DNS Private Resolver live:

# ARO creates this automatically — linked to ARO spoke VNet
# You must manually link it to the hub VNet

PRIVATE_ZONE=$(az network private-dns zone list \
  --resource-group $(az aro show -g rg-aro -n aro-prod \
    --query clusterProfile.resourceGroupId -o tsv | tr -d '\n') \
  --query "[?contains(name,'aroapp.io')].name" -o tsv)

# Link to hub VNet
az network private-dns link vnet create \
  --resource-group <aro-managed-rg> \
  --zone-name $PRIVATE_ZONE \
  --name link-to-hub-vnet \
  --virtual-network $(az network vnet show \
    --resource-group rg-hub \
    --name hub-vnet --query id -o tsv) \
  --registration-enabled false

Without this link, VMs in the hub VNet cannot resolve api.cluster.eastus.aroapp.io — DNS queries fall through to public DNS which returns NXDOMAIN for a private cluster.


Entra ID (AAD) Integration for Developer Access

Replace the kubeadmin local account with Entra ID authentication — developers log in with their corporate credentials:

# Configure AAD OAuth on ARO
az aro update \
  --resource-group rg-aro \
  --name aro-prod \
  --client-id <app-registration-client-id> \
  --client-secret <app-registration-secret>

# Grant cluster-admin to an AAD group
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: aro-cluster-admins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: <aad-group-object-id>    # e.g. Platform Engineering team
---
# Grant view-only to a developer group
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: dev-view
  namespace: my-app
roleRef:
  kind: ClusterRole
  name: view
subjects:
  - kind: Group
    name: <dev-aad-group-object-id>

Developers now login via:

oc login $API_URL # redirects to Microsoft login page
# Enter corporate credentials → MFA → issued a token

Egress from a Private Cluster

A private cluster still needs outbound internet access for Red Hat image registries and update servers. Force all egress through Azure Firewall via UDR on both subnets:

# Route table for master and worker subnets
az network route-table create \
  --resource-group rg-aro-network \
  --name rt-aro-subnets

az network route-table route create \
  --resource-group rg-aro-network \
  --route-table-name rt-aro-subnets \
  --name force-to-firewall \
  --address-prefix 0.0.0.0/0 \
  --next-hop-type VirtualAppliance \
  --next-hop-ip-address 10.0.1.4     # Azure Firewall private IP

# Associate with master subnet
az network vnet subnet update \
  --resource-group rg-aro \
  --vnet-name aro-spoke-vnet \
  --name master-subnet \
  --route-table rt-aro-subnets

# Associate with worker subnet
az network vnet subnet update \
  --resource-group rg-aro \
  --vnet-name aro-spoke-vnet \
  --name worker-subnet \
  --route-table rt-aro-subnets

Required Azure Firewall FQDN allow rules for private ARO:

quay.io                          # Red Hat image registry
registry.redhat.io               # Red Hat registry
registry.access.redhat.com       # RHEL content
cdn.quay.io                      # CDN for quay
*.blob.core.windows.net          # Azure storage (etcd backups, images)
*.servicebus.windows.net         # ARO monitoring
management.azure.com             # Azure ARM API
login.microsoftonline.com        # Entra ID auth

Key Takeaway

A private ARO cluster achieves zero public attack surface by replacing the public API server load balancer with a VNet-internal private endpoint, and replacing the public ingress load balancer with an internal one. DNS resolution of both endpoints stays entirely within Azure’s private network. The only access paths are Azure Bastion for interactive access, ExpressRoute or VPN for on-premises connectivity, and self-hosted CI/CD runners for automation — all travelling over encrypted private paths without a single packet touching the public internet.

Optimize Azure Traffic Flow with UDR

Traffic Flow Through Azure Firewall via UDR

The UDR (User Defined Route) is the mechanism that forces all spoke traffic through Azure Firewall — overriding Azure’s default system routes which would otherwise send traffic directly between peered VNets, bypassing inspection entirely.


Why UDRs Are Necessary

Azure VNet peering by default creates direct routing between peered VNets — packets travel peer-to-peer without touching any intermediate device. This means without UDRs, a VM in Spoke 1 talking to a VM in Spoke 2 completely bypasses Azure Firewall:

Default behaviour (NO UDR):
  Spoke 1 VM (10.1.1.4) → Spoke 2 VM (10.2.1.4)
  System route: 10.2.0.0/16 → VNet peering (direct)
  Result: traffic bypasses firewall entirely 

With UDR applied to spoke subnet:
  Spoke 1 VM (10.1.1.4) → Spoke 2 VM (10.2.1.4)
  UDR overrides: 0.0.0.0/0 → 10.0.1.4 (Firewall private IP)
  Result: traffic hits firewall → inspected → forwarded 

The UDR wins because custom routes always override system routes — Azure’s route selection priority is custom UDR first, then BGP routes, then system routes.


Route Table Structure

A route table is an Azure resource associated with one or more subnets. Every subnet that needs inspection gets the same core UDR:

Route Table: rt-spoke1-subnets
Associated to: Spoke 1 subnet A, Spoke 1 subnet B

Routes:
  Name              Prefix          Next hop type        Next hop IP
  ─────────────────────────────────────────────────────────────────
  force-to-fw       0.0.0.0/0       Virtual Appliance    10.0.1.4

Deployed via ARM / Bicep:

resource routeTable 'Microsoft.Network/routeTables@2023-04-01' = {
  name: 'rt-spoke1-subnets'
  location: location
  properties: {
    disableBgpRoutePropagation: true   // ← critical — explained below
    routes: [
      {
        name: 'force-to-firewall'
        properties: {
          addressPrefix: '0.0.0.0/0'
          nextHopType: 'VirtualAppliance'
          nextHopIpAddress: '10.0.1.4'   // Azure Firewall private IP
        }
      }
    ]
  }
}

// Associate with spoke subnet
resource subnetAssociation 'Microsoft.Network/virtualNetworks/subnets@2023-04-01' = {
  name: 'spoke1/snet-app'
  properties: {
    addressPrefix: '10.1.1.0/24'
    routeTable: {
      id: routeTable.id
    }
  }
}


The Three Traffic Paths

Path 1 — North-South Outbound (spoke VM → internet)

Step 1: Spoke 1 VM (10.1.1.4) sends packet to 8.8.8.8
Step 2: Subnet route table consulted
UDR match: 0.0.0.0/0 → next hop 10.0.1.4 (Firewall)
Step 3: Packet arrives at Azure Firewall private IP
Step 4: Firewall evaluates application rules
Rule: allow src=10.1.0.0/16 dest=*.google.com proto=HTTPS → ALLOW
Step 5: Firewall SNATs packet
Source IP changed: 10.1.1.4 → Firewall public IP (20.x.x.x)
Step 6: Packet exits to internet from Firewall public IP
Return traffic arrives at Firewall public IP
Step 7: Firewall translates back → forwards to 10.1.1.4

SNAT is automatic for internet-bound traffic — the spoke VM’s private IP is never exposed to the internet. Azure Firewall’s public IP is the only address the internet sees.


Path 2 — North-South Inbound (internet → spoke VM)

Step 1: External client sends to Firewall public IP 20.x.x.x:443
Step 2: Firewall DNAT rule fires:
dest 20.x.x.x:443 → translated to 10.1.4.5:443 (spoke VM)
Step 3: Firewall forwards to 10.1.4.5 via VNet peering path
Step 4: Packet arrives at spoke VM — no public IP needed on VM
Step 5: VM responds to Firewall private IP (it sees FW as source)
UDR on VM subnet ensures return goes back through Firewall
Step 6: Firewall forwards return to external client

Path 3 — East-West (spoke 1 VM → spoke 2 VM)

This is the most important path for security — lateral movement between spokes must be inspected:

Step 1: Spoke 1 VM (10.1.1.4) sends packet to Spoke 2 VM (10.2.1.8)
Step 2: Spoke 1 subnet route table consulted
UDR: 0.0.0.0/0 → 10.0.1.4 (matches — more specific than system route)
Step 3: Packet arrives at Azure Firewall
Step 4: Firewall evaluates network rules
Rule: allow src=10.1.0.0/16 dest=10.2.1.8 port=443 → ALLOW
(or deny if no rule matches)
Step 5: Firewall forwards to 10.2.1.8 via peering to Spoke 2
Step 6: Spoke 2 subnet route table:
UDR: 0.0.0.0/0 → 10.0.1.4
Return traffic: 10.2.1.8 → 10.1.1.4
UDR forces return through Firewall too
Step 7: Firewall forwards return packet to Spoke 1 VM

Both directions of every connection traverse the Firewall — request and response. This is essential for stateful inspection — if only one direction went through the Firewall, the session state table would be incomplete.


disableBgpRoutePropagation — Why It Matters

Every route table has a disableBgpRoutePropagation flag. On spoke subnets this must be set to true:

disableBgpRoutePropagation: false (default)
→ VPN Gateway pushes on-premises routes into spoke effective routes
→ Spoke VM sends on-premises traffic directly to Gateway
→ Bypasses Firewall for on-premises bound traffic ❌
disableBgpRoutePropagation: true (required for spoke subnets)
→ VPN Gateway routes suppressed on spoke subnets
→ Only UDR routes active: 0.0.0.0/0 → Firewall
→ All traffic including on-premises bound goes through Firewall ✅

Forgetting this setting is one of the most common misconfiguration errors in hub and spoke deployments — on-premises traffic silently bypasses the Firewall even though the UDR looks correct.


UDR on GatewaySubnet — On-Premises to Spoke

To inspect traffic arriving from on-premises destined for spoke VNets, a UDR must also be applied to the GatewaySubnet:

Route Table: rt-gateway-subnet
Associated to: GatewaySubnet
Routes:
Name Prefix Next hop type Next hop IP
────────────────────────────────────────────────────────────────
to-spoke1 10.1.0.0/16 VirtualAppliance 10.0.1.4
to-spoke2 10.2.0.0/16 VirtualAppliance 10.0.1.4
to-spoke3 10.3.0.0/16 VirtualAppliance 10.0.1.4

Note this uses specific spoke prefixes rather than 0.0.0.0/0 — applying a default route to GatewaySubnet breaks the gateway’s ability to communicate with Azure control plane endpoints.


Effective Route Inspection

You can verify UDRs are working correctly by checking a VM’s effective routes in the Azure portal or CLI:

az network nic show-effective-route-table \
--resource-group rg-spoke1 \
--name vm-prod-01-nic \
--output table
Source State Address Prefix Next Hop Type Next Hop IP
──────── ─────── ──────────────── ────────────────── ──────────
Default Active 10.1.0.0/16 VnetLocal
Default Invalid 0.0.0.0/0 Internet ← overridden
User Active 0.0.0.0/0 VirtualAppliance 10.0.1.4 ✅
Default Active 10.0.0.0/16 VNetPeering
Default Active 10.2.0.0/16 VNetPeering

The default 0.0.0.0/0 → Internet system route shows as Invalid — it has been overridden by the custom UDR pointing to the Firewall. This confirms all traffic is force-tunnelled correctly.


Common Misconfiguration Pitfalls

Forgetting disableBgpRoutePropagation — gateway-learned routes override UDRs for on-premises prefixes, silently bypassing Firewall for hybrid traffic.

Missing return path UDR — if Spoke 2 subnet has no UDR, return traffic goes directly back to Spoke 1 via peering, creating an asymmetric routing loop that breaks TCP sessions.

Applying UDR to AzureBastionSubnet — Bastion requires direct internet connectivity for its management plane. A UDR with 0.0.0.0/0 → Firewall on AzureBastionSubnet breaks Bastion entirely. Bastion subnet must have no UDR or a very specific one that excludes Bastion management ranges.

Applying 0.0.0.0/0 UDR to GatewaySubnet — breaks gateway health probes and control plane communication. Always use specific spoke prefixes on GatewaySubnet, never a default route.

Firewall private IP not static — Azure Firewall’s private IP should be configured as static during deployment. If it changes, every UDR next-hop entry becomes invalid and traffic black-holes.


Key Takeaway

The UDR is a deceptively simple mechanism — a single route entry 0.0.0.0/0 → Virtual Appliance → 10.0.1.4 — that transforms Azure’s default direct peering behaviour into a fully inspected, security-enforced network. Applied correctly to every spoke subnet with disableBgpRoutePropagation enabled, it ensures no traffic — outbound internet, inbound DNAT, or lateral east-west — can bypass Azure Firewall, giving you complete visibility and control over your entire hub and spoke estate.

Understanding Azure Bastion: A Complete Guide

Azure Bastion

Azure Bastion is a fully managed PaaS service that provides secure RDP and SSH connectivity to your virtual machines directly through the Azure portal or native client — over TLS on port 443 — without exposing any public IP address on the VM itself.


Why Bastion Exists — The Problem It Solves

The traditional way to RDP or SSH into an Azure VM was to assign it a public IP and open port 3389 or 22 to the internet. This creates serious exposure:

Old approach:
VM has public IP 20.x.x.x → port 3389 open to internet
→ Constant brute-force attacks (thousands/day)
→ Any misconfigured NSG = immediate compromise
→ Public IP costs, management overhead
→ No session recording or audit trail
With Bastion:
VM has no public IP — completely unreachable from internet
→ Admin connects via browser to Bastion on port 443
→ Bastion proxies RDP/SSH over private VNet path
→ Session fully audited in Azure Monitor
→ No attack surface on the VM itself

How the Connection Works — Step by Step

1. Admin opens Azure portal or native client
Selects a VM → Connect → Bastion
2. Portal establishes WebSocket connection to Bastion
over HTTPS port 443 (same as normal web traffic)
→ passes through corporate firewalls without issue
3. Bastion authenticates the admin
via Azure AD / RBAC — checks if admin has
"Bastion User" role on the Bastion resource
4. Bastion opens RDP (:3389) or SSH (:22) connection
from its private NIC directly to the VM's private IP
entirely inside the VNet / peered spoke
5. RDP or SSH session streams back to browser
rendered as HTML5 — no RDP client needed
(Standard SKU supports native RDP/SSH client too)
6. Session ends → connection torn down
Audit log written to Azure Monitor / Log Analytics

The VM never sees a connection from the internet — it only sees a private IP connection from within the VNet.


AzureBastionSubnet Requirements

Bastion requires a dedicated subnet with a very specific name — Azure will refuse to deploy it anywhere else:

Subnet name: AzureBastionSubnet ← exact name required
Min size: /26 (64 addresses) ← Basic SKU minimum
Recommended: /24 or /25 ← room to scale instances
Location: Hub VNet only ← one Bastion serves all peered spokes

No other resources — VMs, firewalls, private endpoints — can share this subnet. The /26 minimum exists because Bastion deploys multiple managed instances internally and Azure reserves addresses for platform use.


Bastion SKUs

FeatureBasicStandardPremium
Browser-based RDP/SSH
Native client (RDP/SSH app)
Shareable links
IP-based connection (no Azure VM)
VNet peering support
Session scaling (instances)2 fixed2–50 scalable2–50 scalable
File transfer (upload/download)
Clipboard (copy/paste)✅ text✅ text + file✅ text + file
Session recording
Private-only Bastion (no public IP)
Kerberos authentication
Pricing~$140/month~$280/month~$470/month

Standard is the right choice for most enterprises — it adds native client support (so admins can use their local RDP or SSH client instead of the browser) and instance scaling for large teams.

Premium adds session recording — every RDP/SSH session is captured and stored in a storage account — critical for compliance environments (PCI-DSS, SOC 2) where you must prove what privileged users did during a session. It also supports fully private Bastion with no public IP at all, using a private endpoint instead.


NSG Rules Required

Two NSGs must be configured correctly — one on the AzureBastionSubnet and one on the target VM subnet:

NSG on AzureBastionSubnet

Inbound rules:

PrioritySourceDestPortProtocolActionPurpose
100InternetAny443TCPAllowAdmin HTTPS inbound
110GatewayManagerAny443TCPAllowAzure control plane
120AzureLoadBalancerAny443TCPAllowHealth probes

Outbound rules:

PrioritySourceDestPortProtocolActionPurpose
100AnyVirtualNetwork3389, 22TCPAllowRDP/SSH to VMs
110AnyAzureCloud443TCPAllowBastion diagnostics

NSG on target VM subnet

PrioritySourceDestPortProtocolActionPurpose
100VirtualNetworkAny3389TCPAllowRDP from Bastion
110VirtualNetworkAny22TCPAllowSSH from Bastion

The VM subnet must allow inbound from VirtualNetwork service tag — which includes all peered VNets including the hub where Bastion lives. Critically, there is no rule allowing port 3389 or 22 from Internet — the VM is completely shielded.


RBAC Roles for Bastion Access

Bastion access is controlled by Azure RBAC — not just network connectivity:

RoleWhat it allows
Reader on BastionSee the Bastion resource
Bastion ReaderConnect through Bastion (read-only session)
Virtual Machine Contributor on VMNeeded to initiate the connection
Bastion ContributorManage the Bastion resource

A typical setup grants an admin team the Virtual Machine Contributor role on specific spoke resource groups, plus Reader on the Bastion. They can then connect to VMs they have rights to — but cannot modify the Bastion configuration itself or connect to VMs outside their scope.


Bastion in Hub and Spoke — One Bastion for All Spokes

A single Bastion in the hub VNet reaches VMs in all peered spoke VNets — you do not need a Bastion in every spoke. This is a key cost and management advantage:

Hub VNet → AzureBastionSubnet → Azure Bastion
↓ peering allows Bastion to reach
Spoke 1 VMs (production)
Spoke 2 VMs (development)
Spoke 3 VMs (shared services)
Spoke 4 VMs (DMZ)

One Bastion instance, one Standard public IP, one monthly cost — covering your entire estate. The only requirement is that the hub-to-spoke peering has Allow gateway transit enabled on the hub side and Use remote gateways enabled on the spoke side (the same peering settings used for routing).


Key Takeaway

Azure Bastion eliminates the biggest attack surface in most Azure environments — publicly exposed management ports on VMs. By proxying all RDP and SSH through a hardened, Microsoft-managed PaaS service over HTTPS, it gives you secure, auditable VM access with no public IPs on your workloads, no VPN client requirement for browser sessions, and full RBAC control over who can connect to what.

Understanding Azure DDoS Protection Standard

Azure DDoS Protection Standard

Azure DDoS Protection Standard is a managed, always-on service that detects and mitigates volumetric, protocol, and application-layer DDoS attacks against your Azure public IP addresses — automatically, without any configuration changes during an attack.


The Three Attack Categories It Defends Against

Layer 3/4 — Volumetric attacks

These flood your network bandwidth with massive traffic volumes — UDP floods, ICMP floods, amplification attacks (DNS, NTP, memcached). Azure absorbs these at the network edge using its global 60+ Tbps scrubbing capacity, before the traffic ever reaches your VNet or gateway.

Layer 3/4 — Protocol attacks

These exhaust connection state tables on firewalls, load balancers, and gateways. SYN floods send millions of half-open TCP connections; Smurf attacks abuse ICMP broadcasts. DDoS Protection mitigates these by validating TCP handshakes and rate-limiting malformed packets at the edge.

Layer 7 — Resource layer attacks

HTTP floods, Slowloris, and application-specific attacks target your app’s compute rather than your bandwidth. DDoS Protection Standard alone does not fully mitigate Layer 7 attacks — these require Azure WAF on Application Gateway or Azure Front Door working alongside DDoS Protection. The two services are designed to be used together for full-stack protection.


How Adaptive Tuning Works

This is the core differentiator versus the free Basic tier. DDoS Protection Standard builds a per-public-IP traffic baseline using machine learning:

Normal Monday traffic profile for your VPN Gateway public IP:
- Avg 2,000 packets/sec
- Peak 8,000 packets/sec
- Protocol mix: 70% TCP, 20% UDP, 10% ICMP
- Geographic distribution: CA, US, EU
Attack detected when:
- Packets jump to 4,000,000/sec ← 500× normal
- 99% from single ASN in one region
- All UDP port 53 (DNS amplification)
Response: automatic mitigation within seconds
- Rate limit traffic matching attack signature
- Pass legitimate traffic through
- Alert via Azure Monitor

Baselines are built per public IP, per protocol, per port — so the service understands what normal looks like for your VPN Gateway vs your Application Gateway vs your load balancer, and tuning is automatic as your traffic patterns change.


DDoS Protection Tiers Compared

FeatureBasic (free)Network (Standard)IP Protection
Always-on monitoring
Automatic attack mitigation✅ basic✅ advanced✅ advanced
Adaptive ML tuning per IP
Attack analytics & metrics
Attack mitigation reports
Attack mitigation flow logs
Azure Monitor alerts
WAF policy integration
DDoS Rapid Response (Microsoft experts)
Cost protection (service credit)
ScopeAll Azure (shared)Per VNet (plan)Per public IP
PricingFree~$2,944/month + per IP~$199/IP/month

Network Protection (the classic “Standard” tier) is applied at the VNet level via a DDoS Protection Plan — every public IP in all linked VNets is automatically covered.

IP Protection is a newer, per-IP option introduced for smaller deployments where you only need to protect a handful of public IPs without paying for a full plan.


What a DDoS Protection Plan Covers

A single DDoS Protection Plan can be linked to multiple VNets across multiple subscriptions in the same tenant. This is the right model for hub and spoke — one plan at the hub subscription level covers everything:

DDoS Protection Plan (resource group: rg-network-hub)
↓ linked to
Hub VNet → VPN Gateway public IP protected
→ Azure Firewall public IP protected
→ Bastion public IP protected
Spoke 4 (DMZ) → App Gateway public IP protected
→ Load Balancer public IP protected

The first 100 public IPs are included in the plan price. Beyond 100, you pay per additional IP.


Monitoring and Alerting

During and after an attack, DDoS Protection surfaces detailed metrics in Azure Monitor:

MetricWhat it shows
Under DDoS attackBoolean — 1 if attack active on this IP
Inbound packets dropped DDoSPackets/sec being scrubbed
Inbound packets forwarded DDoSClean packets/sec passing through
Inbound bytes DDoSRaw attack volume in bytes/sec
Mitigation reasonSYN flood, UDP flood, etc.

Set an alert rule on Under DDoS attack = 1 to fire a notification to your security team or trigger a Logic App / n8n workflow the moment an attack begins.


When Should You Enable It?

Enable DDoS Protection Standard when any of these are true

Any public-facing production workload with real business impact if taken offline — an internet-facing Application Gateway, a VPN Gateway handling thousands of remote users, or a load balancer fronting a revenue-generating application — warrants the protection. The ~$3K/month cost is trivial compared to the revenue loss and incident response cost of a successful multi-hour DDoS attack.

You also need it when compliance frameworks require it. PCI-DSS, HIPAA, and ISO 27001 environments often require documented DDoS mitigation controls, and DDoS Protection Standard gives you the attack reports and flow logs needed to satisfy auditors.

The cost protection feature is a practical reason too — if an attack causes your Azure compute or bandwidth costs to spike (autoscaled VMs spun up to handle flood traffic, for example), Microsoft will credit those costs back when DDoS Protection was enabled.

You can skip it when

Dev/test environments, internal-only workloads with no public IPs, and resources entirely behind Azure Front Door or a third-party CDN that absorbs attacks upstream don’t need the plan — the Basic tier’s shared protection is sufficient, and the CDN/Front Door layer already absorbs volumetric attacks before they reach your origin.


In a Hub and Spoke Context

The recommended placement is one DDoS Protection Plan at the hub subscription, linked to the hub VNet and any spoke VNets that have public IPs (typically the DMZ spoke with App Gateway and WAF). Pair it with Azure Firewall for Layer 3/4 east-west filtering and Azure WAF on Application Gateway for Layer 7 protection, and you have defence-in-depth across all three attack categories.

Managing DNS Efficiently with Azure Private Resolver

Azure DNS Private Resolver

Azure DNS Private Resolver is a fully managed, highly available DNS service that lets your spoke VNets and on-premises networks resolve Azure private DNS zones — without deploying and managing DNS virtual machines.

Before Private Resolver existed, enterprises had to run Windows Server DNS VMs in the hub to forward queries between on-premises and Azure. Private Resolver replaces that entirely with a managed service.


The Two Endpoints

DNS Private Resolver has two endpoint types, each deployed into a dedicated delegated subnet inside the hub VNet.

Inbound Endpoint

Receives DNS queries from outside Azure — from on-premises DNS servers or spoke VNets that point their DNS server setting directly at this IP.

  • Gets a static private IP from your hub VNet subnet (e.g. 10.0.5.4)
  • Reachable over VPN Gateway or ExpressRoute from on-premises
  • On-premises DNS servers forward specific zones (e.g. *.privatelink.blob.core.windows.net) to this IP
  • Requires a dedicated /28 subnet named however you like (e.g. snet-dns-inbound)

Outbound Endpoint

Forwards DNS queries from Azure to external resolvers — typically to on-premises DNS servers for resolving internal corp domains like *.contoso.local.

  • Also gets a private IP from a dedicated /28 subnet
  • Does not receive queries directly — works only through a Forwarding Ruleset
  • Requires a dedicated subnet separate from the inbound endpoint subnet

Forwarding Ruleset

A Forwarding Ruleset is a collection of conditional forwarding rules attached to the outbound endpoint. You then link the ruleset to spoke VNets so those spokes inherit the forwarding rules automatically.

Example ruleset rules

DomainForwarding targetUse
contoso.local.192.168.1.10:53 (on-prem DNS)Resolve internal corp names
corp.contoso.com.192.168.1.10:53Resolve corp public domain internally
prod.internal.10.3.2.5:53Resolve shared-services zone
. (wildcard)168.63.129.16:53All other queries → Azure public DNS

The wildcard . rule is the catch-all — any query that doesn’t match a specific rule falls through to Azure’s public DNS resolver.


How Spoke VNets Resolve Private DNS Zones

There are two patterns depending on whether you want centralised or per-VNet control.

Pattern 1 — Custom DNS server pointing to inbound endpoint (recommended)

Each spoke VNet sets its DNS server to the inbound endpoint’s private IP (10.0.5.4) instead of the default Azure DNS (168.63.129.16):

Spoke VM queries: myaccount.blob.core.windows.net
Spoke VNet DNS setting → 10.0.5.4 (inbound endpoint)
DNS Private Resolver checks private DNS zones
Finds: myaccount.blob.core.windows.net → 10.1.8.4 (private endpoint IP)
Returns private IP — traffic stays on private network

This is set at the VNet level in Azure portal or via ARM:

{
"dhcpOptions": {
"dnsServers": ["10.0.5.4"]
}
}

Pattern 2 — Ruleset linked to spoke VNets

Link the hub’s forwarding ruleset directly to each spoke VNet. The spoke VNets keep the default Azure DNS (168.63.129.16) but inherit the conditional forwarding rules from the ruleset:

Spoke VM queries: server01.contoso.local
Azure DNS (168.63.129.16) — checks linked ruleset
Ruleset rule: contoso.local → forward to 192.168.1.10
Outbound endpoint forwards to on-premises DNS
On-premises DNS returns 192.168.10.45

Pattern 2 avoids changing the DNS server setting on every spoke and is easier to manage at scale — you just link new spokes to the existing ruleset.


Private DNS Zone Resolution Flow (Full Detail)

1. Developer VM in prod spoke queries:
mydb.privatelink.database.windows.net
2. Query goes to 10.0.5.4 (inbound endpoint)
3. DNS Private Resolver checks:
→ Is there a forwarding rule for this domain? No
→ Is there a linked private DNS zone? Yes
Zone: privatelink.database.windows.net
Record: mydb → 10.1.9.6 (private endpoint NIC IP)
4. Returns: mydb.privatelink.database.windows.net = 10.1.9.6
5. VM connects to SQL on 10.1.9.6
Traffic never leaves Azure backbone

Without DNS Private Resolver, the public DNS record for mydb.database.windows.net would resolve to the public IP, bypassing your private endpoint entirely.


Private DNS Zone Auto-Registration

When you create Azure PaaS resources with private endpoints, you link them to a private DNS zone. Common zones:

ServicePrivate DNS zone
Azure Blob Storageprivatelink.blob.core.windows.net
Azure SQL Databaseprivatelink.database.windows.net
Azure Key Vaultprivatelink.vaultcore.azure.net
Azure Container Registryprivatelink.azurecr.io
Azure Kubernetes Serviceprivatelink.{region}.azmk8s.io
Azure Monitorprivatelink.monitor.azure.com
Azure Service Busprivatelink.servicebus.windows.net

All these zones are linked to the hub VNet where DNS Private Resolver lives. Because all spokes resolve through the resolver, they automatically get the private IPs for these services.


Subnet Requirements

EndpointSubnet name (your choice)Min sizeDelegation
Inbounde.g. snet-dns-inbound/28 (16 IPs)Microsoft.Network/dnsResolvers
Outbounde.g. snet-dns-outbound/28 (16 IPs)Microsoft.Network/dnsResolvers

Both subnets must be in the hub VNet, must be separate from each other, and must not contain any other resources. The delegation is set automatically when you create the endpoint.


Before vs After Private Resolver

Before (DNS VMs)After (DNS Private Resolver)
Infrastructure2+ Windows DNS VMs in hubZero VMs — fully managed
High availabilityManual VM HA, availability setsBuilt-in, 99.99% SLA
MaintenancePatch, monitor, backup VMsNone
Conditional forwardingConfigured per-VMForwarding rulesets, linked to VNets
On-premises resolutionRequires VM reachabilityInbound endpoint IP, reachable over VPN/ER
CostVM compute + licencesPer-endpoint + per-query pricing

DNS Private Resolver is one of the clearest examples in Azure of a managed service eliminating operational overhead — the old pattern of DNS VMs in the hub was fragile, expensive, and easy to misconfigure.

Azure Firewall Rule Types in Hub and Spoke

Azure Firewall enforces three distinct rule collections processed in a strict priority order. Understanding all three is essential to designing a secure hub and spoke topology.

The three rule types are processed top to bottom — DNAT first, then network, then application. A match at any layer stops processing. If nothing matches, traffic is implicitly denied.


Rule Type 1 — DNAT Rules

Destination Network Address Translation rewrites the destination IP (and optionally port) of inbound traffic hitting the firewall’s public IP, redirecting it to a private backend inside a spoke VNet.

What it does

Internet client → Firewall public IP (52.x.x.x:443)
↓ DNAT rule fires
Rewrites destination to 10.1.4.5:443
Backend VM in production spoke

Example DNAT rules

NameProtocolSourceDest (public IP)Dest portTranslated IPTranslated port
allow-web-inboundTCP*52.10.20.3044310.1.4.5443
allow-rdp-adminTCP203.0.113.0/2452.10.20.30338910.0.3.103389
allow-api-gatewayTCP*52.10.20.3180,44310.4.2.88080

Key rules about DNAT

  • DNAT rules implicitly create a matching network rule to allow the translated traffic through — you don’t need a separate network rule for the return path.
  • DNAT only applies to inbound traffic — traffic arriving at the firewall’s public IP from outside.
  • You cannot DNAT to a broadcast or multicast address.
  • After translation, the packet is treated as if it came from the firewall’s private IP — so your backend VMs see the firewall, not the original client. Preserve source IP with SNAT if needed.

Rule Type 2 — Network Rules

Network rules are Layer 3/4 filters — they match on source IP, destination IP, port, and protocol. No payload inspection. This is the right tool for non-HTTP traffic: SQL, RDP, SMB, DNS, NTP, custom protocols.

What it does

Spoke VM (10.1.2.5) → SQL Server (10.3.4.10:1433)
Network rule: allow src=10.1.0.0/16 dest=10.3.4.10 port=1433 proto=TCP
Traffic passes — no application-layer inspection

Example network rules

NameSourceDestinationProtocolPortAction
allow-spoke-to-ad10.0.0.0/810.3.2.0/24TCP/UDP53,88,389,636Allow
allow-prod-to-sql10.1.0.0/1610.3.4.10TCP1433Allow
allow-mgmt-rdp10.0.3.0/2410.0.0.0/8TCP3389Allow
allow-ntp10.0.0.0/8*UDP123Allow
deny-dev-to-prod10.2.0.0/1610.1.0.0/16AnyAnyDeny
allow-internet-out10.0.0.0/8*TCP80,443Allow

Network rule features

IP Groups — reusable objects containing lists of IPs and CIDRs, so you don’t repeat 10.1.0.0/16, 10.2.0.0/16, 10.3.0.0/16 in every rule:

IPGroup: "all-spokes" = [10.1.0.0/16, 10.2.0.0/16, 10.3.0.0/16, 10.4.0.0/16]

FQDN in network rules — you can use FQDNs as destinations in network rules (e.g. *.windows.update.com) but only for TCP/UDP. The firewall resolves the FQDN using its DNS settings and matches on the resolved IP.

Service Tags — Microsoft-managed groups of IP ranges for Azure services:

Source: 10.0.0.0/8 → Destination: AzureMonitor → Port: 443 → Allow

Common tags: AzureCloud, AzureMonitor, Storage, Sql, WindowsUpdate, MicrosoftDefenderForEndpoint


Rule Type 3 — Application Rules

Application rules operate at Layer 7 — they can inspect the HTTP/HTTPS host header and URL path, enforce FQDN allow-lists, apply web category filtering, and (with Premium SKU) perform full TLS inspection.

What it does

Spoke VM → HTTPS request to api.github.com
Application rule: allow src=10.1.0.0/16 target=*.github.com proto=Https
Firewall checks SNI / Host header — matches rule
Traffic passes (or inspected if TLS inspection enabled)

Example application rules

NameSourceTarget FQDNsProtocolAction
allow-windows-update10.0.0.0/8*.update.microsoft.com, *.windowsupdate.comHTTP, HTTPSAllow
allow-azure-services10.0.0.0/8*.azure.com, *.core.windows.netHTTPSAllow
allow-dev-package-mgrs10.2.0.0/16*.npmjs.org, *.pypi.org, *.nuget.orgHTTPSAllow
allow-github10.1.0.0/16*.github.com, *.githubusercontent.comHTTPSAllow
deny-social-media10.0.0.0/8Web category: SocialNetworkingHTTP, HTTPSDeny
allow-all-outbound10.0.0.0/8*HTTP, HTTPSAllow

FQDN Tags — Microsoft-managed bundles

Instead of listing dozens of Microsoft service URLs manually, use built-in FQDN tags:

FQDN TagWhat it covers
WindowsUpdateAll Windows Update endpoints
WindowsDiagnosticsTelemetry and diagnostics endpoints
MicrosoftActiveProtectionServiceDefender update endpoints
AppServiceEnvironmentASE management traffic

Azure Firewall SKUs

FeatureBasicStandardPremium
DNAT rules
Network rules
Application rules
FQDN filteringLimited
Web category filtering
Threat intelligenceAlert only✅ Alert + deny✅ Alert + deny
TLS inspection
IDPS (intrusion detection)
URL filtering (path-level)
Use caseDev/testMost enterprisesHigh-security / compliance

TLS inspection (Premium only) decrypts HTTPS traffic, inspects the payload with IDPS signatures, then re-encrypts it. Requires deploying a CA certificate chain trusted by all spoke VMs — typically distributed via Group Policy or Intune.


Firewall Policy vs Classic Rules

Modern Azure Firewall uses Firewall Policy — an ARM resource that holds all rule collections and can be shared across multiple firewall instances:

Firewall Policy (parent — global rules)
↓ inheritance
Firewall Policy (child — environment-specific rules)
↓ applied to
Azure Firewall instance (hub VNet)

This lets you enforce baseline rules (e.g. deny dev→prod) at the parent policy level across all environments, while child policies add environment-specific rules. Child policies cannot override parent deny rules.


Rule Processing Priority — the Full Picture

Priority 100 (lowest number = highest priority)
DNAT collection A → rules evaluated top to bottom
DNAT collection B
Priority 200
Network collection A → IP/port rules
Network collection B
Priority 300
Application collection A → FQDN / URL rules
Application collection B
Priority 65000 (built-in)
Allow Azure infrastructure FQDNs (IMDS, DNS, etc.)
Priority 65500 (built-in)
Implicit deny all

Rule collections within each type are evaluated by priority number — lowest number wins. Within a collection, rules are evaluated top to bottom and the first match wins.

Azure VPN Gateway: A Guide to Connection Types and Benefits

What is Azure VPN Gateway?

An Azure VPN Gateway is a managed network gateway service that sends encrypted traffic between an Azure Virtual Network and an on-premises location (or another Azure VNet) over the public internet using IPsec/IKE tunnels. It’s the primary service that bridges your on-premises network to Azure in a hub and spoke topology.


Three Connection Types

Site-to-Site (S2S) connects your entire on-premises network to Azure over an IPsec/IKE tunnel. Your on-premises VPN device (router or firewall) terminates the tunnel. This is the most common type used in hub and spoke.

Point-to-Site (P2S) connects individual remote devices (laptops, phones) directly to the Azure VNet. Uses OpenVPN, SSTP, or IKEv2 protocols. No on-premises device required — just a VPN client app.

VNet-to-VNet connects two Azure VNets in different regions using the same IPsec tunnel mechanism as S2S. For same-region connections, VNet peering is cheaper and faster — VNet-to-VNet is mainly used cross-region or across subscriptions/tenants.


How It Works Internally

On-premises VPN device
↓ IPsec/IKE tunnel (encrypted)
Azure VPN Gateway (2 VM instances in GatewaySubnet)
↓ internal routing
Hub VNet → UDR propagation → Spoke VNets

The gateway always deploys as two instances for high availability. You choose between active-passive (one standby, ~10s failover) or active-active (both instances forward traffic simultaneously, faster failover).


SKUs — Full Breakdown

SKUs are grouped into generations. Generation 2 is current and recommended for all new deployments.

Generation 1 (legacy — avoid for new deployments)

SKUMax throughputS2S tunnelsP2S connectionsBGPZone-redundant
Basic100 Mbps10128
VpnGw1650 Mbps30250
VpnGw21 Gbps30500
VpnGw31.25 Gbps301,000

Generation 2 (current — recommended)

SKUMax throughputS2S tunnelsP2S connectionsBGPZone-redundant
VpnGw1650 Mbps30250
VpnGw21 Gbps30500
VpnGw31.25 Gbps301,000
VpnGw45 Gbps1005,000
VpnGw510 Gbps10010,000
VpnGw1AZ650 Mbps30250
VpnGw2AZ1 Gbps30500
VpnGw3AZ1.25 Gbps301,000
VpnGw4AZ5 Gbps1005,000
VpnGw5AZ10 Gbps10010,000

The AZ suffix means the gateway is deployed across Availability Zones — its instances span physically separate datacentre buildings, protecting against a full zone failure. This is the right choice for production workloads with strict uptime requirements.


SKU Selection Guide

ScenarioRecommended SKU
Dev/test only, no BGP neededBasic
Small org, <30 branch officesVpnGw1AZ
Mid-size enterpriseVpnGw2AZ or VpnGw3AZ
Large enterprise, many tunnelsVpnGw4AZ
Very high throughput (10 Gbps)VpnGw5AZ
High SLA required in productionAny AZ SKU

Key Concepts to Know

BGP (Border Gateway Protocol) — enables dynamic route exchange between Azure and your on-premises router. Without BGP, you must manually define every on-premises subnet in the Local Network Gateway. With BGP, routes are exchanged automatically. Required for active-active configurations and most enterprise setups.

GatewaySubnet — a dedicated subnet in your hub VNet that must be named exactly GatewaySubnet. Minimum /27 (32 addresses), recommended /26 or larger for future gateway coexistence (VPN + ExpressRoute). No other resources should be placed in this subnet.

Local Network Gateway — an Azure resource that represents your on-premises VPN device. You define its public IP address and the address space of your on-premises network here.

Active-Active mode — both gateway instances are active simultaneously, each with its own public IP. Your on-premises VPN device must support two tunnels. Provides near-zero downtime failover and higher aggregate throughput.

IKE versions — the gateway supports IKEv1 and IKEv2. IKEv2 is preferred — it’s faster to negotiate, more secure, and required for P2S with IKEv2 clients.


VPN Gateway vs ExpressRoute Gateway

VPN GatewayExpressRoute Gateway
TransportPublic internet (encrypted)Private MPLS circuit (unencrypted at layer)
Max throughput10 Gbps (VpnGw5AZ)Up to 100 Gbps (UltraPerformance)
LatencyVariable (internet)Consistent, low latency
CostLowerHigher (circuit + gateway)
Use caseMost enterprisesFinancial, healthcare, high-compliance

In many enterprise deployments both coexist in the same GatewaySubnet — ExpressRoute as the primary path, VPN as the failover.