Managing DNS in OpenShift (OCP): A Comprehensive Guide

techhadoop OCP

DNS Operator in OpenShift

The DNS Operator deploys, configures, and continuously manages CoreDNS, which provides DNS resolution and Kubernetes Service discovery for Pods inside an OpenShift cluster.

A simple way to remember it:

The DNS Operator manages the DNS platform; CoreDNS answers the DNS queries.

DNS Operator
     │
     ▼
CoreDNS DaemonSet
     │
     ▼
DNS Service IP
     │
     ▼
Pods resolve Services and external names

The DNS Operator is installed automatically during OpenShift installation. It runs as a Deployment, while CoreDNS runs as a DaemonSet managed by that Operator. (Red Hat Documentation)

DNS Operator vs CoreDNS

These are separate components:

ComponentResponsibility
DNS OperatorManages DNS configuration and CoreDNS lifecycle
CoreDNSProcesses DNS queries
DNS ServiceProvides a stable ClusterIP for DNS queries
kubeletConfigures each Pod to use the cluster DNS Service
Node resolverMaintains node hostname entries where required
DNS Operator
      │
      ▼
Creates and manages
      │
      ├── CoreDNS DaemonSet
      ├── DNS Service
      ├── CoreDNS ConfigMap
      └── Node resolver DaemonSet

The Operator implements the cluster-scoped DNS API in the operator.openshift.io API group. (Red Hat Documentation)

Where the Components Run

The DNS Operator normally runs in:

openshift-dns-operator

CoreDNS and node-resolver components normally run in:

openshift-dns

Check the Operator:

oc get deployment -n openshift-dns-operator

Check DNS Pods:

oc get pods -n openshift-dns -o wide

Check DaemonSets:

oc get daemonset -n openshift-dns

Typical output includes:

dns-default
node-resolver

Main DNS Architecture

Application Pod
      │
      │ DNS query
      ▼
Cluster DNS Service IP
      │
      ▼
CoreDNS Pod
      │
      ├── Internal cluster name?
      │       └── Answer from Kubernetes API data
      │
      └── External name?
              └── Forward to upstream DNS resolver

The default internal cluster domain is:

cluster.local

CoreDNS provides resolution for internal names such as:

service.namespace.svc.cluster.local

The default DNS resource is named default. (Red Hat Documentation)

Kubernetes Service Discovery

Suppose you create this Service:

apiVersion: v1
kind: Service
metadata:
  name: payments-api
  namespace: banking
spec:
  selector:
    app: payments-api
  ports:
    - port: 8443

CoreDNS creates DNS-based service discovery for it.

A Pod in the same namespace can use:

payments-api

A Pod in another namespace can use:

payments-api.banking

The complete DNS name is:

payments-api.banking.svc.cluster.local

Flow:

Client Pod
    │
    ▼
payments-api.banking.svc.cluster.local
    │
    ▼
CoreDNS
    │
    ▼
Service ClusterIP
    │
    ▼
Ready application Pods

Pod DNS Configuration

The kubelet configures each normal Pod to use the cluster DNS Service.

Inside a Pod:

cat /etc/resolv.conf

Typical content:

search banking.svc.cluster.local svc.cluster.local cluster.local
nameserver 172.30.0.10
options ndots:5

The exact DNS Service IP depends on the cluster Service network. OpenShift commonly assigns the DNS Service a stable IP from the Service CIDR. (Red Hat Documentation)

You can check it with:

oc get dns.operator/default -o yaml

Look under:

status:
  clusterDomain: cluster.local
  clusterIP: 172.30.0.10

Why CoreDNS Runs as a DaemonSet

CoreDNS normally runs through the dns-default DaemonSet.

Node 1          Node 2          Node 3
  │               │               │
CoreDNS         CoreDNS         CoreDNS

Benefits:

  • DNS capacity scales as nodes are added.
  • DNS is distributed across the cluster.
  • A single DNS Pod failure does not stop name resolution.
  • Requests can be handled close to workloads.
  • DNS remains available if individual nodes fail.

The DNS Operator creates the CoreDNS DaemonSet and exposes it through a Service with a stable IP. (Red Hat Documentation)

Internal Name Resolution

CoreDNS watches the Kubernetes API for Services, EndpointSlices, namespaces, and other relevant resources.

For a Service query:

Pod asks for:
payments-api.banking.svc.cluster.local
                │
                ▼
CoreDNS queries its Kubernetes data
                │
                ▼
Returns the Service ClusterIP

For a headless Service:

spec:
  clusterIP: None

CoreDNS can return Pod or endpoint IPs instead of one Service ClusterIP.

This is useful for:

  • StatefulSets
  • Databases
  • Cluster members
  • Direct endpoint discovery

External DNS Resolution

If a Pod asks for an external name:

api.example.com

CoreDNS normally forwards the query to an upstream resolver.

Application Pod
      │
      ▼
CoreDNS
      │
      ▼
Corporate or cloud DNS
      │
      ▼
External DNS result

Upstream resolvers normally come from node resolver configuration or from explicit DNS Operator settings.

DNS Forwarding

The DNS Operator supports forwarding particular DNS zones to specific resolvers.

Example requirement:

*.corp.bank.local
        │
        ▼
Corporate DNS servers

A simplified configuration is:

apiVersion: operator.openshift.io/v1
kind: DNS
metadata:
  name: default
spec:
  servers:
    - name: corporate-dns
      zones:
        - corp.bank.local
      forwardPlugin:
        upstreams:
          - 10.20.30.10
          - 10.20.30.11

Flow:

Query: database.corp.bank.local
              │
              ▼
CoreDNS matches corp.bank.local
              │
              ▼
Forwards to 10.20.30.10 / 10.20.30.11

Queries that do not match a configured zone fall back to the configured upstream resolvers. (Red Hat Documentation)

Edit DNS configuration with:

oc edit dns.operator/default

Do not manually edit the Operator-generated CoreDNS ConfigMap as the primary configuration method.

DNS Caching

CoreDNS caches successful and unsuccessful DNS responses.

This reduces:

  • Query latency
  • Load on upstream DNS servers
  • Repeated external lookups
  • Network traffic

OpenShift supports configuring positive and negative cache TTLs through the DNS Operator resource. (Red Hat Documentation)

Example:

apiVersion: operator.openshift.io/v1
kind: DNS
metadata:
  name: default
spec:
  cache:
    positiveTTL: 1h
    negativeTTL: 30s

Be careful with TTL tuning:

  • Very low TTLs increase DNS load.
  • Very high TTLs can retain stale results longer.
  • Negative caching can make a recently created record appear unavailable until the negative TTL expires.

Node Resolver

The DNS Operator also manages a node-resolver DaemonSet.

Check it with:

oc get daemonset node-resolver -n openshift-dns

The node-resolver component helps maintain node-level name resolution information, including managed entries in the node’s /etc/hosts where required.

DNS Operator
      │
      ├── dns-default DaemonSet
      │       └── Cluster DNS
      │
      └── node-resolver DaemonSet
              └── Node hostname resolution support

Reconciliation Loop

The DNS Operator continuously compares desired DNS configuration with the actual resources.

DNS resource: default
        │
        ▼
DNS Operator reads desired state
        │
        ▼
Checks DaemonSets, Service and ConfigMap
        │
        ▼
Difference found?
     ┌──┴──┐
     │     │
    No    Yes
     │     │
     ▼     ▼
  Wait   Recreate or update resources
               │
               ▼
         Validate DNS availability
               │
               ▼
        Update Operator status

Examples that trigger reconciliation:

  • CoreDNS Pod fails.
  • DNS forwarding configuration changes.
  • Cache settings change.
  • OpenShift is upgraded.
  • A managed ConfigMap changes.
  • Node placement is modified.
  • A DaemonSet does not match the desired configuration.

DNS Operator Health

Check the ClusterOperator:

oc get clusteroperator dns

Healthy output:

NAME   AVAILABLE   PROGRESSING   DEGRADED
dns    True        False         False

Interpretation:

ConditionMeaning
Available=TrueDNS service is operational
Progressing=TrueDNS resources are being changed
Degraded=TrueA DNS component or configuration is failing

Detailed information:

oc describe clusteroperator dns

The Operator considers DNS available when the DNS Service has a ClusterIP and at least one CoreDNS Pod is available. (Red Hat Documentation)

Useful Commands

Check the Operator
oc get deployment dns-operator \
  -n openshift-dns-operator
oc logs -n openshift-dns-operator \
  deployment/dns-operator
Check DNS configuration
oc get dns.operator/default -o yaml
oc describe dns.operator/default
Check CoreDNS Pods
oc get pods -n openshift-dns -o wide
Check DaemonSets
oc get daemonset -n openshift-dns
Check DNS Service
oc get service dns-default -n openshift-dns
Check generated configuration
oc get configmap dns-default \
  -n openshift-dns \
  -o yaml
Check events
oc get events -n openshift-dns \
  --sort-by='.lastTimestamp'

Testing DNS from a Pod

Create a temporary troubleshooting Pod:

oc run dns-test \
  --image=registry.access.redhat.com/ubi9/ubi-minimal \
  --restart=Never \
  -- sleep 3600

Enter it:

oc rsh dns-test

Check its resolver configuration:

cat /etc/resolv.conf

Test an internal Service:

getent hosts kubernetes.default.svc.cluster.local

Test your application:

getent hosts payments-api.banking.svc.cluster.local

Test an external address:

getent hosts example.com

Delete the Pod afterward:

oc delete pod dns-test

Troubleshooting Flow

Use this sequence:

Application DNS error
        │
        ▼
Check Pod /etc/resolv.conf
        │
        ▼
Test short and full service names
        │
        ▼
Check Service and EndpointSlices
        │
        ▼
Check DNS Service ClusterIP
        │
        ▼
Check CoreDNS Pods
        │
        ▼
Check DNS Operator status
        │
        ▼
Check CoreDNS and Operator logs
        │
        ▼
Check upstream DNS and network policies

Scenario 1: Internal Service Does Not Resolve

Example:

payments-api.banking.svc.cluster.local

Check the Service:

oc get svc payments-api -n banking

Check namespace and spelling:

oc get namespace banking

Test the full name:

getent hosts payments-api.banking.svc.cluster.local

Then check:

oc get pods -n openshift-dns
oc get svc dns-default -n openshift-dns

Remember:

  • DNS can resolve a Service even if its backend Pods are unhealthy.
  • DNS resolution does not prove the application itself is reachable.
  • EndpointSlices affect application traffic, not necessarily the existence of the Service DNS record.

Scenario 2: External Names Fail but Internal Names Work

For example:

payments-api.banking.svc.cluster.local → works
example.com                            → fails

This usually indicates an upstream-forwarding issue.

Check:

  • Upstream DNS server availability
  • DNS Operator forwarding configuration
  • Firewall access to UDP/TCP port 53
  • Node /etc/resolv.conf
  • Corporate DNS reachability
  • Egress restrictions
  • Forwarding loops

Review CoreDNS logs:

oc logs -n openshift-dns <dns-default-pod> \
  -c dns

Scenario 3: One Node Has DNS Problems

If Pods on one node fail DNS while other nodes work:

oc get pods -n openshift-dns -o wide

Check whether the affected node has a healthy CoreDNS Pod.

Then examine:

  • Node networking
  • OVN connectivity
  • DNS DaemonSet Pod
  • kubelet configuration
  • Service routing
  • Firewall rules
  • MTU problems
  • Node resource pressure

Test from Pods scheduled on both a healthy and affected node.

Scenario 4: DNS Query Is Slow

Possible causes:

  • Slow upstream resolver
  • DNS forwarding loop
  • Packet loss
  • Excessive query volume
  • Too-low cache TTL
  • CoreDNS CPU throttling
  • Node network problems
  • Search-domain expansion caused by ndots
  • External queries being tried as multiple internal names first

Measure lookup time:

time getent hosts external.example.com

Compare internal and external queries separately.

Scenario 5: CoreDNS Pods Are Pending

Check:

oc describe pod <dns-pod> -n openshift-dns

Possible causes:

  • Node selector mismatch
  • Missing toleration
  • Insufficient CPU or memory
  • Node taints
  • Scheduling restrictions
  • Image pull problem

CoreDNS and node-resolver placement can be controlled using node selectors and tolerations in the DNS Operator configuration. (Red Hat Documentation)

Common DNS Errors

Could not resolve host

Possible causes:

  • CoreDNS unavailable
  • Wrong Pod resolver configuration
  • Upstream DNS failure
  • NetworkPolicy or firewall blocking DNS
  • Typographical error

SERVFAIL

Possible causes:

  • Upstream resolver failure
  • DNS forwarding loop
  • Invalid zone configuration
  • DNSSEC or upstream issue
NXDOMAIN

Means the requested name does not exist according to the resolver.

Check:

  • Service name
  • Namespace
  • DNS zone
  • Negative cache
  • External record creation
DNS works but connection fails

DNS only returned an IP address. Check:

  • Service port
  • EndpointSlices
  • Pod readiness
  • NetworkPolicy
  • Application process
  • TLS configuration

DNS Operator vs DNS Operator Configuration

Avoid directly editing:

CoreDNS DaemonSet
dns-default Service
generated dns-default ConfigMap

These are Operator-managed and changes may be reverted.

Configure DNS using:

oc edit dns.operator/default

The Operator then generates the appropriate CoreDNS configuration and performs reconciliation.

Relationship with Other Operators

Cluster Version Operator
        │
        ▼
DNS Operator
        │
        ├── CoreDNS DaemonSet
        ├── Node Resolver DaemonSet
        ├── DNS Service
        └── DNS ConfigMap

Dependencies include:

ComponentRelationship
Network OperatorProvides connectivity to DNS Pods and Service IP
kubeletPlaces cluster DNS information into Pod resolver configuration
API ServerProvides Service and Endpoint data
Ingress OperatorDepends on external wildcard DNS for application routes
MonitoringCollects DNS component metrics and alerts
CVOInstalls and upgrades the DNS Operator

Interview Answer

The OpenShift DNS Operator deploys and manages CoreDNS to provide internal name resolution and Kubernetes Service discovery. The Operator runs as a Deployment in openshift-dns-operator, while it manages the dns-default CoreDNS DaemonSet, the node-resolver DaemonSet, a DNS Service with a stable ClusterIP, and the generated CoreDNS configuration in openshift-dns.

Pods send DNS queries to the DNS Service IP configured in their /etc/resolv.conf. CoreDNS resolves internal names such as service.namespace.svc.cluster.local using Kubernetes API information and forwards external or configured private-zone queries to upstream resolvers. The Operator continuously reconciles these resources and supports configuration through the cluster-scoped dns.operator/default object, including forwarding, caching, and node placement.

For troubleshooting, I start with oc get co dns, inspect the DNS Operator and CoreDNS Pods, check the dns-default Service and ConfigMap, test DNS from a Pod, and determine whether the failure affects internal names, external names, or only Pods on a particular node. I then check upstream resolvers, OVN connectivity, NetworkPolicies, firewall rules and CoreDNS logs.

Leave a Reply