Understanding Kubernetes API Server Operator in OpenShift

API Server Operator in OpenShift

The Kubernetes API Server Operator manages the lifecycle and configuration of the Kubernetes API server in OpenShift.

Its main responsibility is to make sure the kube-apiserver is:

Installed
Correctly configured
Highly available
Using valid certificates
Running the version required by OpenShift
Automatically recovered if a component fails
Safely rolled out during upgrades

The Operator runs in:

openshift-kube-apiserver-operator

It manages API server instances in:

openshift-kube-apiserver

The Operator is installed and updated through the Cluster Version Operator. Its cluster-scoped configuration resource is KubeAPIServer, named cluster. (Red Hat Documentation)

Where It Fits

			
Users, kubelets, Operators and controllers
                    │
                    ▼
        API load balancer :6443
                    │
       ┌────────────┼────────────┐
       ▼            ▼            ▼
   master-0      master-1      master-2
 kube-apiserver kube-apiserver kube-apiserver
       ▲            ▲            ▲
       └────────────┼────────────┘
                    │
       Kubernetes API Server Operator
                    │
                    ▼
       Configuration, rollout and health
                    │
                    ▼
                   etcd

		

The API server is the main entry point into the cluster. Commands such as:

			
oc get pods
oc create deployment
oc apply -f app.yaml

all pass through the Kubernetes API server.

Kubernetes API Server vs API Server Operator

These are different components.

Component	Purpose
`kube-apiserver`	Processes Kubernetes API requests
Kubernetes API Server Operator	Installs, configures and maintains `kube-apiserver`
OpenShift API Server	Provides OpenShift-specific APIs
OpenShift API Server Operator	Maintains the OpenShift API server

The flow is:

			
API Server Operator
        │
        ▼
Manages kube-apiserver
        │
        ▼
kube-apiserver handles API requests

		

Kubernetes API Server vs OpenShift API Server

OpenShift has two related API server layers.

Kubernetes API server

Handles standard Kubernetes resources:

Pods
Deployments
Services
Secrets
ConfigMaps
Nodes
RBAC
StatefulSets

OpenShift API server

Handles OpenShift-specific APIs, such as certain:

Projects
Routes-related platform integrations
Security and authorization extensions
OpenShift-specific resources

			
Client request
      │
      ▼
Kubernetes API aggregation layer
      │
      ├── Kubernetes APIs
      │
      └── OpenShift-specific APIs

		

The Kubernetes API Server Operator manages kube-apiserver, while the OpenShift API Server Operator installs and maintains openshift-apiserver. (Red Hat Documentation)

Main Responsibilities

1. Deploying API server static pods

On each control-plane node, the API server runs as a static pod.

			
/etc/kubernetes/manifests/
        │
        ▼
kubelet detects manifest
        │
        ▼
kube-apiserver pod starts

		

Typical API server pods can be viewed with:

oc get pods -n openshift-kube-apiserver -o wide

Example:

			
kube-apiserver-master-0
kube-apiserver-master-1
kube-apiserver-master-2

Static pods are controlled by the kubelet directly rather than by a Deployment.

2. Managing revisions

The Operator creates versioned API server revisions.

			
Revision 12
     │
     ▼
Configuration changed
     │
     ▼
Revision 13 created
     │
     ▼
Control-plane nodes updated gradually

		

You can see revision resources and related configuration in the API server namespace:

			
oc get configmaps -n openshift-kube-apiserver
oc get secrets -n openshift-kube-apiserver

Revision-based management allows the Operator to roll out a consistent configuration and diagnose which revision each node is running.

3. Rolling updates

The Operator avoids replacing every API server simultaneously.

			
master-0
Update → Ready
          │
          ▼
master-1
Update → Ready
          │
          ▼
master-2
Update → Ready

		

This preserves API availability as long as:

The load balancer has healthy backends.
Enough control-plane nodes remain available.
etcd retains quorum.
The new revision becomes healthy.

During an OpenShift upgrade, the CVO delivers the new release state and the API Server Operator reconciles the API server toward that version.

4. Certificate management

The API server requires several certificates for:

External API access
Internal API access
Communication with etcd
Authentication of clients
Communication with aggregated APIs
Service-network endpoints

The Operator helps manage and rotate API server certificates.

			
Certificate approaches rotation point
               │
               ▼
Operator creates updated certificate resources
               │
               ▼
New static-pod revision
               │
               ▼
API servers roll out incrementally

		

This reduces the chance of an API outage caused by expired certificates.

5. API server configuration

The cluster-scoped resource is:

oc get kubeapiserver cluster -o yaml

The corresponding API is:

			
operator.openshift.io/v1
kind: KubeAPIServer
metadata:
  name: cluster

The resource provides configuration for the Operator that manages kube-apiserver. (Red Hat Documentation)

A simplified example:

			
apiVersion: operator.openshift.io/v1
kind: KubeAPIServer
metadata:
  name: cluster
spec:
  audit:
    profile: Default

		

Do not add unsupported fields or manually edit generated static-pod manifests.

Use the supported cluster API:

oc edit kubeapiserver cluster

Red Hat identifies oc edit kubeapiserver as the configuration interface for the Kubernetes API Server Operator. (Red Hat Documentation)

6. Audit policy configuration

The Operator applies the configured API audit profile.

Audit records may include:

User identity
Service account identity
API resource
Operation or verb
Source IP
Request result
Timestamp

Example operations:

			
create
update
patch
delete
get
list

		

Audit configuration should be changed through the KubeAPIServer resource, not by directly modifying the API server static-pod command arguments.

7. etcd connectivity

The API server reads and writes cluster state in etcd.

			
oc request
    │
    ▼
kube-apiserver
    │
    ▼
Authentication and authorization
    │
    ▼
Admission controls
    │
    ▼
etcd read/write
    │
    ▼
Response

		

The Operator manages the API server configuration required to connect securely to etcd, but the etcd cluster itself is maintained by the etcd Operator.

A slow etcd backend directly affects API performance:

			
Slow etcd disk
      │
      ▼
Slow etcd transaction
      │
      ▼
Slow API response
      │
      ▼
Slow oc commands and controller reconciliation

		

8. Health monitoring

The Operator monitors whether the API server is:

Available
Progressing
Degraded
Running the expected revision
Responding to health checks

Check the ClusterOperator:

oc get clusteroperator kube-apiserver

Healthy status:

			
AVAILABLE     True
PROGRESSING   False
DEGRADED      False

Detailed information:

oc describe clusteroperator kube-apiserver

Reconciliation Process

The Operator continuously performs this loop:

			
Observe desired configuration
             │
             ▼
Inspect current API server revision
             │
             ▼
Compare desired and actual state
             │
       ┌─────┴─────┐
       │           │
   Matches      Difference
       │           │
       ▼           ▼
 No action    Create new revision
                   │
                   ▼
            Roll out static pods
                   │
                   ▼
              Check readiness
                   │
                   ▼
              Update status

		

Example: the audit profile changes.

			
Administrator updates KubeAPIServer CR
                    │
                    ▼
Operator notices configuration change
                    │
                    ▼
New revision generated
                    │
                    ▼
API server nodes updated incrementally
                    │
                    ▼
Operator reports Available

		

Request Processing Through the API Server

A request normally passes through several stages:

			
oc apply
   │
   ▼
Load balancer
   │
   ▼
kube-apiserver
   │
   ├── TLS validation
   ├── Authentication
   ├── Authorization
   ├── Admission control
   ├── Resource validation
   └── etcd persistence
          │
          ▼
       Response

		

For example:

oc create deployment nginx --image=nginx

The API server:

Authenticates the user.
Checks RBAC authorization.
Runs admission controls.
Validates the Deployment.
Stores it in etcd.
Returns the result.
Controllers later create the ReplicaSet and Pods.

The Operator maintains the API server that performs these steps; it does not process user API requests itself.

Useful Commands

Check ClusterOperator status

			
oc get co kube-apiserver
oc describe co kube-apiserver

Check Operator pods

oc get pods -n openshift-kube-apiserver-operator

Check Operator logs

			
oc logs -n openshift-kube-apiserver-operator \
  deployment/kube-apiserver-operator

Check API server pods

oc get pods -n openshift-kube-apiserver -o wide

Check a specific API server pod

			
oc describe pod -n openshift-kube-apiserver \
  kube-apiserver-master-0

Check API readiness

oc get --raw='/readyz?verbose'

Check API liveness

oc get --raw='/livez?verbose'

Check configuration

oc get kubeapiserver cluster -o yaml

Check recent events

			
oc get events -n openshift-kube-apiserver \
  --sort-by='.lastTimestamp'

Troubleshooting a Degraded API Server Operator

Use this sequence:

			
ClusterOperator
      ↓
Operator conditions
      ↓
Operator pod and logs
      ↓
API server static pods
      ↓
Node health
      ↓
etcd health
      ↓
Certificates and load balancer

		

Step 1: Check status

			
oc get co kube-apiserver
oc describe co kube-apiserver

Look at the condition messages under:

Available
Progressing
Degraded

The condition message usually points to the affected revision or node.

Step 2: Check the Operator

oc get pods -n openshift-kube-apiserver-operator

			
oc logs -n openshift-kube-apiserver-operator \
  deployment/kube-apiserver-operator \
  --since=1h

Look for:

Revision installation failure
Certificate errors
Missing ConfigMaps or Secrets
Static-pod rollout timeout
Node installer failure

Step 3: Check API server pods

oc get pods -n openshift-kube-apiserver -o wide

Check containers in a failing static pod:

			
oc describe pod -n openshift-kube-apiserver \
  <kube-apiserver-pod>

			
oc logs -n openshift-kube-apiserver \
  <kube-apiserver-pod> \
  -c kube-apiserver \
  --since=1h

Step 4: Check etcd

			
oc get co etcd
oc get pods -n openshift-etcd -o wide

API server symptoms can be caused by:

etcd disk latency
Lost etcd quorum
Slow network between control-plane nodes
etcd certificate errors
etcd database pressure

Step 5: Check the affected node

			
oc get nodes
oc describe node <master-node>

For host-level investigation:

			
oc debug node/<master-node>
chroot /host
systemctl status kubelet
journalctl -u kubelet --since "1 hour ago"

Step 6: Check the load balancer

Test the cluster endpoint:

curl -k https://api.<cluster-domain>:6443/readyz

Test each control-plane backend separately:

			
curl -k https://<master-0>:6443/readyz
curl -k https://<master-1>:6443/readyz
curl -k https://<master-2>:6443/readyz

Possible problems:

An unhealthy master remains in the pool.
Health check is incorrect.
Port 6443 is blocked.
TLS inspection interferes with API traffic.
DNS resolves to the wrong VIP.

What Not to Do

Avoid:

			
Editing static-pod manifests manually
Deleting API server certificates
Restarting all control-plane nodes together
Deleting revision resources without Red Hat guidance
Editing Operator-managed Deployments directly
Changing etcd data manually

		

Operator-managed resources will often be restored, and unsafe changes could make the API unavailable.

Always use supported configuration resources and preserve etcd quorum.

Relationship with Other Operators

			
Cluster Version Operator
          │
          ▼
Kubernetes API Server Operator
          │
          ├── works with etcd Operator
          ├── relies on Machine Config Operator
          ├── exposes health to Monitoring
          └── provides APIs used by all other Operators

		

Operator	Relationship
CVO	Installs and upgrades the API Server Operator
etcd Operator	Provides the persistent API backend
MCO	Maintains control-plane node OS configuration
Authentication Operator	Supports user login and OAuth flows
Monitoring	Scrapes API server and Operator metrics
Network Operator	Provides required control-plane networking

Interview Answer

The Kubernetes API Server Operator manages and updates the kube-apiserver instances running as static pods on the OpenShift control-plane nodes. It is installed through the Cluster Version Operator and continuously reconciles the API server’s desired configuration with its actual state. It manages versioned static-pod revisions, certificates, audit configuration, etcd connectivity and rolling updates across the control-plane nodes.

The API Server Operator itself does not process application requests; the kube-apiserver does that. The Operator makes sure the API servers remain correctly configured and highly available. During a configuration change or upgrade, it creates a new revision and rolls it out incrementally, checking readiness before completing the transition. For troubleshooting, I start with oc get co kube-apiserver, inspect its conditions, check the Operator logs and API server static pods, then validate etcd, control-plane node health, certificates and the external API load balancer.