For a Cluster Administrator role, the interview questions will pivot away from application development (like S2I or basic deployments) and focus heavily on infrastructure, cluster stability, day-2 operations, security, and underlying architecture.

Here are the high-yield, advanced OpenShift interview questions tailored specifically for a Cluster Administrator.

1. Installation, Infrastructure & Architecture

Q1: What is the difference between IPI (Installer-Provisioned Infrastructure) and UPI (User-Provisioned Infrastructure)? When would you choose one over the other?

• IPI (Full Automation): The OpenShift installer controls everything. It talks directly to the cloud provider API (like AWS, Azure, or vSphere), provisions the networking, load balancers, storage, virtual machines, and installs the cluster.
  • When to use: When you want a quick, standard, hands-off deployment and have full administrative rights to the underlying cloud/infra provider.
• UPI (Customized Control): The administrator manually provisions the infrastructure (compute, networking, storage, firewalls) ahead of time. The OpenShift installer is only used to generate the ignition files to boot the nodes.
  • When to use: Essential for strict enterprise environments. If you have complex pre-existing networking (DMZs, specific subnets), custom firewalls, disconnected (air-gapped) environments, or strict security governance where an installer cannot be given API access to create infrastructure.

Q2: What is Red Hat Enterprise Linux CoreOS (RHCOS), and why does OpenShift require it for the Control Plane?

RHCOS is a minimal, monolithic, container-optimized operating system.

• Immutability: The underlying host OS is read-only (except for /etc and /var). This prevents “configuration drift” where individual administrators make untracked manual changes to specific nodes.
• Managed by the Cluster: RHCOS is managed directly by the cluster itself via the Machine Config Operator (MCO). Upgrading OpenShift automatically upgrades the OS on the nodes. You treat nodes as cattle, not pets.
• Control Plane Requirement: OpenShift strictly requires RHCOS for master nodes to ensure total predictability, security, and atomic updates of the control plane.

2. Day-2 Operations & Upgrades

Q3: You are planning a cluster upgrade from version 4.x to 4.y. Walk me through your pre-requisites and execution steps.

A seasoned admin doesn’t just click “Upgrade”. The response should show a structured process:

1. Check the Upgrade Graph: Use the OpenShift Update Graph tool or oc adm upgrade to verify a valid, supported path exists between your current version and the target version.
2. Evaluate Operator Compatibility: Check the Operator Lifecycle Manager (OLM) to ensure all installed 3rd-party operators (e.g., databases, service meshes) are compatible with the target OpenShift version.
3. Verify Cluster Health: Ensure all ClusterOperators are Available=True, Progressing=False, and Degraded=False. Never upgrade a degraded cluster.
4. Backup the etcd Database: Take a manual etcd snapshot before initiating the upgrade (oc debug node/... -- chroot /host cluster-etcd-operator/etcd-snapshot-backup.sh).
5. Monitor Worker Node Capacity: Ensure there is enough spare capacity in the cluster. Because nodes are drained and rebooted sequentially during an upgrade, the remaining nodes must be able to handle the shifted workload.
6. Trigger and Monitor: Execute oc adm upgrade channel=<channel> then oc adm upgrade --to=<version>. Monitor via oc get clusterversion.

Q4: How do you handle a scenario where a Worker Node becomes NotReady due to disk pressure?

1. Identify the Culprit: Use oc describe node <node-name> to confirm DiskPressure is the active taint. Look at the conditions.
2. Determine the Cause: Access node metrics or use oc debug node/<node-name> to check if the issue is in /var/lib/containers (stuck/bloated container logs, uncleaned images) or a specific application writing local data.
3. Short-Term Remediation: * OpenShift’s Kubelet should automatically trigger garbage collection for unused images. If it fails, manual clearing of stopped containers or safe log rotation might be necessary.
   • Evict pods if necessary, though the DiskPressure taint should stop new pods from scheduling there.
4. Long-Term Root Cause Analysis:
   • Implement stricter log limits in application Console logging.
   • Adjust the evictionHard thresholds in the KubeletConfig to trigger garbage collection earlier.
   • Consider scaling up node disk sizes or adding more worker nodes.

3. Advanced Networking, Security & Storage

Q5: What is the default CNI for modern OpenShift 4 clusters, and how does it differ from its predecessor?

• Modern OpenShift clusters use OVN-Kubernetes (Open Virtual Network) as the default container network interface (CNI). It replaced OpenShift SDN.
• Key Advantages of OVN-K:
  • It supports dual-stack IPv4/IPv6 networking out of the box.
  • It includes native support for Kubernetes NetworkPolicies and advanced routing.
  • Better integration with hybrid cloud environments and massive scalability compared to the older OVS-based OpenShift SDN.

Q6: How would you secure a multi-tenant OpenShift cluster where Team A and Team B must share the same hardware but cannot communicate?

1. Network Isolation: Implement NetworkPolicies in each namespace. By default, namespaces can talk to each other. I would apply a default-deny policy for cross-namespace ingress traffic and explicitly whitelist only what is necessary.
2. RBAC (Role-Based Access Control): Bind Team A’s users to a specific ClusterRole (like admin or edit) scoped strictly to Team A’s namespaces using RoleBindings (not ClusterRoleBindings).
3. Resource Quotas and LimitRanges: Prevent one team from starving the other of resources. Apply ResourceQuotas to limit maximum CPU/Memory per namespace and LimitRanges to enforce default requests/limits on pods.
4. Security Context Constraints (SCC): Ensure both teams are locked into the default restricted-v2 SCC so neither team can escalate privileges to the underlying worker node.

4. Scenario-Based Troubleshooting (The “Hero” Questions)

Q7: The etcd leader crashes, or the etcd cluster loses quorum. How do you recover?

Losing etcd quorum is a critical P1 outage.

1. Assess the Damage: Check the status of the etcd pods using oc get pods -n openshift-etcd. If quorum is lost, the control plane APIs will stop responding (oc commands will timeout).
2. Locate the Backup: Hopefully, automated or recent manual etcd backups exist on a secure external server (as performed in Q3).
3. Perform an etcd Recovery: * You must access one of the Master nodes directly via SSH (using the core user and recovery keys).
   • Stop the etcd static pods.
   • Run the recovery script provided by OpenShift (/usr/local/bin/etcd-snapshot-restore.sh) initialization using the backup snapshot.
   • This script rewrites the etcd data directory and forces a single-node etcd cluster, allowing the other masters to slowly re-sync once the API comes back online.

Q8: A critical security vulnerability (CVE) is announced for the Linux Kernel. How do you patch all 100+ worker nodes in your cluster without disrupting application traffic?

You leverage the power of the Machine Config Operator (MCO):

1. Red Hat will release an updated RHCOS node image or a MachineConfig package containing the patch.
2. When the cluster updates its configuration via a new MachineConfig, the MCO takes over.
3. The MCO acts as a rolling controller: It selects a worker node, marks it as unschedulable (cordon), safely evicts all running pods (drain), applies the kernel patch/reboot, waits for it to come back to Ready status, and moves to the next node.
4. Because this happens sequentially, applications with proper replica counts and Pod Disruption Budgets (PDBs) suffer exactly zero downtime.

etcd recovery and MachineConfig concepts

Let’s break these two critical Cluster Administration concepts down to the architectural and operational level. Mastering etcd management and the Machine Config Operator (MCO) is what separates a junior administrator from a senior cluster engineer.

1. Deep Dive: etcd Recovery & Architecture

etcd is a distributed, consistent key-value store that acts as OpenShift’s single source of truth. Every single object—Pods, Routes, Secrets, CRDs, and ConfigMaps—is stored here.

The Quorum Mechanics

etcd uses the Raft consensus algorithm to ensure data consistency across the master nodes. To function, it requires a strict majority (quorum) of healthy members. The formula for quorum is:

{Quorum} = {N}/{2} + 1

Where N is the total number of members in the cluster.

• For a standard 3-master cluster, quorum is 3/2 + 1 = 2. You can lose 1 master without losing quorum.
• If 2 masters fail simultaneously, you have 1 node left. 1 < 2, so quorum is lost. The cluster API immediately locks up and stops responding.

Scenario: Step-by-Step Loss of Quorum Recovery

When quorum is broken, you cannot use oc commands because the API server depends on etcd. You must bypass the API entirely and interact directly with the master host operating system via SSH.

Step 1: Access a Surviving Master Node

SSH into one of the surviving master nodes using your cluster’s private SSH key:

Bash

ssh core@master-0.example.com

Step 2: Run the Backup Script (Pre-requisite Check)

Before recovering, ensure you actually have a valid snapshot. By default, backups are stored in /var/lib/etcd/ if configured, or a custom external path. A valid backup looks like a .db file accompanied by cluster metadata.

Step 3: Initiate the Single-Node Recovery

OpenShift provides a built-in recovery script located inside the cluster-etcd-operator container image, but it is exposed to the host path. Run the recovery script on the master node:

sudo -i
/usr/local/bin/etcd-snapshot-restore.sh /home/core/assets/backup/snapshot_v4.14.db

What this script does under the hood:

1. Stops the static pods: It moves the manifests for the etcd, kube-apiserver, and kube-controller-manager out of /etc/kubernetes/manifests/ so the Kubelet stops trying to run them.
2. Wipes existing data: It clears the corrupted/out-of-sync etcd data directory (/var/lib/etcd/).
3. Restores the snapshot: It unpacks your .db file into /var/lib/etcd/.
4. Rewrites the cluster membership: It modifies the etcd configuration to trick the node into believing it is a single-node cluster ($N=1$, meaning quorum = 1). It erases the metadata of the other dead masters.
5. Restarts the static pods: It moves the manifests back, forcing the API server to spin up using this new single-member database.

Step 4: Re-syncing the Other Masters

Once the API server is back up on master-0, use your local terminal again. The remaining master nodes (master-1 and master-2) will still be out of sync.

You don’t need to manually restore them. Instead, you force the Cluster Etcd Operator to redeploy them by clearing their old member states:

Bash

# Force etcd operator to regenerate secret keys and re-sync members
oc patch etcd/cluster --type=merge -p '{"spec": {"unsupportedConfigOverrides": {"useUnsupportedArchitectureControl": "true"}}}'

The operator will detect that master-1 and master-2 are missing from the current active state, pave their etcd directories automatically, and catch them up via Raft replication from master-0.

2. Deep Dive: The Machine Config Operator (MCO)

In vanilla Kubernetes, if you want to change a kernel parameter (sysctl), add an SSH key, or configure an enterprise container registry mirror on your worker nodes, you have to use external configuration management tools like Ansible, Puppet, or SaltStack.

OpenShift discards external tools entirely and uses the Machine Config Operator (MCO) to manage the operating system (RHCOS) natively through Kubernetes Custom Resources.

The MCO Component Hierarchy

To understand how a configuration change reaches a node, you must understand these 4 core objects:

1. MachineConfig (MC): A YAML file that outlines the exact state you want the OS to be in. It can contain files to write, systemd units to enable, or kernel settings to apply.
2. MachineConfigPool (MCP): A grouping of nodes that should receive the same configurations (typically mapped to roles, like master or worker).
3. Controller: Monitors the MachineConfigs and compiles them into a single, master “target” configuration for the entire pool.
4. Machine Config Daemon (MCD): A pod that runs as a DaemonSet on every single node in the cluster. It runs with root privileges (chroot /host) and is responsible for actually writing the changes to its local disk.

Scenario: Writing a Custom Security File to 100+ Nodes

Let’s look at how the MCO executes a change. Suppose your security team requires a corporate banner (/etc/issue) to be present on every worker node.

Step 1: Create the MachineConfig Object

The files inside a MachineConfig must be encoded in Ignition format (which uses URL-encoded or base64 text).

YAML

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker # Ties this config to the worker pool
  name: 99-worker-corporate-banner
spec:
  config:
    ignition:
      version: 3.2.0
    storage:
      files:
      - contents:
          source: data:,WARNING%3A%20Authorized%20Access%20Only%21%0A
        mode: 420 # Octal for file permissions (0644)
        path: /etc/issue

Step 2: The MCO Reconciliation Flow

When you apply this file (oc apply -f banner.yaml), the following chain reaction occurs:

1. Compilation: The MCO notices a new MachineConfig with the worker label. It takes this file and merges it with all existing worker configs into a new, single cryptographic hash string.
2. Pool Update: The MachineConfigPool for workers changes its status to UPDATING=True.
3. The Rollout Lifecycle (Node by Node):
   • The Machine Config Daemon (MCD) running on worker-0 notices the pool’s target hash no longer matches its local current hash.
   • The MCD requests the cluster to drain the node. The cluster cordons the node and gracefully evicts all workloads to other worker nodes.
   • Once empty, the MCD steps out of the container boundary using chroot, accesses the host file system, and writes the string WARNING: Authorized Access Only! into /etc/issue.
   • If the change requires a reboot (like a kernel parameter change), the MCD triggers a host reboot. If it doesn’t require a reboot (like our file change), it skips this step.
   • The MCD verifies the file exists and is correct, then uncordons the node, marking it Ready.
4. Next Node: The MCO moves to worker-1, repeating the exact same process.

How to Monitor This as an Administrator

During a massive rollout, you monitor the orchestration via the pools:

oc get mcp

Output example during update:

Plaintext

NAME     CONFIG                               UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT
master   rendered-master-1a2b3c...           True      False      False      3              3
worker   rendered-worker-4f5e6d...           False     True       False      100            45

If a node fails to apply a configuration (e.g., a systemd service fails to start), the pool will mark DEGRADED=True and immediately halt the entire rollout to prevent breaking the remaining 54 nodes.

If you were to encounter a Degraded MachineConfigPool in production, your immediate next step would be to check the logs of the specific daemon pod on the failing node.

OCP Interview Questions on OADP (OpenShift API for Data Protection)

Core Concepts

Q: What is OADP? OADP (OpenShift API for Data Protection) is a Red Hat operator built on top of Velero that provides backup, restore, and disaster recovery for OpenShift workloads. It backs up Kubernetes resources (manifests) and persistent volume data, storing them in object storage (S3-compatible).

Q: What are the core components of OADP?

Q: What Kubernetes/OpenShift resources does OADP back up?

• All K8s API objects: Deployments, Services, ConfigMaps, Secrets, PVCs, RBAC, etc.
• Namespace-scoped and cluster-scoped resources
• Persistent Volume data (via Restic/Kopia or CSI snapshots)
• Custom Resources (CRDs + CRs)

Q: What are the CRDs introduced by OADP?

DataProtectionApplication (DPA) Configuration

Q: Walk me through a DPA manifest.

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: dpa-sample
  namespace: openshift-adp          # OADP always installs here
spec:
  # --- Velero configuration ---
  configuration:
    velero:
      defaultPlugins:
        - openshift                 # Required for OCP — handles SCCs, routes, etc.
        - aws                       # Or gcp, azure, csi
        - csi                       # Required for CSI snapshot support
      resourceTimeout: 10m
      logLevel: info
      featureFlags:
        - EnableCSIVolumeSnapshots  # Required for CSI DataMover
    # --- File-level backup agent ---
    nodeAgent:                      # Was "restic" in older OADP versions
      enable: true
      uploaderType: kopia           # kopia (default 1.2+) or restic
      podConfig:
        resourceAllocations:
          limits:
            cpu: "2"
            memory: 2Gi
          requests:
            cpu: 500m
            memory: 256Mi
  # --- Object storage backend ---
  backupStorageLocations:
    - name: default
      provider: aws                 # aws | gcp | azure | oracle
      default: true
      objectStorage:
        bucket: my-oadp-bucket
        prefix: ocp-cluster-1      # Useful when sharing bucket across clusters
      credential:
        name: cloud-credentials    # Secret with cloud provider creds
        key: cloud
      config:
        region: us-east-1
        s3ForcePathStyle: "true"   # Required for MinIO/on-prem S3
        s3Url: https://minio.example.com  # For non-AWS S3
  # --- Volume snapshot backend (optional) ---
  volumeSnapshotLocations:
    - name: default
      provider: aws
      credential:
        name: cloud-credentials
        key: cloud
      config:
        region: us-east-1

Q: What is the openshift plugin and why is it required?

The openshift Velero plugin handles OpenShift-specific resources that vanilla Velero doesn’t understand:

• Strips build configs and image stream tags that shouldn’t be restored verbatim
• Preserves SCC assignments
• Handles routes and OpenShift-specific annotations
• Manages UID/GID range preservation across namespaces

Without it, restores may fail or produce broken workloads in OpenShift.

Backup

Q: How do you create a backup?

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: my-backup
  namespace: openshift-adp
spec:
  includedNamespaces:
    - my-app-namespace
  excludedNamespaces:
    - kube-system
  includedResources:              # Optional: specific resource types
    - deployments
    - services
    - persistentvolumeclaims
  excludedResources:
    - events
    - events.events.k8s.io
  labelSelector:                  # Only back up resources with this label
    matchLabels:
      app: my-app
  ttl: 720h                       # Retention — delete backup after 30 days
  storageLocation: default        # References BSL name
  snapshotVolumes: false          # Use Restic/Kopia instead of snapshots
  defaultVolumesToFsBackup: true  # File-level backup for ALL PVs
  hooks:                          # Pre/post backup hooks
    resources:
      - name: db-backup-hook
        includedNamespaces:
          - my-app-namespace
        labelSelector:
          matchLabels:
            app: postgres
        pre:
          - exec:
              container: postgres
              command:
                - /bin/bash
                - -c
                - pg_dump -U postgres mydb > /backup/dump.sql
              timeout: 5m

Trigger via CLI:

velero backup create my-backup \
  --include-namespaces my-app-namespace \
  --default-volumes-to-fs-backup \
  --ttl 720h

Q: What is the difference between snapshotVolumes and defaultVolumesToFsBackup?

Q: What are Backup Hooks and when do you use them?

Hooks are commands run inside containers before (pre) or after (post) backup. Common uses:

• Pre-hook: quiesce a database, flush cache, create consistent dump
• Post-hook: unquiesce, resume writes, clean up temp files

Restore

Q: How do you create a restore?

apiVersion: velero.io/v1
kind: Restore
metadata:
  name: my-restore
  namespace: openshift-adp
spec:
  backupName: my-backup           # Must reference an existing Backup object
  includedNamespaces:
    - my-app-namespace
  excludedResources:
    - nodes
    - events
    - persistentvolumes           # Let OCP provision new PVs
  namespaceMapping:
    my-app-namespace: my-app-restored   # Restore into a DIFFERENT namespace
  restorePVs: true
  preserveNodePorts: false        # Don't restore specific NodePort numbers
  existingResourcePolicy: update  # none | update — what to do if resource exists

CLI:

velero restore create --from-backup my-backup \
  --include-namespaces my-app-namespace \
  --namespace-mappings my-app-namespace:my-app-restored

Q: What is namespaceMapping and when is it useful?

It maps source namespace → target namespace during restore. Use cases:

• Restoring to a different environment (prod backup → staging)
• Restoring into a new namespace without overwriting existing workloads
• Cloning an application for testing

Q: What resources are excluded by default during restore?

• nodes
• events
• PersistentVolumes (cluster-scoped; OCP re-provisions via PVC)
• Velero/OADP’s own CRDs
• Resources with annotation velero.io/exclude-from-backup: "true"

Scheduling

Q: How do you schedule recurring backups?

apiVersion: velero.io/v1
kind: Schedule
metadata:
  name: daily-backup
  namespace: openshift-adp
spec:
  schedule: "0 2 * * *"           # Cron — 2AM daily
  template:
    includedNamespaces:
      - my-app-namespace
    ttl: 168h                     # Keep 7 days of backups
    defaultVolumesToFsBackup: true
    storageLocation: default
  useOwnerReferencesInBackup: false

# Check scheduled backups
oc get schedule -n openshift-adp
velero schedule get

Troubleshooting

Q: A backup is stuck in InProgress. What do you check?

# 1. Describe the backup object
oc describe backup my-backup -n openshift-adp
# 2. Check Velero pod logs
oc logs -n openshift-adp -l app.kubernetes.io/name=velero
# 3. Check node-agent (Kopia/Restic) pod logs on affected node
oc logs -n openshift-adp -l app.kubernetes.io/name=node-agent
# 4. Check DataUpload objects (CSI path)
oc get dataupload -n openshift-adp
# 5. Check BSL connectivity
oc describe backupstoragelocation default -n openshift-adp

Q: A restore fails with “namespace already exists.” What do you do?

Options:

• Use namespaceMapping to restore into a new namespace
• Delete the existing namespace first (if safe)
• Set existingResourcePolicy: update to overwrite existing resources
• Use --existing-resource-policy=update in the CLI

Q: PV data is not being restored. What do you check?

# 1. Was the backup taken with volume backup enabled?
oc get backup my-backup -o yaml | grep -E "defaultVolumesToFsBackup|snapshotVolumes"
# 2. Check BackupRepository status
oc get backuprepository -n openshift-adp
oc describe backuprepository <name> -n openshift-adp
# 3. Check DataDownload objects
oc get datadownload -n openshift-adp
# 4. Verify PVC is in the restore spec (restorePVs: true)
# 5. Check node-agent logs
oc logs -n openshift-adp -l app.kubernetes.io/name=node-agent -f

Common Errors & Fixes

Quick Reference Commands

# OADP operator and pod status
oc get pods -n openshift-adp
# All backup objects
oc get backup,restore,schedule,bsl,vsl -n openshift-adp
# Backup details and phase
velero backup describe my-backup --details
# Restore logs
velero restore logs my-restore
# BSL status
velero backup-location get
# Force BSL re-validation
oc patch backupstoragelocation default \
  -n openshift-adp \
  --type merge \
  -p '{"spec":{"accessMode":"ReadWrite"}}'
# Delete a failed backup and retry
velero backup delete my-backup

Q: What is the difference between OADP 1.1 and 1.2+?

OCP Interview Questions on SCC (Security Context Constraints)

Here’s a comprehensive set of interview Q&A on SCCs in OpenShift Container Platform:

Core Concepts

Q: What is an SCC in OpenShift? A Security Context Constraint (SCC) is an OpenShift-specific resource that controls what permissions a pod can request at runtime — things like running as root, using host networking, mounting certain volume types, and more. SCCs are similar to Kubernetes PodSecurityAdmission but more granular and flexible.

Q: How does SCC differ from Kubernetes RBAC? RBAC controls what API operations a user/service account can perform (create, get, delete resources). SCC controls what a pod can do at runtime — its security posture on the node. Both are needed; they complement each other.

Q: What are the built-in SCCs in OpenShift?

Assignment & Priority

Q: How does OpenShift assign an SCC to a pod? OpenShift evaluates all SCCs available to the pod’s service account (via RBAC), then picks the lowest-priority SCC that can validate the pod’s security requirements. If multiple SCCs have the same priority, it picks alphabetically.

Q: How do you grant an SCC to a service account?

oc adm policy add-scc-to-user anyuid -z my-serviceaccount -n my-namespace

This creates a RoleBinding (or ClusterRoleBinding) that allows the service account to use the specified SCC.

Q: What is SCC priority and why does it matter? Each SCC has a numeric priority field (higher number = higher priority). When multiple SCCs could satisfy a pod, the one with the highest priority wins. This lets you promote a custom SCC above built-ins without renaming them.

Common Scenarios

Q: A pod fails with “unable to validate against any security context constraint.” What do you do?

1. Check the pod’s service account: oc get pod <pod> -o yaml | grep serviceAccount
2. List SCCs available to it: oc get rolebindings,clusterrolebindings -o yaml | grep <sa-name>
3. Check what the pod is requesting (runAsUser, capabilities, volumes, etc.)
4. Use: oc adm policy who-can use scc/<scc-name>
5. Either adjust the workload spec or grant the appropriate SCC

Q: How do you check which SCC was applied to a running pod?

oc get pod <pod-name> -o yaml | grep openshift.io/scc

The annotation openshift.io/scc: <scc-name> shows the applied SCC.

Q: When would you create a custom SCC instead of using built-ins? When built-ins are either too permissive or too restrictive for your use case. For example, you need a specific UID range, specific Linux capabilities (like NET_BIND_SERVICE), or specific volume types — but don’t want to grant full anyuid or privileged.

Security & Best Practices

Q: What is the principle of least privilege for SCCs? Always grant the most restrictive SCC that still allows the workload to function. Avoid privileged or anyuid unless absolutely necessary. Use service accounts per application, not the default SA.

Q: What’s the risk of granting anyuid to a service account? It allows pods to run as root (UID 0), which means a container escape could give an attacker root on the node. It also allows running as any arbitrary UID, bypassing namespace UID isolation.

Q: How do SCCs interact with Pod Security Admission (PSA) in newer OCP versions? OCP 4.11+ introduced restricted-v2 aligned with Kubernetes PSA standards. SCCs remain the primary enforcement mechanism in OCP, but namespaces can also have PSA labels. OCP’s SCC system takes precedence — if a pod passes SCC validation, PSA warnings may still appear but won’t block the pod by default.

Quick-Fire Questions

• Can a user with cluster-admin bypass SCCs? Yes — cluster-admin is granted the privileged SCC.
• Are SCCs namespaced? No, SCCs are cluster-scoped resources, but their assignment via RoleBindings can be namespace-scoped.
• What volume types does restricted SCC allow? ConfigMap, downwardAPI, emptyDir, persistentVolumeClaim, projected, secret.
• What command lists all SCCs? oc get scc
• How do you describe an SCC? oc describe scc restricted

SCC Creation & Troubleshooting — Deep Dive

Custom SCC Creation

When to Create a Custom SCC

• Built-in SCCs are too permissive (security risk) or too restrictive (app won’t run)
• Need a specific UID/GID range for a workload
• Need specific Linux capabilities without granting full privileged
• Need specific volume types not allowed by restricted

SCC Manifest — Annotated

apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
metadata:
  name: my-custom-scc
  annotations:
    kubernetes.io/description: "Custom SCC for my-app — allows NET_BIND_SERVICE only"
# --- Priority ---
priority: 10                        # Higher than restricted(null=0); lower than privileged(200)
# --- User Control ---
runAsUser:
  type: MustRunAsRange              # Options: MustRunAs, MustRunAsNonRoot, MustRunAsRange, RunAsAny
  uidRangeMin: 1000
  uidRangeMax: 65535
# --- Group Control ---
fsGroup:
  type: MustRunAs
  ranges:
    - min: 1000
      max: 65535
supplementalGroups:
  type: RunAsAny                    # Or MustRunAs with ranges
# --- Privilege Escalation ---
allowPrivilegeEscalation: false     # Prevents sudo/setuid inside container
defaultAllowPrivilegeEscalation: false
allowPrivilegedContainer: false     # Never allow --privileged
# --- Linux Capabilities ---
allowedCapabilities: []             # Caps pods can REQUEST
defaultAddCapabilities:
  - NET_BIND_SERVICE                # Always added to every pod using this SCC
requiredDropCapabilities:
  - ALL                             # Drop everything first, then add back selectively
# --- Host Access ---
allowHostDirVolumePlugin: false
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowHostPorts: false
# --- Volumes ---
volumes:
  - configMap
  - downwardAPI
  - emptyDir
  - persistentVolumeClaim
  - projected
  - secret
  # - hostPath  ← only add if truly needed
# --- Seccomp / AppArmor ---
seccompProfiles:
  - runtime/default
# annotations for AppArmor:
# apparmor.security.beta.kubernetes.io/allowedProfiles: runtime/default
# --- SELinux ---
seLinuxContext:
  type: MustRunAs                   # Enforces SELinux label
  # seLinuxOptions:
  #   level: "s0:c123,c456"        # Uncomment for specific MCS label
# --- Read-only root filesystem ---
readOnlyRootFilesystem: false       # Set true for hardened workloads
# --- Users/Groups allowed to use this SCC ---
users: []                           # Specific users (avoid; prefer service accounts)
groups: []                          # Specific groups

Apply it:

oc apply -f my-custom-scc.yaml

Grant the SCC to a Service Account

# Namespace-scoped (preferred)
oc adm policy add-scc-to-user my-custom-scc \
  -z my-app-serviceaccount \
  -n my-namespace
# What this actually creates under the hood:
oc get rolebinding -n my-namespace | grep my-custom-scc

Verify:

oc describe sa my-app-serviceaccount -n my-namespace
oc get rolebindings -n my-namespace -o yaml | grep -A5 my-custom-scc

runAsUser Types — Cheat Sheet

Troubleshooting SCCs

The Troubleshooting Mental Model

Pod fails to schedule/start
        │
        ├─ Check Events → "unable to validate against any SCC"
        │         │
        │         ├─ What is the pod REQUESTING? (securityContext fields)
        │         └─ What SCCs are AVAILABLE to the service account?
        │
        ├─ Check annotation → which SCC was actually applied?
        │
        └─ Gap between what pod requests vs what SCC allows

Step-by-Step Troubleshooting Workflow

Step 1 — Read the error event

oc describe pod <pod-name> -n <namespace>
# Look for: "unable to validate against any security context constraint"
# The message lists WHICH fields failed and WHY

Example error output:

unable to validate against any security context constraint: 
[provider restricted: .spec.securityContext.runAsUser: Invalid value: 0: 
must be in the ranges: [1000640000, 1000649999]]

This tells you exactly: runAsUser: 0 was requested, but the SCC only allows 1000640000–1000649999.

Step 2 — Identify the service account

oc get pod <pod-name> -o jsonpath='{.spec.serviceAccountName}'
# Default if blank: "default"

Step 3 — Check what SCCs the SA can use

# Method 1: Check who can use a specific SCC
oc adm policy who-can use scc/restricted
oc adm policy who-can use scc/anyuid
# Method 2: Check all bindings for the SA
oc get clusterrolebindings,rolebindings -A -o yaml \
  | grep -B5 -A10 "my-app-serviceaccount"
# Method 3: Direct check (OCP 4.x)
oc get clusterrolebindings -o json | \
  jq '.items[] | select(.subjects[]?.name=="my-app-serviceaccount")'

Step 4 — Inspect what the pod is requesting

oc get pod <pod-name> -o yaml | grep -A20 securityContext

Key fields to check:

securityContext:
  runAsUser: 0           # ← root? needs anyuid or privileged
  runAsGroup: 0
  fsGroup: 0
  privileged: true       # ← needs privileged SCC
  capabilities:
    add: ["NET_ADMIN"]   # ← needs allowedCapabilities
  hostNetwork: true      # ← needs hostnetwork SCC
  hostPID: true          # ← needs privileged SCC

Step 5 — Check which SCC was actually applied (running pod)

oc get pod <pod-name> -o jsonpath='{.metadata.annotations.openshift\.io/scc}'

If it says restricted but you expected my-custom-scc, check priority and RBAC binding.

Step 6 — Simulate SCC admission (dry-run)

# Check if a service account can create a pod with a given SCC
oc auth can-i use scc/my-custom-scc \
  --as=system:serviceaccount:my-namespace:my-app-sa

Common Errors & Fixes

Useful Diagnostic Commands — Quick Reference

# List all SCCs with priorities
oc get scc -o custom-columns=\
NAME:.metadata.name,\
PRIORITY:.priority,\
RUNASUSER:.runAsUser.type,\
FSGROUP:.fsGroup.type
# Full details of an SCC
oc describe scc my-custom-scc
# Who can use a given SCC
oc adm policy who-can use scc/anyuid
# Which SCC is on a running pod
oc get pod <pod> -o jsonpath='{.metadata.annotations.openshift\.io/scc}'
# All pods in a namespace and their SCCs
oc get pods -n <ns> -o json | \
  jq '.items[] | {name: .metadata.name, scc: .metadata.annotations["openshift.io/scc"]}'
# Remove an SCC from a service account
oc adm policy remove-scc-from-user anyuid -z my-sa -n my-namespace
# Audit: find all SAs with privileged SCC
oc get clusterrolebindings -o json | \
  jq '.items[] | select(.roleRef.name=="system:openshift:scc:privileged")'

Pro Tips for Interviews

• Never grant privileged or anyuid cluster-wide — always scope to a specific SA in a specific namespace via RoleBinding, not ClusterRoleBinding
• Custom SCCs > modifying built-ins — built-in SCCs can be reset by OCP upgrades
• Use requiredDropCapabilities: [ALL] + selectively add back — defense in depth
• The annotation openshift.io/scc is your best friend for verifying what’s actually running
• SCC priority ties go alphabetical — name your custom SCC carefully if priority matters

Security Context Constraints (SCC) in OpenShift

SCCs are OpenShift’s mechanism for controlling what a pod is allowed to do at the OS and kernel level — think of them as a security policy that sits between your pod spec and the Linux kernel.

They are OpenShift’s equivalent of Kubernetes’ PodSecurity Admission (PSA), but significantly more powerful and flexible.

The Core Problem SCCs Solve

Without SCCs, any pod could potentially:

• Run as root
• Mount any host path
• Use host networking
• Load kernel modules
• Escape the container boundary

SCCs define the boundary of what’s allowed.

Developer submits Pod
        ↓
OpenShift Admission Controller
        ↓  checks
   Does the pod's requested
   capabilities fit within
   an SCC the pod's SA has?
        ↓ yes          ↓ no
   Pod scheduled    Pod rejected

Built-in SCCs

OpenShift ships with several predefined SCCs, ordered from most to least restrictive:

Most pods should run under restricted or restricted-v2. Granting privileged should be rare and deliberate.

Anatomy of an SCC

apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
metadata:
  name: my-custom-scc
# ── User / Group control ──────────────────────────────────────────
allowedUsers: []
allowedGroups: []
runAsUser:
  type: MustRunAsRange      # MustRunAsNonRoot | MustRunAs | RunAsAny
  uidRangeMin: 1000
  uidRangeMax: 65535
seLinuxContext:
  type: MustRunAs           # RunAsAny | MustRunAs
  seLinuxOptions:
    level: "s0:c123,c456"
# ── Capabilities ──────────────────────────────────────────────────
allowPrivilegeEscalation: false
allowPrivilegedContainer: false
defaultAddCapabilities: []
requiredDropCapabilities:
  - ALL                     # drop every Linux capability by default
allowedCapabilities:
  - NET_BIND_SERVICE        # only re-add what you need
# ── Volume control ────────────────────────────────────────────────
volumes:
  - configMap
  - secret
  - emptyDir
  - persistentVolumeClaim
  # - hostPath  ← only add if truly needed
allowHostDirVolumePlugin: false
allowHostNetwork:          false
allowHostPID:              false
allowHostIPC:              false
allowHostPorts:            false
# ── Filesystem ────────────────────────────────────────────────────
readOnlyRootFilesystem:    false
fsGroup:
  type: MustRunAs
  ranges:
    - min: 1000
      max: 65535
supplementalGroups:
  type: MustRunAs
  ranges:
    - min: 1000
      max: 65535
# ── Who can use this SCC ─────────────────────────────────────────
users:
  - system:serviceaccount:my-namespace:my-sa
groups:
  - system:authenticated

How OpenShift Assigns SCCs

When a pod is created, the admission controller runs through this process:

1. Collect all SCCs available to the pod's Service Account
        ↓
2. Sort SCCs by priority (higher priority checked first)
        ↓
3. For each SCC (most → least restrictive):
   Can this pod's spec be satisfied by this SCC?
        ↓ yes
4. Mutate the pod spec to conform (e.g. inject UID range)
        ↓
5. Annotate pod with the SCC used:
   openshift.io/scc: restricted

The annotation tells you which SCC won:

kubectl get pod my-pod -o jsonpath='{.metadata.annotations.openshift\.io/scc}'
# → restricted

runAsUser Strategies

This is one of the most important settings:

OpenShift namespaces have a UID range annotation that restricted SCC uses automatically:

kubectl get namespace my-namespace \
  -o jsonpath='{.metadata.annotations.openshift\.io/sa\.scc\.uid-range}'
# → 1000700000/10000
# meaning UIDs 1000700000 – 1000709999 are valid for this namespace

Pods in restricted are assigned a random UID from this range — so your app must not assume it runs as a specific UID.

Granting an SCC to a Service Account

The most common operation you’ll do:

# Create a dedicated service account
kubectl create serviceaccount my-app-sa -n my-namespace
# Grant an SCC to it
oc adm policy add-scc-to-user anyuid \
  -z my-app-sa \
  -n my-namespace
# Or grant to a group
oc adm policy add-scc-to-group nonroot \
  system:serviceaccounts:my-namespace

Then reference the SA in your deployment:

spec:
  template:
    spec:
      serviceAccountName: my-app-sa
      containers:
        - name: app
          image: my-image

Writing a Custom SCC

Say you have an app that needs to bind to port 80 (requires NET_BIND_SERVICE) but nothing else special:

apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
metadata:
  name: net-bind-scc
  annotations:
    kubernetes.io/description: "Allows NET_BIND_SERVICE only"
allowPrivilegedContainer: false
allowPrivilegeEscalation: false
runAsUser:
  type: MustRunAsNonRoot
seLinuxContext:
  type: MustRunAs
requiredDropCapabilities:
  - ALL
allowedCapabilities:
  - NET_BIND_SERVICE
volumes:
  - configMap
  - secret
  - emptyDir
  - persistentVolumeClaim
allowHostNetwork: false
allowHostPID:     false
allowHostIPC:     false
fsGroup:
  type: RunAsAny
users:
  - system:serviceaccount:my-namespace:my-app-sa

kubectl apply -f net-bind-scc.yaml

Common Scenarios & Solutions

Legacy app runs as root

# Least bad option — grant anyuid only to its SA
oc adm policy add-scc-to-user anyuid -z legacy-app-sa -n my-namespace

Better long-term: fix the image to run as a non-root UID.

App writes to /tmp but gets permission denied

The restricted SCC assigns a random UID — if the image has /tmp owned by root, the random UID can’t write. Fix the Dockerfile:

RUN chmod 1777 /tmp \
 && chown -R 1001:0 /app \
 && chmod -R g=u /app
USER 1001

The g=u trick makes the group permissions match user permissions — OpenShift always runs with GID 0, so this ensures the random UID can still write.

Prometheus node exporter needs host access

oc adm policy add-scc-to-user hostaccess \
  -z prometheus-node-exporter \
  -n monitoring

Init container needs to set sysctl

securityContext:
  sysctls:
    - name: net.core.somaxconn
      value: "1024"

You also need an SCC with allowedUnsafeSysctls or the sysctl listed under forbiddenSysctls removed.

SCC vs Kubernetes PodSecurity Admission

SCCs predate PSA and are more powerful. OpenShift also supports PSA alongside SCCs in newer versions.

Debugging SCC Issues

# See which SCC a running pod got
oc get pod my-pod -o jsonpath='{.metadata.annotations.openshift\.io/scc}'
# Simulate which SCC would be used (dry run)
oc adm policy scc-review -z my-sa -n my-namespace
# Check what SCCs a service account can use
oc adm policy who-can use scc restricted
# See all SCCs sorted by priority
oc get scc --sort-by=.priority
# Describe an SCC in full
oc describe scc restricted
# Events when a pod is rejected
kubectl get events -n my-namespace --field-selector reason=FailedCreate

Best Practices

1. Never grant privileged or anyuid cluster-wide — scope to a specific SA in a specific namespace
2. Drop ALL capabilities, then add back only what you need — principle of least privilege
3. Fix images instead of granting wider SCCs — chown/chmod in Dockerfile, use non-root USER
4. Use dedicated service accounts per app — never use default
5. Audit regularly — oc get scc and review who has privileged or anyuid
6. Prefer custom SCCs over broad built-ins — create a tailored SCC rather than granting anyuid just because it’s easy
7. Use restricted-v2 as your baseline in OCP 4.11+ — it’s stricter and aligns with upstream PSA

Key Takeaways

• SCCs are OpenShift’s pod-level security policy — more powerful than Kubernetes PSA
• Every pod gets an SCC — the most restrictive one it qualifies for wins
• Grant SCCs to service accounts, not to pods directly
• restricted / restricted-v2 should be the default — escalate only with justification
• Fix your images rather than loosening SCCs wherever possible
• Debug with oc adm policy scc-review and pod annotations