Understanding RHCOS Boot Process for OpenShift

techhadoop OCP

Booting new VMs with Red Hat Enterprise Linux CoreOS (RHCOS) is a critical part of the OpenShift installation process. During installation, the installer provisions the VMs and boots them from an immutable RHCOS image. These nodes then join the cluster and are managed automatically.

OpenShift Boot Process with RHCOS

                User runs openshift-install
                           │
                           ▼
               Ignition configuration generated
                           │
                           ▼
         Infrastructure creates new RHCOS VM
                           │
                           ▼
              VM boots immutable RHCOS image
                           │
                           ▼
             Ignition downloads machine config
                           │
                           ▼
          Configure hostname, networking, SSH
                           │
                           ▼
               kubelet service starts
                           │
                           ▼
        kubelet connects to Kubernetes API
                           │
                           ▼
      CSR generated and approved automatically
                           │
                           ▼
          Node joins the OpenShift cluster
                           │
                           ▼
Machine Config Operator manages the node

Step 1 – Installer Creates Ignition Files

The installer generates three Ignition configurations:

bootstrap.ign
master.ign
worker.ign

Example:

openshift-install create ignition-configs

Each Ignition file tells the node:

  • hostname
  • SSH keys
  • certificates
  • kubelet configuration
  • pull secret
  • networking
  • MachineConfig information

Step 2 – VM Boots RHCOS

The VM boots using the Red Hat CoreOS image.

Examples:

  • VMware
  • KVM
  • Hyper-V
  • AWS EC2
  • Azure VM
  • GCP Compute Engine

Unlike traditional Linux:

No kickstart
No cloud-init
No Ansible required

Instead it boots directly into RHCOS.

Step 3 – Ignition Runs (First Boot Only)

During the first boot:

systemd
      │
      ▼
Ignition

Ignition configures:

Filesystem

/var
/etc

Users

core

SSH Keys

Certificates

Networking

Kubelet configuration

CRI-O configuration

Machine Config

This happens only once.

Step 4 – Immutable Operating System

RHCOS is immutable.

/

is read-only.

Applications never modify the OS.

Instead they use

/var

Only Machine Config Operator changes the OS.

Step 5 – kubelet Starts

Systemd starts:

crio

then

kubelet

Example:

systemctl status kubelet

Output:

Active: active (running)

Step 6 – kubelet Contacts API Server

kubelet connects to

https://api.cluster.example.com:6443

It authenticates using bootstrap credentials.

Step 7 – CSR Created

Each node creates a Certificate Signing Request.

View pending requests:

oc get csr

Example:

csr-abc123 Pending
csr-def456 Pending

Normally OpenShift approves them automatically.

Approve manually:

oc adm certificate approve csr-abc123

Step 8 – Node Joins Cluster

Check:

oc get nodes

Example:

master-0 Ready
master-1 Ready
master-2 Ready
worker-0 Ready
worker-1 Ready

Step 9 – Machine Config Operator Takes Over

After joining:

Machine Config Operator

manages:

Kernel arguments

OS updates

Certificates

Kubelet config

CRI-O config

Network files

SSH keys

You never manually patch RHCOS.

What Actually Boots?

RHCOS includes:

Linux Kernel
systemd
CRI-O
kubelet
podman
rpm-ostree
Ignition
NetworkManager
SELinux
OpenShift components

Boot Sequence Inside RHCOS

BIOS / UEFI
      │
      ▼
GRUB
      │
      ▼
Linux Kernel
      │
      ▼
initramfs
      │
      ▼
Ignition
      │
      ▼
systemd
      │
      ▼
NetworkManager
      │
      ▼
CRI-O
      │
      ▼
kubelet
      │
      ▼
API Server
      │
      ▼
Node Ready

Troubleshooting RHCOS Boot

View Ignition logs
journalctl -b -u ignition
Check kubelet
journalctl -u kubelet
Check CRI-O
journalctl -u crio
Verify Machine Config
oc get mcp

Healthy output:

NAME     UPDATED
master   True
worker   True
Check Machine Config Daemon
oc get pods -n openshift-machine-config-operator
Check Ignition file retrieval

On the node:

journalctl -b | grep ignition
Check node status
oc describe node worker-0
Interview Answer (2-minute version)

In OpenShift, new VMs boot from an immutable Red Hat CoreOS image rather than a traditional Linux installation. During installation, the openshift-install utility generates Ignition configuration files for bootstrap, control plane, and worker nodes. On first boot, the Ignition service configures networking, SSH keys, certificates, kubelet, and other system settings. After that, systemd starts CRI-O and the kubelet, which contacts the Kubernetes API server, generates a certificate signing request (CSR), and joins the cluster once approved. From that point onward, the Machine Config Operator (MCO) manages all operating system configuration and updates using rpm-ostree, ensuring every RHCOS node remains consistent, immutable, and centrally managed. This design improves security, simplifies lifecycle management, and reduces configuration drift across the cluster.

Leave a Reply