Comprehensive Guide to OCP Cluster Topology

techhadoop OCP

Cluster topology overview (physical and logical layout of the full OCP enterprise cluster)

Diagram 2 – Networking & security zones (SDN, NetworkPolicy, and multi-tenancy boundaries)

Diagram 3 : Day-2 GitOps and CI/CD pipeline (how code flows from Git to production)

Now let’s deploy. Here are the complete manifests and scripts covering every layer of the architecture above.

Deployment playbook

Prerequisites checklist

Before running anything, confirm:

  • Pull secret from cloud.redhat.com downloaded as pull-secret.json
  • DNS wildcard *.apps.<cluster>.<domain> and api.<cluster>.<domain> resolving
  • 3 master nodes: 8 vCPU / 32 GB RAM / 120 GB OS disk each
  • Minimum 3 worker nodes: 8 vCPU / 32 GB RAM / 100 GB OS disk + data disks
  • NTP synchronized across all nodes
  • openshift-install, oc, and helm binaries installed
1 · Install config — install-config.yaml
apiVersion: v1
baseDomain: example.com
metadata:
  name: prod-cluster
controlPlane:
  architecture: amd64
  hyperthreading: Enabled
  name: master
  replicas: 3
  platform:
    vsphere:                          # swap for aws/azure/baremetal as needed
      cpus: 8
      coresPerSocket: 4
      memoryMB: 32768
      osDisk:
        diskSizeGB: 120
compute:
  - architecture: amd64
    hyperthreading: Enabled
    name: worker
    replicas: 3
    platform:
      vsphere:
        cpus: 8
        coresPerSocket: 4
        memoryMB: 32768
        osDisk:
          diskSizeGB: 100
networking:
  clusterNetwork:
    - cidr: 10.128.0.0/14
      hostPrefix: 23
  machineNetwork:
    - cidr: 192.168.10.0/24
  networkType: OVNKubernetes
  serviceNetwork:
    - 172.30.0.0/16
platform:
  vsphere:
    vcenter: vcenter.example.com
    username: administrator@vsphere.local
    password: "{{ VCENTER_PASSWORD }}"
    datacenter: DC1
    defaultDatastore: datastore1
    folder: /DC1/vm/OCP
    network: VM Network
pullSecret: '{{ PULL_SECRET }}'
sshKey: '{{ SSH_PUBLIC_KEY }}'
fips: false
# Bootstrap the cluster
openshift-install create cluster --dir ./install-config --log-level=info
2 · Infrastructure MachineConfigPool — move platform workloads off workers
# infra-mcp.yaml
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
  name: infra
spec:
  machineConfigSelector:
    matchExpressions:
      - key: machineconfiguration.openshift.io/role
        operator: In
        values: [worker, infra]
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/infra: ""
# Label your infrastructure nodes
oc label node infra-node-1 infra-node-2 infra-node-3 \
    node-role.kubernetes.io/infra=""
# Apply the MachineConfigPool
oc apply -f infra-mcp.yaml
# Move the ingress controller to infra nodes
oc patch ingresscontroller/default -n openshift-ingress-operator \
  --type=merge -p '{
    "spec": {
      "nodePlacement": {
        "nodeSelector": {
          "matchLabels": {"node-role.kubernetes.io/infra": ""}
        },
        "tolerations": [{
          "key": "node-role.kubernetes.io/infra",
          "effect": "NoSchedule"
        }]
      }
    }
  }'
# Move the internal image registry to infra nodes
oc patch config/cluster -n openshift-image-registry \
  --type=merge -p '{
    "spec": {
      "nodeSelector": {"node-role.kubernetes.io/infra": ""},
      "tolerations": [{
        "key": "node-role.kubernetes.io/infra",
        "effect": "NoSchedule"
      }]
    }
  }'
3 · Cluster autoscaler + MachineAutoscaler
# cluster-autoscaler.yaml
apiVersion: autoscaling.openshift.io/v1
kind: ClusterAutoscaler
metadata:
  name: default
spec:
  resourceLimits:
    maxNodesTotal: 24
    cores:
      min: 8
      max: 192
    memory:
      min: 4
      max: 768
  scaleDown:
    enabled: true
    delayAfterAdd: 10m
    delayAfterDelete: 5m
    delayAfterFailure: 30s
    unneededTime: 5m
    utilizationThreshold: "0.4"
---
apiVersion: autoscaling.openshift.io/v1beta1
kind: MachineAutoscaler
metadata:
  name: worker-autoscaler
  namespace: openshift-machine-api
spec:
  minReplicas: 3
  maxReplicas: 12
  scaleTargetRef:
    apiVersion: machine.openshift.io/v1beta1
    kind: MachineSet
    name: prod-cluster-worker
4 · NetworkPolicy — production namespace baseline
# netpol-production.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-from-ingress
  namespace: production
spec:
  podSelector: {}
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              network.openshift.io/policy-group: ingress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-same-namespace
  namespace: production
spec:
  podSelector: {}
  ingress:
    - from:
        - podSelector: {}
  egress:
    - to:
        - podSelector: {}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: production
spec:
  podSelector: {}
  egress:
    - ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
5 · ResourceQuota + LimitRange
# quota-production.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: production-quota
  namespace: production
spec:
  hard:
    requests.cpu: "40"
    requests.memory: 80Gi
    limits.cpu: "80"
    limits.memory: 160Gi
    persistentvolumeclaims: "20"
    pods: "200"
    services: "50"
    services.loadbalancers: "2"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: production-limits
  namespace: production
spec:
  limits:
    - type: Container
      default:
        cpu: 500m
        memory: 512Mi
      defaultRequest:
        cpu: 100m
        memory: 128Mi
      max:
        cpu: "8"
        memory: 16Gi
    - type: Pod
      max:
        cpu: "16"
        memory: 32Gi
    - type: PersistentVolumeClaim
      max:
        storage: 500Gi
6 · LDAP / OAuth identity provider
# oauth-ldap.yaml
apiVersion: config.openshift.io/v1
kind: OAuth
metadata:
  name: cluster
spec:
  identityProviders:
    - name: corporate-ldap
      mappingMethod: claim
      type: LDAP
      ldap:
        attributes:
          id: [dn]
          email: [mail]
          name: [cn]
          preferredUsername: [sAMAccountName]
        bindDN: "CN=ocp-bind,OU=ServiceAccounts,DC=example,DC=com"
        bindPassword:
          name: ldap-bind-secret
        ca:
          name: ldap-ca-cert
        insecure: false
        url: "ldaps://ldap.example.com/OU=Users,DC=example,DC=com?sAMAccountName?sub"
  tokenConfig:
    accessTokenMaxAgeSeconds: 86400
# Create bind password secret
oc create secret generic ldap-bind-secret \
  --from-literal=bindPassword='<BIND_PASSWORD>' \
  -n openshift-config
# Create CA configmap
oc create configmap ldap-ca-cert \
  --from-file=ca.crt=/path/to/ldap-ca.crt \
  -n openshift-config
oc apply -f oauth-ldap.yaml
# Remove the default kubeadmin after verifying LDAP login
oc delete secret kubeadmin -n kube-system
7 · RBAC — cluster roles for enterprise teams
# rbac-teams.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: platform-admins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: Group
    name: ocp-platform-admins   # synced from LDAP
    apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: dev-team-edit
  namespace: production
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: edit
subjects:
  - kind: Group
    name: dev-team
    apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: qa-team-view
  namespace: production
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
  - kind: Group
    name: qa-team
    apiGroup: rbac.authorization.k8s.io
8 · OpenShift GitOps (ArgoCD) — operator + App of Apps
# Install OpenShift GitOps operator via CLI
cat <<EOF | oc apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: openshift-gitops-operator
  namespace: openshift-operators
spec:
  channel: latest
  name: openshift-gitops-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace
EOF
# Wait for ArgoCD instance to be ready
oc wait --for=condition=Available deployment/openshift-gitops-server \
  -n openshift-gitops --timeout=300s
# app-of-apps.yaml — root ArgoCD application
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: cluster-config
  namespace: openshift-gitops
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/ocp-cluster-config
    targetRevision: main
    path: apps
  destination:
    server: https://kubernetes.default.svc
    namespace: openshift-gitops
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - PrunePropagationPolicy=foreground
9 · Cluster monitoring customization
# monitoring-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-monitoring-config
  namespace: openshift-monitoring
data:
  config.yaml: |
    prometheusK8s:
      nodeSelector:
        node-role.kubernetes.io/infra: ""
      tolerations:
        - key: node-role.kubernetes.io/infra
          effect: NoSchedule
      retention: 15d
      volumeClaimTemplate:
        spec:
          storageClassName: ocs-storagecluster-ceph-rbd
          resources:
            requests:
              storage: 500Gi
    alertmanagerMain:
      nodeSelector:
        node-role.kubernetes.io/infra: ""
      tolerations:
        - key: node-role.kubernetes.io/infra
          effect: NoSchedule
    grafana:
      enabled: true
    telemeterClient:
      enabled: false          # disable telemetry for air-gapped / regulated envs
10 · etcd encryption + backup
# etcd-encryption.yaml
apiVersion: config.openshift.io/v1
kind: APIServer
metadata:
  name: cluster
spec:
  encryption:
    type: aescbc            # enables encryption-at-rest for etcd
# Automated etcd backup — run as a CronJob on master nodes
cat <<'EOF' | oc apply -f -
apiVersion: batch/v1
kind: CronJob
metadata:
  name: etcd-backup
  namespace: openshift-etcd
spec:
  schedule: "0 2 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          hostNetwork: true
          hostPID: true
          nodeSelector:
            node-role.kubernetes.io/master: ""
          tolerations:
            - operator: Exists
          serviceAccountName: etcd-backup-sa
          containers:
            - name: etcd-backup
              image: registry.redhat.io/openshift4/ose-cli:latest
              command:
                - /bin/bash
                - -c
                - |
                  /usr/local/bin/cluster-backup.sh /home/core/etcd-backup
              volumeMounts:
                - mountPath: /home/core/etcd-backup
                  name: backup-dir
          volumes:
            - name: backup-dir
              hostPath:
                path: /home/core/etcd-backup
          restartPolicy: OnFailure
EOF
11 · Image policy — only allow signed images in production
# image-policy.yaml
apiVersion: config.openshift.io/v1alpha1
kind: ImagePolicy
metadata:
  name: production-image-policy
spec:
  scopes:
    - "registry.example.com/prod/*"
  policy:
    rootOfTrust:
      policyType: PublicKey
      publicKey:
        keyData: "{{ BASE64_COSIGN_PUBLIC_KEY }}"
        rekorKeyData: "{{ BASE64_REKOR_PUBLIC_KEY }}"
12 · Post-install validation
#!/usr/bin/env bash
set -euo pipefail
echo "=== Cluster health ==="
oc get nodes
oc get clusteroperators | grep -v "True.*False.*False"
echo "=== etcd member health ==="
oc rsh -n openshift-etcd etcd-$(oc get nodes -l node-role.kubernetes.io/master \
  -o jsonpath='{.items[0].metadata.name}') \
  etcdctl member list --write-out=table
echo "=== Operator status ==="
oc get co | awk '$3=="False" || $4=="True" || $5=="True"'
echo "=== Certificate expiry (alert if < 30d) ==="
oc get secret -A -o json | \
  python3 -c "
import json,sys,base64,datetime
from cryptography import x509
data=json.load(sys.stdin)
for item in data['items']:
  for k,v in item.get('data',{}).items():
    if k.endswith('.crt'):
      try:
        cert=x509.load_pem_x509_certificate(base64.b64decode(v))
        days=(cert.not_valid_after_utc.replace(tzinfo=None)-datetime.datetime.utcnow()).days
        if days < 30:
          print(f\"WARN: {item['metadata']['namespace']}/{item['metadata']['name']} expires in {days}d\")
      except: pass
"
echo "=== MachineConfigPool status ==="
oc get mcp
echo "=== All checks complete ==="

Enterprise best-practice summary

ConcernDecision
Network pluginOVN-Kubernetes (required for EgressIP, EgressFirewall, network segmentation)
Node topologySeparate master / infra / compute pools — never mix roles
etcdEncryption at rest (aescbc) + nightly backup to S3 or NFS
IdentityLDAP/OIDC with group sync — remove kubeadmin after day 1
Multi-tenancydefault-deny-all NetworkPolicy per namespace + ResourceQuota + LimitRange
SCCsrestricted-v2 for all workloads unless a specific capability is justified
Image supply chainBuild → scan (ACS/Trivy) → sign (cosign) → promote digest (never mutable tags)
GitOpsArgoCD App of Apps, selfHeal: true, manual sync gate for production
ObservabilityPrometheus on infra nodes, 15-day retention, alert to PagerDuty/Slack
Cluster upgradesStable channel, pause MCPs before upgrade, upgrade masters then workers
Backupetcd nightly + Velero for namespace-level PV snapshots

Leave a Reply