Understanding OpenShift Logging Architecture: A Shift to Vector and Loki

Red Hat OpenShift Logging provides a managed, cluster-wide framework for collecting, forwarding, storing, and visualizing logs generated by the node OS, core platform services, and user applications.

In modern OpenShift releases (OCP 4.12+), the logging architecture underwent a major shift, transitioning from the legacy Fluentd / Elasticsearch / Kibana (EFK) stack toward Vector, Loki, and OpenSearch (or external SIEMs).

1. Architectural Blueprint: Collection vs. Storage

OpenShift separates the logging stack into two independent architectural concerns: Log Forwarding (Collection) and Log Storage/Visualization.

Plaintext

  LOG SOURCES                        LOG COLLECTOR                   LOG STORES / SIEMs
┌──────────────┐ ┌───────────┐ ┌────────────────────┐
│ Applications │ ────────┐ │ │ ──────────────► │ OpenShift Loki │
└──────────────┘ │ │ │ └────────────────────┘
┌──────────────┐ ├─────────► │ Vector │ ┌────────────────────┐
│Infrastructure│ ───────┤ │ Collector │ ──────────────► │ OpenSearch / ELK │
└──────────────┘ │ │ │ └────────────────────┘
┌──────────────┐ │ │ │ ┌────────────────────┐
│ Audit Logs │ ────────┘ └───────────┘ ──────────────► │ Kafka / Splunk/ S3 │
└──────────────┘ └────────────────────┘

A. Log Categories

OpenShift categorizes all generated logs into three streams:

  • Application: Logs generated by user workloads running inside application namespaces (excluding system namespaces).
  • Infrastructure: Logs generated by OpenShift system pods (namespaces like openshift-*, kube-*) and host node services (journald, systemd).
  • Audit: Security audit trails generated by kube-apiserver, openshift-apiserver, OVN-Kubernetes firewall logs, and host auditd.

2. The Core Components Explained

Vector (The High-Performance Collector)

Replaces Fluentd as the default log collector. Vector is an ultra-fast, memory-efficient log agent written in Rust.

  • Role: Runs as a DaemonSet across all nodes. It scrapes container log files (/var/log/pods) and host journal logs (/var/log/journal), parses and enriches them with Kubernetes metadata (namespace, pod name, labels), and routes them to destinations.
  • Why Vector? Significantly lower CPU and memory footprint compared to Ruby-based Fluentd, with higher throughput capabilities under high log volume bursts.

Loki (The Preferred Native Log Store)

Loki Stack Operator is the default, Red Hat-supported log store for long-term log retention in modern OCP environments.

  • Role: An index-free, cloud-native log aggregation system inspired by Prometheus. Instead of indexing the full text of every log line, Loki only indexes the metadata labels (e.g., namespace="prod", app="payment").
  • Storage Model: Stores compressed log chunks directly in S3-compatible Object Storage (AWS S3, MinIO, Azure Blob, ODF NooBaa). This makes log storage drastically cheaper and highly scalable compared to Elasticsearch.

OpenSearch (The Search-Heavy Alternative)

Replaces the deprecated Elasticsearch operator.

  • Role: A distributed search and analytics engine derived from Elasticsearch. It performs full-text indexing on every word inside a log entry.
  • Trade-off: Excellent for complex wildcard string queries and heavy analytical dashboards, but consumes significantly more RAM, CPU, and high-performance block storage compared to Loki.

3. Declarative Setup Example: ClusterLogForwarder

To route logs using Vector, administrators create a custom resource called ClusterLogForwarder in the openshift-logging namespace.

YAML

apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
name: instance
namespace: openshift-logging
spec:
# Define external or internal destinations
outputs:
- name: loki-internal
type: lokiStack
lokiStack:
target:
name: logging-loki
url: 'https://logging-loki-gateway.openshift-logging.svc:8080'
- name: external-splunk
type: splunk
url: 'https://splunk.corp.internal:8088'
secret:
name: splunk-hec-token
# Route specific log types to desired outputs
pipelines:
- name: app-to-loki
inputRefs:
- application
outputRefs:
- loki-internal
- name: audit-to-splunk
inputRefs:
- audit
- infrastructure
outputRefs:
- external-splunk

4. Summary Matrix: Component Comparison

ComponentRoleResource OverheadIndexing MethodPrimary Storage
VectorCollector / RouterVery Low (Rust binary)N/A (Streaming buffer)Ephemeral node memory / disk
LokiNative Log StoreLow / MediumMetadata Labels OnlyObject Storage (S3 / Blob)
OpenSearchSearch Engine StoreHigh (JVM-based)Full-Text Inverted IndexHigh-IOPS Block Storage (PVCs)

Leave a Reply