Kong – full mini project folder

April 18, 2026April 18, 2026 techhadoop kong, Uncategorized ai, artificial-intelligence, devops, llm, technology

Here’s a full mini project folder for Kong that you can copy as-is.

It uses Kong Gateway in DB-less mode, so all config lives in one declarative kong.yml file. That mode is a good fit for CI/CD and Git-managed config, but the Admin API is effectively read-only for config changes in this setup. (Kong Docs)

Folder structure

			
kong-mini-project/
├── app/
│   ├── package.json
│   └── server.js
├── kong/
│   └── kong.yml
├── .dockerignore
├── Dockerfile
└── compose.yml

		

1) `app/package.json`

			
{
  "name": "kong-mini-project",
  "version": "1.0.0",
  "description": "Node app behind Kong Gateway",
  "main": "server.js",
  "scripts": {
    "start": "node server.js"
  },
  "license": "MIT"
}

		

2) `app/server.js`

			
const http = require("http");
const PORT = process.env.PORT || 3000;
const server = http.createServer((req, res) => {
  if (req.url === "/healthz") {
    res.writeHead(200, { "Content-Type": "application/json" });
    return res.end(JSON.stringify({ ok: true }));
  }
  const body = {
    ok: true,
    message: "Hello from app behind Kong",
    method: req.method,
    url: req.url,
    host: req.headers.host,
    time: new Date().toISOString()
  };
  res.writeHead(200, { "Content-Type": "application/json" });
  res.end(JSON.stringify(body, null, 2));
});
server.listen(PORT, () => {
  console.log(`Server listening on ${PORT}`);
});

		

3) `Dockerfile`

			
FROM node:20-alpine
WORKDIR /app
COPY app/package.json ./
RUN npm install --omit=dev
COPY app/server.js ./
ENV PORT=3000
EXPOSE 3000
CMD ["npm", "start"]

		

4) `.dockerignore`

			
node_modules
npm-debug.log
.git
.github

5) `kong/kong.yml`

This is the heart of the project. It defines:

one upstream Service
one public Route
a key-auth plugin
a rate-limiting plugin
one Consumer with an API key

Kong’s declarative config format supports entities like Services, Routes, Consumers, and Plugins in DB-less mode. The Key Auth plugin can require API keys, and the Rate Limiting plugin can throttle requests by time window such as per minute. When authentication is present, rate limiting uses the authenticated Consumer identity. (Kong Docs)

			
_format_version: "3.0"
services:
  - name: app-service
    url: http://app:3000
    routes:
      - name: app-route
        paths:
          - /api
        protocols:
          - http
          - https
plugins:
  - name: key-auth
    service: app-service
    config:
      key_names:
        - apikey
  - name: rate-limiting
    service: app-service
    config:
      minute: 5
      policy: local
consumers:
  - username: demo-client
    keyauth_credentials:
      - key: super-secret-demo-key

		

A note on policy: local: that works well for a single local node, but Kong notes that plugins needing shared database state do not fully function in DB-less mode, so this is best for learning or single-node setups rather than clustered distributed quotas. (Kong Docs)

6) `compose.yml`

Kong’s Docker docs support running Kong with Docker Compose, and the read-only Docker Compose guide for DB-less mode uses KONG_DATABASE=off plus KONG_DECLARATIVE_CONFIG pointing to the config file. (Kong Docs)

			
services:
  kong:
    image: kong:3.10
    environment:
      KONG_DATABASE: "off"
      KONG_DECLARATIVE_CONFIG: /kong/declarative/kong.yml
      KONG_PROXY_ACCESS_LOG: /dev/stdout
      KONG_ADMIN_ACCESS_LOG: /dev/stdout
      KONG_PROXY_ERROR_LOG: /dev/stderr
      KONG_ADMIN_ERROR_LOG: /dev/stderr
      KONG_ADMIN_LISTEN: 0.0.0.0:8001
    ports:
      - "8000:8000"   # public proxy
      - "8001:8001"   # admin api (read-only for config in DB-less mode)
    volumes:
      - ./kong/kong.yml:/kong/declarative/kong.yml:ro
  app:
    build:
      context: .
      dockerfile: Dockerfile

		

7) Run it

docker compose up -d --build

Then test it.

Without an API key, access should fail because the route is protected by the Key Auth plugin. (Kong Docs)

curl -i http://localhost:8000/api

With the API key in a header, it should succeed. Kong’s Key Auth plugin supports reading keys from headers, query parameters, or request body, depending on config. (Kong Docs)

			
curl -i \
  -H "apikey: super-secret-demo-key" \
  http://localhost:8000/api

You can also use a query string:

curl -i "http://localhost:8000/api?apikey=super-secret-demo-key"

8) Test rate limiting

The plugin is set to 5 requests per minute, so the sixth quick request should return 429. Kong’s rate-limiting plugin supports time windows including seconds, minutes, hours, days, months, and years. (Kong Docs)

			
for i in {1..6}; do
  curl -s -o /dev/null -w "%{http_code}\n" \
    -H "apikey: super-secret-demo-key" \
    http://localhost:8000/api
done

		

9) Useful checks

See running containers:

docker compose ps

Follow Kong logs:

docker compose logs -f kong

Follow app logs:

docker compose logs -f app

Read the service list from the Admin API:

curl http://localhost:8001/services

In DB-less mode, that Admin API is useful for inspection, but Kong’s docs say you cannot use it for normal write-based configuration management because the declarative file is the source of truth. (Kong Docs)

10) What makes this different from Traefik

With Traefik, the main workflow was “discover containers and route traffic to them.” With Kong, the model is “define Services and Routes, then attach policy plugins like auth and rate limiting.” Kong’s docs emphasize entities such as Services, Routes, Consumers, Upstreams, and Plugins as the core gateway model. (Kong Docs)

So in practice:

Traefik is great for app routing and reverse proxying.
Kong is better when you want API-specific control like identity, quotas, and policy.

11) Resume line

Built a containerized API behind Kong Gateway in DB-less mode using declarative configuration, API key authentication, and per-consumer rate limiting.

12) Best next upgrade

The strongest next step is to add JWT auth or request transformation, because those show off Kong as an API gateway rather than just a reverse proxy. Kong’s plugin ecosystem is one of its main strengths. (Kong Docs)

KONG

August 25, 2025April 26, 2026 techhadoop kong

Got it — here’s a clean, practical way to wire Client → Cloudflare → F5 → Kong → Upstream so your normal Kong flow
Client → [Kong Route matched] → [Service invoked] → Upstream API works end-to-end.

0) Topology (recommended)

Client ─TLS─&gt; Cloudflare ─TLS─&gt; F5 VIP ─HTTP─&gt; Kong (8000) ─(HTTP/HTTPS)─&gt; Upstream

Cloudflare terminates client TLS + WAF/bot/DDOS.
F5 is your “origin” for Cloudflare, then load balances across Kong nodes.
Kong routes/plugins/auth, then proxies to your upstream.

You can also do F5→Kong over HTTPS (8443). Start with HTTP (simpler), add re-encryption later.

1) Cloudflare (edge) setup

DNS: api.example.com → orange-cloud (proxied) to your F5 public IP.
SSL/TLS: set mode Full (strict).
Authenticated Origin Pull (AOP):
- Enable it in Cloudflare.
- On F5, require the Cloudflare client cert (see F5 step) so only CF can hit your VIP.
Origin server certificate (on F5): either
- a normal public cert (LetsEncrypt, etc.), or
- a Cloudflare Origin Certificate (valid only to CF; fine if you never bypass CF).
API cache rules: bypass cache for /api/*, enable WebSockets if you use them.

2) F5 LTM (VIP to Kong) essentials

Virtual server (HTTPS on 443) → pool (Kong nodes on 8000)

Client SSL profile: present your cert (public or CF Origin Cert).
(Optional but recommended) verify Cloudflare client cert for AOP:
import Cloudflare Origin Pull CA on F5 and set it as Trusted CA; require client cert.
HTTP profile: enable; Insert X-Forwarded-For; preserve headers.
OneConnect: enable for keep-alives to Kong.
Pool members: all Kong nodes, port 8000 (or 8443 if you re-encrypt).
Health monitor: HTTP GET to Kong status (see Kong step).

Header hygiene (iRule – optional, if you want to be explicit):

when HTTP_REQUEST {
  # Preserve real client IP from Cloudflare into X-Forwarded-For
  if { [HTTP::header exists "CF-Connecting-IP"] } {
    set cfip [HTTP::header value "CF-Connecting-IP"]
    if { [HTTP::header exists "X-Forwarded-For"] } {
      HTTP::header replace "X-Forwarded-For" "[HTTP::header value X-Forwarded-For], $cfip"
    } else {
      HTTP::header insert "X-Forwarded-For" $cfip
    }
  }
  HTTP::header replace "X-Forwarded-Proto" "https"
}

(Or just enable “X-Forwarded-For: append” in the HTTP profile and set XFP via policy.)

3) Kong Gateway settings (behind F5)

Environment (docker-compose/env vars):

KONG_PROXY_LISTEN=0.0.0.0:8000
# Trust only your F5 addresses/CIDRs (do NOT trust 0.0.0.0/0)
KONG_TRUSTED_IPS=10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,&lt;F5_VIP_or_SNATs&gt;
KONG_REAL_IP_HEADER=X-Forwarded-For
KONG_REAL_IP_RECURSIVE=on
# helpful during testing
KONG_HEADERS=latency_tokens
# health endpoint for F5 monitor (internal only)
KONG_STATUS_LISTEN=0.0.0.0:8100

F5 health monitor target (HTTP):

URL: http://<kong-node>:8100/status
Expect: 200 (Kong’s status port; safe to expose internally)

Define Service/Route (example):

# Service to your upstream (HTTP)
curl -sX POST :8001/services \
  -d name=orders-svc \
  -d host=tomcat-app \
  -d port=8080 \
  -d protocol=http

# Route that matches your API path/host
curl -sX POST :8001/routes \
  -d service.name=orders-svc \
  -d hosts[]=api.example.com \
  -d paths[]=/v1

(If upstream is HTTPS, set protocol=https, port=443, and sni=upstream.host.)

4) Putting it together (request path)

Client → https://api.example.com/v1/...
Cloudflare terminates TLS, adds CF-Connecting-IP, forwards to F5.
F5 validates CF client cert (AOP), appends XFF, sets X-Forwarded-Proto:https, LB to a Kong node.
Kong trusts F5 IP, extracts the real client IP from XFF, matches Route (Host=api.example.com, path=/v1), invokes Service, proxies to Upstream.
Response flows back; Kong adds latency headers (if enabled); F5 returns to CF; CF returns to client.

5) Testing (in layers)

Direct to Kong (bypass CF/F5) on the private network: curl -i -H 'Host: api.example.com' http://<kong-node>:8000/v1/ping -H 'Kong-Debug: 1'
Through F5 VIP (simulate Cloudflare off): curl -i -H 'Host: api.example.com' https://<f5-public-ip>/v1/ping --resolve api.example.com:<f5-public-ip>:443
Through Cloudflare (real path): curl -i https://api.example.com/v1/ping -H 'Kong-Debug: 1'

Check headers:

X-Kong-Upstream-Latency (upstream time)
X-Kong-Proxy-Latency (Kong internal)
Logs in Kong show client_ip = real client IP (not F5/CF) if trusted IPs are correct.

6) Observability (recommended)

Kong plugins:
- prometheus (metrics), correlation-id (X-Request-ID), http-log or file-log → ELK.
F5: request logging profile to ELK; LTM stats.
Cloudflare: Security events/Edge logs (if plan allows).

7) Common gotchas (and quick fixes)

Client IP shows F5/Cloudflare → set KONG_TRUSTED_IPS, REAL_IP_*; ensure F5 appends XFF and passes CF-Connecting-IP.
OIDC header lost → make sure F5 policy does not strip Authorization.
Wrong redirects / mixed content → ensure X-Forwarded-Proto: https reaches Kong & upstream.
“Server busy” / spikes → raise F5 OneConnect pool, ensure Kong backlog/somaxconn, keepalive to upstream.
Health checks hitting Admin API → use KONG_STATUS_LISTEN (8100) instead, not port 8001 (Admin).

Cloudflare → F5 → Kong API stack

August 25, 2025April 26, 2026 techhadoop kong

Here’s the clean mental model for a Cloudflare → F5 → Kong API stack, plus the key headers/TLS choices and the minimum config knobs so it “just works.”

1) The traffic flow (happy path)

Client ─(HTTPS/HTTP2/3)→ Cloudflare ─(TLS to origin)→ F5 VIP ─(HTTP/HTTPS)→ Kong cluster ─→ Upstream services

Cloudflare terminates the client TLS (always, if the orange-cloud proxy is on).
- Applies WAF, DDoS, bot rules, rate limits, geo rules, etc.
- Forwards to your F5 VIP as the “origin”.
F5 LTM receives Cloudflare’s request.
- Usually terminates TLS again (re-encrypt to Kong or send plain HTTP on the inside).
- Load-balances across Kong nodes (pool members in Zone A/B).
Kong Gateway routes by Host/path to your backend (service), runs plugins (OIDC, rate-limit, etc.), and proxies to the upstream.

2) TLS choices (pick one per hop)

Cloudflare → F5 (origin TLS):

Set Cloudflare SSL mode to Full (strict).
Enable Authenticated Origin Pull so only Cloudflare can hit F5.
On F5, trust Cloudflare’s Origin Pull CA and require client cert.

F5 → Kong:

Simple: terminate on F5 and send HTTP to Kong on the private VLAN.
End-to-end TLS: client-SSL on F5, server-SSL from F5 to Kong (re-encrypt), SNI kong.internal (or node name).

Kong → Upstream:

Match your upstream: protocol=http|https, SNI if TLS, optionally mTLS to sensitive services.

3) Real client IP (do this or logs/limits will be wrong)

Cloudflare sets:

CF-Connecting-IP (client IP)
X-Forwarded-For (appends client IP)
X-Forwarded-Proto: https

F5 should preserve (not overwrite) X-Forwarded-For and pass X-Forwarded-Proto.

Kong must trust the proxy chain so it can compute the real client IP:

Set (env or kong.conf):
- KONG_TRUSTED_IPS=<F5 private CIDRs or F5 VIPs> (don’t trust 0.0.0.0/0)
- KONG_REAL_IP_HEADER=X-Forwarded-For
- KONG_REAL_IP_RECURSIVE=on
Then client_ip in logs, rate-limit/correlation will be the actual user IP from Cloudflare.

If you prefer using Cloudflare’s header explicitly, you can have F5 copy CF-Connecting-IP into the leftmost position of X-Forwarded-For.

4) Load balancing & health checks (avoid double “mystery” failover)

Cloudflare (optional LB): usually point it at a single F5 VIP per region and let F5 do node health.
F5 → Kong nodes: HTTP health monitor (e.g., GET /status/health on each Kong).
Kong → upstreams: use Kong Upstreams/Targets with active + passive health checks to eject bad app pods.

Pick one layer to be the source of truth per hop (Cloudflare LB or F5, Kong or upstream LB) to avoid contradictory decisions.

5) Protocols & connections

HTTP versions: client can be HTTP/2 or HTTP/3 to Cloudflare. Cloudflare→F5 is HTTP/1.1 or HTTP/2 (CF may downgrade). F5→Kong is typically HTTP/1.1.
Keep-alive: enable OneConnect on F5 and keep-alive to Kong to avoid connection churn.
WebSockets/gRPC: supported end-to-end; ensure Upgrade/HTTP2 is enabled through F5 and Kong Routes/Services.

6) Minimal config snippets

F5 (HTTP profile / header handling):

Enable “Insert X-Forwarded-For” (or an iRule to append not overwrite).
Preserve X-Forwarded-Proto = https.
If using Authenticated Origin Pull: client-SSL requires CF client cert; trust CF Origin CA.

Kong (env):

KONG_HEADERS=latency_tokens
KONG_TRUSTED_IPS=10.0.0.0/8,172.16.0.0/12,192.168.0.0/16, &lt;your F5 VIPs/CIDRs&gt;
KONG_REAL_IP_HEADER=X-Forwarded-For
KONG_REAL_IP_RECURSIVE=on

Kong Service (HTTPS upstream with SNI):

protocol: https
host: api.internal.example
port: 443
tls_verify: true
sni: api.internal.example

Kong rate limiting behind proxies: use consumer or ip policy; with trusted IPs set, ip uses the real client.

7) Observability (what to turn on)

Cloudflare: (if plan allows) Edge logs / Security events (attack/waf/bot).
F5: LTM logs + request logging profiles; export to ELK.
Kong: enable Prometheus plugin, correlation-id, and http-log/file-log to ELK.
Make sure your ELK/Loki sees client_ip, X-Request-ID, service, route, upstream_status, latencies.{kong,proxy,request}.

8) Common pitfalls & quick fixes

Client IP shows F5/Cloudflare: set KONG_TRUSTED_IPS and KONG_REAL_IP_* as above; ensure F5 appends—not overwrites—X-Forwarded-For.
OIDC 401s behind CF/F5: your LB stripped Authorization. Validate with an echo upstream; ensure F5 leaves Authorization intact.
Mixed-content / wrong redirects: preserve X-Forwarded-Proto: https through to Kong and upstream so apps build HTTPS URLs.
Header too large (Set-Cookie): raise Kong Nginx proxy buffers if you see upstream sent too big header.
Double LB flaps: don’t health-check the same hop in both Cloudflare and F5 with different intervals/paths.

9) Example end-to-end (typical, secure)

Cloudflare Full (strict) + Authenticated Origin Pull → F5 public VIP api.example.com:443.
F5 terminates TLS, adds XFF if missing, passes X-Forwarded-Proto: https, LB to kongA:8000, kongB:8000.
Kong trusts F5 IPs, pulls real client IP from XFF, runs OIDC/rate-limit, sends to upstream (HTTPS with SNI).
Logs/metrics from all 3 land in ELK/Grafana.

Kong / our app is build using Java Spring Boot

August 12, 2025April 26, 2026 techhadoop kong cloud, science, security, technology

When integrating certificates into Spring Boot, especially for mTLS, HTTPS, or secure inter-service communication, there are several important certificate handling considerations you should account for:

1. Keystore / Truststore Formats

Store	Purpose	Common Format
Keystore	Holds your application’s private key + cert	`JKS`, `PKCS12` (`.p12` / `.pfx`)
Truststore	Holds trusted CA certs to verify clients or servers	`JKS`, `PKCS12`

JKS is Java-native, but PKCS12 is preferred for modern apps (more portable + supports OpenSSL).

Example Spring Boot Config:

server.ssl.key-store=classpath:server.p12
server.ssl.key-store-password=changeit
server.ssl.key-store-type=PKCS12
server.ssl.trust-store=classpath:truststore.p12
server.ssl.trust-store-password=changeit
server.ssl.trust-store-type=PKCS12

2. Certificate Chain Requirements

Spring Boot expects:

Full certificate chain in the keystore if issued by an intermediate CA
Trusted root CA in the truststore (for mTLS)

🔸 Failure to include the full chain often results in handshake errors or “unable to find valid certification path” errors.

3. Protocol Configuration (TLS)

Spring Boot (via embedded Tomcat) uses TLS 1.2+ by default.

To restrict or specify:

server.ssl.enabled-protocols=TLSv1.2,TLSv1.3

Disable TLS 1.0/1.1 — they are deprecated and insecure.

4. Cipher Suites

Spring Boot (via Tomcat) negotiates secure ciphers by default.
You can explicitly define them:

server.ssl.ciphers=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,...

Use modern ciphers like ECDHE + AES_GCM.
Avoid RC4, 3DES, NULL, or EXPORT ciphers.

5. Mutual TLS (mTLS)

To require client certs:

server.ssl.client-auth=need

Modes:

none: default (no client cert)
want: optional client cert
need: mandatory client cert (for mTLS)

6. Generating Keystore & Truststore

Convert PEM to PKCS12:

openssl pkcs12 -export \
  -in client.crt \
  -inkey client.key \
  -certfile ca.crt \
  -out client-keystore.p12 \
  -name client

Then import trusted CA into truststore (if using JKS):

keytool -import -alias myca -file ca.crt -keystore truststore.jks

7. Spring Boot with Reverse Proxies (e.g., Kong, F5)

If TLS termination is done by Kong/F5 and Spring Boot sits behind it:

Use X-Forwarded headers: server.forward-headers-strategy=framework
Do not terminate TLS inside Spring unless you need end-to-end TLS.

Summary: What to Account For

Area	Key Considerations
Format	Use `PKCS12` for best compatibility
Trust	Truststore must contain root CA or intermediate
Chain	Include full cert chain in keystore
Protocol	Enable TLSv1.2+ only
Ciphers	Use strong, modern suites
mTLS	Set `client-auth=need` + configure truststore
Proxy	Handle `X-Forwarded-*` if behind Kong/F5

KONG 401 error caused by F5

August 6, 2025April 26, 2026 techhadoop kong

F5 (BIG-IP) or any load balancer can cause a 401 Unauthorized error in Kong, depending on how it’s configured. Here’s how:

How F5 Can Cause 401 Errors in Kong

F5 Misconfiguration	Effect on Kong
Strips Authorization headers	Kong never receives the token/key — authentication fails with 401.
*Overwrites headers (like `Host` or `X-Forwarded-`)**	Breaks routing or causes Kong to reject the request.
SSL Termination issues	OIDC or JWT validation fails due to invalid scheme or issuer mismatch.
Rewrites path incorrectly	Kong cannot match the route, plugin doesn’t apply properly, or token is invalid.
Session persistence or cookie manipulation	Disrupts OIDC or session-based auth flows.

How to Test if F5 is the Cause

Bypass F5 (send request directly to Kong): curl -i http://<kong-IP>:8000/<route-path> \ -H "Authorization: Bearer <your-token>"
- If this works and through F5 it fails → F5 is modifying/blocking something.
Inspect headers from F5:
- Add a logging plugin in Kong to dump all headers.
- Use request-transformer or a custom plugin to inspect incoming headers.
Check F5 HTTP Profile:
- Ensure Authorization headers are not removed.
  - F5 → Virtual Server → HTTP Profile → Request Header Erase should not include Authorization.
Enable debug logs in Kong:
- Will show missing/malformed headers.

Fix Recommendations

Preserve Authorization headers in F5 config.
Ensure correct SSL termination and forwarding.
Keep route paths intact when forwarding.
Use F5 “passthrough” mode if possible for auth-related traffic.

KONG 401 & 400 error

July 23, 2025April 26, 2026 techhadoop kong news, politics, security, technology

If you’re getting 401 and 400 intermittently from the upstream service, that strongly suggests authentication-related issues, often tied to token forwarding, expiration, or format mismatch.

Quick Summary of Key Differences

Status	Meaning	Common Cause
401	Unauthorized	Missing/invalid/expired credentials
400	Bad Request	Malformed or incomplete request (e.g. OIDC token request)

Intermittent 401 + 400: Common Root Causes

1. Expired or Reused Tokens

Kong gets a token once, caches it, and keeps using it—but upstream expects a fresh one.
Especially common with client credentials or authorization code flows.

Solution:

Set token caching to short duration or disable it in the OIDC plugin: config: cache_ttl: 0 # Or a very short TTL like 5

2. Multiple Consumers with Invalid Secrets

One client (consumer) is configured correctly, others are not.
You see 401/400 when the bad client makes a request.

Solution:

Enable verbose logging in Kong: export KONG_LOG_LEVEL=debug kong reload Then correlate consumer_id with the error.

3. Kong Not Forwarding Tokens Correctly

Kong authenticates but doesn’t forward Authorization header to the upstream.
Some plugins strip headers by default.

Solution:

Add request-transformer plugin to pass the token: curl -X POST http://localhost:8001/services/YOUR_SERVICE/plugins \ --data "name=request-transformer" \ --data "config.add.headers=Authorization:Bearer $(jwt_token)"

🔸 4. OIDC Plugin Misconfiguration

If you’re using the OpenID Connect plugin:

grant_type, client_id, or redirect_uri may be wrong or missing intermittently.
Kong might request a new token but fail to pass a correct one.

Check:

Kong OIDC plugin config
Errors like: error=invalid_request error_description="Unsupported client authentication method"

How to Debug Effectively

Set KONG_LOG_LEVEL=debug, reload Kong, and tail the logs: tail -f /usr/local/kong/logs/error.log
Inspect Upstream Request:
- Look for what headers/body Kong is sending.
- Especially Authorization, Content-Type, and request body if OIDC is involved.
Track Errors to a Specific Consumer:
- Use consumer_id in the access log to trace.
- Maybe only some consumers are misconfigured.
Try Curling the Upstream Directly with the exact payload Kong sends (use Postman or curl): curl -X POST https://upstream/token \ -H "Authorization: Bearer <your_token>" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "grant_type=client_credentials&client_id=...&client_secret=..."

to check

Kong plugin configs (especially OIDC/JWT)
A few lines from Kong’s debug logs showing the upstream request/response
Whether you’re using Ping Identity or a custom upstream

kong – 2 services same route

July 8, 2025April 26, 2026 techhadoop kong ai, cloud, programming, python, technology

In Kong Gateway, you cannot have two services bound to the exact same route — a route must be unique in terms of its combination of matching rules (such as paths, hosts, methods, etc.).

Why You Can’t Have Duplicate Routes

Kong uses the route’s matching criteria to determine which service should handle a request. If two routes have the same matching rules, Kong can’t disambiguate, which leads to a conflict error when trying to create the second route.

For example:

# Service A
curl -i -X POST http://localhost:8001/services/ --data name=service-a --data url=http://api-a
curl -i -X POST http://localhost:8001/routes --data name=route-a --data service.name=service-a --data paths[]=/api

# Service B
curl -i -X POST http://localhost:8001/services/ --data name=service-b --data url=http://api-b
curl -i -X POST http://localhost:8001/routes --data name=route-b --data service.name=service-b --data paths[]=/api

The second POST /routes will fail with:

HTTP 409 Conflict – duplicate entry

Workarounds / Alternatives

Goal	Solution
Blue/Green or Canary Deployments	Use Kong Plugins (e.g., `traffic-split`, `canary`) or upstreams and targets instead of duplicate routes.
Different consumers/users hitting different backends	Use request transformers, ACLs, or Kong Enterprise Route-by-header/Route-by-consumer plugins.
Same path, different method or host	You can differentiate routes by `methods[]`, `hosts[]`, or `headers`.

🧪 Example: Two Routes with Same Path, Different Hosts

# Route 1
curl -i -X POST http://localhost:8001/routes \
  --data name=api-v1 \
  --data paths[]=/api \
  --data hosts[]=v1.example.com \
  --data service.name=service-a

# Route 2
curl -i -X POST http://localhost:8001/routes \
  --data name=api-v2 \
  --data paths[]=/api \
  --data hosts[]=v2.example.com \
  --data service.name=service-b

These can coexist because their hosts[] fields are different.

Summary

Route Matching Rule	Must Be Unique For
`paths[]`	Same host/methods/headers
`hosts[]`	If combined with same path
`methods[]`, `headers`	Can disambiguate routes with same path

KONG – 504

July 2, 2025April 26, 2026 techhadoop kong

Common Causes

Upstream service is slow or down
DNS resolution delay or failure
Incorrect protocol (HTTP vs HTTPS)
Upstream timeout too low in Kong config
Load balancer or firewall in between is blocking/delaying

If you’re seeing upstream_status=504 in Kong logs, it means:

Kong sent the request to the upstream (your backend service), but did not receive a response within the configured timeout, so it returned a 504 Gateway Timeout to the client.

🔍 What to Check When You See `upstream_status=504`

1. Verify Upstream is Healthy

Try to access your upstream from the Kong host directly:

curl -v http://&lt;upstream-host&gt;:&lt;port&gt;/&lt;endpoint&gt;

If this is slow or hangs → your backend is the problem.
If it fails to connect → network or DNS issue.

2. Increase Kong Timeouts

Timeouts are per-service. You can increase them using the Admin API:

curl -X PATCH http://localhost:8001/services/{service_name} \
  --data "connect_timeout=10000" \
  --data "read_timeout=15000" \
  --data "write_timeout=10000"

All values are in milliseconds.

3. Check Protocol Mismatch (HTTP vs HTTPS)

If your upstream uses HTTPS but the service is configured with protocol = http, Kong cannot complete the TLS handshake and may hang.

Fix it with:

curl -X PATCH http://localhost:8001/services/{service_name} \
  --data "protocol=https"

4. Check Kong Error Log

Run:

tail -n 100 /usr/local/kong/logs/error.log | grep timeout

You may see:

[error] ... upstream timed out (110: Connection timed out) while reading response header from upstream

5. Check Health of Upstream Targets (If Using Upstreams)

curl http://localhost:8001/upstreams/{upstream_name}/health

If all targets are unhealthy, Kong will return 504s.

6. Review Load Balancer / Firewall Rules

If Kong goes through:

A corporate firewall
A cloud load balancer
An IPsec/VPN tunnel

Timeouts or dropped packets there can also lead to 504.

Example Full Troubleshooting Flow

Run from Kong host/container: curl -v http://your-upstream-host:port/health
If that times out:
- Check if service is running.
- Check firewall rules.
Increase Kong timeouts: curl -X PATCH http://localhost:8001/services/my-api \ --data "read_timeout=20000"
Check logs: tail -f /usr/local/kong/logs/error.log

If you paste the exact Kong service config and a log snippet with upstream_status=504, I can give you a more specific fix.

A 504 Gateway Timeout from Kong means:

Kong Gateway sent a request to your upstream service, but didn’t get a response in time, so it gave up and returned a 504 to the client.

QUICK FIX CHECKLIST

Check	Action
⏱ Timeout too short?	Increase timeout values for the service
🔌 Backend down?	Test the backend directly from Kong
❌ Wrong protocol?	Ensure the `protocol` is set correctly (`http` vs `https`)
🔐 Firewall?	Check network/firewall between Kong and upstream
📉 Backend overloaded?	Review CPU/memory on backend service
🔁 Load balancer in the path?	Check intermediate LB timeouts

1. Test the Backend Directly

From Kong host or container:

curl -v http://&lt;upstream-host&gt;:&lt;port&gt;/health

If this hangs or fails, the issue is upstream—not Kong.

2. Increase Kong Timeouts

Use the Admin API:

curl -X PATCH http://localhost:8001/services/{service_name} \
  --data "connect_timeout=10000" \
  --data "read_timeout=15000" \
  --data "write_timeout=10000"

(All values are in milliseconds.)

3. Check Protocol Mismatch

Is your backend using HTTPS but your service uses HTTP?

Fix:

curl -X PATCH http://localhost:8001/services/{service_name} \
  --data "protocol=https"

4. Check Kong Logs

Look in:

/usr/local/kong/logs/error.log

Typical timeout error:

upstream timed out (110: Connection timed out) while reading response header from upstream

5. Enable Correlation ID for Tracing

Enable this to track requests:

curl -X POST http://localhost:8001/plugins \
  --data "name=correlation-id" \
  --data "config.header_name=X-Correlation-ID" \
  --data "config.generator=uuid" \
  --data "config.echo_downstream=true"

Now check logs using the X-Correlation-ID.

If you send your service config or a log example with upstream_status=504, I’ll pinpoint the issue faster.

The error “An invalid response from upstream server” in Kong typically means:

Kong was able to connect to the upstream service, but the response it got back was malformed, empty, or violated protocol expectations.

Common Root Causes

Cause	Description
❌ Empty response	Upstream sent no body or headers back
⚠️ Invalid HTTP format	Response doesn’t follow HTTP spec
🔐 SSL/TLS mismatch	HTTP used with HTTPS backend or vice versa
🧱 Protocol error	Upstream speaking gRPC, WebSocket, or another protocol unexpectedly
🔌 Upstream closed connection	Before Kong finished reading the response

How to Troubleshoot

1. Check Logs

Look for this in Kong’s error log:

tail -f /usr/local/kong/logs/error.log

Common message:

upstream prematurely closed connection while reading response header

2. Test Upstream Manually

Run from the Kong host or container:

curl -i http://&lt;upstream-host&gt;:&lt;port&gt;/&lt;path&gt;

If you get no response or a malformed one, the problem is the upstream.

3. Verify Protocol

Make sure you’re not calling an HTTPS backend as HTTP:

# Fix protocol
curl -X PATCH http://localhost:8001/services/{service_name} \
  --data "protocol=https"

4. Increase Read Timeout

Sometimes upstreams are just too slow:

curl -X PATCH http://localhost:8001/services/{service_name} \
  --data "read_timeout=15000"

5. Check Upstream for Non-Standard Response

If the upstream returns:

Non-HTTP data (e.g., binary, gRPC without HTTP/1.1 framing)
Incorrect headers
Incomplete response

It will break Kong’s HTTP parser.

🧪 Bonus: Use Request Termination for Testing

Temporarily override the upstream to test Kong behavior:

curl -X POST http://localhost:8001/services/{service_name}/plugins \
  --data "name=request-termination" \
  --data "config.status_code=200"

If this works, the issue is 100% upstream-related.

F5 – kong configuration

November 13, 2024July 28, 2026 techhadoop kong azure, cybersecurity, linux, security, ssl

Configure the F5 Load Balancer with VIP and SSL Certificate

Create a Virtual Server (VIP):
- Log in to your F5 management console.
- Navigate to Local Traffic > Virtual Servers > Virtual Server List.
- Click Create and configure the following:
  - Name: Give the VIP a meaningful name, like Kong_VIP.
  - Destination Address: Specify the IP address for the VIP.
  - Service Port: Set to 443 for HTTPS.
Assign an SSL Certificate to the VIP:
- Under the SSL Profile (Client) section, select Custom.
- For Client SSL Profile, choose an existing SSL profile, or create a new one if needed:
  - Go to Local Traffic > Profiles > SSL > Client.
  - Click Create and provide a name, then upload the SSL certificate and key.
- Assign this SSL profile to your VIP.
Configure Load Balancing Method:
- Under Load Balancing Method, choose a method that best fits your setup, such as Round Robin or Least Connections.
Set Up Pool and Pool Members:
- In the Pool section, create or select a pool to add your Kong instances as members:
  - Go to Local Traffic > Pools > Pool List, then Create a new pool.
  - Assign Kong instances as Pool Members using their internal IP addresses and ports (usually port 8000 for HTTP or 8443 for HTTPS if Kong is configured with SSL).
- Make sure health monitors are set up for these pool members to detect when a Kong instance goes down.

Setup

Whether you need certificates on both the F5 load balancer and the Kong servers depends on how you plan to manage SSL/TLS termination and the level of encryption required for traffic between the F5 and Kong.

Here are two common setups:

1. SSL Termination on the F5 (Most Common)

Certificate Location: Only on the F5 load balancer.
How It Works: The F5 terminates the SSL connection with clients, decrypts the incoming HTTPS traffic, and forwards it to the Kong servers as plain HTTP traffic.
Benefits: Reduces the overhead on Kong servers because they don’t need to handle SSL encryption. It’s simpler to manage as only the F5 requires an SSL certificate.
Considerations: Traffic between the F5 and Kong servers is unencrypted, which is typically acceptable in private or secured networks (e.g., within a secure data center or VPC).

Configuration Steps:

Install and configure the SSL certificate only on the F5.
Set the F5 VIP to listen on HTTPS (port 443).
Configure Kong to listen on HTTP (port 8000 or a custom port).

This setup is generally sufficient if Kong instances and the F5 are within a trusted network.

2. End-to-End SSL (SSL Termination on Both F5 and Kong Servers)

Certificate Location: On both the F5 load balancer and the Kong servers.
How It Works: The F5 terminates the initial SSL connection from the client, but then re-encrypts the traffic before forwarding it to Kong. Kong servers also have SSL certificates, allowing them to decrypt this re-encrypted traffic.
Benefits: Ensures encrypted communication all the way from the client to the Kong servers, providing an extra layer of security.
Considerations: Requires SSL certificates on both F5 and Kong, and introduces some additional CPU overhead on Kong due to the need to decrypt/encrypt traffic.

Configuration Steps:

Install and configure an SSL certificate on the F5 for the VIP, and configure the VIP to listen on HTTPS.
Install an SSL certificate on each Kong server (these can be the same certificate as the F5 or separate ones).
Configure Kong to listen on HTTPS (port 8443 or another SSL-enabled port).
Configure the F5 to forward encrypted traffic to the Kong servers’ HTTPS port.

This approach is beneficial if security policies require end-to-end encryption, especially if the connection between the F5 and Kong instances crosses less secure network segments.

Summary

If you’re terminating SSL on the F5 only: A certificate is only required on the F5.
If you need end-to-end encryption: SSL certificates are required on both the F5 and each Kong server.

In most enterprise environments, SSL termination on the F5 is sufficient. However, for environments with stringent security requirements, end-to-end SSL provides additional security by ensuring all connections remain encrypted.

mTLS

Certificates:
- The F5 load balancer needs server and client certificates and CA certificates to validate clients (client mTLS) and backend servers (backend mTLS).
- Backend servers need server certificates and may require a CA certificate to verify the F5’s client certificate.
mTLS Setup:
- For mTLS from the client to the F5, configure the Client SSL Profile on the F5.
- For mTLS between the F5 and backend servers, configure the Server SSL Profile and set backend servers (like Kong) to require mTLS by validating the F5’s client certificate.

This setup provides mutual authentication between clients, the F5, and backend servers, ensuring secure, verified communication across all components in the chain.

do i need a certificate on both servers and F5 load balancer also how i integrate with mtls ?

For mTLS (mutual TLS) integration with an F5 load balancer and downstream servers like Kong or another backend, it is essential to determine where the certificates are needed and how to configure mTLS for mutual authentication between clients, the F5, and backend servers.

Certificates on Both the F5 Load Balancer and Backend Servers

Certificates on the F5 Load Balancer:
- The F5 acts as the entry point for client connections and needs a server certificate to handle HTTPS traffic.
- If using mTLS, the F5 will also need a client certificate and a trusted certificate authority (CA) to validate incoming client certificates.
- The F5 can be configured to terminate SSL and optionally re-encrypt traffic to backend servers.
Certificates on Backend Servers:
- For end-to-end encryption (where traffic from the F5 to backend servers remains encrypted), each backend server (e.g., Kong) also needs a server certificate.
- If mutual TLS is required between the F5 and backend servers, the backend servers also need to verify the client (F5’s) certificate, so you’ll need to import the F5’s client certificate or a shared CA certificate on backend servers.

Configuring mTLS on F5 Load Balancer

Here’s how you can set up mTLS on an F5 load balancer to handle mutual authentication with clients and potentially with backend servers:

1. Configure mTLS Between Client and F5

Client SSL Profile:
- Go to Local Traffic > Profiles > SSL > Client.
- Create a Client SSL Profile for the VIP and enable Client Certificate Authentication by selecting Require under Client Certificate.
- Import or reference a CA certificate that you trust to sign client certificates. This CA will validate client certificates.
Assign SSL Profile to VIP:
- Attach this client SSL profile to the VIP handling client requests.
- The F5 will now require clients to present a valid certificate from the specified CA to establish a secure connection.

2. mTLS Between F5 and Backend Servers (Optional)

If you want end-to-end mTLS (client to F5 and F5 to backend):

Server SSL Profile:
- Go to Local Traffic > Profiles > SSL > Server.
- Create a Server SSL Profile and enable the Authenticate option to require the backend server to present a valid certificate.
- Specify a trusted CA certificate to validate the backend server’s certificate.
Assign Server SSL Profile to Pool:
- Attach this server SSL profile to the backend pool so that the F5 will establish an mTLS connection when connecting to each backend server.

Backend Server Configuration (e.g., Kong)

If Kong is the backend server, configure Kong to:

Present a server certificate to the F5 for mutual authentication.
Verify client certificates if mTLS is required from F5 to Kong:
- Set client_ssl = on and configure ssl_client_certificate to reference the CA certificate or client certificates you trust.

Example snippet for kong.conf:

client_ssl = on

ssl_cert = /path/to/server.crt

ssl_cert_key = /path/to/server.key

ssl_client_certificate = /path/to/ca.crt # This will be used to verify F5’s client certificate

Summary

Certificates:
- The F5 load balancer needs server and client certificates and CA certificates to validate clients (client mTLS) and backend servers (backend mTLS).
- Backend servers need server certificates and may require a CA certificate to verify the F5’s client certificate.
mTLS Setup:
- For mTLS from the client to the F5, configure the Client SSL Profile on the F5.
- For mTLS between the F5 and backend servers, configure the Server SSL Profile and set backend servers (like Kong) to require mTLS by validating the F5’s client certificate.

This setup provides mutual authentication between clients, the F5, and backend servers, ensuring secure, verified communication across all components in the chain.

setup both SSL and mTLS

Yes, you can absolutely have a setup with both SSL termination and mTLS on the F5 load balancer. Here’s how the setup would work, allowing for both standard SSL connections (for regular HTTPS traffic) and mTLS (for additional security and mutual authentication) on the same VIP.

Mixed SSL and mTLS on F5

The configuration would involve:

Standard SSL Termination for clients that only need secure (HTTPS) connections.
mTLS configuration for clients requiring mutual authentication (client certificate verification).

Steps to Set Up SSL and mTLS on F5

1. Configure VIP for SSL Termination with Optional mTLS

Create a Client SSL Profile for Standard SSL:
- Go to Local Traffic > Profiles > SSL > Client.
- Create a new Client SSL profile for the VIP.
- Import and assign the server certificate and private key for the F5 load balancer, enabling standard SSL termination for incoming HTTPS requests.
- Set Client Certificate to Ignore or Optional for this profile. This setting allows both clients that do not have a client certificate and clients with a certificate to connect securely.
Create an Additional Client SSL Profile for mTLS:
- Create a second Client SSL Profile specifically for mTLS.
- Assign the F5’s server certificate and private key as before.
- Set Client Certificate to Require and specify the CA certificate that will validate incoming client certificates.
- In Configuration > Authentication, select Require or Request to mandate client certificate validation for mTLS connections.
Attach Both SSL Profiles to the VIP:
- Attach both the standard SSL profile and mTLS SSL profile to the same VIP.
- The F5 will now support both types of SSL connections (standard and mTLS) for incoming traffic.

2. Backend SSL Configuration (Optional)

If you want end-to-end SSL or mTLS between the F5 and backend servers:

Create a Server SSL Profile for Backend SSL:
- Go to Local Traffic > Profiles > SSL > Server and create a new Server SSL Profile.
- Specify a trusted CA certificate if backend servers require validation of the F5’s certificate for mTLS.
- Attach this Server SSL Profile to the backend pool so the F5 will establish an encrypted connection to the backend servers.
- For mutual TLS to backend servers, configure the backend servers (e.g., Kong) to validate the F5’s client certificate.

3. Test SSL and mTLS Connections

SSL Connection:
- Test a standard SSL connection by accessing the VIP without providing a client certificate.
- The F5 should accept the connection securely without requiring a client certificate.
mTLS Connection:
- Test an mTLS connection by providing a valid client certificate signed by the trusted CA.
- The F5 should validate the client certificate before establishing the connection.

Summary

SSL and mTLS Profiles: Attach both a standard SSL profile (with client certificate optional or ignored) and an mTLS SSL profile (with client certificate required) to the same VIP.
Optional Backend mTLS: Optionally, configure mTLS for connections from the F5 to backend servers if end-to-end mutual authentication is required.
Client Experience: Clients that support mTLS can authenticate with certificates, while clients without certificates can still connect over standard SSL.

This configuration allows the F5 to handle both SSL and mTLS connections on the same endpoint, supporting secure flexibility in handling a range of client needs and security requirements.

Common Issues and Resolutions

1. Certificate Verification Failed

If Kong logs errors like:

unable to get local issuer certificate
certificate verify failed

Cause

F5 is presenting a certificate that Kong cannot validate because the CA is not trusted or the certificate chain is incomplete.

Solution

Verify F5 Certificate Chain:
- Ensure F5 is presenting the full certificate chain, including intermediate and root certificates.
- On F5, upload the intermediate and root certificates alongside the server certificate.

Steps in F5:

Go to System → File Management → SSL Certificate List.
Import the intermediate and root certificates if missing.
Assign them to the SSL profile.
Add the Root CA to Kong:
- Export the root certificate (and intermediate certificate, if needed) from F5.
- Add the CA to Kong’s trusted store:

curl -i -X POST http://<KONG_ADMIN_API>:8001/ca_certificates \

–data “cert=$(cat /path/to/root_ca.pem)”

Enable Certificate Validation in Kong:
- Ensure the tls_verify option is enabled for services connecting to F5:

curl -i -X PATCH http://<KONG_ADMIN_API>:8001/services/<SERVICE_NAME_OR_ID> \

–data “tls_verify=true”

2. SNI Mismatch

If Kong logs errors like:

SSL: certificate name does not match

Cause

The Server Name Indication (SNI) sent by Kong does not match the hostname in F5’s SSL certificate.

Solution

Verify F5 SSL Certificate:
- Ensure the certificate on F5 is issued for the hostname used by Kong.
- Use a tool like openssl to check the F5 certificate:

openssl s_client -connect <F5_VIP>:443 -showcerts

Set SNI in Kong:
- Specify the correct SNI for the service in Kong:

bash

Copy code

curl -i -X PATCH http://<KONG_ADMIN_API>:8001/services/<SERVICE_NAME_OR_ID> \

–data “tls_verify=true” \

–data “tls_verify_depth=2” \

–data “sni=<F5_HOSTNAME>”

3. Mutual TLS (mTLS) Configuration

If using mTLS, errors may include:

SSL handshake failed
no client certificate presented

Cause

Kong is not presenting a client certificate, or F5 is not configured to validate the client certificate.

Solution

Upload Client Certificate to Kong:
- Add the client certificate and private key to Kong:

bash

curl -i -X POST http://<KONG_ADMIN_API>:8001/certificates \

–data “cert=$(cat /path/to/client_certificate.pem)” \

–data “key=$(cat /path/to/client_key.pem)”

Associate the Certificate with the Service:
- Attach the certificate to the service connecting to F5:

bash

Copy code

curl -i -X PATCH http://<KONG_ADMIN_API>:8001/services/<SERVICE_NAME_OR_ID> \

–data “client_certificate=<CERTIFICATE_ID>”

Enable Client Certificate Validation on F5:
- On F5, enable client certificate authentication in the SSL profile:
  - Go to Local Traffic → SSL Profiles → Edit the profile.
  - Enable Require Client Certificate.
  - Upload the CA certificate that issued the client certificate.

4. Protocol or Cipher Mismatch

Errors like:

SSL routines:ssl_choose_client_version:unsupported protocol
ssl_cipher_list failure

Cause

Mismatch in SSL protocols or ciphers supported by F5 and Kong.

Solution

Check SSL Protocols and Ciphers on F5:
- Ensure F5 supports the protocols (e.g., TLS 1.2/1.3) and ciphers used by Kong.
- Modify the F5 SSL profile to include compatible protocols and ciphers.
Set Cipher Suites in Kong:
- Update Kong’s nginx_kong.conf to include compatible ciphers:

nginx

ssl_ciphers HIGH:!aNULL:!MD5;

ssl_protocols TLSv1.2 TLSv1.3;

Restart Kong after the update.

5. Untrusted Self-Signed Certificate

If F5 uses a self-signed certificate, Kong cannot validate it by default.

Solution

Export the self-signed certificate from F5.
Add the certificate to Kong’s trusted CA store:

bash

Copy code

curl -i -X POST http://<KONG_ADMIN_API>:8001/ca_certificates \

–data “cert=$(cat /path/to/self_signed_certificate.pem)”

Best Practices

Use Valid Certificates:
- Always use certificates from trusted Certificate Authorities (CAs) for production systems.
Enable Logging:
- Monitor logs in Kong and F5 to troubleshoot SSL/TLS issues.
Regular Certificate Rotation:
- Ensure certificates are renewed and updated before expiry.
Secure Configuration:
- Use modern TLS protocols (e.g., TLS 1.2/1.3) and strong cipher suites.

By addressing these common issues, you can ensure smooth integration between F5 and Kong Gateway with robust SSL/TLS security.

Common Issues and Resolutions

1. Certificate Verification Failed

If Kong logs errors like:

unable to get local issuer certificate
certificate verify failed

Cause

F5 is presenting a certificate that Kong cannot validate because the CA is not trusted or the certificate chain is incomplete.

Solution

Verify F5 Certificate Chain:
- Ensure F5 is presenting the full certificate chain, including intermediate and root certificates.
- On F5, upload the intermediate and root certificates alongside the server certificate.

Steps in F5:

Go to System → File Management → SSL Certificate List.
Import the intermediate and root certificates if missing.
Assign them to the SSL profile.
Add the Root CA to Kong:
- Export the root certificate (and intermediate certificate, if needed) from F5.
- Add the CA to Kong’s trusted store:

curl -i -X POST http://<KONG_ADMIN_API>:8001/ca_certificates \

–data “cert=$(cat /path/to/root_ca.pem)”

Enable Certificate Validation in Kong:
- Ensure the tls_verify option is enabled for services connecting to F5:

curl -i -X PATCH http://<KONG_ADMIN_API>:8001/services/<SERVICE_NAME_OR_ID> \

–data “tls_verify=true”

2. SNI Mismatch

If Kong logs errors like:

SSL: certificate name does not match

Cause

The Server Name Indication (SNI) sent by Kong does not match the hostname in F5’s SSL certificate.

Solution

Verify F5 SSL Certificate:
- Ensure the certificate on F5 is issued for the hostname used by Kong.
- Use a tool like openssl to check the F5 certificate:

openssl s_client -connect <F5_VIP>:443 -showcerts

Set SNI in Kong:
- Specify the correct SNI for the service in Kong:

curl -i -X PATCH http://<KONG_ADMIN_API>:8001/services/<SERVICE_NAME_OR_ID> \

–data “tls_verify=true” \

–data “tls_verify_depth=2” \

–data “sni=<F5_HOSTNAME>”

3. Mutual TLS (mTLS) Configuration

If using mTLS, errors may include:

SSL handshake failed
no client certificate presented

Cause

Kong is not presenting a client certificate, or F5 is not configured to validate the client certificate.

Solution

Upload Client Certificate to Kong:
- Add the client certificate and private key to Kong:

curl -i -X POST http://<KONG_ADMIN_API>:8001/certificates \

–data “cert=$(cat /path/to/client_certificate.pem)” \

–data “key=$(cat /path/to/client_key.pem)”

Associate the Certificate with the Service:
- Attach the certificate to the service connecting to F5:

curl -i -X PATCH http://<KONG_ADMIN_API>:8001/services/<SERVICE_NAME_OR_ID> \

–data “client_certificate=<CERTIFICATE_ID>”

Enable Client Certificate Validation on F5:
- On F5, enable client certificate authentication in the SSL profile:
  - Go to Local Traffic → SSL Profiles → Edit the profile.
  - Enable Require Client Certificate.
  - Upload the CA certificate that issued the client certificate.

4. Protocol or Cipher Mismatch

Errors like:

SSL routines:ssl_choose_client_version:unsupported protocol
ssl_cipher_list failure

Cause

Mismatch in SSL protocols or ciphers supported by F5 and Kong.

Solution

Check SSL Protocols and Ciphers on F5:
- Ensure F5 supports the protocols (e.g., TLS 1.2/1.3) and ciphers used by Kong.
- Modify the F5 SSL profile to include compatible protocols and ciphers.
Set Cipher Suites in Kong:
- Update Kong’s nginx_kong.conf to include compatible ciphers:

nginx

ssl_ciphers HIGH:!aNULL:!MD5;

ssl_protocols TLSv1.2 TLSv1.3;

Restart Kong after the update.

5. Untrusted Self-Signed Certificate

If F5 uses a self-signed certificate, Kong cannot validate it by default.

Solution

Export the self-signed certificate from F5.
Add the certificate to Kong’s trusted CA store:

curl -i -X POST http://<KONG_ADMIN_API>:8001/ca_certificates \

–data “cert=$(cat /path/to/self_signed_certificate.pem)”

Best Practices

Use Valid Certificates:
- Always use certificates from trusted Certificate Authorities (CAs) for production systems.
Enable Logging:
- Monitor logs in Kong and F5 to troubleshoot SSL/TLS issues.
Regular Certificate Rotation:
- Ensure certificates are renewed and updated before expiry.
Secure Configuration:
- Use modern TLS protocols (e.g., TLS 1.2/1.3) and strong cipher suites.

By addressing these common issues, you can ensure smooth integration between F5 and Kong Gateway with robust SSL/TLS security.

Folder structure

1) app/package.json

2) app/server.js

3) Dockerfile

4) .dockerignore

5) kong/kong.yml

6) compose.yml

7) Run it

8) Test rate limiting

9) Useful checks

10) What makes this different from Traefik

11) Resume line

12) Best next upgrade

0) Topology (recommended)

1) Cloudflare (edge) setup

2) F5 LTM (VIP to Kong) essentials

3) Kong Gateway settings (behind F5)

4) Putting it together (request path)

5) Testing (in layers)

6) Observability (recommended)

7) Common gotchas (and quick fixes)

1) The traffic flow (happy path)

2) TLS choices (pick one per hop)

3) Real client IP (do this or logs/limits will be wrong)

4) Load balancing & health checks (avoid double “mystery” failover)

5) Protocols & connections

6) Minimal config snippets

7) Observability (what to turn on)

8) Common pitfalls & quick fixes

9) Example end-to-end (typical, secure)

1. Keystore / Truststore Formats

Example Spring Boot Config:

2. Certificate Chain Requirements

3. Protocol Configuration (TLS)

4. Cipher Suites

5. Mutual TLS (mTLS)

6. Generating Keystore & Truststore

7. Spring Boot with Reverse Proxies (e.g., Kong, F5)

Summary: What to Account For

How F5 Can Cause 401 Errors in Kong

How to Test if F5 is the Cause

Fix Recommendations

Quick Summary of Key Differences

Intermittent 401 + 400: Common Root Causes

1. Expired or Reused Tokens

2. Multiple Consumers with Invalid Secrets

3. Kong Not Forwarding Tokens Correctly

🔸 4. OIDC Plugin Misconfiguration

How to Debug Effectively

Why You Can’t Have Duplicate Routes

Workarounds / Alternatives

🧪 Example: Two Routes with Same Path, Different Hosts

Summary

Common Causes

🔍 What to Check When You See upstream_status=504

1. Verify Upstream is Healthy

2. Increase Kong Timeouts

3. Check Protocol Mismatch (HTTP vs HTTPS)

4. Check Kong Error Log

5. Check Health of Upstream Targets (If Using Upstreams)

6. Review Load Balancer / Firewall Rules

Example Full Troubleshooting Flow

QUICK FIX CHECKLIST

1. Test the Backend Directly

2. Increase Kong Timeouts

3. Check Protocol Mismatch

4. Check Kong Logs

5. Enable Correlation ID for Tracing

Common Root Causes

How to Troubleshoot

1. Check Logs

2. Test Upstream Manually

3. Verify Protocol

4. Increase Read Timeout

5. Check Upstream for Non-Standard Response

🧪 Bonus: Use Request Termination for Testing

1) `app/package.json`

2) `app/server.js`

3) `Dockerfile`

4) `.dockerignore`

5) `kong/kong.yml`

6) `compose.yml`

🔍 What to Check When You See `upstream_status=504`