error kong

techhadoop

tail -f /usr/local/kong/logs/error.log | egrep -i ‘oidc|jwt|introspect|jwks|bearer|signature|aud|iss’
Typical log clues → fixes:

Got it—here’s a fast, practical playbook for “invalid bearer token” in a Kong + Ping Identity setup (with F5 in front).

Quick triage (60‑second checks)

  1. Header format
    • Must be exactly: Authorization: Bearer <access_token> (case‑sensitive “Bearer”, single space, no quotes).
  2. Right token type
    • Use the Access Token, not the ID token.
  3. Header delivery
    • Ensure F5/LB isn’t stripping Authorization. Hit Kong directly to compare.
# Direct to Kong (bypass F5 if you can)
curl -si https://kong.example.com/api \
  -H "Authorization: Bearer $AT"

Validate the JWT (if your AT is JWT)

1) Is it a JWT?

A JWT has two dots: xxxxx.yyyyy.zzzzz. If not, it’s likely opaque → skip to “Opaque tokens”.

2) Decode & inspect claims

# header
cut -d. -f1 <<<"$AT" | base64 -d 2>/dev/null | jq
# payload
cut -d. -f2 <<<"$AT" | base64 -d 2>/dev/null | jq

Check:

  • iss equals your Ping issuer in Kong config
  • aud includes your API audience expected by Kong
  • exp is in the future; nbf is not in the future (watch clock skew)
  • scope / roles include what your route requires

3) Signature / key material

From the JWT header (kid, alg) confirm Kong can fetch the matching JWKS:

  • Kong must reach Ping’s /.well-known/openid-configuration → jwks_uri
  • No corporate proxy/DNS blocking
  • alg matches what the IdP issues (e.g., RS256)

Validate with Kong

Turn on debug temporarily and reproduce once:

KONG_LOG_LEVEL=debug kong restart
tail -f /usr/local/kong/logs/error.log | egrep -i 'oidc|jwt|introspect|jwks|bearer|signature|aud|iss'

Typical log clues → fixes:

  • signature verification failed → wrong JWKS, stale kid, or mismatched alg.
  • token expired → exp passed; consider small clock_skew_leeway.
  • audience not allowed / aud mismatch → adjust audience in IdP or audience_claim/allowed_audiences in plugin.
  • issuer not allowed → fix issuer (exact string match).
  • could not get jwks → networking/proxy or TLS trust to Ping.

Common F5/LB gotchas

  • Missing Authorization header forwarding. On F5, ensure policies/iRules preserve it.
  • Rewriting to lower/upper case header names is fine; removing isn’t.

Opaque tokens (non‑JWT access tokens)

If the access token looks random (no dots), enable introspection:

  • Kong openid-connect plugin must have introspection_endpoint, client_id, client_secret.
  • Kong needs network access to Ping’s introspection endpoint.
  • Expect log lines: introspection active / token active:false if invalid or revoked.

Kong plugin configuration — key items to review

# For openid-connect plugin (per service/route)
config:
  issuer: "https://ping.example.com/as"
  discovery: "https://ping.example.com/as/.well-known/openid-configuration"
  # If using opaque tokens:
  introspection_endpoint: "https://ping.example.com/as/introspect.oauth2"
  client_id: "<kong-oidc-client>"
  client_secret: "<secret>"
  # Validation:
  verify_signature: true
  verify_claims: true
  allowed_audiences:
    - "api://my-api"
  scopes_required:
    - "api.read"
  # Optional:
  clock_skew_leeway: 60
  bearer_only: true   # if you don’t want redirects (API-only)

Upstream expectations (post‑auth)

Decide what the upstream needs:

  • Forward original token: config.upstream_access_token_header = "Authorization" (default)
  • Or forward derived identity headers (customize via config.upstream_headers_claims).
    Ensure your upstream checks the same audience/scope semantics (or trusts Kong’s verdict and uses headers only).

Quick curl matrix

# Good: Access token with proper prefix
curl -H "Authorization: Bearer $AT" https://api.example.com/orders

# Bad: ID token by mistake
curl -H "Authorization: Bearer $IDT" https://api.example.com/orders   # expect 401

# Test over F5 vs direct Kong
curl -si -H "Authorization: Bearer $AT" https://edge.example.com/orders
curl -si -H "Authorization: Bearer $AT" https://kong.example.com/orders

Frequent root causes & fixes

  • Typo / missing “Bearer ” → fix header format.
  • Using ID token → request AT (auth code or client credentials).
  • Expired token → renew; verify system clocks (NTP) on F5, Kong, and upstream.
  • Issuer/audience mismatch → align Ping client + API resources with Kong allowed_audiences and issuer.
  • JWKS fetch failure → open egress, fix proxy/TLS trust, or pin jwks_uri.
  • Opaque token with no introspection configured → enable introspection_endpoint + client creds.
  • F5 stripped header → preserve Authorization.
  • Scope missing → include required scopes in token request or relax scopes_required (if appropriate).

If you paste a redacted sample token header/payload (no signature needed) and your current openid-connect plugin snippet, I’ll pinpoint exactly which check is failing and provide the minimal config change or Ping policy tweak.

Leave a comment