A 502 Bad Gateway error in your setup:
Client → LB → KONG → LB → Server
means that Kong Gateway cannot successfully connect to the upstream server behind the second load balancer. Here’s how to troubleshoot and resolve it step by step:
🔁 Understanding the Flow
- Client hits the first LB (e.g., AWS ELB, NGINX).
- That LB forwards traffic to Kong Gateway.
- Kong receives the request and proxies it to a second LB.
- That LB routes to your actual upstream service.
🛠️ Troubleshooting Checklist
✅ 1. Check Upstream URL in Kong
- Run:
curl -s http://<KONG_ADMIN>:8001/services/<service-name> - Ensure the upstream URL (
urlorhost) points to a valid, resolvable address (DNS or IP) and correct port.
Example:
{
"host": "internal-lb.yourdomain.local",
"port": 8080,
"protocol": "http"
}
✅ 2. Test Kong’s Network Reachability
From the Kong container or host:
curl -v http://internal-lb.yourdomain.local:8080/
If this fails:
- DNS resolution might be broken
- Port might be blocked
- LB might not route correctly
✅ 3. Enable Debug Logs in Kong
In kong.conf or via environment variable:
log_level = debug
Then check:
tail -f /usr/local/kong/logs/error.log
Look for messages like:
upstream timed outcould not resolve hostconnection refused
✅ 4. Check Health of Second LB and Backend
- Ensure second LB is up
- Verify backend servers are healthy and accepting connections
- Check if Kong’s IP is allowed (firewall or security group)
✅ 5. Check Kong Route & Service Configuration
Validate route is defined correctly:
curl -s http://<KONG_ADMIN>:8001/routes
Make sure paths, hosts, or methods match the request.
🧪 Example Kong Service & Route Setup
# Service pointing to internal load balancer
curl -i -X POST http://localhost:8001/services \
--data name=upstream-service \
--data url=http://internal-lb.yourdomain.local:8080
# Route for the service
curl -i -X POST http://localhost:8001/services/upstream-service/routes \
--data paths[]=/api
🚫 Common Causes of 502 with LB Behind Kong
| Problem | Solution |
|---|---|
| DNS resolution failure | Use IP or fix /etc/resolv.conf or CoreDNS |
| Port not exposed or wrong | Confirm port with nc or curl |
| Second LB not forwarding correctly | Check LB target groups and health checks |
| Kong plugins (e.g., OIDC, rate-limit) error | Disable plugins temporarily to isolate |
| HTTP vs HTTPS mismatch | Ensure protocol matches (http vs https) |
| Timeout too short | Increase proxy_read_timeout or similar |
✅ Final Tips
- Try
curldirectly from Kong to the backend server. - Use Kong’s health check endpoint if you’re using upstream targets:
curl http://localhost:8001/upstreams/<name>/health
If you share:
- the exact
curlcall to Kong - the relevant Kong service/route config
- error.log content from Kong
The error message “upstream prematurely closed connection while reading response header from upstream” in Kong Gateway indicates that Kong attempted to read the response headers from the upstream service, but the connection was closed unexpectedly before the headers were fully received. This typically results in a 502 Bad Gateway error.
Common Causes
- Upstream Service Crashes or Terminates Connection Early:
- The upstream application may crash, encounter an error, or intentionally close the connection before sending a complete response.
- Timeouts:
- The upstream service takes too long to respond, exceeding Kong’s configured timeouts.
- Keepalive Connection Issues:
- Persistent connections (keepalive) between Kong and the upstream service may be closed unexpectedly by the upstream, leading to this error.
- Protocol Mismatch:
- Kong expects a certain protocol (e.g., HTTP/1.1), but the upstream service responds differently or uses an incompatible protocol.
- Large Response Headers:
- The upstream service sends headers that exceed Kong’s buffer sizes, causing the connection to be closed prematurely.
es, a mismatch between the protocol specified in Kong’s service configuration and the actual protocol used by the upstream service can lead to the error:
“upstream prematurely closed connection while reading response header from upstream”
This typically occurs when Kong attempts to communicate with an upstream service over HTTP, but the upstream expects HTTPS, or vice versa.
🔍 Understanding the Issue
When Kong is configured to connect to an upstream service, it uses the protocol specified in the service’s configuration. If the upstream service expects HTTPS connections and Kong is configured to use HTTP, the SSL/TLS handshake will fail, leading to the connection being closed prematurely.
Yes, a mismatch between the protocol specified in Kong’s service configuration and the actual protocol used by the upstream service can lead to the error:
“upstream prematurely closed connection while reading response header from upstream”
This typically occurs when Kong attempts to communicate with an upstream service over HTTP, but the upstream expects HTTPS, or vice versa.
🔍 Understanding the Issue
When Kong is configured to connect to an upstream service, it uses the protocol specified in the service’s configuration. If the upstream service expects HTTPS connections and Kong is configured to use HTTP, the SSL/TLS handshake will fail, leading to the connection being closed prematurely.
For example, if your upstream service is accessible at https://api.example.com, but Kong is configured with:
bashCopyEditcurl -i -X POST http://localhost:8001/services \
--data name=example-service \
--data url=http://api.example.com
Kong will attempt to connect over HTTP, resulting in a failed connection.
✅ Solution
Ensure that the protocol in Kong’s service configuration matches the protocol expected by the upstream service.
If the upstream service requires HTTPS, configure the service in Kong accordingly:
bashCopyEditcurl -i -X POST http://localhost:8001/services \
--data name=example-service \
--data url=https://api.example.com
This ensures that Kong establishes a secure connection using HTTPS, aligning with the upstream service’s expectations.
Infra Cloud Solutions 