latency

techhadoop

That message comes from Nginx (which Kong runs on). It means the upstream response didn’t fit in memory buffers, so Nginx spooled it to disk under /usr/local/kong/proxy_temp. It’s not fatal, but it adds I/O and latency.

Here are practical fixes—pick what matches your API pattern.

What you can do

1) Quick stopgap: don’t write to disk

Prevents temp-file writes; response still buffered in memory.

proxy_buffering on;
proxy_max_temp_file_size 0;

2) Stream responses (no buffering at all)

Great for large downloads/streaming; reduces latency & disk I/O. (Backpressure goes directly to upstream.)

proxy_buffering off;

3) Increase memory buffers (keep buffering, avoid disk)

Size these to your typical response size and concurrency.

proxy_buffering on;
proxy_buffer_size 64k;        # header/first buffer
proxy_buffers 32 64k;         # total ~2 MB per connection here
proxy_busy_buffers_size 256k; # busy threshold before spooling

4) If you must spool, make it fast

Put Kong’s proxy temp on tmpfs or faster disk:

  • Mount /usr/local/kong/proxy_temp on tmpfs (container/VM)
  • Or move it via: proxy_temp_path /usr/local/kong/proxy_temp 1 2;

5) Tame slow clients (common cause of spooling)

Slow downloads force Nginx to hold data. Tighten/adjust:

send_timeout 30s;
tcp_nodelay on;
keepalive_timeout 65s;

(Shorter timeouts reduce long-lived slow sends.)

How to set these in Kong

Pick one method you already use:

A) With kong.conf

Add proxy-level directives (no need to rebuild Kong):

nginx_proxy_directives = 
  proxy_buffering=on,
  proxy_max_temp_file_size=0,
  proxy_buffer_size=64k,
  proxy_buffers=32 64k,
  proxy_busy_buffers_size=256k

(Comma-separated key=value pairs; restart/reload Kong.)

B) With environment variables (containers)

KONG_NGINX_PROXY_DIRECTIVES=proxy_buffering=on,proxy_max_temp_file_size=0,proxy_buffer_size=64k,proxy_buffers=32 64k,proxy_busy_buffers_size=256k

C) With a custom Nginx template

If you already manage an nginx-kong.conf template, put the directives in the proxy context and redeploy.

D) Helm (Kubernetes)

If using the Kong Helm chart, set the proxy directives block (values key often named like proxy.nginx.* or nginxProxyDirectives) to include the same directives. (Exact key names vary by chart version—mirror the options above.)

What to choose

  • APIs returning big files / streaming → proxy_buffering off;
  • APIs with occasional large JSON → keep buffering on and increase proxy_buffer_* sizes; also set proxy_max_temp_file_size 0;
  • Mobile or very slow clients → consider buffering on + larger buffers, or keep temp on tmpfs

Validate

  1. Hit the route and watch /usr/local/kong/proxy_temp usage.
  2. Check error log: the “buffered to a temporary file” line should stop.
  3. Watch p95/p99 request latency; it usually drops when disk spooling stops.

If you share your response size range and traffic pattern (download/JSON/streaming), I’ll tune exact buffer counts/sizes so you don’t overspend RAM.

Leave a comment