F5 – kong configuration

techhadoop , , , ,

Configure the F5 Load Balancer with VIP and SSL Certificate

  1. Create a Virtual Server (VIP):
    • Log in to your F5 management console.
    • Navigate to Local Traffic > Virtual Servers > Virtual Server List.
    • Click Create and configure the following:
      • Name: Give the VIP a meaningful name, like Kong_VIP.
      • Destination Address: Specify the IP address for the VIP.
      • Service Port: Set to 443 for HTTPS.
  2. Assign an SSL Certificate to the VIP:
    • Under the SSL Profile (Client) section, select Custom.
    • For Client SSL Profile, choose an existing SSL profile, or create a new one if needed:
      • Go to Local Traffic > Profiles > SSL > Client.
      • Click Create and provide a name, then upload the SSL certificate and key.
    • Assign this SSL profile to your VIP.
  3. Configure Load Balancing Method:
    • Under Load Balancing Method, choose a method that best fits your setup, such as Round Robin or Least Connections.
  4. Set Up Pool and Pool Members:
    • In the Pool section, create or select a pool to add your Kong instances as members:
      • Go to Local Traffic > Pools > Pool List, then Create a new pool.
      • Assign Kong instances as Pool Members using their internal IP addresses and ports (usually port 8000 for HTTP or 8443 for HTTPS if Kong is configured with SSL).
    • Make sure health monitors are set up for these pool members to detect when a Kong instance goes down.

Setup

Whether you need certificates on both the F5 load balancer and the Kong servers depends on how you plan to manage SSL/TLS termination and the level of encryption required for traffic between the F5 and Kong.

Here are two common setups:

1. SSL Termination on the F5 (Most Common)

  • Certificate Location: Only on the F5 load balancer.
  • How It Works: The F5 terminates the SSL connection with clients, decrypts the incoming HTTPS traffic, and forwards it to the Kong servers as plain HTTP traffic.
  • Benefits: Reduces the overhead on Kong servers because they don’t need to handle SSL encryption. It’s simpler to manage as only the F5 requires an SSL certificate.
  • Considerations: Traffic between the F5 and Kong servers is unencrypted, which is typically acceptable in private or secured networks (e.g., within a secure data center or VPC).

Configuration Steps:

  • Install and configure the SSL certificate only on the F5.
  • Set the F5 VIP to listen on HTTPS (port 443).
  • Configure Kong to listen on HTTP (port 8000 or a custom port).

This setup is generally sufficient if Kong instances and the F5 are within a trusted network.

2. End-to-End SSL (SSL Termination on Both F5 and Kong Servers)

  • Certificate Location: On both the F5 load balancer and the Kong servers.
  • How It Works: The F5 terminates the initial SSL connection from the client, but then re-encrypts the traffic before forwarding it to Kong. Kong servers also have SSL certificates, allowing them to decrypt this re-encrypted traffic.
  • Benefits: Ensures encrypted communication all the way from the client to the Kong servers, providing an extra layer of security.
  • Considerations: Requires SSL certificates on both F5 and Kong, and introduces some additional CPU overhead on Kong due to the need to decrypt/encrypt traffic.

Configuration Steps:

  • Install and configure an SSL certificate on the F5 for the VIP, and configure the VIP to listen on HTTPS.
  • Install an SSL certificate on each Kong server (these can be the same certificate as the F5 or separate ones).
  • Configure Kong to listen on HTTPS (port 8443 or another SSL-enabled port).
  • Configure the F5 to forward encrypted traffic to the Kong servers’ HTTPS port.

This approach is beneficial if security policies require end-to-end encryption, especially if the connection between the F5 and Kong instances crosses less secure network segments.

Summary

  • If you’re terminating SSL on the F5 only: A certificate is only required on the F5.
  • If you need end-to-end encryption: SSL certificates are required on both the F5 and each Kong server.

In most enterprise environments, SSL termination on the F5 is sufficient. However, for environments with stringent security requirements, end-to-end SSL provides additional security by ensuring all connections remain encrypted.

mTLS

  • Certificates:
    • The F5 load balancer needs server and client certificates and CA certificates to validate clients (client mTLS) and backend servers (backend mTLS).
    • Backend servers need server certificates and may require a CA certificate to verify the F5’s client certificate.
  • mTLS Setup:
    • For mTLS from the client to the F5, configure the Client SSL Profile on the F5.
    • For mTLS between the F5 and backend servers, configure the Server SSL Profile and set backend servers (like Kong) to require mTLS by validating the F5’s client certificate.

This setup provides mutual authentication between clients, the F5, and backend servers, ensuring secure, verified communication across all components in the chain.

do i need a certificate on both servers and F5 load balancer also how i integrate with mtls ?

For mTLS (mutual TLS) integration with an F5 load balancer and downstream servers like Kong or another backend, it is essential to determine where the certificates are needed and how to configure mTLS for mutual authentication between clients, the F5, and backend servers.

Certificates on Both the F5 Load Balancer and Backend Servers

  1. Certificates on the F5 Load Balancer:
    • The F5 acts as the entry point for client connections and needs a server certificate to handle HTTPS traffic.
    • If using mTLS, the F5 will also need a client certificate and a trusted certificate authority (CA) to validate incoming client certificates.
    • The F5 can be configured to terminate SSL and optionally re-encrypt traffic to backend servers.
  2. Certificates on Backend Servers:
    • For end-to-end encryption (where traffic from the F5 to backend servers remains encrypted), each backend server (e.g., Kong) also needs a server certificate.
    • If mutual TLS is required between the F5 and backend servers, the backend servers also need to verify the client (F5’s) certificate, so you’ll need to import the F5’s client certificate or a shared CA certificate on backend servers.

Configuring mTLS on F5 Load Balancer

Here’s how you can set up mTLS on an F5 load balancer to handle mutual authentication with clients and potentially with backend servers:

1. Configure mTLS Between Client and F5

  • Client SSL Profile:
    • Go to Local Traffic > Profiles > SSL > Client.
    • Create a Client SSL Profile for the VIP and enable Client Certificate Authentication by selecting Require under Client Certificate.
    • Import or reference a CA certificate that you trust to sign client certificates. This CA will validate client certificates.
  • Assign SSL Profile to VIP:
    • Attach this client SSL profile to the VIP handling client requests.
    • The F5 will now require clients to present a valid certificate from the specified CA to establish a secure connection.

2. mTLS Between F5 and Backend Servers (Optional)

If you want end-to-end mTLS (client to F5 and F5 to backend):

  • Server SSL Profile:
    • Go to Local Traffic > Profiles > SSL > Server.
    • Create a Server SSL Profile and enable the Authenticate option to require the backend server to present a valid certificate.
    • Specify a trusted CA certificate to validate the backend server’s certificate.
  • Assign Server SSL Profile to Pool:
    • Attach this server SSL profile to the backend pool so that the F5 will establish an mTLS connection when connecting to each backend server.

Backend Server Configuration (e.g., Kong)

If Kong is the backend server, configure Kong to:

  • Present a server certificate to the F5 for mutual authentication.
  • Verify client certificates if mTLS is required from F5 to Kong:
    • Set client_ssl = on and configure ssl_client_certificate to reference the CA certificate or client certificates you trust.

Example snippet for kong.conf:

client_ssl = on

ssl_cert = /path/to/server.crt

ssl_cert_key = /path/to/server.key

ssl_client_certificate = /path/to/ca.crt  # This will be used to verify F5’s client certificate

Summary

  • Certificates:
    • The F5 load balancer needs server and client certificates and CA certificates to validate clients (client mTLS) and backend servers (backend mTLS).
    • Backend servers need server certificates and may require a CA certificate to verify the F5’s client certificate.
  • mTLS Setup:
    • For mTLS from the client to the F5, configure the Client SSL Profile on the F5.
    • For mTLS between the F5 and backend servers, configure the Server SSL Profile and set backend servers (like Kong) to require mTLS by validating the F5’s client certificate.

This setup provides mutual authentication between clients, the F5, and backend servers, ensuring secure, verified communication across all components in the chain.

setup both SSL and mTLS

Yes, you can absolutely have a setup with both SSL termination and mTLS on the F5 load balancer. Here’s how the setup would work, allowing for both standard SSL connections (for regular HTTPS traffic) and mTLS (for additional security and mutual authentication) on the same VIP.

Mixed SSL and mTLS on F5

The configuration would involve:

  1. Standard SSL Termination for clients that only need secure (HTTPS) connections.
  2. mTLS configuration for clients requiring mutual authentication (client certificate verification).

Steps to Set Up SSL and mTLS on F5

1. Configure VIP for SSL Termination with Optional mTLS

  1. Create a Client SSL Profile for Standard SSL:
    • Go to Local Traffic > Profiles > SSL > Client.
    • Create a new Client SSL profile for the VIP.
    • Import and assign the server certificate and private key for the F5 load balancer, enabling standard SSL termination for incoming HTTPS requests.
    • Set Client Certificate to Ignore or Optional for this profile. This setting allows both clients that do not have a client certificate and clients with a certificate to connect securely.
  2. Create an Additional Client SSL Profile for mTLS:
    • Create a second Client SSL Profile specifically for mTLS.
    • Assign the F5’s server certificate and private key as before.
    • Set Client Certificate to Require and specify the CA certificate that will validate incoming client certificates.
    • In Configuration > Authentication, select Require or Request to mandate client certificate validation for mTLS connections.
  3. Attach Both SSL Profiles to the VIP:
    • Attach both the standard SSL profile and mTLS SSL profile to the same VIP.
    • The F5 will now support both types of SSL connections (standard and mTLS) for incoming traffic.

2. Backend SSL Configuration (Optional)

If you want end-to-end SSL or mTLS between the F5 and backend servers:

  1. Create a Server SSL Profile for Backend SSL:
    • Go to Local Traffic > Profiles > SSL > Server and create a new Server SSL Profile.
    • Specify a trusted CA certificate if backend servers require validation of the F5’s certificate for mTLS.
    • Attach this Server SSL Profile to the backend pool so the F5 will establish an encrypted connection to the backend servers.
    • For mutual TLS to backend servers, configure the backend servers (e.g., Kong) to validate the F5’s client certificate.

3. Test SSL and mTLS Connections

  1. SSL Connection:
    • Test a standard SSL connection by accessing the VIP without providing a client certificate.
    • The F5 should accept the connection securely without requiring a client certificate.
  2. mTLS Connection:
    • Test an mTLS connection by providing a valid client certificate signed by the trusted CA.
    • The F5 should validate the client certificate before establishing the connection.

Summary

  • SSL and mTLS Profiles: Attach both a standard SSL profile (with client certificate optional or ignored) and an mTLS SSL profile (with client certificate required) to the same VIP.
  • Optional Backend mTLS: Optionally, configure mTLS for connections from the F5 to backend servers if end-to-end mutual authentication is required.
  • Client Experience: Clients that support mTLS can authenticate with certificates, while clients without certificates can still connect over standard SSL.

This configuration allows the F5 to handle both SSL and mTLS connections on the same endpoint, supporting secure flexibility in handling a range of client needs and security requirements.

Common Issues and Resolutions

1. Certificate Verification Failed

If Kong logs errors like:

  • unable to get local issuer certificate
  • certificate verify failed

Cause

  • F5 is presenting a certificate that Kong cannot validate because the CA is not trusted or the certificate chain is incomplete.

Solution

  1. Verify F5 Certificate Chain:
    • Ensure F5 is presenting the full certificate chain, including intermediate and root certificates.
    • On F5, upload the intermediate and root certificates alongside the server certificate.

Steps in F5:

  1. Go to System → File Management → SSL Certificate List.
  2. Import the intermediate and root certificates if missing.
  3. Assign them to the SSL profile.
  4. Add the Root CA to Kong:
    • Export the root certificate (and intermediate certificate, if needed) from F5.
    • Add the CA to Kong’s trusted store:

curl -i -X POST http://<KONG_ADMIN_API&gt;:8001/ca_certificates \

  –data “cert=$(cat /path/to/root_ca.pem)”

  1. Enable Certificate Validation in Kong:
    • Ensure the tls_verify option is enabled for services connecting to F5:

curl -i -X PATCH http://<KONG_ADMIN_API&gt;:8001/services/<SERVICE_NAME_OR_ID> \

  –data “tls_verify=true”

2. SNI Mismatch

If Kong logs errors like:

  • SSL: certificate name does not match

Cause

  • The Server Name Indication (SNI) sent by Kong does not match the hostname in F5’s SSL certificate.

Solution

  1. Verify F5 SSL Certificate:
    • Ensure the certificate on F5 is issued for the hostname used by Kong.
    • Use a tool like openssl to check the F5 certificate:

openssl s_client -connect <F5_VIP>:443 -showcerts

  1. Set SNI in Kong:
    • Specify the correct SNI for the service in Kong:

bash

Copy code

curl -i -X PATCH http://<KONG_ADMIN_API&gt;:8001/services/<SERVICE_NAME_OR_ID> \

  –data “tls_verify=true” \

  –data “tls_verify_depth=2” \

  –data “sni=<F5_HOSTNAME>”

3. Mutual TLS (mTLS) Configuration

If using mTLS, errors may include:

  • SSL handshake failed
  • no client certificate presented

Cause

  • Kong is not presenting a client certificate, or F5 is not configured to validate the client certificate.

Solution

  1. Upload Client Certificate to Kong:
    • Add the client certificate and private key to Kong:

bash

curl -i -X POST http://<KONG_ADMIN_API&gt;:8001/certificates \

  –data “cert=$(cat /path/to/client_certificate.pem)” \

  –data “key=$(cat /path/to/client_key.pem)”

  1. Associate the Certificate with the Service:
    • Attach the certificate to the service connecting to F5:

bash

Copy code

curl -i -X PATCH http://<KONG_ADMIN_API&gt;:8001/services/<SERVICE_NAME_OR_ID> \

  –data “client_certificate=<CERTIFICATE_ID>”

  1. Enable Client Certificate Validation on F5:
    • On F5, enable client certificate authentication in the SSL profile:
      • Go to Local Traffic → SSL Profiles → Edit the profile.
      • Enable Require Client Certificate.
      • Upload the CA certificate that issued the client certificate.

4. Protocol or Cipher Mismatch

Errors like:

  • SSL routines:ssl_choose_client_version:unsupported protocol
  • ssl_cipher_list failure

Cause

  • Mismatch in SSL protocols or ciphers supported by F5 and Kong.

Solution

  1. Check SSL Protocols and Ciphers on F5:
    • Ensure F5 supports the protocols (e.g., TLS 1.2/1.3) and ciphers used by Kong.
    • Modify the F5 SSL profile to include compatible protocols and ciphers.
  2. Set Cipher Suites in Kong:
    • Update Kong’s nginx_kong.conf to include compatible ciphers:

nginx

ssl_ciphers HIGH:!aNULL:!MD5;

ssl_protocols TLSv1.2 TLSv1.3;

  1. Restart Kong after the update.

5. Untrusted Self-Signed Certificate

If F5 uses a self-signed certificate, Kong cannot validate it by default.

Solution

  1. Export the self-signed certificate from F5.
  2. Add the certificate to Kong’s trusted CA store:

bash

Copy code

curl -i -X POST http://<KONG_ADMIN_API&gt;:8001/ca_certificates \

  –data “cert=$(cat /path/to/self_signed_certificate.pem)”

Best Practices

  • Use Valid Certificates:
    • Always use certificates from trusted Certificate Authorities (CAs) for production systems.
  • Enable Logging:
    • Monitor logs in Kong and F5 to troubleshoot SSL/TLS issues.
  • Regular Certificate Rotation:
    • Ensure certificates are renewed and updated before expiry.
  • Secure Configuration:
    • Use modern TLS protocols (e.g., TLS 1.2/1.3) and strong cipher suites.

By addressing these common issues, you can ensure smooth integration between F5 and Kong Gateway with robust SSL/TLS security. Let me know if you need detailed help with any specific step!

Common Issues and Resolutions

1. Certificate Verification Failed

If Kong logs errors like:

  • unable to get local issuer certificate
  • certificate verify failed

Cause

  • F5 is presenting a certificate that Kong cannot validate because the CA is not trusted or the certificate chain is incomplete.

Solution

  1. Verify F5 Certificate Chain:
    • Ensure F5 is presenting the full certificate chain, including intermediate and root certificates.
    • On F5, upload the intermediate and root certificates alongside the server certificate.

Steps in F5:

  1. Go to System → File Management → SSL Certificate List.
  2. Import the intermediate and root certificates if missing.
  3. Assign them to the SSL profile.
  4. Add the Root CA to Kong:
    • Export the root certificate (and intermediate certificate, if needed) from F5.
    • Add the CA to Kong’s trusted store:

curl -i -X POST http://<KONG_ADMIN_API&gt;:8001/ca_certificates \

  –data “cert=$(cat /path/to/root_ca.pem)”

  1. Enable Certificate Validation in Kong:
    • Ensure the tls_verify option is enabled for services connecting to F5:

curl -i -X PATCH http://<KONG_ADMIN_API&gt;:8001/services/<SERVICE_NAME_OR_ID> \

  –data “tls_verify=true”

2. SNI Mismatch

If Kong logs errors like:

  • SSL: certificate name does not match

Cause

  • The Server Name Indication (SNI) sent by Kong does not match the hostname in F5’s SSL certificate.

Solution

  1. Verify F5 SSL Certificate:
    • Ensure the certificate on F5 is issued for the hostname used by Kong.
    • Use a tool like openssl to check the F5 certificate:

openssl s_client -connect <F5_VIP>:443 -showcerts

  1. Set SNI in Kong:
    • Specify the correct SNI for the service in Kong:

curl -i -X PATCH http://<KONG_ADMIN_API&gt;:8001/services/<SERVICE_NAME_OR_ID> \

  –data “tls_verify=true” \

  –data “tls_verify_depth=2” \

  –data “sni=<F5_HOSTNAME>”

3. Mutual TLS (mTLS) Configuration

If using mTLS, errors may include:

  • SSL handshake failed
  • no client certificate presented

Cause

  • Kong is not presenting a client certificate, or F5 is not configured to validate the client certificate.

Solution

  1. Upload Client Certificate to Kong:
    • Add the client certificate and private key to Kong:

curl -i -X POST http://<KONG_ADMIN_API&gt;:8001/certificates \

  –data “cert=$(cat /path/to/client_certificate.pem)” \

  –data “key=$(cat /path/to/client_key.pem)”

  1. Associate the Certificate with the Service:
    • Attach the certificate to the service connecting to F5:

curl -i -X PATCH http://<KONG_ADMIN_API&gt;:8001/services/<SERVICE_NAME_OR_ID> \

  –data “client_certificate=<CERTIFICATE_ID>”

  1. Enable Client Certificate Validation on F5:
    • On F5, enable client certificate authentication in the SSL profile:
      • Go to Local Traffic → SSL Profiles → Edit the profile.
      • Enable Require Client Certificate.
      • Upload the CA certificate that issued the client certificate.

4. Protocol or Cipher Mismatch

Errors like:

  • SSL routines:ssl_choose_client_version:unsupported protocol
  • ssl_cipher_list failure

Cause

  • Mismatch in SSL protocols or ciphers supported by F5 and Kong.

Solution

  1. Check SSL Protocols and Ciphers on F5:
    • Ensure F5 supports the protocols (e.g., TLS 1.2/1.3) and ciphers used by Kong.
    • Modify the F5 SSL profile to include compatible protocols and ciphers.
  2. Set Cipher Suites in Kong:
    • Update Kong’s nginx_kong.conf to include compatible ciphers:

nginx

ssl_ciphers HIGH:!aNULL:!MD5;

ssl_protocols TLSv1.2 TLSv1.3;

  1. Restart Kong after the update.

5. Untrusted Self-Signed Certificate

If F5 uses a self-signed certificate, Kong cannot validate it by default.

Solution

  1. Export the self-signed certificate from F5.
  2. Add the certificate to Kong’s trusted CA store:

curl -i -X POST http://<KONG_ADMIN_API&gt;:8001/ca_certificates \

  –data “cert=$(cat /path/to/self_signed_certificate.pem)”

Best Practices

  • Use Valid Certificates:
    • Always use certificates from trusted Certificate Authorities (CAs) for production systems.
  • Enable Logging:
    • Monitor logs in Kong and F5 to troubleshoot SSL/TLS issues.
  • Regular Certificate Rotation:
    • Ensure certificates are renewed and updated before expiry.
  • Secure Configuration:
    • Use modern TLS protocols (e.g., TLS 1.2/1.3) and strong cipher suites.

By addressing these common issues, you can ensure smooth integration between F5 and Kong Gateway with robust SSL/TLS security. Let me know if you need detailed help with any specific step!

Leave a comment