Kong – Ldap setting

October 29, 2024 techhadoop azure, cybersecurity, ldap, microsoft, security

For Kong’s Admin API to have visibility into LDAP users and roles, the following steps ensure LDAP users are recognized and mapped to roles in Kong’s RBAC system. Here’s an overview of how it works and how to set it up:

1. Enable LDAP Authentication on the Admin API

Configure Kong to authenticate users from an LDAP server by setting up the ldap-auth plugin on the Admin API. This allows the Admin API to recognize LDAP credentials and authenticate users.
This configuration is typically done in kong.conf or using environment variables when launching Kong:

export KONG_ADMIN_LISTEN=”0.0.0.0:8001″

export KONG_LDAP_HOST=”ldap-server.example.com”

export KONG_LDAP_PORT=389

export KONG_LDAP_BASE_DN=”ou=users,dc=example,dc=com”

export KONG_LDAP_BIND_DN=”cn=admin,dc=example,dc=com”

export KONG_LDAP_BIND_PASSWORD=”admin_password”

2. Configure LDAP Bindings for Users in RBAC

After LDAP is enabled, Kong must map LDAP users to RBAC roles. This can be done by associating Kong roles with the LDAP user groups or specific LDAP users through RBAC settings.
You can create roles and assign permissions to them in Kong’s RBAC configuration by using Admin API requests. For example:

# Create a custom role (if you don’t want to use kong-admin)

curl -i -X POST http://localhost:8001/rbac/roles \

–data “name=admin-role”

# Assign permissions to the role

curl -i -X POST http://localhost:8001/rbac/roles/admin-role/endpoints \

–data “workspace=default” \

–data “endpoint=/services” \

–data “actions=read,update”

3. Map LDAP Users to Roles

Once the roles are set up, map LDAP users to the created roles. You can do this by adding RBAC permissions based on LDAP username:

# Assign the LDAP user to the role

curl -i -X POST http://localhost:8001/rbac/users \

–data “username=<ldap-username>” \

–data “custom_id=<unique-ldap-id>” \

–data “roles=admin-role”

Here, <ldap-username> is the LDAP user, and <unique-ldap-id> is the identifier used in LDAP (e.g., uid=…).

4. Authenticate via LDAP User to Access Admin API

After assigning the role to the LDAP user, authenticate as the LDAP user using the Admin API. Kong will check the LDAP server for credentials and match the user to the associated RBAC role.
Once authenticated, LDAP users with RBAC roles are granted access based on their assigned permissions in Kong.

5. Verify Configuration

Test that your LDAP users can access Kong’s Admin API endpoints according to their role permissions by using curl or another HTTP client, as previously described.

ansible-playbook

October 25, 2024 techhadoop java, programming, python, security, spring-boot

---
- name: Delete multiple routes in Kong API Gateway
  hosts: localhost
  gather_facts: false
  tasks:
    - name: Get all routes from Kong
      uri:
        url: "http://localhost:8001/routes"
        method: GET
        return_content: yes
        status_code: 200
      register: kong_routes

    - name: Extract the list of route IDs to delete
      set_fact:
        route_ids: "{{ kong_routes.json.data | map(attribute='id') | list }}"

    - name: Show the route IDs to delete
      debug:
        msg: "Route ID: {{ item }}"
      loop: "{{ route_ids }}"

    - name: Delete each route by its ID
      uri:
        url: "http://localhost:8001/routes/{{ item }}"
        method: DELETE
        status_code: 204
      loop: "{{ route_ids }}"
      when: item is defined

---
- name: Delete multiple services in Kong API Gateway
  hosts: localhost
  gather_facts: false
  tasks:
    - name: Get all services from Kong
      uri:
        url: "http://localhost:8001/services"
        method: GET
        return_content: yes
        status_code: 200
      register: kong_services

    - name: Extract the list of route IDs to delete
      set_fact:
        service_ids: "{{ kong_services.json.data | map(attribute='id') | list }}"

    - name: Show the service IDs to delete
      debug:
        msg: "Service ID: {{ item }}"
      loop: "{{ service_ids }}"

    - name: Delete each service by its ID
      uri:
        url: "http://localhost:8001/services/{{ item }}"
        method: DELETE
        status_code: 204
      loop: "{{ service_ids }}"
      when: item is defined

Kong Service

October 25, 2024November 23, 2024 techhadoop docker, java, programming, python, security

---
- name: Create Services with Service Paths and Routes in Kong API Gateway
  hosts: localhost
  tasks:

    # Define a list of services, their service paths, and routes
    - name: Define a list of services, service paths, and routes
      set_fact:
        services:
          - { name: service1, service_url: http://example-service1.com:8080/service1, route_name: route1, route_path: /service1 }
          - { name: service2, service_url: http://example-service2.com:8080/service2, route_name: route2, route_path: /service2 }
          - { name: service3, service_url: http://example-service3.com:8080/service3, route_name: route3, route_path: /service3 }

    # Create a Service in Kong for each service defined, including service path
    - name: Create Service in Kong
      uri:
        url: http://localhost:8001/services
        method: POST
        body_format: json
        body:
          name: "{{ item.name }}"
          url: "{{ item.service_url }}"  # Service URL (including the service path)
        status_code: 201
      loop: "{{ services }}"
      register: service_creation

# Create a Route for each Service
    - name: Create Route for the Service
      uri:
        url: http://localhost:8001/routes
        method: POST
        body_format: json
        body:
          service:
            name: "{{ item.name }}"
          name: "{{ item.route_name }}"  # Route name
          paths:
            - "{{ item.route_path }}"  # Route path (external access path)
        status_code: 201
      loop: "{{ services }}"
      when: service_creation is succeeded

    # Optionally verify that services and routes were created
    - name: Verify Service and Route creation
      uri:
        url: http://localhost:8001/services/{{ item.name }}
        method: GET
        status_code: 200
      loop: "{{ services }}"

unable to verify the first certificate – KONG

October 24, 2024 techhadoop azure, certificates, security, ssl, technology

unable to verify the first certificate

The error “unable to verify the first certificate” typically indicates that the client (or server) cannot verify the certificate because it does not have the correct root certificate, intermediate certificate, or the certificate chain is incomplete. Here’s how you can troubleshoot and resolve this issue:

Common Causes of the Error:

Missing Root or Intermediate Certificates
- The server or client lacks the necessary CA certificates to complete the chain of trust.
Self-Signed Certificate
- If the certificate is self-signed, the server or client must explicitly trust the certificate.
Incomplete Certificate Chain
- The server might not be sending the entire certificate chain (intermediate certificates) along with the server certificate.
Incorrect Client/Server Configuration
- The client or server may not be configured to trust the CA that issued the certificate.

Steps to Fix the Issue:

1. Verify the Certificate Chain

Check whether the certificate chain is complete. You can use openssl to check the chain from the client side:

openssl s_client -connect your-kong-api:443 -showcerts

This will show the certificates presented by the server. Verify if the server provides the full chain, including the intermediate certificates.

2. Install Missing CA Certificates on the Client

If the client does not trust the CA that issued the server certificate, you need to install the root CA certificate on the client. For example, on most systems, you can install the CA certificates by adding them to the trusted certificate store.

Linux (e.g., Ubuntu/Debian): Copy the CA certificate (e.g., ca.crt) to the /usr/local/share/ca-certificates/ directory and then update the certificate store:

sudo cp ca.crt /usr/local/share/ca-certificates/

sudo update-ca-certificates

Windows: Import the root certificate into the Trusted Root Certification Authorities store via the Certificate Manager.
macOS: You can import the CA certificate into Keychain Access and mark it as trusted.

3. Verify the Server-Side Configuration

If you’re managing the server (e.g., Kong API Gateway), ensure that the server is sending the complete certificate chain. You can provide both the server certificate and any intermediate certificates in the configuration.

For example, in Kong, you can configure SSL certificates like this:

curl -i -X POST http://localhost:8001/certificates \

–form “cert=@/path/to/server.crt” \

–form “key=@/path/to/server.key” \

–form “cert_alt=@/path/to/intermediate.crt”

Ensure that the server certificate and intermediate certificates are included in the cert field.

4. Test the Connection with the Correct CA

When testing using curl, ensure you include the correct root CA or intermediate CA:

curl -v –cacert /path/to/ca.crt https://your-kong-api/your-route

This will make curl use the specified CA certificate for validation.

5. Check for Self-Signed Certificates

If you’re using self-signed certificates, you’ll need to make sure that both the client and server are explicitly configured to trust the self-signed certificate.

For example, when using curl:

curl -v –key client.key –cert client.crt –cacert ca.crt https://your-kong-api/

If the certificate is self-signed and you want to bypass certificate verification (not recommended in production), you can use:

curl -v –insecure https://your-kong-api/

6. Include the Correct Intermediate Certificates

If the server is not sending the intermediate certificate (or you forgot to add it), make sure that the intermediate certificate is included in the chain. You can concatenate the server certificate and intermediate certificate into one file:

cat server.crt intermediate.crt > full_chain.crt

Then, use the full chain certificate for your server configuration.

Summary of Steps:

Check the certificate chain using openssl s_client to see if all necessary certificates are presented.
Ensure the client trusts the root CA by installing the root or intermediate CA on the client.
Ensure the server presents the full certificate chain (server certificate + intermediate certificates).
Test the connection using the correct CA with curl or another tool.
Handle self-signed certificates by explicitly trusting them or bypassing verification in non-production environments.

By following these steps, you should be able to resolve the “unable to verify the first certificate” issue.

com.pingidentity.pf.datastore an error occured while testing the connection Error PKIX path building failed

October 23, 2024 techhadoop certificates, cybersecurity, security, ssl, technology

The error you’re encountering, PKIX path building failed, is related to SSL certificate validation in PingIdentity while attempting to establish a connection. Specifically, it indicates that the system could not verify the certificate path back to a trusted root certificate authority (CA). This is a common issue when a server certificate is either self-signed or not recognized by the trust store.

Here’s a breakdown of the error:

PKIX path building failed: This means that Java’s SSL/TLS system could not build a valid certificate chain back to a trusted root certificate authority.
SunCertPathBuilderException: unable to find valid certification path to requested target: This suggests that the certificate presented by the remote system is not trusted by the client making the request. The client’s trust store (Java keystore) does not have the necessary CA certificates to validate the server certificate.

Causes:

Self-signed certificate: If the server you’re trying to connect to uses a self-signed certificate, it won’t be trusted automatically.
Untrusted CA: The certificate is signed by a CA that’s not included in the default trust store of the Java Virtual Machine (JVM).
Missing intermediate certificates: The certificate chain might be incomplete, missing intermediate certificates between the server’s certificate and the trusted root.
Expired or revoked certificates: The server certificate could be expired or revoked, leading to validation failure.

Solutions:

1. Import the Server’s Certificate to the Java Truststore

You need to import the server’s certificate (or the intermediate CA certificates) into the Java trust store to ensure it’s recognized as a trusted certificate.

Steps:

Obtain the server certificate:

openssl s_client -connect <server-host>:<port> -showcerts

This will return the server’s SSL certificate chain. Copy the relevant certificate (in PEM format).

Save the certificate as server.crt.
Import the certificate into the Java trust store:

keytool -import -alias <alias_name> -keystore <path_to_java_home>/lib/security/cacerts -file server.crt

The default password for the trust store is usually changeit, but this can vary.

Restart your PingFederate server or application to ensure the new trust store is loaded.

2. Use a Valid SSL Certificate

If the server is using a self-signed certificate, consider replacing it with one signed by a trusted public CA (e.g., Let’s Encrypt, GlobalSign, etc.).
Ensure the entire certificate chain, including intermediate certificates, is properly configured on the server.

3. Disable SSL Validation (Not Recommended in Production)

You can temporarily disable SSL validation to allow connections without certificate verification. This is usually done in testing environments or when working with self-signed certificates.

Example: In Java, you can disable SSL certificate validation by setting a custom trust manager, but this approach is not secure and should be avoided in production:

import javax.net.ssl.*;

import java.security.SecureRandom;

import java.security.cert.X509Certificate;

TrustManager[] trustAllCerts = new TrustManager[]{

new X509TrustManager() {

public X509Certificate[] getAcceptedIssuers() {

return null;

}

public void checkClientTrusted(X509Certificate[] certs, String authType) {}

public void checkServerTrusted(X509Certificate[] certs, String authType) {}

}

};

SSLContext sc = SSLContext.getInstance(“SSL”);

sc.init(null, trustAllCerts, new SecureRandom());

HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

This solution is not recommended for production environments.

4. Verify the Server Certificate

Ensure that the certificate on the server is not expired, properly configured, and includes all intermediate certificates. You can use tools like:

SSL Labs: https://www.ssllabs.com/ssltest/ (to check certificate configuration)
OpenSSL to verify the certificate chain:

openssl s_client -connect <server-host>:<port> -showcerts

5. Update the JVM Truststore (if outdated)

If you are using an old version of Java, the default trust store may not include modern root CAs. You can update your JVM or manually update the CA certificates:

Download a fresh version of the cacerts file from the latest JVM or a trusted source.
Replace your current cacerts file (located in JAVA_HOME/lib/security) with the new one.

6. Proxy Configuration

If you’re using a proxy, make sure the proxy server has the necessary CA certificates and that PingFederate is properly configured to connect through the proxy.

Conclusion

To resolve the PKIX path building failed error, you will likely need to add the server certificate (or its CA) to your Java trust store. Ensure the server’s certificate chain is correctly configured and, if using self-signed certificates, import them into the trust store. Avoid disabling SSL validation in production environments due to security risks.

PING Identity

October 16, 2024 techhadoop certificates, java, security, ssl, technology

The error you’re encountering is related to certificate validation and occurs when the system is unable to establish a valid certificate chain to the requested target (usually an external service or API). Specifically, the error message:

SunCertPathBuilderException: unable to find valid certification path to requested target

indicates that the Java application (in this case, Ping Identity) is trying to connect to an HTTPS service, but the service’s SSL/TLS certificate is either:

Self-signed, or
Issued by a Certificate Authority (CA) that is not trusted by the Java trust store.

Here’s how you can troubleshoot and resolve this issue:

Step 1: Verify the SSL/TLS Certificate

Check the service’s certificate: Use a browser or a tool like openssl to verify the service’s certificate chain:

openssl s_client -connect <hostname>:<port> -showcerts

This will display the certificate chain used by the server. Ensure that the server certificate is valid and that intermediate and root certificates are also included.

Check if the certificate is self-signed: If the service is using a self-signed certificate or a certificate from a CA not included in the default Java trust store, you’ll need to manually trust it.

Step 2: Add the Certificate to the Java Trust Store

You’ll need to import the certificate into the Java trust store so that Java can trust it.

Export the server certificate:
- Save the certificate to a file using your browser or the openssl command.

For example:

echo | openssl s_client -connect <hostname>:<port> | sed -ne ‘/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p’ > server-cert.pem

Import the certificate into Java’s trust store: Use the keytool command to import the certificate:

sudo keytool -import -alias <alias-name> -file <path-to-cert> -keystore $JAVA_HOME/lib/security/cacerts

Replace:

<alias-name>: A unique alias for the certificate.
<path-to-cert>: The path to the certificate file you saved.

The default password for the Java trust store is usually changeit.

Verify the certificate import: You can verify if the certificate has been successfully imported by listing the contents of the trust store:

sudo keytool -list -keystore $JAVA_HOME/lib/security/cacerts

Step 3: Test the Connection Again

After importing the certificate, restart your application and test the connection again to ensure the error is resolved.

Additional Considerations:

Custom Trust Store: If your application is using a custom trust store (not the default Java trust store), ensure that the certificate is added to that trust store instead.
CA Certificates: If the certificate is from a trusted CA, ensure that your system has the correct root CA certificates in its trust store.

By importing the certificate into the Java trust store, you should be able to resolve the PKIX path building failed error and establish a successful connection.

How to verify mtls client cert via curl in Kong API gateway

October 10, 2024 techhadoop certificates, linux, security, ssl, technology

To verify mTLS (Mutual TLS) client certificates in Kong API Gateway using curl, you will need to have:

A valid client certificate and key that the server can verify.
The server’s CA certificate to trust the server’s certificate.

Here’s a step-by-step guide to verify the mTLS setup using curl:

1. Ensure mTLS is enabled in Kong

Make sure that the mTLS authentication plugin is enabled on the route, service, or globally in your Kong instance. For example, you can check if the mtls-auth plugin is applied:

curl http://localhost:8001/plugins

If it’s not enabled, you can add the mTLS plugin to a specific route or service:

curl -i -X POST http://localhost:8001/services/{service-id}/plugins \

–data “name=mtls-auth”

2. Ensure the Client Certificate is Associated with a Consumer

To match the client certificate with a consumer in Kong, you need to associate the Common Name (CN) or Subject Alternative Name (SAN) in the certificate with a consumer.

curl -i -X POST http://localhost:8001/consumers/{consumer-id}/mtls_auth \

–data “subject_name=client.example.com”

This ensures that when the client presents the certificate, Kong will map the request to the correct consumer based on the CN.

3. Verify mTLS using curl

To perform mTLS using curl, you will need:

The client certificate and client key.
The CA certificate that Kong uses to verify the client certificate.

Use the following curl command to verify mTLS:

bash

Copy code

curl -v https://{KONG_HOST}:{KONG_PORT}/{route-path} \

–cert /path/to/client-cert.pem \

–key /path/to/client-key.pem \

–cacert /path/to/ca-cert.pem

–cert /path/to/client-cert.pem: Path to the client certificate file.
–key /path/to/client-key.pem: Path to the client private key file.
–cacert /path/to/ca-cert.pem: Path to the CA certificate that Kong uses to verify the client certificate.
-v: Verbose output to see the handshake details.

4. Example Command

For example, if Kong is running on localhost, the route is /secure-service, and you have the client certificate and key:

curl -v https://localhost:8443/secure-service \

–cert /etc/ssl/client-cert.pem \

–key /etc/ssl/client-key.pem \

–cacert /etc/ssl/ca-cert.pem

5. Check the Response

If the client certificate is valid and trusted, Kong will allow the request, and you will see a 200 OK response (or the relevant service response).
If the certificate validation fails, you may see errors like:
- 400 Bad Request: Indicates issues with the certificate verification.
- 403 Forbidden: Indicates the certificate was valid but the client was not authorized for the route.

6. Common Errors and Troubleshooting

400 Bad Request: SSL certificate validation failed: This usually happens when the client certificate is not signed by the trusted CA or doesn’t match the subject name expected by Kong.
403 Forbidden: mTLS authentication failed: This happens if the certificate’s CN or SAN does not match any configured consumers in Kong.

Conclusion

Verifying mTLS in Kong API Gateway via curl involves ensuring that the client certificate and key are properly configured and Kong is set up to validate them. If configured correctly, Kong will authenticate the client using the certificate, and the request will proceed.

Hadoop ha Active/Active vs Active/Passive

October 8, 2024 techhadoop azure, cloud, disaster-recovery, security, technology

Hadoop High Availability (HA): Active/Active vs. Active/Passive

When designing a Hadoop High Availability (HA) solution, two common approaches are Active/Active and Active/Passive. These strategies help ensure data and service availability across failures and disasters. Let’s compare them in detail to help you understand their differences, benefits, challenges, and use cases.

1. Active/Active Hadoop Architecture

Overview:

Both sites are fully operational and handling workloads simultaneously.
Both clusters actively serve requests, and the load can be distributed between them.
Data is replicated between the sites, ensuring both sites are synchronized.

Key Components:

HDFS Federation: Each site has its own NameNode that manages a portion of the HDFS namespace.
YARN ResourceManager: Each site runs its own ResourceManager, coordinating job execution locally, but the jobs can be balanced between sites.
Zookeeper & JournalNodes Quorum: Spread across both sites to provide consistency and manage service coordination.
Cross-Site Replication: Hadoop’s DistCp or HDFS replication is used to replicate data across sites.
Hive/Impala Metastore: Shared between sites, ensuring consistent metadata.

Advantages:

Load Balancing: Traffic and workloads can be distributed between the two active sites, reducing pressure on a single site.
Low Recovery Time: In case of a site failure, the other site can immediately handle all workloads without downtime.
Improved Resource Utilization: Both sites are fully operational, utilizing available resources efficiently.
Fast Failover: If one site fails, the remaining site continues operating without needing to bring up services.

Challenges:

Increased Complexity: Managing two active sites involves more complex setup, including federation, data replication, and synchronization.
Data Consistency: Ensuring both sites have up-to-date data requires robust replication mechanisms and careful coordination.
Conflict Resolution: Handling conflicting updates across both sites requires careful planning and automated conflict resolution strategies.

Operational Considerations:

Synchronization of Data: Ensure real-time or near real-time data replication across both sites.
Federated HDFS: Requires splitting data across multiple namespaces with NameNodes in each site.
Network Requirements: Reliable, high-bandwidth network links are essential for cross-site replication and synchronization.
Monitoring and Automation: Continuous monitoring of job failures, resource usage, and automatic load balancing/failover processes.

Best Use Cases:

Mission-Critical Workloads: Where zero downtime and continuous availability are essential.
Geographically Distributed Sites: When there is a need for global load balancing or when sites are geographically distant but still need to function as one.
High Load Systems: Systems that need to distribute workloads across multiple data centers to balance processing power.

2. Active/Passive Hadoop Architecture