technology

Kong error

techhadoop Uncategorized , , ,

The error:

no credentials found for given iss

in the Kong OIDC plugin means:

Kong received a token or request with an issuer (iss) claim, but it cannot find a matching configuration for that issuer.

🔍 Why This Happens

This usually occurs in one of these scenarios:

1. Missing or Incorrect config.issuer in the OIDC plugin

  • You did not set the exact issuer URL from Ping Identity in the plugin config.
  • Or you set it, but it’s spelled or formatted differently than what’s in the JWT’s iss claim.

Example:
If the token contains:

"iss": "https://auth.pingone.com/12345/as"

Then your OIDC plugin must be configured with:

config.issuer=https://auth.pingone.com/12345/as

2. Wrong Discovery Document

Kong uses the issuer to retrieve:

<issuer>/.well-known/openid-configuration

If the URL is incorrect or inaccessible, it can’t discover metadata and fails.

3. Token Issuer Doesn’t Match Plugin Config

  • You’re sending a JWT token manually (e.g., in Authorization: Bearer) and it’s not issued by the issuer you configured.
  • Kong expects the iss claim to match the config.issuer.

✅ Fix Steps

Step 1: Extract the Token’s Issuer

Run:

jwt decode <your-token>

Or paste the JWT into jwt.io and check the iss field.

Step 2: Update Plugin Config

Ensure the OIDC plugin config matches the exact iss value:

curl -X PATCH http://localhost:8001/plugins/<plugin-id> \
  --data "config.issuer=https://auth.pingone.com/12345/as"

Or if creating fresh:

curl -X POST http://localhost:8001/services/my-service/plugins \
  --data "name=openid-connect" \
  --data "config.issuer=https://auth.pingone.com/12345/as" \
  ...

🧪 Test Again

Once updated:

  1. Clear browser session or test in Postman
  2. Trigger the login flow again
  3. Ensure Kong receives a token from Ping with matching iss

Would you like help decoding the token or verifying your current plugin setup? You can paste the token header (without the signature) and I’ll check it.

Deploy AWX (Rancher )

techhadoop Uncategorized , , , ,

If you turn on (start/deploy) cattle-system pods (especially cattle-cluster-agent) and it crashes your cluster,
it means the Rancher agents are broken or misconfigured and are overloading, blocking, or breaking Kubernetes internally.

In detail, here’s why this can happen:

CauseWhat HappensWhy It Crushes the Cluster
❌ Rancher agents fail to connect and keep retryingThey flood the Kubernetes API server with reconnect attemptsAPI server gets overloaded, becomes unresponsive
❌ Wrong Rancher URL or network brokenAgents enter infinite loops trying to reach RancherNode CPU/memory gets exhausted
❌ Authentication errors (cert expired, token invalid)Agents spam auth failures on kube-apiserverAPI server becomes slow or hangs
❌ Agent version mismatchOlder agents send bad requests to newer serversAPI server rejects them violently
❌ Cluster registration issues (wrong cluster state in Rancher DB)Rancher tries to sync invalid resourcesetcd or kube-apiserver crash
❌ cattle-node-agent overloadEach node spawns bad agents, DaemonSet restarts infinitelykubelet, container runtime get overloaded

🚨 Important:

  • Rancher’s cattle-cluster-agent talks both to your Kubernetes API server and to Rancher’s API.
  • If it is broken (wrong URL, invalid token, expired cert, bad Rancher setup), it spams the cluster.
  • Symptoms you usually see:
    • kubectl get nodes hangs
    • kubectl get pods -A freezes
    • CPU of kube-apiserver jumps high
    • Cluster nodes become NotReady
    • etcd warnings appear if etcd is running locally

✅ Proof: If you kill (scale to 0) the cattle-cluster-agent, the cluster usually becomes healthy again.
Example:

kubectl scale deployment cattle-cluster-agent -n cattle-system --replicas=0

→ Cluster becomes stable.

💬 Summary in simple words:

Rancher agents are deeply tied into your Kubernetes cluster.
If they are broken, they spam your Kubernetes system like crazy, and that overloads and crashes your cluster.

🔎 Next step: To solve it properly:

  1. Collect logs: kubectl logs -n cattle-system cattle-cluster-agent-xxxx kubectl logs -n cattle-system cattle-node-agent-xxxx
  2. Fix whatever is wrong (wrong URL, expired cert, wrong Rancher token, network broken).
  3. Then redeploy cattle-system carefully.

👉 Question for you to proceed:

  • Did you recently upgrade Rancher or move Rancher server (IP change, domain change)?
  • Did you renew SSL/TLS certificates on Rancher?
  • Or is this a new cluster registration you are trying?

Depending on your answer, I will tell you exactly how to fix it.
Would you like me to guide you step-by-step? 🚀
(If yes, tell me Rancher version too — 2.6.x, 2.7.x, 2.8.x, etc.)

This is a serious architecture conflict:

Rancher’s cattle-node-agent is destabilizing your AWX deployment.

And the reason is resource exhaustion or cluster network/messaging overload.

🔥 Why does this happen?

CauseDescriptionWhy it breaks AWX
Resource exhaustionRancher’s cattle-node-agent is heavy (especially when reconnecting or retrying). It uses a lot of CPU, memory, and bandwidth.AWX Postgres and Web pods starve for CPU/memory or lose network
Node pressureWhen 2 cattle-node-agents come alive, node CPU/mem pressure increases.Kubernetes starts evicting pods (including AWX pods)
Pod preemptionIf no resource limits are properly set, kube-scheduler kills “low-priority” pods to save critical pods.Your AWX web or postgres pod might get evicted
Flapping networkingRancher’s agents may try to reach Rancher server and overload K8s networking.AWX needs stable DNS, API server access; it breaks
PersistentVolume interruptionsIf Rancher agents cause node instability, your Postgres PVCs (storage) might get stuck or remount improperly.AWX relies heavily on Postgres; database disruptions kill it

🧠 Let’s go deep technical quickly:

If you run:

kubectl get events -n awx --sort-by='.metadata.creationTimestamp'

or

kubectl describe pod <awx-pod-name> -n awx

you will likely see:

  • Evicted messages
  • NodeAffinity errors
  • OutOfcpu
  • OutOfmemory
  • Pod sandbox changed
  • Volume mount errors
  • Postgres connection refused or file corruption

🛠 How to properly fix this

✅ Step 1: Check Node Resources

Look at:

kubectl describe node <node-name>

Check for:

  • Allocatable CPU/memory
  • Current Usage
  • Conditions like MemoryPressure, DiskPressure

✅ Step 2: Set Strong Resource Requests & Limits for AWX

In your AWX YAML (awx.yaml), add hard limits like this:

yamlCopyEditspec:
  web_resource_requirements:
    requests:
      memory: 1Gi
      cpu: 500m
    limits:
      memory: 2Gi
      cpu: 1000m
  task_resource_requirements:
    requests:
      memory: 1Gi
      cpu: 500m
    limits:
      memory: 2Gi
      cpu: 1000m
  postgres_resource_requirements:
    requests:
      memory: 1Gi
      cpu: 500m
    limits:
      memory: 2Gi
      cpu: 1000m

✅ Step 3: Pin AWX pods to 1 specific node (optional)

If your nodes are unstable, you can force AWX to run only on the “good” node:

spec:
  node_selector:
    node-role.kubernetes.io/awx: "true"

Or use taints and tolerations.

✅ Step 4: Fix Rancher agents

If your Rancher agents (cattle-node-agent) are going crazy (e.g., retrying, error loops):

  • Make sure nodes can reach Rancher server stably
  • Update the registration URL if needed
  • Fix certificate errors

You can check the cattle-node-agent logs:

kubectl logs -n cattle-system cattle-node-agent-xxxxx

✅ Step 5: Stabilize PVCs

If AWX Postgres uses PVC (Persistent Volume Claim), ensure that:

  • Storage class is fast and resilient
  • Volumes aren’t remounting during node pressure

If you use Longhorn or local-path, make sure your storage is attached and healthy.

⚡ Quick Summary

SymptomRoot CauseFix
AWX web and Postgres pods crash when Rancher agent wakes upNode pressure, network overload, volume flappingSet AWX hard resource requests/limits, stabilize Rancher agent, pin critical pods to best nodes, ensure fast storage

✍️ What I Recommend for You Right Now

  1. Look at AWX pods: kubectl get pods -n awx kubectl describe pod <awx-pod> -n awx kubectl get events -n awx --sort-by='.metadata.creationTimestamp'
  2. Look at Node status: bashCopyEditkubectl describe node <node-name>
  3. Look at Rancher cattle-node-agent logs: bashCopyEditkubectl logs -n cattle-system cattle-node-agent-xxxx

Paste me (if you want) the errors and I can tell you exactly what’s killing AWX.
You’re very close to solving it. 🚀

❓ Quick Questions

create a better AWX deployment YAML that sets:

  • Node Affinity
  • Hard CPU/memory limits
  • Stable PVC
  • Tolerations (if needed)

Troubleshooting

kubectl get secret awx-admin-password -n awx -o jsonpath=”{.data.password}” | base64 –decode

Allow LDAP users to access the Kong Manager GUI in Kong Gateway

techhadoop Uncategorized , , , ,

To allow LDAP users to access the Kong Manager GUI in Kong Gateway Enterprise 3.4, you’ll need to integrate LDAP authentication via the Kong Enterprise Role-Based Access Control (RBAC) system.

Here’s how you can get it working step-by-step 👇

👤 Step 1: Configure LDAP Authentication for Kong Manager

Edit your kong.conf or pass these as environment variables if you’re using a container setup.

admin_gui_auth = ldap-auth
admin_gui_auth_conf = {
  "ldap_host": "ldap.example.com",
  "ldap_port": 389,
  "ldap_base_dn": "dc=example,dc=com",
  "ldap_attribute": "uid",
  "ldap_bind_dn": "cn=admin,dc=example,dc=com",
  "ldap_password": "adminpassword",
  "start_tls": false,
  "verify_ldap_host": false
}

✅ If you’re using LDAPS, set ldap_port = 636 and start_tls = false or configure accordingly.

Restart Kong after updating this config.

👥 Step 2: Create an RBAC User Linked to the LDAP Username

Kong still needs an RBAC user that maps to the LDAP-authenticated identity.

curl -i -X POST http://localhost:8001/rbac/users \
  --data "name=jdoe" \
  --data "user_token=jdoe-admin-token"

The name here must match the LDAP uid or whatever attribute you configured with ldap_attribute.

🔐 Step 3: Assign a Role to the RBAC User

curl -i -X POST http://localhost:8001/rbac/users/jdoe/roles \
  --data "roles=read-only"  # Or "admin", "super-admin", etc.

Available roles: read-only, admin, super-admin, or your own custom roles.

🔓 Step 4: Log into Kong Manager with LDAP User

Go to your Kong Manager GUI:

https://<KONG_MANAGER_URL>:8445

Enter:

  • Username: jdoe (LDAP uid)
  • Password: LDAP user’s actual password (Kong will bind to LDAP and verify it)

🛠️ Optional: Test LDAP Config from CLI

You can test the LDAP binding from Kong CLI:

curl -i -X POST http://localhost:8001/rbac/users \
  --data "name=testuser" \
  --data "user_token=test123"

Then try logging into Kong Manager with testuser using their LDAP password.

Kong HA

techhadoop Uncategorized , , , ,

Setting up Kong Gateway with high availability (HA) on-premise on bare metal servers involves several steps. Below is a comprehensive guide to achieve this setup:

Prerequisites

  1. Bare Metal Servers: Ensure you have multiple physical servers available.
  2. Network Configuration: Ensure all servers are on the same network and can communicate with each other.
  3. Data Store: Kong Gateway requires a shared data store like PostgreSQL or Cassandra. Ensure you have a highly available setup for your data store.
  4. Load Balancer: A hardware or software load balancer to distribute traffic across Kong Gateway nodes.

Step-by-Step Guide

1. Install PostgreSQL for the Shared Data Store

  1. Install PostgreSQL:

sudo apt-get update

sudo apt-get install -y postgresql postgresql-contrib

  1. Configure PostgreSQL for High Availability:
    • Set up replication between multiple PostgreSQL instances.
    • Ensure that the primary and standby instances are configured correctly.
  2. Create a Kong Database:

sudo -u postgres psql

CREATE DATABASE kong;

CREATE USER kong WITH PASSWORD ‘yourpassword’;

GRANT ALL PRIVILEGES ON DATABASE kong TO kong;

\q

2. Install Kong Gateway on Each Server

  1. Install Kong Gateway:

sudo apt-get update

sudo apt-get install -y apt-transport-https

curl -s https://packages.konghq.com/keys/kong.key | sudo apt-key add –

echo “deb https://packages.konghq.com/debian/ $(lsb_release -sc) main” | sudo tee -a /etc/apt/sources.list

sudo apt-get update

sudo apt-get install -y kong

  1. Configure Kong Gateway:
    • Create a kong.conf file on each server with the following configuration:

database = postgres

pg_host = <primary_postgresql_host>

pg_port = 5432

pg_user = kong

pg_password = yourpassword

pg_database = kong

  1. Start Kong Gateway:

kong migrations bootstrap

kong start

3. Configure Load Balancer

  1. Set Up a Load Balancer:
    • Configure your load balancer to distribute traffic across the Kong Gateway nodes.
    • Ensure the load balancer is set up for high availability (e.g., using a failover IP or DNS).
  2. Configure Health Checks:
    • Configure health checks on the load balancer to monitor the health of each Kong Gateway node.
    • Ensure that traffic is only sent to healthy nodes.

4. Set Up Failover Mechanism

  1. Database Failover:
    • Ensure your PostgreSQL setup has a failover mechanism in place (e.g., using Patroni or pgpool-II).
  2. Kong Gateway Failover:
    • Ensure that the load balancer can detect when a Kong Gateway node is down and redirect traffic to other nodes.

5. Implement Monitoring and Alerts

  1. Set Up Monitoring:
    • Use tools like Prometheus and Grafana to monitor the health and performance of your Kong Gateway nodes and PostgreSQL database.
  2. Set Up Alerts:
    • Configure alerts to notify you of any issues with the Kong Gateway nodes or the PostgreSQL database.

Example Configuration Files

PostgreSQL Configuration (pg_hba.conf):

# TYPE  DATABASE        USER            ADDRESS                 METHOD

host    kong            kong            192.168.1.0/24          md5

Kong Gateway Configuration (kong.conf):

database = postgres

pg_host = 192.168.1.10

pg_port = 5432

pg_user = kong

pg_password = yourpassword

pg_database = kong

Summary

By following these steps, you can set up a highly available Kong Gateway on bare metal servers. This setup ensures that your API gateway remains reliable and performs well under various conditions. Make sure to thoroughly test your setup to ensure that failover and load balancing work as expected.

LEAPP upgrade – multiple kernel issue

techhadoop Uncategorized , , , ,

Newest installed kernel not in use

If the newest installed kernel is not in use even after rebooting, it typically means your system is booting into an older kernel version. This happens due to the default GRUB settings or other configuration issues.

Here’s how you can resolve the issue step-by-step:

1. Verify the Currently Running Kernel

  • Check the kernel currently in use:

uname -r

  • Example output:

CopyEdit

3.10.0-1127.el7.x86_64

  • Compare this with the newest installed kernel version (see step 2).

2. List All Installed Kernels

  • List all installed kernel packages:

rpm -qa | grep kernel | grep -v firmware

  • Example output:

kernel-3.10.0-1127.el7.x86_64

kernel-3.10.0-1160.el7.x86_64

  • Identify the newest kernel (e.g., 3.10.0-1160.el7.x86_64).

3. Check GRUB Default Boot Entry

  • Display the current GRUB default boot entry:

grub2-editenv list

  • You should see something like this:

saved_entry=0

  • If the saved_entry is pointing to an older kernel, the system is booting into it by default.

4. Update GRUB to Use the Newest Kernel

Option A: Use the First Kernel (Default GRUB Behavior)

  • Edit the GRUB configuration:

sudo vi /etc/default/grub

  • Set the GRUB_DEFAULT value to 0 (the first kernel in the GRUB list):

GRUB_DEFAULT=0

  • Save and exit the file.
  • Regenerate the GRUB configuration:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Option B: Manually Specify the Newest Kernel

  • Run this command to view all available kernel entries:

awk -F\’ ‘$1==”menuentry ” {print i++ ” : ” $2}’ /boot/grub2/grub.cfg

  • Example output:

0 : CentOS Linux (3.10.0-1160.el7.x86_64)

1 : CentOS Linux (3.10.0-1127.el7.x86_64)

2 : CentOS Linux (0-rescue-…)

  • Set GRUB_DEFAULT to the corresponding entry for the newest kernel (e.g., 0):

sudo grub2-set-default 0

  • Verify the setting:

grub2-editenv list

5. Reboot the System

  • Reboot your system to load the correct kernel:

sudo reboot

  • After reboot, confirm that the new kernel is in use:

uname -r

6. Remove Old Kernels (Optional)

  • To prevent confusion in the future, you can remove unused older kernels:

sudo package-cleanup –oldkernels –count=1

  • This retains only the most recent kernel.

7. Troubleshooting

  • Manually Select the Kernel: If the system still boots into the wrong kernel, you can manually select the desired kernel during the GRUB menu at boot time. To enable the GRUB menu:
    1. Edit /etc/default/grub:

sudo vi /etc/default/grub

  • Set GRUB_TIMEOUT to a non-zero value (e.g., 5):

GRUB_TIMEOUT=5

  • Save and regenerate the GRUB configuration:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

  • Kernel Missing After Update: Ensure the new kernel is properly installed. Reinstall it if necessary:

sudo yum install kernel

Ansible playbook to run a shell script on your remote hosts:

techhadoop Uncategorized , , , , 

#!/bin/bash
# example_script.sh

echo "Hello, this is a sample script."
echo "Current date and time: $(date)"

ansible 

---
- name: Run Shell Script on Remote Hosts
  hosts: all
  become: yes
  tasks:
    - name: Copy Shell Script to Remote Hosts
      copy:
        src: /path/to/example_script.sh
        dest: /tmp/example_script.sh
        mode: '0755'

    - name: Run Shell Script
      command: /tmp/example_script.sh

    - name: Remove Shell Script from Remote Hosts
      file:
        path: /tmp/example_script.sh
        state: absent

Explanation:
Copy Shell Script: The copy module is used to transfer the shell script from the local machine to the remote hosts. The mode parameter ensures that the script is executable.

Run Shell Script: The command module is used to execute the shell script on the remote hosts.

Remove Shell Script: The file module is used to delete the shell script from the remote hosts after execution to clean up.

Running the Playbook:
To run the playbook, use the following command:

initramfs

techhadoop Uncategorized , , , ,

What is initramfs ?

initramfs stands for initial RAM filesystem. It plays a crucial role in the Linux boot process by providing a temporary root filesystem that is loaded into memory. This temporary root filesystem contains the necessary drivers, tools, and scripts needed to mount the real root filesystem and continue the boot process.

In simpler terms, it acts as a bridge between the bootloader and the main operating system, ensuring that the system has everything it needs to boot successfully.

Key Concepts of initramfs:

FeatureDescription
Temporary FilesystemIt’s loaded into memory as a temporary root filesystem.
Kernel ModulesContains drivers (kernel modules) required to access disks, filesystems, and other hardware.
ScriptsContains initialization scripts to prepare the system for booting the real root filesystem.
Critical FilesIncludes essential tools like mount, udev, bash, and libraries.

Key Functions of initramfs:

  1. Kernel Initialization: During the boot process, the Linux kernel loads the initramfs into memory.
  2. Loading Drivers: initramfs includes essential drivers needed to access hardware components, such as storage devices and filesystems.
  3. Mounting Root Filesystem: The primary function of initramfs is to mount the real root filesystem from a storage device (e.g., hard drive, SSD).
  4. Transitioning to Real Root: Once the real root filesystem is mounted, the initramfs transitions control to the system’s main init process, allowing the boot process to continue.

How initramfs Works:

  1. Bootloader Stage: The bootloader (e.g., GRUB) loads the Linux kernel and initramfs into memory.
  2. Kernel Stage: The kernel initializes and mounts the initramfs as the root filesystem.
  3. Init Stage: The init script or program within initramfs runs, performing tasks such as loading additional drivers, mounting filesystems, and locating the real root filesystem.
  4. Switch Root: The initramfs mounts the real root filesystem and switches control to it, allowing the system to boot normally.

Customizing initramfs:

You can customize the initramfs by including specific drivers, tools, and scripts. This is useful for scenarios where the default initramfs does not include the necessary components for your system.

Tools for Managing initramfs:

  • mkinitramfs: A tool to create initramfs images.
  • update-initramfs: A tool to update existing initramfs images.

Difference Between initramfs and initrd

Featureinitramfsinitrd
Formatcpio archive (compressed)Disk image (block device)
MountingExtracted directly into RAM as a rootfsMounted as a loop device
FlexibilityMore flexible and fasterLess flexible, older technology

Location of initramfs

On most Linux distributions, the initramfs file is located in the /boot directory:

ls /boot/initramfs-*.img

How to Rebuild initramfs

If you’ve made changes to the kernel, /etc/fstab, or storage configuration (e.g., LUKS, LVM), you may need to rebuild the initramfs.

Rebuild initramfs on RHEL/CentOS:

sudo dracut -f

Rebuild initramfs on Ubuntu/Debian:

sudo update-initramfs -u

Common Issues Related to initramfs

IssueCauseSolution
Dropped into initramfs shellKernel can’t find the root filesystemCheck /etc/fstab, rebuild initramfs, or fix missing drivers.
Boot failure after kernel updateMissing or corrupt initramfsRebuild initramfs.
Filesystem not mountingIncorrect or missing drivers in initramfsEnsure necessary drivers are included and rebuild.

By understanding how initramfs works, you can better appreciate its role in the Linux boot process and customize it to suit your needs.

Upgrade RHEL 7 to RHEL 8 – ansible playbook

techhadoop Uncategorized , , , , 

---
- name: Upgrade RHEL 7 to RHEL 8
  hosts: all
  become: true
  vars:
    leapp_repo: "rhel-7-server-extras-rpms"

  tasks:
    - name: Ensure system is up to date
      yum:
        name: '*'
        state: latest

    - name: Install the Leapp utility
      yum:
        name: leapp
        state: present
        enablerepo: "{{ leapp_repo }}"

    - name: Run the pre-upgrade check with Leapp
      shell: leapp preupgrade
      register: leapp_preupgrade
      ignore_errors: true

    - name: Check for pre-upgrade issues
      debug:
        msg: "Leapp pre-upgrade check output: {{ leapp_preupgrade.stdout }}"

    - name: Fix any issues identified by Leapp (manual step)
      pause:
        prompt: "Please review the pre-upgrade report at /var/log/leapp/leapp-report.txt and fix any blocking issues. Press Enter to continue."

    - name: Proceed with the upgrade
      shell: leapp upgrade
      ignore_errors: true

    - name: Reboot the server to complete the upgrade
      reboot:
        reboot_timeout: 1200

    - name: Verify the OS version after reboot
      shell: cat /etc/redhat-release
      register: os_version

    - name: Display the new OS version
      debug:
        msg: "The server has been successfully upgraded to: {{ os_version.stdout }}"

How to use LUKS data disk encryption in MapR

techhadoop Uncategorized , , , ,

How to use LUKS data disk encryption in MapR

MapR (now part of HPE Ezmeral) supports encryption at various levels, but using LUKS (Linux Unified Key Setup) encryption for data disks is a system-level operation that must be done outside of MapR’s native encryption features. Here’s a step-by-step guide to set up LUKS disk encryption on a MapR node and ensure MapR can access the encrypted disk after unlocking it.

Steps to Set Up LUKS Disk Encryption for MapR Data Disks

1. Identify the Disk to Encrypt

Find the disk you want to encrypt using the lsblk or fdisk command:

bash

lsblk

fdisk -l

For example, if the disk to be encrypted is /dev/sdb, use that in the following steps.

2. Install the Necessary Packages

Ensure you have the required tools to set up LUKS encryption:

sudo apt-get install cryptsetup  # For Ubuntu/Debian

sudo yum install cryptsetup      # For CentOS/RHEL

3. Set Up LUKS Encryption on the Disk

Run the following command to initialize the disk with LUKS encryption:

sudo cryptsetup luksFormat /dev/sdb

You’ll be prompted to confirm the operation and set a passphrase.

⚠️ Warning: This will erase all existing data on the disk.

4. Open and Map the Encrypted Disk

Unlock the encrypted disk and map it to a device:

sudo cryptsetup open /dev/sdb mapr_data_disk

You can verify that the encrypted device is available:

lsblk

5. Format the Encrypted Disk

Format the newly mapped device with a filesystem that MapR supports (typically ext4 or xfs):

sudo mkfs.ext4 /dev/mapper/mapr_data_disk

6. Mount the Encrypted Disk

Create a mount point and mount the encrypted disk:

sudo mkdir -p /opt/mapr/data

sudo mount /dev/mapper/mapr_data_disk /opt/mapr/data

7. Make the Mount Persistent

Edit the /etc/crypttab file to automatically unlock the disk at boot:

echo “mapr_data_disk /dev/sdb none luks” | sudo tee -a /etc/crypttab

Then, add an entry to /etc/fstab to mount the disk automatically after it is unlocked:

echo “/dev/mapper/mapr_data_disk /opt/mapr/data ext4 defaults 0 0” | sudo tee -a /etc/fstab

8. Ensure MapR Can Access the Disk

Make sure the MapR user has the necessary permissions to access the encrypted disk:

sudo chown -R mapr:mapr /opt/mapr/data

9. Test the Setup

Reboot the system to ensure the encrypted disk is unlocked and mounted correctly:

sudo reboot

After the system reboots, verify that the disk is unlocked and mounted:

lsblk

df -h

10. Verify MapR Storage Pools

After the encrypted disk is mounted, add it to the MapR storage pool:

maprcli disk add -server <server_name> -disks /dev/mapper/mapr_data_disk

Additional Considerations

  • Passphrase Management: Consider integrating with a key management system (KMS) to avoid manual passphrase entry.
  • Performance Impact: Encryption may introduce some performance overhead, so test accordingly.
  • Backup Configuration Files: Ensure you back up /etc/crypttab and /etc/fstab for disaster recovery.

LUKS – disks encrypt

techhadoop Uncategorized , , , , 

#!/bin/bash

# Variables
DISKS=("/dev/sdb" "/dev/sdc") # List of disks to encrypt
KEYFILE="/etc/luks/keyfile"   # Keyfile path
MOUNT_POINTS=("/mnt/disk1" "/mnt/disk2") # Corresponding mount points

# Check for root privileges
if [ "$(id -u)" -ne 0 ]; then
    echo "This script must be run as root. Exiting."
    exit 1
fi

# Create the keyfile if it doesn't exist
if [ ! -f "$KEYFILE" ]; then
    echo "Creating LUKS keyfile..."
    mkdir -p "$(dirname "$KEYFILE")"
    dd if=/dev/urandom of="$KEYFILE" bs=4096 count=1
    chmod 600 "$KEYFILE"
fi

# Function to encrypt and set up a disk
encrypt_disk() {
    local DISK=$1
    local MAPPER_NAME=$2
    local MOUNT_POINT=$3

    echo "Processing $DISK..."
    
    # Check if the disk is already encrypted
    if cryptsetup isLuks "$DISK"; then
        echo "$DISK is already encrypted. Skipping."
        return
    fi

    # Format the disk with LUKS encryption
    echo "Encrypting $DISK..."
    cryptsetup luksFormat "$DISK" "$KEYFILE"
    if [ $? -ne 0 ]; then
        echo "Failed to encrypt $DISK. Exiting."
        exit 1
    fi

    # Open the encrypted disk
    echo "Opening $DISK..."
    cryptsetup luksOpen "$DISK" "$MAPPER_NAME" --key-file "$KEYFILE"

    # Create a filesystem on the encrypted disk
    echo "Creating filesystem on /dev/mapper/$MAPPER_NAME..."
    mkfs.ext4 "/dev/mapper/$MAPPER_NAME"

    # Create the mount point if it doesn't exist
    mkdir -p "$MOUNT_POINT"

    # Add entry to /etc/fstab for automatic mounting
    echo "Adding $DISK to /etc/fstab..."
    UUID=$(blkid -s UUID -o value "/dev/mapper/$MAPPER_NAME")
    echo "UUID=$UUID $MOUNT_POINT ext4 defaults 0 2" >> /etc/fstab

    # Mount the disk
    echo "Mounting $MOUNT_POINT..."
    mount "$MOUNT_POINT"
}

# Loop through disks and encrypt each one
for i in "${!DISKS[@]}"; do
    DISK="${DISKS[$i]}"
    MAPPER_NAME="luks_disk_$i"
    MOUNT_POINT="${MOUNT_POINTS[$i]}"

    encrypt_disk "$DISK" "$MAPPER_NAME" "$MOUNT_POINT"
done

echo "All disks have been encrypted and mounted."