Install Prometheus Alertmanager on Ubuntu: Step-by-Step Guide

techhadoop linux, monitoring, Prometheus

Here’s a complete step-by-step guide to install and configure Prometheus Alertmanager on Ubuntu.

Step 1: Update System

sudo apt update && sudo apt upgrade -y

Step 2: Create a Dedicated User

sudo useradd --no-create-home --shell /bin/false alertmanager

Step 3: Download Alertmanager

Download the latest release from GitHub:

cd /tmp
wget https://github.com/prometheus/alertmanager/releases/latest/download/alertmanager-0.27.0.linux-amd64.tar.gz
tar -xvf alertmanager-0.27.0.linux-amd64.tar.gz
cd alertmanager-0.27.0.linux-amd64

Check https://github.com/prometheus/alertmanager/releases for the latest version and substitute accordingly.

Step 4: Install Binaries

Move the binaries to /usr/local/bin/:

sudo mv alertmanager /usr/local/bin/
sudo mv amtool /usr/local/bin/

Step 5: Create Directories

Create config and data directories and set ownership:

sudo mkdir /etc/alertmanager
sudo mkdir /var/lib/alertmanager
sudo chown alertmanager:alertmanager /etc/alertmanager
sudo chown alertmanager:alertmanager /var/lib/alertmanager

Step 6: Create the Config File

sudo nano /etc/alertmanager/alertmanager.yml

Paste a basic configuration (example with email):

global:
  smtp_smarthost: 'smtp.gmail.com:587'
  smtp_from: 'alertmanager@example.com'
  smtp_auth_username: 'your@gmail.com'
  smtp_auth_password: 'your-app-password'
route:
  group_by: ['alertname']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 1h
  receiver: 'email-alerts'
receivers:
  - name: 'email-alerts'
    email_configs:
      - to: 'you@example.com'
inhibit_rules:
  - source_match:
      severity: 'critical'
    target_match:
      severity: 'warning'
    equal: ['alertname', 'dev', 'instance']

Set proper ownership:

sudo chown alertmanager:alertmanager /etc/alertmanager/alertmanager.yml

Step 7: Create a systemd Service

Create the service file:

sudo nano /etc/systemd/system/alertmanager.service
[Unit]
Description=Prometheus Alertmanager
Wants=network-online.target
After=network-online.target
[Service]
User=alertmanager
Group=alertmanager
Type=simple
ExecStart=/usr/local/bin/alertmanager \
  --config.file=/etc/alertmanager/alertmanager.yml \
  --storage.path=/var/lib/alertmanager
[Install]
WantedBy=multi-user.target

Step 8: Start and Enable the Service

sudo systemctl daemon-reload
sudo systemctl start alertmanager
sudo systemctl enable alertmanager
sudo systemctl status alertmanager

Alertmanager will be available at http://<your-server-ip>:9093.

Step 9: Integrate with Prometheus

Add this to your prometheus.yml to wire Alertmanager in:

alerting:
  alertmanagers:
    - static_configs:
        - targets: ['localhost:9093']

Optionally, also scrape Alertmanager’s own metrics:

scrape_configs:
  - job_name: alertmanager
    static_configs:
      - targets: ['localhost:9093']

Then restart Prometheus:

sudo systemctl restart prometheus

Step 10: (Optional) Firewall Rules

If Alertmanager should only be accessible locally:

sudo ufw allow from 127.0.0.1 to any port 9093
sudo ufw deny 9093

Verify & Troubleshoot

# Check logs
sudo journalctl -u alertmanager -f
# Validate config file
amtool check-config /etc/alertmanager/alertmanager.yml
Common Receivers Beyond Email
ReceiverKey Config Field
Slackslack_configs with api_url
PagerDutypagerduty_configs with service_key
Webhookwebhook_configs with url
OpsGenieopsgenie_configs with api_key

Here’s the complete email configuration for Alertmanager:

Update /etc/alertmanager/alertmanager.yml

sudo nano /etc/alertmanager/alertmanager.yml
global:
  smtp_smarthost: 'smtp.gmail.com:587'
  smtp_from: 'alertmanager@yourdomain.com'
  smtp_auth_username: 'your@gmail.com'
  smtp_auth_password: 'your-app-password'   # Gmail App Password, NOT your login password
  smtp_require_tls: true
route:
  receiver: 'email-alerts'
  group_by: ['alertname', 'severity']
  group_wait: 30s        # Wait before sending first alert in a group
  group_interval: 5m     # Wait before sending new alerts for an existing group
  repeat_interval: 4h    # Resend if alert is still firing after this duration
  # Optional: route critical alerts separately
  routes:
    - match:
        severity: critical
      receiver: 'email-critical'
      repeat_interval: 1h
receivers:
  - name: 'email-alerts'
    email_configs:
      - to: 'your-team@example.com'
        send_resolved: true    # Also notify when alert resolves
  - name: 'email-critical'
    email_configs:
      - to: 'oncall@example.com'
        send_resolved: true
        headers:
          Subject: '[CRITICAL] {{ .GroupLabels.alertname }}'
inhibit_rules:
  - source_match:
      severity: 'critical'
    target_match:
      severity: 'warning'
    equal: ['alertname', 'instance']

Gmail Setup (Important)

Gmail blocks plain passwords — you need an App Password:

  1. Go to https://myaccount.google.com/security
  2. Enable 2-Step Verification (required)
  3. Go to App Passwords → select “Mail” → generate
  4. Use that 16-character password in smtp_auth_password

For other providers, swap out the SMTP settings:

Providersmtp_smarthost
Gmailsmtp.gmail.com:587
Outlook/Office365smtp.office365.com:587
SendGridsmtp.sendgrid.net:587
AWS SESemail-smtp.<region>.amazonaws.com:587

Apply & Test

# Validate config before restarting
amtool check-config /etc/alertmanager/alertmanager.yml
# Restart to apply
sudo systemctl restart alertmanager
# Send a test alert manually
amtool alert add alertname="TestAlert" severity="warning" \
  --alertmanager.url=http://localhost:9093
# Check it appears
amtool alert --alertmanager.url=http://localhost:9093

Check Logs if Email Fails

sudo journalctl -u alertmanager -f

Common issues to look for:

  • authentication failed → wrong App Password
  • connection refused → check SMTP host/port
  • tls: no supported versions → set smtp_require_tls: false for port 25

Once you confirm email is working, you can layer in Slack or PagerDuty alongside it in the same config.

Leave a Reply