Alertmanager is the specialized component of the Prometheus ecosystem that handles alerts.

While Prometheus is responsible for recording metrics and triggering an alert when something goes wrong (like a server running out of RAM), Prometheus itself has no idea how to send an email, slack message, or page an engineer. It hands that problem off to Alertmanager.

The Workflow: How an Alert Moves

[ Your Servers ] ──(Metrics)──► [ Prometheus ]
                                       │
                                       ▼ (Metrics hit a threshold)
                                [ Alertmanager ]
                                       │
                ┌──────────────────────┼──────────────────────┐
                ▼                      ▼                      ▼
           [ Email ]               [ Slack ]             [ PagerDuty ]

1. Prometheus constantly evaluates your data against rules you write (e.g., Is CPU usage > 90% for more than 5 minutes?). If yes, Prometheus fires the alert and shoots it over to Alertmanager.
2. Alertmanager takes that raw alert and figures out how to handle it intelligently.
3. The Receiver (Slack, Email, PagerDuty, Discord, etc.) actually delivers the message to your team.

Why do we need a separate tool just for alerts?

You might wonder why Prometheus doesn’t just send emails directly. Alertmanager exists because in a large infrastructure (like your 20+ servers), raw alerts can quickly turn into a nightmare without three critical features:

1. Grouping (Preventing “Alert Fatigue”)

Imagine a network switch dies, cutting off 15 of your Docker hosts at the exact same moment. If Prometheus sent alerts directly, you would get 15 individual emails or Slack pings within 10 seconds, blowing up your phone.

Alertmanager groups them. It sees that 15 servers failed for the same reason, bundles them into one single notification: “Alert: 15 hosts are currently unreachable.”

2. Inhibition (Muting Dependent Alerts)

If a physical server completely crashes, both the Host Down alert and the 10 Containers on this Host are Down alerts will trigger. You don’t need to be told the containers are down if you already know the whole machine is dead. Alertmanager allows you to mute (inhibit) the container alerts if the main host alert is already active.

3. Silencing

If you are doing planned maintenance on server-05 on a Tuesday morning, you can log into Alertmanager’s web interface and click Silence for that specific server for 2 hours. Prometheus will keep tracking the metrics, but Alertmanager will block any notifications from waking up your team while you work on it.

A Quick Look at How It Connects to Your Stack

Since you are running your central monitoring stack via Portainer, you would add an Alertmanager service to your existing Docker Compose file right alongside Prometheus and Grafana:

YAML

  alertmanager:
    image: prom/alertmanager:latest
    container_name: central-alertmanager
    ports:
      - "9093:9093"
    volumes:
      - /opt/alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml
    restart: unless-stopped

In your alertmanager.yml file, you define your routing. For example, telling it to route critical alerts to Slack and warning alerts to Email:

YAML

route:
  receiver: 'slack-notifications'
receivers:
- name: 'slack-notifications'
  slack_configs:
  - api_url: 'https://hooks.slack.com/services/T000/B000/XXXXXX'
    channel: '#ops-alerts'

Alertmanager vs. Grafana Alerting

Note that Grafana can also send alerts based on your dashboards. However, in professional setups using Prometheus, engineers prefer Alertmanager because it can process math across your entire fleet instantly at the database level, rather than waiting for a Grafana graph to load and trigger.