alertmanager: When a group has multiple alarms,cannot receive resolved notice

What did you do? There are ten alarms in a notice group. after a time,one of them has resolved,But I didn’t receive any notice. repeat_interval later, I got a notice, it has nine alarms. if the group only have one alarm, When the alarm is resolved, I can receive the notice immediately.

What did you expect to see? When a group has multiple alarms , One of them resolved, I should be notified immediately.

What did you see instead? Under which circumstances? nothing

Environment

  • System information:

        Linux 2.6.32-573.22.1.el6.centos.plus.x86_64 x86_64
    
  • Alertmanager version: version 0.15.0-rc.1

  • Prometheus version:

    prometheus 2.1.0

  • Alertmanager configuration file:

global:
  resolve_timeout: 5m 
  smtp_smarthost: '192.168.3.101:25'
  smtp_from: 'prometheus@alter.com'
  smtp_require_tls: false
  
templates:
   - /prometheus/test/alertmanager-0.15.0-rc.1.linux-amd64/test.tmpl
route:
  group_by: ['alertname', 'job_name','severity']
  group_wait: 5s
  group_interval: 1m
  repeat_interval: 10m
  receiver:  team-X-mails
receivers:
- name: 'team-X-mails'
  email_configs:
  - to: 'xxxxxx@139.com'
    send_resolved: true
    #html: '{{ template "email.test.html" . }}'
    #headers: {Subject: '{{ template  "subject_email" . }}'}

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 2
  • Comments: 20 (8 by maintainers)

Commits related to this issue

Most upvoted comments

I’m in the need of this feature too, or at least to add it with a configuration. I will have to get back to a previous version due to this issue.

I hope to receive every resovled notification,What should I do? now, When a group has more than one alarm, some resolve, I can’t get notified

When a group has multiple alarms , One of them resolved, I should be notified immediately.

This isn’t the current expectation. group_interval only applies when new alerts are added to the group, not when existing alerts resolve.

From the docs:

# How long to wait before sending a notification about new alerts that
# are added to a group of alerts for which an initial notification has
# already been sent. (Usually ~5m or more.)
[ group_interval: <duration> | default = 5m ]