prometheus: promtool check config complains about missing region in EC2 SD config

Use case. Why is this important?

I want to validate my prometheus configuration file

Bug Report

What did you do?

I run promtool check config against my prometheus.yml and got an error saying:

Checking packer/files/prometheus/config/prometheus.yml
  FAILED: parsing YAML file packer/files/prometheus/config/prometheus.yml: EC2 SD configuration requires a region

What did you expect to see?

I expected to see no errors as not having a region in the EC2 SD configuration is supported. Per the documentation: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#ec2_sd_config

# The AWS region. If blank, the region from the instance metadata is used.

Environment

  • System information:

    Linux 4.15.0-65-generic x86_64

  • Prometheus version:

prometheus, version 2.12.0 (branch: HEAD, revision: 43acd0e2e93f9f70c49b2267efa0124f1e759e86)
  build user:       root@7a9dbdbe0cc7
  build date:       20190818-13:53:16
  go version:       go1.12.8
  • Prometheus configuration file:
global:
  scrape_interval: 10s

rule_files:
  - 'alerts/*.yml'

alerting:
  alertmanagers:
  - static_configs:
    - targets:
      - localhost:9093

scrape_configs:
  - job_name: 'prometheus'
    ec2_sd_configs:
      - port: 9090
    relabel_configs:
      - source_labels: [__meta_ec2_tag_Name]
        regex: prometheus
        action: keep
      - source_labels: [__meta_ec2_tag_Name]
        target_label: instance
      - source_labels: [__meta_ec2_tag_Environment]
        target_label: environment

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 1
  • Comments: 18 (9 by maintainers)

Most upvoted comments

Is it possible to add a workaround such as passing the region via a CLI flag? For my use case I want to validate the configuration before building our Prometheus AMI via pre-commit hooks.

Right now I’m able to validate my Alertmanager configuration and Prometheus alert rules so this would be a very nice to have for our needs.

The reason is that the code deserializing the YAML data queries the EC2 metadata service to get the region thus it will fail if promtool isn’t executed from an EC2 instance. One could argue that it shouldn’t be done at this stage.

https://github.com/prometheus/prometheus/blob/53ea6d63901b96a13bb79ff598c96af4a979adce/discovery/ec2/ec2.go#L91-L102