statsd_exporter: Reject invalid metrics/mappings early instead of breaking `/metrics`

When a user submits an invalid metric (such as mapping a counter and a gauge to the same metric name), we do not log anything, but from that point on (until the exporter is restarted) /metrics will return a 500. We ought to not accept the second (then invalid) sample at all, and log a descriptive error message instead. This way users do not lose visibility into their other metrics.

Original issue:

Statsd exporter results an internal error (Showed in Prometheus) when malformed statsd lines are sent. However, this does not show any error in the log as well.

Steps to recreate:

  1. Use the statsd package ‘hot-shots’ (nodejs)
  2. Send an ‘increment’ without specifying the increment number.

import {StatsD} from 'hot-shots'; let client = new StatsD(this.initConfig); client.increment('adapter.' + adapter + '.errors'); //note the buggy invocation On Prometheus targets, exporter goes to down state. No logs will be shown.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 15 (9 by maintainers)

Most upvoted comments

I’m going to rename the issue, and add a high-level description at the top, in case anyone wants to pick this up.