opentelemetry-js: opentelemetry/exporter-metrics-otlp-http Does not send metrics to collector (with last collector version 0.40.0)

Hello guys, We wanted to use https://github.com/GoogleChrome/web-vitals#browser-support to get some metrics from our browser instrumentation and send them through opentelemetry-js. In order to accomplish that we started to make some tests with the examples provided here.

What version of OpenTelemetry are you using?

I’m using 0.27.0 opentelemetry-js version (this repo in main branch) and otelcontribcol 0.40.0

What version of Node are you using?

v12.16.3

Please provide the code you used to setup the OpenTelemetry SDK

At the moment the exporter does not work with the examples provided. Just changed this in the example (tracer-web/metrics/index.js) const labels = { pid: 1, environment: 'staging' }; because ‘process’ is undefined

What did you do?

We just simply start up the ‘tracer-web’ examples. All the traces examples work as expected and arrive to our collector, but metrics doesn’t.

If possible, provide a recipe for reproducing the error.

Have your collector running
cd opentelemetry-js/examples/tracer-web
npm install
in metrics/index.js change: const labels = { pid: process.pid, environment: 'staging' }; to const labels = { pid: 1, environment: 'staging' };
npm run start
Go to http://localhost:8090/metrics/ and click ‘Start metrics’

What did you expect to see?

I expected to see the metrics in our collector/backed, like all the other traces examples that works correctly.

What did you see instead?

In Network tab from the browser I have a 400 Bad request response from the collector.

{"code":3,"message":"unknown field \"doubleSum\" in v1.Metric"}

Additional context

traces are exported to localhost:55681/v1/traces (default legacy endpoint)
metrics are exported to localhost:55681/v1/metrics (default legacy endpoint)
Docker compose collector file:

version: "2"
services:

  # Collector
  otel-collector:
    image: otel/opentelemetry-collector-contrib:0.40.0
    command: ["--config=/etc/otel-collector-local-config.yaml"]
    volumes:
      - ./otel-collector-local-config.yaml:/etc/otel-collector-local-config.yaml
      - /home/fernando/workspace/khoros/agent/collector-volume:/etc/collector-logging/
    environment:
      LOG_EXPORTER_LOG_LEVEL: "DEBUG"
      NEW_RELIC_API_KEY: "key123"
    ports:
      - "1888:1888"   # pprof extension
      - "13133:13133" # health_check extension
      - "4317:4317"   # OTLP gRPC receiver
      - "4318:4318"   # OTLP HTTP receiver
      - "55681:55681" # default legacy OTLP HTTP receiver
      - "55670:55679" # zpages extension
      - "8888:8888"   # Prometheus metrics exposed by the collector
      - "8889:8889"   # Prometheus exporter metrics

Collector yml config file:

receivers:
  otlp:
    protocols:
      grpc:
      http:
        cors_allowed_origins:
        - http://*
        - https://*        
exporters:
  logging:
    logLevel: $LOG_EXPORTER_LOG_LEVEL
  otlp:
    endpoint: https://otlp.nr-data.net:4317
    headers:
      "api-key": $NEW_RELIC_API_KEY
processors:
  batch:
extensions:
  health_check: {}
service:
  extensions: [health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp, logging]
    metrics:
      receivers: [otlp]
      exporters: [otlp, logging]

Hope the information is enough to reproduce the issue, if not, please reach me and I provide more details.

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 6
Comments: 33 (16 by maintainers)

Most upvoted comments

In order to get this working, the proto transformations need to be updated. The transformations are in experimental/packages/opentelemetry-exporter-metrics-otlp-http/src/transformMetrics.ts and depend on the types in packages/exporter-trace-otlp-http/src/types.ts. Because of this unfortunate coupling, the proto files, their types (hand rolled), and the transformations (also hand rolled) need to be updated for all 6 otlp exporters at the same time. If you want to get the fix out ASAP, I would recommend you do that.

Alternatively, I am working on a separate package which will handle transformations into the intermediate OTLP JSON format and serialization as protobuf in #2691. Once that package is completed, the metrics exporters will be refactored to use it (and it already has updated proto). Depending on how time-sensitive this is for you, it may be prudent to just wait for that (i would estimate a couple weeks).

+10

dyladan on Jan 10, 2022

"message":"Bad Request","name":"OTLPExporterError","data":"{\"code\":3,\"message\":\"unknown field \\\"doubleSum\\\" in v1.Metric\"}","code":"400"}

This is the error I’m receiving when using metrics exporter

  otlp:
    protocols:
      grpc:
      http:
  zipkin:

exporters:
  jaeger:
    endpoint: jaeger:14250
    tls:
      insecure: true
  logging:
  zipkin:
    endpoint: "http://zipkin:9411/api/v2/spans"

processors:
  batch:

extensions:
  health_check:
  pprof:
  zpages:

service:
  extensions: [pprof, zpages, health_check]
  pipelines:
    traces:
      receivers: [otlp, zipkin]
      exporters: [zipkin, jaeger, logging]
      processors: [batch]
    metrics:
      receivers: [ otlp ]
      exporters: [ logging ]

const metricExporter = new OTLPMetricExporter({
  url: 'http://localhost:4318/v1/metrics',
});

hejkerooo on Apr 13, 2022

@dyladan sorry for pinging you, is there any success?

morigs on Dec 21, 2021

I won’t have time to tackle this until monday so if you want to do it before then I will try to keep an eye out for notifications and merge/release the fix over the weekend. If you can wait until monday i’ll just handle it then.

dyladan on Dec 17, 2021

Yeah our proto is a little out of date. i’ll update it

dyladan on Dec 17, 2021

It’s still an issue with "@opentelemetry/exporter-metrics-otlp-http": "0.28.0" and otel/opentelemetry-collector-contrib:0.51.0. Any ideas how to fix it? Thanks

radoslavirha on May 18, 2022

Same issue here. Also tested with otelcol 0.41.0, otelcontribcol 0.40.0 and otelcontribcol 0.41.0

morigs on Dec 17, 2021

Running into this again on 0.35.1. I’m using RemixInstrumentation. Maybe that is sending the wrong shape?

lamroger on Jan 30, 2023

Hi @lamroger, thanks for reaching out. Could you please open a new bug for the problem you’re seeing?

It has gotten a bit hard to follow the thread here, and a lot has changed since this issue here was first opened.

pichlermarc on Jan 31, 2023

@mplachter you’re using @opentelemetry/exporter-metrics-otlp-grpc, right? 🤔

I looked into your issue and found that the @opentelemetry/exporter-metrics-otlp-http works, but @opentelemetry/exporter-metrics-otlp-grpc is currently broken. It seems like a PR after #2886 introduced another bug. This causes the exporter to try and open a secure connection on an insecure server.

I’m currently working on a fix.

Edit: the fix should be ready over at #3019 🙂

pichlermarc on Jun 3, 2022

@pichlermarc if you have time can you look into this?

dyladan on Jun 2, 2022

Same issue here. I thought the latests 0.28.0 was going to have a newer otlp metric proto but it does not and still will not work with anything passed 0.42.0

mplachter on May 26, 2022