opentelemetry-java: Unable to reset/reinitialise OpenTelemetrySdk
Describe the bug I have a class to setup and stop opentelemetry with those methods given as follows
fun setupTelemetry(exporter: SpanExporter) {
OpenTelemetrySdk
.getGlobalTracerManagement()
.addSpanProcessor(SimpleSpanProcessor.builder(exporter).build())
OpenTelemetry.setGlobalPropagators(
DefaultContextPropagators.builder().addTextMapPropagator(HttpTraceContext.getInstance()).build()
)
}
fun stop() {
OpenTelemetrySdk.getGlobalTracerManagement().shutdown()
}
Since this code is used in AWS Lambda, it might happen that setupTelemetry is called after stop has been called. This is something we can’t change, since this is fixed behaviour set by AWS. Once shutdown has been called on the tracer management, it is propagated further through TracerSdkProvider to TracerSharedState. The latter sets a flag internally and at this point every call to spanBuilder delegates to Tracer.getDefault().spanBuilder, which creates a no-op span. An attempt to create a new TracerProvider fails, since the sdk only allows to use instances of ObfuscatedTracerProvider as tracer providers. However, those can’t be instantiated from outside of the library.
On the other side, shutdown on the tracer management is responsible for killing connections established by the span exporters to talk to backend systems, e.g. Jaeger. Beside that fact, that one doesn’t want to manager those exporters manually, since this is done by the sdk, those can’t be removed from TracerSharedState::registeredSpanProcessors
, which potentially impacts the performance in the case described above.
Steps to reproduce Call
setupTelemetry(exporter)
stop()
setupTelemetry(exporter)
From now on, only no-op spans are produced.
What did you expect to see? There should be a possibility to initialise the sdk after is has been shut down. Thus spans are created and exported.
What did you see instead?
TracerSdk::spanBuilder
uses Tracer.getDefault()
What version and what artifacts are you using?
Artifacts: ,opentelemtry-api
, opentelemetry-sdk
, exporters Zipkin, Jaeger
Version: v0.10.0
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 30 (17 by maintainers)
Hi @jkwatson,
I’m very busy these days. I’ll have a look and give you feedback no later than next Monday.
Hi @midu-vtg
In 0.10 we have
OpenTelemtry.set
andOpenTelemetrySdk.builder
- does this allow you to reinitialize the SDK?Also if it’s possible, can you describe how this code is wired up? We generally expect spans to need to be flushed every request using
forceFlush
since it’s unknown when the lambda function will be frozen, would that remove the need to call shutdown? I haven’t heard of a lambda function callback for clean shutdown yet but I may be missing it, would be great to hear what you’re using.