istio: opentelemetry tracing breaks sidecar injection
Is this the right place to submit this?
- This is not a security vulnerability or a crashing bug
- This is not a question about how to use Istio
Bug Description
When deploying istio with helm and the following values for tracing:
global:
proxy:
tracer: opentelemetry
meshConfig:
enableTracing: true
defaultConfig:
tracing: {}
defaultProviders:
tracing:
- "opentelemetry"
extensionProviders:
- name: "opentelemetry"
opentelemetry:
service: otel-node-collector.kube-system.svc.cluster.local
port: 4317
proxy:
enableTracing: true
results in the following meshConfig that seem to track with the required settings for opentelemetry tracing:
data:
mesh: |-
defaultConfig:
discoveryAddress: istiod.istio-system.svc:15012
tracing: {}
defaultProviders:
tracing:
- opentelemetry
enablePrometheusMerge: true
enableTracing: true
extensionProviders:
- name: opentelemetry
opentelemetry:
port: 4317
service: otel-node-collector.kube-system.svc.cluster.local
rootNamespace: istio-system
trustDomain: cluster.local
meshNetworks: 'networks: {}'
With this config in place, pods cannot be created. The logs for istiod show the follow
2024-01-09T08:40:35.239386Z error Pod injection failed: failed to run injection template: failed applying injection overlay: unmarshal patched pod: json: cannot unmarshal string into Go struct field Probe.spec.containers.readinessProbe.failureThreshold of type int32
And an event with the same text appears in the relevant replicatset or statefulset.
Removing the global.proxy.tracer setting allows pods to start, but the resulting mesh.defaultConfig (combined with a Telemetry object), no longer results in traces being sent to the collector.
Version
istio:
client version: 1.20.0
control plane version: 1.20.1
data plane version: 1.20.1 (6 proxies)
kubernetes:
Client Version: v1.27.5-dispatcher
Kustomize Version: v5.0.1
Server Version: v1.27.3
Additional Information
No response
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 18 (9 by maintainers)
the following configuration work as expected:
I don’t think we can reasonably partially fail. We could fail and either not start Istiod at all, or fail with a more explicit injection message (“failed to read config” not “<obscure parsing error”). I’d lean towards the latter.