spark-on-k8s-operator: the operator is broken when using prometheus

the config map for prometheus jmx exporter is not being mounted anymore as a consequence the driver fail to start when using prometheus

commit b8c7edca0eeb372fb68089b433d559f721734711 might have broken it

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 15 (12 by maintainers)

Most upvoted comments

I found what the issue is. We recently changed the way the webhook works. It used to rely on annotations added by the submission to find things it needs to patch. The new way is the webhook actually gets the SparkApplication object for the driver/executor pod it tries to patch. However, the ConfigMap for Prometheus configuration is added at runtime into .spec.driver.configMaps and .spec.executor.configMaps. But the updated SparkApplication object is not written back to the API server, causing the webhook to miss the ConfigMap for Prometheus. I will fix it. Sorry for the trouble.

liyinan926 on Apr 19, 2019