rancher: Monitoring - Grafana use wrong prometheus-operated url

What kind of request is this (question/bug/enhancement/feature request):

Bug

Steps to reproduce (least amount of steps as possible):

• Enable Cluster monitoring
• Enable Project monitoring
• Try to access Grafana

Result:

Encounter Bad Gateway (502) in Grafana dashboard

Other details that may be helpful:

Cluster monitoring exists under namespace cattle-prometheus. Project monitoring creates another namespace cattle-prometheus-p-xxxxx. Logs of Grafana shows that it tries to access prometheus-operated:9090, while it should try prometheus-operated.cattle-prometheus.

Editing the configMaps related to Grafana (grafana-project-monitoring-nginx and grafana-project-monitoring-provisionings) to replace the occurences of prometheus-operated by prometheus-operated.cattle-prometheus solved the problem.

We see in the Helm app that prometheus.project.alertManagerTarget and prometheus.sync.target have the correct host, prometheus-operated.cattle-prometheus. We may have a similar property for Grafana.

NOTE I may be able to submit a PR, but I did not found the repository of the associated Helm chart. If you can point me to it, that would be wonderful !

Environment information

• Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): v2.3.4 (was not working with v2.3.2 either)
• Installation option (single install/HA): HA