grafana-operator: [Bug] Can't create Dashboards

Describe the bug I can’t create dashboards after updating to Operator v3.10.0.

The errors are:

{"level":"error","ts":1620139043.2435157,"logger":"controller_grafanadashboard","msg":"failed to get or create namespace folder Dev for dashboard ","error":"Get \"http://admin:***@grafana-service:3000/api/folders\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/hstefans/go/pkg/mod/github.com/go-logr/zapr@v0.1.1/zapr.go:128\ngithub.com/integr8ly/grafana-operator/v3/pkg/controller/grafanadashboard.(*ReconcileGrafanaDashboard).reconcileDashboards\n\tgrafana-operator/pkg/controller/grafanadashboard/dashboard_controller.go:253\ngithub.com/integr8ly/grafana-operator/v3/pkg/controller/grafanadashboard.(*ReconcileGrafanaDashboard).Reconcile\n\tgrafana-operator/pkg/controller/grafanadashboard/dashboard_controller.go:136\ngithub.com/integr8ly/grafana-operator/v3/pkg/controller/grafanadashboard.add.func1\n\tgrafana-operator/pkg/controller/grafanadashboard/dashboard_controller.go:86\ngithub.com/integr8ly/grafana-operator/v3/pkg/controller/grafanadashboard.add.func2\n\tgrafana-operator/pkg/controller/grafanadashboard/dashboard_controller.go:92"}
{"level":"error","ts":1620139043.2436557,"logger":"controller_grafanadashboard","msg":"error updating dashboard","error":"Get \"http://admin:***@grafana-service:3000/api/folders\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/hstefans/go/pkg/mod/github.com/go-logr/zapr@v0.1.1/zapr.go:128\ngithub.com/integr8ly/grafana-operator/v3/pkg/controller/grafanadashboard.(*ReconcileGrafanaDashboard).manageError\n\tgrafana-operator/pkg/controller/grafanadashboard/dashboard_controller.go:366\ngithub.com/integr8ly/grafana-operator/v3/pkg/controller/grafanadashboard.(*ReconcileGrafanaDashboard).reconcileDashboards\n\tgrafana-operator/pkg/controller/grafanadashboard/dashboard_controller.go:254\ngithub.com/integr8ly/grafana-operator/v3/pkg/controller/grafanadashboard.(*ReconcileGrafanaDashboard).Reconcile\n\tgrafana-operator/pkg/controller/grafanadashboard/dashboard_controller.go:136\ngithub.com/integr8ly/grafana-operator/v3/pkg/controller/grafanadashboard.add.func1\n\tgrafana-operator/pkg/controller/grafanadashboard/dashboard_controller.go:86\ngithub.com/integr8ly/grafana-operator/v3/pkg/controller/grafanadashboard.add.func2\n\tgrafana-operator/pkg/controller/grafanadashboard/dashboard_controller.go:92"}

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 18

Most upvoted comments

Finally, I found an issue, it was Security Group for Openshift worker nodes. Port 3000 should be opened in case of hostNetwork: true usage and I don’t know why it worked before with Operator v3.9.0 (probably because of changes introduced for preferService in v3.10.X). Thank you for your help.

@alrf there is a change incoming where we fix the service DNS name: #438

Once that’s landed, i’ll reach out and we can give it another try.