grafana-operator: [Bug] 'deployment not ready' is reported and hang 'Phase: reconciling' status for grafana instance using v4.4.1 operator

Describe the bug ‘deployment not ready’ is reported in grafana instance and event using v4.4.1 operator, after operator is installed and grafana cr instance is created.

Version v4.4.1

To Reproduce Not a full reproduction but this is how I found it:

Installed operator v4.4.1 to a single namespace grafana in OCP operator hub.
Created CR for an instance
check instance, found Warning ProcessingError 105m (x4188 over 14h) GrafanaDashboard deployment not ready
in OCP console web ui, go to Networking > Route and click grafana-route, grafana UI loading failed.

Expected behavior No error found and all cr up.

Runtime (please complete the following information):

OS: Linux
Grafana Operator Version v4.4.1
Environment: OCP 4.10.3
Deployment type: Installed operator v4.4.1 to a single namespace grafana in OCP operator hub.

Additional context

$ oc get pod
NAME                                                   READY   STATUS    RESTARTS   AGE
grafana-deployment-6c6d7cc447-dx6kh                    1/1     Running   0          104m
grafana-operator-controller-manager-6444c498b5-dgzrx   2/2     Running   0          14h

$ oc get grafanas.integreatly.org
NAME              AGE
example-grafana   14h

$ oc describe grafanas.integreatly.org example-grafana
...
Status:                                                                                                                                                                                                                                    
   Message:                success                                                                                                                                                                                                          
   Phase:                  reconciling                                                                                                                                                                                                      
   Previous Service Name:  grafana-service
Events:
  Type     Reason           Age                    From              Message
  ----     ------           ----                   ----              -------
  Warning  ProcessingError  105m (x4188 over 14h)  GrafanaDashboard  deployment not ready

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 2
Comments: 25

Most upvoted comments

We are seeing this on Grafana Operator v4.4.1 in OpenShift.

Seems like the API Schema for GrafanaDashboard has the “labels” object, while the GrafanaDataSource and GrafanaNotificationChannel API Schema does not on integreatly.org | v1alpha1

A notificationchannel deployment with label app=grafana would result in instant reconcilling of object.

Redeploying the deployment without the label app=grafana would be successful.

We need the app=grafana was that is matchlabel for our operator to find the monitoring object for multi-namespaces.

I don’t see how this cannot be resolvable if we simply add the label object into all the Grafana Objects API Schema.

gr-ahh on Oct 19, 2022

My bad. Did not noticed that setting dashboardLabelSelector is mandatory. After that dashboards and notifications are loaded successfully.

marcinsztalmi on Aug 19, 2022