origin: Duplicate caches in OpenShift master

Here is a list of all unique reflector caches, their type, and the location that initializes them in origin.

*api.Build from github.com/openshift/origin/pkg/build/controller/factory/factory.go:125
*api.Build from github.com/openshift/origin/pkg/build/controller/factory/factory.go:186
*api.Build from github.com/openshift/origin/pkg/build/controller/factory/factory.go:74
*api.BuildConfig from github.com/openshift/origin/pkg/build/controller/factory/factory.go:282
*api.BuildConfig from github.com/openshift/origin/pkg/build/controller/factory/factory.go:317
*api.ClusterPolicy from github.com/openshift/origin/pkg/authorization/cache/clusterpolicy.go:32
*api.ClusterPolicyBinding from github.com/openshift/origin/pkg/authorization/cache/clusterpolicybinding.go:32
*api.ConfigMap from controller/resourcequota/resource_quota_controller.go:185
*api.DeploymentConfig from github.com/openshift/origin/pkg/deploy/controller/configchange/factory.go:43
*api.DeploymentConfig from github.com/openshift/origin/pkg/deploy/controller/deploymentconfig/factory.go:43
*api.DeploymentConfig from github.com/openshift/origin/pkg/deploy/controller/imagechange/factory.go:48
*api.Group from github.com/openshift/origin/pkg/user/cache/groups.go:38
*api.ImageStream from controller/resourcequota/resource_quota_controller.go:185
*api.ImageStream from github.com/openshift/origin/pkg/build/controller/factory/factory.go:279
*api.ImageStream from github.com/openshift/origin/pkg/deploy/controller/imagechange/factory.go:37
*api.ImageStream from github.com/openshift/origin/pkg/image/controller/factory.go:40
*api.LimitRange from k8s.io/kubernetes/plugin/pkg/admission/limitranger/admission.go:156
*api.LimitRange from k8s.io/kubernetes/plugin/pkg/admission/limitranger/admission.go:156
*api.Namespace from controller/namespace/namespace_controller.go:168
*api.Namespace from controller/serviceaccount/serviceaccounts_controller.go:140
*api.Namespace from github.com/openshift/origin/pkg/project/auth/cache.go:154
*api.Namespace from github.com/openshift/origin/pkg/project/cache/cache.go:84
*api.Namespace from github.com/openshift/origin/pkg/project/controller/factory.go:37
*api.Namespace from github.com/openshift/origin/pkg/security/controller/factory.go:40
*api.Namespace from k8s.io/kubernetes/plugin/pkg/admission/namespace/lifecycle/admission.go:114
*api.Node from controller/daemon/controller.go:212
*api.Node from controller/node/nodecontroller.go:222
*api.Node from k8s.io/kubernetes/plugin/pkg/scheduler/factory/factory.go:220
*api.PersistentVolume from controller/persistentvolume/persistentvolume_claim_binder_controller.go:459
*api.PersistentVolume from controller/persistentvolume/persistentvolume_recycler_controller.go:325
*api.PersistentVolume from k8s.io/kubernetes/plugin/pkg/scheduler/factory/factory.go:224
*api.PersistentVolumeClaim from controller/persistentvolume/persistentvolume_claim_binder_controller.go:464
*api.PersistentVolumeClaim from controller/resourcequota/resource_quota_controller.go:185
*api.PersistentVolumeClaim from k8s.io/kubernetes/plugin/pkg/scheduler/factory/factory.go:225
*api.Pod from controller/daemon/controller.go:211
*api.Pod from controller/endpoint/endpoints_controller.go:130
*api.Pod from controller/gc/gc_controller.go:81
*api.Pod from controller/job/controller.go:141
*api.Pod from controller/node/nodecontroller.go:223
*api.Pod from controller/replication/replication_controller.go:210
*api.Pod from controller/resourcequota/resource_quota_controller.go:185
*api.Pod from github.com/openshift/origin/pkg/build/controller/factory/factory.go:189
*api.Pod from github.com/openshift/origin/pkg/build/controller/factory/factory.go:240
*api.Pod from github.com/openshift/origin/pkg/deploy/controller/deployerpod/factory.go:57
*api.Pod from k8s.io/kubernetes/plugin/pkg/scheduler/factory/factory.go:213
*api.Pod from k8s.io/kubernetes/plugin/pkg/scheduler/factory/factory.go:216
*api.Policy from github.com/openshift/origin/pkg/authorization/cache/policy.go:32
*api.PolicyBinding from github.com/openshift/origin/pkg/authorization/cache/policybinding.go:32
*api.ReplicationController from controller/replication/replication_controller.go:209
*api.ReplicationController from controller/resourcequota/resource_quota_controller.go:185
*api.ReplicationController from github.com/openshift/origin/pkg/deploy/controller/deployerpod/factory.go:39
*api.ReplicationController from github.com/openshift/origin/pkg/deploy/controller/deployment/factory.go:50
*api.ReplicationController from k8s.io/kubernetes/plugin/pkg/scheduler/factory/factory.go:235
*api.ResourceQuota from controller/resourcequota/resource_quota_controller.go:182
*api.ResourceQuota from controller/resourcequota/resource_quota_controller.go:182
*api.ResourceQuota from k8s.io/kubernetes/plugin/pkg/admission/resourcequota/admission.go:86
*api.ResourceQuota from k8s.io/kubernetes/plugin/pkg/admission/resourcequota/admission.go:86
*api.Secret from controller/resourcequota/resource_quota_controller.go:185
*api.Secret from controller/serviceaccount/tokens_controller.go:145
*api.Secret from github.com/openshift/origin/pkg/serviceaccounts/controllers/deleted_dockercfg_secrets.go:73
*api.Secret from github.com/openshift/origin/pkg/serviceaccounts/controllers/deleted_token_secrets.go:67
*api.Secret from k8s.io/kubernetes/plugin/pkg/admission/serviceaccount/admission.go:118
*api.SecurityContextConstraints from github.com/openshift/origin/pkg/security/admission/admission.go:51
*api.SecurityContextConstraints from github.com/openshift/origin/pkg/security/admission/admission.go:51
*api.SecurityContextConstraints from github.com/openshift/origin/pkg/security/admission/admission.go:51
*api.Service from controller/endpoint/endpoints_controller.go:129
*api.Service from controller/resourcequota/resource_quota_controller.go:185
*api.Service from github.com/openshift/origin/pkg/serviceaccounts/controllers/docker_registry_service.go:85
*api.Service from k8s.io/kubernetes/plugin/pkg/scheduler/factory/factory.go:230
*api.ServiceAccount from controller/serviceaccount/serviceaccounts_controller.go:139
*api.ServiceAccount from controller/serviceaccount/tokens_controller.go:144
*api.ServiceAccount from github.com/openshift/origin/pkg/serviceaccounts/controllers/create_dockercfg_secrets.go:83
*api.ServiceAccount from k8s.io/kubernetes/plugin/pkg/admission/serviceaccount/admission.go:102
*extensions.DaemonSet from controller/daemon/controller.go:210
*extensions.DaemonSet from controller/node/nodecontroller.go:224
*extensions.HorizontalPodAutoscaler from controller/podautoscaler/horizontal.go:121
*extensions.Job from controller/job/controller.go:140
*extensions.ReplicaSet from k8s.io/kubernetes/plugin/pkg/scheduler/factory/factory.go:240

  12 *api.Pod
   7 *api.Namespace
   5 *api.ReplicationController
   5 *api.Secret
   4 *api.ImageStream
   4 *api.ResourceQuota
   4 *api.Service
   4 *api.ServiceAccount
   3 *api.Build
   3 *api.DeploymentConfig
   3 *api.Node
   3 *api.PersistentVolume
   3 *api.PersistentVolumeClaim
   3 *api.SecurityContextConstraints
   2 *api.BuildConfig
   2 *api.LimitRange
   2 *extensions.DaemonSet
   1 *api.ClusterPolicy
   1 *api.ClusterPolicyBinding
   1 *api.ConfigMap
   1 *api.Group
   1 *api.Policy
   1 *api.PolicyBinding
   1 *extensions.HorizontalPodAutoscaler
   1 *extensions.Job
   1 *extensions.ReplicaSet

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 72 (69 by maintainers)

Most upvoted comments

I can probably get gc to use shared informers.

On Sat, Apr 29, 2017 at 10:06 AM Clayton Coleman notifications@github.com wrote:

Well, we have the 3.6 number.

126

Mostly garbage collectors, will post the breakdown. This is too much of a regression for 3.6, so marking this p0 until we are sure we can fix or tolerate.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/openshift/origin/issues/8229#issuecomment-298171007, or mute the thread https://github.com/notifications/unsubscribe-auth/AAABYlvFnXsVLGjQXIvfrY8o_eSFzjIUks5r00PHgaJpZM4H3aGO .

ncdc on Apr 29, 2017

sudo openshift start master --loglevel=4 2>&1 | grep reflector.go

Will print all of these

On Feb 10, 2017, at 4:42 PM, jay vyas notifications@github.com wrote:

@smarterclayton https://github.com/smarterclayton we’re digging in, real stupid question before we start… to reproduce this, you’re using pprof somehow? Or some other tool ?

i assume maybe on a real cluster deployment ? the api.ImageStream (10m0s) from github.com format seems to be generated by some kind of tool but not sure its the same as the https://blog.golang.org/profiling-go-programs.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openshift/origin/issues/8229#issuecomment-279073972, or mute the thread https://github.com/notifications/unsubscribe-auth/ABG_p5QJHIg4dYuan_x7RN5RBsJNmoZBks5rbNm1gaJpZM4H3aGO .

smarterclayton on Feb 11, 2017

I mostly want to get a determination that a) the fact that we have more is not a regression, and b) that the scale team is ok with us actually holding more things in memory (so memory scale performance gets worse in 1.5) and c) that we do the minimum due diligence as a team before moving it out.

Regressing total memory used would be a reason to hold a release in many cases, especially if we had promised to improve it for some use cases.

smarterclayton on Feb 2, 2017