spring-cloud-kubernetes: KubernetesServerList is not able to fetch endpoints for all namespaces

Hi,

during our investigations how to handle services deployed in several projects within openshift using Zuul as api gateway we ran into a major problem when trying to fetch pods/endpoints

Scenario:

  • a Zuul as api gateway using spring-cloud-starter-kubernetes-netflix running a dedicated project inside openshift
  • at least one downstream service which is deployed in an other project

When sticking to the fabric8’s DefaultKubernetesClient implementation there are two steps needed in order to perform service discovery:

1) Perform service discovery based on kubernetes services by invoking client.services() In our scenario we want to tweak this a little to discover kubernetes services with a certain label within all namespaces. For example ‘example.com/stage: testing’

This step works fine by extending the DefaultKubernetesClient:

    @Bean
    public KubernetesClient kubernetesClient() {
        DefaultKubernetesClient client = new DefaultKubernetesClient() {
            @Override
            public MixedOperation<Service, ServiceList, DoneableService, Resource<Service, DoneableService>> services() {
                ServiceOperationsImpl serviceOperations = new ServiceOperationsImpl(httpClient, getConfiguration(), null);
                serviceOperations.withLabels(labels);

                return serviceOperations;
            }

        };

        return client;
    }

2) Fetching endpoints for all discovered kubernetes services Here we’re facing the problem: The KubernetesServerList reads the configured namespace from the configuration file/environment (for ‘any namespace’ this property should be null). However, as you can see in the initWithNiwsConfig() method this property is treaded as a string:

this.namespace = clientConfig.getPropertyAsString(KubernetesConfigKey.Namespace, client.getNamespace());

Now, in the getUpdatedListOfServers() method which is invoked to return the list of available instances, a NULL is expected in case you want to fetch all namespaces:

        Endpoints endpoints = namespace != null
                ? client.endpoints().inNamespace(namespace).withName(serviceId).get()
                : client.endpoints().withName(serviceId).get();

Now, of course, an empty string is not null so the client tries to discover endpoints for namespace: “” and it fails.

Unfortunately, even if you try to execute client.endpoints().withName(serviceId).get() it returns NULL because the kubernetes API returns 404. Maybe the API has changed?

So maybe this call should be change to something like

client.endpoints().inAnyNamespace().withLabel("name", serviceId).list()

NOTE: .withLabel(“name”, ServiceId) is not the correct equivalent to .withName(serviceId) In our case we can use a alternative filter here so it’s not a problem for us. But I can see that a generic solution should be implemented here.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 3
  • Comments: 34 (19 by maintainers)

Most upvoted comments

@claitonp Hi, this is still pending discussion with the team, but that should happen soon 😃

@PatrickHuetter thanks for your feedback!

I have added a for team discussion label so we can go over this with the team and see what the best way to move forward might be. Just thinking about it from a high level perspective leads me to believe that is should not be too difficult, but we probably need to put some more thought into it.

I also think this would be a good enhancement. We also have multiple namespaces e.g. “spring-server” where services like config-server, api-gateway, spring-boot-admin or other server applications are contained. The “spring-service” namespace contains all business logic relevant services. At this time it seems that it’s not possible to configure the kubernetes discovery client to obtain services from these both namespaces. Please correct me if i’m wrong.

Hi, @spencergibb

we had planned to upstream our changes to fix the issues we’ve faced here. I’ve seen that you started to rework this project. Are you interested in our changes? Does it make sense to PR those?

Our work includes:

  • implementing the metadata in the KubernetesDiscoveryClient
  • fixing the issues with client.endpoints()
  • rework of the namespace logic in order to gain better control over the namespace configuration

AFAIK the kubernetes client will send a GET request to /api/v1/endpoints with request parameters to retrieve all the endpoints across all namespaces. If the service account being used doesn’t have access to a specific namespace then that one will be filtered out. Those are my assumptions though, this needs to be verified.

EDIT: Looking at the API documentation of v1.10 it will always return a 200 status code, so I guess it’s safe to assume that the discovery client won’t fail and will function normal.

I just now saw that #396 is related.

@PatrickHuetter re-reading the whole issue again (in light of the other issue as well), it seems like my comment would only get you half way (since getInstances would still not work).

So to summarize, what you would like to do is have things work out of the box when no namespace is set (which would mean checking all namespaces)?