kubefwd: kubefwd does not work for ClusterIP without any selectors

We have a service of ClusterIP type without any selectors (backed by manual Endpoint) which is defined as so:

apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  namespace: prod
spec:
  ports:
  - name: http
    protocol: TCP
    port: 9200
    targetPort: 9200
---
kind: Endpoints
apiVersion: v1
metadata:
  name: elasticsearch
  namespace: prod
subsets:
  - addresses:
      - ip: 10.132.0.6
    ports:
      - name: http
        port: 9200
        protocol: TCP

When trying to access it thru kubefwd it fails with:

2019/01/07 18:41:40 Runtime error: an error occurred forwarding 9200 -> 9200: error forwarding port 9200 to pod d3dd8d6c2ccd2f3576689afe4868cc30a6a4394cff74e97a8ea1dbabb407da0b, uid : exit status 1: 2019/01/07 17:41:40 socat[39057] E connect(5, AF=2 127.0.0.1:9200, 16): Connection refused

Forwarding to all the other services backed by pods works ok.

Accessing the service from withing the cluster also works just fine:

root@some_other_pod_in_the_cluster:/usr/local/tomcat# curl -v elasticsearch:9200/
*   Trying 10.15.254.34...
* TCP_NODELAY set
* Connected to elasticsearch (10.15.254.34) port 9200 (#0)
> GET / HTTP/1.1
> Host: elasticsearch:9200
> User-Agent: curl/7.52.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 493
...

// The 10.15.254.34 IP is indeed the one allocated to the elasticsearch service.

I hope I provided enough info. Thanks for your time.

Best.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 15 (8 by maintainers)

Most upvoted comments

I ended up doing something like this.

I created an ExternalName service as above,

apiVersion: v1
kind: Service
metadata:
  name: mysql
  namespace: my-namespace
  labels:
    app: mysql
spec:
  externalName: rds-db.us-east-1.rds.amazonaws.com
  ports:
  - port: 3306
    protocol: TCP
    targetPort: 3306
  type: ExternalName

Then I ran a temporary deployment + service in the same namespace,

kubectl --namespace my-namespace run mysql-tunnel-$USER \
    -it --image=alpine/socat \
    --tty --rm --expose=true --port=3306 \
    tcp-listen:3306,fork,reuseaddr tcp-connect:mysql:3306

And proxied to that,

kubectl -n my-namespace port-forward svc/mysql-tunnel-$USER 3306:3306

https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/

You’ll see an example of forwarding a service. kubectl port-forward svc/myservice 8000

Same case here. In my case, I’m using an ExternalName to point to an RDS server,

2019/02/11 13:07:49 Runtime error: an error occurred forwarding 3306 -> 3306: error forwarding port 3306 to pod 12def1bc08d1df9342824e6dfb658ea7b5af6d9e95dfc733853f2ae7cea9acef, uid : exit status 1: 2019/02/11 21:07:49 socat[10037] E connect(5, AF=2 127.0.0.1:3306, 16): Connection refused

Service looks like:

apiVersion: v1
kind: Service
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  externalName: rds-db.us-east-1.rds.amazonaws.com
  ports:
  - port: 3306
    protocol: TCP
    targetPort: 3306
  type: ExternalName

@sgandon I guess there are probably quite a number of different use cases. When I am developing on our platforms we have verbose logging turned on for the microservices services (pods) I am connecting to, we also have Prometheus gathering metrics, so when I run experiments I need to know how the underlying pod is dealing with the communication.

I think a good solution to this might be a 2.x version of kubefwd with args something like this:

kubefwd svc -n namespace <- forwards services kubefwd pods -n namespace <- forwards one pod per service

Just a thought.

@willaustin kubectl port-forward svc/myservice still requires that the Service has a Pod backing it. kubefwd uses the same method.

The kubectl documentation shows all the different forwarding commands as equivalent because they are simply means of selecting Pods.

I don’t believe the Kubernetes API has any support for forwarding to a Service.

Here is the API documentation for Pod port-forwarding: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.15/#create-connect-portforward-pod-v1-core

No equivalent API for Service port-forwarding: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.15/#-strong-proxy-operations-service-v1-core-strong-

@cjimti Kubectl can now port-forward to services. I would appreciate a feature such as the one mentioned. I’m working with some pod-less services and would love to have them included.

@sgandon sounds good to me. I can tag it as an enhancement. I won’t have time to work on it till next week but I think having kubefwd svc and kubefwd pods would be a great addition and close a few of these issues. (I have quite a bit of technical debt to clean up as well)

Thanks @sgandon @lumberj and OP @goodhoko

In that case, you are essentially doing what Kubefwd does, which is port forward a Pod. Technically you could create a service that selects your mysql-tunnel-$USER Pod and Kubefwd would work. Not a great workaround and tedious if you have quite a few Pod-less services.

I would like to keep Kubefwd usable without ever having to modify the cluster to accommodate it. I personally plan to use a few Services that point to external (and internal) endpoints so I’ll need this feature soon myself.