skaffold: Port forwarding not work when pods recreated
I’m making a simple golang app and trying to use skaffold for local development withskaffold dev --port-forward.
However, it only works for the first time, then fails to open ports again when files changed and pods recreated. I even tried v0.33.0 and the latest version 2f2a6f4, neither works.
I checked issue #1594 #1815 and tried v0.29.0, and it works as I expected.
I notice that when I use v0.33.0, every time I changed files, skaffold will start a new pod then exit (delete?) the old one, but when I use v0.29.0, only the new one starts, and there is no exit log. So I guess is the old one blocks port-forward somehow since v0.29.0
Expected behavior
Recreate port forwarding correctly just like v0.29.0 did.
Actual behavior
Only successful for the first time.
Information
- Skaffold version: v0.33.0
- Operating system: MacOSX 10.14.5
- Contents of skaffold.yaml:
apiVersion: skaffold/v1beta10
kind: Config
build:
artifacts:
- image: api
context: ./api/
deploy:
kubectl:
manifests:
- ./manifests/*
profiles:
- name: minikube
activation:
- kubeContext: minikube
command: dev
Steps to reproduce the behavior
- use minikube and v0.33.0, checkout a minimal app with service and deployment only.
- run
skaffold dev --port-forward, can see port-forward works for the first time. - change the source codes, trigger the pod recreate.
- from the logs, you can see skaffold starts a new pod with a different name with the old one, which will be deleted after the new one started listening on the port.
- stop skaffold and rerun the same command
skaffold dev --port-forward, it works again.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 6
- Comments: 32 (22 by maintainers)
Hey @cmoad @demisx I found some bugs in port forwarding and just opened #2477, once it’s merged can you let me know if it fixes your issue?
@demisx thanks for the info, and glad to hear you’re seeing the issue fixed with the latest commits! this has obviously been a pain for us recently so any progress is good progress 😃 since this particular issue seems to be resolved I’m gonna close this issue, but if you are still seeing the other problems you reported here, please open them up in separate issues so we can track them 👍
@japsen I think this issue should be fixed with #2359 – the healthcheck will wait until deployments have stabilized, and then port forwarding will start, so the pods should be ready by the time port forwarding starts.
cc @tejal29
I’ve been testing with the latest bleeding version
4ba3d06today and all issues reported earlier still exist. The port forwarding seems got worse, though. I can’t startskaffold devwith all of ports forwarded properly. Something is missing each time and I have to forward manually.@priyawadhwa I too have this issue. Environment
Observed Behaviour
skaffold dev --port-forwardsuccessfully opens ports for the initial build.Steps to Reproduce Have recreated using several of the sample applications, most recently I modified the nodejs sample. I made two changes:
nodemon src/index.jsfor nodenode src/indexskaffold dev --port-forward -v infowhich successfully opened port3000locallyhttp://localhost:3000in my browser several times and it worked as expectedsrc/index.jswhich prompted a reload3001http://localhost:3001and got nothinghttp://localhost:3000again and got nothingYou can find my full console output here, let me know if I can provide anything else that might help:
@priyawadhwa On first run I can see the services getting successfully forwarded:
However, if the underlying pods change during a deployment update the service port forward becomes unresponsive.
It looks like the service is still forwarding to the pods correctly inside the minikube VM:
One more test to confirm the tunnel is dropped: