garden: In-cluster building doesn't work on DigitalOcean (doks)

Bug

Current Behavior

Currently, in-cluster building doesn’t work with DigitalOcean doks clusters.

The garden-system all deploy as normal, but the cluster is unable to pull from the in-cluster registry. The proximal cause appears to be that hostPort pods (through DaemonSets) can’t be reached (connection refused), even for other services. I’ve tried all manner of things and am stuck on fixing this.

Expected behavior

For in-cluster building to work, same as on GKE, AKS etc.

Reproducible example

Try configuring a doks cluster environment in the demo-project and deploying the project. It will eventually fail with ImagePullBackOff because the cluster is unable to reach the in-cluster registry.

Workaround

Use the default local-docker build mode when deploying to doks clusters.

Suggested solution(s)

We need to reach out to DO to figure out why hostPort services refuse connections.

Your environment

Latest master.