kubernetes: kubeconfig config set-context fails with mix of writable & non-writable entries in KUBECONFIG

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug

/kind feature

What happened:

with KUBECONFIG=/home/pdbogen/.kube/config:/etc/kube/config, kubeconfig config set current-context sandbox returns error: open /etc/kube/config.lock: permission denied

What you expected to happen:

kubeconfig should lock only /home/pdbogen/.kube/config and set the indicated setting there

How to reproduce it (as minimally and precisely as possible):

$ mkdir readonly writable
$ chmod -w readonly
$ KUBECONFIG=$PWD/writable/config:$PWD/readonly/config kubectl config set current-context example
error: open /home/pdbogen/readonly/config.lock: permission denied

Anything else we need to know?:

I want to be able to provide my users a system-managed kubernetes config that describes various clusters. I would like them to be able to specify things like current-context easily, via kubectl config.

Even when ~/.kube/config exists and contains a current-context setting, kubectl config set current-context still tries to lock /etc/kube/config.

Environment:

  • Kubernetes version (use kubectl version): Client Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.5-16", GitCommit:"32ac1c9073b132b8ba18aa830f46b77dcceb0723", GitTreeState:"clean", BuildDate:"2018-06-27T17:46:30Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"} (we build our own, currently can’t use 1.11)
  • Cloud provider or hardware configuration: n/a; but, Amazon EKS
  • OS (e.g. from /etc/os-release): Ubuntu Xenial 16.04.5
  • Kernel (e.g. uname -a): 4.4.0-1063-aws
  • Install tools: na
  • Others: na

About this issue

  • Original URL
  • State: open
  • Created 6 years ago
  • Comments: 23 (9 by maintainers)

Commits related to this issue

Most upvoted comments

I had the same issue. Please make sure that $HOME/.kube/ directory has the same permission as $HOME/.kube/config.

$ sudo chown $(id -u):$(id -g) $HOME/.kube/ 
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

I got this error when running kubectl config use-context "clustercontextname2 whiting a devcontainer. if the same command is executed outside the devcontainer no error is reported. Notice that config.lock is created with 000 ownership so it’s unaccessible to kubectl. No chance to use the command within the devcontainer.

vscode@docker-desktop is the devcontainer /home/user/.kube/ is the home user and .kube dir in the host

vscode@docker-desktop:/home/vscode$ kubectl config use-context "clustercontextname2"
error: open /home/vscode/.kube/config.lock: permission denied
vscode@docker-desktop:/home/vscode$
vscode@docker-desktop:/home/vscode$ ll .kube/
total 20K
drwx------  5 vscode vscode  160 Mar 30 21:34 ./
drwxr-xr-x 81 vscode vscode 2.6K Mar 30 21:34 ../
drwxr-x---  4 vscode vscode  128 Mar 30 17:08 cache/
-rw-------  1 vscode vscode  19K Mar 30 21:31 config
----------  1 vscode vscode    0 Mar 30 21:37 config.lock
drwxr-x--- 60 vscode vscode 1.9K Mar 30 20:29 http-cache/
vscode@docker-desktop:/home/vscode$

vscode@docker-desktop:~$ ll .kube 
lrwxrwxrwx 1 vscode vscode 18 Mar 30 16:28 .kube -> /home/user/.kube/
vscode@docker-desktop:/home/vscode$ docker version
Client: Docker Engine - Community
 Version:           20.10.14
 API version:       1.41
 Go version:        go1.16.15
 Git commit:        a224086
 Built:             Thu Mar 24 01:48:13 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.6.1 (76265)
 Engine:
  Version:          20.10.13
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.15
  Git commit:       906f57f
  Built:            Thu Mar 10 14:06:05 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.5.10
  GitCommit:        2a1d4dbdb2a1030dc5b01e96fb110a9d9f150ecc
 runc:
  Version:          1.0.3
  GitCommit:        v1.0.3-0-gf46b6ba
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
  
vscode@docker-desktop:/home/vscode$ kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:58:47Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"802eff1fe87ad2dd737ebbe891f30500b88beb00", GitTreeState:"clean", BuildDate:"2021-11-15T08:35:41Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.23) and server (1.21) exceeds the supported minor version skew of +/-1
# kube config
contexts:
  - context:
      cluster: cluster1
      user: clusterUser1
    name: clustercontextname1
  - context:
      cluster: cluster2
      namespace: namespace2
      user: clusterUser2
    name: clustercontextname2
current-context: clustercontextname2

Update: Uncheck Enable VirtioFS accelerated directory sharing then no more config.lock problem

I have in fact discovered a usable workaround for this.

If the system-level config files are symlinked into a directory the user can write to, then kubectl will lock the symlinks instead of the actual config files. kubectl can’t write to these files, so some config change operations may fail; but as long as a writable config precedes the non-writable config in KUBECONFIG, it can work at least for setting the context. Here’s an example:

$ mkdir -p ~/.kube
$ ln -s /etc/kube/config ~/.kube/system
$ touch ~/.kube/config
$ # important note, the writable file `config` precedes the unwritable symlink `system`
$ export KUBECONFIG="$HOME/.kube/config:$HOME/.kube/system"
$ kubectl config set current-context example
Property "current-context" set.
$ cat .kube/config 
apiVersion: v1
clusters: []
contexts: []
current-context: example
kind: Config
preferences: {}
users: []

Still, I think that kubectl should only try to lock a file just before writing to it, which would make this workaround unnecessary. Or, perhaps, it should actively recognize when files are not writable, so not candidates for writing a config. This would be even better, because then a KUBECONFIG like $HOME/.kube/config:/etc/kube/config could automatically create the .kube/config when needed.

We just ran into the exact same scenario as above, and had the developer uncheck Enable VirtioFS accelerated directory sharing and it worked immediately. We could not explain the issue otherwise

This is ridiculous. The use-context subcommand at the VERY LEAST should NOT care if the filesystem in which the config is on is writable. All of these workarounds are ridiculous.