kubernetes: Pods stuck in terminating state on Windows nodes

Is this a BUG REPORT or FEATURE REQUEST?:

/kind bug

What happened: Pods stuck in terminating state on Windows node after deleting it

What you expected to happen: Pods get terminated and should disappear from kubectl

How to reproduce it (as minimally and precisely as possible): Run a pod with windows container Delete it Pods are still terminating

Anything else we need to know?: Could you pls help to resolve this issue. K8s pods stuck as Terminating after getting deleted on windows node with windows container(IIS). Docker ps -a not showing this container, so it is completely deleted, but in kubectl it is marked as Terminating. For networking we using OVN. kubectl logs:

$kubectl logs windows-IIS-container1 
failed to get container status {"" ""}: rpc error: code = OutOfRange desc = EOF

kubelet logs:

Line 11214: W0323 15:19:52.550689    1592 docker_sandbox.go:340] failed to read pod IP from plugin/docker: Couldn't find network status for default/windows-IIS-container1 through plugin: invalid network status for
	Line 11258: E0323 15:21:54.711806    1592 kubelet.go:1629] Unable to mount volumes for pod "windows-IIS-container1_default(57827f1c-2ea4-11e8-b7b6-00155d00680b)": timeout expired waiting for volumes to attach/mount for pod "default"/"windows-IIS-container1". list of unattached/unmounted volumes=[default-token-mpzlg]; skipping pod
	Line 11258: E0323 15:21:54.711806    1592 kubelet.go:1629] Unable to mount volumes for pod "windows-IIS-container1_default(57827f1c-2ea4-11e8-b7b6-00155d00680b)": timeout expired waiting for volumes to attach/mount for pod "default"/"windows-IIS-container1". list of unattached/unmounted volumes=[default-token-mpzlg]; skipping pod
	Line 11259: E0323 15:21:54.711806    1592 pod_workers.go:186] Error syncing pod 57827f1c-2ea4-11e8-b7b6-00155d00680b ("windows-IIS-container1_default(57827f1c-2ea4-11e8-b7b6-00155d00680b)"), skipping: timeout expired waiting for volumes to attach/mount for pod "default"/"windows-IIS-container1". list of unattached/unmounted volumes=[default-token-mpzlg]
	

Environment:

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.2", GitCommit:"5fa2db2bd46ac79e5e00a4e6ed24191080aa463b", GitTreeState:"clean", BuildDate:"2018-01-18T21:11:08Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.4", GitCommit:"bee2d1505c4fe820744d26d41ecd3fdd4a3d6546", GitTreeState:"clean", BuildDate:"2018-03-12T16:21:35Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

• Cloud provider or hardware configuration: Hyper-V VMs

• OS (e.g. from /etc/os-release): Windows Node:

Major  Minor  Build  Revision
-----  -----  -----  --------
10     0      14393  0

Master Node: NAME=“Ubuntu” VERSION=“16.04.4 LTS (Xenial Xerus)” ID=ubuntu ID_LIKE=debian PRETTY_NAME=“Ubuntu 16.04.4 LTS” VERSION_ID=“16.04” HOME_URL=“http://www.ubuntu.com/” SUPPORT_URL=“http://help.ubuntu.com/” BUG_REPORT_URL=“http://bugs.launchpad.net/ubuntu/” VERSION_CODENAME=xenial UBUNTU_CODENAME=xenial

• Kernel (e.g. uname -a): Linux esd-kbrnts-lnx-1 4.13.0-37-generic #42~16.04.1-Ubuntu SMP Wed Mar 7 16:03:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

• Install tools: OVN 2.7 on windows OVN 2.7.2 on linux

• Others: Docker Version on Windows node: Server Version: 17.06.2-ee-6

@kubernetes/sig-windows-bugs @kubernetes/sig-scheduling