containerd: [windows] Setting containerd root to a VHD mounted drive does not work
Description
I’m pretty stumped as to why this would be an issue. I’m trying to store the containerd root and state on an NTFS volume backed by a VHDX, and the logs are emitting:
[03/05/2023 18:40:44] time="2023-03-05T10:40:44.197531800-08:00" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:win-test-nano-2j2nc,Uid:77b93fe0-f0c4-432e-973c-1597c2bd8cf9,Namespace:default,Attempt:0,} failed, error" error="failed to get sandbox image \"registry.k8s.io/pause:3.6\": failed to pull image \"registry.k8s.io/pause:3.6\": failed to pull and unpack image \"registry.k8s.io/pause:3.6\": failed to extract layer sha256:f2032cdb21dfb9f3d6fdd3797ea992ec05415e5a6a57f9eb36b6646496db8d09: Failed to safefile.OpenRelative (name: Files, root: \\\\?\\D:\\containerd\\root\\io.containerd.snapshotter.v1.windows\\snapshots\\203) failed in Win32: open \\\\?\\D:\\containerd\\root\\io.containerd.snapshotter.v1.windows\\snapshots\\203\\Files: Access is denied. (0x1f) Files: unknown"
I’ve granted “Everyone” full control on the disk and the containerd folders to try and fix it, but it still happens. I also tried patching out the “winapi.OBJ_DONT_REPARSE” requirement from safeopen.OpenRelative, but that didn’t make a difference either.
Steps to reproduce the issue
- Create a dynamically sized VHD. You can do this from Disk Management or Hyper-V.
- Attach the disk to the computer and initialize it. Mount it to the D: drive. It should look something like this when you’re done:
- In your containerd config, set:
root = "D:\\containerd\\root"
state = "D:\\containerd\\state"
- Schedule any kind of Windows container (including host processes) and watch containerd fail to extract the images to the disk.
Describe the results you received and expected
The error log is included above. I’d expect this to work since a volume on a mounted VHD just appears as any other kind of drive. I’d be curious to know whether placing the containerd root on anything other than C: works, but I don’t have a computer with two physical drives that I can easily test that on.
I also tried mounting the VHDX at a path inside C:\ and having the root point there, but that got me the same result. All-in-all very weird behaviour.
What version of containerd are you using?
1.16.18
Any other relevant information
No response
Show configuration if it is related to CRI plugin.
No response
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 16 (16 by maintainers)
Oh hey, I didn’t see the comments on this thread since for some reason I don’t get notifications on GitHub.
I can also confirm that it is a VHDX specific issue. We tried using WinSPD to create a virtual disk, formatted it NTFS and used that as the root for containerd and it worked fine.
This currently isn’t a high priority issue for me, so I don’t think I’ll get time to test a specific fix. I ran into it while exploring the possibility of a Windows snapshotter that uses VHDX mounts for the layers (instead of having to extract images), since combined with WinFSP for visualizing the VHDX storage, that would allow us to have a lazy pulled (stargz) snapshotter for Windows.