WCOW: buildkitd can't spin up Hyper-V isolated containers for RUN commands

[riverar]

Summary

Attempting to build an image on Windows vNext 27774 or Windows 11 results in failure due to mismatching container/host operating system versions. Buildkitd has no apparent way to force the use of Hyper-V isolated containers.

Microsoft has not provided containers compatible with newer versions of Windows forcing Hyper-V isolation use.

Details

PS C:\containerd\sample> buildctl.exe build --frontend=dockerfile.v0 --local context=. --local dockerfile=. --output type=image,name=localhost/hello,push=true
[+] Building 250.8s (5/9)
 => [internal] load build definition from Dockerfile                                                                                                0.1s
 => => transferring dockerfile: 378B                                                                                                                0.0s
 => [internal] load metadata for mcr.microsoft.com/windows:ltsc2019-amd64                                                                           0.2s
 => [internal] load .dockerignore                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                     0.0s
 => [1/6] FROM mcr.microsoft.com/windows:ltsc2019-amd64@sha256:4e9bb33a0297c75a0f803d8b234f25f67624fa2c0f11b96b001338d43a3e4206                   249.1s
 => => resolve mcr.microsoft.com/windows:ltsc2019-amd64@sha256:4e9bb33a0297c75a0f803d8b234f25f67624fa2c0f11b96b001338d43a3e4206                     0.0s
 => => extracting sha256:7293265e0c3e99d74f84da6c264417dc49da57ac45e26f6c58e5e62a02b5a023                                                         249.0s
 => ERROR [2/6] RUN net user /add Demo                                                                                                            1.2s
------
 > [2/6] RUN net user /add Demo:
------
Dockerfile:3
--------------------
   1 |     FROM mcr.microsoft.com/windows:ltsc2019-amd64
   2 |     USER ContainerAdministrator
   3 | >>> RUN net user /add Demo
...
--------------------
error: failed to solve: process "cmd /S /C net user /add Demo" did not complete successfully: hcs::System::Start t4e3dlwzslv1pmq7jmwsp1s2d: The container operating system does not match the host operating system.: unknown

Versions

buildctl github.com/moby/buildkit v0.19.0 3637d1b
buildkitd github.com/moby/buildkit v0.19.0 3637d1b

Questions

• Is it possible to force Hyper-V isolation for all containers at the containerd level?

[profnandaa]

@riverar -- I will scope this and get back to you. Given that the classic docker supports docker build --isolation hyperv .., this is a valid ask.

However, getting this on the containerd side [1], will dig through ...

__
[1] see some ctr run vs. docker run on Win 11 (Build 22631), what might I be missing?

$ ctr run --rm mcr.microsoft.com/windows/nanoserver:ltsc2019 hyp-test-1 echo "testing"
ctr: no match for platform in manifest sha256:d96a6c2108e1449c872cc2b224a9b3444fa1415b4d6947280aba2d2bb3bd0025: not found
$ ctr run --rm --isolated=true mcr.microsoft.com/windows/nanoserver:ltsc2019 hyp-test-1 echo "testing"
ctr: no match for platform in manifest sha256:d96a6c2108e1449c872cc2b224a9b3444fa1415b4d6947280aba2d2bb3bd0025: not found

--
$ docker run --rm --isolation hyperv mcr.microsoft.com/windows/nanoserver:ltsc2019 cmd.exe /C echo "testing"
testing

[profnandaa]

Update, needed to specify -amd64 on the tag, so ctr run --isolated works okay:

ctr run --rm --isolated=true mcr.microsoft.com/windows/nanoserver:ltsc2019-amd64 hyp-test-1 cmd.exe /c echo "testing"
testing

[profnandaa]

@riverar -- can take a look at this #5753

[riverar]

@profnandaa You're a machine! Will take a look in the AM and get you some feedback.