-Old distributions
-
-
-
-### Debian GNU/Linux 10
-Add `kernel.unprivileged_userns_clone=1` to `/etc/sysctl.conf` (or `/etc/sysctl.d`) and run `sudo sysctl -p`.
-This step is not needed for Debian GNU/Linux 11 and later.
-
-### RHEL/CentOS 7
-Add `user.max_user_namespaces=28633` to `/etc/sysctl.conf` (or `/etc/sysctl.d`) and run `sudo sysctl -p`.
-This step is not needed for RHEL/CentOS 8 and later.
-
-### Fedora, before kernel 5.13
-You may have to disable SELinux, or run BuildKit with `--oci-worker-snapshotter=fuse-overlayfs`.
-
-
-
-
## Known limitations
* Using the `overlayfs` snapshotter requires kernel >= 5.11 or Ubuntu kernel.
On kernel >= 4.18, the `fuse-overlayfs` snapshotter is used instead of `overlayfs`.
@@ -93,37 +41,6 @@ Then let buildkitd join the same namespace as containerd.
$ containerd-rootless-setuptool.sh nsenter -- buildkitd --oci-worker=false --containerd-worker=true --containerd-worker-snapshotter=native
```
-## Troubleshooting
-
-### Error related to `overlayfs`
-Try running `buildkitd` with `--oci-worker-snapshotter=fuse-overlayfs`:
-
-```console
-$ rootlesskit buildkitd --oci-worker-snapshotter=fuse-overlayfs
-```
-
-### Error related to `fuse-overlayfs`
-Try running `buildkitd` with `--oci-worker-snapshotter=native`:
-
-```console
-$ rootlesskit buildkitd --oci-worker-snapshotter=native
-```
-
-### Error related to `newuidmap` or `/etc/subuid`
-See https://rootlesscontaine.rs/getting-started/common/subuid/
-
-### Error `Options:[rbind ro]}]: operation not permitted`
-Make sure to mount an `emptyDir` volume on `/home/user/.local/share/buildkit` .
-
-### Error `fork/exec /proc/self/exe: no space left on device` with `level=warning msg="/proc/sys/user/max_user_namespaces needs to be set to non-zero."`
-Run `sysctl -w user.max_user_namespaces=N` (N=positive integer, like 63359) on the host nodes.
-
-See [`../examples/kubernetes/sysctl-userns.privileged.yaml`](../examples/kubernetes/sysctl-userns.privileged.yaml).
-
-### Error `mount proc:/proc (via /proc/self/fd/6), flags: 0xe: operation not permitted`
-This error is known to happen when BuildKit is executed in a container without the `--oci-worker-no-sandbox` flag.
-Make sure that `--oci-worker-no-process-sandbox` is specified (See [below](#docker)).
-
## Containerized deployment
### Kubernetes
@@ -193,3 +110,85 @@ $ make images
$ docker run ... moby/buildkit:local-rootless ...
```
+## Troubleshooting
+
+### Error related to `overlayfs`
+Try running `buildkitd` with `--oci-worker-snapshotter=fuse-overlayfs`:
+
+```console
+$ rootlesskit buildkitd --oci-worker-snapshotter=fuse-overlayfs
+```
+
+### Error related to `fuse-overlayfs`
+Try running `buildkitd` with `--oci-worker-snapshotter=native`:
+
+```console
+$ rootlesskit buildkitd --oci-worker-snapshotter=native
+```
+
+### Error related to `newuidmap` or `/etc/subuid`
+See https://rootlesscontaine.rs/getting-started/common/subuid/
+
+### Error `Options:[rbind ro]}]: operation not permitted`
+Make sure to mount an `emptyDir` volume on `/home/user/.local/share/buildkit` .
+
+### Error `fork/exec /proc/self/exe: no space left on device` with `level=warning msg="/proc/sys/user/max_user_namespaces needs to be set to non-zero."`
+Run `sysctl -w user.max_user_namespaces=N` (N=positive integer, like 63359) on the host nodes.
+
+See [`../examples/kubernetes/sysctl-userns.privileged.yaml`](../examples/kubernetes/sysctl-userns.privileged.yaml).
+
+### Error `mount proc:/proc (via /proc/self/fd/6), flags: 0xe: operation not permitted`
+This error is known to happen when BuildKit is executed in a container without the `--oci-worker-no-sandbox` flag.
+Make sure that `--oci-worker-no-process-sandbox` is specified (See [below](#docker)).
+
+## Distribution-specific hint
+Using Ubuntu kernel is recommended.
+### Container-Optimized OS from Google
+Make sure to have an `emptyDir` volume below:
+```yaml
+spec:
+ containers:
+ - name: buildkitd
+ volumeMounts:
+ # Dockerfile has `VOLUME /home/user/.local/share/buildkit` by default too,
+ # but the default VOLUME does not work with rootless on Google's Container-Optimized OS
+ # as it is mounted with `nosuid,nodev`.
+ # https://github.com/moby/buildkit/issues/879#issuecomment-1240347038
+ - mountPath: /home/user/.local/share/buildkit
+ name: buildkitd
+ volumes:
+ - name: buildkitd
+ emptyDir: {}
+```
+
+See also the [example manifests](#Kubernetes).
+
+### Bottlerocket OS
+
+Needs to set the max user namespaces to a positive integer, through the [API settings](https://github.com/bottlerocket-os/bottlerocket#kernel-settings):
+
+```toml
+[settings.kernel.sysctl]
+"user.max_user_namespaces" = "16384"
+```
+
+See [`../examples/eksctl/bottlerocket.yaml`](../examples/eksctl/bottlerocket.yaml) for an example to configure a Node Group in EKS.
+
+