Add experimental "rootless" dind variant #174

tianon · 2019-07-30T20:39:05Z

This is an alternative proposal to #165 + #168.

This should have roughly the same behavior as the dockerd-rootless.sh script but without the extra depdendency (and frankly I'd love to pull some official vpnkit release binaries too instead of pulling the extras bundle, but I couldn't find any official binary releases nor did it look trivial to build from source). This allows us to iterate on this in between Docker releases as well, if necessary (like pulling in a newer rootlesskit release, for example).

I was toying with automatically adding --experimental as well, but wanted to enforce that being opt-in so that users don't switch without realizing what they're doing (since it does require some changes in usage).

Generally everything should "just work" as automatically as possible, but with the appropriate escape hatches for users who need/want to control the underlying knobs more directly.

Update (2021-03-04): as of 20.10.x, --experimental is no longer required (moby/moby#40759)

AkihiroSuda · 2019-07-30T20:46:01Z

Thanks for working on this, but I think dockerd-rootless.sh should be still kept as a separate script, so that we don't need to maintain the both upstream script and the downstream script in this repo.

Also, it is highly likely that the content of the script will differ in future revisions.

AkihiroSuda · 2019-07-30T20:50:28Z

Dockerfile-dind-rootless.template

+
+# create a default user preconfigured for running rootless dockerd
+RUN set -eux; \
+	adduser -h /home/rootless -g 'Rootless' -D -u 1000 rootless; \


the UID should be configurable as ARG https://github.com/docker-library/docker/pull/165/files#diff-3a278e88442db689fbd14924162365f4R12

Why? Folks shouldn't have to / shouldn't be rebuilding the image (I think the usability of that is way too low) -- the way it's configured, setting a reasonable value for HOME and setting up /etc/subuid and /etc/subgid should be enough for things to work (although some of the tools in use might also require /etc/passwd to be updated, but these are all pretty normal things for running a container as an "arbitrary user" and avoids requiring users to build their own custom images).

the UID 1000 is likely to overwrap with other users on the host, and it might break the user's env when the daemon got compromised.

Also the number of subuids should be reconfigurable, as some user might want more than 65536 subuids

The goal here is that the user can do something as trivial as the following in order to customize their environment (and thus adjust the numbers to their heart's content):

FROM docker:19.03-dind-rootless USER root RUN adduser -h /home/user -D -u 1234 user RUN echo 'user:101234:65536' >> /etc/subuid RUN echo 'user:101234:65536' >> /etc/subgid USER user

(most of this is also easily accomplishable via a docker run line, although a bit more gymnastics are involved)

tianon · 2019-07-30T21:48:32Z

So, the reason I avoided dockerd-rootless.sh is because the entire script boiled down to setting a couple default values (based on autodetection that we can already assume based on what we include in the image) and doing a couple rm commands after running rootlesskit -- it doesn't seem worth using that script. Do you anticipate more custom setup being required? If so, what kinds of things do you expect and/or why can't dockerd itself perform the necessary setup?

AkihiroSuda · 2019-07-30T22:17:47Z

Do you anticipate more custom setup being required?

I don't know, but something may change when we get support fo cgroup (v2).
Anyway, we can revisit this when we face some real issue.

18.09-rc/dind/dockerd-entrypoint.sh

AkihiroSuda · 2019-07-30T22:20:32Z

19.03-rc/dind-rootless/Dockerfile

+		*) echo >&2 "error: unsupported architecture ($apkArch)"; exit 1 ;;\
+	esac; \
+	\
+	if ! wget -O rootless.tgz "https://download.docker.com/linux/static/${DOCKER_CHANNEL}/${dockerArch}/docker-rootless-extras-${DOCKER_VERSION}.tgz"; then \


FYI: the origin of the binary is djs55/vpnkit image https://github.com/moby/moby/blob/7cfd8146dc6089a5f21415dc4c0a609a93c507fe/Dockerfile#L249

FYI: If you don't like the VPNKit binary footprint, https://github.com/rootless-containers/slirp4netns can be used instead of VPNKit.

slirp4netns is licensed under GPL2 and not included in the docker-rootless-extras.tgz because they didn't want to include GPL2 binary in the same tgz, but probably it doesn't hurt to put slirp4netns in Docker images.

I don't mind the footprint (or vpnkit in general -- I understand it's been used in Docker Desktop for a pretty respectable amount of time now, so it's "tested, tried, and true" so to speak) -- the issue I've got is with the need to pull this extra bundle which we then have to wait for a new Docker release to get an update to if there happens to be some issue that's resolved in a newer release of vpnkit, so it'd be really neat to get some official binary releases there, but I think that's a pretty minor issue at this point (vpnkit is pretty provably solid, as noted).

19.03-rc/dind-rootless/Dockerfile

AkihiroSuda · 2019-07-30T22:28:59Z

19.03-rc/dind-rootless/Dockerfile

+		/home/rootless/.local/share/docker \
+		/home/rootless/certs
+ENV DOCKER_TLS_CERTDIR=/home/rootless/certs
+VOLUME /home/rootless/.local/share/docker


I'd prefer setting ENV XDG_RUNTIME_DIR here so that it is visible to docker exec processes

and also DOCKER_HOST for docker exec docker

The expected usage there would be either docker exec -it docker-entrypoint.sh sh or docker exec docker-entrypoint.sh version, etc (using the same functionality as docker run docker ... does to set those same variables automatically/appropriately). Along those lines, I'm also going to update dockerd-entrypoint.sh to pass not only docker ... through to docker-entrypoint.sh, but any command that isn't dockerd (which should help with this as well).

tianon · 2019-07-31T21:53:17Z

Tested latest push via:

$ docker run -it --rm --name dind --privileged -v omg-certs:/certs/client docker:19.03-dind-rootless --experimental
...

$ docker run -it --rm --link dind:docker -v omg-certs:/certs/client docker:19.03-dind-rootless version
Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89e8a
 Built:             Thu Jul 25 21:17:37 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89e8a
  Built:            Thu Jul 25 21:27:55 2019
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Changes: - docker-library/docker@2285205: Merge pull request docker-library/docker#174 from infosiftr/rootless - docker-library/docker@c01ffa4: Adjust bits per review - docker-library/docker@7db7ecc: Add experimental "rootless" dind variant

docker-library#174 added an experimental "rootless" variant of the DinD service. While the change itself is simple and should not affect current usages, a `DOCKER_HOST` variable was used to define which docker.sock path should be used depending on the process owner's ID. Chosing DOCKER_HOST variable for this is unfortunately not the best option: 1. It's not the best semantic choice - the code that uses it defines a socket file, not the host. DOCKER_SOCKET just seems to fit better the purpose. 1. DOCKER_HOST is already a variable, that is used by users to define where the Docker daemon is listening. In case when it's added to the DinD container (which is a common situation for example for GitLab CI jobs that are using DinD as a service), it finally ends with assigning two times the same port, while once it uses 0.0.0.0 address, and once some unresolvable domain name (depending on what user defined; most probably `docker`). This commit proposes a change of the variable name to DOCKER_SOCKET, which will better match it purpose and additionally it will stop breaking configurations of many of docker:dind image users. Signed-off-by: Tomasz Maczukin <[email protected]>

related to docker-library/docker#174 (comment)

tim-gp · 2019-08-13T16:02:20Z

I've tried to spin up without certs per #174 (comment)

$ docker run -it --rm --name dind --privileged docker:19.03.1-dind-rootless --experimental

I get [rootlesskit:child ] error: executing [[ip tuntap add name tap0 mode tap] [ip link set tap0 address <MAC>]]: exit status 1 on Docker Desktop for Mac 2.1.0.0.

AkihiroSuda · 2019-08-13T16:29:44Z

opened docker/for-mac#3838 for Docker fo Mac issue

Add experimental "rootless" dind variant

7db7ecc

AkihiroSuda reviewed Jul 30, 2019

View reviewed changes

18.09-rc/dind/dockerd-entrypoint.sh Outdated Show resolved Hide resolved

AkihiroSuda reviewed Jul 30, 2019

View reviewed changes

19.03-rc/dind-rootless/Dockerfile Show resolved Hide resolved

AkihiroSuda reviewed Jul 30, 2019

View reviewed changes

19.03-rc/dind-rootless/Dockerfile Outdated Show resolved Hide resolved

AkihiroSuda reviewed Jul 30, 2019

View reviewed changes

This was referenced Jul 31, 2019

Add dind-rootless #165

Closed

split tls-generate-certs.sh from dockerd-entrypoint.sh #168

Closed

Adjust bits per review

c01ffa4

tianon force-pushed the rootless branch from 61bf8a9 to c01ffa4 Compare July 31, 2019 21:50

AkihiroSuda approved these changes Aug 1, 2019

View reviewed changes

yosifkit merged commit 2285205 into docker-library:master Aug 1, 2019

yosifkit deleted the rootless branch August 1, 2019 22:20

tianon mentioned this pull request Aug 1, 2019

Add a short blurb linking to the "dind-rootless" PR docker-library/docs#1547

Merged

tianon mentioned this pull request Aug 1, 2019

Update docker docker-library/official-images#6400

Merged

tmaczukin mentioned this pull request Aug 2, 2019

Rename variable used for defining used docker.sock file #177

Merged

This comment has been minimized.

Sign in to view

tianon mentioned this pull request Aug 2, 2019

publish docker:rootless #151

Closed

thomasleveil added a commit to thomasleveil/docker-jenkins-dsl-ready that referenced this pull request Aug 3, 2019

update dind test and documentation

c62ad97

related to docker-library/docker#174 (comment)

benabineri mentioned this pull request Aug 7, 2019

Rootless variant on Amazon Linux 2 - /proc/sys/kernel/unprivileged_userns_clone #183

Closed

AkihiroSuda mentioned this pull request Aug 13, 2019

rootless dind does not work on Docker for Mac docker/for-mac#3838

Open

2 tasks

viceice mentioned this pull request Sep 2, 2019

rootless dind does not work on Docker for Windows docker/for-win#4627

Closed

2 tasks

tianon mentioned this pull request Mar 5, 2020

docker:19.03.6-dind: chown not working with "docker build" when dockerd is run in userns #217

Closed

yosifkit mentioned this pull request Jul 17, 2020

Unable to run Docker Daemon in the dind-rootless image #245

Closed

yosifkit mentioned this pull request Oct 27, 2020

rootless: set ENV DOCKER_HOST=unix:///run/user/1000/docker.sock #266

Closed

tianon mentioned this pull request Dec 14, 2020

19.03-dind-rootless image fails lacking experimental #281

Closed

wglambert mentioned this pull request Dec 20, 2021

More detailed description on image version? #340

Closed

tianon mentioned this pull request Feb 26, 2025

Move rootless notes to a variant stub docker-library/docs#2546

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add experimental "rootless" dind variant #174

Add experimental "rootless" dind variant #174

tianon commented Jul 30, 2019 •

edited

Loading

AkihiroSuda commented Jul 30, 2019

AkihiroSuda Jul 30, 2019

tianon Jul 30, 2019

AkihiroSuda Jul 30, 2019

AkihiroSuda Jul 30, 2019

tianon Jul 31, 2019

tianon commented Jul 30, 2019

AkihiroSuda commented Jul 30, 2019

AkihiroSuda Jul 30, 2019

AkihiroSuda Jul 30, 2019

tianon Jul 31, 2019

AkihiroSuda Jul 30, 2019

AkihiroSuda Jul 30, 2019

tianon Jul 31, 2019

tianon commented Jul 31, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

tim-gp commented Aug 13, 2019

AkihiroSuda commented Aug 13, 2019

Add experimental "rootless" dind variant #174

Add experimental "rootless" dind variant #174

Conversation

tianon commented Jul 30, 2019 • edited Loading

AkihiroSuda commented Jul 30, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tianon commented Jul 30, 2019

AkihiroSuda commented Jul 30, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tianon commented Jul 31, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

tim-gp commented Aug 13, 2019

AkihiroSuda commented Aug 13, 2019

tianon commented Jul 30, 2019 •

edited

Loading