From 283a289a69e1fcca0fd0f755c1db5ba272209636 Mon Sep 17 00:00:00 2001 From: Steve Azzopardi Date: Tue, 23 Jul 2019 14:43:40 +0200 Subject: Update DinD example for 19.03 Docker 19.03 enables TLS by default: ``` Starting in 18.09+, the dind variants of this image will automatically generate TLS certificates in the directory specified by the DOCKER_TLS_CERTDIR environment variable. Warning: in 18.09, this behavior is disabled by default (for compatibility). If you use --network=host, shared network namespaces (as in Kubernetes pods), or otherwise have network access to the container (including containers started within the dind instance via their gateway interface), this is a potential security issue (which can lead to access to the host system, for example). It is recommended to enable TLS by setting the variable to an appropriate value (-e DOCKER_TLS_CERTDIR=/certs or similar). In 19.03+, this behavior is enabled by default. ``` Update the example to use docker over TLS. --- doc/ci/docker/using_docker_build.md | 204 ++++++++++++++++++++++++++---------- 1 file changed, 150 insertions(+), 54 deletions(-) diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 278a0d6e934..2cbad5f101c 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -98,9 +98,62 @@ The second approach is to use the special docker-in-docker (dind) (`docker`) and run the job script in context of that image in privileged mode. -NOTE: **Note:** `docker-compose` is not part of docker-in-docker (dind). In case you'd like to use `docker-compose` in your CI builds, please follow the [installation instructions for docker-compose](https://docs.docker.com/compose/install/) provided by docker. +NOTE: **Note:** +`docker-compose` is not part of docker-in-docker (dind). To use `docker-compose` in your +CI builds, follow the `docker-compose` +[installation instructions](https://docs.docker.com/compose/install/). -In order to do that, follow the steps: +DANGER: **Danger:** +By enabling `--docker-privileged`, you are effectively disabling all of +the security mechanisms of containers and exposing your host to privilege +escalation which can lead to container breakout. For more information, check +out the official Docker documentation on +[Runtime privilege and Linux capabilities][docker-cap]. + +Docker-in-Docker works well, and is the recommended configuration, but it is +not without its own challenges: + +- When using docker-in-docker, each job is in a clean environment without the past + history. Concurrent jobs work fine because every build gets it's own + instance of Docker engine so they won't conflict with each other. But this + also means jobs can be slower because there's no caching of layers. +- By default, `docker:dind` uses `--storage-driver vfs` which is the slowest + form offered. To use a different driver, see + [Using the overlayfs driver](#using-the-overlayfs-driver). +- Since the `docker:dind` container and the runner container don't share their + root filesystem, the job's working directory can be used as a mount point for + child containers. For example, if you have files you want to share with a + child container, you may create a subdirectory under `/builds/$CI_PROJECT_PATH` + and use it as your mount point (for a more thorough explanation, check [issue + #41227](https://gitlab.com/gitlab-org/gitlab-ce/issues/41227)): + + ```yaml + variables: + MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt + + script: + - mkdir -p "$MOUNT_POINT" + - docker run -v "$MOUNT_POINT:/mnt" my-docker-image + ``` + +An example project using this approach can be found here: . + +In the examples below, we are using Docker images tags to specify a +specific version, such as `docker:19.03.1`. If tags like `docker:stable` +are used, you have no control over what version is going to be used and this +can lead to unpredictable behavior, especially when new versions are +released. + +#### TLS enabled + +NOTE: **Note** +This requires GitLab Runner 11.11 or higher. + +The Docker daemon supports connection over TLS and it's done by default +for Docker 19.03.1 or higher. This is the **suggested** way to use the +docker-in-docker service and +[GitLab.com Shared Runners](../../user/gitlab_com/index.html#shared-runners) +support this. 1. Install [GitLab Runner](https://docs.gitlab.com/runner/install). @@ -113,22 +166,21 @@ In order to do that, follow the steps: --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "docker:stable" \ - --docker-privileged + --docker-image "docker:19.03.1" \ + --docker-privileged \ + --docker-volumes "/certs/client" ``` The above command will register a new Runner to use the special - `docker:stable` image which is provided by Docker. **Notice that it's using - the `privileged` mode to start the build and service containers.** If you - want to use [docker-in-docker] mode, you always have to use `privileged = true` - in your Docker containers. + `docker:19.03.1` image, which is provided by Docker. **Notice that it's + using the `privileged` mode to start the build and service + containers.** If you want to use [docker-in-docker] mode, you always + have to use `privileged = true` in your Docker containers. - DANGER: **Danger:** - By enabling `--docker-privileged`, you are effectively disabling all of - the security mechanisms of containers and exposing your host to privilege - escalation which can lead to container breakout. For more information, check - out the official Docker documentation on - [Runtime privilege and Linux capabilities][docker-cap]. + This will also mount `/certs/client` for the service and build + container, which is needed for the docker client to use the + certificates inside of that directory. For more information how + Docker with TLS works check . The above command will create a `config.toml` entry similar to this: @@ -139,41 +191,48 @@ In order to do that, follow the steps: executor = "docker" [runners.docker] tls_verify = false - image = "docker:stable" + image = "docker:19.03.1" privileged = true disable_cache = false - volumes = ["/cache"] + volumes = ["/certs/client", "/cache"] [runners.cache] - Insecure = false - ``` + [runners.cache.s3] + [runners.cache.gcs] + ``` 1. You can now use `docker` in the build script (note the inclusion of the - `docker:dind` service): + `docker:19.03.1-dind` service): ```yaml - image: docker:stable + image: docker:19.03.1 variables: - # When using dind service we need to instruct docker, to talk with the - # daemon started inside of the service. The daemon is available with - # a network connection instead of the default /var/run/docker.sock socket. + # When using dind service, we need to instruct docker, to talk with + # the daemon started inside of the service. The daemon is available + # with a network connection instead of the default + # /var/run/docker.sock socket. docker:19.03.1 does this automatically + # by setting the DOCKER_HOST in + # https://github.com/docker-library/docker/blob/d45051476babc297257df490d22cbd806f1b11e4/19.03.1/docker-entrypoint.sh#L23-L29 # # The 'docker' hostname is the alias of the service container as described at - # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. # - # Note that if you're using the Kubernetes executor, the variable should be set to - # tcp://localhost:2375/ because of how the Kubernetes executor connects services - # to the job container - # DOCKER_HOST: tcp://localhost:2375/ + # Note that if you're using the Kubernetes executor, the variable + # should be set to tcp://localhost:2376/ because of how the + # Kubernetes executor connects services to the job container + # DOCKER_HOST: tcp://localhost:2376/ # - # For non-Kubernetes executors, we use tcp://docker:2375/ - DOCKER_HOST: tcp://docker:2375/ # When using dind, it's wise to use the overlayfs driver for # improved performance. DOCKER_DRIVER: overlay2 + # Specify to Docker where to create the certificates, Docker will + # create them automatically on boot, and will create + # `/certs/client` that will be shared between the service and job + # container, thanks to volume mount from config.toml + DOCKER_TLS_CERTDIR: "/certs" services: - - docker:dind + - docker:19.03.1-dind before_script: - docker info @@ -185,33 +244,70 @@ In order to do that, follow the steps: - docker run my-docker-image /script/to/run/tests ``` -Docker-in-Docker works well, and is the recommended configuration, but it is -not without its own challenges: +#### TLS disabled -- When using docker-in-docker, each job is in a clean environment without the past - history. Concurrent jobs work fine because every build gets it's own - instance of Docker engine so they won't conflict with each other. But this - also means jobs can be slower because there's no caching of layers. -- By default, `docker:dind` uses `--storage-driver vfs` which is the slowest - form offered. To use a different driver, see - [Using the overlayfs driver](#using-the-overlayfs-driver). -- Since the `docker:dind` container and the runner container don't share their - root filesystem, the job's working directory can be used as a mount point for - children containers. For example, if you have files you want to share with a - child container, you may create a subdirectory under `/builds/$CI_PROJECT_PATH` - and use it as your mount point (for a more thorough explanation, check [issue - #41227](https://gitlab.com/gitlab-org/gitlab-ce/issues/41227)): +Sometimes there are legitimate reasons why you might want to disable TLS. +For example, you have no control over the GitLab Runner configuration +that you are using. - ```yaml - variables: - MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt +Assuming that the Runner `config.toml` is similar to: - script: - - mkdir -p "$MOUNT_POINT" - - docker run -v "$MOUNT_POINT:/mnt" my-docker-image - ``` +```toml +[[runners]] + url = "https://gitlab.com/" + token = TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:19.03.1" + privileged = true + disable_cache = false + volumes = ["/cache"] + [runners.cache] + [runners.cache.s3] + [runners.cache.gcs] +``` -An example project using this approach can be found here: . +You can now use `docker` in the build script (note the inclusion of the +`docker:19.03.1-dind` service): + +```yaml +image: docker:19.03.1 + +variables: + # When using dind service we need to instruct docker, to talk with the + # daemon started inside of the service. The daemon is available with + # a network connection instead of the default /var/run/docker.sock socket. + # + # The 'docker' hostname is the alias of the service container as described at + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services + # + # Note that if you're using the Kubernetes executor, the variable should be set to + # tcp://localhost:2375/ because of how the Kubernetes executor connects services + # to the job container + # DOCKER_HOST: tcp://localhost:2375/ + # + # For non-Kubernetes executors, we use tcp://docker:2375/ + DOCKER_HOST: tcp://docker:2375/ + # When using dind, it's wise to use the overlayfs driver for + # improved performance. + DOCKER_DRIVER: overlay2 + # + # This will instruct Docker not to start over TLS. + DOCKER_TLS_CERTDIR: "" + +services: + - docker:19.03.1-dind + +before_script: + - docker info + +build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests +``` ### Use Docker socket binding -- cgit v1.2.3