diff options
Diffstat (limited to 'doc/ci/docker/using_docker_build.md')
| -rw-r--r-- | doc/ci/docker/using_docker_build.md | 239 |
1 files changed, 150 insertions, 89 deletions
diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index ebbfde09c67..c8eeb5c222a 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -152,9 +152,9 @@ Docker-in-Docker service and [GitLab.com shared runners](../../user/gitlab_com/index.md#shared-runners) support this. -GitLab Runner 11.11 or later is required, but it is not supported if GitLab -Runner is installed using the [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html). -See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/83) for details. +##### Docker + +> Introduced in GitLab Runner 11.11. 1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). 1. Register GitLab Runner from the command line to use `docker` and `privileged` @@ -217,6 +217,62 @@ See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/iss # The 'docker' hostname is the alias of the service container as described at # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. # + # Specify to Docker where to create the certificates, Docker will + # create them automatically on boot, and will create + # `/certs/client` that will be shared between the service and job + # container, thanks to volume mount from config.toml + DOCKER_TLS_CERTDIR: "/certs" + + services: + - docker:19.03.12-dind + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` + +##### Kubernetes + +> [Introduced](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/106) in GitLab Runner Helm Chart 0.23.0. + +1. Using the + [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html), update the + [`values.yml` file](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/00c1a2098f303dffb910714752e9a981e119f5b5/values.yaml#L133-137) + to specify a volume mount. + + ```yaml + runners: + config: | + [[runners]] + [runners.kubernetes] + image = "ubuntu:20.04" + privileged = true + [[runners.kubernetes.volumes.empty_dir]] + name = "docker-certs" + mount_path = "/certs/client" + medium = "Memory" + ``` + +1. You can now use `docker` in the build script (note the inclusion of the + `docker:19.03.13-dind` service): + + ```yaml + image: docker:19.03.13 + + variables: + # When using dind service, we need to instruct docker to talk with + # the daemon started inside of the service. The daemon is available + # with a network connection instead of the default + # /var/run/docker.sock socket. + DOCKER_HOST: tcp://docker:2376 + # + # The 'docker' hostname is the alias of the service container as described at + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier, # the variable must be set to tcp://localhost:2376 because of how the # Kubernetes executor connects services to the job container @@ -227,9 +283,14 @@ See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/iss # `/certs/client` that will be shared between the service and job # container, thanks to volume mount from config.toml DOCKER_TLS_CERTDIR: "/certs" + # These are usually specified by the entrypoint, however the + # Kubernetes executor doesn't run entrypoints + # https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4125 + DOCKER_TLS_VERIFY: 1 + DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client" services: - - docker:19.03.12-dind + - docker:19.03.13-dind before_script: - docker info @@ -302,6 +363,90 @@ build: - docker run my-docker-image /script/to/run/tests ``` +#### Use Docker socket binding + +The third approach is to bind-mount `/var/run/docker.sock` into the +container so that Docker is available in the context of that image. + +NOTE: **Note:** +If you bind the Docker socket and you are +[using GitLab Runner 11.11 or later](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), +you can no longer use `docker:19.03.12-dind` as a service. Volume bindings +are done to the services as well, making these incompatible. + +To make Docker available in the context of the image: + +1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). +1. From the command line, register a runner with the `docker` executor and share `/var/run/docker.sock`: + + ```shell + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:19.03.12" \ + --docker-volumes /var/run/docker.sock:/var/run/docker.sock + ``` + + This command registers a new runner to use the special + `docker:19.03.12` image, which is provided by Docker. **The command uses + the Docker daemon of the runner itself. Any containers spawned by Docker + commands are siblings of the runner rather than children of the runner.** + This may have complications and limitations that are unsuitable for your workflow. + + Your `config.toml` file should not have an entry like this: + + ```toml + [[runners]] + url = "https://gitlab.com/" + token = REGISTRATION_TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:19.03.12" + privileged = false + disable_cache = false + volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] + [runners.cache] + Insecure = false + ``` + +1. Use `docker` in the build script. You don't need to + include the `docker:19.03.12-dind` service, like you do when you're using + the Docker-in-Docker executor: + + ```yaml + image: docker:19.03.12 + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` + +This method avoids using Docker in privileged mode. However, +the implications of this method are: + +- By sharing the Docker daemon, you are effectively disabling all + the security mechanisms of containers and exposing your host to privilege + escalation, which can lead to container breakout. For example, if a project + ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner + containers. +- Concurrent jobs may not work; if your tests + create containers with specific names, they may conflict with each other. +- Sharing files and directories from the source repository into containers may not + work as expected. Volume mounting is done in the context of the host + machine, not the build container. For example: + + ```shell + docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests + ``` + #### Enable registry mirror for `docker:dind` service When the Docker daemon starts inside of the service container, it uses @@ -381,7 +526,7 @@ content: Update the `config.toml` file to mount the file to `/etc/docker/daemon.json`. This would mount the file for **every** -container that is created by GitLab Runner. The configuration will be +container that is created by GitLab Runner. The configuration is picked up by the `dind` service. ```toml @@ -444,90 +589,6 @@ The configuration is picked up by the `dind` service. sub_path = "daemon.json" ``` -#### Use Docker socket binding - -The third approach is to bind-mount `/var/run/docker.sock` into the -container so that Docker is available in the context of that image. - -NOTE: **Note:** -If you bind the Docker socket [when using GitLab Runner 11.11 or -newer](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), -you can no longer use `docker:19.03.12-dind` as a service because volume bindings -are done to the services as well, making these incompatible. - -In order to do that, follow the steps: - -1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). -1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: - - ```shell - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:19.03.12" \ - --docker-volumes /var/run/docker.sock:/var/run/docker.sock - ``` - - The above command registers a new runner to use the special - `docker:19.03.12` image which is provided by Docker. **Notice that it's using - the Docker daemon of the runner itself, and any containers spawned by Docker - commands are siblings of the runner rather than children of the runner.** - This may have complications and limitations that are unsuitable for your workflow. - - The above command creates a `config.toml` entry similar to this: - - ```toml - [[runners]] - url = "https://gitlab.com/" - token = REGISTRATION_TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:19.03.12" - privileged = false - disable_cache = false - volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] - [runners.cache] - Insecure = false - ``` - -1. You can now use `docker` in the build script (note that you don't need to - include the `docker:19.03.12-dind` service as when using the Docker in Docker - executor): - - ```yaml - image: docker:19.03.12 - - before_script: - - docker info - - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` - -While the above method avoids using Docker in privileged mode, you should be -aware of the following implications: - -- By sharing the Docker daemon, you are effectively disabling all - the security mechanisms of containers and exposing your host to privilege - escalation which can lead to container breakout. For example, if a project - ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner - containers. -- Concurrent jobs may not work; if your tests - create containers with specific names, they may conflict with each other. -- Sharing files and directories from the source repository into containers may not - work as expected since volume mounting is done in the context of the host - machine, not the build container. For example: - - ```shell - docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests - ``` - ## Making Docker-in-Docker builds faster with Docker layer caching When using Docker-in-Docker, Docker downloads all layers of your image every |