diff options
Diffstat (limited to 'doc/ci/docker/using_docker_build.md')
| -rw-r--r-- | doc/ci/docker/using_docker_build.md | 73 |
1 files changed, 38 insertions, 35 deletions
diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index f992af6c8a5..65b9c03186b 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -1,10 +1,13 @@ --- +stage: Verify +group: Continuous Integration +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: concepts, howto --- # Building Docker images with GitLab CI/CD -GitLab CI/CD allows you to use Docker Engine to build and test docker-based projects. +GitLab CI/CD allows you to use Docker Engine to build and test Docker-based projects. One of the new trends in Continuous Integration/Deployment is to: @@ -91,15 +94,15 @@ NOTE: **Note:** By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions. For more information please read [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). -### Use docker-in-docker workflow with Docker executor +### Use Docker-in-Docker workflow with Docker executor -The second approach is to use the special docker-in-docker (dind) +The second approach is to use the special Docker-in-Docker (dind) [Docker image](https://hub.docker.com/_/docker/) with all tools installed (`docker`) and run the job script in context of that image in privileged mode. NOTE: **Note:** -`docker-compose` is not part of docker-in-docker (dind). To use `docker-compose` in your +`docker-compose` is not part of Docker-in-Docker (dind). To use `docker-compose` in your CI builds, follow the `docker-compose` [installation instructions](https://docs.docker.com/compose/install/). @@ -113,19 +116,19 @@ out the official Docker documentation on Docker-in-Docker works well, and is the recommended configuration, but it is not without its own challenges: -- When using docker-in-docker, each job is in a clean environment without the past +- When using Docker-in-Docker, each job is in a clean environment without the past history. Concurrent jobs work fine because every build gets its own instance of Docker engine so they won't conflict with each other. But this also means that jobs can be slower because there's no caching of layers. - By default, Docker 17.09 and higher uses `--storage-driver overlay2` which is the recommended storage driver. See [Using the overlayfs driver](#use-the-overlayfs-driver) for details. -- Since the `docker:19.03.8-dind` container and the Runner container don't share their +- Since the `docker:19.03.11-dind` container and the Runner container don't share their root filesystem, the job's working directory can be used as a mount point for child containers. For example, if you have files you want to share with a child container, you may create a subdirectory under `/builds/$CI_PROJECT_PATH` and use it as your mount point (for a more thorough explanation, check [issue - #41227](https://gitlab.com/gitlab-org/gitlab-foss/issues/41227)): + #41227](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41227)): ```yaml variables: @@ -139,7 +142,7 @@ not without its own challenges: An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. In the examples below, we are using Docker images tags to specify a -specific version, such as `docker:19.03.8`. If tags like `docker:stable` +specific version, such as `docker:19.03.11`. If tags like `docker:stable` are used, you have no control over what version is going to be used and this can lead to unpredictable behavior, especially when new versions are released. @@ -151,12 +154,12 @@ Requires GitLab Runner 11.11 or later, but is not supported if GitLab Runner is installed using the [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html). See the [related -issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/issues/83) for +issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/83) for details. The Docker daemon supports connection over TLS and it's done by default -for Docker 19.03.8 or higher. This is the **suggested** way to use the -docker-in-docker service and +for Docker 19.03.11 or higher. This is the **suggested** way to use the +Docker-in-Docker service and [GitLab.com Shared Runners](../../user/gitlab_com/index.md#shared-runners) support this. @@ -171,19 +174,19 @@ support this. --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "docker:19.03.8" \ + --docker-image "docker:19.03.11" \ --docker-privileged \ --docker-volumes "/certs/client" ``` The above command will register a new Runner to use the special - `docker:19.03.8` image, which is provided by Docker. **Notice that it's + `docker:19.03.11` image, which is provided by Docker. **Notice that it's using the `privileged` mode to start the build and service - containers.** If you want to use [docker-in-docker](https://www.docker.com/blog/docker-can-now-run-within-docker/) mode, you always + containers.** If you want to use [Docker-in-Docker](https://www.docker.com/blog/docker-can-now-run-within-docker/) mode, you always have to use `privileged = true` in your Docker containers. This will also mount `/certs/client` for the service and build - container, which is needed for the docker client to use the + container, which is needed for the Docker client to use the certificates inside of that directory. For more information how Docker with TLS works check <https://hub.docker.com/_/docker/#tls>. @@ -196,7 +199,7 @@ support this. executor = "docker" [runners.docker] tls_verify = false - image = "docker:19.03.8" + image = "docker:19.03.11" privileged = true disable_cache = false volumes = ["/certs/client", "/cache"] @@ -206,10 +209,10 @@ support this. ``` 1. You can now use `docker` in the build script (note the inclusion of the - `docker:19.03.8-dind` service): + `docker:19.03.11-dind` service): ```yaml - image: docker:19.03.8 + image: docker:19.03.11 variables: # When using dind service, we need to instruct docker, to talk with @@ -234,7 +237,7 @@ support this. DOCKER_TLS_CERTDIR: "/certs" services: - - docker:19.03.8-dind + - docker:19.03.11-dind before_script: - docker info @@ -261,7 +264,7 @@ Assuming that the Runner `config.toml` is similar to: executor = "docker" [runners.docker] tls_verify = false - image = "docker:19.03.8" + image = "docker:19.03.11" privileged = true disable_cache = false volumes = ["/cache"] @@ -271,10 +274,10 @@ Assuming that the Runner `config.toml` is similar to: ``` You can now use `docker` in the build script (note the inclusion of the -`docker:19.03.8-dind` service): +`docker:19.03.11-dind` service): ```yaml -image: docker:19.03.8 +image: docker:19.03.11 variables: # When using dind service we need to instruct docker, to talk with the @@ -295,7 +298,7 @@ variables: DOCKER_TLS_CERTDIR: "" services: - - docker:19.03.8-dind + - docker:19.03.11-dind before_script: - docker info @@ -315,7 +318,7 @@ container so that Docker is available in the context of that image. NOTE: **Note:** If you bind the Docker socket [when using GitLab Runner 11.11 or newer](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), -you can no longer use `docker:19.03.8-dind` as a service because volume bindings +you can no longer use `docker:19.03.11-dind` as a service because volume bindings are done to the services as well, making these incompatible. In order to do that, follow the steps: @@ -330,12 +333,12 @@ In order to do that, follow the steps: --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "docker:19.03.8" \ + --docker-image "docker:19.03.11" \ --docker-volumes /var/run/docker.sock:/var/run/docker.sock ``` The above command will register a new Runner to use the special - `docker:19.03.8` image which is provided by Docker. **Notice that it's using + `docker:19.03.11` image which is provided by Docker. **Notice that it's using the Docker daemon of the Runner itself, and any containers spawned by Docker commands will be siblings of the Runner rather than children of the Runner.** This may have complications and limitations that are unsuitable for your workflow. @@ -349,7 +352,7 @@ In order to do that, follow the steps: executor = "docker" [runners.docker] tls_verify = false - image = "docker:19.03.8" + image = "docker:19.03.11" privileged = false disable_cache = false volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] @@ -358,11 +361,11 @@ In order to do that, follow the steps: ``` 1. You can now use `docker` in the build script (note that you don't need to - include the `docker:19.03.8-dind` service as when using the Docker in Docker + include the `docker:19.03.11-dind` service as when using the Docker in Docker executor): ```yaml - image: docker:19.03.8 + image: docker:19.03.11 before_script: - docker info @@ -377,7 +380,7 @@ In order to do that, follow the steps: While the above method avoids using Docker in privileged mode, you should be aware of the following implications: -- By sharing the docker daemon, you are effectively disabling all +- By sharing the Docker daemon, you are effectively disabling all the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For example, if a project ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner @@ -392,9 +395,9 @@ aware of the following implications: docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests ``` -## Making docker-in-docker builds faster with Docker layer caching +## Making Docker-in-Docker builds faster with Docker layer caching -When using docker-in-docker, Docker will download all layers of your image every +When using Docker-in-Docker, Docker will download all layers of your image every time you create a build. Recent versions of Docker (Docker 1.13 and above) can use a pre-existing image as a cache during the `docker build` step, considerably speeding up the build process. @@ -416,10 +419,10 @@ any image that's used with the `--cache-from` argument must first be pulled Here's a `.gitlab-ci.yml` file showing how Docker caching can be used: ```yaml -image: docker:19.03.8 +image: docker:19.03.11 services: - - docker:19.03.8-dind + - docker:19.03.11-dind variables: # Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled @@ -514,7 +517,7 @@ Once you've built a Docker image, you can push it up to the built-in ## Troubleshooting -### docker: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running? +### `docker: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?` This is a common error when you are using [Docker in Docker](#use-docker-in-docker-workflow-with-docker-executor) |