diff options
author | Marcel Amirault <mamirault@gitlab.com> | 2019-08-01 05:29:16 +0300 |
---|---|---|
committer | Evan Read <eread@gitlab.com> | 2019-08-01 05:29:16 +0300 |
commit | a696695f31a2df748eed0b9cf9454a6046fdc71e (patch) | |
tree | 1afce82578050fd82a34e7b2a65d5f12ef20db58 /doc/ci/docker | |
parent | cfb7f11644d41bedc423865caa28b4266840f8e4 (diff) |
Fix whitespace in ci docs
Many code blocks are 4spaced, and they render in GitLab
without coloring as a result, even though they are
fenced with a language label. If in a list, other items
will render as being in a code block too, even if not
meant to. This fixes all these issues for most docs in
/ci, and cleans up other minor whitespace issues too.
Diffstat (limited to 'doc/ci/docker')
-rw-r--r-- | doc/ci/docker/using_docker_build.md | 342 | ||||
-rw-r--r-- | doc/ci/docker/using_docker_images.md | 131 |
2 files changed, 238 insertions, 235 deletions
diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 10a21edbd34..278a0d6e934 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -48,42 +48,42 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. 1. During GitLab Runner installation select `shell` as method of executing job scripts or use command: - ```bash - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor shell \ - --description "My Runner" - ``` + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor shell \ + --description "My Runner" + ``` 1. Install Docker Engine on server. - For more information how to install Docker Engine on different systems - checkout the [Supported installations](https://docs.docker.com/engine/installation/). + For more information how to install Docker Engine on different systems + checkout the [Supported installations](https://docs.docker.com/engine/installation/). 1. Add `gitlab-runner` user to `docker` group: - ```bash - sudo usermod -aG docker gitlab-runner - ``` + ```bash + sudo usermod -aG docker gitlab-runner + ``` 1. Verify that `gitlab-runner` has access to Docker: - ```bash - sudo -u gitlab-runner -H docker info - ``` + ```bash + sudo -u gitlab-runner -H docker info + ``` - You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`: + You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`: - ```yaml - before_script: - - docker info + ```yaml + before_script: + - docker info - build_image: - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` + build_image: + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` 1. You can now use `docker` command (and **install** `docker-compose` if needed). @@ -107,83 +107,83 @@ In order to do that, follow the steps: 1. Register GitLab Runner from the command line to use `docker` and `privileged` mode: - ```bash - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:stable" \ - --docker-privileged - ``` - - The above command will register a new Runner to use the special - `docker:stable` image which is provided by Docker. **Notice that it's using - the `privileged` mode to start the build and service containers.** If you - want to use [docker-in-docker] mode, you always have to use `privileged = true` - in your Docker containers. - - DANGER: **Danger:** - By enabling `--docker-privileged`, you are effectively disabling all of - the security mechanisms of containers and exposing your host to privilege - escalation which can lead to container breakout. For more information, check - out the official Docker documentation on - [Runtime privilege and Linux capabilities][docker-cap]. - - The above command will create a `config.toml` entry similar to this: - - ```toml - [[runners]] - url = "https://gitlab.com/" - token = TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:stable" - privileged = true - disable_cache = false - volumes = ["/cache"] - [runners.cache] - Insecure = false - ``` + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:stable" \ + --docker-privileged + ``` + + The above command will register a new Runner to use the special + `docker:stable` image which is provided by Docker. **Notice that it's using + the `privileged` mode to start the build and service containers.** If you + want to use [docker-in-docker] mode, you always have to use `privileged = true` + in your Docker containers. + + DANGER: **Danger:** + By enabling `--docker-privileged`, you are effectively disabling all of + the security mechanisms of containers and exposing your host to privilege + escalation which can lead to container breakout. For more information, check + out the official Docker documentation on + [Runtime privilege and Linux capabilities][docker-cap]. + + The above command will create a `config.toml` entry similar to this: + + ```toml + [[runners]] + url = "https://gitlab.com/" + token = TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:stable" + privileged = true + disable_cache = false + volumes = ["/cache"] + [runners.cache] + Insecure = false + ``` 1. You can now use `docker` in the build script (note the inclusion of the `docker:dind` service): - ```yaml - image: docker:stable - - variables: - # When using dind service we need to instruct docker, to talk with the - # daemon started inside of the service. The daemon is available with - # a network connection instead of the default /var/run/docker.sock socket. - # - # The 'docker' hostname is the alias of the service container as described at - # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services - # - # Note that if you're using the Kubernetes executor, the variable should be set to - # tcp://localhost:2375/ because of how the Kubernetes executor connects services - # to the job container - # DOCKER_HOST: tcp://localhost:2375/ - # - # For non-Kubernetes executors, we use tcp://docker:2375/ - DOCKER_HOST: tcp://docker:2375/ - # When using dind, it's wise to use the overlayfs driver for - # improved performance. - DOCKER_DRIVER: overlay2 - - services: - - docker:dind - - before_script: - - docker info - - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` + ```yaml + image: docker:stable + + variables: + # When using dind service we need to instruct docker, to talk with the + # daemon started inside of the service. The daemon is available with + # a network connection instead of the default /var/run/docker.sock socket. + # + # The 'docker' hostname is the alias of the service container as described at + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services + # + # Note that if you're using the Kubernetes executor, the variable should be set to + # tcp://localhost:2375/ because of how the Kubernetes executor connects services + # to the job container + # DOCKER_HOST: tcp://localhost:2375/ + # + # For non-Kubernetes executors, we use tcp://docker:2375/ + DOCKER_HOST: tcp://docker:2375/ + # When using dind, it's wise to use the overlayfs driver for + # improved performance. + DOCKER_DRIVER: overlay2 + + services: + - docker:dind + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` Docker-in-Docker works well, and is the recommended configuration, but it is not without its own challenges: @@ -202,14 +202,14 @@ not without its own challenges: and use it as your mount point (for a more thorough explanation, check [issue #41227](https://gitlab.com/gitlab-org/gitlab-ce/issues/41227)): - ```yaml - variables: - MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt + ```yaml + variables: + MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt - script: - - mkdir -p "$MOUNT_POINT" - - docker run -v "$MOUNT_POINT:/mnt" my-docker-image - ``` + script: + - mkdir -p "$MOUNT_POINT" + - docker run -v "$MOUNT_POINT:/mnt" my-docker-image + ``` An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. @@ -230,54 +230,54 @@ In order to do that, follow the steps: 1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: - ```bash - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:stable" \ - --docker-volumes /var/run/docker.sock:/var/run/docker.sock - ``` - - The above command will register a new Runner to use the special - `docker:stable` image which is provided by Docker. **Notice that it's using - the Docker daemon of the Runner itself, and any containers spawned by docker - commands will be siblings of the Runner rather than children of the runner.** - This may have complications and limitations that are unsuitable for your workflow. - - The above command will create a `config.toml` entry similar to this: - - ```toml - [[runners]] - url = "https://gitlab.com/" - token = REGISTRATION_TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:stable" - privileged = false - disable_cache = false - volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] - [runners.cache] - Insecure = false - ``` + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:stable" \ + --docker-volumes /var/run/docker.sock:/var/run/docker.sock + ``` + + The above command will register a new Runner to use the special + `docker:stable` image which is provided by Docker. **Notice that it's using + the Docker daemon of the Runner itself, and any containers spawned by docker + commands will be siblings of the Runner rather than children of the runner.** + This may have complications and limitations that are unsuitable for your workflow. + + The above command will create a `config.toml` entry similar to this: + + ```toml + [[runners]] + url = "https://gitlab.com/" + token = REGISTRATION_TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:stable" + privileged = false + disable_cache = false + volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] + [runners.cache] + Insecure = false + ``` 1. You can now use `docker` in the build script (note that you don't need to include the `docker:dind` service as when using the Docker in Docker executor): - ```yaml - image: docker:stable + ```yaml + image: docker:stable - before_script: - - docker info + before_script: + - docker info - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` While the above method avoids using Docker in privileged mode, you should be aware of the following implications: @@ -293,9 +293,9 @@ aware of the following implications: work as expected since volume mounting is done in the context of the host machine, not the build container. For example: - ```sh - docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests - ``` + ```sh + docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests + ``` ## Making docker-in-docker builds faster with Docker layer caching @@ -366,23 +366,23 @@ which can be avoided if a different driver is used, for example `overlay2`. 1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: - ```sh - sudo lsmod | grep overlay - ``` + ```sh + sudo lsmod | grep overlay + ``` - If you see no result, then it isn't loaded. To load it use: + If you see no result, then it isn't loaded. To load it use: - ```sh - sudo modprobe overlay - ``` + ```sh + sudo modprobe overlay + ``` - If everything went fine, you need to make sure module is loaded on reboot. - On Ubuntu systems, this is done by editing `/etc/modules`. Just add the - following line into it: + If everything went fine, you need to make sure module is loaded on reboot. + On Ubuntu systems, this is done by editing `/etc/modules`. Just add the + following line into it: - ```text - overlay - ``` + ```text + overlay + ``` ### Use driver per project @@ -450,9 +450,9 @@ For all projects, mostly suitable for public ones: your Docker images and has read/write access to the Registry. This is ephemeral, so it's only valid for one job. You can use the following example as-is: - ```sh - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - ``` + ```sh + docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY + ``` For private and internal projects: @@ -465,9 +465,9 @@ For private and internal projects: Replace the `<username>` and `<access_token>` in the following example: - ```sh - docker login -u <username> -p <access_token> $CI_REGISTRY - ``` + ```sh + docker login -u <username> -p <access_token> $CI_REGISTRY + ``` - **Using the GitLab Deploy Token**: You can create and use a [special deploy token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token) @@ -475,9 +475,9 @@ For private and internal projects: Once created, you can use the special environment variables, and GitLab CI/CD will fill them in for you. You can use the following example as-is: - ```sh - docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY - ``` + ```sh + docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY + ``` ### Container Registry examples diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index f3896c5232c..d5056568dff 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -305,25 +305,25 @@ For example, the following two definitions are equal: 1. Using a string as an option to `image` and `services`: - ```yaml - image: "registry.example.com/my/image:latest" + ```yaml + image: "registry.example.com/my/image:latest" - services: - - postgresql:9.4 - - redis:latest - ``` + services: + - postgresql:9.4 + - redis:latest + ``` 1. Using a map as an option to `image` and `services`. The use of `image:name` is required: - ```yaml - image: - name: "registry.example.com/my/image:latest" + ```yaml + image: + name: "registry.example.com/my/image:latest" - services: - - name: postgresql:9.4 - - name: redis:latest - ``` + services: + - name: postgresql:9.4 + - name: redis:latest + ``` ### Available settings for `image` @@ -526,6 +526,7 @@ it's provided as an environment variable. This is because GitLab Runnner uses ** runtime. ### Using statically-defined credentials + There are two approaches that you can take in order to access a private registry. Both require setting the environment variable `DOCKER_AUTH_CONFIG` with appropriate authentication info. @@ -555,18 +556,18 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`: - **First way -** Do a `docker login` on your local machine: - ```bash - docker login registry.example.com:5000 --username my_username --password my_password - ``` + ```bash + docker login registry.example.com:5000 --username my_username --password my_password + ``` - Then copy the content of `~/.docker/config.json`. + Then copy the content of `~/.docker/config.json`. - If you don't need access to the registry from your computer, you - can do a `docker logout`: + If you don't need access to the registry from your computer, you + can do a `docker logout`: - ```bash - docker logout registry.example.com:5000 - ``` + ```bash + docker logout registry.example.com:5000 + ``` - **Second way -** In some setups, it's possible that Docker client will use the available system keystore to store the result of `docker @@ -575,12 +576,12 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`: `${username}:${password}` manually. Open a terminal and execute the following command: - ```bash - echo -n "my_username:my_password" | base64 + ```bash + echo -n "my_username:my_password" | base64 - # Example output to copy - bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= - ``` + # Example output to copy + bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= + ``` #### Configuring a job @@ -590,25 +591,25 @@ follow these steps: 1. Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: - ```json - { - "auths": { - "registry.example.com:5000": { - "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" - } - } - } - ``` + ```json + { + "auths": { + "registry.example.com:5000": { + "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" + } + } + } + ``` 1. You can now use any private image from `registry.example.com:5000` defined in `image` and/or `services` in your `.gitlab-ci.yml` file: - ```yaml - image: registry.example.com:5000/namespace/image:tag - ``` + ```yaml + image: registry.example.com:5000/namespace/image:tag + ``` - In the example above, GitLab Runner will look at `registry.example.com:5000` for the - image `namespace/image:tag`. + In the example above, GitLab Runner will look at `registry.example.com:5000` for the + image `namespace/image:tag`. You can add configuration for as many registries as you want, adding more registries to the `"auths"` hash as described above. @@ -637,10 +638,10 @@ To add `DOCKER_AUTH_CONFIG` to a Runner: 1. Modify the Runner's `config.toml` file as follows: - ```toml - [[runners]] - environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"registry.example.com:5000\":{\"auth\":\"bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=\"}}}"] - ``` + ```toml + [[runners]] + environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"registry.example.com:5000\":{\"auth\":\"bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=\"}}}"] + ``` 1. Restart the Runner service. @@ -662,16 +663,17 @@ To configure credentials store, follow these steps: Make sure helper program is available in GitLab Runner `$PATH`. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: + - Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the - Docker configuration file as the value: + Docker configuration file as the value: - ```json - { - "credsStore": "osxkeychain" - } - ``` + ```json + { + "credsStore": "osxkeychain" + } + ``` - Or, if you are running self-hosted Runners, add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`. GitLab Runner will read this config file @@ -693,17 +695,18 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th 1. Make sure `docker-credential-ecr-login` is available in GitLab Runner's `$PATH`. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: + - Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the - Docker configuration file as the value: + Docker configuration file as the value: - ```json - { - "credHelpers": { - "aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login" - } - } - ``` + ```json + { + "credHelpers": { + "aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login" + } + } + ``` - Or, if you are running self-hosted Runners, add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`. @@ -713,12 +716,12 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th 1. You can now use any private image from `aws_account_id.dkr.ecr.region.amazonaws.com` defined in `image` and/or `services` in your `.gitlab-ci.yml` file: - ```yaml - image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest - ``` + ```yaml + image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest + ``` - In the example above, GitLab Runner will look at `aws_account_id.dkr.ecr.region.amazonaws.com` for the - image `private/image:latest`. + In the example above, GitLab Runner will look at `aws_account_id.dkr.ecr.region.amazonaws.com` for the + image `private/image:latest`. You can add configuration for as many registries as you want, adding more registries to the `"credHelpers"` hash as described above. |