diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-03-20 18:19:03 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-03-20 18:19:03 +0300 |
commit | 14bd84b61276ef29b97d23642d698de769bacfd2 (patch) | |
tree | f9eba90140c1bd874211dea17750a0d422c04080 /doc/ci | |
parent | 891c388697b2db0d8ee0c8358a9bdbf6dc56d581 (diff) |
Add latest changes from gitlab-org/gitlab@15-10-stable-eev15.10.0-rc42
Diffstat (limited to 'doc/ci')
75 files changed, 536 insertions, 425 deletions
diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 1aa579b842a..d7c5b089116 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -139,13 +139,14 @@ You can override cache settings without overwriting the global cache by using `policy` for one job: ```yaml -cache: &global_cache - key: $CI_COMMIT_REF_SLUG - paths: - - node_modules/ - - public/ - - vendor/ - policy: pull-push +default: + cache: &global_cache + key: $CI_COMMIT_REF_SLUG + paths: + - node_modules/ + - public/ + - vendor/ + policy: pull-push job: cache: @@ -570,7 +571,7 @@ You can clear the cache in the GitLab UI: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. -1. In the upper right, select **Clear runner caches**. +1. In the upper-right corner, select **Clear runner caches**. On the next commit, your CI/CD jobs use a new cache. diff --git a/doc/ci/chatops/img/gitlab-chatops-icon-small.png b/doc/ci/chatops/img/gitlab-chatops-icon-small.png Binary files differdeleted file mode 100644 index 71cc5dba5cf..00000000000 --- a/doc/ci/chatops/img/gitlab-chatops-icon-small.png +++ /dev/null diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md index f0efb5fc884..bb3c8d537f3 100644 --- a/doc/ci/chatops/index.md +++ b/doc/ci/chatops/index.md @@ -9,62 +9,70 @@ type: index, concepts, howto > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4466) in GitLab Ultimate 10.6. > - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24780) to GitLab Free in 11.9. -> - `CHAT_USER_ID` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/341798) in GitLab 14.4. +> - `CHAT_USER_ID` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/341798) in GitLab 14.4. -GitLab ChatOps provides a method to interact with CI/CD jobs through chat services -like Slack. Many organizations' discussion, collaboration, and troubleshooting takes -place in chat services. Having a method to run CI/CD jobs with output -posted back to the channel can significantly augment your team's workflow. +Use GitLab ChatOps to interact with CI/CD jobs through chat services +like Slack. -## How GitLab ChatOps works +Many organizations use chat services to collaborate, troubleshoot, and plan work. With ChatOps, +you can discuss work with your team, run CI/CD jobs, and view job output, all from the same +application. -GitLab ChatOps is built upon [GitLab CI/CD](../index.md) and -[Slack Slash Commands](../../user/project/integrations/slack_slash_commands.md). -ChatOps provides a `run` action for [slash commands](../../integration/slash_commands.md) -with the following arguments: +## ChatOps workflow and CI/CD configuration -- A `<job name>` to execute. -- The `<job arguments>`. +ChatOps looks for the specified job in the +[`.gitlab-ci.yml`](../yaml/index.md) on the project's default +branch. If the job is found, ChatOps creates a pipeline that contains +only the specified job. If you set `when: manual`, ChatOps creates the +pipeline, but the job doesn't start automatically. + +A job run with ChatOps has the same functionality as a job run from +GitLab. The job can use existing [CI/CD variables](../variables/index.md#predefined-cicd-variables) like +`GITLAB_USER_ID` to perform additional rights validation, but these +variables can be [overridden](../variables/index.md#cicd-variable-precedence). + +You should set [`rules`](../yaml/index.md#rules) so the job does not +run as part of the standard CI/CD pipeline. ChatOps passes the following [CI/CD variables](../variables/index.md#predefined-cicd-variables) to the job: -- `CHAT_INPUT` contains any additional arguments. -- `CHAT_CHANNEL` is set to the name of channel the action was triggered in. -- `CHAT_USER_ID` is set to the chat service's user ID of the user who triggered the slash command. +- `CHAT_INPUT` - The arguments passed to `/project-name run`. +- `CHAT_CHANNEL` - The name of the chat channel the job is run from. +- `CHAT_USER_ID` - The chat service ID of the user who runs the job. -When executed, ChatOps looks up the specified job name and attempts to match it -to a corresponding job in [`.gitlab-ci.yml`](../yaml/index.md). If a matching job -is found on the default branch, a pipeline containing only that job is scheduled. After the -job completes: +When the job runs: -- If the job completes in *less than 30 minutes*, the ChatOps sends the job's output to Slack. -- If the job completes in *more than 30 minutes*, the job must use the +- If the job completes in less than 30 minutes, ChatOps sends the job output to the chat channel. +- If the job completes in more than 30 minutes, you must use a method like the [Slack API](https://api.slack.com/) to send data to the channel. -To use the `run` command, you must have at least the -Developer role. -If a job shouldn't be able to be triggered from chat, you can set the job to `except: [chat]`. +## Run a CI/CD job + +You can run a CI/CD job from chat with the `/project-name run` +[slash command](../../integration/slash_commands.md). + +Prerequisites: + +- You must have at least the Developer role for the project. + +To run a CI/CD job: + +- In the chat client, enter `/project-name run <job name> <arguments>`. -## Best practices for ChatOps CI jobs +ChatOps schedules a pipeline that contains only the specified job. -Since ChatOps is built upon GitLab CI/CD, the job has all the same features and -functions available. Consider these best practices when creating ChatOps jobs: +### Exclude a job from ChatOps -- GitLab strongly recommends you set [`rules`](../yaml/index.md#rules) so the job does not run as part - of the standard CI pipeline. -- If the job is set to `when: manual`, ChatOps creates the pipeline, but the job waits to be started. -- ChatOps provides limited support for access control. If the user triggering the - slash command has at least the Developer role - in the project, the job runs. The job itself can use existing - [CI/CD variables](../variables/index.md#predefined-cicd-variables) like - `GITLAB_USER_ID` to perform additional rights validation, but - these variables can be [overridden](../variables/index.md#cicd-variable-precedence). +To prevent a job from being run from chat: -### Controlling the ChatOps reply +- In `.gitlab-ci.yml`, set the job to `except: [chat]`. -The output for jobs with a single command is sent to the channel as a reply. For -example, the chat reply of the following job is `Hello World` in the channel: +## Customize the ChatOps reply + +ChatOps sends the output for a job with a single command to the +channel as a reply. For example, when the following job runs, +the chat reply is `Hello world`: ```yaml stages: @@ -78,13 +86,12 @@ hello-world: - echo "Hello World" ``` -Jobs that contain multiple commands (or `before_script`) return additional -content in the chat reply. In these cases, both the commands and their output are -included, with the commands wrapped in ANSI color codes. +If the job contains multiple commands, or if `before_script` is set, ChatOps sends the commands +and their output to the channel. The commands are wrapped in ANSI color codes. -To selectively reply with the output of one command, its output must be bounded by -the `chat_reply` section. For example, the following job lists the files in the -current directory: +To selectively reply with the output of one command, place the output +in a `chat_reply` section. For example, the following job lists the +files in the current directory: ```yaml stages: @@ -99,28 +106,8 @@ ls: - echo -e "section_start:$( date +%s ):chat_reply\r\033[0K\n$( ls -la )\nsection_end:$( date +%s ):chat_reply\r\033[0K" ``` -## GitLab ChatOps examples - -The GitLab.com team created a repository of [common ChatOps scripts](https://gitlab.com/gitlab-com/chatops) -they use to interact with our Production instance of GitLab. Administrators of -other GitLab instances may find them useful. They can serve as inspiration for ChatOps -scripts you can write to interact with your own applications. - -## GitLab ChatOps icon - -The [official GitLab ChatOps icon](img/gitlab-chatops-icon.png) is available for download. -You can find and download the official GitLab ChatOps icon here. - -![GitLab ChatOps bot icon](img/gitlab-chatops-icon-small.png) - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +## Related topics -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +- Download the [official GitLab ChatOps icon](img/gitlab-chatops-icon.png). +- The GitLab team maintains a [repository of common ChatOps scripts](https://gitlab.com/gitlab-com/chatops) + they use to interact with GitLab.com. diff --git a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md index 920e7cca2cb..71210467018 100644 --- a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md +++ b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/cloud_deployment/heroku.md b/doc/ci/cloud_deployment/heroku.md index 541ba89dca8..7c15bf5b8cd 100644 --- a/doc/ci/cloud_deployment/heroku.md +++ b/doc/ci/cloud_deployment/heroku.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index a8826a6fdb5..79684d9d627 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: howto --- diff --git a/doc/ci/cloud_services/aws/index.md b/doc/ci/cloud_services/aws/index.md index 484a159cd2b..733105e9ddf 100644 --- a/doc/ci/cloud_services/aws/index.md +++ b/doc/ci/cloud_services/aws/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/cloud_services/azure/index.md b/doc/ci/cloud_services/azure/index.md index 321f9849632..c99e30e8bac 100644 --- a/doc/ci/cloud_services/azure/index.md +++ b/doc/ci/cloud_services/azure/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/cloud_services/google_cloud/index.md b/doc/ci/cloud_services/google_cloud/index.md index 516a2d05cd1..bcb2681403a 100644 --- a/doc/ci/cloud_services/google_cloud/index.md +++ b/doc/ci/cloud_services/google_cloud/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md index 9304e3562d9..88e7d88e22a 100644 --- a/doc/ci/cloud_services/index.md +++ b/doc/ci/cloud_services/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index a8c192dc944..674f9b6035a 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -9,12 +9,12 @@ type: concepts, howto You can use GitLab CI/CD with Docker to create Docker images. For example, you can create a Docker image of your application, -test it, and publish it to a container registry. +test it, and push it to a container registry. To run Docker commands in your CI/CD jobs, you must configure -GitLab Runner to support `docker` commands. +GitLab Runner to support `docker` commands. This method requires `privileged` mode. -If you want to build Docker images without enabling privileged mode on the runner, +If you want to build Docker images without enabling `privileged` mode on the runner, you can use a [Docker alternative](#docker-alternatives). ## Enable Docker commands in your CI/CD jobs @@ -58,7 +58,7 @@ the Docker commands, but needs permission to do so. sudo -u gitlab-runner -H docker info ``` -1. In GitLab, to verify that everything works, add `docker info` to `.gitlab-ci.yml`: +1. In GitLab, add `docker info` to `.gitlab-ci.yml` to verify that Docker is working: ```yaml before_script: @@ -70,10 +70,10 @@ the Docker commands, but needs permission to do so. - docker run my-docker-image /script/to/run/tests ``` -You can now use `docker` commands (and install `docker-compose` if needed). +You can now use `docker` commands (and install Docker Compose if needed). -When you add `gitlab-runner` to the `docker` group, you are effectively granting `gitlab-runner` full root permissions. -For more information, see the [security of the `docker` group](https://blog.zopyx.com/on-docker-security-docker-group-considered-harmful/). +When you add `gitlab-runner` to the `docker` group, you effectively grant `gitlab-runner` full root permissions. +For more information, see [security of the `docker` group](https://blog.zopyx.com/on-docker-security-docker-group-considered-harmful/). ### Use Docker-in-Docker @@ -83,15 +83,15 @@ For more information, see the [security of the `docker` group](https://blog.zopy - The executor uses a [container image of Docker](https://hub.docker.com/_/docker/), provided by Docker, to run your CI/CD jobs. -The Docker image has all of the `docker` tools installed and can run +The Docker image includes all of the `docker` tools and can run the job script in context of the image in privileged mode. -We recommend you use Docker-in-Docker with TLS enabled, +You should use Docker-in-Docker with TLS enabled, which is supported by [GitLab.com shared runners](../runners/index.md). -You should always specify a specific version of the image, like `docker:20.10.16`. +You should always pin a specific version of the image, like `docker:20.10.16`. If you use a tag like `docker:stable`, you have no control over which version is used. -Unpredictable behavior can result, especially when new versions are released. +This can cause incompatibility problems when new versions are released. #### Use the Docker executor with Docker-in-Docker @@ -101,14 +101,11 @@ You can use the Docker executor to run jobs in a Docker container. > Introduced in GitLab Runner 11.11. -The Docker daemon supports connections over TLS. In Docker 19.03.12 and later, -TLS is the default. +The Docker daemon supports connections over TLS. TLS is the default in Docker 19.03.12 and later. WARNING: -This task enables `--docker-privileged`. When you do this, you are effectively disabling all of -the security mechanisms of containers and exposing your host to privilege -escalation. Doing this can lead to container breakout. For more information, -see the official Docker documentation about +This task enables `--docker-privileged`, which effectively disables the container's security mechanisms and exposes your host to privilege +escalation. This action can cause container breakout. For more information, see [runtime privilege and Linux capabilities](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities). To use Docker-in-Docker with TLS enabled: @@ -134,10 +131,9 @@ To use Docker-in-Docker with TLS enabled: you must always use `privileged = true` in your Docker containers. - This command mounts `/certs/client` for the service and build container, which is needed for the Docker client to use the - certificates in that directory. For more information on how - Docker with TLS works, see <https://hub.docker.com/_/docker/#tls>. + certificates in that directory. For more information, see [the Docker image documentation](https://hub.docker.com/_/docker/). - The previous command creates a `config.toml` entry similar to this: + The previous command creates a `config.toml` entry similar to the following example: ```toml [[runners]] @@ -155,7 +151,7 @@ To use Docker-in-Docker with TLS enabled: [runners.cache.gcs] ``` -1. You can now use `docker` in the job script. Note the inclusion of the +1. You can now use `docker` in the job script. You should include the `docker:20.10.16-dind` service: ```yaml @@ -193,7 +189,7 @@ To use Docker-in-Docker with TLS enabled: ##### Docker-in-Docker with TLS disabled in the Docker executor -Sometimes you might have legitimate reasons to disable TLS. +Sometimes there are legitimate reasons to disable TLS. For example, you have no control over the GitLab Runner configuration that you are using. @@ -215,7 +211,7 @@ Assuming that the runner's `config.toml` is similar to: [runners.cache.gcs] ``` -You can now use `docker` in the job script. Note the inclusion of the +You can now use `docker` in the job script. You should include the `docker:20.10.16-dind` service: ```yaml @@ -254,7 +250,7 @@ build: #### Use the Kubernetes executor with Docker-in-Docker -You can use the Kubernetes executor to run jobs in a Docker container. +You can use the [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html) to run jobs in a Docker container. ##### Docker-in-Docker with TLS enabled in Kubernetes @@ -280,7 +276,7 @@ To use Docker-in-Docker with TLS enabled in Kubernetes: medium = "Memory" ``` -1. You can now use `docker` in the job script. Note the inclusion of the +1. You can now use `docker` in the job script. You should include the `docker:20.10.16-dind` service: ```yaml @@ -324,25 +320,23 @@ To use Docker-in-Docker with TLS enabled in Kubernetes: - docker run my-docker-image /script/to/run/tests ``` -#### Limitations of Docker-in-Docker +#### Known issues with Docker-in-Docker -Docker-in-Docker is the recommended configuration, but is -not without its own challenges: +Docker-in-Docker is the recommended configuration, but you should be aware of the following issues: - **The `docker-compose` command**: This command is not available in this configuration by default. - To use `docker-compose` in your job scripts, follow the `docker-compose` + To use `docker-compose` in your job scripts, follow the Docker Compose [installation instructions](https://docs.docker.com/compose/install/). -- **Cache**: Each job runs in a new environment. Concurrent jobs work fine, - because every build gets its own instance of Docker engine and they don't conflict with each other. - However, jobs can be slower because there's no caching of layers. +- **Cache**: Each job runs in a new environment. Because every build gets its own instance of the Docker engine, concurrent jobs do not cause conflicts. + However, jobs can be slower because there's no caching of layers. See [Docker layer caching](#make-docker-in-docker-builds-faster-with-docker-layer-caching). - **Storage drivers**: By default, earlier versions of Docker use the `vfs` storage driver, which copies the file system for each job. Docker 17.09 and later use `--storage-driver overlay2`, which is the recommended storage driver. See [Using the OverlayFS driver](#use-the-overlayfs-driver) for details. -- **Root file system**: Because the `docker:20.10.16-dind` container and the runner container don't share their +- **Root file system**: Because the `docker:20.10.16-dind` container and the runner container do not share their root file system, you can use the job's working directory as a mount point for child containers. For example, if you have files you want to share with a - child container, you might create a subdirectory under `/builds/$CI_PROJECT_PATH` - and use it as your mount point. For a more detailed explanation, view + child container, you could create a subdirectory under `/builds/$CI_PROJECT_PATH` + and use it as your mount point. For a more detailed explanation, see [issue #41227](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41227). ```yaml @@ -362,16 +356,16 @@ NOTE: If you bind the Docker socket and you are [using GitLab Runner 11.11 or later](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), you can no longer use `docker:20.10.16-dind` as a service. Volume bindings -are done to the services as well, making these incompatible. +also affect services, making them incompatible. #### Use the Docker executor with Docker socket binding To make Docker available in the context of the image, you need to mount `/var/run/docker.sock` into the launched containers. To do this with the Docker -executor, you need to add `"/var/run/docker.sock:/var/run/docker.sock"` to the +executor, add `"/var/run/docker.sock:/var/run/docker.sock"` to the [Volumes in the `[runners.docker]` section](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#volumes-in-the-runnersdocker-section). -Your configuration should look something like this: +Your configuration should look similar to this example: ```toml [[runners]] @@ -388,7 +382,7 @@ Your configuration should look something like this: Insecure = false ``` -You can also do this while registering your runner by providing the following options: +To mount `/var/run/docker.sock` while registering your runner, include the following options: ```shell sudo gitlab-runner register -n \ @@ -402,10 +396,10 @@ sudo gitlab-runner register -n \ ##### Enable registry mirror for `docker:dind` service -When the Docker daemon starts inside of the service container, it uses -the default configuration. You may want to configure a +When the Docker daemon starts inside the service container, it uses +the default configuration. You might want to configure a [registry mirror](https://docs.docker.com/registry/recipes/mirror/) for -performance improvements and to ensure you don't reach Docker Hub rate limits. +performance improvements and to ensure you do not exceed Docker Hub rate limits. ###### The service in the `.gitlab-ci.yml` file @@ -474,9 +468,9 @@ content: ``` Update the `config.toml` file to mount the file to -`/etc/docker/daemon.json`. This would mount the file for **every** -container that is created by GitLab Runner. The configuration is -picked up by the `dind` service. +`/etc/docker/daemon.json`. This mounts the file for **every** +container created by GitLab Runner. The configuration is +detected by the `dind` service. ```toml [[runners]] @@ -516,13 +510,12 @@ kubectl create configmap docker-daemon --namespace gitlab-runner --from-file /tm ``` NOTE: -Make sure to use the namespace that the Kubernetes executor for GitLab Runner uses -to create job pods in. +You must use the namespace that the Kubernetes executor for GitLab Runner uses to create job pods. After the ConfigMap is created, you can update the `config.toml` file to mount the file to `/etc/docker/daemon.json`. This update -mounts the file for **every** container that is created by GitLab Runner. -The configuration is picked up by the `dind` service. +mounts the file for **every** container created by GitLab Runner. +The `dind` service detects this configuration. ```toml [[runners]] @@ -537,22 +530,21 @@ The configuration is picked up by the `dind` service. sub_path = "daemon.json" ``` -##### Limitations of Docker socket binding +##### Known issues with Docker socket binding When you use Docker socket binding, you avoid running Docker in privileged mode. However, the implications of this method are: -- By sharing the Docker daemon, you are effectively disabling all - the security mechanisms of containers and exposing your host to privilege - escalation, which can lead to container breakout. For example, if a project - ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner +- By sharing the Docker daemon, you effectively disable all + the container's security mechanisms and expose your host to privilege + escalation. This can cause container breakout. For example, if a project + ran `docker rm -f $(docker ps -a -q)`, it would remove the GitLab Runner containers. -- Concurrent jobs may not work; if your tests - create containers with specific names, they may conflict with each other. -- Any containers spawned by Docker commands are siblings of the runner rather - than children of the runner. This may have complications and limitations that - are unsuitable for your workflow. -- Sharing files and directories from the source repository into containers may not +- Concurrent jobs might not work. If your tests + create containers with specific names, they might conflict with each other. +- Any containers created by Docker commands are siblings of the runner, rather + than children of the runner. This might cause complications for your workflow. +- Sharing files and directories from the source repository into containers might not work as expected. Volume mounting is done in the context of the host machine, not the build container. For example: @@ -560,8 +552,8 @@ the implications of this method are: docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests ``` -You don't need to include the `docker:20.10.16-dind` service, like you do when -you're using the Docker-in-Docker executor: +You do not need to include the `docker:20.10.16-dind` service, like you do when +you use the Docker-in-Docker executor: ```yaml image: docker:20.10.16 @@ -580,7 +572,7 @@ build: When you use Docker-in-Docker, the [standard authentication methods](using_docker_images.md#access-an-image-from-a-private-container-registry) -don't work because a fresh Docker daemon is started with the service. +do not work, because a fresh Docker daemon is started with the service. ### Option 1: Run `docker login` @@ -612,14 +604,14 @@ empty or remove it. If you are an administrator for GitLab Runner, you can mount a file with the authentication configuration to `~/.docker/config.json`. -Then every job that the runner picks up is authenticated already. If you +Then every job that the runner picks up is already authenticated. If you are using the official `docker:20.10.16` image, the home directory is under `/root`. If you mount the configuration file, any `docker` command that modifies the `~/.docker/config.json` fails. For example, `docker login` fails, because the file is mounted as read-only. Do not change it from -read-only, because problems occur. +read-only, because this causes problems. Here is an example of `/opt/.docker/config.json` that follows the [`DOCKER_AUTH_CONFIG`](using_docker_images.md#determine-your-docker_auth_config-data) @@ -685,7 +677,7 @@ If you already have defined, you can use the variable and save it in `~/.docker/config.json`. -There are multiple ways to define this authentication: +You can define this authentication in several ways: - In [`pre_build_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) in the runner configuration file. @@ -718,24 +710,23 @@ build: When using Docker-in-Docker, Docker downloads all layers of your image every time you create a build. Recent versions of Docker (Docker 1.13 and later) can -use a pre-existing image as a cache during the `docker build` step. This considerably -speeds up the build process. +use a pre-existing image as a cache during the `docker build` step. This significantly +accelerates the build process. ### How Docker caching works -When running `docker build`, each command in `Dockerfile` results in a layer. -These layers are kept around as a cache and can be reused if there haven't been -any changes. Change in one layer causes all subsequent layers to be recreated. +When running `docker build`, each command in `Dockerfile` creates a layer. +These layers are retained as a cache and can be reused if there have been no changes. Change in one layer causes the recreation of all subsequent layers. -You can specify a tagged image to be used as a cache source for the `docker build` -command by using the `--cache-from` argument. Multiple images can be specified +To specify a tagged image to be used as a cache source for the `docker build` +command, use the `--cache-from` argument. Multiple images can be specified as a cache source by using multiple `--cache-from` arguments. Any image that's used -with the `--cache-from` argument must first be pulled +with the `--cache-from` argument must be pulled (using `docker pull`) before it can be used as a cache source. ### Docker caching example -Here's a `.gitlab-ci.yml` file that shows how to use Docker caching: +This example `.gitlab-ci.yml` file shows how to use Docker caching: ```yaml image: docker:20.10.16 @@ -768,35 +759,33 @@ In the `script` section for the `build` stage: cache (see the `--cache-from $CI_REGISTRY_IMAGE:latest` argument) if available, and tags it. 1. The last two commands push the tagged Docker images to the container registry - so that they may also be used as cache for subsequent builds. + so that they can also be used as cache for subsequent builds. ## Use the OverlayFS driver NOTE: The shared runners on GitLab.com use the `overlay2` driver by default. -By default, when using `docker:dind`, Docker uses the `vfs` storage driver which -copies the file system on every run. This is a disk-intensive operation -which can be avoided if a different driver is used, for example `overlay2`. +By default, when using `docker:dind`, Docker uses the `vfs` storage driver, which +copies the file system on every run. You can avoid this disk-intensive operation by using a different driver, for example `overlay2`. ### Requirements -1. Make sure a recent kernel is used, preferably `>= 4.2`. +1. Ensure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: ```shell sudo lsmod | grep overlay ``` - If you see no result, then it isn't loaded. To load it use: + If you see no result, then the module is not loaded. To load the module, use: ```shell sudo modprobe overlay ``` - If everything went fine, you need to make sure module is loaded on reboot. - On Ubuntu systems, this is done by editing `/etc/modules`. Just add the - following line into it: + If the module loaded, you must make sure the module loads on reboot. + On Ubuntu systems, do this by adding the following line to `/etc/modules`: ```plaintext overlay @@ -823,7 +812,7 @@ environment variable in the environment = ["DOCKER_DRIVER=overlay2"] ``` -If you're running multiple runners, you have to modify all configuration files. +If you're running multiple runners, you must modify all configuration files. Read more about the [runner configuration](https://docs.gitlab.com/runner/configuration/) and [using the OverlayFS storage driver](https://docs.docker.com/storage/storagedriver/overlayfs-driver/). @@ -833,8 +822,8 @@ and [using the OverlayFS storage driver](https://docs.docker.com/storage/storage To build Docker images without enabling privileged mode on the runner, you can use one of these alternatives: -- [`kaniko`](using_kaniko.md). -- [`buildah`](https://github.com/containers/buildah). +- [`kaniko`](using_kaniko.md) +- [`buildah`](https://github.com/containers/buildah) For example, with `buildah`: @@ -868,7 +857,7 @@ build: ## Use the GitLab Container Registry -After you've built a Docker image, you can push it up to the built-in +After you've built a Docker image, you can push it to the [GitLab Container Registry](../../user/packages/container_registry/build_and_push_images.md#use-gitlab-cicd). ## Troubleshooting @@ -879,18 +868,18 @@ This is a common error when you are using [Docker-in-Docker](#use-docker-in-docker) v19.03 or later. -This issue occurs because Docker starts on TLS automatically. +This error occurs because Docker starts on TLS automatically. -- If this is your first time setting it up, read +- If this is your first time setting it up, see [use the Docker executor with the Docker image](#use-docker-in-docker). -- If you are upgrading from v18.09 or earlier, read our +- If you are upgrading from v18.09 or earlier, see the [upgrade guide](https://about.gitlab.com/blog/2019/07/31/docker-in-docker-with-docker-19-dot-03/). -This error can also occur with the [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html#using-dockerdind) when attempts are made to access the Docker-in-Docker service before it has had time to fully start up. For a more detailed explanation, see [this issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27215). +This error can also occur with the [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html#using-dockerdind) when attempts are made to access the Docker-in-Docker service before it has fully started up. For a more detailed explanation, see [issue 27215](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27215). ### Docker `no such host` error -You may get an error that says +You might get an error that says `docker: error during connect: Post https://docker:2376/v1.40/containers/create: dial tcp: lookup docker on x.x.x.x:53: no such host`. This issue can occur when the service's image name diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index 089ecefb373..c4f685b18ec 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: Require approvals prior to deploying to a Protected Environment --- diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index cf82238564e..327118c8a86 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md index 479b783202d..2ee0dc2ee11 100644 --- a/doc/ci/environments/environments_dashboard.md +++ b/doc/ci/environments/environments_dashboard.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/ci/environments/external_deployment_tools.md b/doc/ci/environments/external_deployment_tools.md index ff3172f0e02..89a1f8565a9 100644 --- a/doc/ci/environments/external_deployment_tools.md +++ b/doc/ci/environments/external_deployment_tools.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- @@ -42,45 +42,45 @@ Here is an example setup that creates a `success` deployment record in GitLab wh 1. Create a new webhook. You can save the following manifest file and apply it by `kubectl apply -n argocd -f <manifiest-file-path>`: - ```yaml - apiVersion: v1 - kind: ConfigMap - metadata: - name: argocd-notifications-cm - data: - trigger.on-deployed: | - - description: Application is synced and healthy. Triggered once per commit. - oncePer: app.status.sync.revision - send: - - gitlab-deployment-status - when: app.status.operationState.phase in ['Succeeded'] and app.status.health.status == 'Healthy' - template.gitlab-deployment-status: | - webhook: - gitlab: - method: POST - path: /projects/<your-project-id>/deployments - body: | - { - "status": "success", - "environment": "production", - "sha": "{{.app.status.operationState.operation.sync.revision}}", - "ref": "main", - "tag": "false" - } - service.webhook.gitlab: | - url: https://gitlab.com/api/v4 - headers: - - name: PRIVATE-TOKEN - value: <your-access-token> - - name: Content-type - value: application/json - ``` + ```yaml + apiVersion: v1 + kind: ConfigMap + metadata: + name: argocd-notifications-cm + data: + trigger.on-deployed: | + - description: Application is synced and healthy. Triggered once per commit. + oncePer: app.status.sync.revision + send: + - gitlab-deployment-status + when: app.status.operationState.phase in ['Succeeded'] and app.status.health.status == 'Healthy' + template.gitlab-deployment-status: | + webhook: + gitlab: + method: POST + path: /projects/<your-project-id>/deployments + body: | + { + "status": "success", + "environment": "production", + "sha": "{{.app.status.operationState.operation.sync.revision}}", + "ref": "main", + "tag": "false" + } + service.webhook.gitlab: | + url: https://gitlab.com/api/v4 + headers: + - name: PRIVATE-TOKEN + value: <your-access-token> + - name: Content-type + value: application/json + ``` 1. Create a new subscription in your application: - ```shell - kubectl patch app <your-app-name> -n argocd -p '{"metadata": {"annotations": {"notifications.argoproj.io/subscribe.on-deployed.gitlab":""}}}' --type merge - ``` + ```shell + kubectl patch app <your-app-name> -n argocd -p '{"metadata": {"annotations": {"notifications.argoproj.io/subscribe.on-deployed.gitlab":""}}}' --type merge + ``` NOTE: If a deployment wasn't created as expected, you can troubleshoot with [`argocd-notifications` tool](https://argocd-notifications.readthedocs.io/en/stable/troubleshooting/). diff --git a/doc/ci/environments/img/environments_project_home.png b/doc/ci/environments/img/environments_project_home.png Binary files differindex 36b33e260f0..10d9de4f415 100644 --- a/doc/ci/environments/img/environments_project_home.png +++ b/doc/ci/environments/img/environments_project_home.png diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md index 10cda68c4b5..8df2d8ead6c 100644 --- a/doc/ci/environments/incremental_rollouts.md +++ b/doc/ci/environments/incremental_rollouts.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 60450692794..c9b212c2915 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference disqus_identifier: 'https://docs.gitlab.com/ee/ci/environments.html' @@ -159,6 +159,19 @@ deploy_review_app: - main ``` +### Rename an environment + +> - Renaming an environment by using the UI was [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68550) in GitLab 14.3. +> - Renaming an environment by using the API was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/338897) in GitLab 15.9 and is planned for removal in GitLab 16.0. + +You cannot rename an environment by using the UI, and the API method was deprecated in GitLab 15.9. + +To achieve the same result as renaming an environment: + +1. [Stop the existing environment](#stop-an-environment-through-the-ui). +1. [Delete the existing environment](#delete-an-environment). +1. [Create a new environment](#create-a-static-environment) with the desired name. + ## Deployment tier of environments > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300741) in GitLab 13.10. @@ -207,7 +220,7 @@ The `when: manual` action: You can find the play button in the pipelines, environments, deployments, and jobs views. -## Configure Kubernetes deployments (DEPRECATED) +## Configure Kubernetes deployments (deprecated) > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. @@ -708,7 +721,7 @@ You can view a deployment's expiration date in the GitLab UI. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the name of the deployment. -In the upper left, next to the environment name, the expiration date is displayed. +In the upper-left corner, next to the environment name, the expiration date is displayed. #### Override a deployment's scheduled stop time @@ -717,7 +730,7 @@ You can manually override a deployment's expiration date. 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the deployment name. -1. in the upper right, select the thumbtack (**{thumbtack}**). +1. in the upper-right corner, select the thumbtack (**{thumbtack}**). ![Environment auto stop](img/environment_auto_stop_v13_10.png) @@ -866,7 +879,7 @@ It may take a minute or two for data to appear after initial deployment. Metric charts can be embedded in GitLab Flavored Markdown. See [Embedding Metrics in GitLab Flavored Markdown](../../operations/metrics/embed.md) for more details. -### Web terminals (DEPRECATED) +### Web terminals (deprecated) > [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. @@ -991,19 +1004,6 @@ like [Review Apps](../review_apps/index.md) (`review/*`). The most specific spec takes precedence over the other wildcard matching. In this case, the `review/feature-1` spec takes precedence over `review/*` and `*` specs. -### Rename an environment - -> Renaming environments through the UI was [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68550) in GitLab 14.3. Renaming environments through the API was deprecated and [is planned to be removed](https://gitlab.com/gitlab-org/gitlab/-/issues/338897) in GitLab 15.0. - -Renaming an environment through the UI is not possible. -Instead, you need to delete the old environment and create a new one: - -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. -1. Find the environment and stop it. -1. Delete the environment. -1. Create a new environment with your preferred name. - ## Related topics - [Use GitLab CI to deploy to multiple environments (blog post)](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/) diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index fc49be798ec..c81a5ab1378 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index 7bb14c4c900..6295c131fb9 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 533b6519d9a..81e70e5cf82 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/deployment/index.md b/doc/ci/examples/deployment/index.md index 67b4ec6b279..74b4581af99 100644 --- a/doc/ci/examples/deployment/index.md +++ b/doc/ci/examples/deployment/index.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 7b5690e2d3d..3385aca1ef2 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments author: Vincent Tunru author_gitlab: Vinnl diff --git a/doc/ci/examples/index.md b/doc/ci/examples/index.md index 6738c55b6fa..b05c872d624 100644 --- a/doc/ci/examples/index.md +++ b/doc/ci/examples/index.md @@ -76,7 +76,7 @@ choose one of these templates: - [dotNET Core (`dotNET-Core.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET-Core.gitlab-ci.yml) - [Elixir (`Elixir.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Elixir.gitlab-ci.yml) - [Flutter (`Flutter.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Flutter.gitlab-ci.yml) -- [Golang (`Go.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Go.gitlab-ci.yml) +- [Go (`Go.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Go.gitlab-ci.yml) - [Gradle (`Gradle.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Gradle.gitlab-ci.yml) - [Grails (`Grails.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Grails.gitlab-ci.yml) - [iOS with fastlane (`iOS-Fastlane.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/iOS-Fastlane.gitlab-ci.yml) diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 062bd602c29..d56f14617db 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -126,7 +126,7 @@ We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our ![variables page](img/variables_page.png) -We also need to add the public key to **Project** > **Settings** > **Repository** as a [Deploy Key](../../../user/project/deploy_keys/index.md), which gives us the ability to access our repository from the server through [SSH protocol](../../../gitlab-basics/command-line-commands.md#start-working-on-your-project). +We also need to add the public key to **Project** > **Settings** > **Repository** as a [Deploy Key](../../../user/project/deploy_keys/index.md), which gives us the ability to access our repository from the server through the SSH protocol. ```shell # As the deployer user on the server diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index 9f299448d52..0b4b2054610 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -90,7 +90,7 @@ As part of publishing a package, semantic-release increases the version number i <!-- markdownlint-disable MD044 --> -1. On the top bar, in the upper right, select your avatar. +1. On the top bar, in the upper-right corner, select your avatar. 1. On the left sidebar, select **Access Tokens**. 1. In the **Token name** box, enter a token name. 1. Under **select scopes**, select the **api** checkbox. diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index e62c38fc1ec..bf07af5e761 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Execution +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: "An overview of Continuous Integration, Continuous Delivery, and Continuous Deployment, as well as an introduction to GitLab CI/CD." type: concepts diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index d9cfbdf124e..db916d8d125 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Execution +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -65,7 +65,9 @@ tries to steal tokens from other jobs. You can control what projects a CI/CD job token can access to increase the job token's security. A job token might give extra permissions that aren't necessary -to access specific private resources. +to access specific private resources. The job token scope only controls access +to private projects. If an accessed project is public or internal, token scoping does +not apply. If a job token is leaked, it could potentially be used to access private data to the job token's user. By limiting the job token access scope, private data cannot @@ -78,14 +80,15 @@ see [epic 3559](https://gitlab.com/groups/gitlab-org/-/epics/3559). > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346298/) in GitLab 15.9. [Deployed behind the `:inbound_ci_scoped_job_token` feature flag](../../user/feature_flags.md), enabled by default. -Control your project's job token scope by creating an **inbound** allowlist of projects which can -access your project through its `CI_JOB_TOKEN`. +Create an **inbound** allowlist of projects which can access your project through +their `CI_JOB_TOKEN`. -For example, you can add project `B` to the inbound allowlist for project `A`. Jobs -in the pipeline for "allowed project" `B` can now use the CI/CD job token to authenticate -API calls to access project `A`. +For example, project `A` can add project `B` to the inbound allowlist. CI/CD jobs +in project `B` (the "allowed project") can now use their CI/CD job token to +authenticate API calls to access project `A`. If project `A` is public or internal, +the project can be accessed by project `B` without adding it to the inbound allowlist. -By default the allowlist includes your current project. +By default the inbound allowlist of any project only includes itself. It is a security risk to disable this feature, so project maintainers or owners should keep this setting enabled at all times. Add projects to the allowlist only when cross-project @@ -113,6 +116,8 @@ To disable the inbound job token scope allowlist: 1. Toggle **Allow access to this project with a CI_JOB_TOKEN** to disabled. Enabled by default in new projects. +You can also disable the allowlist [with the API](../../api/graphql/reference/index.md#mutationprojectcicdsettingsupdate). + ### Add a project to the inbound job token scope allowlist You can add projects to the inbound allowlist for a project. Projects added to the allowlist @@ -133,6 +138,8 @@ To add a project: 1. Under **Allow CI job tokens from the following projects to access this project**, add projects to the allowlist. +You can also add a target project to the allowlist [with the API](../../api/graphql/reference/index.md#mutationcijobtokenscopeaddproject). + ### Limit your project's job token access > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/328553) in GitLab 14.1. [Deployed behind the `:ci_scoped_job_token` feature flag](../../user/feature_flags.md), disabled by default. @@ -155,8 +162,8 @@ limited only by the user's access permissions. For example, when the setting is enabled, jobs in a pipeline in project `A` have a `CI_JOB_TOKEN` scope limited to project `A`. If the job needs to use the token to make an API request to a private project `B`, then `B` must be added to the allowlist for `A`. -If project `B` is public or internal, it's not required to be added to the allowlist. -The job token scope is only for controlling access to private projects. +If project `B` is public or internal, you do not need to add +`B` to the allowlist to grant access. ### Configure the outbound job token scope diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index 5e69ecff7b8..b9c2ee409b8 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Execution +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -339,8 +339,8 @@ In the example above: - `date +%s`: The Unix timestamp (for example `1560896352`). - `my_first_section`: The name given to the section. - `\r\e[0K`: Prevents the section markers from displaying in the rendered (colored) - job log, but they are displayed in the raw job log. To see them, in the upper right - of the job log, select **{doc-text}** (**Show complete raw**). + job log, but they are displayed in the raw job log. To see them, in the upper-right corner + of the job log, select **Show complete raw** (**{doc-text}**). - `\r`: carriage return. - `\e[0K`: clear line ANSI escape code. diff --git a/doc/ci/large_repositories/index.md b/doc/ci/large_repositories/index.md index 5a553228ba1..f728e9b9e17 100644 --- a/doc/ci/large_repositories/index.md +++ b/doc/ci/large_repositories/index.md @@ -255,7 +255,7 @@ For very active repositories with a large number of references and files, you ca enabled on all Gitaly servers, we found that we no longer need a pre-clone step for `gitlab-org/gitlab` development. - Optimize your CI/CD jobs by seeding repository data in a pre-clone step with the [`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) of GitLab Runner. See - [SaaS runners on Linux](../runners/saas/linux_saas_runner.md#pre-clone-script) for details. + [SaaS runners on Linux](../runners/saas/linux_saas_runner.md#pre-clone-script-deprecated) for details. Besides speeding up pipelines in large and active projects, seeding the repository data also helps avoid `429 Too many requests` errors from Cloudflare. diff --git a/doc/ci/lint.md b/doc/ci/lint.md index 74e0f0cd5ef..119a0e5853e 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -26,7 +26,7 @@ To check CI/CD configuration with the CI lint tool: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. -1. In the upper right, select **CI lint**. +1. In the upper-right corner, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. 1. Select **Validate**. @@ -47,7 +47,7 @@ To simulate a pipeline: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **CI/CD > Pipelines**. -1. In the upper right, select **CI lint**. +1. In the upper-right corner, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. 1. Select **Simulate pipeline creation for the default branch**. 1. Select **Validate**. diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index 727977b3bc8..1a7155c8195 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -70,7 +70,7 @@ simulations in the [CI Lint tool](../lint.md#simulate-a-pipeline). > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357219) in GitLab 15.1. You can review configuration added with the [`include`](../yaml/index.md#include) -keyword in the pipeline editor. In the upper right, select the file tree (**{file-tree}**) +keyword in the pipeline editor. In the upper-right corner, select the file tree (**{file-tree}**) to see a list of all included configuration files. Selected files open in a new tab for review. diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index 4cbaa77b029..e1eb2f53910 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -19,7 +19,7 @@ end-to-end duration of a pipeline. On GitLab.com: -- CI/CD minutes quotas are enabled for both public and private projects, but public +- CI/CD minutes quotas are enabled for all projects, but certain projects [consume CI/CD minutes at a slower rate](#cost-factor). - The base monthly CI/CD minutes quota for a GitLab.com [namespace](../../user/namespace/index.md) is determined by its [license tier](https://about.gitlab.com/pricing/). @@ -201,13 +201,12 @@ can be higher than the end-to-end duration of a pipeline. The cost factors for jobs running on shared runners on GitLab.com are: -- `1` for internal and private projects. -- `0.5` for public projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). -- `0.008` for public forks of public projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). For every 125 minutes of job execution time, +- `1` for internal, public, and private projects. +- Exceptions for public projects: + - `0.5` for projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). + - `0.008` for forks of projects in the [GitLab for Open Source program](../../subscriptions/index.md#gitlab-for-open-source). For every 125 minutes of job execution time, you use 1 CI/CD minute. -- `1` for other public projects, after October 1, 2022 (previously `0.04`). - For every 1 minute of job execution time, you use 1 CI/CD minute. -- Calculated differently for [community contributions to GitLab projects](#cost-factor-for-community-contributions-to-gitlab-projects). +- Discounted dynamically for [community contributions to GitLab projects](#cost-factor-for-community-contributions-to-gitlab-projects). The cost factors on self-managed instances are: diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md index e4560cd882d..1e4654e69fe 100644 --- a/doc/ci/pipelines/downstream_pipelines.md +++ b/doc/ci/pipelines/downstream_pipelines.md @@ -231,37 +231,43 @@ configuration for jobs that use the Windows runner, like scripts, use <code>\ ### Run child pipelines with merge request pipelines -To trigger a child pipeline as a [merge request pipeline](merge_request_pipelines.md): +Pipelines, including child pipelines, run as branch pipelines by default when not using +[`rules`](../yaml/index.md#rules) or [`workflow:rules`](../yaml/index.md#workflowrules). +To configure child pipelines to run when triggered from a [merge request (parent) pipeline](merge_request_pipelines.md), use `rules` or `workflow:rules`. +For example, using `rules`: -1. Set the trigger job to run on merge requests in the parent pipeline's configuration file: +1. Set the parent pipeline's trigger job to run on merge requests: ```yaml - microservice_a: + trigger-child-pipeline-job: trigger: - include: path/to/microservice_a.yml + include: path/to/child-pipeline-configuration.yml rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" ``` -1. Configure the child pipeline jobs to run in merge request pipelines with [`rules`](../yaml/index.md#rules) - or [`workflow:rules`](../yaml/index.md#workflowrules). - For example, with `rules` in a child pipeline's configuration file: +1. Use `rules` to configure the child pipeline jobs to run when triggered by the parent pipeline: ```yaml job1: - script: echo "Child pipeline job 1" + script: echo "This child pipeline job runs any time the parent pipeline triggers it." rules: - - if: $CI_MERGE_REQUEST_ID + - if: $CI_PIPELINE_SOURCE == "parent_pipeline" job2: - script: echo "Child pipeline job 2" + script: echo "This child pipeline job runs only when the parent pipeline is a merge request pipeline" rules: - if: $CI_MERGE_REQUEST_ID ``` - In child pipelines, `$CI_PIPELINE_SOURCE` always has a value of `parent_pipeline` - and cannot be used to identify merge request pipelines. Use `$CI_MERGE_REQUEST_ID` - instead, which is always present in merge request pipelines. +In child pipelines, `$CI_PIPELINE_SOURCE` always has a value of `parent_pipeline`, so: + +- You can use `if: $CI_PIPELINE_SOURCE == "parent_pipeline"` to ensure child pipeline jobs always run. +- You _can't_ use `if: $CI_PIPELINE_SOURCE == "merge_request_event"` to configure child pipeline + jobs to run for merge request pipelines. Instead, use `if: $CI_MERGE_REQUEST_ID` + to set child pipeline jobs to run only when the parent pipeline is a merge request pipeline. The parent pipeline's + [`CI_MERGE_REQUEST_*` predefined variables](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) + are passed to the child pipeline jobs. ### Specify a branch for multi-project pipelines @@ -281,7 +287,7 @@ Use: - The `project` keyword to specify the full path to the downstream project. In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/367660), you can use [variable expansion](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). -- The `branch` keyword to specify the name of a branch or [tag](../../topics/git/tags.md) +- The `branch` keyword to specify the name of a branch or [tag](../../user/project/repository/tags/index.md) in the project specified by `project`. You can use variable expansion. ## Trigger a multi-project pipeline by using the API @@ -309,18 +315,32 @@ trigger_pipeline: > Hover behavior for pipeline cards [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/197140/) in GitLab 13.2. In the [pipeline graph view](index.md#view-full-pipeline-graph), downstream pipelines display -as a list of cards on the right of the graph. Hover over the pipeline's card to view -which job triggered the downstream pipeline. +as a list of cards on the right of the graph. From this view, you can: -### Retry a downstream pipeline +- Select a trigger job to see the triggered downstream pipeline's jobs. +- Select **Expand jobs** **{chevron-lg-right}** on a pipeline card to expand the view + with the downstream pipeline's jobs. You can view one downstream pipeline at a time. +- Hover over a pipeline card to have the job that triggered the downstream pipeline highlighted. + +### Retry failed and canceled jobs in a downstream pipeline > - Retry from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. > - Retry from graph view [generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. -To retry a completed downstream pipeline, select **Retry** (**{retry}**): +To retry failed and canceled jobs, select **Retry** (**{retry}**): - From the downstream pipeline's details page. -- On the pipeline's card in the [pipeline graph view](index.md#view-full-pipeline-graph). +- On the pipeline's card in the pipeline graph view. + +### Recreate a downstream pipeline + +> Retry trigger job from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367547) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `ci_recreate_downstream_pipeline`. Disabled by default. + +You can recreate a downstream pipeline by retrying its corresponding trigger job. The newly created downstream pipeline replaces the current downstream pipeline in the pipeline graph. + +To recreate a downstream pipeline: + +- Select **Run again** (**{retry}**) on the trigger job's card in the pipeline graph view. ### Cancel a downstream pipeline @@ -330,7 +350,7 @@ To retry a completed downstream pipeline, select **Retry** (**{retry}**): To cancel a downstream pipeline that is still running, select **Cancel** (**{cancel}**): - From the downstream pipeline's details page. -- On the pipeline's card in the [pipeline graph view](index.md#view-full-pipeline-graph). +- On the pipeline's card in the pipeline graph view. ### Mirror the status of a downstream pipeline in the trigger job @@ -365,13 +385,9 @@ trigger_job: After you trigger a multi-project pipeline, the downstream pipeline displays to the right of the [pipeline graph](index.md#visualize-pipelines). -![Multi-project pipeline graph](img/multi_project_pipeline_graph_v14_3.png) - In [pipeline mini graphs](index.md#pipeline-mini-graphs), the downstream pipeline displays to the right of the mini graph. -![Multi-project pipeline mini graph](img/pipeline_mini_graph_v15_0.png) - ## Fetch artifacts from an upstream pipeline Use [`needs:project`](../yaml/index.md#needsproject) to fetch artifacts from an @@ -604,8 +620,9 @@ the ones defined in the upstream project take precedence. ### Pass dotenv variables created in a job **(PREMIUM)** -You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](../variables/index.md#pass-an-environment-variable-to-another-job) -and [`needs:project`](../yaml/index.md#needsproject). +You can pass variables to a downstream job with [`dotenv` variable inheritance](../variables/index.md#pass-an-environment-variable-to-another-job) +and [`needs:project`](../yaml/index.md#needsproject). These variables are only available in +the script of the job and can't be used to configure it, for example with `rules` or `artifact:paths`. For example, in a [multi-project pipeline](#multi-project-pipelines): @@ -656,6 +673,16 @@ With multi-project pipelines, the trigger job fails and does not create the down to run pipelines against the protected branch. See [pipeline security for protected branches](index.md#pipeline-security-on-protected-branches) for more information. +### Job in child pipeline is not created when the pipeline runs + +If the parent pipeline is a [merge request pipeline](merge_request_pipelines.md), +the child pipeline must [use `workflow:rules` or `rules` to ensure the jobs run](#run-child-pipelines-with-merge-request-pipelines). + +If no jobs in the child pipeline can run due to missing or incorrect `rules` configuration: + +- The child pipeline fails to start. +- The parent pipeline's trigger job fails with: `downstream pipeline can not be created, Pipeline will not run for the selected trigger. The rules configuration prevented any jobs from being added to the pipeline.` + ### `Ref is ambiguous` You cannot trigger a multi-project pipeline with a tag when a branch exists with the same diff --git a/doc/ci/pipelines/img/multi_project_pipeline_graph_v14_3.png b/doc/ci/pipelines/img/multi_project_pipeline_graph_v14_3.png Binary files differdeleted file mode 100644 index aadf8bb0979..00000000000 --- a/doc/ci/pipelines/img/multi_project_pipeline_graph_v14_3.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipeline_mini_graph_v15_0.png b/doc/ci/pipelines/img/pipeline_mini_graph_v15_0.png Binary files differdeleted file mode 100644 index 48a0ca9d84f..00000000000 --- a/doc/ci/pipelines/img/pipeline_mini_graph_v15_0.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index fa04cb6cb92..d2038dbcb62 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -423,8 +423,7 @@ You can group the jobs by: - [Job dependencies](#view-job-dependencies-in-the-pipeline-graph), which arranges jobs based on their [`needs`](../yaml/index.md#needs) dependencies. -[Multi-project pipeline graphs](downstream_pipelines.md#view-multi-project-pipelines-in-pipeline-graphs) help -you visualize the entire pipeline, including all cross-project inter-dependencies. +Multi-project pipeline graphs help you visualize the entire pipeline, including all cross-project inter-dependencies. If a stage contains more than 100 jobs, only the first 100 jobs are listed in the pipeline graph. The remaining jobs still run as usual. To see the jobs: diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index 75c9852d3d0..7db4596bb1d 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/job_artifacts.html' --- @@ -264,7 +264,7 @@ artifacts and log. You must be: To delete a job: 1. Go to a job's detail page. -1. In the upper right of the job's log, select **Erase job log** (**{remove}**). +1. In the upper-right corner of the job's log, select **Erase job log** (**{remove}**). 1. On the confirmation dialog, select **OK**. ## Expose job artifacts in the merge request UI diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 7e18c11f234..881e94a6559 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -140,14 +140,7 @@ the parent project. Additionally, if you do not trust the fork project's runner, running the pipeline in the parent project uses the parent project's trusted runners. WARNING: -Fork merge requests can contain malicious code that tries to steal secrets in the -parent project when the pipeline runs, even before merge. As a reviewer, carefully -check the changes in the merge request before triggering the pipeline. If you trigger -the pipeline by selecting **Run pipeline** or applying a suggestion, GitLab shows -a warning that you must accept before the pipeline runs. If you trigger the pipeline -by using any other method, including the API, [`/rebase` quick action](../../user/project/quick_actions.md#issues-merge-requests-and-epics), -or [**Rebase** option](../../user/project/merge_requests/methods/index.md#rebasing-in-semi-linear-merge-methods), -**no warning displays**. +Fork merge requests can contain malicious code that tries to steal secrets in the parent project when the pipeline runs, even before merge. As a reviewer, carefully check the changes in the merge request before triggering the pipeline. Unless you trigger the pipeline through the API or the [`/rebase` quick action](../../user/project/quick_actions.md#issues-merge-requests-and-epics), GitLab shows a warning that you must accept before the pipeline runs. Otherwise, **no warning displays**. Prerequisites: diff --git a/doc/ci/pipelines/pipeline_architectures.md b/doc/ci/pipelines/pipeline_architectures.md index e4a3416f60b..dadf631e2bb 100644 --- a/doc/ci/pipelines/pipeline_architectures.md +++ b/doc/ci/pipelines/pipeline_architectures.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Execution +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/ci/pipelines/pipeline_artifacts.md b/doc/ci/pipelines/pipeline_artifacts.md index aacaa110041..31c848d5274 100644 --- a/doc/ci/pipelines/pipeline_artifacts.md +++ b/doc/ci/pipelines/pipeline_artifacts.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 3633863915c..98dc02c1041 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -71,7 +71,7 @@ is selected. ## Auto-cancel redundant pipelines -You can set pending or running pipelines to cancel automatically when a new pipeline runs on the same branch. You can enable this in the project settings: +You can set pending or running pipelines to cancel automatically when a pipeline for new changes runs on the same branch. You can enable this in the project settings: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md index 7cf71f2a3ea..3835b5f2df2 100644 --- a/doc/ci/resource_groups/index.md +++ b/doc/ci/resource_groups/index.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Configure +group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: Control the job concurrency in GitLab CI/CD --- diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 7a0f43f9ec5..9e69ab1ccb8 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index 0fd4bff1bff..41dda5da0b2 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -155,8 +155,11 @@ different places. ### Determine the IP address of a shared runner -To view the IP address of a shared runner you must have administrator access to -the GitLab instance. To determine this: +Prerequisite: + +- You must have administrator access to the instance. + +To determine the IP address of a shared runner: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **CI/CD > Runners**. diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index 41cde709143..d3bbd7d8816 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -20,7 +20,7 @@ in your [subscription plan](https://about.gitlab.com/pricing/). ## Security for GitLab SaaS runners -GitLab SaaS runners on Linux and Windows run on Google Compute Platform. The [Google Infrastructure Security Design Overview whitepaper](https://cloud.google.com/docs/security/infrastructure/design/resources/google_infrastructure_whitepaper_fa.pdf) provides an overview of how Google designs security into its technical infrastructure. The GitLab [Trust Center](https://about.gitlab.com/security/) and [GitLab Security Compliance Controls](https://about.staging.gitlab.com/handbook/engineering/security/security-assurance/security-compliance/sec-controls.html) pages provide an overview of the Security and Compliance controls that govern the GitLab SaaS runners. +GitLab SaaS runners on Linux and Windows run on Google Compute Platform. The [Google Infrastructure Security Design Overview whitepaper](https://cloud.google.com/docs/security/infrastructure/design/resources/google_infrastructure_whitepaper_fa.pdf) provides an overview of how Google designs security into its technical infrastructure. The GitLab [Trust Center](https://about.gitlab.com/security/) and [GitLab Security Compliance Controls](https://about.staging.gitlab.com/handbook/engineering/security/security-assurance/security-compliance/sec-controls.html) pages provide an overview of the security and compliance controls that govern the GitLab SaaS runners. The runner that serves as a Runner Manager automatically initiates the creation and deletion of the virtual machines (VMs) used for CI jobs. When the Runner Manager picks up a GitLab SaaS CI job, it automatically executes that job on a new VM. There is no human or manual intervention in this process. The following section provides an overview of the additional built-in layers that harden the security of the GitLab Runner SaaS CI build environment. diff --git a/doc/ci/runners/register_runner.md b/doc/ci/runners/register_runner.md new file mode 100644 index 00000000000..ec10a1299dc --- /dev/null +++ b/doc/ci/runners/register_runner.md @@ -0,0 +1,33 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Register a shared runner **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383139) in GitLab 15.10. [Deployed behind the `create_runner_workflow_for_admin` flag](../../administration/feature_flags.md), disabled by default. + +FLAG: +On self-managed GitLab, this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `create_runner_workflow_for_admin`. +On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. + +Prerequisites: + +- You must be an administrator to register a shared runner. + +To register a [shared runner](../runners/runners_scope.md#shared-runners): + +1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, select **CI/CD > Runners**. +1. Select **New instance runner**. +1. Select a platform. +1. Optional. Enter a description. +1. Optional. Enter a maintenance note. +1. Optional. Enter configurations for the runner. +1. Select **Submit**. +1. Follow the instructions to register the runner from the command line. + +NOTE: +The token only displays in the UI for a short period of time during registration, +and is then saved in `config.toml`. diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md index 58c8a6d0815..e9ac91409af 100644 --- a/doc/ci/runners/saas/linux_saas_runner.md +++ b/doc/ci/runners/saas/linux_saas_runner.md @@ -88,12 +88,15 @@ Below are the settings for SaaS runners on Linux. and [#4070](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4070). NOTE: -The final disk space your jobs can use is less than 25 GB. Some disk space -allocated to the instance is occupied by the operating system, the Docker image, -and a copy of your cloned repository. +SaaS runner instances are provisioned with a 25 GB storage volume. The underlying disk space of the storage volume +is shared by the operating system, the Docker image, and a copy of your cloned repository. +This means that the available free disk space that your jobs can use is **less than 25 GB**. -## Pre-clone script +## Pre-clone script (deprecated) +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/391896) in GitLab 15.9 +and is planned for removal in 16.0. Use [`pre_get_sources_script`](../../../ci/yaml/index.md#hookspre_get_sources_script) instead. This change is a breaking change. With SaaS runners on Linux, you can run commands in a CI/CD job before the runner attempts to run `git init` and `git fetch` to download a GitLab repository. The diff --git a/doc/ci/runners/saas/macos/environment.md b/doc/ci/runners/saas/macos/environment.md index 1fc7afea7e6..7aa0f33fc59 100644 --- a/doc/ci/runners/saas/macos/environment.md +++ b/doc/ci/runners/saas/macos/environment.md @@ -20,11 +20,11 @@ Each time you run a job that requires tooling or dependencies not available in t GitLab SaaS provides macOS build machines on Apple servers with Intel x86-64 processors. The expectation is that virtual machines running on the Apple M1 chip will be available in the second half of 2022. -At this time there is only one available machine type offered, `gbc-macos-large`. +At this time there is only one available machine type offered, `shared-macos-amd64`. | Instance type | vCPUS | Memory (GB) | | --------- | --- | ------- | -| `gbc-macos-large` | 4 | 10 | +| `shared-macos-amd64` | 4 | 10 | ## VM images diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index 96e294ff828..21120732fbe 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -84,4 +84,4 @@ In SaaS runners on macOS, the objective is to make 90% of CI jobs start executin - If the VM image does not include the specific software version you need for your job, then the job execution time will increase as the required software needs to be fetched and installed. - At this time, it is not possible to bring your own OS image. -- The keychain for user `gitlab` is not publicly available. You must create a keychain instead. +- The keychain for user `gitlab` is not publicly available. You must create a keychain instead. diff --git a/doc/ci/secrets/id_token_authentication.md b/doc/ci/secrets/id_token_authentication.md index f622bc2a7b1..b10763b40d6 100644 --- a/doc/ci/secrets/id_token_authentication.md +++ b/doc/ci/secrets/id_token_authentication.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: tutorial --- @@ -45,7 +45,7 @@ The following fields are included in each ID token: | [`iss`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.1) | Always | Issuer of the token, which is the domain of the GitLab instance ("issuer" claim). | | [`jti`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.7) | Always | Unique identifier for the token ("JWT ID" claim). | | [`nbf`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.5) | Always | The time after which the token becomes valid ("not before" claim). | -| [`sub`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.2) | Always | The job ID ("subject" claim). | +| [`sub`](https://www.rfc-editor.org/rfc/rfc7519.html#section-4.1.2) | Always | `project_path:{group}/{project}:ref_type:{type}:ref:{branch_name}` ("subject" claim). | | `deployment_tier` | Job specifies an environment | [Deployment tier](../environments/index.md#deployment-tier-of-environments) of the environment the job specifies. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363590) in GitLab 15.2. | | `environment_protected` | Job specifies an environment | `true` if specified environment is protected, `false` otherwise. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9. | | `environment` | Job specifies an environment | Environment the job specifies. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9. | diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md index 14f771431e5..fa8fe2a36a8 100644 --- a/doc/ci/secrets/index.md +++ b/doc/ci/secrets/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/ci/secure_files/index.md b/doc/ci/secure_files/index.md index 10baa57cabb..d432db351bb 100644 --- a/doc/ci/secure_files/index.md +++ b/doc/ci/secure_files/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index f26751c56e7..eca4c62deb2 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -269,7 +269,7 @@ test: | `entrypoint` | no | 9.4 |Command or script to execute as the container's entrypoint. It's translated to the Docker `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. | | `command` | no | 9.4 |Command or script that should be used as the container's command. It's translated to arguments passed to Docker after the image's name. The syntax is similar to [`Dockerfile`'s `CMD`](https://docs.docker.com/engine/reference/builder/#cmd) directive, where each shell token is a separate string in the array. | | `alias` (1) | no | 9.4 | Additional alias that can be used to access the service from the job's container. Read [Accessing the services](#accessing-the-services) for more information. | -| `variables` (2) | no | 14.5 | Additional environment variables that are passed exclusively to the service. The syntax is the same as [Job Variables](../variables/index.md). | +| `variables` (2) | no | 14.5 | Additional environment variables that are passed exclusively to the service. The syntax is the same as [Job Variables](../variables/index.md). Service variables cannot reference themselves. | (1) Alias support for the Kubernetes executor was [introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2229) in GitLab Runner 12.8, and is only available for Kubernetes version 1.7 or later. diff --git a/doc/ci/ssh_keys/index.md b/doc/ci/ssh_keys/index.md index c1154485018..ab767697cbc 100644 --- a/doc/ci/ssh_keys/index.md +++ b/doc/ci/ssh_keys/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Execution +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/test_cases/index.md b/doc/ci/test_cases/index.md index 4088e5e82c6..8ad9e27de8b 100644 --- a/doc/ci/test_cases/index.md +++ b/doc/ci/test_cases/index.md @@ -1,12 +1,12 @@ --- stage: Plan -group: Certify +group: Product Planning info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments description: Test cases in GitLab can help your teams create testing scenarios in their existing development platform. type: reference --- -# Test Cases **(ULTIMATE)** +# Test cases **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233479) in GitLab 13.6. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/241983) in GitLab 13.7. @@ -24,14 +24,14 @@ Prerequisite: To create a test case in a GitLab project: -1. Go to **CI/CD > Test Cases**. +1. Go to **CI/CD > Test cases**. 1. Select **New test case**. You are taken to the new test case form. Here you can enter the new case's title, [description](../../user/markdown.md), attach a file, and assign [labels](../../user/project/labels.md). 1. Select **Submit test case**. You are taken to view the new test case. ## View a test case -You can view all test cases in the project in the Test Cases list. Filter the +You can view all test cases in the project in the test cases list. Filter the issue list with a search query, including labels or the test case's title. Prerequisite: @@ -43,7 +43,7 @@ Whether you can view an test case depends on the [project visibility level](../. To view a test case: -1. In a project, go to **CI/CD > Test Cases**. +1. In a project, go to **CI/CD > Test cases**. 1. Select the title of the test case you want to view. You are taken to the test case page. ![An example test case page](img/test_case_show_v13_10.png) @@ -77,7 +77,7 @@ To archive a test case, on the test case's page, select **Archive test case**. To view archived test cases: -1. Go to **CI/CD > Test Cases**. +1. Go to **CI/CD > Test cases**. 1. Select **Archived**. ## Reopen an archived test case diff --git a/doc/ci/testing/accessibility_testing.md b/doc/ci/testing/accessibility_testing.md index fa57371a7d5..58bb4308095 100644 --- a/doc/ci/testing/accessibility_testing.md +++ b/doc/ci/testing/accessibility_testing.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/testing/browser_performance_testing.md b/doc/ci/testing/browser_performance_testing.md index ff013f0037e..175ecef7f09 100644 --- a/doc/ci/testing/browser_performance_testing.md +++ b/doc/ci/testing/browser_performance_testing.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/testing/code_quality.md b/doc/ci/testing/code_quality.md index 4b826991bb5..ff7e11e9c71 100644 --- a/doc/ci/testing/code_quality.md +++ b/doc/ci/testing/code_quality.md @@ -44,6 +44,7 @@ Code Quality results are shown in the: - Merge request widget - Merge request changes view - Pipeline details view +- Project quality view ### Merge request widget @@ -73,6 +74,12 @@ tab of the pipeline's details page. ![Code Quality Report](img/code_quality_report_13_11.png) +### Project quality view + +The project quality view displays an overview of the code quality findings. + +![Code Quality Summary](img/code_quality_summary_15_9.png) + ## Enable Code Quality Prerequisites: diff --git a/doc/ci/testing/fail_fast_testing.md b/doc/ci/testing/fail_fast_testing.md index 4ad1656c8d6..57e21beef06 100644 --- a/doc/ci/testing/fail_fast_testing.md +++ b/doc/ci/testing/fail_fast_testing.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/testing/img/code_quality_summary_15_9.png b/doc/ci/testing/img/code_quality_summary_15_9.png Binary files differnew file mode 100644 index 00000000000..60077123488 --- /dev/null +++ b/doc/ci/testing/img/code_quality_summary_15_9.png diff --git a/doc/ci/testing/index.md b/doc/ci/testing/index.md index 41d474f0e60..5fd86561b9e 100644 --- a/doc/ci/testing/index.md +++ b/doc/ci/testing/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/testing/load_performance_testing.md b/doc/ci/testing/load_performance_testing.md index 7e527fae562..cf870bcec55 100644 --- a/doc/ci/testing/load_performance_testing.md +++ b/doc/ci/testing/load_performance_testing.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/testing/metrics_reports.md b/doc/ci/testing/metrics_reports.md index e084e4d3bc7..9257ba8990e 100644 --- a/doc/ci/testing/metrics_reports.md +++ b/doc/ci/testing/metrics_reports.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/testing/test_coverage_visualization.md b/doc/ci/testing/test_coverage_visualization.md index e2a1a4a2c5e..1b29304d1af 100644 --- a/doc/ci/testing/test_coverage_visualization.md +++ b/doc/ci/testing/test_coverage_visualization.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -33,7 +33,7 @@ This format was originally developed for Java, but most coverage analysis framew for other languages have plugins to add support for it, like: - [simplecov-cobertura](https://rubygems.org/gems/simplecov-cobertura) (Ruby) -- [gocover-cobertura](https://github.com/boumenot/gocover-cobertura) (Golang) +- [gocover-cobertura](https://github.com/boumenot/gocover-cobertura) (Go) Other coverage analysis frameworks support the format out of the box, for example: diff --git a/doc/ci/testing/unit_test_report_examples.md b/doc/ci/testing/unit_test_report_examples.md index 87426fc8496..c63e225a2a7 100644 --- a/doc/ci/testing/unit_test_report_examples.md +++ b/doc/ci/testing/unit_test_report_examples.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -274,3 +274,24 @@ phpunit: reports: junit: report.xml ``` + +## Rust + +This example uses [cargo2junit](https://crates.io/crates/cargo2junit), +which is installed in the current directory. +To retrieve JSON output from `cargo test`, you must enable the nightly compiler. + +```yaml +run unittests: + image: rust:latest + stage: test + before_script: + - cargo install --root . cargo2junit + script: + - cargo test -- -Z unstable-options --format json --report-time | bin/cargo2junit > report.xml + artifacts: + when: always + reports: + junit: + - report.xml +``` diff --git a/doc/ci/testing/unit_test_reports.md b/doc/ci/testing/unit_test_reports.md index 0fe9b2b6d64..88a5d90d30e 100644 --- a/doc/ci/testing/unit_test_reports.md +++ b/doc/ci/testing/unit_test_reports.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index 16530ea20b6..393b128c658 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Execution +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: tutorial --- @@ -201,3 +201,13 @@ doesn't exist, GitLab returns `The requested URL returned error: 400`. For example, you might accidentally use `main` for the branch name in a project that uses a different branch name for its default branch. + +Another possible cause for this error is a rule that prevents creation of the pipelines when `CI_PIPELINE_SOURCE` value is `trigger`, such as: + +```yaml +rules: + - if: $CI_PIPELINE_SOURCE == "trigger" + when: never +``` + +Review your [`workflow:rules`](../yaml/index.md#workflowrules) to ensure a pipeline can be created when `CI_PIPELINE_SOURCE` value is `trigger`. diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 17ce184ee28..30c067db36a 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Execution +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- @@ -37,7 +37,7 @@ If you need to link to the schema directly, it is at: ```plaintext -https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/editor/schema/ci.json. +https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/editor/schema/ci.json ``` To see the full list of custom tags covered by the CI/CD schema, check the diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index ec5c6c240a6..24eee4253a8 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- @@ -19,6 +19,12 @@ or have them [prefilled in manual pipelines](../pipelines/index.md#prefill-varia Variable names are limited by the [shell the runner uses](https://docs.gitlab.com/runner/shells/index.html) to execute scripts. Each shell has its own set of reserved variable names. +To ensure consistent behavior, you should always put variable values in single or double quotes. +Variables are internally parsed by the [Psych YAML parser](https://docs.ruby-lang.org/en/master/Psych.html), +so quoted and unquoted variables might be parsed differently. For example, `VAR1: 012345` +is interpreted as an octal value, so the value becomes `5349`, but `VAR1: "012345"` is parsed +as a string with a value of `012345`. + > For more information about advanced use of GitLab CI/CD: > > - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Get to productivity faster with these [7 advanced GitLab CI workflow hacks](https://about.gitlab.com/webcast/7cicd-hacks/) @@ -189,7 +195,7 @@ You can make a CI/CD variable available to all projects and groups in a GitLab i Prerequisite: -- You must have administrator access. +- You must have administrator access to the instance. To add an instance variable: diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index b9557b98066..45dc1a7fe37 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index 6ea1f454467..c20d9be51e7 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Authoring +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: reference --- diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index 151a043da5e..8740cf100ac 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Insights +group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -22,7 +22,7 @@ date for the artifacts. Some `artifacts:reports` types can be generated by multiple jobs in the same pipeline, and used by merge request or pipeline features from each job. -To be able to browse the report output files, include the [`artifacts:paths`](index.md#artifactspaths) keyword. +To browse the report output files, ensure you include the [`artifacts:paths`](index.md#artifactspaths) keyword in your job definition. NOTE: Combined reports in parent pipelines using [artifacts from child pipelines](index.md#needspipelinejob) is diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index bf0b7444e78..6d34a3034f7 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -20,13 +20,6 @@ To include a single configuration file, use either of these syntax options: include: '/templates/.after-script-template.yml' ``` -- `include` with a single file, and you specify the `include` type: - - ```yaml - include: - remote: 'https://gitlab.com/awesome-project/raw/main/.before-script-template.yml' - ``` - ## Include an array of configuration files You can include an array of configuration files: @@ -159,12 +152,18 @@ do not change. This method is called *merging*. ### Merge method for `include` -For a file containing `include` directives, the included files are read in order (possibly -recursively), and the configuration in these files is likewise merged in order. If the parameters overlap, the last included file takes precedence. Finally, the directives in the -file itself are merged with the configuration from the included files. +The `include` configuration merges with the main configuration file with this process: + +- Included files are read in the order defined in the configuration file, and + the included configuration is merged together in the same order. +- If an included file also uses `include`, that nested `include` configuration is merged first (recursively). +- If parameters overlap, the last included file takes precedence when merging the configuration + from the included files. +- After all configuration added with `include` is merged together, the main configuration + is merged with the included configuration. This merge method is a _deep merge_, where hash maps are merged at any depth in the -configuration. To merge hash map A (containing the configuration merged so far) and B (the next piece +configuration. To merge hash map "A" (that contains the configuration merged so far) and "B" (the next piece of configuration), the keys and values are processed as follows: - When the key only exists in A, use the key and value from A. @@ -172,9 +171,7 @@ of configuration), the keys and values are processed as follows: - When the key exists in both A and B, and one of the values is not a hash map, use the value from B. - Otherwise, use the key and value from B. -For example: - -We have a configuration consisting of two files. +For example, with a configuration that consists of two files: - The `.gitlab-ci.yml` file: @@ -211,7 +208,7 @@ We have a configuration consisting of two files. dotenv: deploy.env ``` -The merged result: +The merged result is: ```yaml variables: @@ -374,7 +371,7 @@ In `include` sections in your `.gitlab-ci.yml` file, you can use: - [Project variables](../variables/index.md#for-a-project). - [Group variables](../variables/index.md#for-a-group). - [Instance variables](../variables/index.md#for-an-instance). -- Project [predefined variables](../variables/predefined_variables.md). +- Project [predefined variables](../variables/predefined_variables.md) (`CI_PROJECT_*`). - In GitLab 14.2 and later, the `$CI_COMMIT_REF_NAME` [predefined variable](../variables/predefined_variables.md). When used in `include`, the `CI_COMMIT_REF_NAME` variable returns the full @@ -386,15 +383,7 @@ In GitLab 14.5 and later, you can also use: - [Trigger variables](../triggers/index.md#pass-cicd-variables-in-the-api-call). - [Scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule). - [Manual pipeline run variables](../pipelines/index.md#run-a-pipeline-manually). -- Pipeline [predefined variables](../variables/predefined_variables.md). - - YAML files are parsed before the pipeline is created, so the following pipeline predefined variables - are **not** available: - - - `CI_PIPELINE_ID` - - `CI_PIPELINE_URL` - - `CI_PIPELINE_IID` - - `CI_PIPELINE_CREATED_AT` +- The `CI_PIPELINE_SOURCE` and `CI_PIPELINE_TRIGGERED` [predefined variables](../variables/predefined_variables.md). For example: @@ -517,3 +506,20 @@ When the pipeline runs, GitLab: # This matches all `.yml` files only in subfolders of `configs`. include: 'configs/**/*.yml' ``` + +## Troubleshooting + +### `Maximum of 150 nested includes are allowed!` error + +The maximum number of [nested included files](#use-nested-includes) for a pipeline is 150. +If you receive the `Maximum 150 includes are allowed` error message in your pipeline, +it's likely that either: + +- Some of the nested configuration includes an overly large number of additional nested `include` configuration. +- There is an accidental loop in the nested includes. For example, `include1.yml` includes + `include2.yml` which includes `include1.yml`, creating a recursive loop. + +To help reduce the risk of this happening, edit the pipeline configuration file +with the [pipeline editor](../pipeline_editor/index.md), which validates if the +limit is reached. You can remove one included file at a time to try to narrow down +which configuration file is the source of the loop or excessive included files. diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index d5c0fe1d41d..6ba9d455278 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -136,11 +136,17 @@ The `include` files are: - Always evaluated first and then merged with the content of the `.gitlab-ci.yml` file, regardless of the position of the `include` keyword. -You can [nest](includes.md#use-nested-includes) up to 100 includes. In [GitLab 14.9 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/28987), -the same file can be included multiple times in nested includes, but duplicates are ignored. +You can have up to 150 includes per pipeline, including [nested](includes.md#use-nested-includes) includes: -In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/28212), -the time limit to resolve all files is 30 seconds. +- In [GitLab 15.10 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/367150) you can have up to 150 includes. + In nested includes, the same file can be included multiple times, but duplicated includes + count towards the limit. +- From [GitLab 14.9 to GitLab 15.9](https://gitlab.com/gitlab-org/gitlab/-/issues/28987), you can have up to 100 includes. + The same file can be included multiple times in nested includes, but duplicates are ignored. +- In GitLab 14.9 and earlier you can have up to 100 includes, but the same file can not + be included multiple times. + +The time limit to resolve all files is 30 seconds. **Keyword type**: Global keyword. @@ -153,6 +159,8 @@ the time limit to resolve all files is 30 seconds. **Additional details**: +- Only [certain CI/CD variables](includes.md#use-variables-with-include) can be used + with `include` keywords. - Use merging to customize and override included CI/CD configurations with local - You can override included configuration by having the same job name or global keyword in the `.gitlab-ci.yml` file. The two configurations are merged together, and the @@ -171,7 +179,7 @@ the time limit to resolve all files is 30 seconds. #### `include:local` -Use `include:local` to include a file that is in the same repository as the project running the pipeline. +Use `include:local` to include a file that is in the same repository as the configuration file containing the `include` keyword. Use `include:local` instead of symbolic links. **Keyword type**: Global keyword. @@ -201,8 +209,8 @@ include: '.gitlab-ci-production.yml' - The `.gitlab-ci.yml` file and the local file must be on the same branch. - You can't include local files through Git submodules paths. -- All [nested includes](includes.md#use-nested-includes) are executed in the scope of the same project, - so you can use local, project, remote, or template includes. +- All [nested includes](includes.md#use-nested-includes) are executed in the scope of the project containing the configuration file with the `include` keyword, not the project running the pipeline. + You can use local, project, remote, or template includes. #### `include:project` @@ -220,8 +228,7 @@ use `include:project` and `include:file`. The YAML files must have the `.yml` or `.yaml` extension. - `include:ref`: Optional. The ref to retrieve the file from. Defaults to the `HEAD` of the project when not specified. - -You can use [certain CI/CD variables](includes.md#use-variables-with-include). +- You can use [certain CI/CD variables](includes.md#use-variables-with-include). **Example of `include:project`**: @@ -252,8 +259,8 @@ include: **Additional details**: -- All [nested includes](includes.md#use-nested-includes) are executed in the scope of the target project. - You can use `local` (relative to the target project), `project`, `remote`, or `template` includes. +- All [nested includes](includes.md#use-nested-includes) are executed in the scope of the project containing the configuration file with the nested `include` keyword. + You can use `local` (relative to the project containing the configuration file with the `include` keyword), `project`, `remote`, or `template` includes. - When the pipeline starts, the `.gitlab-ci.yml` file configuration included by all methods is evaluated. The configuration is a snapshot in time and persists in the database. GitLab does not reflect any changes to the referenced `.gitlab-ci.yml` file configuration until the next pipeline starts. @@ -422,23 +429,30 @@ A configuration with different pipeline names depending on the pipeline conditio ```yaml variables: - PIPELINE_NAME: 'Default pipeline name' # A default is not required. + PROJECT1_PIPELINE_NAME: 'Default pipeline name' # A default is not required. workflow: - name: '$PIPELINE_NAME' + name: '$PROJECT1_PIPELINE_NAME' rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' variables: - PIPELINE_NAME: 'MR pipeline: $CI_COMMIT_BRANCH' + PROJECT1_PIPELINE_NAME: 'MR pipeline: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME' - if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-in-ruby3/' variables: - PIPELINE_NAME: 'Ruby 3 pipeline' + PROJECT1_PIPELINE_NAME: 'Ruby 3 pipeline' ``` **Additional details**: - If the name is an empty string, the pipeline is not assigned a name. A name consisting of only CI/CD variables could evaluate to an empty string if all the variables are also empty. +- `workflow:rules:variables` become [global variables](#variables) available in all jobs, + including [`trigger`](#trigger) jobs which forward variables to downstream pipelines by default. + If the downstream pipeline uses the same variable, the [variable is overwritten](../variables/index.md#cicd-variable-precedence) + by the upstream variable value. Be sure to either: + - Use a unique variable name in every project's pipeline configuration, like `PROJECT1_PIPELINE_NAME`. + - Use [`inherit:variables`](#inheritvariables) in the trigger job and list the + exact variables you want to forward to the downstream pipeline. #### `workflow:rules` @@ -552,6 +566,16 @@ When the branch is something else: - job1's `DEPLOY_VARIABLE` is `job1-default-deploy`. - job2's `DEPLOY_VARIABLE` is `default-deploy`. +**Additional details**: + +- `workflow:rules:variables` become [global variables](#variables) available in all jobs, + including [`trigger`](#trigger) jobs which forward variables to downstream pipelines by default. + If the downstream pipeline uses the same variable, the [variable is overwritten](../variables/index.md#cicd-variable-precedence) + by the upstream variable value. Be sure to either: + - Use unique variable names in every project's pipeline configuration, like `PROJECT1_VARIABLE_NAME`. + - Use [`inherit:variables`](#inheritvariables) in the trigger job and list the + exact variables you want to forward to the downstream pipeline. + ## Job keywords The following topics explain how to use keywords to configure CI/CD pipelines. @@ -938,10 +962,8 @@ job: #### `artifacts:public` -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49775) in GitLab 13.8 -> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -> - It's disabled on GitLab.com. -> - It's recommended for production use. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223273) in GitLab 13.8 [with a flag](../../user/feature_flags.md) named `non_public_artifacts`, disabled by default. +> - [Updated](https://gitlab.com/gitlab-org/gitlab/-/issues/322454) in GitLab 15.10. Artifacts created with `artifacts:public` before 15.10 are not guaranteed to remain private after this update. FLAG: On self-managed GitLab, by default this feature is not available. To make it available, @@ -1924,12 +1946,8 @@ rspec: ### `hooks` -> Introduced in GitLab 15.6 [with a flag](../../administration/feature_flags.md) named `ci_hooks_pre_get_sources_script`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `ci_hooks_pre_get_sources_script`. -The feature is not ready for production use. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356850) in GitLab 15.6 [with a flag](../../administration/feature_flags.md) named `ci_hooks_pre_get_sources_script`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/381840) in GitLab 15.10. Feature flag `ci_hooks_pre_get_sources_script` removed. Use `hooks` to specify lists of commands to execute on the runner at certain stages of job execution, like before retrieving the Git repository. @@ -1943,7 +1961,8 @@ at certain stages of job execution, like before retrieving the Git repository. #### `hooks:pre_get_sources_script` -> Introduced in GitLab 15.6 [with a flag](../../administration/feature_flags.md) named `ci_hooks_pre_get_sources_script`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356850) in GitLab 15.6 [with a flag](../../administration/feature_flags.md) named `ci_hooks_pre_get_sources_script`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/381840) in GitLab 15.10. Feature flag `ci_hooks_pre_get_sources_script` removed. Use `hooks:pre_get_sources_script` to specify a list of commands to execute on the runner before retrieving the Git repository and any submodules. You can use it @@ -4348,18 +4367,19 @@ child3: ### `variables` -Use `variables` to define [CI/CD variables](../variables/index.md#define-a-cicd-variable-in-the-gitlab-ciyml-file), -which are configurable values that are passed to jobs. - -Variables are always available in `script`, `before_script`, and `after_script` commands. -You can also use variables as inputs in some job keywords. +Use `variables` to define [CI/CD variables](../variables/index.md#define-a-cicd-variable-in-the-gitlab-ciyml-file) for jobs. **Keyword type**: Global and job keyword. You can use it at the global level, and also at the job level. -If you define `variables` at the global level, each variable is copied to -every job configuration when the pipeline is created. If the job already has that -variable defined, the [job-level variable takes precedence](../variables/index.md#cicd-variable-precedence). +If you define `variables` as a [global keyword](#keywords), it behaves like default variables +for all jobs. Each variable is copied to every job configuration when the pipeline is created. +If the job already has that variable defined, the [job-level variable takes precedence](../variables/index.md#cicd-variable-precedence). + +Variables defined at the global-level cannot be used as inputs for other global keywords +like [`include`](includes.md#use-variables-with-include). These variables can only +be used at the job-level, in `script`, `before_script`, and `after_script` sections, +as well as inputs in some job keywords like [`rules`](../jobs/job_control.md#cicd-variable-expressions). **Possible inputs**: Variable name and value pairs: |