diff options
Diffstat (limited to 'doc/ci')
105 files changed, 1119 insertions, 514 deletions
diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index d14c9563642..be5112251a4 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -4,7 +4,7 @@ group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Caching in GitLab CI/CD **(FREE)** +# Caching in GitLab CI/CD **(FREE ALL)** A cache is one or more files a job downloads and saves. Subsequent jobs that use the same cache don't have to download the files again, so they execute more quickly. diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md index 09f2758113f..10276df6291 100644 --- a/doc/ci/chatops/index.md +++ b/doc/ci/chatops/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index, concepts, howto --- -# GitLab ChatOps **(FREE)** +# GitLab ChatOps **(FREE ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4466) in GitLab Ultimate 10.6. > - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24780) to GitLab Free in 11.9. diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index 5494e5fad1c..7164fae10a1 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Using GitLab CI/CD with a Bitbucket Cloud repository **(PREMIUM)** +# Using GitLab CI/CD with a Bitbucket Cloud repository **(PREMIUM ALL)** GitLab CI/CD can be used with Bitbucket Cloud by: diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index feb01a0fc4a..1fad7ad5a53 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Using GitLab CI/CD with a GitHub repository **(PREMIUM)** +# Using GitLab CI/CD with a GitHub repository **(PREMIUM ALL)** GitLab CI/CD can be used with **GitHub.com** and **GitHub Enterprise** by creating a [CI/CD project](index.md) to connect your GitHub repository to diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index 7f454cabcf4..a9093632a8c 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index, howto --- -# GitLab CI/CD for external repositories **(PREMIUM)** +# GitLab CI/CD for external repositories **(PREMIUM ALL)** >[Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4642) in GitLab 10.6. @@ -103,3 +103,8 @@ references an open Pull Request, both contribute to the status of the Pull Reque via GitHub integration. If you want to exclusively run pipelines on external pull requests and not on branches you can add `except: [branches]` to the job specs. [Read more](https://gitlab.com/gitlab-org/gitlab/-/issues/24089#workaround). + +## Troubleshooting + +- [Pull mirroring is not triggering pipelines](../../user/project/repository/mirror/troubleshooting.md#pull-mirroring-is-not-triggering-pipelines). +- [Fix hard failures when mirroring](../../user/project/repository/mirror/pull.md#fix-hard-failures-when-mirroring). diff --git a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md index f7604689b78..f50b7be855a 100644 --- a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md +++ b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md @@ -4,7 +4,7 @@ group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Deploy to Amazon Elastic Container Service **(FREE)** +# Deploy to Amazon Elastic Container Service **(FREE ALL)** This step-by-step guide helps you deploy a project hosted on GitLab.com to the Amazon [Elastic Container Service (ECS)](https://aws.amazon.com/ecs/). @@ -217,14 +217,14 @@ These variables are injected into the pipeline jobs and can access the ECS API. 1. Go to **Settings > CI/CD > Variables**. 1. Select **Add Variable** and set the following key-value pairs. - |Key|Value|Note| - |---|---|---| - |`AWS_ACCESS_KEY_ID`|`<Access key ID of the deployer>`| For authenticating `aws` CLI. | - |`AWS_SECRET_ACCESS_KEY`|`<Secret access key of the deployer>`| For authenticating `aws` CLI. | - |`AWS_DEFAULT_REGION`|`us-east-2`| For authenticating `aws` CLI. | - |`CI_AWS_ECS_CLUSTER`|`ecs-demo`| The ECS cluster is accessed by `production_ecs` job. | - |`CI_AWS_ECS_SERVICE`|`ecs_demo`| The ECS service of the cluster is updated by `production_ecs` job. Ensure that this variable is scoped to the appropriate environment (`production`, `staging`, `review/*`). | - |`CI_AWS_ECS_TASK_DEFINITION`|`ecs_demo`| The ECS task definition is updated by `production_ecs` job. | + | Key | Value | Note | + |------------------------------|---------------------------------------|------| + | `AWS_ACCESS_KEY_ID` | `<Access key ID of the deployer>` | For authenticating `aws` CLI. | + | `AWS_SECRET_ACCESS_KEY` | `<Secret access key of the deployer>` | For authenticating `aws` CLI. | + | `AWS_DEFAULT_REGION` | `us-east-2` | For authenticating `aws` CLI. | + | `CI_AWS_ECS_CLUSTER` | `ecs-demo` | The ECS cluster is accessed by `production_ecs` job. | + | `CI_AWS_ECS_SERVICE` | `ecs_demo` | The ECS service of the cluster is updated by `production_ecs` job. Ensure that this variable is scoped to the appropriate environment (`production`, `staging`, `review/*`). | + | `CI_AWS_ECS_TASK_DEFINITION` | `ecs_demo` | The ECS task definition is updated by `production_ecs` job. | ### Make a change to the demo application diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index 3e77e8c6c25..16bfced9c34 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Deploy to AWS from GitLab CI/CD **(FREE)** +# Deploy to AWS from GitLab CI/CD **(FREE ALL)** GitLab provides Docker images with the libraries and tools you need to deploy to AWS. You can reference these images in your CI/CD pipeline. diff --git a/doc/ci/cloud_services/aws/index.md b/doc/ci/cloud_services/aws/index.md index b1148d3a258..f0183fc7ba2 100644 --- a/doc/ci/cloud_services/aws/index.md +++ b/doc/ci/cloud_services/aws/index.md @@ -4,7 +4,7 @@ group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Configure OpenID Connect in AWS to retrieve temporary credentials **(FREE)** +# Configure OpenID Connect in AWS to retrieve temporary credentials **(FREE ALL)** WARNING: `CI_JOB_JWT_V2` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) diff --git a/doc/ci/cloud_services/azure/index.md b/doc/ci/cloud_services/azure/index.md index b921dabc4e2..fa8ff506dd0 100644 --- a/doc/ci/cloud_services/azure/index.md +++ b/doc/ci/cloud_services/azure/index.md @@ -4,7 +4,7 @@ group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Configure OpenID Connect in Azure to retrieve temporary credentials **(FREE)** +# Configure OpenID Connect in Azure to retrieve temporary credentials **(FREE ALL)** WARNING: `CI_JOB_JWT_V2` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) diff --git a/doc/ci/cloud_services/google_cloud/index.md b/doc/ci/cloud_services/google_cloud/index.md index d99b50b5013..f2318b818d8 100644 --- a/doc/ci/cloud_services/google_cloud/index.md +++ b/doc/ci/cloud_services/google_cloud/index.md @@ -4,7 +4,7 @@ group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Configure OpenID Connect with GCP Workload Identity Federation **(FREE)** +# Configure OpenID Connect with GCP Workload Identity Federation **(FREE ALL)** WARNING: `CI_JOB_JWT_V2` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md index eadf0656d48..6896fffcce7 100644 --- a/doc/ci/cloud_services/index.md +++ b/doc/ci/cloud_services/index.md @@ -4,7 +4,7 @@ group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Connect to cloud services **(FREE)** +# Connect to cloud services **(FREE ALL)** > - `CI_JOB_JWT` variable for reading secrets from Vault [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207125) in GitLab 12.10. > - `CI_JOB_JWT_V2` variable to support additional OIDC providers [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346737) in GitLab 14.7. @@ -95,9 +95,9 @@ The condition is validated against the JWT to create a trust specifically agains - Subject or `sub`: A concatenation of metadata describing the GitLab CI/CD workflow including the group, project, branch, and tag. The `sub` field is in the following format: - `project_path:{group}/{project}:ref_type:{type}:ref:{branch_name}` -| Filter type | Example | -| ------------------------------------ | ------------------------------------------------------------ | -| Filter to main branch | `project_path:mygroup/myproject:ref_type:branch:ref:main` | +| Filter type | Example | +|--------------------------------------|---------| +| Filter to main branch | `project_path:mygroup/myproject:ref_type:branch:ref:main` | | Filter to any branch | Wildcard supported. `project_path:mygroup/myproject:ref_type:branch:ref:*` | | Filter to specific project | `project_path:mygroup/myproject:ref_type:branch:ref:main` | | Filter to all projects under a group | Wildcard supported. `project_path:mygroup/*:ref_type:branch:ref:main` | diff --git a/doc/ci/components/index.md b/doc/ci/components/index.md index ae35b3779c3..4a739bdfcf6 100644 --- a/doc/ci/components/index.md +++ b/doc/ci/components/index.md @@ -7,12 +7,9 @@ type: reference # CI/CD Components (Experimental) -> Introduced in GitLab 16.0 as an [experimental feature](../../policy/experiment-beta-support.md). - -FLAG: -On self-managed GitLab, by default this feature is not available. -To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `ci_namespace_catalog_experimental`. -On GitLab.com, this feature is not available. This feature is not ready for production use. +> - Introduced as an [experimental feature](../../policy/experiment-beta-support.md) in GitLab 16.0, [with a flag](../../administration/feature_flags.md) named `ci_namespace_catalog_experimental`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/groups/gitlab-org/-/epics/9897) in GitLab 16.2. +> - [Feature flag `ci_namespace_catalog_experimental` removed.](https://gitlab.com/gitlab-org/gitlab/-/issues/394772) in GitLab 16.3. This feature is an experimental feature and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/9897) to track future work. Tell us about your use case by leaving comments in the epic. @@ -339,7 +336,7 @@ We recommend adopting at least the `MAJOR.MINOR` format. For example: `2.1`, `1.0.0`, `1.0.0-alpha`, `2.1.3`, `3.0.0-rc.1`. -## CI/CD Catalog +## CI/CD Catalog **(PREMIUM ALL)** The CI/CD Catalog is a list of [components repositories](#components-repository), each containing resources that you can add to your CI/CD pipelines. @@ -367,7 +364,7 @@ This action is not reversible. Any existing CI template, that you share with other projects via `include:` syntax, can be converted to a CI component. -1. Decide whether you want the component to be part of an existing [components repository](#components-repository), +1. Decide whether you want the component to be part of an existing [components repository](#components-repository), if you want to logically group components together. Create and setup a [components repository](#components-repository) otherwise. 1. Create a YAML file in the components repository according to the expected [directory structure](#directory-structure). 1. Copy the content of the template YAML file into the new component YAML file. diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index e3bbe5b66c7..a1163a3acff 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Directed Acyclic Graph **(FREE)** +# Directed Acyclic Graph **(FREE ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47063) in GitLab 12.2. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/206902) in GitLab 12.10. @@ -36,8 +36,8 @@ Consider a monorepo as follows: It has a pipeline that looks like the following: -| build | test | deploy | -| ----- | ---- | ------ | +| build | test | deploy | +|-----------|----------|--------| | `build_a` | `test_a` | `deploy_a` | | `build_b` | `test_b` | `deploy_b` | | `build_c` | `test_c` | `deploy_c` | diff --git a/doc/ci/docker/index.md b/doc/ci/docker/index.md index 162d67286ad..49a9e1f8eea 100644 --- a/doc/ci/docker/index.md +++ b/doc/ci/docker/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index --- -# Docker integration **(FREE)** +# Docker integration **(FREE ALL)** There are two primary ways to incorporate [Docker](https://www.docker.com) into your CI/CD workflow: diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 5e8f2b36620..fb38de0f7de 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: concepts, howto --- -# Use Docker to build Docker images **(FREE)** +# Use Docker to build Docker images **(FREE ALL)** You can use GitLab CI/CD with Docker to create Docker images. For example, you can create a Docker image of your application, @@ -37,7 +37,7 @@ the Docker commands, but needs permission to do so. ```shell sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ + --url "https://gitlab.com/" \ --registration-token REGISTRATION_TOKEN \ --executor shell \ --description "My Runner" @@ -91,7 +91,7 @@ You should use Docker-in-Docker with TLS enabled, which is supported by [GitLab.com shared runners](../runners/index.md). You should always pin a specific version of the image, like `docker:20.10.16`. -If you use a tag like `docker:stable`, you have no control over which version is used. +If you use a tag like `docker:latest`, you have no control over which version is used. This can cause incompatibility problems when new versions are released. #### Use the Docker executor with Docker-in-Docker @@ -117,7 +117,7 @@ To use Docker-in-Docker with TLS enabled: ```shell sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ + --url "https://gitlab.com/" \ --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ @@ -381,7 +381,7 @@ To mount `/var/run/docker.sock` while registering your runner, include the follo ```shell sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ + --url "https://gitlab.com/" \ --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index d3a2f4ece06..455731f6c65 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: concepts, howto --- -# Run your CI/CD jobs in Docker containers **(FREE)** +# Run your CI/CD jobs in Docker containers **(FREE ALL)** You can run your CI/CD jobs in separate, isolated Docker containers. diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index b7affe28984..568f4977c2f 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Use kaniko to build Docker images **(FREE)** +# Use kaniko to build Docker images **(FREE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45512) in GitLab 11.2. Requires GitLab Runner 11.2 and above. @@ -73,9 +73,11 @@ If you authenticate against the [Dependency Proxy](../../user/packages/dependenc you must add the corresponding CI/CD variables for authentication to the `config.json` file: ```yaml -- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"},\"$CI_DEPENDENCY_PROXY_SERVER\":{\"auth\":\"$(printf "%s:%s" ${CI_DEPENDENCY_PROXY_USER} "${CI_DEPENDENCY_PROXY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json +- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"},\"$(echo -n $CI_DEPENDENCY_PROXY_SERVER | awk -F[:] '{print $1}')\":{\"auth\":\"$(printf "%s:%s" ${CI_DEPENDENCY_PROXY_USER} "${CI_DEPENDENCY_PROXY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json ``` +This command strips the port, for example `:443`, from `CI_DEPENDENCY_PROXY_SERVER`, so you don't have to include it when referencing images. + ### Building an image with kaniko behind a proxy If you use a custom GitLab Runner behind an http(s) proxy, kaniko needs to be set diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index e332b040fbc..395a07f48c8 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Disabling GitLab CI/CD **(FREE)** +# Disabling GitLab CI/CD **(FREE ALL)** GitLab CI/CD is enabled by default on all new projects. If you use an external CI/CD server like Jenkins or Drone CI, you can diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index 2a9b381c517..2933b25e09b 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w description: Require approvals prior to deploying to a Protected Environment --- -# Deployment approvals **(PREMIUM)** +# Deployment approvals **(PREMIUM ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343864) in GitLab 14.7 with a flag named `deployment_approvals`. Disabled by default. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/347342) in GitLab 14.8. diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index e15e09b27c1..b34fc61ba66 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -4,7 +4,7 @@ group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Deployment safety **(FREE)** +# Deployment safety **(FREE ALL)** [Deployment jobs](../jobs/index.md#deployment-jobs) are a specific kind of CI/CD job. They can be more sensitive than other jobs in a pipeline, @@ -93,15 +93,17 @@ When an older deployment job is manual, the play button is disabled with a messa Job age is determined by the job start time, not the commit time, so a newer commit can be prevented in some circumstances. -### How to rollback to an outdated deployment +### Job retries for rollback deployments -> In GitLab 15.6, [rollback via job retry was introduced back](https://gitlab.com/gitlab-org/gitlab/-/issues/378359). +> - Rollback via job retry [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378359) in GitLab 15.6. +> - Job retries for rollback deployments checkbox [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/410427) in GitLab 16.3. -In some cases, you need to rollback to an outdated deployment. -This feature explicitly allows rollback via [Environment Rollback](index.md#environment-rollback), -so that you can quickly rollback in an urgent case. +You might need to quickly roll back to a stable, outdated deployment. +By default, pipeline job retries for [deployment rollback](index.md#environment-rollback) are enabled. -Alternatively, you can run a new pipeline with a previous commit. It contains newer deployment jobs than the latest deployment. +To disable pipeline retries, clear the **Allow job retries for rollback deployments** checkbox. You should disable pipeline retries in sensitive projects. + +When a rollback is required, you must run a new pipeline with a previous commit. ### Example diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md index ce219e5d746..e98205f45b9 100644 --- a/doc/ci/environments/environments_dashboard.md +++ b/doc/ci/environments/environments_dashboard.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Environments Dashboard **(PREMIUM)** +# Environments Dashboard **(PREMIUM ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3713) in GitLab 12.5. diff --git a/doc/ci/environments/external_deployment_tools.md b/doc/ci/environments/external_deployment_tools.md index 37c4cf05f13..ac31d196887 100644 --- a/doc/ci/environments/external_deployment_tools.md +++ b/doc/ci/environments/external_deployment_tools.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Track deployments of an external deployment tool **(FREE)** +# Track deployments of an external deployment tool **(FREE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22513) in GitLab 12.5. diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md index c032ae3be60..a498e525b54 100644 --- a/doc/ci/environments/incremental_rollouts.md +++ b/doc/ci/environments/incremental_rollouts.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: concepts, howto --- -# Incremental rollouts with GitLab CI/CD **(FREE)** +# Incremental rollouts with GitLab CI/CD **(FREE ALL)** When rolling out changes to your application, it is possible to release production changes to only a portion of your Kubernetes pods as a risk mitigation strategy. By releasing diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 689f92723ee..c91018980f0 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Environments and deployments **(FREE)** +# Environments and deployments **(FREE ALL)** Environments describe where code is deployed. @@ -343,7 +343,7 @@ the job's `script`. If there is a problem with a deployment, you can retry it or roll it back. -To retry or rollback a deployment: +To retry or roll back a deployment: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Operate > Environments**. @@ -355,7 +355,7 @@ To retry or rollback a deployment: NOTE: If you have [prevented outdated deployment jobs](deployment_safety.md#prevent-outdated-deployment-jobs) in your project, the rollback buttons might be hidden or disabled. -In this case, see [how to rollback to an outdated deployment](deployment_safety.md#how-to-rollback-to-an-outdated-deployment). +In this case, see [job retries for rollback deployments](deployment_safety.md#job-retries-for-rollback-deployments). ### Environment URL @@ -729,7 +729,7 @@ or human error can cause major issues with an environment. Things like: You can use [incident management](../../operations/incident_management/index.md) to get alerts when there are critical issues that need immediate attention. -#### View the latest alerts for environments **(ULTIMATE)** +#### View the latest alerts for environments **(ULTIMATE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214634) in GitLab 13.4. @@ -745,7 +745,7 @@ longer visible on the environments page. If the alert requires a [rollback](#retry-or-roll-back-a-deployment), you can select the deployment tab from the environment page and select which deployment to roll back to. -#### Auto Rollback **(ULTIMATE)** +#### Auto Rollback **(ULTIMATE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35404) in GitLab 13.7. diff --git a/doc/ci/environments/kubernetes_dashboard.md b/doc/ci/environments/kubernetes_dashboard.md index ee16294e654..98b6731e469 100644 --- a/doc/ci/environments/kubernetes_dashboard.md +++ b/doc/ci/environments/kubernetes_dashboard.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Dashboard for Kubernetes (Beta) **(FREE)** +# Dashboard for Kubernetes (Beta) **(FREE ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390769) in GitLab 16.1, with [flags](../../administration/feature_flags.md) named `environment_settings_to_graphql`, `kas_user_access`, `kas_user_access_project`, and `expose_authorized_cluster_agents`. This feature is in [Beta](../../policy/experiment-beta-support.md#beta). > - Feature flag `environment_settings_to_graphql` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124177) in GitLab 16.2. @@ -20,6 +20,13 @@ For Flux users, the synchronization status of a given environment is not display ## Configure a dashboard +> - Filtering resources by namespace [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/403618) in GitLab 16.2 [with a flag](../../administration/feature_flags.md) named `kubernetes_namespace_for_environment`. Disabled by default. +> - Filtering resources by namespace [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/127043) in GitLab 16.3. Feature flag `kubernetes_namespace_for_environment` removed. +> - Selecting the related Flux resource [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/128857) in GitLab 16.3 [with a flag](../../administration/feature_flags.md) named `flux_resource_for_environment`. Disabled by default. + +FLAG: +On self-managed GitLab, by default selecting a Flux resource is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `flux_resource_for_environment`. On GitLab.com, this feature is not available. + Configure a dashboard to use it for a given environment. You can configure dashboard for an environment that already exists, or add one when you create an environment. @@ -36,6 +43,8 @@ Prerequisites: 1. Select the environment to be associated with the Kubernetes. 1. Select **Edit**. 1. Select a GitLab agent for Kubernetes. +1. Optional. From the **Kubernetes namespace** dropdown list, select a namespace. +1. Optional. From the **Flux resource** dropdown list, select a Flux resource. 1. Select **Save**. ### The environment doesn't exist @@ -45,6 +54,8 @@ Prerequisites: 1. Select **New environment**. 1. Complete the **Name** field. 1. Select a GitLab agent for Kubernetes. +1. Optional. From the **Kubernetes namespace** dropdown list, select a namespace. +1. Optional. From the **Flux resource** dropdown list, select a Flux resource. 1. Select **Save**. ## View a dashboard @@ -56,6 +67,27 @@ To view a configured dashboard: 1. Expand the environment associated with GitLab agent for Kubernetes. 1. Expand **Kubernetes overview**. +### Flux sync status + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/391581) in GitLab 16.3. +> - Customizing the name of the Flux resource [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/128857) in GitLab 16.3 [with a flag](../../administration/feature_flags.md) named `flux_resource_for_environment`. Disabled by default. + +A dashboard displays the sync status of your Flux deployments. + +| Status | Description | +|---------|-------------| +| **Reconciled** | The deployment successfully reconciled with its environment. | +| **Reconciling** | A reconciliation is in progress. | +| **Stalled** | A reconciliation is stuck because of an error that cannot be resolved without human intervention. | +| **Failed** | The deployment couldn't reconcile because of an unrecoverable error. | +| **Unknown** | The sync status of the deployment couldn't be retrieved. | +| **Unavailable** | The `Kustomization` or `HelmRelease` resource couldn't be retrieved. | + +Deployments rely on Flux `Kustomization` and `HelmRelease` resources to gather +the status of a given environment, which requires a namespace to be configured for the environment. +By default, GitLab searches the `Kustomization` and `HelmRelease` resources for the name of the project slug. +You can customize the name GitLab looks for in the environment settings. + ## Troubleshooting When working with the Dashboard for Kubernetes, you might encounter the following issues. diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 61b59ceedb2..fd789ea798d 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -4,7 +4,7 @@ group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Protected environments **(PREMIUM)** +# Protected environments **(PREMIUM ALL)** [Environments](../environments/index.md) can be used for both testing and production reasons. @@ -33,6 +33,7 @@ To protect an environment: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > CI/CD**. 1. Expand **Protected environments**. +1. Select **Protect an environment**. 1. From the **Environment** list, select the environment you want to protect. 1. In the **Allowed to deploy** list, select the role, users, or groups you want to give deploy access to. Keep in mind that: @@ -50,7 +51,7 @@ To protect an environment: - **Developers**: Allows access to all of the project's users with the Maintainer and Developer role. - You can select groups that are already associated with the project only. - Users must have at least the Developer role to appear in - the **Allowed to deploy** list. + the **Approvers** list. 1. In the **Approval rules** section: diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index f45c60bdd1f..737c95cf747 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -5,13 +5,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Authenticating and reading secrets with HashiCorp Vault (Deprecated) **(PREMIUM)** +# Authenticating and reading secrets with HashiCorp Vault **(PREMIUM ALL)** WARNING: -Authenticating with HashiCorp Vault by using `CI_JOB_JWT` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) -and the token is scheduled to be removed in GitLab 16.5. This change is a breaking change. -Use [ID tokens to authenticate with HashiCorp Vault](../../secrets/id_token_authentication.md#automatic-id-token-authentication-with-hashicorp-vault) -instead. +Authenticating with `CI_JOB_JWT` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) +and the token is scheduled to be removed in GitLab 16.5. Use +[ID tokens to authenticate with HashiCorp Vault](../../secrets/id_token_authentication.md#automatic-id-token-authentication-with-hashicorp-vault) +instead, as demonstrated on this page. This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD. @@ -33,30 +33,30 @@ ID tokens are JSON Web Tokens (JWTs) used for OIDC authentication with third-par The following fields are included in the JWT: -| Field | When | Description | -|-------------------------|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `jti` | Always | Unique identifier for this token | -| `iss` | Always | Issuer, the domain of your GitLab instance | -| `iat` | Always | Issued at | -| `nbf` | Always | Not valid before | -| `exp` | Always | Expires at | -| `sub` | Always | Subject (job ID) | -| `namespace_id` | Always | Use this to scope to group or user level namespace by ID | -| `namespace_path` | Always | Use this to scope to group or user level namespace by path | -| `project_id` | Always | Use this to scope to project by ID | -| `project_path` | Always | Use this to scope to project by path | -| `user_id` | Always | ID of the user executing the job | -| `user_login` | Always | Username of the user executing the job | -| `user_email` | Always | Email of the user executing the job | -| `pipeline_id` | Always | ID of this pipeline | -| `pipeline_source` | Always | [Pipeline source](../../jobs/job_control.md#common-if-clauses-for-rules) | -| `job_id` | Always | ID of this job | -| `ref` | Always | Git ref for this job | -| `ref_type` | Always | Git ref type, either `branch` or `tag` | +| Field | When | Description | +|-------------------------|------------------------------|-------------| +| `jti` | Always | Unique identifier for this token | +| `iss` | Always | Issuer, the domain of your GitLab instance | +| `iat` | Always | Issued at | +| `nbf` | Always | Not valid before | +| `exp` | Always | Expires at | +| `sub` | Always | Subject (job ID) | +| `namespace_id` | Always | Use this to scope to group or user level namespace by ID | +| `namespace_path` | Always | Use this to scope to group or user level namespace by path | +| `project_id` | Always | Use this to scope to project by ID | +| `project_path` | Always | Use this to scope to project by path | +| `user_id` | Always | ID of the user executing the job | +| `user_login` | Always | Username of the user executing the job | +| `user_email` | Always | Email of the user executing the job | +| `pipeline_id` | Always | ID of this pipeline | +| `pipeline_source` | Always | [Pipeline source](../../jobs/job_control.md#common-if-clauses-for-rules) | +| `job_id` | Always | ID of this job | +| `ref` | Always | Git ref for this job | +| `ref_type` | Always | Git ref type, either `branch` or `tag` | | `ref_path` | Always | Fully qualified ref for the job. For example, `refs/heads/main`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119075) in GitLab 16.0. | -| `ref_protected` | Always | `true` if this Git ref is protected, `false` otherwise | -| `environment` | Job specifies an environment | Environment this job specifies ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9) | -| `environment_protected` | Job specifies an environment | `true` if specified environment is protected, `false` otherwise ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9) | +| `ref_protected` | Always | `true` if this Git ref is protected, `false` otherwise | +| `environment` | Job specifies an environment | Environment this job specifies ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9) | +| `environment_protected` | Job specifies an environment | `true` if specified environment is protected, `false` otherwise ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294440) in GitLab 13.9) | | `deployment_tier` | Job specifies an environment | [Deployment tier](../../environments/index.md#deployment-tier-of-environments) of environment this job specifies ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363590) in GitLab 15.2) | Example JWT payload: @@ -249,7 +249,7 @@ Now, configure the JWT Authentication method: ```shell $ vault write auth/jwt/config \ jwks_url="https://gitlab.example.com/-/jwks" \ - bound_issuer="gitlab.example.com" + bound_issuer="https://gitlab.example.com" ``` [`bound_issuer`](https://developer.hashicorp.com/vault/api-docs/auth/jwt#bound_issuer) specifies that only a JWT with the issuer (that is, the `iss` claim) set to `gitlab.example.com` can use this method to authenticate, and that the JWKS endpoint (`https://gitlab.example.com/-/jwks`) should be used to validate the token. diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 5d1ae0048af..dfdaa1866c3 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Running Composer and npm scripts with deployment via SCP in GitLab CI/CD **(FREE)** +# Running Composer and npm scripts with deployment via SCP in GitLab CI/CD **(FREE ALL)** This guide covers the building of dependencies of a PHP project while compiling assets via an npm script using [GitLab CI/CD](../../index.md). diff --git a/doc/ci/examples/deployment/index.md b/doc/ci/examples/deployment/index.md index 793e60d517c..f9360faedac 100644 --- a/doc/ci/examples/deployment/index.md +++ b/doc/ci/examples/deployment/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Using Dpl as a deployment tool **(FREE)** +# Using Dpl as a deployment tool **(FREE ALL)** [Dpl](https://github.com/travis-ci/dpl) (pronounced like the letters D-P-L) is a deploy tool made for continuous deployment that's developed and used by Travis CI, but can also be diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index ee8b6bb30b6..10ec18a6147 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -9,7 +9,7 @@ description: 'Confidence checking your entire app every time a new feature is ad <!-- vale off --> -# End-to-end testing with GitLab CI/CD and WebdriverIO **(FREE)** +# End-to-end testing with GitLab CI/CD and WebdriverIO **(FREE ALL)** [Review Apps](../../review_apps/index.md) are great: for every merge request (or branch, for that matter), the new code can be copied and deployed to a fresh production-like live @@ -154,7 +154,7 @@ to interact with your application, so we need to install and run them. Furthermore, WebdriverIO uses Selenium as a common interface to control different browsers, so we need to install and run Selenium as well. Luckily, the Selenium project provides the Docker images for Firefox [standalone-firefox](https://hub.docker.com/r/selenium/standalone-firefox/) and -and for Chrome [standalone-chrome](https://hub.docker.com/r/selenium/standalone-chrome/). +and for Chrome [standalone-chrome](https://hub.docker.com/r/selenium/standalone-chrome/). (Since Safari and Internet Explorer/Edge are not open source and not available for Linux, we are unfortunately unable to use those in GitLab CI/CD). diff --git a/doc/ci/examples/index.md b/doc/ci/examples/index.md index 514817adc91..aa3be8f2ab0 100644 --- a/doc/ci/examples/index.md +++ b/doc/ci/examples/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index --- -# GitLab CI/CD Examples **(FREE)** +# GitLab CI/CD examples **(FREE ALL)** This page contains links to a variety of examples that can help you understand how to implement [GitLab CI/CD](../index.md) for your specific use case. diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 14d8e0fc72f..ea936b66d31 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -8,7 +8,7 @@ author_gitlab: mehranrasulian <!-- vale off --> -# Test and deploy Laravel applications with GitLab CI/CD and Envoy **(FREE)** +# Test and deploy Laravel applications with GitLab CI/CD and Envoy **(FREE ALL)** ## Introduction @@ -66,7 +66,7 @@ This test will be used later for continuously testing our app with GitLab CI/CD. ### Push to GitLab Since we have our app up and running locally, it's time to push the codebase to our remote repository. -Let's create [a new project](../../../user/project/index.md#create-a-project) in GitLab named `laravel-sample`. +Let's create [a new project](../../../user/project/index.md) in GitLab named `laravel-sample`. After that, follow the command line instructions displayed on the project's homepage to initiate the repository on our machine and push the first commit. ```shell @@ -376,7 +376,7 @@ These are persistent data and will be shared to every new release. Now, we would need to deploy our app by running `envoy run deploy`, but it won't be necessary since GitLab can handle that for us with CI's [environments](../../environments/index.md), which will be described [later](#setting-up-gitlab-cicd) in this tutorial. Now it's time to commit [Envoy.blade.php](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Envoy.blade.php) and push it to the `main` branch. -To keep things simple, we commit directly to `main`, without using [feature-branches](../../../topics/gitlab_flow.md#github-flow-as-a-simpler-alternative) since collaboration is beyond the scope of this tutorial. +To keep things simple, we commit directly to `main`, without using feature branches, since collaboration is beyond the scope of this tutorial. In a real world project, teams may use [Issue Tracker](../../../user/project/issues/index.md) and [merge requests](../../../user/project/merge_requests/index.md) to move their code across branches: ```shell diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 244c1e379da..0e0d40f12f6 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Testing PHP projects **(FREE)** +# Testing PHP projects **(FREE ALL)** This guide covers basic building instructions for PHP projects. diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index 00260199179..b284e2b2dc1 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -4,7 +4,7 @@ group: Package Registry info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Publish npm packages to the GitLab Package Registry using semantic-release **(FREE)** +# Publish npm packages to the GitLab Package Registry using semantic-release **(FREE ALL)** This guide demonstrates how to automatically publish npm packages to the [GitLab Package Registry](../../user/packages/npm_registry/index.md) by using [semantic-release](https://github.com/semantic-release/semantic-release). diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 31cb6bc9946..2561ab6465e 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Using Git submodules with GitLab CI/CD **(FREE)** +# Using Git submodules with GitLab CI/CD **(FREE ALL)** Use [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules) to keep a Git repository as a subdirectory of another Git repository. You can clone another @@ -116,7 +116,7 @@ To make submodules work correctly in CI/CD jobs: If you use the [`CI_JOB_TOKEN`](jobs/ci_job_token.md) to clone a submodule in a pipeline job, the user executing the job must be assigned to a role that has [permission](../user/permissions.md#gitlab-cicd-permissions) to trigger a pipeline -in the upstream submodule project. +in the upstream submodule project. Additionally, [CI/CD job token access](jobs/ci_job_token.md#configure-cicd-job-token-access) must be properly configured in the upstream submodule project. ## Troubleshooting diff --git a/doc/ci/index.md b/doc/ci/index.md index a3106a2475c..21fe122f503 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -6,7 +6,7 @@ description: "Learn how to use GitLab CI/CD, the GitLab built-in Continuous Inte type: index --- -# GitLab CI/CD **(FREE)** +# GitLab CI/CD **(FREE ALL)** GitLab CI/CD is a tool for software development using the continuous methodologies: @@ -104,7 +104,7 @@ GitLab CI/CD features, grouped by DevOps stage, include: | [Dynamic Application Security Testing](../user/application_security/dast/index.md) | Test your application's runtime behavior for vulnerabilities. | | [Dependency Scanning](../user/application_security/dependency_scanning/index.md) | Analyze your dependencies for known vulnerabilities. | | [Infrastructure as Code scanning](../user/application_security/iac_scanning/index.md) | Scan your IaC configuration files for known vulnerabilities. | -| [License Compliance](../user/compliance/license_compliance/index.md) | Search your project dependencies for their licenses. | +| [License Scanning](../user/compliance/license_scanning_of_cyclonedx_files/index.md) | Search your project dependencies for their licenses. | Search your project dependencies for their licenses. | | [Secret Detection](../user/application_security/secret_detection/index.md) | Search your application's source code for secrets. | | [Static Application Security Testing](../user/application_security/sast/index.md) | Test your application's source code for known vulnerabilities. | | [Web API fuzz testing](../user/application_security/api_fuzzing/index.md) | Test your application's API behavior by providing randomized input. | diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index a7923cb84a0..1125061d830 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Interactive web terminals **(FREE)** +# Interactive web terminals **(FREE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/50144) in GitLab 11.3. @@ -61,7 +61,7 @@ improving this behavior. Sometimes, when a job is running, things don't go as you would expect, and it would be helpful if one can have a shell to aid debugging. When a job is running, on the right panel you can see a button `debug` that opens the terminal -for the current job. +for the current job. Only the person who started a job can debug it.  diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index 895aa8551d7..35692ed7b7e 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -6,7 +6,7 @@ description: "An overview of Continuous Integration, Continuous Delivery, and Co type: concepts --- -# CI/CD concepts **(FREE)** +# CI/CD concepts **(FREE ALL)** With the continuous method of software development, you continuously build, test, and deploy iterative code changes. This iterative process helps reduce diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index c2fe3071b52..dee078c21e0 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -4,7 +4,7 @@ group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# GitLab CI/CD job token **(FREE)** +# GitLab CI/CD job token **(FREE ALL)** When a pipeline job is about to run, GitLab generates a unique token and injects it as the [`CI_JOB_TOKEN` predefined variable](../variables/predefined_variables.md). @@ -105,6 +105,8 @@ access is needed. ### Disable the job token scope allowlist +> **Allow access to this project with a CI_JOB_TOKEN** setting [renamed to **Limit access _to_ this project**](https://gitlab.com/gitlab-org/gitlab/-/issues/411406) in GitLab 16.3. + WARNING: It is a security risk to disable the allowlist. A malicious user could try to compromise a pipeline created in an unauthorized project. If the pipeline was created by one of @@ -122,28 +124,30 @@ To disable the job token scope allowlist: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > CI/CD**. 1. Expand **Token Access**. -1. Toggle **Allow access to this project with a CI_JOB_TOKEN** to disabled. +1. Toggle **Limit access _to_ this project** to disabled. Enabled by default in new projects. You can also disable the allowlist [with the API](../../api/graphql/reference/index.md#mutationprojectcicdsettingsupdate). ### Add a project to the job token scope allowlist +> **Allow access to this project with a CI_JOB_TOKEN** setting [renamed to **Limit access _to_ this project**](https://gitlab.com/gitlab-org/gitlab/-/issues/411406) in GitLab 16.3. + You can add projects to the allowlist for a project. Projects added to the allowlist can make API calls from running pipelines by using the CI/CD job token. Prerequisite: -- You must have at least the Maintainer role in the current project and at least - the Guest role in the allowed project. -- You must not have more than 100 projects added to the allowlist. +- You must have at least the Maintainer role in the current project. If the allowed project + is internal or private, you must have at least the Guest role in that project. +- You must not have more than 200 projects added to the allowlist. To add a project: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > CI/CD**. 1. Expand **Token Access**. -1. Verify **Allow access to this project with a CI_JOB_TOKEN** is enabled. +1. Verify **Limit access _to_ this project** is enabled. 1. Under **Allow CI job tokens from the following projects to access this project**, add projects to the allowlist. @@ -176,20 +180,22 @@ If project `B` is public or internal, you do not need to add ### Configure the job token scope +> **Limit CI_JOB_TOKEN access** setting [renamed to **Limit access _from_ this project**](https://gitlab.com/gitlab-org/gitlab/-/issues/411406) in GitLab 16.3. + Prerequisite: -- You must not have more than 100 projects added to the token's scope. +- You must not have more than 200 projects added to the token's scope. To configure the job token scope: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > CI/CD**. 1. Expand **Token Access**. -1. Toggle **Limit CI_JOB_TOKEN access** to enabled. +1. Toggle **Limit access _from_ this project** to enabled. 1. Optional. Add existing projects to the token's access scope. The user adding a project must have the Maintainer role in both projects. -## Download an artifact from a different pipeline **(PREMIUM)** +## Download an artifact from a different pipeline **(PREMIUM ALL)** > `CI_JOB_TOKEN` for artifacts download with the API was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/2346) in GitLab 9.5. diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index a0c0fc43502..762fb717505 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -4,7 +4,7 @@ group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Jobs **(FREE)** +# Jobs **(FREE ALL)** Pipeline configuration begins with jobs. Jobs are the most fundamental element of a `.gitlab-ci.yml` file. diff --git a/doc/ci/jobs/job_artifacts.md b/doc/ci/jobs/job_artifacts.md index 3dd43a9760c..b080a4fd1e9 100644 --- a/doc/ci/jobs/job_artifacts.md +++ b/doc/ci/jobs/job_artifacts.md @@ -4,7 +4,7 @@ group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Job artifacts **(FREE)** +# Job artifacts **(FREE ALL)** Jobs can output an archive of files and directories. This output is known as a job artifact. diff --git a/doc/ci/jobs/job_artifacts_troubleshooting.md b/doc/ci/jobs/job_artifacts_troubleshooting.md index f5951350085..ebed347e966 100644 --- a/doc/ci/jobs/job_artifacts_troubleshooting.md +++ b/doc/ci/jobs/job_artifacts_troubleshooting.md @@ -23,7 +23,7 @@ the keyword reference for information on how to fetch artifacts with these keywo ## Job artifacts use too much disk space If job artifacts are using too much disk space, see the -[job artifacts administration documentation](../../administration/job_artifacts.md#job-artifacts-using-too-much-disk-space). +[job artifacts administration documentation](../../administration/job_artifacts_troubleshooting.md#job-artifacts-using-too-much-disk-space). ## Error message `No files to upload` diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 09c7500a883..770e1d38297 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -4,7 +4,7 @@ group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Choose when to run jobs **(FREE)** +# Choose when to run jobs **(FREE ALL)** When a new pipeline starts, GitLab checks the pipeline configuration to determine which jobs should run in that pipeline. You can configure jobs to run depending on @@ -259,20 +259,20 @@ runs in all cases except merge requests. For behavior similar to the [`only`/`except` keywords](../yaml/index.md#only--except), you can check the value of the `$CI_PIPELINE_SOURCE` variable: -| Value | Description | -|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `api` | For pipelines triggered by the [pipelines API](../../api/pipelines.md#create-a-new-pipeline). | -| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/index.md) command. | -| `external` | When you use CI services other than GitLab. | -| `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). | +| Value | Description | +|-------------------------------|-------------| +| `api` | For pipelines triggered by the [pipelines API](../../api/pipelines.md#create-a-new-pipeline). | +| `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/index.md) command. | +| `external` | When you use CI services other than GitLab. | +| `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). | | `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/merged_results_pipelines.md), and [merge trains](../pipelines/merge_trains.md). | -| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | +| `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../pipelines/downstream_pipelines.md#parent-child-pipelines) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | | `pipeline` | For [multi-project pipelines](../pipelines/downstream_pipelines.md#multi-project-pipelines) created by [using the API with `CI_JOB_TOKEN`](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api), or the [`trigger`](../yaml/index.md#trigger) keyword. | -| `push` | For pipelines triggered by a `git push` event, including for branches and tags. | -| `schedule` | For [scheduled pipelines](../pipelines/schedules.md). | -| `trigger` | For pipelines created by using a [trigger token](../triggers/index.md#configure-cicd-jobs-to-run-in-triggered-pipelines). | -| `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **Build > Pipelines** section. | -| `webide` | For pipelines created by using the [WebIDE](../../user/project/web_ide/index.md). | +| `push` | For pipelines triggered by a `git push` event, including for branches and tags. | +| `schedule` | For [scheduled pipelines](../pipelines/schedules.md). | +| `trigger` | For pipelines created by using a [trigger token](../triggers/index.md#configure-cicd-jobs-to-run-in-triggered-pipelines). | +| `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **Build > Pipelines** section. | +| `webide` | For pipelines created by using the [WebIDE](../../user/project/web_ide/index.md). | The following example runs the job as a manual job in scheduled pipelines or in push pipelines (to branches or tags), with `when: on_success` (default). It does not @@ -617,7 +617,7 @@ To run a manual job, you must have permission to merge to the assigned branch: You can also [add custom CI/CD variables when running a manual job](index.md#specifying-variables-when-running-manual-jobs). -### Protect manual jobs **(PREMIUM)** +### Protect manual jobs **(PREMIUM ALL)** Use [protected environments](../environments/protected_environments.md) to define a list of users authorized to run a manual job. You can authorize only @@ -832,6 +832,83 @@ deploy: Quotes around the `dependencies` entry are required. +## Specify a parallelized job using needs with multiple parallelized jobs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254821) in GitLab 16.3. + +You can use variables defined in [`needs:parallel:matrix`](../yaml/index.md#needsparallelmatrix) with multiple parallelized jobs. + +For example: + +```yaml +linux:build: + stage: build + script: echo "Building linux..." + parallel: + matrix: + - PROVIDER: aws + STACK: + - monitoring + - app1 + - app2 + +mac:build: + stage: build + script: echo "Building mac..." + parallel: + matrix: + - PROVIDER: [gcp, vultr] + STACK: [data, processing] + +linux:rspec: + stage: test + needs: + - job: linux:build + parallel: + matrix: + - PROVIDER: aws + - STACK: app1 + script: echo "Running rspec on linux..." + +mac:rspec: + stage: test + needs: + - job: mac:build + parallel: + matrix: + - PROVIDER: [gcp, vultr] + - STACK: [data] + script: echo "Running rspec on mac..." + +production: + stage: deploy + script: echo "Running production..." + environment: production +``` + +This example generates several jobs. The parallel jobs each have different values +for `PROVIDER` and `STACK`. + +- 3 parallel `linux:build` jobs: + - `linux:build: [aws, monitoring]` + - `linux:build: [aws, app1]` + - `linux:build: [aws, app2]` +- 4 parallel `mac:build` jobs: + - `mac:build: [gcp, data]` + - `mac:build: [gcp, processing]` + - `mac:build: [vultr, data]` + - `mac:build: [vultr, processing]` +- A `linux:rspec` job. +- A `production` job. + +The jobs have three paths of execution: + +- Linux path: The `linux:rspec` job runs as soon as the `linux:build: [aws, app1]` + job finishes, without waiting for `mac:build` to finish. +- macOS path: The `mac:rspec` job runs as soon as the `mac:build: [gcp, data]` and + `mac:build: [vultr, data]` jobs finish, without waiting for `linux:build` to finish. +- The `production` job runs as soon as all previous jobs finish. + ## Use predefined CI/CD variables to run jobs only in specific pipeline types You can use [predefined CI/CD variables](../variables/predefined_variables.md) to choose diff --git a/doc/ci/large_repositories/index.md b/doc/ci/large_repositories/index.md index 4b17f3354aa..da5ebe21341 100644 --- a/doc/ci/large_repositories/index.md +++ b/doc/ci/large_repositories/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Optimize GitLab for large repositories **(FREE)** +# Optimize GitLab for large repositories **(FREE ALL)** Large repositories consisting of more than 50k files in a worktree may require more optimizations beyond @@ -234,7 +234,6 @@ concurrent = 4 cache_dir = "/cache" environment = [ - "GIT_DEPTH=10", "GIT_CLONE_PATH=$CI_BUILDS_DIR/$CI_CONCURRENT_ID/$CI_PROJECT_NAME" ] diff --git a/doc/ci/lint.md b/doc/ci/lint.md index f4c8a377c31..0ca0469f5e9 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -4,7 +4,7 @@ group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Validate GitLab CI/CD configuration **(FREE)** +# Validate GitLab CI/CD configuration **(FREE ALL)** Use the CI Lint tool to check the validity of GitLab CI/CD configuration. You can validate the syntax from a `.gitlab-ci.yml` file or any other sample CI/CD configuration. diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index a05fe87013c..d70794639d1 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index, howto --- -# Migrating from CircleCI **(FREE)** +# Migrating from CircleCI **(FREE ALL)** If you are currently using CircleCI, you can migrate your CI/CD pipelines to [GitLab CI/CD](../introduction/index.md), and start making use of all its powerful features. Check out our diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index f3f14da037f..54ea3812029 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index, howto --- -# Migrating from Jenkins **(FREE)** +# Migrating from Jenkins **(FREE ALL)** A lot of GitLab users have successfully migrated to GitLab CI/CD from Jenkins. We've collected several resources here that you might find informative if you're just getting started. diff --git a/doc/ci/mobile_devops.md b/doc/ci/mobile_devops.md index a85de5e2a51..6969b3d4ef4 100644 --- a/doc/ci/mobile_devops.md +++ b/doc/ci/mobile_devops.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Mobile DevOps (Experimental) +# Mobile DevOps (Experiment) Use GitLab Mobile DevOps to quickly build, sign, and release native and cross-platform mobile apps for Android and iOS using GitLab CI/CD. Mobile DevOps is an experimental feature developed by @@ -41,10 +41,9 @@ test: ### iOS build environments -[GitLab SaaS runners on macOS](../ci/runners/saas/macos_saas_runner.md) are currently available in beta. +[GitLab SaaS runners on macOS](../ci/runners/saas/macos_saas_runner.md) are in Beta. -After you are granted access to the beta macOS runners, [choose an image](../ci/runners/saas/macos_saas_runner.md#supported-macos-images) -and add it to your `.gitlab-ci.yml` file. +[Choose an image](../ci/runners/saas/macos_saas_runner.md#supported-macos-images) to run a job on a macOS GitLab SaaS runner and add it to your `.gitlab-ci.yml` file. For example: @@ -89,7 +88,7 @@ For an overview, see [How to build and release an Android app to Google Play wit Run the following command to generate a keystore file if you don't already have one: ```shell -keytool -genkey -v -keystore release-keystore.jks -storepass password -alias release -keypass password -keyalg RSA -keysize 2048 -validity 10000 +keytool -genkey -v -keystore release-keystore.jks -storepass password -alias release -keypass password -keyalg RSA -keysize 2048 -validity 10000 ``` Next, put the keystore configuration in a file called `release-keystore.properties`, @@ -122,7 +121,7 @@ The next step is to configure Gradle to use the newly created keystore. In the a } ``` -1. Anywhere within the `android` block, add: +1. Anywhere in the `android` block, add: ```gradle signingConfigs { @@ -243,7 +242,7 @@ For example: ```ruby default_platform(:ios) - + platform :ios do desc "Build and sign the application for development" lane :build do @@ -299,7 +298,7 @@ to build and release Android apps. To enable the integration: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > Integrations**. 1. Select **Google Play**. -1. In **Enable integration**, select the **Active** checkbox. +1. Under **Enable integration**, select the **Active** checkbox. 1. In **Package name**, enter the package name of the app. For example, `com.gitlab.app_name`. 1. In **Service account key (.JSON)** drag or upload your key file. 1. Select **Save changes**. @@ -356,7 +355,7 @@ to build and release apps for iOS, iPadOS, macOS, tvOS, and watchOS. To enable t 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > Integrations**. 1. Select **Apple App Store**. -1. Turn on the **Active** toggle under **Enable Integration**. +1. Under **Enable integration**, select the **Active** checkbox. 1. Provide the Apple App Store Connect configuration information: - **Issuer ID**: You can find the Apple App Store Connect Issuer ID in the **Keys** section under **Users and Access** in the Apple App Store Connect portal. - **Key ID**: The key ID of the new private key that was just generated. diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index b6f30627b7e..5291a8abb49 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Pipeline Editor **(FREE)** +# Pipeline Editor **(FREE ALL)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4540) in GitLab 13.8. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/270059) in GitLab 13.10. diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index 6c7b00c827a..4e822cf3edd 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Compute quota **(PREMIUM)** +# Compute quota **(PREMIUM ALL)** > [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" or "compute minutes" in GitLab 16.1. @@ -152,7 +152,7 @@ For example, with a GitLab SaaS Premium license: If you use `13,000` compute minutes during the month, the next month your additional compute minutes become `2,000`. If you use `9,000` compute minutes during the month, your additional compute minutes remain the same. -If you bought additional compute minutes while on a trial subscription, those compute minutes are available after the trial ends or you upgrade to a paid plan. +Additional compute minutes bought on a trial subscription are available after the trial ends or upgrading to a paid plan. You can find pricing for additional compute minutes on the [GitLab Pricing page](https://about.gitlab.com/pricing/). @@ -269,8 +269,10 @@ GitLab SaaS runners have different cost factors, depending on the runner type (L | Linux OS amd64 | small | 1 | | Linux OS amd64 | medium | 2 | | Linux OS amd64 | large | 3 | +| Linux OS amd64 | xlarge | 6 | +| Linux OS amd64 | 2xlarge | 12 | | Linux OS amd64 + GPU-enabled | medium, GPU standard | 7 | -| macOS M1 | Medium | 6 | +| macOS M1 | medium | 6 (Beta) | | Windows Server | - | 1 (Beta) | ### Monthly reset of compute usage diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md index 686020fc17a..fca6e8407ef 100644 --- a/doc/ci/pipelines/downstream_pipelines.md +++ b/doc/ci/pipelines/downstream_pipelines.md @@ -4,7 +4,7 @@ group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Downstream pipelines **(FREE)** +# Downstream pipelines **(FREE ALL)** A downstream pipeline is any GitLab CI/CD pipeline triggered by another pipeline. Downstream pipelines run independently and concurrently to the upstream pipeline @@ -209,8 +209,9 @@ To trigger a child pipeline from a dynamically generated configuration file: - generated-config.yml ``` -1. Configure the trigger job to run after the job that generated the configuration file, - and set `include: artifact` to the generated artifact: +1. Configure the trigger job to run after the job that generated the configuration file. + Set `include: artifact` to the generated artifact, and set `include: job` to + the job that created the artifact: ```yaml child-pipeline: @@ -381,7 +382,7 @@ trigger_job: ::EndTabs -### View multi-project pipelines in pipeline graphs **(PREMIUM)** +### View multi-project pipelines in pipeline graphs **(PREMIUM ALL)** After you trigger a multi-project pipeline, the downstream pipeline displays to the right of the [pipeline graph](index.md#visualize-pipelines). @@ -389,7 +390,51 @@ to the right of the [pipeline graph](index.md#visualize-pipelines). In [pipeline mini graphs](index.md#pipeline-mini-graphs), the downstream pipeline displays to the right of the mini graph. -## Fetch artifacts from an upstream pipeline **(PREMIUM)** +## Fetch artifacts from an upstream pipeline **(PREMIUM ALL)** + +::Tabs + +:::TabTitle Parent-child pipeline + +Use [`needs:pipeline:job`](../yaml/index.md#needspipelinejob) to fetch artifacts from an +upstream pipeline: + +1. In the upstream pipeline, save the artifacts in a job with the [`artifacts`](../yaml/index.md#artifacts) + keyword, then trigger the downstream pipeline with a trigger job: + + ```yaml + build_artifacts: + stage: build + script: + - echo "This is a test artifact!" >> artifact.txt + artifacts: + paths: + - artifact.txt + + deploy: + stage: deploy + trigger: + include: + - local: path/to/child-pipeline.yml + variables: + PARENT_PIPELINE_ID: $CI_PIPELINE_ID + ``` + +1. Use `needs:pipeline:job` in a job in the downstream pipeline to fetch the artifacts. + + ```yaml + test: + stage: test + script: + - cat artifact.txt + needs: + - pipeline: $PARENT_PIPELINE_ID + job: build_artifacts + ``` + + Set `job` to the job in the upstream pipeline that created the artifacts. + +:::TabTitle Multi-project pipeline Use [`needs:project`](../yaml/index.md#needsproject) to fetch artifacts from an upstream pipeline: @@ -408,7 +453,7 @@ upstream pipeline: deploy: stage: deploy - trigger: my/downstream_project + trigger: my/downstream_project # Path to the project to trigger a pipeline in ``` 1. Use `needs:project` in a job in the downstream pipeline to fetch the artifacts. @@ -431,6 +476,8 @@ upstream pipeline: - `ref` to the branch. - `artifacts` to `true`. +::EndTabs + ### Fetch artifacts from an upstream merge request pipeline When you use `needs:project` to [pass artifacts to a downstream pipeline](#fetch-artifacts-from-an-upstream-pipeline), @@ -623,7 +670,7 @@ Upstream pipelines take precedence over downstream ones. If there are two variables with the same name defined in both upstream and downstream projects, the ones defined in the upstream project take precedence. -### Pass dotenv variables created in a job **(PREMIUM)** +### Pass dotenv variables created in a job **(PREMIUM ALL)** You can pass variables to a downstream job with [`dotenv` variable inheritance](../variables/index.md#pass-an-environment-variable-to-another-job) and [`needs:project`](../yaml/index.md#needsproject). These variables are only available in diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 45af40a4cea..7cde8b50524 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# CI/CD pipelines **(FREE)** +# CI/CD pipelines **(FREE ALL)** NOTE: Watch the @@ -84,11 +84,11 @@ project repository. This table lists the refspecs injected for each pipeline type: -| Pipeline type | Refspecs | -|--------------- |---------------------------------------- | -| pipeline for branches | `+<sha>:refs/pipelines/<id>` and `+refs/heads/<name>:refs/remotes/origin/<name>` | -| pipeline for tags | `+<sha>:refs/pipelines/<id>` and `+refs/tags/<name>:refs/tags/<name>` | -| [merge request pipeline](../pipelines/merge_request_pipelines.md) | `+refs/pipelines/<id>:refs/pipelines/<id>` | +| Pipeline type | Refspecs | +|-------------------------------------------------------------------|----------| +| pipeline for branches | `+<sha>:refs/pipelines/<id>` and `+refs/heads/<name>:refs/remotes/origin/<name>` | +| pipeline for tags | `+<sha>:refs/pipelines/<id>` and `+refs/tags/<name>:refs/tags/<name>` | +| [merge request pipeline](../pipelines/merge_request_pipelines.md) | `+refs/pipelines/<id>:refs/pipelines/<id>` | The refs `refs/heads/<name>` and `refs/tags/<name>` exist in your project repository. GitLab generates the special ref `refs/pipelines/<id>` during a @@ -334,7 +334,7 @@ runners do not use regular runners, they must be tagged accordingly. Review the [deployment safety](../environments/deployment_safety.md) page for additional security recommendations for securing your pipelines. -## Trigger a pipeline when an upstream project is rebuilt **(PREMIUM)** +## Trigger a pipeline when an upstream project is rebuilt **(PREMIUM ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9045) in GitLab 12.8. @@ -352,6 +352,7 @@ To trigger the pipeline when the upstream project is rebuilt: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > CI/CD**. 1. Expand **Pipeline subscriptions**. +1. Select **Add project**. 1. Enter the project you want to subscribe to, in the format `<namespace>/<project>`. For example, if the project is `https://gitlab.com/gitlab-org/gitlab`, use `gitlab-org/gitlab`. 1. Select **Subscribe**. diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 045fa8dc16c..356b97aacc0 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w --- -# Merge request pipelines **(FREE)** +# Merge request pipelines **(FREE ALL)** > [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/351192) from `pipelines for merge requests` to `merge request pipelines` in GitLab 14.8. @@ -67,7 +67,7 @@ To use merge request pipelines: ## Use `rules` to add jobs -You can use the [`rules`](../yaml/index.md#rules) keyword to configure jobs to run in +Use the [`rules`](../yaml/index.md#rules) keyword to configure jobs to run in merge request pipelines. For example: ```yaml @@ -95,10 +95,21 @@ job2: - echo "This job also runs in merge request pipelines" ``` +A common `workflow` configuration is to have pipelines run for merge requests, tags, and the default branch. For example: + +```yaml +workflow: + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' + - if: $CI_COMMIT_TAG + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH +``` + ## Use `only` to add jobs -You can use the [`only`](../yaml/index.md#onlyrefs--exceptrefs) keyword with `merge_requests` -to configure jobs to run in merge request pipelines. +[`rules`](#use-rules-to-add-jobs) is the preferred method, but you can also use +the [`only`](../yaml/index.md#onlyrefs--exceptrefs) keyword with `merge_requests` +to configure jobs to run in merge request pipelines. For example: ```yaml job1: @@ -125,7 +136,7 @@ Pipelines for forks display with the **fork** badge in the parent project:  -### Run pipelines in the parent project **(PREMIUM)** +### Run pipelines in the parent project **(PREMIUM ALL)** Project members in the parent project can trigger a merge request pipeline for a merge request submitted from a fork project. This pipeline: diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index c2fdbe3f6e5..c2bf9743e4f 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Merge trains **(PREMIUM)** +# Merge trains **(PREMIUM ALL)** NOTE: [In GitLab 16.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/359057), the **Start merge train** diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md index 16120924f37..51678e64b10 100644 --- a/doc/ci/pipelines/merged_results_pipelines.md +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Merged results pipelines **(PREMIUM)** +# Merged results pipelines **(PREMIUM ALL)** > - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/351192) from `pipelines for merged results` to `merged results pipelines` in GitLab 14.8. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91849) in GitLab 15.1, merged results pipelines also run on [Draft merge requests](../../user/project/merge_requests/drafts.md). diff --git a/doc/ci/pipelines/pipeline_architectures.md b/doc/ci/pipelines/pipeline_architectures.md index ac4c8c1a731..d0324f16ffb 100644 --- a/doc/ci/pipelines/pipeline_architectures.md +++ b/doc/ci/pipelines/pipeline_architectures.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Pipeline architecture **(FREE)** +# Pipeline architecture **(FREE ALL)** Pipelines are the fundamental building blocks for CI/CD in GitLab. This page documents some of the important concepts related to them. diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md index c0c8d60b9d6..1d7d6c3289a 100644 --- a/doc/ci/pipelines/pipeline_efficiency.md +++ b/doc/ci/pipelines/pipeline_efficiency.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Pipeline efficiency **(FREE)** +# Pipeline efficiency **(FREE ALL)** [CI/CD Pipelines](index.md) are the fundamental building blocks for [GitLab CI/CD](../index.md). Making pipelines more efficient helps you save developer time, which: diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 49b129f11aa..75a7d373203 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference, howto --- -# Scheduled pipelines **(FREE)** +# Scheduled pipelines **(FREE ALL)** Use scheduled pipelines to run GitLab CI/CD [pipelines](index.md) at regular intervals. @@ -67,6 +67,9 @@ the next scheduled time: You can manually run scheduled pipelines once per minute. +When you run a scheduled pipeline manually, the pipeline runs with the +permissions of the user who triggered it, not the permissions of the schedule owner. + ## Take ownership Scheduled pipelines execute with the permissions of the user diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index fe6c88c9c4d..b9c95c63098 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference, howto --- -# Customize pipeline configuration **(FREE)** +# Customize pipeline configuration **(FREE ALL)** You can customize how pipelines run for your project. @@ -98,6 +98,7 @@ To avoid this scenario: 1. Select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Select the **Prevent outdated deployment jobs** checkbox. +1. Optional. Clear the **Allow job retries for rollback deployments** checkbox. 1. Select **Save changes**. For more information, see [Deployment safety](../environments/deployment_safety.md#prevent-outdated-deployment-jobs). diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index 80bb9f63413..8e6fa965aa4 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Tutorial: Create and run your first GitLab CI/CD pipeline **(FREE)** +# Tutorial: Create and run your first GitLab CI/CD pipeline **(FREE ALL)** This tutorial shows you how to configure and run your first CI/CD pipeline in GitLab. diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md index 74e3e493adb..cf29ab62240 100644 --- a/doc/ci/resource_groups/index.md +++ b/doc/ci/resource_groups/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w description: Control the job concurrency in GitLab CI/CD --- -# Resource group **(FREE)** +# Resource group **(FREE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15536) in GitLab 12.7. @@ -314,4 +314,4 @@ To get job information from the GraphQL API: } ``` - If the status is not `running` or `pending`, open a new issue. + If the status is not `running` or `pending`, [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and [contact support](https://about.gitlab.com/support/#contact-support) so they can apply the correct labels to the issue. diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index e1be6118bab..d05861818e2 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Review apps **(FREE)** +# Review apps **(FREE ALL)** Review apps are a collaboration tool that provide an environment to showcase product changes. @@ -191,7 +191,7 @@ After you have the route mapping set up, it takes effect in the following locati - In the blob file view, by selecting **View** (**{external-link}**) next to the file. <!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> -## Visual Reviews (deprecated) **(PREMIUM)** +## Visual Reviews (deprecated) **(PREMIUM ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10761) in GitLab 12.0. > - [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index 9424f8ea846..3e26c120f74 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -4,7 +4,7 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Configuring runners **(FREE)** +# Configuring runners **(FREE ALL)** If you have installed your own runners, you can configure and secure them in GitLab. @@ -57,60 +57,53 @@ How this feature works: 1. You start a job 1. The job, if running longer, times out after **30 minutes** -## Be careful with sensitive information +## Protecting sensitive information -With some [runner executors](https://docs.gitlab.com/runner/executors/), -if you can run a job on the runner, you can get full access to the file system, -and thus any code it runs as well as the token of the runner. With shared runners, this means that anyone -that runs jobs on the runner, can access another user's code that runs on the -runner. +To avoid exposing sensitive information, you can restrict the usage +of shared runners on large GitLab instances. This ensures that you +control access to your GitLab instances and secure [runner executors](https://docs.gitlab.com/runner/executors/). -In addition, because you can get access to the runner token, it is possible -to create a clone of a runner and submit false jobs, for example. - -The above is easily avoided by restricting the usage of shared runners -on large public GitLab instances, controlling access to your GitLab instance, -and using more secure [runner executors](https://docs.gitlab.com/runner/executors/). +If certain executors run a job, the file system, the code the runner executes, +and the runner token may be exposed. This means that anyone that runs jobs +on a _shared runner_ can access another user's code that runs on the runner. +Users with access to the runner token can use it to create a clone of +a runner and submit false jobs in a vector attack. For more information, see [Security Considerations](https://docs.gitlab.com/runner/security/). ### Prevent runners from revealing sensitive information -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13194) in GitLab 10.0. - -You can protect runners so they don't reveal sensitive information. -When a runner is protected, the runner picks jobs created on -[protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md) only, -and ignores other jobs. +To ensure runners don't reveal sensitive information, you can configure them to only run jobs +on [protected branches](../../user/project/protected_branches.md), or jobs that have [protected tags](../../user/project/protected_tags.md). -To protect or unprotect a runner: +To prevent runners from revealing sensitive information: -1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. -1. Find the runner you want to protect or unprotect. Make sure it's enabled. -1. Select the pencil button. -1. Check the **Protected** option. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. +1. Expand **Runners**. +1. Find the runner you want to protect or unprotect. Make sure the runner is enabled. +1. Select **Edit** (**{pencil}**). +1. Select the **Protected** checkbox. 1. Select **Save changes**. - - -### Forks +### Using shared runners in forked projects -Whenever a project is forked, it copies the settings of the jobs that relate -to it. This means that if you have shared runners set up for a project and -someone forks that project, the shared runners serve jobs of this project. +When a project is forked, the job settings related to jobs are copied. If you have shared runners +configured for a project and a user forks that project, the shared runners serve jobs of this project. -Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/364303), you might encounter the message `An error occurred while forking the project. Please try again.` if the runner settings of the project you are forking does not match the new project namespace. +Due to a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/364303), if the runner settings +of the forked project does not match the new project namespace, the following message displays: +`An error occurred while forking the project. Please try again.`. -To work around this issue, you should make sure that the shared runner settings are consistent in the forked project and the new namespace. +To work around this issue, ensure that the shared runner settings are consistent in the forked project and the new namespace. - If shared runners are **enabled** on the forked project, then this should also be **enabled** on the new namespace. - If shared runners are **disabled** on the forked project, then this should also be **disabled** on the new namespace. -### Attack vectors in runners +### Reset the runner registration token for a project (deprecated) -Mentioned briefly earlier, but the following things of runners can be exploited. -We're always looking for contributions that can mitigate these -[Security Considerations](https://docs.gitlab.com/runner/security/). - -### Reset the runner registration token for a project +WARNING: +The ability to pass a runner registration token, and support for certain configuration arguments was +[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6 and will be removed in GitLab 17.0. Authentication tokens +should be used instead. For more information, see [Migrating to the new runner registration workflow](new_creation_workflow.md). If you think that a registration token for a project was revealed, you should reset it. A registration token can be used to register another runner for the project. @@ -118,43 +111,42 @@ That new runner may then be used to obtain the values of secret variables or to To reset the registration token: -1. Go to the project's **Settings > CI/CD**. -1. Expand the **General pipelines settings** section. -1. Find the **Runner token** form field and select **Reveal value**. -1. Delete the value and save the form. -1. After the page is refreshed, expand the **Runners settings** section - and check the registration token - it should be changed. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. +1. Expand **Runners**. +1. To the right of **New project runner**, select the vertical ellipsis (**{ellipsis_v}**). +1. Select **Reset registration token**. +1. Select **Reset token**. -From now on the old token is no longer valid and does not register -any new runners to the project. If you are using any tools to provision and -register new runners, the tokens used in those tools should be updated to reflect the -value of the new token. +After you reset the registration token, it is no longer valid and does not register +any new runners to the project. You should also update the registration token in tools +you use to provision and register new values. ### Reset the runner authentication token -If you think that an authentication token for a runner was revealed, you should -reset it. An attacker could use the token to [clone a runner](https://docs.gitlab.com/runner/security/#cloning-a-runner). +If an authentication token is revealed, an attacker could use the token to [clone a runner](https://docs.gitlab.com/runner/security/#cloning-a-runner). -To reset the authentication token, [unregister the runner](https://docs.gitlab.com/runner/commands/#gitlab-runner-unregister) -and then [register](https://docs.gitlab.com/runner/commands/#gitlab-runner-register) it again. +To reset the authentication token: -To verify that the previous authentication token has been revoked, use the [Runners API](../../api/runners.md#verify-authentication-for-a-registered-runner). +1. Delete the runner: + - [Delete a shared runner](runners_scope.md#delete-shared-runners). + - [Delete a group runner](runners_scope.md#delete-a-group-runner). + - [Delete a project runner](runners_scope.md#delete-a-project-runner). +1. Create a new runner so that it is assigned a new authentication token: + - [Create a shared runner](runners_scope.md#create-a-shared-runner-with-an-authentication-token). + - [Create a group runner](runners_scope.md#create-a-group-runner-with-an-authentication-token). + - [Create a project runner](runners_scope.md#create-a-project-runner-with-an-authentication-token). +1. Optional. To verify that the previous authentication token has been revoked, use the [Runners API](../../api/runners.md#verify-authentication-for-a-registered-runner). ## Use tags to control which jobs a runner can run -You must set up a runner to be able to run all the different types of jobs -that it may encounter on the projects it's shared over. This would be -problematic for large amounts of projects, if it weren't for tags. +You can use [tags](../yaml/index.md#tags) to ensure that runners only run the jobs they are equipped +to run. For example, you can specify the `rails` tag for runners that have the dependencies to run +Rails test suites. -GitLab CI/CD tags are not the same as Git tags. GitLab CI/CD tags are associated with runners. +GitLab CI/CD tags are different to Git tags. GitLab CI/CD tags are associated with runners. Git tags are associated with commits. -By tagging a runner for the types of jobs it can handle, you can make sure -shared runners will [only run the jobs they are equipped to run](../yaml/index.md#tags). - -For instance, at GitLab we have runners tagged with `rails` if they contain -the appropriate dependencies to run Rails test suites. - ### Set a runner to run untagged jobs When you [register a runner](https://docs.gitlab.com/runner/register/), its default behavior is to **only pick** @@ -302,9 +294,6 @@ When using the Kubernetes executor, you can use variables to ### Git strategy -> - Introduced in GitLab 8.9 as an experimental feature. -> - `GIT_STRATEGY=none` requires GitLab Runner v1.7+. - You can set the `GIT_STRATEGY` used to fetch the repository content, either globally or per-job in the [`variables`](../yaml/index.md#variables) section: @@ -341,8 +330,6 @@ rely on files brought into the local working copy from cache or artifacts. ### Git submodule strategy -> Requires GitLab Runner v1.10+. - The `GIT_SUBMODULE_STRATEGY` variable is used to control if / how Git submodules are included when fetching the code before a build. You can set them globally or per-job in the [`variables`](../yaml/index.md#variables) section. @@ -381,8 +368,6 @@ You can provide additional flags to control advanced behavior using [`GIT_SUBMOD ### Git checkout -> Introduced in GitLab Runner 9.3. - The `GIT_CHECKOUT` variable can be used when the `GIT_STRATEGY` is set to either `clone` or `fetch` to specify whether a `git checkout` should be run. If not specified, it defaults to true. You can set them globally or per-job in the @@ -410,8 +395,6 @@ script: ### Git clean flags -> Introduced in GitLab Runner 11.10 - The `GIT_CLEAN_FLAGS` variable is used to control the default behavior of `git clean` after checking out the sources. You can set it globally or per-job in the [`variables`](../yaml/index.md#variables) section. @@ -437,8 +420,6 @@ script: ### Git fetch extra flags -> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4142) in GitLab Runner 13.1. - Use the `GIT_FETCH_EXTRA_FLAGS` variable to control the behavior of `git fetch`. You can set it globally or per-job in the [`variables`](../yaml/index.md#variables) section. @@ -503,8 +484,6 @@ to wrap the string in single quotes so the YAML can be parsed successfully. ### Git submodule update flags -> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3192) in GitLab Runner 14.8. - Use the `GIT_SUBMODULE_UPDATE_FLAGS` variable to control the behavior of `git submodule update` when [`GIT_SUBMODULE_STRATEGY`](#git-submodule-strategy) is set to either `normal` or `recursive`. You can set it globally or per-job in the [`variables`](../yaml/index.md#variables) section. @@ -561,8 +540,6 @@ the permissions of the user executing the job, and does not require SSH credenti ### Shallow cloning -> Introduced in GitLab 8.9 as an experimental feature. - You can specify the depth of fetching and cloning using `GIT_DEPTH`. `GIT_DEPTH` does a shallow clone of the repository and can significantly speed up cloning. It can be helpful for repositories with a large number of commits or old, large binaries. The value is @@ -613,8 +590,6 @@ variables: ### Custom build directories -> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2211) in GitLab Runner 11.10. - By default, GitLab Runner clones the repository in a unique subpath of the `$CI_BUILDS_DIR` directory. However, your project might require the code in a specific directory (Go projects, for example). In that case, you can specify @@ -696,8 +671,6 @@ because `$CI_BUILDS_DIR` is not expanded. ### Job stages attempts -> Introduced in GitLab, it requires GitLab Runner v1.9+. - You can set the number of attempts that the running job tries to execute the following stages: @@ -725,8 +698,6 @@ GitLab.com shared runners run on CoreOS. This means that you cannot use some sys ## Artifact and cache settings -> Introduced in GitLab Runner 13.9. - Artifact and cache settings control the compression ratio of artifacts and caches. Use these settings to specify the size of the archive produced by a job. diff --git a/doc/ci/runners/img/protected_runners_check_box_v14_1.png b/doc/ci/runners/img/protected_runners_check_box_v14_1.png Binary files differdeleted file mode 100644 index d67085d83f9..00000000000 --- a/doc/ci/runners/img/protected_runners_check_box_v14_1.png +++ /dev/null diff --git a/doc/ci/runners/new_creation_workflow.md b/doc/ci/runners/new_creation_workflow.md index fce4b57b2a1..55ff5165ff7 100644 --- a/doc/ci/runners/new_creation_workflow.md +++ b/doc/ci/runners/new_creation_workflow.md @@ -4,7 +4,7 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Migrating to the new runner registration workflow **(FREE)** +# Migrating to the new runner registration workflow **(FREE ALL)** DISCLAIMER: This page contains information related to upcoming products, features, and functionality. diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index 840eb7fe93b..fff695eb606 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -40,7 +40,7 @@ If you are using GitLab.com: > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/389269) in GitLab 16.0. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/415447) in GitLab 16.2. Feature flag `create_runner_workflow_for_admin` removed. -Prerequisites: +Prerequisite: - You must be an administrator. @@ -69,7 +69,7 @@ The ability to pass a runner registration token, and support for certain configu [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6 and will be removed in GitLab 17.0. Authentication tokens should be used instead. For more information, see [Migrating to the new runner registration workflow](new_creation_workflow.md). -Prerequisites: +Prerequisite: - You must be an administrator. @@ -82,6 +82,44 @@ To create a shared runner: 1. Copy the registration token. 1. [Register the runner](https://docs.gitlab.com/runner/register/). +### Pause or resume a shared runner + +Prerequisite: + +- You must be an administrator. + +You can pause a runner so that it does not accept jobs from groups and projects in the GitLab instance. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **CI/CD > Runners**. +1. In the search box, enter the runner description or filter the runner list. +1. In the runner list, to the right of the runner: + - To pause the runner, select **Pause** (**{pause}**). + - To resume the runner, select **Resume** (**{play}**). + +### Delete shared runners + +Prerequisite: + +- You must be an administrator. + +When you delete a shared runner, it is permanently deleted from the GitLab instance and can +no longer be used by groups and projects. If you want to temporarily stop the runner from accepting +jobs, you can [pause](#pause-or-resume-a-shared-runner) the runner instead. + +To delete a single or multiple shared runners: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **CI/CD > Runners**. +1. In the search box, enter the runner description or filter the list of runners. +1. Delete the shared runner: + - To delete a single runner, next to the runner, select **Delete runner** (**{remove}**). + - To delete multiple shared runners, select the checkbox for each runner and select **Delete selected**. + - To delete all runners, select the checkbox at the top of the runner list and select **Delete selected**. +1. Select **Permanently delete runner**. + ### Enable shared runners for a project On GitLab.com, [shared runners](index.md) are enabled in all projects by @@ -182,7 +220,7 @@ When only one job runs at a time, the fair usage algorithm assigns jobs in this Use _group runners_ when you want all projects in a group to have access to a set of runners. -Group runners process jobs by using a first in, first out ([FIFO](https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics))) queue. +Group runners process jobs by using a first in, first out queue. ### Create a group runner with an authentication token @@ -237,8 +275,6 @@ how to [register a runner](https://docs.gitlab.com/runner/register/). ### View and manage group runners -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/37366/) in GitLab 13.2. - You can view and manage all runners for a group, its subgroups, and projects. You can do this for your self-managed GitLab instance or for GitLab.com. You must have the Owner role for the group. @@ -248,25 +284,6 @@ You must have the Owner role for the group. From this page, you can edit, pause, and remove runners from the group, its subgroups, and projects. -#### Delete multiple group runners - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361721/) in GitLab 15.6 - -Prerequisites: - -- You must have either: - - Owner role for the group. - - Access to delete any runners in the group. - -To delete multiple runners in a single action in the group list: - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. -1. Select **Build > Runners**. -1. To delete multiple runners, you can either: - - Select the checkbox next to the runner. - - Select the checkbox at the top of the runner list to select all runners in the list. -1. To delete the runners, select **Delete selected**. - #### Filter group runners to show only inherited > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/337838/) in GitLab 15.5. @@ -284,20 +301,46 @@ those in other groups: 1. Select **Build > Runners**. 1. Above the list, turn off the **Show only inherited** toggle. -### Pause or remove a group runner +### Pause or resume a group runner -You can pause or remove a group runner for your self-managed GitLab instance or for GitLab.com. -You must have the Owner role for the group. +Prerequisite: + +- You must be an administrator or have the Owner role for the group. + +You can pause a runner so that it does not accept jobs from subgroups and projects in the GitLab +instance. If you pause a group runner that is used by multiple projects, the runner pauses for all projects. + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Build > Runners**. +1. In the search box, enter the runner description or filter the runner list. +1. In the runner list, to the right of the runner: + - To pause the runner, select **Pause** (**{pause}**). + - To resume the runner, select **Resume** (**{play}**). + +### Delete a group runner + +> Multiple runner deletion [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361721/) in GitLab 15.6. + +Prerequisite: + +- You must be an administrator or have the Owner role for the group. + +When you delete a group runner, it is permanently deleted from the GitLab instance and can +no longer be used by subgroups and projects. If you want to temporarily stop the runner from accepting +jobs, you can [pause](#pause-or-resume-a-group-runner) the runner instead. + +To delete a single or multiple group runners: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. 1. Select **Build > Runners**. -1. Select **Pause** or **Remove runner**. - - If you pause a group runner that is used by multiple projects, the runner pauses for all projects. - - From the group view, you cannot remove a runner that is assigned to more than one project. - You must remove it from each project first. -1. On the confirmation dialog, select **OK**. +1. In the search box, enter the runner description or filter the list of runners. +1. Delete the group runner: + - To delete a single runner, next to the runner, select **Delete runner** (**{remove}**). + - To delete multiple shared runners, select the checkbox for each runner and select **Delete selected**. + - To delete all runners, select the checkbox at the top of the runner list and select **Delete selected**. +1. Select **Permanently delete runner**. -### Clean up stale group runners **(ULTIMATE)** +### Clean up stale group runners **(ULTIMATE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363012) in GitLab 15.1. @@ -407,6 +450,43 @@ To create a project runner: The runner is now enabled for the project. +### Pause or resume a project runner + +Prerequisite: + +- You must be an administrator, or have the Maintainer role for the project. + +You can pause a project runner so that it does not accept jobs from projects it's assigned to +in the GitLab instance. + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to + find the project where you want to enable the runner. +1. Select **Settings > CI/CD**. +1. Expand **Runners**. +1. In the **Assigned project runners** section, find the runner. +1. To the right of the runner: + - To pause the runner, select **Pause** (**{pause}**), then select **Pause**. + - To resume the runner, select **Resume** (**{play}**). + +### Delete a project runner + +Prerequisites: + +- You must be an administrator, or have the Maintainer role for the project. +- You cannot delete a project runner that is assigned to more than one project. Before you can delete the runner, you must [disable](#enable-a-project-runner-for-a-different-project) it in all projects where it is enabled. + +When you delete a project runner, it is permanently deleted from the GitLab instance and can +no longer be used by projects. If you want to temporarily stop the runner from accepting +jobs, you can [pause](#pause-or-resume-a-project-runner) the runner instead. + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to + find the project where you want to enable the runner. +1. Select **Settings > CI/CD**. +1. Expand **Runners**. +1. In the **Assigned project runners** section, find the runner. +1. To the right of the runner, select **Remove runner**. +1. To delete the runner, select **Remove**. + ### Enable a project runner for a different project After a project runner is created, you can enable it for other projects. @@ -460,26 +540,25 @@ A runner can have one of the following statuses. | `stale` | The runner has not contacted GitLab in more than 3 months. If the runner was created more than 3 months ago, but it never contacted the instance, it is also considered **stale**. | | `never_contacted` | The runner has never contacted GitLab. To make the runner contact GitLab, run `gitlab-runner run`. | -## View statistics for runner performance **(ULTIMATE)** +## View statistics for runner performance **(ULTIMATE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/377963) in GitLab 15.8. As an administrator, you can view runner statistics to learn about the performance of your runner fleet. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. On the left sidebar, select **CI/CD > Runners**. -1. Select **View metrics**. - -The **Median job queued time** value is calculated by sampling the queue duration of the +- The **Median job queued time** value is calculated by sampling the queue duration of the most recent 100 jobs that were run by Instance runners. Jobs from only the latest 5000 runners are considered. - -The median is a value that falls into the 50th percentile: half of the jobs +- The median is a value that falls into the 50th percentile: half of the jobs queued for longer than the median value, and half of the jobs queued for less than the median value. -## Determine which runners need to be upgraded **(ULTIMATE)** +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. On the left sidebar, select **CI/CD > Runners**. +1. Select **View metrics**. + +## Determine which runners need to be upgraded **(ULTIMATE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/365078) in GitLab 15.3. diff --git a/doc/ci/runners/saas/gpu_saas_runner.md b/doc/ci/runners/saas/gpu_saas_runner.md index a05b162ac84..6dc9d8c9534 100644 --- a/doc/ci/runners/saas/gpu_saas_runner.md +++ b/doc/ci/runners/saas/gpu_saas_runner.md @@ -44,3 +44,10 @@ gpu-job: ``` If you don't want to install larger libraries such as Tensorflow or XGBoost each time you run a job, you can create your own image with all the required components pre-installed. +Watch this demo to learn how to leverage GPU-enabled SaaS runners to train an XGBoost model: +<div class="video-fallback"> + Video demonstration of GitLab GPU-Enabled SaaS runners: <a href="https://youtu.be/tElegG4NCZ0">Train XGboost models with GitLab</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/tElegG4NCZ0" frameborder="0" allowfullscreen> </iframe> +</figure> diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md index 95917bbc300..dd5381f3cd6 100644 --- a/doc/ci/runners/saas/linux_saas_runner.md +++ b/doc/ci/runners/saas/linux_saas_runner.md @@ -20,8 +20,10 @@ For Free, Premium, and Ultimate plan customers, jobs on these instances consume | Runner Tag | vCPUs | Memory | Storage | |-----------------------------------------------|-------|--------|---------| | `saas-linux-small-amd64` | 2 | 8 GB | 25 GB | -| `saas-linux-medium-amd64` **(PREMIUM SAAS)** | 4 | 16 GB | 50 GB | -| `saas-linux-large-amd64` **(PREMIUM SAAS)** | 8 | 32 GB | 100 GB | +| `saas-linux-medium-amd64` | 4 | 16 GB | 50 GB | +| `saas-linux-large-amd64` **(PREMIUM SAAS)** | 8 | 32 GB | 100 GB | +| `saas-linux-xlarge-amd64` **(PREMIUM SAAS)** | 16 | 64 GB | 200 GB | +| `saas-linux-2xlarge-amd64` **(PREMIUM SAAS)** | 32 | 128 GB | 200 GB | The `small` machine type is set as default. If no [tag](../../yaml/index.md#tags) keyword in your `.gitlab-ci.yml` file is specified, the jobs will run on this default runner. diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index a559fc7d53e..44f99ed6ccc 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -117,6 +117,51 @@ variables: HOMEBREW_NO_AUTO_UPDATE: 1 ``` +## Optimizing Cocoapods + +If you use Cocoapods in a project, you should consider the following optimizations to improve CI performance. + +**Cocoapods CDN** + +You can use content delivery network (CDN) access to download packages from the CDN instead of having to clone an entire +project repository. CDN access is available in Cocoapods 1.8 or later and is supported by all GitLab SaaS runners on macOS. + +To enable CDN access, ensure your Podfile starts with: + +```ruby +source 'https://cdn.cocoapods.org/' +``` + +**Use GitLab caching** + +Use caching in Cocoapods packages in GitLab to only run `pod install` +when pods change, which can improve build performance. + +To [configure caching](../../../ci/caching/index.md) for your project: + +1. Add the `cache` configuration to your `.gitlab-ci.yml` file: + + ```yaml + cache: + key: + files: + - Podfile.lock + paths: + - Pods + ``` + +1. Add the [`cocoapods-check`](https://guides.cocoapods.org/plugins/optimising-ci-times.html) plugin to your project. +1. Update the job script to check for installed dependencies before it calls `pod install`: + + ```shell + bundle exec pod check || bundle exec pod install + ``` + +**Include pods in source control** + +You can also [include the pods directory in source control](https://guides.cocoapods.org/using/using-cocoapods.html#should-i-check-the-pods-directory-into-source-control). This eliminates the need to install pods as part of the CI job, +but it does increase the overall size of your project's repository. + ## Known issues and usage constraints - If the VM image does not include the specific software version you need for your job, the required software must be fetched and installed. This causes an increase in job execution time. diff --git a/doc/ci/secrets/azure_key_vault.md b/doc/ci/secrets/azure_key_vault.md new file mode 100644 index 00000000000..645ab5db0d1 --- /dev/null +++ b/doc/ci/secrets/azure_key_vault.md @@ -0,0 +1,49 @@ +--- +stage: Verify +group: Pipeline Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: concepts, howto +--- + +# Use Azure Key Vault secrets in GitLab CI/CD **(PREMIUM ALL)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271271) in GitLab and GitLab Runner 16.3. + +You can use secrets stored in the [Azure Key Vault](https://azure.microsoft.com/en-us/products/key-vault/) +in your GitLab CI/CD pipelines. + +Prerequisites: + +- Have a key vault on Azure. +- Have an application with key vault permissions. +- [Configure OpenID Connect in Azure to retrieve temporary credentials](../../ci/cloud_services/azure/index.md). +- Add [CI/CD variables to your project](../variables/index.md#for-a-project) to provide details about your Vault server: + - `AZURE_KEY_VAULT_SERVER_URL`: The URL of your Azure Key Vault server, such as `https://vault.example.com`. + - `AZURE_CLIENT_ID`: The client ID of the Azure application. + - `AZURE_TENANT_ID`: The tenant ID of the Azure application. + +## Use Azure Key Vault secrets in a CI/CD job + +You can use a secret stored in your Azure Key Vault in a job by defining it with the +[`azure_key_vault`](../yaml/index.md#secretsazure_key_vault) keyword: + +```yaml +job: + id_tokens: + AZURE_JWT: + aud: 'azure' + secrets: + DATABASE_PASSWORD: + token: AZURE_JWT + azure_key_vault: + name: 'test' + version: 'test' +``` + +In this example: + +- `name` is the name of the secret. +- `version` is the version of the secret. +- GitLab fetches the secret from Azure Key Vault and stores the value in a temporary file. + The path to this file is stored in a `DATABASE_PASSWORD` CI/CD variable, similar to + [file type CI/CD variables](../variables/index.md#use-file-type-cicd-variables). diff --git a/doc/ci/secrets/convert-to-id-tokens.md b/doc/ci/secrets/convert-to-id-tokens.md index e9f0d0ef5b0..18803d4de72 100644 --- a/doc/ci/secrets/convert-to-id-tokens.md +++ b/doc/ci/secrets/convert-to-id-tokens.md @@ -5,9 +5,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Tutorial: Update HashiCorp Vault configuration to use ID Tokens **(PREMIUM)** +# Tutorial: Update HashiCorp Vault configuration to use ID Tokens **(PREMIUM ALL)** -This tutorial demonstrates how to convert your existing CI/CI secrets configuration to use [ID Tokens](../secrets/id_token_authentication.md). +This tutorial demonstrates how to convert your existing CI/CD secrets configuration to use [ID Tokens](../secrets/id_token_authentication.md). The `CI_JOB_JWT` variables are deprecated, but updating to ID tokens requires some important configuration changes to work with Vault. If you have more than a handful of jobs, converting everything at once is a daunting task. @@ -20,10 +20,15 @@ Jobs that don't use `secrets:vault` can still use `CI_JOB_JWT` tokens. This tutorial will focus on v16 onwards, if you are running a slightly older version you will need to toggle the `Limit JSON Web Token (JWT) access` setting as appropriate. -To update your vault configuration to use ID tokens: +There isn't one standard method to migrate to [ID tokens](../secrets/id_token_authentication.md), so this tutorial includes two variations for how to convert your existing CI/CD secrets. Choose the method that is most appropriate for your use case: -1. [Create a second JWT authentication path in Vault](#create-a-second-jwt-authentication-path-in-vault) -1. [Recreate roles to use the new authentication path](#recreate-roles-to-use-the-new-authentication-path) +1. Update your Vault configuration: + - Method A: Migrate JWT roles to the new Vault auth method + 1. [Create a second JWT authentication path in Vault](#create-a-second-jwt-authentication-path-in-vault) + 1. [Recreate roles to use the new authentication path](#recreate-roles-to-use-the-new-authentication-path) + - Method B: Move `iss` claim to roles for the migration window + 1. [Add `bound_issuers` claim map to each role](#add-bound_issuers-claim-map-to-each-role) + 1. [Remove `bound_issuers` claim from auth method](#remove-bound_issuers-claim-from-auth-method) 1. [Update your CI/CD Jobs](#update-your-cicd-jobs) ## Prerequisites @@ -36,10 +41,17 @@ To follow along, you must have: - A Vault server that you are already using. - CI/CD jobs retrieving secrets from Vault with `CI_JOB_JWT`. -In the examples below, replace `vault.example.com` with the URL of your Vault server, -and `gitlab.example.com` with the URL of your GitLab instance. +In the examples below, replace: -## Create a second JWT authentication path in Vault +- `vault.example.com` with the URL of your Vault server. +- `gitlab.example.com` with the URL of your GitLab instance. +- `jwt` or `jwt_v2` with your auth method names. + +## Method A: Migrate JWT roles to the new Vault auth method + +This method creates a second JWT auth method in parallel to the existing one in use. Afterwards all Vault roles used for the GitLab integration are recreated in this new auth method. + +### Create a second JWT authentication path in Vault As part of the transition from `CI_JOB_JWT` to ID tokens, you must update the `bound_issuer` in Vault to include `https://`: @@ -69,7 +81,7 @@ You can create multiple authentication paths in Vault, which enable you to trans bound_issuer="https://gitlab.example.com" ``` -## Recreate roles to use the new authentication path +### Recreate roles to use the new authentication path Roles are bound to a specific authentication path so you need to add new roles for each job. @@ -113,6 +125,63 @@ Roles are bound to a specific authentication path so you need to add new roles f You only need to update `jwt` to `jwt_v2` in the `vault` command, do not change the `role_type` inside the role. +## Method B: Move `iss` claim to roles for migration window + +This method doesn't require Vault administrators to create a second JWT auth method and recreate all GitLab related roles. + +### Add `bound_issuers` claim map to each role + +Vault doesn't allow multiple `iss` claims on the JWT auth method level, as the [`bound_issuer`](https://developer.hashicorp.com/vault/api-docs/auth/jwt#bound_issuer) +directive on this level only accepts a single value. However, multiple claims can be configured +on the role level by using the [`bound_claims`](https://developer.hashicorp.com/vault/api-docs/auth/jwt#bound_claims) +map configuration directive. + +With this method you can provide Vault with multiple options for the `iss` claim validation. This supports the `https://` prefixed GitLab instance hostname claim that comes with the `id_tokens`, as well as the old non-prefixed claim. + +To add the [`bound_claims`](https://developer.hashicorp.com/vault/api-docs/auth/jwt#bound_claims) configuration to the required roles, run: + +```shell +$ vault write auth/jwt/role/myproject-staging - <<EOF +{ + "role_type": "jwt", + "policies": ["myproject-staging"], + "token_explicit_max_ttl": 60, + "user_claim": "user_email", + "bound_claims": { + "iss": [ + "https://gitlab.example.com", + "gitlab.example.com" + ], + "project_id": "22", + "ref": "master", + "ref_type": "branch" + } +} +EOF +``` + +You do not need to alter any existing role configurations except for the `bound_claims` section +Make sure to add the `iss` configuration as shown above to ensure Vault accepts +the prefixed and non-prefixed `iss` claim for this role. + +You must apply this change to all JWT roles used for the GitLab integration before moving on to the next step. + +You can revert the migration of the `iss` claim validation from the auth method to the roles if desired, +after all projects have been migrated and you no longer need parallel support for `CI_JOB_JWT` and ID tokens. + +### Remove `bound_issuers` claim from auth method + +After all roles have been updated with the `bound_claims.iss` claims, you can remove the auth method level configuration for this validation: + +```shell +$ vault write auth/jwt/config \ + jwks_url="https://gitlab.example.com/-/jwks" \ + bound_issuer="" +``` + +Setting the `bound_issuer` directive to an empty string removes the issuer validation on the auth method level. +However, as we have moved this validation to the role level, this configuration is still secure. + ## Update your CI/CD Jobs Vault has two different [KV Secrets Engines](https://developer.hashicorp.com/vault/docs/secrets/kv) and the version you are using impacts how you define secrets in CI/CD. @@ -124,7 +193,12 @@ Also, if needed you can review the CI/CD documentation for: - [`secrets:`](../yaml/index.md#secrets) - [`id_tokens:`](../yaml/index.md#id_tokens) -The following examples show how to obtain the staging database password written to the `password` field in `secret/myproject/staging/db` +The following examples show how to obtain the staging database password written to the `password` field in `secret/myproject/staging/db`. + +The value for the `VAULT_AUTH_PATH` variable depends on the migration method you used: + +- Method A (Migrate JWT roles to the new Vault auth method): Use `jwt_v2`. +- Method B (Move `iss` claim to roles for migration window): Use `jwt`. ### KV Secrets Engine v1 @@ -134,7 +208,7 @@ The [`secrets:vault`](../yaml/index.md#secretsvault) keyword defaults to v2 of t job: variables: VAULT_SERVER_URL: https://vault.example.com - VAULT_AUTH_PATH: jwt_v2 + VAULT_AUTH_PATH: jwt_v2 # or "jwt" if you used method B VAULT_AUTH_ROLE: myproject-staging id_tokens: VAULT_ID_TOKEN: @@ -165,7 +239,7 @@ Long format: job: variables: VAULT_SERVER_URL: https://vault.example.com - VAULT_AUTH_PATH: jwt_v2 + VAULT_AUTH_PATH: jwt_v2 # or "jwt" if you used method B VAULT_AUTH_ROLE: myproject-staging id_tokens: VAULT_ID_TOKEN: @@ -189,7 +263,7 @@ You can also use a short format: job: variables: VAULT_SERVER_URL: https://vault.example.com - VAULT_AUTH_PATH: jwt_v2 + VAULT_AUTH_PATH: jwt_v2 # or "jwt" if you used method B VAULT_AUTH_ROLE: myproject-staging id_tokens: VAULT_ID_TOKEN: @@ -201,3 +275,5 @@ job: ``` After you commit the updated CI/CD configuration, your jobs will be fetching secrets with ID Tokens, congratulations! + +If you have migrated all projects to fetch secrets with ID Tokens and used method B for the migration, it is now possible to move the `iss` claim validation back to the auth method configuration if you desire. diff --git a/doc/ci/secrets/id_token_authentication.md b/doc/ci/secrets/id_token_authentication.md index 509bb6f07cf..22a260e4bb6 100644 --- a/doc/ci/secrets/id_token_authentication.md +++ b/doc/ci/secrets/id_token_authentication.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# OpenID Connect (OIDC) Authentication Using ID Tokens **(FREE)** +# OpenID Connect (OIDC) Authentication Using ID Tokens **(FREE ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356986) in GitLab 15.7. @@ -76,6 +76,7 @@ The token also includes custom claims provided by GitLab: | `sha` | Always | The commit SHA for the job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.0. | | `ci_config_ref_uri` | Always | The ref path to the top-level pipeline definition, for example, `gitlab.example.com/my-group/my-project//.gitlab-ci.yml@refs/heads/main`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.2. This claim is `null` unless the pipeline definition is located in the same project. | | `ci_config_sha` | Always | Git commit SHA for the `ci_config_ref_uri`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.2. This claim is `null` unless the pipeline definition is located in the same project. | +| `project_visibility` | Always | The [visibility](../../user/public_access.md) of the project where the pipeline is running. Can be `internal`, `private`, or `public`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/418810) in GitLab 16.3. | ```json { @@ -103,6 +104,7 @@ The token also includes custom claims provided by GitLab: "runner_id": 1, "runner_environment": "self-hosted", "sha": "714a629c0b401fdce83e847fc9589983fc6f46bc", + "project_visibility": "public", "ci_config_ref_uri": "gitlab.example.com/my-group/my-project//.gitlab-ci.yml@refs/heads/main", "ci_config_sha": "714a629c0b401fdce83e847fc9589983fc6f46bc", "jti": "235b3a54-b797-45c7-ae9a-f72d7bc6ef5b", @@ -136,7 +138,7 @@ manual_authentication: - my-authentication-script.sh $VAULT_TOKEN $PASSWORD ``` -## Automatic ID Token authentication with HashiCorp Vault **(PREMIUM)** +## Automatic ID Token authentication with HashiCorp Vault **(PREMIUM ALL)** You can use ID tokens to automatically fetch secrets from HashiCorp Vault with the [`secrets`](../yaml/index.md#secrets) keyword. diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md index db9f36f295a..c184102d948 100644 --- a/doc/ci/secrets/index.md +++ b/doc/ci/secrets/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: concepts, howto --- -# Using external secrets in CI **(FREE)** +# Using external secrets in CI **(FREE ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218746) in GitLab 13.4 and GitLab Runner 13.4. > - `file` setting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250695) in GitLab 14.1 and GitLab Runner 14.1. @@ -97,7 +97,7 @@ To configure your Vault server: NOTE: Support for providing these values in the user interface [is tracked in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/218677). -## Use Vault secrets in a CI job **(PREMIUM)** +## Use Vault secrets in a CI job **(PREMIUM ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/28321) in GitLab 13.4 and GitLab Runner 13.4. diff --git a/doc/ci/secure_files/index.md b/doc/ci/secure_files/index.md index b3ccf071996..37c453a5b9d 100644 --- a/doc/ci/secure_files/index.md +++ b/doc/ci/secure_files/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Project-level Secure Files **(FREE)** +# Project-level Secure Files **(FREE ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78227) in GitLab 14.8 [with a flag](../../administration/feature_flags.md) named `ci_secure_files`. Disabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/350748) in GitLab 15.7. Feature flag `ci_secure_files` removed. diff --git a/doc/ci/services/gitlab.md b/doc/ci/services/gitlab.md index bd31f1b8e91..54fc664ef12 100644 --- a/doc/ci/services/gitlab.md +++ b/doc/ci/services/gitlab.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Use GitLab as a microservice **(FREE)** +# Use GitLab as a microservice **(FREE ALL)** Many applications need to access JSON APIs, so application tests might need access to APIs too. The following example shows how to use GitLab as a microservice to give diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index f4c90934e06..f92c50fcf3f 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index --- -# Services **(FREE)** +# Services **(FREE ALL)** When you configure CI/CD, you specify an image, which is used to create the container where your jobs run. To specify this image, you use the `image` keyword. @@ -48,7 +48,7 @@ socket or `localhost`. Read more in [accessing the services](#accessing-the-serv ## How the health check of services works Services are designed to provide additional features which are **network accessible**. -They may be a database like MySQL, or Redis, and even `docker:stable-dind` which +They may be a database like MySQL, or Redis, and even `docker:dind` which allows you to use Docker-in-Docker. It can be practically anything that's required for the CI/CD job to proceed, and is accessed by network. diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index e795c4ffc5f..d02df1faf55 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Using MySQL **(FREE)** +# Using MySQL **(FREE ALL)** Many applications depend on MySQL as their database, and you may need it for your tests to run. diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index ab38d2ce934..b491786b4a1 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Using PostgreSQL **(FREE)** +# Using PostgreSQL **(FREE ALL)** As many applications depend on PostgreSQL as their database, you eventually need it in order for your tests to run. Below you are guided how to diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index a7a28116aa4..d86ffdfabd6 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Using Redis **(FREE)** +# Using Redis **(FREE ALL)** As many applications depend on Redis as their key-value store, you eventually need it in order for your tests to run. Below you are guided how to diff --git a/doc/ci/ssh_keys/index.md b/doc/ci/ssh_keys/index.md index 15b731f88c8..ad45345fa38 100644 --- a/doc/ci/ssh_keys/index.md +++ b/doc/ci/ssh_keys/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Using SSH keys with GitLab CI/CD **(FREE)** +# Using SSH keys with GitLab CI/CD **(FREE ALL)** GitLab currently doesn't have built-in support for managing SSH keys in a build environment (where the GitLab Runner runs). diff --git a/doc/ci/test_cases/index.md b/doc/ci/test_cases/index.md index 9667daf7501..0bc9ae7776e 100644 --- a/doc/ci/test_cases/index.md +++ b/doc/ci/test_cases/index.md @@ -6,7 +6,7 @@ description: Test cases in GitLab can help your teams create testing scenarios i type: reference --- -# Test cases **(ULTIMATE)** +# Test cases **(ULTIMATE ALL)** Test cases in GitLab can help your teams create testing scenarios in their existing development platform. diff --git a/doc/ci/testing/accessibility_testing.md b/doc/ci/testing/accessibility_testing.md index f6d0468c876..783e787f8a7 100644 --- a/doc/ci/testing/accessibility_testing.md +++ b/doc/ci/testing/accessibility_testing.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Accessibility testing **(FREE)** +# Accessibility testing **(FREE ALL)** If your application offers a web interface, you can use [GitLab CI/CD](../index.md) to determine the accessibility diff --git a/doc/ci/testing/browser_performance_testing.md b/doc/ci/testing/browser_performance_testing.md index 059cd637f9e..9e81f243e50 100644 --- a/doc/ci/testing/browser_performance_testing.md +++ b/doc/ci/testing/browser_performance_testing.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Browser Performance Testing **(PREMIUM)** +# Browser Performance Testing **(PREMIUM ALL)** If your application offers a web interface and you're using [GitLab CI/CD](../index.md), you can quickly determine the rendering performance diff --git a/doc/ci/testing/code_coverage.md b/doc/ci/testing/code_coverage.md index 8ffd2cfb727..90a07314083 100644 --- a/doc/ci/testing/code_coverage.md +++ b/doc/ci/testing/code_coverage.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Code coverage **(FREE)** +# Code coverage **(FREE ALL)** Use code coverage to provide insights on what source code is being validated by a test suite. Code coverage is one of many test metrics that can determine software performance and quality. @@ -81,7 +81,7 @@ To view a CSV file of the data, select **Download raw data (`.csv`)**.  -### View history of group code coverage **(PREMIUM)** +### View history of group code coverage **(PREMIUM ALL)** To see the all the project's code coverage under a group over time, you can find view [group repository analytics](../../user/group/repositories_analytics/index.md). @@ -92,7 +92,7 @@ To see the all the project's code coverage under a group over time, you can find You can use [pipeline badges](../../user/project/badges.md#test-coverage-report-badges) to indicate the pipeline status and test coverage of your projects. These badges are determined by the latest successful pipeline. -## Coverage check approval rule **(PREMIUM)** +## Coverage check approval rule **(PREMIUM ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15765) in GitLab 14.0. > - [Made configurable in Project Settings](https://gitlab.com/gitlab-org/gitlab/-/issues/331001) in GitLab 14.1. diff --git a/doc/ci/testing/code_quality.md b/doc/ci/testing/code_quality.md index fcb5a23742a..1d857b8f543 100644 --- a/doc/ci/testing/code_quality.md +++ b/doc/ci/testing/code_quality.md @@ -4,7 +4,7 @@ group: Static Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Code Quality **(FREE)** +# Code Quality **(FREE ALL)** > [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) to GitLab Free in 13.2. @@ -56,7 +56,7 @@ were introduced by the changes made in the merge request.  -### Merge request changes view **(ULTIMATE)** +### Merge request changes view **(ULTIMATE ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267612) in GitLab 13.11, disabled by default behind the `codequality_mr_diff` [feature flag](../../administration/feature_flags.md). > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/284140) in GitLab 13.12. @@ -68,7 +68,7 @@ issues are marked by an indicator beside the gutter. Hover over the marker for d  -### Pipeline details view **(PREMIUM)** +### Pipeline details view **(PREMIUM ALL)** The full list of Code Quality violations generated by a pipeline is shown in the **Code Quality** tab of the pipeline's details page. The pipeline details view displays all Code Quality findings @@ -76,7 +76,7 @@ that were found on the branch it was run on.  -### Project quality view **(ULTIMATE)** +### Project quality view **(ULTIMATE ALL)** The project quality view displays an overview of the code quality findings. The view can be found under **Analyze > CI/CD analytics**, and requires [`project_quality_summary_page`](../../user/feature_flags.md) feature flag to be enabled for this particular project. @@ -136,7 +136,7 @@ To use private runners: ```shell $ gitlab-runner register --executor "docker" \ - --docker-image="docker:stable" \ + --docker-image="docker:latest" \ --url "https://gitlab.com/" \ --description "cq-sans-dind" \ --tag-list "cq-sans-dind" \ @@ -171,7 +171,7 @@ To use private runners: builds_dir = "/tmp/builds" [runners.docker] tls_verify = false - image = "docker:stable" + image = "docker:latest" privileged = false disable_entrypoint_overwrite = false oom_kill_disable = false @@ -454,13 +454,13 @@ You can integrate a custom tool into GitLab to provide Code Quality reports. The Code Quality report artifact JSON file must contain an array of objects with the following properties: -| Name | Description | -| ---------------------- | ----------------------------------------------------------------------------------------- | -| `description` | A description of the code quality violation. | -| `check_name` | A unique name representing the static analysis check that emitted this issue. | -| `fingerprint` | A unique fingerprint to identify the code quality violation. For example, an MD5 hash. | -| `severity` | A severity string (can be `info`, `minor`, `major`, `critical`, or `blocker`). | -| `location.path` | The relative path to the file containing the code quality violation. | +| Name | Description | +|-----------------------------------------------------------|-------------| +| `description` | A description of the code quality violation. | +| `check_name` | A unique name representing the static analysis check that emitted this issue. | +| `fingerprint` | A unique fingerprint to identify the code quality violation. For example, an MD5 hash. | +| `severity` | A severity string (can be `info`, `minor`, `major`, `critical`, or `blocker`). | +| `location.path` | The relative path to the file containing the code quality violation. | | `location.lines.begin` or `location.positions.begin.line` | The line on which the code quality violation occurred. | NOTE: @@ -630,8 +630,9 @@ To work around this problem, set `TIMEOUT_SECONDS` to a higher value in your `.g For example: ```yaml -variables: - TIMEOUT_SECONDS: 3600 +code_quality: + variables: + TIMEOUT_SECONDS: 3600 ``` ### Using Code Quality with Kubernetes CI executor diff --git a/doc/ci/testing/fail_fast_testing.md b/doc/ci/testing/fail_fast_testing.md index 6b1ed4a594d..733c190616c 100644 --- a/doc/ci/testing/fail_fast_testing.md +++ b/doc/ci/testing/fail_fast_testing.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Fail Fast Testing **(PREMIUM)** +# Fail Fast Testing **(PREMIUM ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198550) in GitLab 13.1. diff --git a/doc/ci/testing/index.md b/doc/ci/testing/index.md index a8fb6d688d7..dadbc3821cc 100644 --- a/doc/ci/testing/index.md +++ b/doc/ci/testing/index.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Test with GitLab CI/CD and generate reports in merge requests **(FREE)** +# Test with GitLab CI/CD and generate reports in merge requests **(FREE ALL)** Use GitLab CI/CD to test the changes included in a feature branch. You can also display reports or link to important information directly from [merge requests](../../user/project/merge_requests/index.md). @@ -18,12 +18,12 @@ display reports or link to important information directly from [merge requests]( | [Code Quality](code_quality.md) | Analyze your source code quality using the [Code Climate](https://codeclimate.com/) analyzer and show the Code Climate report right in the merge request widget area. | | [Display arbitrary job artifacts](../yaml/index.md#artifactsexpose_as) | Configure CI pipelines with the `artifacts:expose_as` parameter to directly link to selected [artifacts](../jobs/job_artifacts.md) in merge requests. | | [Unit test reports](unit_test_reports.md) | Configure your CI jobs to use Unit test reports, and let GitLab display a report on the merge request so that it's easier and faster to identify the failure without having to check the entire job log. | -| [License Compliance](../../user/compliance/license_compliance/index.md) | Manage the licenses of your dependencies. | +| [License Scanning](../../user/compliance/license_scanning_of_cyclonedx_files/index.md) | Manage the licenses of your dependencies. | | [Metrics Reports](metrics_reports.md) | Display the Metrics Report on the merge request so that it's fast and easier to identify changes to important metrics. | | [Test Coverage visualization](test_coverage_visualization.md) | See test coverage results for merge requests, in the file diff. | | [Fail fast testing](fail_fast_testing.md) | Run a subset of your RSpec test suite, so failed tests stop the pipeline before the full suite of tests run, saving resources. | -## Security Reports **(ULTIMATE)** +## Security Reports **(ULTIMATE ALL)** In addition to the reports listed above, GitLab can do many types of [Security reports](../../user/application_security/index.md), generated by scanning and reporting any vulnerabilities found in your project: diff --git a/doc/ci/testing/load_performance_testing.md b/doc/ci/testing/load_performance_testing.md index dac4dd555b0..549aa10287e 100644 --- a/doc/ci/testing/load_performance_testing.md +++ b/doc/ci/testing/load_performance_testing.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Load Performance Testing **(PREMIUM)** +# Load Performance Testing **(PREMIUM ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10683) in GitLab 13.2. diff --git a/doc/ci/testing/metrics_reports.md b/doc/ci/testing/metrics_reports.md index 9257ba8990e..0f1eaedfb78 100644 --- a/doc/ci/testing/metrics_reports.md +++ b/doc/ci/testing/metrics_reports.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Metrics Reports **(PREMIUM)** +# Metrics Reports **(PREMIUM ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9788) in GitLab 11.10. Requires GitLab Runner 11.10 and above. diff --git a/doc/ci/testing/test_coverage_visualization.md b/doc/ci/testing/test_coverage_visualization.md index 669157daa21..62b757a93c3 100644 --- a/doc/ci/testing/test_coverage_visualization.md +++ b/doc/ci/testing/test_coverage_visualization.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Test coverage visualization **(FREE)** +# Test coverage visualization **(FREE ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3708) in GitLab 12.9. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/249811) in GitLab 13.5. @@ -19,7 +19,7 @@ MR is merged. ## How test coverage visualization works -Collecting the coverage information is done via GitLab CI/CD's +Collecting the coverage information is done by using the GitLab CI/CD [artifacts reports feature](../yaml/index.md#artifactsreports). You can specify one or more coverage reports to collect, including wildcard paths. GitLab then takes the coverage information in all the files and combines it @@ -41,8 +41,7 @@ Other coverage analysis frameworks support the format out of the box, for exampl - [Coverage.py](https://coverage.readthedocs.io/en/coverage-5.0.4/cmd.html#xml-reporting) (Python) - [PHPUnit](https://github.com/sebastianbergmann/phpunit-documentation-english/blob/master/src/textui.rst#command-line-options) (PHP) -Once configured, if you create a merge request that triggers a pipeline which collects -coverage reports, the coverage is shown in the diff view. This includes reports +After configuration, if your merge request triggers a pipeline that collects coverage reports, the coverage information is displayed in the diff view. This includes reports from any job in any stage in the pipeline. The coverage displays for each line: - `covered` (green): lines which have been checked at least once by tests @@ -431,6 +430,7 @@ the coverage report itself and verify that: to match the files in your repository. - The pipeline has completed. If the pipeline is [blocked on a manual job](../jobs/job_control.md#types-of-manual-jobs), the pipeline is not considered complete. +- The coverage report file does not exceed the [limits](#limits). Report artifacts are not downloadable by default. If you want the report to be downloadable from the job details page, add your coverage report to the artifact `paths`: diff --git a/doc/ci/testing/unit_test_report_examples.md b/doc/ci/testing/unit_test_report_examples.md index 8527d0b712d..fb11467cd7d 100644 --- a/doc/ci/testing/unit_test_report_examples.md +++ b/doc/ci/testing/unit_test_report_examples.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Unit test report examples **(FREE)** +# Unit test report examples **(FREE ALL)** [Unit test reports](unit_test_reports.md) can be generated for many languages and packages. Use these examples as guidelines for configuring your pipeline to generate unit test reports diff --git a/doc/ci/testing/unit_test_reports.md b/doc/ci/testing/unit_test_reports.md index 88a5d90d30e..a8b4c36db7f 100644 --- a/doc/ci/testing/unit_test_reports.md +++ b/doc/ci/testing/unit_test_reports.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Unit test reports **(FREE)** +# Unit test reports **(FREE ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45318) in GitLab 11.2. Requires GitLab Runner 11.2 and above. > - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39737) from JUnit test reports to Unit test reports in GitLab 13.4. diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index a47eaaf0381..506f6fb2106 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -5,19 +5,19 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: tutorial --- -# Trigger pipelines by using the API **(FREE)** +# Trigger pipelines by using the API **(FREE ALL)** To trigger a pipeline for a specific branch or tag, you can use an API call to the [pipeline triggers API endpoint](../../api/pipeline_triggers.md). When authenticating with the API, you can use: -- A [trigger token](#create-a-trigger-token) to trigger a branch or tag pipeline. +- A [pipeline trigger token](#create-a-pipeline-trigger-token) to trigger a branch or tag pipeline. - A [CI/CD job token](../jobs/ci_job_token.md) to [trigger a multi-project pipeline](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api). -## Create a trigger token +## Create a pipeline trigger token -You can trigger a pipeline for a branch or tag by generating a trigger token and using it +You can trigger a pipeline for a branch or tag by generating a pipeline trigger token and using it to authenticate an API call. The token impersonates a user's project access and permissions. Prerequisite: @@ -28,8 +28,9 @@ To create a trigger token: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > CI/CD**. -1. Expand **Pipeline triggers**. -1. Enter a description and select **Add trigger**. +1. Expand **Pipeline trigger tokens**. +1. Select **Add new token** +1. Enter a description and select **Create pipeline trigger token**. - You can view and copy the full token for all triggers you have created. - You can only see the first 4 characters for tokens created by other project members. @@ -41,12 +42,12 @@ to improve the security of trigger tokens. ## Trigger a pipeline -After you [create a trigger token](#create-a-trigger-token), you can use it to trigger +After you [create a pipeline trigger token](#create-a-pipeline-trigger-token), you can use it to trigger pipelines with a tool that can access the API, or a webhook. ### Use cURL -You can use cURL to trigger pipelines with the [pipeline triggers API endpoint](../../api/pipeline_triggers.md). +You can use cURL to trigger pipelines with the [pipeline trigger token API endpoint](../../api/pipeline_triggers.md). For example: - Use a multiline cURL command: @@ -62,7 +63,7 @@ For example: ```shell curl --request POST \ - "https://gitlab.example.com/api/v4/projects/<project_id>/trigger/pipeline?token=<token>&ref=<ref_name>" + "https://gitlab.example.com/api/v4/projects/<project_id>/trigger/pipeline?token=<token>&ref=<ref_name>" ``` In each example, replace: @@ -75,7 +76,7 @@ In each example, replace: ### Use a CI/CD job -You can use a CI/CD job with a triggers token to trigger pipelines when another pipeline +You can use a CI/CD job with a pipeline triggers token to trigger pipelines when another pipeline runs. For example, to trigger a pipeline on the `main` branch of `project-B` when a tag @@ -116,9 +117,9 @@ Replace: - `<ref_name>` with a branch or tag name, like `main`. This value takes precedence over the `ref_name` in the webhook payload. The payload's `ref` is the branch that fired the trigger in the source repository. You must URL-encode the `ref_name` if it contains slashes. -- `<token>` with your trigger token. +- `<token>` with your pipeline trigger token. -#### Use a webhook payload +#### Access webhook payload > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31197) in GitLab 13.9. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/321027) in GitLab 13.11. @@ -138,10 +139,10 @@ The parameter is of the form `variables[key]=value`, for example: ```shell curl --request POST \ - --form token=TOKEN \ - --form ref=main \ - --form "variables[UPLOAD_TO_S3]=true" \ - "https://gitlab.example.com/api/v4/projects/123456/trigger/pipeline" + --form token=TOKEN \ + --form ref=main \ + --form variables[UPLOAD_TO_S3]="true" \ + "https://gitlab.example.com/api/v4/projects/123456/trigger/pipeline" ``` CI/CD variables in triggered pipelines display on each job's page, but only @@ -149,9 +150,9 @@ users with the Owner and Maintainer role can view the values.  -## Revoke a trigger token +## Revoke a pipeline trigger token -To revoke a trigger token: +To revoke a pipeline trigger token: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > CI/CD**. @@ -162,23 +163,24 @@ A revoked trigger token cannot be added back. ## Configure CI/CD jobs to run in triggered pipelines -To [configure when to run jobs](../jobs/job_control.md) in triggered pipelines: +To [configure when to run jobs](../jobs/job_control.md) in triggered pipelines, you can: - Use [`rules`](../yaml/index.md#rules) with the `$CI_PIPELINE_SOURCE` [predefined CI/CD variable](../variables/predefined_variables.md). -- Use [`only`/`except`](../yaml/index.md#onlyrefs--exceptrefs) keywords. +- Use [`only`/`except`](../yaml/index.md#onlyrefs--exceptrefs) keywords, though `rules` + is the preferred keyword. | `$CI_PIPELINE_SOURCE` value | `only`/`except` keywords | Trigger method | |-----------------------------|--------------------------|---------------------| -| `trigger` | `triggers` | In pipelines triggered with the [pipeline triggers API](../../api/pipeline_triggers.md) by using a [trigger token](#create-a-trigger-token). | +| `trigger` | `triggers` | In pipelines triggered with the [pipeline triggers API](../../api/pipeline_triggers.md) by using a [trigger token](#create-a-pipeline-trigger-token). | | `pipeline` | `pipelines` | In [multi-project pipelines](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api) triggered with the [pipeline triggers API](../../api/pipeline_triggers.md) by using the [`$CI_JOB_TOKEN`](../jobs/ci_job_token.md), or by using the [`trigger`](../yaml/index.md#trigger) keyword in the CI/CD configuration file. | Additionally, the `$CI_PIPELINE_TRIGGERED` predefined CI/CD variable is set to `true` -in pipelines triggered with a trigger token. +in pipelines triggered with a pipeline trigger token. -## See which trigger token was used +## See which pipeline trigger token was used -You can see which trigger caused a job to run by visiting the single job page. -A part of the trigger's token displays on the right of the page, under the job details: +You can see which pipeline trigger token caused a job to run by visiting the single job page. +A part of the trigger token displays on the right of the page, under the job details:  @@ -196,7 +198,7 @@ To avoid trigger loops, do not use [pipeline events](../../user/project/integrat A response of `{"message":"404 Not Found"}` when triggering a pipeline might be caused by using a [personal access token](../../user/profile/personal_access_tokens.md) -instead of a trigger token. [Create a new trigger token](#create-a-trigger-token) +instead of a pipeline trigger token. [Create a new trigger token](#create-a-pipeline-trigger-token) and use it instead of the personal access token. ### `The requested URL returned error: 400` when triggering a pipeline diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 623589e2cbc..3d5bcc64889 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Troubleshooting CI/CD **(FREE)** +# Troubleshooting CI/CD **(FREE ALL)** GitLab provides several tools to help make troubleshooting your pipelines easier. @@ -283,6 +283,36 @@ If the merge train pipeline was canceled before the merge request was merged, wi - Add it to the train again. +### Merge request rules widget shows a scan result policy is invalid or duplicated **(ULTIMATE SELF)** + +On GitLab self-managed 15.0 and later, the most likely cause is that the project was exported from a +group and imported into another, and had scan result policy rules. These rules are stored in a +separate project to the one that was exported. As a result, the project contains policy rules that +reference entities that don't exist in the imported project's group. The result is policy rules that +are invalid, duplicated, or both. + +To remove all invalid scan result policy rules from a GitLab instance, an administrator can run +the following script in the [Rails console](../administration/operations/rails_console.md). + +```ruby +Project.joins(:approval_rules).where(approval_rules: { report_type: %i[scan_finding license_scanning] }).where.not(approval_rules: { security_orchestration_policy_configuration_id: nil }).find_in_batches.flat_map do |batch| + batch.map do |project| + # Get projects and their configuration_ids for applicable project rules + [project, project.approval_rules.where(report_type: %i[scan_finding license_scanning]).pluck(:security_orchestration_policy_configuration_id).uniq] + end.uniq.map do |project, configuration_ids| # We take only unique combinations of project + configuration_ids + # If we find more configurations than what is available for the project, we take records with the extra configurations + [project, configuration_ids - project.all_security_orchestration_policy_configurations.pluck(:id)] + end.select { |_project, configuration_ids| configuration_ids.any? } +end.each do |project, configuration_ids| + # For each found pair project + ghost configuration, we remove these rules for a given project + Security::OrchestrationPolicyConfiguration.where(id: configuration_ids).each do |configuration| + configuration.delete_scan_finding_rules_for_project(project.id) + end + # Ensure we sync any potential rules from new group's policy + Security::ScanResultPolicies::SyncProjectWorker.perform_async(project.id) +end +``` + ### Project `group/project` not found or access denied This message is shown if configuration is added with [`include`](yaml/index.md#include) and one of the following: @@ -330,6 +360,8 @@ start a new pipeline to use the new configuration. ### Unable to pull image from another project +> **Allow access to this project with a CI_JOB_TOKEN** setting [renamed to **Limit access _to_ this project**](https://gitlab.com/gitlab-org/gitlab/-/issues/411406) in GitLab 16.3. + When a runner tries to pull an image from a private project, the job could fail with the following error: ```shell @@ -338,7 +370,7 @@ WARNING: Failed to pull image with policy "always": Error response from daemon: This error can happen if the following are both true: -- The **Allow access to this project with a CI_JOB_TOKEN** option is enabled in the private project +- The **Limit access _to_ this project** option is enabled in the private project hosting the image. - The job attempting to fetch the image is running for a project that is not listed in the private project's allowlist. diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index bd066797ada..6280c9080ab 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# GitLab CI/CD variables **(FREE)** +# GitLab CI/CD variables **(FREE ALL)** CI/CD variables are a type of environment variable. You can use them to: diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 73aaafe46c0..6acb254e76f 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Predefined variables reference **(FREE)** +# Predefined variables reference **(FREE ALL)** Predefined [CI/CD variables](index.md) are available in every GitLab CI/CD pipeline. @@ -59,12 +59,12 @@ as it can cause the pipeline to behave unexpectedly. | `CI_DEPLOY_USER` | 10.8 | all | The authentication username of the [GitLab Deploy Token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token), if the project has one. | | `CI_DISPOSABLE_ENVIRONMENT` | all | 10.1 | Only available if the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). `true` when available. | | `CI_ENVIRONMENT_NAME` | 8.15 | all | The name of the environment for this job. Available if [`environment:name`](../yaml/index.md#environmentname) is set. | -| `CI_ENVIRONMENT_SLUG` | 8.15 | all | The simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, and so on. Available if [`environment:name`](../yaml/index.md#environmentname) is set. The slug is [truncated to 24 characters](https://gitlab.com/gitlab-org/gitlab/-/issues/20941). | +| `CI_ENVIRONMENT_SLUG` | 8.15 | all | The simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, and so on. Available if [`environment:name`](../yaml/index.md#environmentname) is set. The slug is [truncated to 24 characters](https://gitlab.com/gitlab-org/gitlab/-/issues/20941). A random suffix is automatically added to [uppercase environment names](https://gitlab.com/gitlab-org/gitlab/-/issues/415526). | | `CI_ENVIRONMENT_URL` | 9.3 | all | The URL of the environment for this job. Available if [`environment:url`](../yaml/index.md#environmenturl) is set. | | `CI_ENVIRONMENT_ACTION` | 13.11 | all | The action annotation specified for this job's environment. Available if [`environment:action`](../yaml/index.md#environmentaction) is set. Can be `start`, `prepare`, or `stop`. | | `CI_ENVIRONMENT_TIER` | 14.0 | all | The [deployment tier of the environment](../environments/index.md#deployment-tier-of-environments) for this job. | | `CI_RELEASE_DESCRIPTION` | 15.5 | all | The description of the release. Available only on pipelines for tags. Description length is limited to first 1024 characters.| -| `CI_GITLAB_FIPS_MODE` | 14.10 | all | The configuration setting for whether FIPS mode is enabled in the GitLab instance. | +| `CI_GITLAB_FIPS_MODE` | 14.10 | all | Only available if [FIPS mode](../../development/fips_compliance.md) is enabled in the GitLab instance. `true` when available. | | `CI_HAS_OPEN_REQUIREMENTS` | 13.1 | all | Only available if the pipeline's project has an open [requirement](../../user/project/requirements/index.md). `true` when available. | | `CI_JOB_ID` | 9.0 | all | The internal ID of the job, unique across all jobs in the GitLab instance. | | `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the Docker image running the job. | @@ -76,7 +76,7 @@ as it can cause the pipeline to behave unexpectedly. | `CI_JOB_NAME_SLUG` | 15.4 | all | `CI_JOB_NAME_SLUG` in lowercase, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in paths. | | `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the job's stage. | | `CI_JOB_STATUS` | all | 13.5 | The status of the job as each runner stage is executed. Use with [`after_script`](../yaml/index.md#after_script). Can be `success`, `failed`, or `canceled`. | -| `CI_JOB_TIMEOUT` | 15.7 | 15.7 | The job timeout value. | +| `CI_JOB_TIMEOUT` | 15.7 | 15.7 | The job timeout, in seconds. | | `CI_JOB_TOKEN` | 9.0 | 1.2 | A token to authenticate with [certain API endpoints](../jobs/ci_job_token.md). The token is valid as long as the job is running. | | `CI_JOB_URL` | 11.1 | 0.5 | The job details URL. | | `CI_JOB_STARTED_AT` | 13.10 | all | The UTC datetime when a job started, in [ISO 8601](https://www.rfc-editor.org/rfc/rfc3339#appendix-A) format. | @@ -92,6 +92,7 @@ as it can cause the pipeline to behave unexpectedly. | `CI_PIPELINE_TRIGGERED` | all | all | `true` if the job was [triggered](../triggers/index.md). | | `CI_PIPELINE_URL` | 11.1 | 0.5 | The URL for the pipeline details. | | `CI_PIPELINE_CREATED_AT` | 13.10 | all | The UTC datetime when the pipeline was created, in [ISO 8601](https://www.rfc-editor.org/rfc/rfc3339#appendix-A) format. | +| `CI_PIPELINE_NAME` | 16.3 | all | The pipeline name defined in [`workflow:name`](../yaml/index.md#workflowname) | | `CI_PROJECT_DIR` | all | all | The full path the repository is cloned to, and where the job runs from. If the GitLab Runner `builds_dir` parameter is set, this variable is set relative to the value of `builds_dir`. For more information, see the [Advanced GitLab Runner configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). | | `CI_PROJECT_ID` | all | all | The ID of the current project. This ID is unique across all projects on the GitLab instance. | | `CI_PROJECT_NAME` | 8.10 | 0.5 | The name of the directory for the project. For example if the project URL is `gitlab.example.com/group-name/project-1`, `CI_PROJECT_NAME` is `project-1`. | @@ -143,7 +144,7 @@ as it can cause the pipeline to behave unexpectedly. | `GITLAB_USER_LOGIN` | 10.0 | all | The username of the user who started the pipeline, unless the job is a manual job. In manual jobs, the value is the username of the user who started the job. | | `GITLAB_USER_NAME` | 10.0 | all | The name of the user who started the pipeline, unless the job is a manual job. In manual jobs, the value is the name of the user who started the job. | | `KUBECONFIG` | 14.2 | all | The path to the `kubeconfig` file with contexts for every shared agent connection. Only available when a [GitLab agent is authorized to access the project](../../user/clusters/agent/ci_cd_workflow.md#authorize-the-agent). | -| `TRIGGER_PAYLOAD` | 13.9 | all | The webhook payload. Only available when a pipeline is [triggered with a webhook](../triggers/index.md#use-a-webhook-payload). | +| `TRIGGER_PAYLOAD` | 13.9 | all | The webhook payload. Only available when a pipeline is [triggered with a webhook](../triggers/index.md#access-webhook-payload). | ## Predefined variables for merge request pipelines diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index 14dfc5dd551..52edeb67f0e 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Where variables can be used **(FREE)** +# Where variables can be used **(FREE ALL)** As it's described in the [CI/CD variables](index.md) documentation, you can define many different variables. Some of them can be used for all GitLab CI/CD diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index 37cb7efdf94..fa7e941ffe5 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -4,7 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# GitLab CI/CD artifacts reports types **(FREE)** +# GitLab CI/CD artifacts reports types **(FREE ALL)** Use [`artifacts:reports`](index.md#artifactsreports) to: @@ -40,7 +40,7 @@ GitLab can display the results of one or more reports in the merge request For more information, see [Accessibility testing](../testing/accessibility_testing.md). -## `artifacts:reports:api_fuzzing` **(ULTIMATE)** +## `artifacts:reports:api_fuzzing` **(ULTIMATE ALL)** > - Introduced in GitLab 13.4. > - Requires GitLab Runner 13.4 or later. @@ -55,7 +55,7 @@ GitLab can display the results of one or more reports in: - The pipeline [**Security** tab](../../user/application_security/vulnerability_report/pipeline.md#view-vulnerabilities-in-a-pipeline). - The [security dashboard](../../user/application_security/api_fuzzing/index.md#security-dashboard). -## `artifacts:reports:browser_performance` **(PREMIUM)** +## `artifacts:reports:browser_performance` **(PREMIUM ALL)** > [Name changed](https://gitlab.com/gitlab-org/gitlab/-/issues/225914) from `artifacts:reports:performance` in GitLab 14.0. @@ -106,7 +106,7 @@ GitLab can display the results of one or more reports in: - The merge request [diff annotations](../testing/code_quality.md#merge-request-changes-view). - The [full report](../testing/metrics_reports.md). -## `artifacts:reports:container_scanning` **(ULTIMATE)** +## `artifacts:reports:container_scanning` **(ULTIMATE ALL)** The `container_scanning` report collects [Container Scanning vulnerabilities](../../user/application_security/container_scanning/index.md). The collected Container Scanning report uploads to GitLab as an artifact. @@ -118,7 +118,7 @@ GitLab can display the results of one or more reports in: - The [security dashboard](../../user/application_security/security_dashboard/index.md). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). -## `artifacts:reports:coverage_fuzzing` **(ULTIMATE)** +## `artifacts:reports:coverage_fuzzing` **(ULTIMATE ALL)** > - Introduced in GitLab 13.4. > - Requires GitLab Runner 13.4 or later. @@ -157,7 +157,7 @@ artifacts: - gl-sbom-bundler-gem.cdx.json ``` -## `artifacts:reports:dast` **(ULTIMATE)** +## `artifacts:reports:dast` **(ULTIMATE ALL)** The `dast` report collects [DAST vulnerabilities](../../user/application_security/dast/index.md). The collected DAST report uploads to GitLab as an artifact. @@ -169,7 +169,7 @@ GitLab can display the results of one or more reports in: - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). - The [security dashboard](../../user/application_security/security_dashboard/index.md). -## `artifacts:reports:dependency_scanning` **(ULTIMATE)** +## `artifacts:reports:dependency_scanning` **(ULTIMATE ALL)** The `dependency_scanning` report collects [Dependency Scanning vulnerabilities](../../user/application_security/dependency_scanning/index.md). The collected Dependency Scanning report uploads to GitLab as an artifact. @@ -245,19 +245,20 @@ concatenate them into a single file. Use either: - A combination of both (`junit: [rspec.xml, test-results/TEST-*.xml]`). - Directories are not supported(`junit: test-results`, `junit: test-results/**`). -## `artifacts:reports:license_scanning` **(ULTIMATE)** +<!--- start_remove The following content will be removed on remove_date: '2023-11-22' --> -> Introduced in GitLab 12.8. +## `artifacts:reports:license_scanning` **(ULTIMATE ALL)** -The License Compliance report collects [Licenses](../../user/compliance/license_compliance/index.md). The License -Compliance report uploads to GitLab as an artifact. +> Introduced in GitLab 12.8. -GitLab can display the results of one or more reports in: +The license scanning report was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387561) +in GitLab 15.9 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/421363) in GitLab 16.3. +You should instead migrate to use [License approval policies](../../user/compliance/license_approval_policies.md) and +the [new method of license scanning](../../user/compliance/license_scanning_of_cyclonedx_files/index.md). -- The merge request [license compliance widget](../../user/compliance/license_compliance/index.md). -- The [license list](../../user/compliance/license_list.md). +<!--- end_remove --> -## `artifacts:reports:load_performance` **(PREMIUM)** +## `artifacts:reports:load_performance` **(PREMIUM ALL)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35260) in GitLab 13.2. > - Requires GitLab Runner 11.5 and above. @@ -270,7 +271,7 @@ GitLab can display the results of only one report in the merge request GitLab cannot display the combined results of multiple `load_performance` reports. -## `artifacts:reports:metrics` **(PREMIUM)** +## `artifacts:reports:metrics` **(PREMIUM ALL)** The `metrics` report collects [Metrics](../testing/metrics_reports.md). The collected Metrics report uploads to GitLab as an artifact. @@ -278,7 +279,7 @@ artifact. GitLab can display the results of one or more reports in the merge request [metrics reports widget](../testing/metrics_reports.md#metrics-reports). -## `artifacts:reports:requirements` **(ULTIMATE)** +## `artifacts:reports:requirements` **(ULTIMATE ALL)** > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2859) in GitLab 13.1. @@ -297,7 +298,7 @@ report uploads to GitLab as an artifact. GitLab can display the results of one or more reports in: -- The merge request [SAST widget](../../user/application_security/sast/index.md#static-application-security-testing-sast). +- The merge request [SAST widget](../../user/application_security/sast/index.md). - The [security dashboard](../../user/application_security/security_dashboard/index.md). ## `artifacts:reports:secret_detection` diff --git a/doc/ci/yaml/gitlab_ci_yaml.md b/doc/ci/yaml/gitlab_ci_yaml.md index 7b4834b3719..920abf50546 100644 --- a/doc/ci/yaml/gitlab_ci_yaml.md +++ b/doc/ci/yaml/gitlab_ci_yaml.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# The `.gitlab-ci.yml` file **(FREE)** +# The `.gitlab-ci.yml` file **(FREE ALL)** To use GitLab CI/CD, you need: diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index 3b6419dbff2..79eb42fd781 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Use CI/CD configuration from other files **(FREE)** +# Use CI/CD configuration from other files **(FREE ALL)** You can use [`include`](index.md#include) to include external YAML files in your CI/CD jobs. @@ -589,6 +589,60 @@ In this example: - `user` is optional. If not defined, the value is `test-user`. - `flags` is optional. If not defined, it has no value. +### Specify functions to manipulate input values + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/409462) in GitLab 16.3. + +You can specify predefined functions in the interpolation block to manipulate the input value. +The format supported is the following: + +```yaml +$[[ input.input-id | <function1> | <function2> | ... <functionN> ]] +``` + +Details: + +- Only [predefined interpolation functions](#predefined-interpolation-functions) are permitted. +- A maximum of 3 functions may be specified in a single interpolation block. +- The functions are executed in the sequence they are specified. + +```yaml +spec: + inputs: + test: + default: '0123456789' +--- + +test-job: + script: echo $[[ inputs.test | truncate(1,3) ]] +``` + +In this example: + +- The function [`truncate`](#truncate) applies to the value of `inputs.test`. +- Assuming the value of `inputs.test` is `0123456789`, then the output of `script` would be `echo 123`. + +### Predefined interpolation functions + +#### Truncate + +Use `truncate` to shorten the interpolated value. For example: + +- `truncate(<offset>,<length>)` + +| Name | Type | Description | +| ---- | ---- | ----------- | +| `offset` | Integer | Number of characters to offset by. | +| `length` | Integer | Number of characters to return after the offset. | + +Example: + +```yaml +$[[ inputs.test | truncate(3,5) ]] +``` + +Assuming the value of `inputs.test` is `0123456789`, then the output would be `34567`. + ### Set input parameter values with `include:inputs` > `include:with` [renamed to `include:inputs`](https://gitlab.com/gitlab-org/gitlab/-/issues/406780) in GitLab 16.0. diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index 9a060c35721..c4f7e6d6e01 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# `.gitlab-ci.yml` keyword reference **(FREE)** +# `.gitlab-ci.yml` keyword reference **(FREE ALL)** This document lists the configuration options for your GitLab `.gitlab-ci.yml` file. @@ -179,7 +179,7 @@ The time limit to resolve all files is 30 seconds. #### `include:local` -Use `include:local` to include a file that is in the same repository as the configuration file containing the `include` keyword. +Use `include:local` to include a file that is in the same repository and branch as the configuration file containing the `include` keyword. Use `include:local` instead of symbolic links. **Keyword type**: Global keyword. @@ -291,8 +291,8 @@ include: **Additional details**: -- All [nested includes](includes.md#use-nested-includes) execute without context as a public user, - so you can only include public projects or templates. +- All [nested includes](includes.md#use-nested-includes) are executed without context as a public user, + so you can only include public projects or templates. No variables are available in the `include` section of nested includes. - Be careful when including a remote CI/CD configuration file. No pipelines or notifications trigger when external CI/CD configuration files change. From a security perspective, this is similar to pulling a third-party dependency. @@ -330,8 +330,8 @@ include: **Additional details**: -- All [nested includes](includes.md#use-nested-includes) are executed only with the permission of the user, - so it's possible to use `project`, `remote`, or `template` includes. +- All [nested includes](includes.md#use-nested-includes) are executed without context as a public user, + so you can only include public projects or templates. No variables are available in the `include` section of nested includes. ### `stages` @@ -1554,7 +1554,7 @@ In this example: is not recorded or displayed. Check [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/280818) for more details. -### `dast_configuration` **(ULTIMATE)** +### `dast_configuration` **(ULTIMATE ALL)** > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5981) in GitLab 14.1. @@ -2389,6 +2389,8 @@ This example creates four paths of execution: it depends on all jobs created in parallel, not just one job. It also downloads artifacts from all the parallel jobs by default. If the artifacts have the same name, they overwrite each other and only the last one downloaded is saved. + - To have `needs` refer to a subset of parallelized jobs (and not all of the parallelized jobs), + use the [`needs:parallel:matrix`](#needsparallelmatrix) keyword. - In [GitLab 14.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/30632) you can refer to jobs in the same stage as the job you are configuring. This feature is enabled on GitLab.com and ready for production use. On self-managed [GitLab 14.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/30632) @@ -2454,7 +2456,7 @@ In this example: - In GitLab 12.6 and later, you can't combine the [`dependencies`](#dependencies) keyword with `needs`. -#### `needs:project` **(PREMIUM)** +#### `needs:project` **(PREMIUM ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14311) in GitLab 12.7. @@ -2679,6 +2681,61 @@ upstream_status: - If you add the `job` keyword to `needs:pipeline`, the job no longer mirrors the pipeline status. The behavior changes to [`needs:pipeline:job`](#needspipelinejob). +#### `needs:parallel:matrix` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254821) in GitLab 16.3. + +Jobs can use [`parallel:matrix`](#parallelmatrix) to run a job multiple times in parallel in a single pipeline, +but with different variable values for each instance of the job. + +Use `needs:parallel:matrix` to execute jobs out-of-order depending on parallelized jobs. + +**Keyword type**: Job keyword. You can use it only as part of a job. Must be used with `needs:job`. + +**Possible inputs**: An array of hashes of variables: + +- The variables and values must be selected from the variables and values defined in the `parallel:matrix` job. + +**Example of `needs:parallel:matrix`**: + +```yaml +linux:build: + stage: build + script: echo "Building linux..." + parallel: + matrix: + - PROVIDER: aws + STACK: + - monitoring + - app1 + - app2 + +linux:rspec: + stage: test + needs: + - job: linux:build + parallel: + matrix: + - PROVIDER: aws + - STACK: app1 + script: echo "Running rspec on linux..." +``` + +The above example generates the following jobs: + +```plaintext +linux:build: [aws, monitoring] +linux:build: [aws, app1] +linux:build: [aws, app2] +linux:rspec +``` + +The `linux:rspec` job runs as soon as the `linux:build: [aws, app1]` job finishes. + +**Related topics**: + +- [Specify a parallelized job using needs with multiple parallelized jobs](../jobs/job_control.md#specify-a-parallelized-job-using-needs-with-multiple-parallelized-jobs). + ### `only` / `except` NOTE: @@ -3858,7 +3915,7 @@ job2: - [Create custom collapsible sections](../jobs/index.md#custom-collapsible-sections) to simplify job log output. -### `secrets` **(PREMIUM)** +### `secrets` **(PREMIUM ALL)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33014) in GitLab 13.4. @@ -3868,8 +3925,6 @@ Use `secrets` to specify [CI/CD secrets](../secrets/index.md) to: - Make available in the job as [CI/CD variables](../variables/index.md) ([`file` type](../variables/index.md#use-file-type-cicd-variables) by default). -This keyword must be used with `secrets:vault`. - #### `secrets:vault` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/28321) in GitLab 13.4 and GitLab Runner 13.4. @@ -3920,6 +3975,34 @@ job: vault: production/db/password@ops # Translates to secret: `ops/data/production/db`, field: `password` ``` +#### `secrets:azure_key_vault` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271271) in GitLab 16.3 and GitLab Runner 16.3. + +Use `secrets:azure_key_vault` to specify secrets provided by a [Azure Key Vault](https://azure.microsoft.com/en-us/products/key-vault/). + +**Keyword type**: Job keyword. You can use it only as part of a job. + +**Possible inputs**: + +- `name`: Name of the secret. +- `version`: Version of the secret. + +**Example of `secrets:azure_key_vault`**: + +```yaml +job: + secrets: + DATABASE_PASSWORD: + azure_key_vault: + name: 'test' + version: 'test' +``` + +**Related topics**: + +- [Use Azure Key Vault secrets in GitLab CI/CD](../secrets/azure_key_vault.md). + #### `secrets:file` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250695) in GitLab 14.1 and GitLab Runner 14.1. diff --git a/doc/ci/yaml/script.md b/doc/ci/yaml/script.md index 1c31191c2f4..71b9e9fbe0b 100644 --- a/doc/ci/yaml/script.md +++ b/doc/ci/yaml/script.md @@ -4,7 +4,7 @@ group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Format scripts and job logs **(FREE)** +# Format scripts and job logs **(FREE ALL)** You can use special syntax in [`script`](index.md#script) sections to: diff --git a/doc/ci/yaml/signing_examples.md b/doc/ci/yaml/signing_examples.md index 5609bd2374e..72e007a749f 100644 --- a/doc/ci/yaml/signing_examples.md +++ b/doc/ci/yaml/signing_examples.md @@ -21,26 +21,38 @@ For details on the mapping between GitLab OIDC claims and Fulcio certificate ext - You must be using GitLab.com. - Your project's CI/CD configuration must be located in the project. -- You must use a version of Cosign that is `>= 2.0.1`. -## Container images +## Sign or verify container images and build artifacts by using Cosign + +You can use Cosign to sign and verify container images and build artifacts. -### Sign a container image with Cosign +**Requirements:** -GitLab [ID tokens](../secrets/id_token_authentication.md) can be used by Cosign for -[keyless signing](https://docs.sigstore.dev/cosign/overview/#example-working-with-containers). The token must have `sigstore` set as the -[`aud`](../secrets/id_token_authentication.md#token-payload) claim. The token can be used by Cosign automatically when -it is set in the `SIGSTORE_ID_TOKEN` environment variable. +- You must use a version of Cosign that is `>= 2.0.1`. **Best practices**: -- Build and sign a container image in the same job to prevent the image from being tampered with before it is - signed. +- Build and sign an image/artifact in the same job to prevent it from being tampered with before it is signed. +- When signing container images, sign the digest (which is immutable) instead of the tag. + +GitLab [ID tokens](../secrets/id_token_authentication.md#id-tokens) can be used by Cosign for +[keyless signing](https://docs.sigstore.dev/cosign/overview/). The token must have +`sigstore` set as the [`aud`](../secrets/id_token_authentication.md#token-payload) claim. The token can be used by Cosign automatically when it is set in the +`SIGSTORE_ID_TOKEN` environment variable. + +To learn more about how to install Cosign, see [Cosign Installation documentation](https://docs.sigstore.dev/cosign/installation/). -See [Cosign documentation](https://docs.sigstore.dev/cosign/signing_with_containers/) to learn more about signing. +### Signing + +#### Container images + +The example below demonstrates how to sign a container image in GitLab CI. The signature is automatically stored in the +same container repository as the image. + +To learn more about signing containers, see [Cosign Signing Containers documentation](https://docs.sigstore.dev/cosign/signing_with_containers/). ```yaml -build_and_sign: +build_and_sign_image: stage: build image: docker:latest services: @@ -60,10 +72,52 @@ build_and_sign: - cosign sign $IMAGE_DIGEST ``` -### Verify a container image with Cosign +#### Build artifacts + +The example below demonstrates how to sign a build artifact in GitLab CI. You should save the `cosign.bundle` file +produced by `cosign sign-blob`, which is used for signature verification. + +To learn more about signing artifacts, see [Cosign Signing Blobs documentation](https://docs.sigstore.dev/cosign/signing_with_blobs/#keyless-signing-of-blobs-and-files). + +```yaml +build_and_sign_artifact: + stage: build + image: alpine:latest + variables: + COSIGN_YES: "true" + id_tokens: + SIGSTORE_ID_TOKEN: + aud: sigstore + before_script: + - apk add --update cosign + script: + - echo "This is a build artifact" > artifact.txt + - cosign sign-blob artifact.txt --bundle cosign.bundle + artifacts: + paths: + - artifact.txt + - cosign.bundle +``` + +### Verification + +**Command-line arguments** + +| Name | Value | +|-----------------------------|-------| +| `--certificate-identity` | The SAN of the signing certificate issued by Fulcio. Can be constructed with the following information from the project where the image/artifact was signed: GitLab instance URL + project path + `//` + CI config path + `@` + ref path. | +| `--certificate-oidc-issuer` | The GitLab instance URL where the image/artifact was signed. For example, `https://gitlab.com`. | +| `--bundle` | The `bundle` file produced by `cosign sign-blob`. Only used for verifying build artifacts. | + +To learn more about verifying signed images/artifacts, see [Cosign Verifying documentation](https://docs.sigstore.dev/cosign/verify/#keyless-verification-using-openid-connect). + +#### Container images + +The example below demonstrates how to verify a signed container image in GitLab CI. The command-line arguments are +described [above](#verification). ```yaml -verify: +verify_image: image: alpine:3.18 stage: verify before_script: @@ -73,6 +127,22 @@ verify: - cosign verify "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA" --certificate-identity "https://gitlab.com/my-group/my-project//path/to/.gitlab-ci.yml@refs/heads/main" --certificate-oidc-issuer "https://gitlab.com" ``` +#### Build artifacts + +The example below demonstrates how to verify a signed build artifact in GitLab CI. Verifying an artifact requires both +the artifact itself and the `cosign.bundle` file produced by `cosign sign-blob`. The command-line arguments are +described [above](#verification). + +```yaml +verify_artifact: + stage: verify + image: alpine:latest + before_script: + - apk add --update cosign + script: + - cosign verify-blob artifact.txt --bundle cosign.bundle --certificate-identity "https://gitlab.com/my-group/my-project//path/to/.gitlab-ci.yml@refs/heads/main" --certificate-oidc-issuer "https://gitlab.com" +``` + ## Use Sigstore and npm to generate keyless provenance You can use Sigstore and npm, together with GitLab CI/CD, to digitally sign build artifacts without the overhead of key management. @@ -253,58 +323,3 @@ predicate: digest: sha1: 6e02e901e936bfac3d4691984dff8c505410cbc3 ``` - -## Build artifacts - -You can use Cosign to both sign build artifacts and verify artifacts that were signed with Cosign. - -### Sign a build artifact with Cosign - -GitLab [ID tokens](../secrets/id_token_authentication.md) can be used by Cosign for keyless signing of build artifacts. -The token must have `sigstore` set as the [`aud`](../secrets/id_token_authentication.md#token-payload) claim. The token can be used by Cosign automatically when -it is set in the `SIGSTORE_ID_TOKEN` environment variable. - -**Best practices**: - -- Build and sign an artifact in the same job to prevent the artifact from being tampered with before it is - signed. - -To learn more about signing artifacts, see [Cosign documentation](https://docs.sigstore.dev/cosign/signing_with_blobs/#keyless-signing-of-blobs-and-files). - -```yaml -sign_artifact: - stage: build - image: alpine:latest - variables: - COSIGN_YES: "true" - id_tokens: - SIGSTORE_ID_TOKEN: - aud: sigstore - before_script: - - apk add --update cosign - script: - - echo "This is a build artifact" > artifact.txt - - cosign sign-blob artifact.txt --bundle cosign.bundle - artifacts: - paths: - - artifact.txt - - cosign.bundle -``` - -### Verify a build artifact with Cosign - -Verifying an artifact requires both the artifact itself and the `cosign.bundle` file produced by `cosign sign-blob`. -The `--certificate-identity` option should reference the project and branch where the artifact was signed. The -`--certificate-oidc-issuer` option should reference the GitLab instance where the artifact was signed. - -To learn more about verifying signed artifacts, see [Cosign documentation](https://docs.sigstore.dev/cosign/verify/). - -```yaml -verify_artifact: - stage: verify - image: alpine:latest - before_script: - - apk add --update cosign - script: - - cosign verify-blob artifact.txt --bundle cosign.bundle --certificate-identity "https://gitlab.com/my-group/my-project//path/to/.gitlab-ci.yml@refs/heads/main" --certificate-oidc-issuer "https://gitlab.com" -``` diff --git a/doc/ci/yaml/workflow.md b/doc/ci/yaml/workflow.md index e88a96ae1f5..af275a15f9c 100644 --- a/doc/ci/yaml/workflow.md +++ b/doc/ci/yaml/workflow.md @@ -4,7 +4,7 @@ group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# GitLab CI/CD `workflow` keyword **(FREE)** +# GitLab CI/CD `workflow` keyword **(FREE ALL)** Use the [`workflow`](index.md#workflow) keyword to control when pipelines are created. @@ -15,12 +15,12 @@ for tags, but the workflow prevents tag pipelines, the job never runs. Some example `if` clauses for `workflow: rules`: -| Example rules | Details | -|------------------------------------------------------|-----------------------------------------------------------| -| `if: '$CI_PIPELINE_SOURCE == "merge_request_event"'` | Control when merge request pipelines run. | +| Example rules | Details | +|------------------------------------------------------|---------| +| `if: '$CI_PIPELINE_SOURCE == "merge_request_event"'` | Control when merge request pipelines run. | | `if: '$CI_PIPELINE_SOURCE == "push"'` | Control when both branch pipelines and tag pipelines run. | -| `if: $CI_COMMIT_TAG` | Control when tag pipelines run. | -| `if: $CI_COMMIT_BRANCH` | Control when branch pipelines run. | +| `if: $CI_COMMIT_TAG` | Control when tag pipelines run. | +| `if: $CI_COMMIT_BRANCH` | Control when branch pipelines run. | See the [common `if` clauses for `rules`](../jobs/job_control.md#common-if-clauses-for-rules) for more examples. @@ -117,8 +117,8 @@ does not block triggered pipelines. ### Git Flow with merge request pipelines -You can use `workflow: rules` as part of [Git Flow or similar strategies](../../topics/gitlab_flow.md) -with merge request pipelines. With these rules, you can use [merge request pipeline features](../pipelines/merge_request_pipelines.md) +You can use `workflow: rules` with merge request pipelines. With these rules, +you can use [merge request pipeline features](../pipelines/merge_request_pipelines.md) with feature branches, while keeping long-lived branches to support multiple versions of your software. diff --git a/doc/ci/yaml/yaml_optimization.md b/doc/ci/yaml/yaml_optimization.md index 2cfda1116fe..84be9c85676 100644 --- a/doc/ci/yaml/yaml_optimization.md +++ b/doc/ci/yaml/yaml_optimization.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# Optimize GitLab CI/CD configuration files **(FREE)** +# Optimize GitLab CI/CD configuration files **(FREE ALL)** You can reduce complexity and duplicated configuration in your GitLab CI/CD configuration files by using: |
