Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/doc/ci
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2024-01-16 13:42:19 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2024-01-16 13:42:19 +0300
commit84d1bd786125c1c14a3ba5f63e38a4cc736a9027 (patch)
treef550fa965f507077e20dbb6d61a8269a99ef7107 /doc/ci
parent3a105e36e689f7b75482236712f1a47fd5a76814 (diff)
Add latest changes from gitlab-org/gitlab@16-8-stable-eev16.8.0-rc42
Diffstat (limited to 'doc/ci')
-rw-r--r--doc/ci/caching/index.md2
-rw-r--r--doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md2
-rw-r--r--doc/ci/cloud_services/azure/index.md2
-rw-r--r--doc/ci/debugging.md2
-rw-r--r--doc/ci/docker/using_docker_build.md2
-rw-r--r--doc/ci/environments/img/kubernetes_summary_ui.pngbin47714 -> 13822 bytes
-rw-r--r--doc/ci/environments/index.md40
-rw-r--r--doc/ci/examples/end_to_end_testing_webdriverio/index.md2
-rw-r--r--doc/ci/index.md7
-rw-r--r--doc/ci/jobs/ci_job_token.md2
-rw-r--r--doc/ci/jobs/index.md2
-rw-r--r--doc/ci/jobs/job_artifacts.md14
-rw-r--r--doc/ci/jobs/job_artifacts_troubleshooting.md23
-rw-r--r--doc/ci/jobs/job_control.md46
-rw-r--r--doc/ci/large_repositories/index.md11
-rw-r--r--doc/ci/migration/bamboo.md6
-rw-r--r--doc/ci/migration/github_actions.md2
-rw-r--r--doc/ci/migration/jenkins.md2
-rw-r--r--doc/ci/mobile_devops.md28
-rw-r--r--doc/ci/pipelines/cicd_minutes.md3
-rw-r--r--doc/ci/pipelines/downstream_pipelines.md4
-rw-r--r--doc/ci/pipelines/index.md2
-rw-r--r--doc/ci/pipelines/merge_request_pipelines.md4
-rw-r--r--doc/ci/pipelines/merged_results_pipelines.md2
-rw-r--r--doc/ci/pipelines/pipeline_efficiency.md2
-rw-r--r--doc/ci/pipelines/schedules.md2
-rw-r--r--doc/ci/quick_start/index.md2
-rw-r--r--doc/ci/quick_start/tutorial.md6
-rw-r--r--doc/ci/review_apps/index.md2
-rw-r--r--doc/ci/runners/configure_runners.md33
-rw-r--r--doc/ci/runners/img/runner_fleet_dashboard.pngbin0 -> 348380 bytes
-rw-r--r--doc/ci/runners/index.md2
-rw-r--r--doc/ci/runners/new_creation_workflow.md8
-rw-r--r--doc/ci/runners/runner_fleet_dashboard.md90
-rw-r--r--doc/ci/runners/runners_scope.md7
-rw-r--r--doc/ci/runners/saas/macos_saas_runner.md5
-rw-r--r--doc/ci/secrets/gcp_secret_manager.md92
-rw-r--r--doc/ci/secrets/index.md6
-rw-r--r--doc/ci/testing/code_quality.md107
-rw-r--r--doc/ci/testing/img/code_quality_inline_indicator_v16_7.pngbin53078 -> 22405 bytes
-rw-r--r--doc/ci/testing/test_coverage_visualization.md16
-rw-r--r--doc/ci/variables/index.md8
-rw-r--r--doc/ci/variables/predefined_variables.md3
-rw-r--r--doc/ci/variables/where_variables_can_be_used.md3
-rw-r--r--doc/ci/yaml/artifacts_reports.md10
-rw-r--r--doc/ci/yaml/index.md166
-rw-r--r--doc/ci/yaml/inputs.md68
47 files changed, 668 insertions, 180 deletions
diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md
index 55f18987490..737b18fb6c2 100644
--- a/doc/ci/caching/index.md
+++ b/doc/ci/caching/index.md
@@ -738,5 +738,5 @@ instance.
To share the cache between concurrent runners, you can either:
- Use the `[runners.docker]` section of the runners' `config.toml` to configure a single mount point on the host that
-is mapped to `/cache` in each container, preventing the runner from creating unique volume names.
+ is mapped to `/cache` in each container, preventing the runner from creating unique volume names.
- Use a distributed cache.
diff --git a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md
index b80bbbc5c98..82f92cdc938 100644
--- a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md
+++ b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md
@@ -276,7 +276,7 @@ include:
To use DAST on the default branch:
1. Set up a new [service](#create-an-ecs-service). This service will be used to deploy a temporary
-DAST environment.
+ DAST environment.
1. Use the `CI_AWS_ECS_SERVICE` variable to set the name.
1. Set the scope to the `dast-default` environment.
1. Add the following to your `.gitlab-ci.yml` file:
diff --git a/doc/ci/cloud_services/azure/index.md b/doc/ci/cloud_services/azure/index.md
index d3bd8e187ba..b3fcaabfdde 100644
--- a/doc/ci/cloud_services/azure/index.md
+++ b/doc/ci/cloud_services/azure/index.md
@@ -153,7 +153,7 @@ you should verify:
- For the `gitlab-group/gitlab-project` project and `main` branch it would be:
`project_path:gitlab-group/gitlab-project:ref_type:branch:ref:main`.
- The correct values of `mygroup` and `myproject` can be retrieved by checking the URL
- when accessing your GitLab project or by selecting the **Clone** option in the project.
+ when accessing your GitLab project or, in the upper-right corner of the project's overview page, selecting **Code**.
- The `Audience` defined in the Azure AD federated identity credentials, for example `https://gitlab.com`
or your own GitLab URL.
diff --git a/doc/ci/debugging.md b/doc/ci/debugging.md
index 8a60b5f649e..84715cda2f5 100644
--- a/doc/ci/debugging.md
+++ b/doc/ci/debugging.md
@@ -143,7 +143,7 @@ configuration into more independent [parent-child pipelines](../ci/pipelines/pip
Pipeline configuration warnings are shown when you:
-- [Validate configuration with the CI Lint tool](yaml/index.md).
+- [Validate configuration with the CI Lint tool](lint.md).
- [Manually run a pipeline](pipelines/index.md#run-a-pipeline-manually).
### `Job may allow multiple pipelines to run for a single action` warning
diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md
index beaa2291eea..bc157c7ae08 100644
--- a/doc/ci/docker/using_docker_build.md
+++ b/doc/ci/docker/using_docker_build.md
@@ -853,7 +853,7 @@ This indicates the GitLab Runner does not have permission to start the
1. Check that `privileged = true` is set in the `config.toml`.
1. Make sure the CI job has the right Runner tags to use these
-privileged runners.
+ privileged runners.
### Error: `cgroups: cgroup mountpoint does not exist: unknown`
diff --git a/doc/ci/environments/img/kubernetes_summary_ui.png b/doc/ci/environments/img/kubernetes_summary_ui.png
index ce51cd8e96f..f8eae88745e 100644
--- a/doc/ci/environments/img/kubernetes_summary_ui.png
+++ b/doc/ci/environments/img/kubernetes_summary_ui.png
Binary files differ
diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md
index 03c9a152d98..c9148e04bf3 100644
--- a/doc/ci/environments/index.md
+++ b/doc/ci/environments/index.md
@@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
Environments describe where code is deployed.
-Each time [GitLab CI/CD](../yaml/index.md) deploys a version of code to an environment,
+Each time [GitLab CI/CD](../index.md) deploys a version of code to an environment,
a deployment is created.
GitLab:
@@ -24,7 +24,7 @@ associated with your project, you can use it to assist with your deployments.
Prerequisites:
-- You must have at least the Reporter role.
+- In a private project, you must have at least the Reporter role. See [Environment permissions](#environment-permissions).
There are a few ways to view a list of environments for a given project:
@@ -108,7 +108,7 @@ To create a static environment, in your `.gitlab-ci.yml` file:
1. Define a job in the `deploy` stage.
1. In the job, define the environment `name` and `url`. If an
-environment of that name doesn't exist when the pipeline runs, it is created.
+ environment of that name doesn't exist when the pipeline runs, it is created.
NOTE:
Some characters cannot be used in environment names. For more information about the
@@ -912,6 +912,38 @@ like [Review Apps](../review_apps/index.md) (`review/*`).
The most specific spec takes precedence over the other wildcard matching. In this case,
the `review/feature-1` spec takes precedence over `review/*` and `*` specs.
+## Environment permissions
+
+Depending on your role, you can interact with environments in public
+and private projects.
+
+### View environments
+
+- In public projects, anyone can view a list of environments, including non-members.
+- In private projects, you must have at least the Reporter role to view a list of environments.
+
+### Create and update environments
+
+- You must have at least the Developer role to create a new environment, or update an existing unprotected environment.
+- If an existing environment is protected and you don't have access to it, you cannot update the environment.
+
+### Stop and delete environments
+
+- You must have at least the Developer role to stop or delete an unprotected environment.
+- If an environment is protected and you don't have access to it, you cannot stop or delete the environment.
+
+### Run deployment jobs in protected environments
+
+If you can push or merge to the protected branch:
+
+- You must have at least the Reporter role.
+
+If you can't push to the protected branch:
+
+- You must be a part of a group with the Reporter role.
+
+See [Deployment-only access to protected environments](protected_environments.md#deployment-only-access-to-protected-environments).
+
## Related topics
- [Dashboard for Kubernetes](kubernetes_dashboard.md)
@@ -1007,7 +1039,7 @@ deploy:
Since `$ENVIRONMENT` variable does not exist in the pipeline, GitLab tries to
create an environment with a name `production/`, which is invalid in
-[the environment name constraint](../yaml/index.md).
+[the environment name constraint](../yaml/index.md#environmentname).
To fix this, use one of the following solutions:
diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md
index 556af355b1a..8872602a027 100644
--- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md
+++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md
@@ -144,7 +144,7 @@ new browser window interacting with your app as you specified.
Which brings us to the exciting part: how do we run this in GitLab CI/CD? There are two things we
need to do for this:
-1. Set up [CI/CD jobs](../../yaml/index.md) that actually have a browser available.
+1. Set up [CI/CD jobs](../../jobs/index.md) that actually have a browser available.
1. Update our WebdriverIO configuration to use those browsers to visit the review apps.
For the scope of this article, we've defined an additional [CI/CD stage](../../yaml/index.md#stages)
diff --git a/doc/ci/index.md b/doc/ci/index.md
index beb7fffeb8a..429db0beede 100644
--- a/doc/ci/index.md
+++ b/doc/ci/index.md
@@ -2,7 +2,6 @@
stage: Verify
group: Pipeline Execution
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
-description: "Learn how to use GitLab CI/CD, the GitLab built-in Continuous Integration, Continuous Deployment, and Continuous Delivery toolset to build, test, and deploy your application."
---
# Get started with GitLab CI/CD **(FREE ALL)**
@@ -22,7 +21,7 @@ If you're new to GitLab CI/CD, start by reviewing some of the commonly used term
To use GitLab CI/CD, you start with a `.gitlab-ci.yml` file at the root of your project
which contains the configuration for your CI/CD pipeline. This file follows the YAML format
-and has its own special syntax.
+and has its own syntax.
You can name this file anything you want, but `.gitlab-ci.yml` is the most common name.
@@ -38,8 +37,8 @@ In the `.gitlab-ci.yml` file, you can define:
**Get started:**
- [Create your first `.gitlab-ci.yml` file](quick_start/index.md).
-- [View all the possible keywords that you can use in the `.gitlab-ci.yml` file](yaml/index.md).
-the configuration.
+- View all the possible keywords that you can use in the `.gitlab-ci.yml` file in
+ the [CI/CD YAML syntax reference](../ci/yaml/index.md).
- Use the [pipeline editor](pipeline_editor/index.md) to edit or [visualize](pipeline_editor/index.md#visualize-ci-configuration)
your CI/CD configuration.
diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md
index ba2d63c12e4..3a787189ac0 100644
--- a/doc/ci/jobs/ci_job_token.md
+++ b/doc/ci/jobs/ci_job_token.md
@@ -17,7 +17,7 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints
- [Container registry](../../user/packages/container_registry/build_and_push_images.md#use-gitlab-cicd)
(the `$CI_REGISTRY_PASSWORD` is `$CI_JOB_TOKEN`).
- [Container registry API](../../api/container_registry.md)
- (scoped to the job's project, when the `ci_job_token_scope` feature flag is enabled).
+ (scoped to the job's project).
- [Get job artifacts](../../api/job_artifacts.md#get-job-artifacts).
- [Get job token's job](../../api/jobs.md#get-job-tokens-job).
- [Pipeline triggers](../../api/pipeline_triggers.md), using the `token=` parameter
diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md
index f4836f93234..3cba8787821 100644
--- a/doc/ci/jobs/index.md
+++ b/doc/ci/jobs/index.md
@@ -134,7 +134,7 @@ jobs. Select to expand them.
![Grouped pipelines](img/pipeline_grouped_jobs_v14_2.png)
-To create a group of jobs, in the [CI/CD pipeline configuration file](../yaml/index.md),
+To create a group of jobs, in the [`.gitlab-ci.yml` file](../index.md#the-gitlab-ciyml-file),
separate each job name with a number and one of the following:
- A slash (`/`), for example, `slash-test 1/3`, `slash-test 2/3`, `slash-test 3/3`.
diff --git a/doc/ci/jobs/job_artifacts.md b/doc/ci/jobs/job_artifacts.md
index f93068faf01..34da3be9370 100644
--- a/doc/ci/jobs/job_artifacts.md
+++ b/doc/ci/jobs/job_artifacts.md
@@ -261,9 +261,17 @@ from:
- The **Artifacts** page. On the right of the job, select **Browse** (**{folder-open}**).
If [GitLab Pages](../../administration/pages/index.md) is enabled in the project, you can preview
-HTML files in the artifacts directly in your browser. If the project is internal or private, you must
-enable [GitLab Pages access control](../../administration/pages/index.md#access-control) to preview
-HTML files.
+some artifacts file extensions directly in your browser. If the project is internal or private, you must enable [GitLab Pages access control](../../administration/pages/index.md#access-control) to enable the preview.
+
+The following extensions are supported:
+
+| File extension | GitLab.com | Linux package with built-in NGINX |
+|----------|---------------------|--------------|
+| `.html` | **{check-circle}** Yes | **{check-circle}** Yes |
+| `.json` | **{check-circle}** Yes | **{check-circle}** Yes |
+| `.xml` | **{check-circle}** Yes | **{check-circle}** Yes |
+| `.txt` | **{dotted-circle}** No | **{check-circle}** Yes |
+| `.log` | **{dotted-circle}** No | **{check-circle}** Yes |
### From a URL
diff --git a/doc/ci/jobs/job_artifacts_troubleshooting.md b/doc/ci/jobs/job_artifacts_troubleshooting.md
index 0b7777d2d82..470c1bf4b55 100644
--- a/doc/ci/jobs/job_artifacts_troubleshooting.md
+++ b/doc/ci/jobs/job_artifacts_troubleshooting.md
@@ -155,3 +155,26 @@ To troubleshoot this error, verify that:
parent-child pipeline hierarchy.
- The `pipeline` and `job` combination exists and resolves to an existing pipeline.
- `dependency-job` has run and finished successfully.
+
+## Jobs show `UnlockPipelinesInQueueWorker` after an upgrade
+
+Jobs might stall and show an error that states `UnlockPipelinesInQueueWorker`.
+
+This issue occurs after an upgrade.
+
+The workaround is to enable the `ci_unlock_pipelines_extra_low` feature flag.
+To toggle feature flags, you must be an administrator.
+
+On GitLab SaaS:
+
+- Run the following [ChatOps](../chatops/index.md) command:
+
+ ```ruby
+ /chatops run feature set ci_unlock_pipelines_extra_low true
+ ```
+
+On GitLab self-managed:
+
+- [Enable the feature flag](../../administration/feature_flags.md) named `ci_unlock_pipelines_extra_low`.
+
+For more information see the comment in [merge request 140318](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/140318#note_1718600424).
diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md
index b432106a2d8..f7b3fed7d74 100644
--- a/doc/ci/jobs/job_control.md
+++ b/doc/ci/jobs/job_control.md
@@ -1218,3 +1218,49 @@ a branch that has an open merge request associated with it.
To [prevent duplicate pipelines](#avoid-duplicate-pipelines), use
[`workflow: rules`](../yaml/index.md#workflow) or rewrite your rules to control
which pipelines can run.
+
+### `This GitLab CI configuration is invalid` for variable expressions
+
+You might receive one of several `This GitLab CI configuration is invalid` errors
+when working with [CI/CD variable expressions](#cicd-variable-expressions).
+These syntax errors can be caused by incorrect usage of quote characters.
+
+In variable expressions, strings should be quoted, while variables should not be quoted.
+For example:
+
+```yaml
+variables:
+ ENVIRONMENT: production
+
+job:
+ script: echo
+ rules:
+ - if: $ENVIRONMENT == "production"
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+```
+
+In this example, both `if:` clauses are valid because the `production` string is quoted,
+and the CI/CD variables are unquoted.
+
+On the other hand, these `if:` clauses are all invalid:
+
+```yaml
+variables:
+ ENVIRONMENT: production
+
+job:
+ script: echo
+ rules: # These rules all cause YAML syntax errors:
+ - if: ${ENVIRONMENT} == "production"
+ - if: "$ENVIRONMENT" == "production"
+ - if: $ENVIRONMENT == production
+ - if: "production" == "production"
+```
+
+In this example:
+
+- `if: ${ENVIRONMENT} == "production"` is invalid, because `${ENVIRONMENT}` is not valid
+ formatting for CI/CD variables in `if:`.
+- `if: "$ENVIRONMENT" == "production"` is invalid, because the variable is quoted.
+- `if: $ENVIRONMENT == production` is invalid, because the string is not quoted.
+- `if: "production" == "production"` is invalid, because there is no CI/CD variable to compare.
diff --git a/doc/ci/large_repositories/index.md b/doc/ci/large_repositories/index.md
deleted file mode 100644
index cef0554bf6c..00000000000
--- a/doc/ci/large_repositories/index.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../../user/project/repository/managing_large_repositories.md'
-remove_date: '2023-11-30'
----
-
-This document was moved to [another location](../../user/project/repository/managing_large_repositories.md).
-
-<!-- This redirect file can be deleted after <2023-11-30>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/ci/migration/bamboo.md b/doc/ci/migration/bamboo.md
index ea9dd666176..b2594e7571e 100644
--- a/doc/ci/migration/bamboo.md
+++ b/doc/ci/migration/bamboo.md
@@ -14,7 +14,7 @@ exported from the Bamboo UI or stored in Spec repositories.
If you are new to GitLab CI/CD, use the [Getting started guide](../index.md) to learn
the basic concepts and how to create your first [`.gitlab-ci.yml` file](../quick_start/index.md).
-If you already have some experience using GitLab CI/CD, you can review [keywords reference documentation](../yaml/index.md)
+If you already have some experience using GitLab CI/CD, you can review [CI/CD YAML syntax reference](../yaml/index.md)
to see the full list of available keywords.
You can also take a look at [Auto DevOps](../../topics/autodevops/index.md), which automatically
@@ -77,7 +77,7 @@ Bamboo Specs can also be [repository-stored](https://confluence.atlassian.com/ba
#### `.gitlab-ci.yml` configuration file
-GitLab, by default, uses a [`.gitlab-ci.yml` file](../yaml/index.md) for CI/CD configuration.
+GitLab, by default, uses a [`.gitlab-ci.yml` file](../index.md#the-gitlab-ciyml-file) for CI/CD configuration.
Alternatively, [Auto DevOps](../../topics/autodevops/index.md) can automatically build,
test, and deploy your application without a manually configured `.gitlab-ci.yml` file.
@@ -754,7 +754,7 @@ Before doing any migration work, you should first:
- Follow tutorials to create [your first GitLab pipeline](../quick_start/index.md)
and [more complex pipelines](../quick_start/tutorial.md) that build, test, and deploy
a static site.
- - Review the [`.gitlab-ci.yml` keyword reference](../yaml/index.md).
+ - Review the [CI/CD YAML syntax reference](../yaml/index.md).
1. Set up and configure GitLab.
1. Test your GitLab instance.
- Ensure [runners](../runners/index.md) are available, either by using shared GitLab.com runners or installing new runners.
diff --git a/doc/ci/migration/github_actions.md b/doc/ci/migration/github_actions.md
index ec55f129c1e..6afb6cebbff 100644
--- a/doc/ci/migration/github_actions.md
+++ b/doc/ci/migration/github_actions.md
@@ -672,7 +672,7 @@ Before doing any migration work, you should first:
1. Get familiar with GitLab.
- Read about the [key GitLab CI/CD features](../../ci/index.md).
- Follow tutorials to create [your first GitLab pipeline](../quick_start/index.md) and [more complex pipelines](../quick_start/tutorial.md) that build, test, and deploys a static site.
- - Review the [`.gitlab-ci.yml` keyword reference](../yaml/index.md).
+ - Review the [CI/CD YAML syntax reference](../yaml/index.md).
1. Set up and configure GitLab.
1. Test your GitLab instance.
- Ensure [runners](../runners/index.md) are available, either by using shared GitLab.com runners or installing new runners.
diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md
index f430b1ac7b9..961799b9564 100644
--- a/doc/ci/migration/jenkins.md
+++ b/doc/ci/migration/jenkins.md
@@ -699,7 +699,7 @@ Before doing any migration work, you should first:
1. Get familiar with GitLab.
- Read about the [key GitLab CI/CD features](../../ci/index.md).
- Follow tutorials to create [your first GitLab pipeline](../quick_start/index.md) and [more complex pipelines](../quick_start/tutorial.md) that build, test, and deploys a static site.
- - Review the [`.gitlab-ci.yml` keyword reference](../yaml/index.md).
+ - Review the [CI/CD YAML syntax reference](../yaml/index.md).
1. Set up and configure GitLab.
1. Test your GitLab instance.
- Ensure [runners](../runners/index.md) are available, either by using shared GitLab.com runners or installing new runners.
diff --git a/doc/ci/mobile_devops.md b/doc/ci/mobile_devops.md
index e871a95d29f..4639967fb1d 100644
--- a/doc/ci/mobile_devops.md
+++ b/doc/ci/mobile_devops.md
@@ -338,26 +338,34 @@ To create an iOS distribution with the Apple Store integration and fastlane, you
1. Generate an API Key for App Store Connect API. In the Apple App Store Connect portal,
[generate a new private key for your project](https://developer.apple.com/documentation/appstoreconnectapi/creating_api_keys_for_app_store_connect_api).
-1. [Enable the Apple App Store integration](#enable-apple-app-store-integration).
+1. [Enable the Apple App Store Connect integration](#enable-the-apple-app-store-connect-integration).
1. Add the release step to your pipeline and fastlane configuration.
<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
-For an overview, see [Apple App Store integration demo](https://youtu.be/CwzAWVgJeK8).
+For an overview, see [Apple App Store Connect integration demo](https://youtu.be/CwzAWVgJeK8).
+<!-- Video published on 2023-03-17 -->
-#### Enable Apple App Store Integration
+#### Enable the Apple App Store Connect integration
-Use the [Apple App Store integration](../user/project/integrations/apple_app_store.md)
-to configure your CI/CD pipelines to connect to [App Store Connect](https://appstoreconnect.apple.com/)
-to build and release apps for iOS, iPadOS, macOS, tvOS, and watchOS. To enable the integration:
+Prerequisites:
+
+- You must have an Apple ID enrolled in the [Apple Developer Program](https://developer.apple.com/programs/enroll/).
+- You must [generate a new private key](https://developer.apple.com/documentation/appstoreconnectapi/creating_api_keys_for_app_store_connect_api) for your project in the Apple App Store Connect portal.
+
+Use the Apple App Store Connect integration to configure your CI/CD pipelines to connect to [App Store Connect](https://appstoreconnect.apple.com).
+With this integration, you can build and release apps for iOS, iPadOS, macOS, tvOS, and watchOS.
+
+To enable the Apple App Store Connect integration in GitLab:
1. On the left sidebar, select **Search or go to** and find your project.
1. Select **Settings > Integrations**.
-1. Select **Apple App Store**.
+1. Select **Apple App Store Connect**.
1. Under **Enable integration**, select the **Active** checkbox.
1. Provide the Apple App Store Connect configuration information:
- - **Issuer ID**: You can find the Apple App Store Connect Issuer ID in the **Keys** section under **Users and Access** in the Apple App Store Connect portal.
- - **Key ID**: The key ID of the new private key that was just generated.
- - **Private Key**: The private key that was just generated. You can only download this key one time.
+ - **Issuer ID**: The Apple App Store Connect issuer ID.
+ - **Key ID**: The key ID of the generated private key.
+ - **Private key**: The generated private key. You can download this key only once.
+ - **Protected branches and tags only**: Enable to set variables on protected branches and tags only.
1. Select **Save changes**.
With the integration enabled, you can use fastlane to distribute a build to TestFlight
diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md
index 699136ccf97..a6c3bb835d0 100644
--- a/doc/ci/pipelines/cicd_minutes.md
+++ b/doc/ci/pipelines/cicd_minutes.md
@@ -1,6 +1,7 @@
---
stage: Verify
group: Pipeline Execution
+description: Calculations, quotas, purchase information.
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -321,7 +322,7 @@ Jobs on project runners are not affected by the compute quota.
### GitLab SaaS usage notifications
-On GitLab SaaS an email notification is sent to the namespace owners when:
+On GitLab SaaS an in-app banner is displayed and an email notification sent to the namespace owners when:
- The remaining compute minutes is below 30% of the quota.
- The remaining compute minutes is below 5% of the quota.
diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md
index 61862a20436..ae2ca74e6f8 100644
--- a/doc/ci/pipelines/downstream_pipelines.md
+++ b/doc/ci/pipelines/downstream_pipelines.md
@@ -379,7 +379,9 @@ trigger_job:
::EndTabs
-### View multi-project pipelines in pipeline graphs **(PREMIUM ALL)**
+### View multi-project pipelines in pipeline graphs
+
+> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/422282) from GitLab Premium to GitLab Free in 16.8.
After you trigger a multi-project pipeline, the downstream pipeline displays
to the right of the [pipeline graph](index.md#visualize-pipelines).
diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md
index 957e0e0de27..0a4f4c3762b 100644
--- a/doc/ci/pipelines/index.md
+++ b/doc/ci/pipelines/index.md
@@ -67,7 +67,7 @@ Pipelines and their component jobs and stages are defined in the CI/CD pipeline
- [Jobs](../jobs/index.md) are the basic configuration component.
- Stages are defined by using the [`stages`](../yaml/index.md#stages) keyword.
-For a list of configuration options in the CI pipeline file, see the [GitLab CI/CD Pipeline Configuration Reference](../yaml/index.md).
+For a list of configuration options in the CI pipeline file, see the [CI/CD YAML syntax reference](../yaml/index.md).
You can also configure specific aspects of your pipelines through the GitLab UI. For example:
diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md
index 0de55d2a488..25358ecd602 100644
--- a/doc/ci/pipelines/merge_request_pipelines.md
+++ b/doc/ci/pipelines/merge_request_pipelines.md
@@ -58,7 +58,7 @@ The three types of merge request pipelines are:
To use merge request pipelines:
-- Your project's [CI/CD configuration file](../yaml/index.md) must be configured with
+- Your project's [`.gitlab-ci.yml` file](../index.md#the-gitlab-ciyml-file) must be configured with
jobs that run in merge request pipelines. To do this, you can use:
- [`rules`](#use-rules-to-add-jobs).
- [`only/except`](#use-only-to-add-jobs).
@@ -160,7 +160,7 @@ GitLab shows a warning that you must accept before the pipeline runs. Otherwise,
Prerequisites:
-- The parent project's [CI/CD configuration file](../yaml/index.md) must be configured to
+- The parent project's [`.gitlab-ci.yml` file](../index.md#the-gitlab-ciyml-file) must be configured to
[run jobs in merge request pipelines](#prerequisites).
- You must be a member of the parent project with [permissions to run CI/CD pipelines](../../user/permissions.md#gitlab-cicd-permissions).
You might need additional permissions if the branch is protected.
diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md
index 213a07f49c4..691de7c3f3c 100644
--- a/doc/ci/pipelines/merged_results_pipelines.md
+++ b/doc/ci/pipelines/merged_results_pipelines.md
@@ -28,7 +28,7 @@ and [is labeled as `merge request`](merge_request_pipelines.md#types-of-merge-re
To use merged results pipelines:
-- Your project's [CI/CD configuration file](../yaml/index.md) must be configured to
+- Your project's [`.gitlab-ci.yml` file](../index.md#the-gitlab-ciyml-file) must be configured to
[run jobs in merge request pipelines](merge_request_pipelines.md#prerequisites).
- Your repository must be a GitLab repository, not an
[external repository](../ci_cd_for_external_repos/index.md).
diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md
index 4a548c8c0af..07fc7c2dfd6 100644
--- a/doc/ci/pipelines/pipeline_efficiency.md
+++ b/doc/ci/pipelines/pipeline_efficiency.md
@@ -27,7 +27,7 @@ The easiest indicators to check for inefficient pipelines are the runtimes of th
stages, and the total runtime of the pipeline itself. The total pipeline duration is
heavily influenced by the:
-- [Size of the repository](../../user/project/repository/managing_large_repositories.md)
+- [Size of the repository](../../user/project/repository/monorepos/index.md)
- Total number of stages and jobs.
- Dependencies between jobs.
- The ["critical path"](#directed-acyclic-graphs-dag-visualization), which represents
diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md
index f83868952f5..9ea635792f3 100644
--- a/doc/ci/pipelines/schedules.md
+++ b/doc/ci/pipelines/schedules.md
@@ -15,7 +15,7 @@ For a scheduled pipeline to run:
- The schedule owner must have the Developer role. For pipelines on protected branches,
the schedule owner must be [allowed to merge](../../user/project/protected_branches.md#add-protection-to-existing-branches)
to the branch.
-- The [CI/CD configuration](../yaml/index.md) must be valid.
+- The [`.gitlab-ci.yml` file](../index.md#the-gitlab-ciyml-file) must have valid syntax.
Otherwise, the pipeline is not created. No error message is displayed.
diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md
index 3da4e5ad023..0dc07a36ef5 100644
--- a/doc/ci/quick_start/index.md
+++ b/doc/ci/quick_start/index.md
@@ -136,7 +136,7 @@ Now you can get started customizing your `.gitlab-ci.yml` and defining more adva
Here are some tips to get started working with the `.gitlab-ci.yml` file.
-For the complete `.gitlab-ci.yml` syntax, see [the full `.gitlab-ci.yml` keyword reference](../yaml/index.md).
+For the complete `.gitlab-ci.yml` syntax, see the full [CI/CD YAML syntax reference](../yaml/index.md).
- Use the [pipeline editor](../pipeline_editor/index.md) to edit your `.gitlab-ci.yml` file.
- Each job contains a script section and belongs to a stage:
diff --git a/doc/ci/quick_start/tutorial.md b/doc/ci/quick_start/tutorial.md
index 389309538e9..bb766436379 100644
--- a/doc/ci/quick_start/tutorial.md
+++ b/doc/ci/quick_start/tutorial.md
@@ -45,7 +45,7 @@ on GitLab.com:
- In the **Project name** field, enter the name of your project, for example `My Pipeline Tutorial Project`.
- Select **Initialize repository with a README**.
1. Select **Create project**.
-1. On the right of the **Project Overview** page for your project, select **Clone**
+1. On the project's overview page, in the upper-right corner, select **Code**
to find the clone paths for your project. Copy the SSH or HTTP path and use the path
to clone the project locally.
@@ -502,5 +502,5 @@ Use a merge request to commit this pipeline configuration to the default branch.
The file is simpler, but it should have the same behavior as the previous step.
You've just created a full pipeline and streamlined it to be more efficient. Nice work!
-Now you can take this knowledge, learn about [the rest of the `.gitlab-ci.yml` keywords](../yaml/index.md),
-and build your own pipelines.
+Now you can take this knowledge, learn about the rest of the `.gitlab-ci.yml` keywords
+in the [CI/CD YAML syntax reference](../yaml/index.md), and build your own pipelines.
diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md
index 3ff25cc8bab..5854704521b 100644
--- a/doc/ci/review_apps/index.md
+++ b/doc/ci/review_apps/index.md
@@ -108,7 +108,7 @@ The following are example projects that demonstrate review app configuration:
Other examples of review apps:
- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
-[Cloud Native Development with GitLab](https://www.youtube.com/watch?v=jfIyQEwrocw).
+ [Cloud Native Development with GitLab](https://www.youtube.com/watch?v=jfIyQEwrocw).
- [Review apps for Android](https://about.gitlab.com/blog/2020/05/06/how-to-create-review-apps-for-android-with-gitlab-fastlane-and-appetize-dot-io/).
## Route Maps
diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md
index 3b21d865d8b..6212c07ce47 100644
--- a/doc/ci/runners/configure_runners.md
+++ b/doc/ci/runners/configure_runners.md
@@ -903,18 +903,41 @@ variables:
| `CACHE_COMPRESSION_LEVEL` | To adjust compression ratio, set to `fastest`, `fast`, `default`, `slow`, or `slowest`. This setting works with the Fastzip archiver only, so the GitLab Runner feature flag [`FF_USE_FASTZIP`](https://docs.gitlab.com/runner/configuration/feature-flags.html#available-feature-flags) must also be enabled. |
| `CACHE_REQUEST_TIMEOUT` | Configure the maximum duration of cache upload and download operations for a single job in minutes. Default is `10` minutes. |
-## Artifact attestation
+## Artifact provenance metadata
> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28940) in GitLab Runner 15.1.
NOTE:
Zip archives are the only supported artifact type. Follow [the issue for details](https://gitlab.com/gitlab-org/gitlab/-/issues/367203).
-GitLab Runner can generate and produce attestation metadata for all build artifacts. To enable this feature, you must set the `RUNNER_GENERATE_ARTIFACTS_METADATA` environment variable to `true`. This variable can either be set globally or it can be set for individual jobs. The metadata is in rendered in a plain text `.json` file that's stored with the artifact. The file name is as follows: `{ARTIFACT_NAME}-metadata.json` where `ARTIFACT_NAME` is what was defined as the [name for the artifact](../jobs/job_artifacts.md#with-a-dynamically-defined-name) in the CI file. The file name, however, defaults to `artifacts-metadata.json` if no name was given to the build artifacts.
+Runners can generate and produce provenance metadata for all build artifacts.
-### Attestation format
+To enable artifact provenance data, set the `RUNNER_GENERATE_ARTIFACTS_METADATA` environment
+variable to `true`. You can set the variable as global or for individual jobs:
-The attestation metadata is generated in the [in-toto attestation format](https://github.com/in-toto/attestation) for spec version [v0.1](https://github.com/in-toto/attestation/tree/v0.1.0/spec). The following fields are populated by default:
+```yaml
+variables:
+ RUNNER_GENERATE_ARTIFACTS_METADATA: "true"
+
+job1:
+ variables:
+ RUNNER_GENERATE_ARTIFACTS_METADATA: "true"
+```
+
+The metadata renders in a plain text `.json` file stored with the artifact. The
+file name is `{ARTIFACT_NAME}-metadata.json`. `ARTIFACT_NAME` is the
+[name for the artifact](../jobs/job_artifacts.md#with-a-dynamically-defined-name)
+defined in the `.gitlab-ci.yml` file. If the name is not defined, the default file name is
+`artifacts-metadata.json`.
+
+### Provenance metadata format
+
+The provenance metadata is generated in the [in-toto attestation format](https://github.com/in-toto/attestation) for spec version [0.1](https://github.com/in-toto/attestation/tree/v0.1.0/spec).
+The runner also produces a statement that adheres to SLSA v0.2 by default.
+
+To opt-in to an SLSA v1.0 statement, set the `SLSA_PROVENANCE_SCHEMA_VERSION=v1` variable in the `.gitlab-ci.yml` file. The v0.2 statement is deprecated and is planned to be removed in the GitLab 17.0 and the v1.0 statement is planned to become the new default format.
+
+The following fields are populated by default:
| Field | Value |
| ------ | ------ |
@@ -938,7 +961,7 @@ The attestation metadata is generated in the [in-toto attestation format](https:
| `metadata.completeness.environment` | Whether the builder's environment is reported. Always `true`. |
| `metadata.completeness.materials` | Whether the build materials are reported. Always `false`. |
-An example of an attestation that the GitLab Runner might generate is as follows:
+An example of provenance metadata that the GitLab Runner might generate is as follows:
```yaml
{
diff --git a/doc/ci/runners/img/runner_fleet_dashboard.png b/doc/ci/runners/img/runner_fleet_dashboard.png
new file mode 100644
index 00000000000..8c77b5a1aa9
--- /dev/null
+++ b/doc/ci/runners/img/runner_fleet_dashboard.png
Binary files differ
diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md
index 1df93ed8896..bfb30a36be2 100644
--- a/doc/ci/runners/index.md
+++ b/doc/ci/runners/index.md
@@ -33,7 +33,7 @@ When you use SaaS runners:
- The VM is active only for the duration of the job and immediately deleted. This means that any changes that your job makes to the virtual machine will not be available to a subsequent job.
- The virtual machine where your job runs has `sudo` access with no password.
- The storage is shared by the operating system, the image with pre-installed software, and a copy of your cloned repository.
-This means that the available free disk space for your jobs to use is reduced.
+ This means that the available free disk space for your jobs to use is reduced.
NOTE:
Jobs handled by SaaS runners on GitLab.com **time out after 3 hours**, regardless of the timeout configured in a project.
diff --git a/doc/ci/runners/new_creation_workflow.md b/doc/ci/runners/new_creation_workflow.md
index c870a89a77a..12bffb79e33 100644
--- a/doc/ci/runners/new_creation_workflow.md
+++ b/doc/ci/runners/new_creation_workflow.md
@@ -39,7 +39,7 @@ The new runner registration workflow has the following benefits:
- Preserved ownership records for runners, and minimized impact on users.
- The addition of a unique system ID ensures that you can reuse the same authentication token across
-multiple runners. For more information, see [Reusing a GitLab Runner configuration](https://docs.gitlab.com/runner/fleet_scaling/#reusing-a-gitlab-runner-configuration).
+ multiple runners. For more information, see [Reusing a GitLab Runner configuration](https://docs.gitlab.com/runner/fleet_scaling/#reusing-a-gitlab-runner-configuration).
## Estimated time frame for planned changes
@@ -61,7 +61,7 @@ To avoid a broken workflow, you must:
1. [Create a shared runner](runners_scope.md#create-a-shared-runner-with-a-runner-authentication-token) and obtain the authentication token.
1. Replace the registration token in your runner registration workflow with the
-authentication token.
+ authentication token.
## Using registration tokens after GitLab 17.0
@@ -159,7 +159,9 @@ Several runner configuration options cannot be set during runner registration. T
The following configuration options are no longer supported in [`values.yaml`](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/main/values.yaml):
```yaml
-## All these fields are DEPRECATED and the runner WILL FAIL TO START if you specify them
+## All these fields are DEPRECATED and the runner WILL FAIL TO START with GitLab Runner 18.0 and later if you specify them.
+## If a runner authentication token is specified in runnerRegistrationToken, the registration will succeed, however the
+## other values will be ignored.
runnerRegistrationToken: ""
locked: true
tags: ""
diff --git a/doc/ci/runners/runner_fleet_dashboard.md b/doc/ci/runners/runner_fleet_dashboard.md
new file mode 100644
index 00000000000..f329561cf4b
--- /dev/null
+++ b/doc/ci/runners/runner_fleet_dashboard.md
@@ -0,0 +1,90 @@
+---
+stage: Verify
+group: Runner
+info: >-
+ To determine the technical writer assigned to the Stage/Group associated with
+ this page, see
+ https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+# Runner Fleet Dashboard **(ULTIMATE)**
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/424495) in GitLab 16.6
+
+GitLab administrators can use the Runner Fleet Dashboard to assess the health of your instance runners.
+The Runner Fleet Dashboard shows:
+
+- Recent CI errors related caused by runner infrastructure.
+- Number of concurrent jobs executed on most busy runners.
+- Histogram of job queue times [(available only with ClickHouse)](#enable-more-ci-analytics-features-with-clickhouse).
+
+Support for usage and cost analysis are proposed in [epic 11183](https://gitlab.com/groups/gitlab-org/-/epics/11183).
+
+![Runner Fleet Dashboard](img/runner_fleet_dashboard.png)
+
+## View the Runner Fleet Dashboard
+
+Prerequisites:
+
+- You must be an administrator.
+
+To view the runner fleet dashboard:
+
+1. On the left sidebar, at the bottom, select **Admin Area**.
+1. Select **Runners**.
+1. Select **Fleet dashboard**.
+
+Most of the dashboard works without any additional actions, with the
+exception of **Wait time to pick a job** chart and features proposed in [epic 11183](https://gitlab.com/groups/gitlab-org/-/epics/11183).
+These features require [setting up an additional infrastructure](#enable-more-ci-analytics-features-with-clickhouse).
+
+## Export compute minutes used by instance runners
+
+Prerequisites:
+
+- You must be an administrator.
+
+To analyze runner usage, you can export a CSV file that contains the number of jobs and executed runner minutes. The
+CSV file shows the runner type and job status for each project. The CSV is sent to your email when the export is completed.
+
+To export compute minutes used by instance runners:
+
+1. On the left sidebar, at the bottom, select **Admin Area**.
+1. Select **Runners**.
+1. Select **Fleet dashboard**.
+1. Select **Export CSV**.
+
+## Enable more CI analytics features with ClickHouse **(ULTIMATE EXPERIMENT)**
+
+> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/11180) in GitLab 16.7 with the [flags](../../administration/feature_flags.md) named `ci_data_ingestion_to_click_house` and `clickhouse_ci_analytics`. Disabled by default.
+> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/424866) in GitLab 16.8. Feature flag `clickhouse_ci_analytics` removed.
+
+This feature is an [Experiment](../../policy/experiment-beta-support.md).
+To test it, we have launched an early adopters program.
+To join the list of users testing this feature, see
+[epic 11180](https://gitlab.com/groups/gitlab-org/-/epics/11180).
+
+### Enable ClickHouse integration
+
+To enable additional CI analytics features:
+
+1. [Configure ClickHouse integration](../../integration/clickhouse.md)
+1. [Enable](../../administration/feature_flags.md#how-to-enable-and-disable-features-behind-flags) the following feature flags:
+
+ | Feature flag name | Purpose | Status |
+ |------------------------------------|---------------------------------------------------------------------------|------------------------------------|
+ | `ci_data_ingestion_to_click_house` | Enables synchronization of new finished CI builds to ClickHouse database. | Enabled by default in GitLab 16.8. |
+ | `clickhouse_ci_analytics` | Enables the **Wait time to pick a job** chart. | Removed in 16.8. |
+
+<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
+For a video walkthrough, see [Setting up Runner Fleet Dashboard with ClickHouse](https://www.youtube.com/watch?v=YpGV95Ctbpk).
+
+## Feedback
+
+To help us improve the Runner Fleet Dashboard, you can provide feedback in
+[issue 421737](https://gitlab.com/gitlab-org/gitlab/-/issues/421737).
+In particular:
+
+- How easy or difficult it was to setup GitLab to make the dashboard work.
+- How useful you found the dashboard.
+- What other information you would like to see on that dashboard.
+- Any other related thoughts and ideas.
diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md
index 48f6a22e9b3..d6a556ffd9b 100644
--- a/doc/ci/runners/runners_scope.md
+++ b/doc/ci/runners/runners_scope.md
@@ -576,13 +576,16 @@ A runner can have one of the following statuses.
As an administrator, you can view runner statistics to learn about the performance of your runner fleet.
-- The **Median job queued time** value is calculated by sampling the queue duration of the
+The **Median job queued time** value is calculated by sampling the queue duration of the
most recent 100 jobs that were run by Instance runners. Jobs from only the latest 5000
runners are considered.
-- The median is a value that falls into the 50th percentile: half of the jobs
+
+The median is a value that falls into the 50th percentile: half of the jobs
queued for longer than the median value, and half of the jobs queued for less than the
median value.
+To view runner statistics:
+
1. On the left sidebar, at the bottom, select **Admin Area**.
1. Select **CI/CD > Runners**.
1. Select **View metrics**.
diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md
index 03c23680c1c..0251c523864 100644
--- a/doc/ci/runners/saas/macos_saas_runner.md
+++ b/doc/ci/runners/saas/macos_saas_runner.md
@@ -21,8 +21,7 @@ You can follow our work towards this goal in the
## Machine types available for macOS
-GitLab SaaS provides macOS build machines on Apple silicon (M1) chips.
-Intel x86-64 runners were deprecated in favor of Apple silicon. To build for an x86-64 target, use Rosetta 2 to emulate an Intel x86-64 build environment.
+GitLab SaaS provides macOS build machines on Apple silicon (M1) chips. To build for an x86-64 target, you can use Rosetta 2 to emulate an Intel x86-64 build environment.
| Runner Tag | vCPUS | Memory | Storage |
| ---------------------- | ----- | ------ | ------- |
@@ -40,7 +39,7 @@ in your `.gitlab-ci.yml` file. Each image runs a specific version of macOS and X
|----------------------------|--------|--------------|
| `macos-12-xcode-14` | `GA` | |
| `macos-13-xcode-14` | `GA` | [Preinstalled Software](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/job-images/-/blob/main/toolchain/macos-13.yml) |
-| `macos-14-xcode-15` | `Beta` | [Preinstalled Software](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/job-images/-/blob/main/toolchain/macos-14.yml) |
+| `macos-14-xcode-15` | `GA` | [Preinstalled Software](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/job-images/-/blob/main/toolchain/macos-14.yml) |
If no image is specified, the macOS runner uses `macos-13-xcode-14`.
diff --git a/doc/ci/secrets/gcp_secret_manager.md b/doc/ci/secrets/gcp_secret_manager.md
new file mode 100644
index 00000000000..ad2a2a269eb
--- /dev/null
+++ b/doc/ci/secrets/gcp_secret_manager.md
@@ -0,0 +1,92 @@
+---
+stage: Verify
+group: Pipeline Security
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Use GCP Secret Manager secrets in GitLab CI/CD **(PREMIUM ALL)**
+
+> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/11739) in GitLab and GitLab Runner 16.8.
+
+You can use secrets stored in the [Google Cloud (GCP) Secret Manager](https://cloud.google.com/security/products/secret-manager)
+in your GitLab CI/CD pipelines.
+
+The flow for using GitLab with GCP Secret Manager
+is summarized by this diagram:
+
+1. GitLab issues ID token to CI/CD job.
+1. The runner authenticates to GCP using an ID token.
+1. GCP verifies the ID token with GitLab.
+1. GCP issues a short-lived access token.
+1. The runner accesses the secret data using the access token.
+1. GCP checks IAM permission on the access token's principal.
+1. GCP returns the secret data to Runner.
+
+To use GitLab with GCP Secret Manager, you must:
+
+- Have secrets stored in [GCP Secret Manager](https://cloud.google.com/security/products/secret-manager).
+- Configure [GCP Workload Identity Federation](#configure-gcp-iam-workload-identify-federation-wif) to include GitLab as an identity provider.
+- Configure [GCP IAM](#grant-access-to-gcp-iam-principal) permissions to grant access to GCP Secret Manager.
+- Configure [GitLab CI/CD with GCP Secret Manager](#configure-gitlab-cicd-to-use-gcp-secret-manager-secrets).
+
+## Configure GCP IAM Workload Identify Federation (WIF)
+
+GCP IAM WIF must be configured to recognize ID tokens issued by GitLab and assign an appropriate principal to them.
+The principal is used to authorize access to the Secret Manager resources:
+
+1. In GCP Console, go to **IAM & Admin > Workload Identity Federation**.
+1. Select **CREATE POOL** and create a new identity pool with a unique name, for example `gitlab-pool`.
+1. Select **ADD PROVIDER** to add a new OIDC Provider to the Identity Pool with a unique name, for example `gitlab-provider`.
+ 1. Set **Issuer (URL)** to the GitLab URL, for example `https://gitlab.com`.
+ 1. Select **Default audience**, or select **Allowed audiences** for a custom audience, which is used in the `aud` for the GitLab CI/CD ID token.
+1. Under **Attribute Mapping**, configure provider attributes, which are mappings between the [OIDC claims](id_token_authentication.md#token-payload)
+ (referred to as "assertion") and Google attributes. These mappings can be used to set fine grained access control.
+ For example, to grant a GitLab project access to Secret Manager secrets, select **ADD MAPPING** and create a mapping of
+ `attribute.gitlab_project_id` to `assertion.project_id`.
+
+## Grant access to GCP IAM principal
+
+After setting up WIF, you must grant the WIF principal access to the secrets in Secret Manager.
+
+1. In GCP Console, go to **IAM & Admin > IAM**.
+1. Select **GRANT ACCESS** to grant access to the principal set created through the WIF provider. For example,
+ to grant IAM access to the principal matching the project with ID `123`, add
+ a new principal like: `principalSet://iam.googleapis.com/projects/[PROJECT_NUMBER]/locations/global/workloadIdentityPools/[POOL_ID]/attribute.gitlab_project_id/[PROJECT_ID]`.
+1. Assign the role **Secret Manager Secret Accessor**.
+1. (Optional) Select **IAM condition (Optional)** to add an IAM condition.
+ Under **Condition Builder**, you can add conditions. For example, you could add two `AND` conditions:
+ - First condition:
+ - **Condition type**: `Type`
+ - **Operator**: `is`
+ - **Resource type**: `secretmanager.googleapis.com/SecretVersion`
+ - Second condition:
+ - **Condition type**: `Name`
+ - **Operator**: `Starts with`
+ - **Value**: The pattern of secrets that you want to grant access to.
+
+You can add additional IAM conditions for fine-grained access controls, including
+accessing secrets with names starting with the project name.
+
+## Configure GitLab CI/CD to use GCP Secret Manager secrets
+
+You can use secrets stored in GCP Secret Manager in CI/CD jobs by defining them with the `gcp_secret_manager` keyword:
+
+```yaml
+job_using_gcp_sm:
+ id_tokens:
+ GCP_ID_TOKEN:
+ # `aud` must match the audience defined in the WIF Identity Pool.
+ aud: https://iam.googleapis.com/projects/1234/locations/global/workloadIdentityPools/gitlab-pool/providers/gitlab-provider
+ secrets:
+ DATABASE_PASSWORD:
+ gcp_secret_manager:
+ name: my-project-secret # This is the name of the secret defined in GCP Secret Manager
+ version: 1 # optional: default to `latest`.
+ token: $GCP_ID_TOKEN
+```
+
+You must also [add these CI/CD variables](../variables/index.md#for-a-project) to provide details about your GCP Secret Manager:
+
+- `GCP_PROJECT_NUMBER`: The GCP [Project Number](https://cloud.google.com/resource-manager/docs/creating-managing-projects)
+- `GCP_WORKLOAD_IDENTITY_FEDERATION_POOL_ID`: The WIF Pool ID (e.g `gitlab-pool`)
+- `GCP_WORKLOAD_IDENTITY_FEDERATION_PROVIDER_ID`: The WIF Provider ID (e.g `gitlab-provider`)
diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md
index e452b26d8a9..96b9709bdef 100644
--- a/doc/ci/secrets/index.md
+++ b/doc/ci/secrets/index.md
@@ -18,6 +18,12 @@ Unlike CI/CD variables, which are always presented to a job, secrets must be exp
required by a job. Read [GitLab CI/CD pipeline configuration reference](../yaml/index.md#secrets)
for more information about the syntax.
+GitLab provides support for the following secret management providers:
+
+1. [Vault by HashiCorp](#use-vault-secrets-in-a-ci-job)
+1. [Google Cloud Secret Manager](gcp_secret_manager.md)
+1. [Azure Key Vault](azure_key_vault.md)
+
GitLab has selected [Vault by HashiCorp](https://www.vaultproject.io) as the
first supported provider, and [KV-V2](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2)
as the first supported secrets engine.
diff --git a/doc/ci/testing/code_quality.md b/doc/ci/testing/code_quality.md
index 23ae615eeb2..9e6c409a0d3 100644
--- a/doc/ci/testing/code_quality.md
+++ b/doc/ci/testing/code_quality.md
@@ -27,14 +27,15 @@ You can extend the code coverage either by using Code Climate
Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/),
as shown in the following table:
-| Capability | In Free | In Premium | In Ultimate |
-|:-----------------------------------------------------------------------|:--------------------|:--------------------|:-------------------|
-| [Configure scanners](#customizing-scan-settings) | **{check-circle}** | **{check-circle}** | **{check-circle}** |
-| [Integrate custom scanners](#implement-a-custom-tool) | **{check-circle}** | **{check-circle}** | **{check-circle}** |
-| [See findings in merge request widget](#merge-request-widget) | **{check-circle}** | **{check-circle}** | **{check-circle}** |
-| [Generate JSON or HTML report artifacts](#output) | **{check-circle}** | **{check-circle}** | **{check-circle}** |
-| [See reports in CI pipelines](#pipeline-details-view) | **{dotted-circle}** | **{check-circle}** | **{check-circle}** |
-| [See findings in merge request diff view](#merge-request-changes-view) | **{dotted-circle}** | **{dotted-circle}** | **{check-circle}** |
+| Feature | In Free | In Premium | In Ultimate |
+|:----------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|
+| [Configure scanners](#customizing-scan-settings) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes |
+| [Integrate custom scanners](#implement-a-custom-tool) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes |
+| [Generate JSON or HTML report artifacts](#output) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes |
+| [Findings in merge request widget](#merge-request-widget) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes |
+| [Findings in pipelines](#pipeline-details-view) | **{dotted-circle}** No | **{check-circle}** Yes | **{check-circle}** Yes |
+| [Findings in merge request changes view](#merge-request-changes-view) | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes |
+| [Summary in project quality view](#project-quality-view) | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes |
## View Code Quality results
@@ -219,63 +220,42 @@ To configure the Code Quality job:
1. Declare a job with the same name as the Code Quality job, after the template's inclusion.
1. Specify additional keys in the job's stanza.
-For an example, see [Download output in JSON format](#download-output-in-json-format).
+For an example, see [Download output in HTML format](#output-in-only-html-format).
-### Available CI/CD variables
-
-> In [GitLab 13.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/11100), the option to override the Code Quality environment variables was added.
+## Available CI/CD variables
Code Quality can be customized by defining available CI/CD variables:
-| CI/CD variable | Description |
-| --------------------------- | ----------- |
-| `SOURCE_CODE` | Path to the source code to scan. |
-| `TIMEOUT_SECONDS` | Custom timeout per engine container for the `codeclimate analyze` command, default is 900 seconds (15 minutes). |
-| `CODECLIMATE_DEBUG` | Set to enable [Code Climate debug mode](https://github.com/codeclimate/codeclimate#environment-variables) |
-| `CODECLIMATE_DEV` | Set to enable `--dev` mode which lets you run engines not known to the CLI. |
-| `REPORT_STDOUT` | Set to print the report to `STDOUT` instead of generating the usual report file. |
-| `REPORT_FORMAT` | Set to control the format of the generated report file. One of: `json\|html`. |
-| `ENGINE_MEMORY_LIMIT_BYTES` | Set the memory limit for engines, default is 1,024,000,000 bytes. |
-| `CODE_QUALITY_DISABLED` | Prevents the Code Quality job from running. |
-| `CODECLIMATE_PREFIX` | Set a prefix to use with all `docker pull` commands in CodeClimate engines. Useful for [offline scanning](https://github.com/codeclimate/codeclimate/pull/948). |
+| CI/CD variable | Description |
+|---------------------------------|-------------|
+| `CODECLIMATE_DEBUG` | Set to enable [Code Climate debug mode](https://github.com/codeclimate/codeclimate#environment-variables). |
+| `CODECLIMATE_DEV` | Set to enable `--dev` mode which lets you run engines not known to the CLI. |
+| `CODECLIMATE_PREFIX` | Set a prefix to use with all `docker pull` commands in CodeClimate engines. Useful for [offline scanning](https://github.com/codeclimate/codeclimate/pull/948). For more information, see [Use a private container registry](#use-a-private-container-image-registry). |
+| `CODECLIMATE_REGISTRY_USERNAME` | Set to specify the username for the registry domain parsed from `CODECLIMATE_PREFIX`. |
+| `CODECLIMATE_REGISTRY_PASSWORD` | Set to specify the password for the registry domain parsed from `CODECLIMATE_PREFIX`. |
+| `CODE_QUALITY_DISABLED` | Prevents the Code Quality job from running. |
+| `CODE_QUALITY_IMAGE` | Set to a fully prefixed image name. Image must be accessible from your job environment. |
+| `ENGINE_MEMORY_LIMIT_BYTES` | Set the memory limit for engines. Default: 1,024,000,000 bytes. |
+| `REPORT_STDOUT` | Set to print the report to `STDOUT` instead of generating the usual report file. |
+| `REPORT_FORMAT` | Set to control the format of the generated report file. Either `json` or `html`. |
+| `SOURCE_CODE` | Path to the source code to scan. |
+| `TIMEOUT_SECONDS` | Custom timeout per engine container for the `codeclimate analyze` command. Default: 900 seconds (15 minutes) |
## Output
-Code Quality creates a file named `gl-code-quality-report.json`. The content of this file is
-processed internally and the results shown in the UI. To see the raw results, you can
-configure the Code Quality job to allow download of this file. Format options are JSON format, HTML
-format, or both. Use the HTML format to view the report in a more human-readable
-format. For example, you could publish the HTML format file on GitLab Pages for even easier
+Code Quality outputs a report containing details of issues found. The content of this report is
+processed internally and the results shown in the UI. The report is also output as a job artifact of
+the `code_quality` job, named `gl-code-quality-report.json`. You can optionally output the report in
+HTML format. For example, you could publish the HTML format file on GitLab Pages for even easier
reviewing.
-### Download output in JSON format
-
-To be able to download the Code Quality report in JSON format, declare the
-`gl-code-quality-report.json` file as an artifact of the `code_quality` job:
-
-```yaml
-include:
- - template: Code-Quality.gitlab-ci.yml
-
-code_quality:
- artifacts:
- paths: [gl-code-quality-report.json]
-```
-
-The full JSON file is available as a
-[downloadable artifact](../jobs/job_artifacts.md#download-job-artifacts) of the `code_quality`
-job.
+### Output in JSON and HTML format
-### Download output in JSON and HTML format
-
-> HTML report format [introduced](https://gitlab.com/gitlab-org/ci-cd/codequality/-/issues/10) in GitLab 13.6.
-
-NOTE:
-To create the HTML format file, the Code Quality job must be run twice, once for each format.
-In this configuration, the JSON format file is created but it is only processed internally.
+To output the Code Quality report in JSON and HTML format, you create an additional job. This requires
+Code Quality to be run twice, once each for file format.
-To be able to download the Code Quality report in both JSON and HTML format, add another job to your
-template by using `extends: code_quality`:
+To output the Code Quality report in HTML format, add another job to your template by using
+`extends: code_quality`:
```yaml
include:
@@ -289,18 +269,17 @@ code_quality_html:
paths: [gl-code-quality-report.html]
```
-Both the JSON and HTML files are available as
-[downloadable artifacts](../jobs/job_artifacts.md#download-job-artifacts) of the `code_quality`
-job.
+Both the JSON and HTML files are output as job artifacts. The HTML file is contained in the
+`artifacts.zip` job artifact.
-### Download output in only HTML format
+### Output in only HTML format
-To download the Code Quality report in _only_ an HTML format file, set `REPORT_FORMAT` to `html` in
-the existing job.
+To download the Code Quality report in _only_ HTML format, set `REPORT_FORMAT` to `html`, overriding
+the default definition of the `code_quality` job.
NOTE:
-This does not create a JSON format file, so Code Quality results are not shown in the
-merge request widget, pipeline report, or changes view.
+This does not create a JSON format file, so Code Quality results are not shown in the merge request
+widget, pipeline report, or changes view.
```yaml
include:
@@ -313,9 +292,7 @@ code_quality:
paths: [gl-code-quality-report.html]
```
-The HTML file is available as a
-[downloadable artifact](../jobs/job_artifacts.md#download-job-artifacts) of the `code_quality`
-job.
+The HTML file is output as a job artifact.
## Use Code Quality with merge request pipelines
diff --git a/doc/ci/testing/img/code_quality_inline_indicator_v16_7.png b/doc/ci/testing/img/code_quality_inline_indicator_v16_7.png
index 0d7d5bb3062..91285493562 100644
--- a/doc/ci/testing/img/code_quality_inline_indicator_v16_7.png
+++ b/doc/ci/testing/img/code_quality_inline_indicator_v16_7.png
Binary files differ
diff --git a/doc/ci/testing/test_coverage_visualization.md b/doc/ci/testing/test_coverage_visualization.md
index ff55f37e1ff..ecd5c794344 100644
--- a/doc/ci/testing/test_coverage_visualization.md
+++ b/doc/ci/testing/test_coverage_visualization.md
@@ -178,7 +178,7 @@ the [`coverage-report`](https://gitlab.com/gitlab-org/ci-sample-projects/coverag
### JavaScript example
-The following [`.gitlab-ci.yml`](../yaml/index.md) example uses [Mocha](https://mochajs.org/)
+The following `.gitlab-ci.yml` example uses [Mocha](https://mochajs.org/)
JavaScript testing and [nyc](https://github.com/istanbuljs/nyc) coverage-tooling to
generate the coverage artifact:
@@ -198,7 +198,7 @@ test:
#### Maven example
-The following [`.gitlab-ci.yml`](../yaml/index.md) example for Java or Kotlin uses [Maven](https://maven.apache.org/)
+The following `.gitlab-ci.yml` example for Java or Kotlin uses [Maven](https://maven.apache.org/)
to build the project and [JaCoCo](https://www.eclemma.org/jacoco/) coverage-tooling to
generate the coverage artifact.
You can check the [Docker image configuration and scripts](https://gitlab.com/haynes/jacoco2cobertura) if you want to build your own image.
@@ -236,7 +236,7 @@ coverage-jdk11:
#### Gradle example
-The following [`.gitlab-ci.yml`](../yaml/index.md) example for Java or Kotlin uses [Gradle](https://gradle.org/)
+The following `.gitlab-ci.yml` example for Java or Kotlin uses [Gradle](https://gradle.org/)
to build the project and [JaCoCo](https://www.eclemma.org/jacoco/) coverage-tooling to
generate the coverage artifact.
You can check the [Docker image configuration and scripts](https://gitlab.com/haynes/jacoco2cobertura) if you want to build your own image.
@@ -274,7 +274,7 @@ coverage-jdk11:
### Python example
-The following [`.gitlab-ci.yml`](../yaml/index.md) example uses [pytest-cov](https://pytest-cov.readthedocs.io/) to collect test coverage data:
+The following `.gitlab-ci.yml` example uses [pytest-cov](https://pytest-cov.readthedocs.io/) to collect test coverage data:
```yaml
run tests:
@@ -293,7 +293,7 @@ run tests:
### PHP example
-The following [`.gitlab-ci.yml`](../yaml/index.md) example for PHP uses [PHPUnit](https://phpunit.readthedocs.io/)
+The following `.gitlab-ci.yml` example for PHP uses [PHPUnit](https://phpunit.readthedocs.io/)
to collect test coverage data and generate the report.
With a minimal [`phpunit.xml`](https://docs.phpunit.de/en/10.2/configuration.html) file (you may reference
@@ -331,7 +331,7 @@ to find Cobertura in the appropriate path.
### C/C++ example
-The following [`.gitlab-ci.yml`](../yaml/index.md) example for C/C++ with
+The following `.gitlab-ci.yml` example for C/C++ with
`gcc` or `g++` as the compiler uses [`gcovr`](https://gcovr.com/en/stable/) to generate the coverage
output file in Cobertura XML format.
@@ -362,7 +362,7 @@ run tests:
### Go example
-The following [`.gitlab-ci.yml`](../yaml/index.md) example for Go uses:
+The following `.gitlab-ci.yml` example for Go uses:
- [`go test`](https://go.dev/doc/tutorial/add-a-test) to run tests.
- [`gocover-cobertura`](https://github.com/boumenot/gocover-cobertura) to convert Go's coverage profile into the Cobertura XML format.
@@ -391,7 +391,7 @@ run tests:
### Ruby example
-The following [`.gitlab-ci.yml`](../yaml/index.md) example for Ruby uses
+The following `.gitlab-ci.yml` example for Ruby uses
- [`rspec`](https://rspec.info/) to run tests.
- [`simplecov`](https://github.com/simplecov-ruby/simplecov) and [`simplecov-cobertura`](https://github.com/dashingrocket/simplecov-cobertura)
diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md
index f247d9609fe..49f5f1edf41 100644
--- a/doc/ci/variables/index.md
+++ b/doc/ci/variables/index.md
@@ -147,7 +147,7 @@ To add or update variables in the project settings:
in job logs. The variable fails to save if the value does not meet the
[masking requirements](#mask-a-cicd-variable).
-After you create a variable, you can use it in the [`.gitlab-ci.yml` configuration](../yaml/index.md)
+After you create a variable, you can use it in the pipeline configuration
or in [job scripts](#use-cicd-variables-in-job-scripts).
### For a group
@@ -244,7 +244,7 @@ malicious code can compromise both masked and protected variables.
Variable values are encrypted using [`aes-256-cbc`](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard)
and stored in the database. This data can only be read and decrypted with a
-valid [secrets file](../../administration/backup_restore/backup_gitlab.md#when-the-secrets-file-is-lost).
+valid [secrets file](../../administration/backup_restore/troubleshooting_backup_gitlab.md#when-the-secrets-file-is-lost).
### Mask a CI/CD variable
@@ -638,7 +638,7 @@ To disable variable expansion for the variable:
## CI/CD variable precedence
-> Scan Execution Policies variable precedence was [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/424028) in GitLab 16.7 [with a flag](../../administration/feature_flags.md) named `security_policies_variables_precedence`. Enabled by default.
+> Scan Execution Policies variable precedence was [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/424028) in GitLab 16.7 [with a flag](../../administration/feature_flags.md) named `security_policies_variables_precedence`. Enabled by default. [Feature flag removed in GitLab 16.8](https://gitlab.com/gitlab-org/gitlab/-/issues/435727).
You can use CI/CD variables with the same name in different places, but the values
can overwrite each other. The type of variable and where they are defined determines
@@ -967,4 +967,4 @@ As a workaround you can either:
- Use [File-type](#use-file-type-cicd-variables) CI/CD variables for large environment variables where possible.
- If a single large variable is larger than `ARG_MAX`, try using [Secure Files](../secure_files/index.md), or
-bring the file to the job through some other mechanism.
+ bring the file to the job through some other mechanism.
diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md
index e8ed47cedd0..470982c7d26 100644
--- a/doc/ci/variables/predefined_variables.md
+++ b/doc/ci/variables/predefined_variables.md
@@ -164,8 +164,9 @@ These variables are available when:
| `CI_MERGE_REQUEST_DIFF_BASE_SHA` | 13.7 | all | The base SHA of the merge request diff. |
| `CI_MERGE_REQUEST_DIFF_ID` | 13.7 | all | The version of the merge request diff. |
| `CI_MERGE_REQUEST_EVENT_TYPE` | 12.3 | all | The event type of the merge request. Can be `detached`, `merged_result` or `merge_train`. |
+| `CI_MERGE_REQUEST_DESCRIPTION` | 16.7 | all | The description of the merge request. If the description is more than 2700 characters long, only the first 2700 characters are stored in the variable. |
+| `CI_MERGE_REQUEST_DESCRIPTION_IS_TRUNCATED` | 16.8 | all | `true` if `CI_MERGE_REQUEST_DESCRIPTION` is truncated down to 2700 characters because the description of the merge request is too long. |
| `CI_MERGE_REQUEST_ID` | 11.6 | all | The instance-level ID of the merge request. This is a unique ID across all projects on the GitLab instance. |
-| `CI_MERGE_REQUEST_DESCRIPTION` | 16.7 | all | The description of the merge request. |
| `CI_MERGE_REQUEST_IID` | 11.6 | all | The project-level IID (internal ID) of the merge request. This ID is unique for the current project, and is the number used in the merge request URL, page title, and other visible locations. |
| `CI_MERGE_REQUEST_LABELS` | 11.9 | all | Comma-separated label names of the merge request. |
| `CI_MERGE_REQUEST_MILESTONE` | 11.9 | all | The milestone title of the merge request. |
diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md
index bddb0947fac..4ea45c9bae4 100644
--- a/doc/ci/variables/where_variables_can_be_used.md
+++ b/doc/ci/variables/where_variables_can_be_used.md
@@ -16,7 +16,7 @@ This document describes where and how the different types of variables can be us
There are two places defined variables can be used. On the:
-1. GitLab side, in the [`.gitlab-ci.yml` file](../yaml/index.md).
+1. GitLab side, in the [`.gitlab-ci.yml` file](../index.md#the-gitlab-ciyml-file).
1. The GitLab Runner side, in `config.toml`.
### `.gitlab-ci.yml` file
@@ -27,6 +27,7 @@ There are two places defined variables can be used. On the:
|:----------------------------------------------------------------------|:-----------------|:-----------------------|:------------|
| [`after_script`](../yaml/index.md#after_script) | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment). |
| [`artifacts:name`](../yaml/index.md#artifactsname) | yes | Runner | The variable expansion is made by GitLab Runner's shell environment. |
+| [`artifacts:paths`](../yaml/index.md#artifactspaths) | yes | Runner | The variable expansion is made by GitLab Runner's shell environment. |
| [`before_script`](../yaml/index.md#before_script) | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment) |
| [`cache:key`](../yaml/index.md#cachekey) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). |
| [`cache:policy`](../yaml/index.md#cachepolicy) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). |
diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md
index 5867a5b3506..131f9e502fe 100644
--- a/doc/ci/yaml/artifacts_reports.md
+++ b/doc/ci/yaml/artifacts_reports.md
@@ -322,13 +322,13 @@ The `repository_xray` report collects information about your repository for use
> [Moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) from GitLab Ultimate to GitLab Free in 13.3.
-The `sast` report collects [SAST vulnerabilities](../../user/application_security/sast/index.md). The collected SAST
-report uploads to GitLab as an artifact.
+The `sast` report collects [SAST vulnerabilities](../../user/application_security/sast/index.md).
+The collected SAST report uploads to GitLab as an artifact.
-GitLab can display the results of one or more reports in:
+For more information, see:
-- The merge request [SAST widget](../../user/application_security/sast/index.md).
-- The [security dashboard](../../user/application_security/security_dashboard/index.md).
+- [View SAST results](../../user/application_security/sast/index.md#view-sast-results)
+- [SAST output](../../user/application_security/sast/index.md#output)
## `artifacts:reports:secret_detection`
diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md
index df2330e04f6..208da96c5ad 100644
--- a/doc/ci/yaml/index.md
+++ b/doc/ci/yaml/index.md
@@ -303,7 +303,9 @@ include:
A `not found or access denied` error may be displayed if the user does not have access to any of the included files.
- Be careful when including another project's CI/CD configuration file. No pipelines or notifications trigger when CI/CD configuration files change.
From a security perspective, this is similar to pulling a third-party dependency. For the `ref`, consider:
- - Using a specific SHA hash, which should be the most stable option.
+ - Using a specific SHA hash, which should be the most stable option. Use the
+ full 40-character SHA hash to ensure the desired commit is referenced, because
+ using a short SHA hash for the `ref` might be ambiguous.
- Applying both [protected branch](../../user/project/protected_branches.md) and [protected tag](../../user/project/protected_tags.md#prevent-tag-creation-with-the-same-name-as-branches) rules to
the `ref` in the other project. Protected tags and branches are more likely to pass through change management before changing.
@@ -480,6 +482,46 @@ You can use some [predefined CI/CD variables](../variables/predefined_variables.
- [`workflow: rules` examples](workflow.md#workflow-rules-examples)
- [Switch between branch pipelines and merge request pipelines](workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines)
+#### `workflow:auto_cancel:on_new_commit`
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/412473) in GitLab 16.8 [with a flag](../../administration/feature_flags.md) named `ci_workflow_auto_cancel_on_new_commit`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available per project or
+for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `ci_workflow_auto_cancel_on_new_commit`.
+On GitLab.com, this feature is not available.
+The feature is not ready for production use.
+
+Use `workflow:auto_cancel:on_new_commit` to configure the behavior of
+the [auto-cancel redundant pipelines](../pipelines/settings.md#auto-cancel-redundant-pipelines) feature.
+
+**Possible inputs**:
+
+- `conservative`: Cancel the pipeline, but only if no jobs with `interruptible: false` have started yet. Default when not defined.
+- `interruptible`: Cancel only jobs with `interruptible: true`.
+- `none`: Do not auto-cancel any jobs.
+
+**Example of `workflow:auto_cancel:on_new_commit`**:
+
+```yaml
+workflow:
+ auto_cancel:
+ on_new_commit: interruptible
+
+job1:
+ interruptible: true
+ script: sleep 60
+
+job2:
+ interruptible: false # Default when not defined.
+ script: sleep 60
+```
+
+In this example:
+
+- When a new commit is pushed to a branch, GitLab creates a new pipeline and `job1` and `job2` start.
+- If a new commit is pushed to the branch before the jobs complete, only `job1` is canceled.
+
#### `workflow:name`
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/372538) in GitLab 15.5 [with a flag](../../administration/feature_flags.md) named `pipeline_name`. Disabled by default.
@@ -657,6 +699,52 @@ When the branch is something else:
- Use [`inherit:variables`](#inheritvariables) in the trigger job and list the
exact variables you want to forward to the downstream pipeline.
+#### `workflow:rules:auto_cancel`
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/436467) in GitLab 16.8 [with a flag](../../administration/feature_flags.md) named `ci_workflow_auto_cancel_on_new_commit`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available per project or
+for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `ci_workflow_auto_cancel_on_new_commit`.
+On GitLab.com, this feature is not available.
+The feature is not ready for production use.
+
+Use `workflow:rules:auto_cancel` to configure the behavior of
+the [`workflow:auto_cancel:on_new_commit`](#workflowauto_cancelon_new_commit) feature.
+
+**Possible inputs**:
+
+- `on_new_commit`: [`workflow:auto_cancel:on_new_commit`](#workflowauto_cancelon_new_commit)
+
+**Example of `workflow:rules:auto_cancel`**:
+
+```yaml
+workflow:
+ auto_cancel:
+ on_new_commit: interruptible
+ rules:
+ - if: $CI_COMMIT_REF_PROTECTED == 'true'
+ auto_cancel:
+ on_new_commit: none
+ - when: always # Run the pipeline in other cases
+
+test-job1:
+ script: sleep 10
+ interruptible: false
+
+test-job2:
+ script: sleep 10
+ interruptible: true
+```
+
+In this example, [`workflow:auto_cancel:on_new_commit`](#workflowauto_cancelon_new_commit)
+is set to `interruptible` for all jobs by default. But if a pipeline runs for a protected branch,
+the rule overrides the default with `on_new_commit: none`. For example, if a pipeline
+is running for:
+
+- A non-protected branch and a new commit is pushed, `test-job1` continues to run and `test-job2` is canceled.
+- A protected branch and a new commit is pushed, both `test-job1` and `test-job2` continue to run.
+
## Header keywords
Some keywords must be defined in a header section of a YAML configuration file.
@@ -719,12 +807,12 @@ scan-website:
Inputs are mandatory when included, unless you set a default value with `spec:inputs:default`.
-Use `default: null` to have no default value.
+Use `default: ''` to have no default value.
**Keyword type**: Header keyword. `specs` must be declared at the top of the configuration file,
in a header section.
-**Possible inputs**: A string representing the default value, or `null`.
+**Possible inputs**: A string representing the default value, or `''`.
**Example of `spec:inputs:default`**:
@@ -735,7 +823,7 @@ spec:
user:
default: 'test-user'
flags:
- default: null
+ default: ''
---
# The pipeline configuration would follow...
@@ -887,7 +975,8 @@ The following topics explain how to use keywords to configure CI/CD pipelines.
### `after_script`
-Use `after_script` to define an array of commands that run after each job, including failed jobs.
+Use `after_script` to define an array of commands that run after a job's `script` section, including failed jobs with failure type of `script_failure`.
+`after_script` commands do not run after [other failure types](#retrywhen).
**Keyword type**: Job keyword. You can use it only as part of a job or in the
[`default` section](#default).
@@ -1077,6 +1166,8 @@ link outside it.
[`doublestar.Glob`](https://pkg.go.dev/github.com/bmatcuk/doublestar@v1.2.2?tab=doc#Match).
- In GitLab Runner 12.10 and earlier, [`filepath.Match`](https://pkg.go.dev/path/filepath#Match).
+CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file).
+
**Example of `artifacts:paths`**:
```yaml
@@ -1272,15 +1363,7 @@ job:
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223273) in GitLab 13.8 [with a flag](../../user/feature_flags.md) named `non_public_artifacts`, disabled by default.
> - [Updated](https://gitlab.com/gitlab-org/gitlab/-/issues/322454) in GitLab 15.10. Artifacts created with `artifacts:public` before 15.10 are not guaranteed to remain private after this update.
-> - [Updated](https://gitlab.com/gitlab-org/gitlab/-/issues/294503) in GitLab 16.7. Rolled out and removed a feature flag named `non_public_artifacts`
-
-WARNING:
-On self-managed GitLab, by default this feature is not available. To make it available,
-an administrator can [enable the feature flag](../../administration/feature_flags.md) named `non_public_artifacts`. On
-GitLab.com, this feature is not available. Due to [issue 413822](https://gitlab.com/gitlab-org/gitlab/-/issues/413822),
-the keyword can be used when the feature flag is disabled, but the feature does not work.
-Do not attempt to use this feature when the feature flag is disabled, and always test
-with non-production data first.
+> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/294503) in GitLab 16.7. Feature flag `non_public_artifacts` removed.
Use `artifacts:public` to determine whether the job artifacts should be
publicly available.
@@ -1593,7 +1676,7 @@ use the new cache, instead of rebuilding the dependencies.
**Additional details**:
- The cache `key` is a SHA computed from the most recent commits
-that changed each listed file.
+ that changed each listed file.
If neither file is changed in any commits, the fallback key is `default`.
##### `cache:key:prefix`
@@ -1909,8 +1992,8 @@ to select a specific site profile and scanner profile.
**Related topics**:
-- [Site profile](../../user/application_security/dast/proxy-based.md#site-profile).
-- [Scanner profile](../../user/application_security/dast/proxy-based.md#scanner-profile).
+- [Site profile](../../user/application_security/dast/on-demand_scan.md#site-profile).
+- [Scanner profile](../../user/application_security/dast/on-demand_scan.md#scanner-profile).
### `dependencies`
@@ -2468,7 +2551,8 @@ image:
#### `image:docker`
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27919) in GitLab 16.7. Requires GitLab Runner 16.7 or later.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27919) in GitLab 16.7. Requires GitLab Runner 16.7 or later.
+> - `user` input option [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/137907) in GitLab 16.8.
Use `image:docker` to pass options to the Docker executor of a GitLab Runner.
@@ -2481,6 +2565,7 @@ A hash of options for the Docker executor, which can include:
- `platform`: Selects the architecture of the image to pull. When not specified,
the default is the same platform as the host runner.
+- `user`: Specify the username or UID to use when running the container.
**Example of `image:docker`**:
@@ -2491,11 +2576,13 @@ arm-sql-job:
name: super/sql:experimental
docker:
platform: arm64/v8
+ user: dave
```
**Additional details**:
- `image:docker:platform` maps to the [`docker pull --platform` option](https://docs.docker.com/engine/reference/commandline/pull/#options).
+- `image:docker:user` maps to the [`docker run --user` option](https://docs.docker.com/engine/reference/commandline/run/#options).
#### `image:pull_policy`
@@ -2620,7 +2707,8 @@ job2:
### `interruptible`
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32022) in GitLab 12.3.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32022) in GitLab 12.3.
+> - Support for `trigger` jobs [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/412473) in GitLab 16.8 [with a flag](../../administration/feature_flags.md) named `ci_workflow_auto_cancel_on_new_commit`. Disabled by default.
Use `interruptible` to configure the [auto-cancel redundant pipelines](../pipelines/settings.md#auto-cancel-redundant-pipelines)
feature to cancel a job before it completes if a new pipeline on the same ref starts for a newer commit. If the feature
@@ -2685,6 +2773,12 @@ In this example, a new pipeline causes a running pipeline to be:
a pipeline to allow users to manually prevent a pipeline from being automatically
cancelled. After a user starts the job, the pipeline cannot be canceled by the
**Auto-cancel redundant pipelines** feature.
+- When using `interruptible` with a [trigger job](#trigger):
+ - The triggered downstream pipeline is never affected by the trigger job's `interruptible` configuration.
+ - If [`workflow:auto_cancel`](#workflowauto_cancelon_new_commit) is set to `conservative`,
+ the trigger job's `interruptible` configuration has no effect.
+ - If [`workflow:auto_cancel`](#workflowauto_cancelon_new_commit) is set to `interruptible`,
+ a trigger job with `interruptible: true` can be automatically cancelled.
### `needs`
@@ -4203,6 +4297,34 @@ job:
vault: production/db/password@ops # Translates to secret: `ops/data/production/db`, field: `password`
```
+#### `secrets:gcp_secret_manager`
+
+> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/11739) in GitLab 16.8 and GitLab Runner 16.8.
+
+Use `secrets:gcp_secret_manager` to specify secrets provided by [GCP Secret Manager](https://cloud.google.com/security/products/secret-manager).
+
+**Keyword type**: Job keyword. You can use it only as part of a job.
+
+**Possible inputs**:
+
+- `name`: Name of the secret.
+- `version`: Version of the secret.
+
+**Example of `secrets:gcp_secret_manager`**:
+
+```yaml
+job:
+ secrets:
+ DATABASE_PASSWORD:
+ gcp_secret_manager:
+ name: 'test'
+ version: 2
+```
+
+**Related topics**:
+
+- [Use GCP Secret Manager secrets in GitLab CI/CD](../secrets/gcp_secret_manager.md).
+
#### `secrets:azure_key_vault`
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271271) in GitLab 16.3 and GitLab Runner 16.3.
@@ -4350,7 +4472,8 @@ In this example, GitLab launches two containers for the job:
#### `services:docker`
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27919) in GitLab 16.7. Requires GitLab Runner 16.7 or later.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27919) in GitLab 16.7. Requires GitLab Runner 16.7 or later.
+> - `user` input option [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/137907) in GitLab 16.8.
Use `services:docker` to pass options to the Docker executor of a GitLab Runner.
@@ -4363,6 +4486,7 @@ A hash of options for the Docker executor, which can include:
- `platform`: Selects the architecture of the image to pull. When not specified,
the default is the same platform as the host runner.
+- `user`: Specify the username or UID to use when running the container.
**Example of `services:docker`**:
@@ -4374,11 +4498,13 @@ arm-sql-job:
- name: super/sql:experimental
docker:
platform: arm64/v8
+ user: dave
```
**Additional details**:
- `services:docker:platform` maps to the [`docker pull --platform` option](https://docs.docker.com/engine/reference/commandline/pull/#options).
+- `services:docker:user` maps to the [`docker run --user` option](https://docs.docker.com/engine/reference/commandline/run/#options).
#### `services:pull_policy`
diff --git a/doc/ci/yaml/inputs.md b/doc/ci/yaml/inputs.md
index 0869be6da9f..18dcb865c06 100644
--- a/doc/ci/yaml/inputs.md
+++ b/doc/ci/yaml/inputs.md
@@ -89,7 +89,7 @@ spec:
default: true
---
-"$[[ job-prefix ]]-scan-website":
+"$[[ inputs.job-prefix ]]-scan-website":
stage: $[[ inputs.job-stage ]]
script:
- echo "scanning website -e $[[ inputs.environment ]] -c $[[ inputs.concurrency ]] -v $[[ inputs.version ]]"
@@ -233,11 +233,18 @@ Only variables you can [use with the `include` keyword](includes.md#use-variable
Example:
```yaml
-$[[ inputs.test | expand_vars ]]
+spec:
+ inputs:
+ test:
+ default: 'test $MY_VAR'
+---
+
+test-job:
+ script: echo $[[ inputs.test | expand_vars ]]
```
-Assuming the value of `inputs.test` is `test $MY_VAR`, and the variable `$MY_VAR` is unmasked
-with a value of `my value`, then the output would be `test my value`.
+In this example, if `$MY_VAR` is unmasked (exposed in job logs) with a value of `my value`, then the input
+would expand to `test my value`.
#### `truncate`
@@ -259,3 +266,56 @@ $[[ inputs.test | truncate(3,5) ]]
```
Assuming the value of `inputs.test` is `0123456789`, then the output would be `34567`.
+
+## Troubleshooting
+
+### YAML syntax errors when using `inputs`
+
+[CI/CD variable expressions](../jobs/job_control.md#cicd-variable-expressions)
+in `rules:if` expect a comparison of a CI/CD variable with a string, otherwise
+[a variety of syntax errors could be returned](../jobs/job_control.md#this-gitlab-ci-configuration-is-invalid-for-variable-expressions).
+
+You must ensure that expressions remain properly formatted after input values are
+inserted into the configuration, which might require the use of additional quote characters.
+
+For example:
+
+```yaml
+spec:
+ inputs:
+ branch:
+ default: $CI_DEFAULT_BRANCH
+---
+
+job-name:
+ rules:
+ - if: $CI_COMMIT_REF_NAME == $[[ inputs.branch ]]
+```
+
+In this example:
+
+- Using `include: inputs: branch: $CI_DEFAULT_BRANCH` is valid. The `if:` clause evaluates to
+ `if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH`, which is a valid variable expression.
+- Using `include: inputs: branch: main` is **invalid**. The `if:` clause evaluates to
+ `if: $CI_COMMIT_REF_NAME == main`, which is invalid because `main` is a string but is not quoted.
+
+Alternatively, add quotes to resolve some variable expression issues. For example:
+
+```yaml
+spec:
+ inputs:
+ environment:
+ default: "$ENVIRONMENT"
+---
+
+$[[ inputs.environment | expand_vars ]] job:
+ script: echo
+ rules:
+ - if: '"$[[ inputs.environment1 | expand_vars ]]" == "production"'
+```
+
+In this example, quoting the input block and also the entire variable expression
+ensures valid `if:` syntax after the input is evaluated. The internal and external quotes
+in the expression must not be the same character. You can use `"` for the internal quotes
+and `'` for the external quotes, or the inverse. On the other hand, the job name does
+not require any quoting.