diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-06-20 13:43:29 +0300 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-06-20 13:43:29 +0300 |
| commit | 3b1af5cc7ed2666ff18b718ce5d30fa5a2756674 (patch) | |
| tree | 3bc4a40e0ee51ec27eabf917c537033c0c5b14d4 /doc/ci | |
| parent | 9bba14be3f2c211bf79e15769cd9b77bc73a13bc (diff) | |
Add latest changes from gitlab-org/gitlab@16-1-stable-eev16.1.0-rc42
Diffstat (limited to 'doc/ci')
83 files changed, 1445 insertions, 1053 deletions
diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index e9d3dae3837..1822e610dad 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -99,7 +99,10 @@ the global fallback cache is fetched every time a cache is not found. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110467) in GitLab 16.0 -Each cache entry supports up-to 5 fallback keys: +Each cache entry supports up to five fallback keys with the [`fallback_keys` keyword](../yaml/index.md#cachefallback_keys). +When a job does not find a cache key, the job attempts to retrieve a fallback cache instead. +Fallback keys are searched in order until a cache is found. If no cache is found, +the job runs without using a cache. For example: ```yaml test-job: @@ -114,11 +117,25 @@ test-job: script: - bundle config set --local path 'vendor/ruby' - bundle install - - yarn install --cache-folder .yarn-cache - echo Run tests... ``` -Fallback keys follows the same processing logic as `cache:key`, meaning that the fullname may include a `-$index` (based on cache clearance) and `-protected`/`-non_protected` (if cache separation enabled on protected branches) suffixes. +In this example: + +1. The job looks for the `cache-$CI_COMMIT_REF_SLUG` cache. +1. If `cache-$CI_COMMIT_REF_SLUG` is not found, the job looks for `cache-$CI_DEFAULT_BRANCH` + as a fallback option. +1. If `cache-$CI_DEFAULT_BRANCH` is also not found, the job looks for `cache-default` + as a second fallback option. +1. If none are found, the job downloads all the Ruby dependencies without using a cache, + but creates a new cache for `cache-$CI_COMMIT_REF_SLUG` when the job completes. + +Fallback keys follow the same processing logic as `cache:key`: + +- If you [clear caches manually](#clear-the-cache-manually), per-cache fallback keys are appended + with an index like other cache keys. +- If the [**Use separate caches for protected branches** setting](#cache-key-names) is enabled, + per-cache fallback keys are appended with `-protected` or `-non_protected`. ### Global fallback key @@ -244,6 +261,39 @@ cache: key: $CI_JOB_NAME ``` +### Use a variable to control a job's cache policy + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371480) in GitLab 16.1. + +To reduce duplication of jobs where the only difference is the pull policy, you can use a [CI/CD variable](../variables/index.md). + +For example: + +```yaml +conditional-policy: + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + variables: + POLICY: pull-push + - if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH + variables: + POLICY: pull + stage: build + cache: + key: gems + policy: $POLICY + paths: + - vendor/bundle + script: + - echo "This job pulls and pushes the cache depending on the branch" + - echo "Downloading dependencies..." +``` + +In this example, the job's cache policy is: + +- `pull-push` for changes to the default branch. +- `pull` for changes to other branches. + ### Cache Node.js dependencies If your project uses [npm](https://www.npmjs.com/) to install Node.js @@ -352,7 +402,7 @@ variables: PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip" # Pip's cache doesn't store the python packages -# https://pip.pypa.io/en/stable/reference/pip_install/#caching +# https://pip.pypa.io/en/stable/topics/caching/ cache: paths: - .cache/pip @@ -485,7 +535,7 @@ be overwritten because caches are restored before artifacts. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330047) in GitLab 15.0. -A suffix is added to the cache key, with the exception of the [fallback cache key](#use-a-fallback-cache-key). +A suffix is added to the cache key, with the exception of the [global fallback cache key](#global-fallback-key). As an example, assuming that `cache.key` is set to `$CI_COMMIT_REF_SLUG`, and that we have two branches `main` and `feature`, then the following table represents the resulting cache keys: @@ -507,8 +557,8 @@ and should only be disabled in an environment where all users with Developer rol To use the same cache for all branches: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Clear the **Use separate caches for protected branches** checkbox. 1. Select **Save changes**. @@ -604,8 +654,8 @@ The next time the pipeline runs, the cache is stored in a different location. You can clear the cache in the GitLab UI: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **CI/CD > Pipelines**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. On the left sidebar, select **Build > Pipelines**. 1. In the upper-right corner, select **Clear runner caches**. On the next commit, your CI/CD jobs use a new cache. diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md index 6dff87ed1c5..7be12d0c0fd 100644 --- a/doc/ci/chatops/index.md +++ b/doc/ci/chatops/index.md @@ -1,6 +1,6 @@ --- -stage: Manage -group: Import and Integrate +stage: Deploy +group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments type: index, concepts, howto --- diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index ceb56b01dcd..5494e5fad1c 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -15,7 +15,8 @@ GitLab CI/CD can be used with Bitbucket Cloud by: To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In GitLab, create a project: - 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **View all your projects**. 1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository**. 1. Select **Repository by URL**. diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index 9933fafcb69..feb01a0fc4a 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -34,7 +34,8 @@ repositories: `repo` and `admin:repo_hook` so that GitLab can access your project, update commit statuses, and create a web hook to notify GitLab of new commits. 1. In GitLab, create a project: - 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **View all your projects**. 1. On the right of the page, select **New project**. 1. Select **Run CI/CD for external repository**. 1. Select **GitHub**. @@ -62,8 +63,9 @@ To manually enable GitLab CI/CD for your repository: 1. Enter a **Token description** and update the scope to allow `repo` so that GitLab can access your project and update commit statuses. 1. In GitLab, create a project: - 1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. - 1. On the right of the page, select **New project**. + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **View all your projects**. + 1. Select **New project**. 1. Select **Run CI/CD for external repository** and **Repository by URL**. 1. In the **Git repository URL** field, enter the HTTPS URL for your GitHub repository. If your project is private, use the personal access token you just created for authentication. diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index 7b9145050bf..6d14928389d 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -24,12 +24,17 @@ snippets disabled. These features To connect to an external repository: -1. In GitLab, on the top bar, select **Main menu > Projects > View all projects**. -1. On the right of the page, select **New project**. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **View all your projects**. +1. Select **New project**. 1. Select **Run CI/CD for external repository**. 1. Select **GitHub** or **Repository by URL**. 1. Complete the fields. +If the **Run CI/CD for external repository** option is not available, the GitLab instance +might not have any import sources configured. Ask an administrator for your instance to check +the [import sources configuration](../../user/admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources). + ## Pipelines for external pull requests > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/65139) in GitLab 12.3. diff --git a/doc/ci/cloud_services/azure/index.md b/doc/ci/cloud_services/azure/index.md index 912e6597985..29d27e440ec 100644 --- a/doc/ci/cloud_services/azure/index.md +++ b/doc/ci/cloud_services/azure/index.md @@ -34,7 +34,7 @@ To complete this tutorial: 1. [Grant permissions for the service principal](#grant-permissions-for-the-service-principal). 1. [Retrieve a temporary credential](#retrieve-a-temporary-credential). -For more information, review Azure's documentation on [Workload identity federation](https://learn.microsoft.com/en-us/azure/active-directory/develop/workload-identity-federation). +For more information, review Azure's documentation on [Workload identity federation](https://learn.microsoft.com/en-us/azure/active-directory/workload-identities/workload-identity-federation). ## Create Azure AD application and service principal diff --git a/doc/ci/cloud_services/google_cloud/index.md b/doc/ci/cloud_services/google_cloud/index.md index 5ed22883518..d99b50b5013 100644 --- a/doc/ci/cloud_services/google_cloud/index.md +++ b/doc/ci/cloud_services/google_cloud/index.md @@ -114,6 +114,17 @@ the assertion in the previous section. After you configure the OIDC and role, the GitLab CI/CD job can retrieve a temporary credential from the [Google Cloud Security Token Service (STS)](https://cloud.google.com/iam/docs/reference/sts/rest). +Add `id_tokens` to your CI/CD job: + +```yaml +job: + id_tokens: + GITLAB_OIDC_TOKEN: + aud: https://gitlab.example.com +``` + +Get temporary credentials using the ID token: + ```shell PAYLOAD="$(cat <<EOF { @@ -122,7 +133,7 @@ PAYLOAD="$(cat <<EOF "requestedTokenType": "urn:ietf:params:oauth:token-type:access_token", "scope": "https://www.googleapis.com/auth/cloud-platform", "subjectTokenType": "urn:ietf:params:oauth:token-type:jwt", - "subjectToken": "${CI_JOB_JWT_V2}" + "subjectToken": "${GITLAB_OIDC_TOKEN}" } EOF )" @@ -142,8 +153,7 @@ Where: - `PROJECT_NUMBER` is your Google Cloud project number (not name). - `POOL_ID` is the ID of the Workload Identity Pool created in the first section. - `PROVIDER_ID` is the ID of the Workload Identity Provider created in the second section. -- `CI_JOB_JWT_V2` is injected into the CI/CD job by GitLab. For more information about - this variable, read [Connect to cloud services](../index.md). +- `GITLAB_OIDC_TOKEN` is an OIDC [ID token](../../yaml/index.md#id_tokens). You can then use the resulting federated token to impersonate the service account created in the previous section: diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md index 54cadc9e1b6..eadf0656d48 100644 --- a/doc/ci/cloud_services/index.md +++ b/doc/ci/cloud_services/index.md @@ -38,8 +38,7 @@ ID tokens support cloud providers with OIDC, including: NOTE: Configuring OIDC enables JWT token access to the target environments for all pipelines. When you configure OIDC for a pipeline, you should complete a software supply chain security -review for the pipeline, focusing on the additional access. You can use the [software supply chain security awareness assessment](https://about.gitlab.com/quiz/software-supply-chain-security/) -as a starting point, and for more information about supply chain attacks, see +review for the pipeline, focusing on the additional access. For more information about supply chain attacks, see [How a DevOps Platform helps protect against supply chain attacks](https://about.gitlab.com/blog/2021/04/28/devops-platform-supply-chain-attacks/). ## Use cases diff --git a/doc/ci/components/index.md b/doc/ci/components/index.md index 95a513220a2..999c9b0c0fb 100644 --- a/doc/ci/components/index.md +++ b/doc/ci/components/index.md @@ -7,7 +7,7 @@ type: reference # CI/CD Components (Experimental) -> Introduced in GitLab 16.0 as an [experimental feature](../../policy/alpha-beta-support.md). +> Introduced in GitLab 16.0 as an [experimental feature](../../policy/experiment-beta-support.md). FLAG: On self-managed GitLab, by default this feature is not available. @@ -173,6 +173,7 @@ For example, for a component repository located at `gitlab-org/dast` on `gitlab. **Additional notes:** +- You can only reference components in the same GitLab instance as your project. - If a tag and branch exist with the same name, the tag takes precedence over the branch. - If a tag is named the same as a commit SHA that exists, like `e3262fdd0914fa823210cdb79a8c421e2cef79d8`, the commit SHA takes precedence over the tag. @@ -189,13 +190,12 @@ After components are added to a components repository, they can immediately be [ However, this repository is not discoverable. You must mark this project as a catalog resource to allow it to be visible in the CI Catalog so other users can discover it. -To mark a project as a catalog resource, run the following [graphQL](../../api/graphql/index.md) -mutation: +To mark a project as a catalog resource: -```graphql -mutation { - catalogResourcesCreate(input: { projectPath: "path-to-project"}) { - errors - } -} -``` +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Visibility, project features, permissions**. +1. Scroll down to **CI/CD Catalog resource** and select the toggle to mark the project as a catalog resource. + +NOTE: +This action is not reversible. diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index f0384fb29c7..e3bbe5b66c7 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -81,7 +81,7 @@ are certain use cases that you may need to work around. For more information, ch ## Needs Visualization -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](../../policy/alpha-beta-support.md#beta). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](../../policy/experiment-beta-support.md#beta). > - It became a [standard feature](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38517) in 13.3. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52208) in GitLab 13.9. diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index fe57b451146..004da63476e 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -352,11 +352,9 @@ Docker-in-Docker is the recommended configuration, but you should be aware of th To use Docker commands in your CI/CD jobs, you can bind-mount `/var/run/docker.sock` into the container. Docker is then available in the context of the image. -NOTE: -If you bind the Docker socket and you are -[using GitLab Runner 11.11 or later](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), -you can no longer use `docker:20.10.16-dind` as a service. Volume bindings -also affect services, making them incompatible. +> If you bind the Docker socket and you are [using GitLab Runner 11.11 or later](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), +> you can no longer use `docker:20.10.16-dind` as a service. +> Volume bindings also affect services, making them incompatible. To make Docker available in the context of the image, you need to mount `/var/run/docker.sock` into the launched containers. To do this with the Docker @@ -392,6 +390,12 @@ sudo gitlab-runner register -n \ --docker-volumes /var/run/docker.sock:/var/run/docker.sock ``` +> If you want to use more complex Docker-in-Docker configurations, like it is necessary to run Code Quality checks with +> Code Climate, you need to ensure that the paths to the build directory are the same on the host as well as inside the +> Docker container. +> See section "[Improve Code Quality performance with private runners](../testing/code_quality.md#improve-code-quality-performance-with-private-runners)" +> in the Code Quality documentation. + #### Enable registry mirror for `docker:dind` service When the Docker daemon starts inside the service container, it uses diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 32f95052980..b7affe28984 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -134,7 +134,7 @@ before_script: - | echo "-----BEGIN CERTIFICATE----- ... - -----END CERTIFICATE-----" >> /kaniko/ssl/certs/additional-ca-cert-bundle.crt + -----END CERTIFICATE-----" >> /kaniko/ssl/certs/ca-certificates.crt ``` ## Video walkthrough of a working example diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index e75f902c153..e332b040fbc 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -30,8 +30,8 @@ When you disable GitLab CI/CD: To disable GitLab CI/CD in your project: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. In the **Repository** section, turn off **CI/CD**. 1. Select **Save changes**. @@ -40,8 +40,8 @@ To disable GitLab CI/CD in your project: To enable GitLab CI/CD in your project: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. In the **Repository** section, turn on **CI/CD**. 1. Select **Save changes**. diff --git a/doc/ci/environments/configure_kubernetes_deployments.md b/doc/ci/environments/configure_kubernetes_deployments.md new file mode 100644 index 00000000000..e618aae7cae --- /dev/null +++ b/doc/ci/environments/configure_kubernetes_deployments.md @@ -0,0 +1,58 @@ +--- +stage: Deploy +group: Environments +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Configure Kubernetes deployments (deprecated) + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. +> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. + +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. + +If you are deploying to a [Kubernetes cluster](../../user/infrastructure/clusters/index.md) +associated with your project, you can configure these deployments from your +`.gitlab-ci.yml` file. + +NOTE: +Kubernetes configuration isn't supported for Kubernetes clusters +[managed by GitLab](../../user/project/clusters/gitlab_managed_clusters.md). + +The following configuration options are supported: + +- [`namespace`](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) + +In the following example, the job deploys your application to the +`production` Kubernetes namespace. + +```yaml +deploy: + stage: deploy + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + kubernetes: + namespace: production + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH +``` + +When you use the GitLab Kubernetes integration to deploy to a Kubernetes cluster, +you can view cluster and namespace information. On the deployment +job page, it's displayed above the job trace: + + + +## Configure incremental rollouts + +Learn how to release production changes to only a portion of your Kubernetes pods with +[incremental rollouts](../environments/incremental_rollouts.md). + +## Related topics + +- [Deploy boards (deprecated)](../../user/project/deploy_boards.md) diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index 96011d5ddff..0ef37452cbb 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -51,7 +51,7 @@ Example: There are two ways to configure the approval requirements: -- [Unified approval setting](#unified-approval-setting) ... You can define who can execute **and** approve deployments. +- [Unified approval setting](#unified-approval-setting-deprecated) ... You can define who can execute **and** approve deployments. This is useful when there is no separation of duties between executors and approvers in your organization. - [Multiple approval rules](#multiple-approval-rules) ... You can define who can execute **or** approve deployments. This is useful when there is a separation of duties between executors and approvers in your organization. @@ -60,11 +60,18 @@ NOTE: Multiple approval rules is a more flexible option than the unified approval setting, thus both configurations shouldn't co-exist and multiple approval rules takes the precedence over the unified approval setting if it happens. -#### Unified approval setting +<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> + +#### Unified approval setting (deprecated) > - UI configuration [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/378447) in GitLab > 15.11. +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/9662) in GitLab 16.1 and is planned for removal +in 17.0. Use [multiple approval rules](https://gitlab.com/gitlab-org/gitlab/-/issues/404579) instead. This change +is a breaking change. + To configure approvals for a protected environment: - Using the [REST API](../../api/protected_environments.md#protect-a-single-environment), @@ -85,6 +92,8 @@ NOTE: To protect, update, or unprotect an environment, you must have at least the Maintainer role. +<!--- end_remove --> + #### Multiple approval rules > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) in GitLab 14.10 with a flag named `deployment_approval_rules`. Disabled by default. @@ -119,6 +128,41 @@ NOTE: To protect, update, or unprotect an environment, you must have at least the Maintainer role. +#### Migrate to multiple approval rules + +You can migrate a protected environment from unified approval rules to multiple +approval rules. Unified approval rules allow all entities that can deploy to an +environment to approve deployment jobs. To migrate to multiple approval rules, +create a new approval rule for each entity allowed to deploy to the environment. + +To migrate with the UI: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. +1. Expand **Protected environments**. +1. From the **Environment** list, select your environment. +1. For each entity allowed to deploy to the environment: + 1. Select **Add approval rules**. + 1. In the modal window, select which entity is allowed to approve the + deployment job. + 1. Enter the number of required approvals. + 1. Select **Save**. + +Each deployment requires the specified number of approvals from each entity. + +For example, the `Production` environment below requires five total approvals, +and allows deployments from only the group `Very Important Group` and the user +`Administrator`: + + + +To migrate, create rules for the `Very Important Group` and `Administrator`. To +preserve the number of required approvals, set the number of required approvals +for `Very Important Group` to four and `Administrator` to one. The new rules +require `Administrator` to approve every deployment job in `Production`. + + + ### Allow self-approval **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381418) in GitLab 15.8. @@ -126,8 +170,8 @@ Maintainer role. By default, the user who triggers a deployment pipeline can't also approve the deployment job. To allow self-approval of a deployment job: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Protected environments**. 1. From the **Approval options**, select the **Allow pipeline triggerer to approve deployment** checkbox. @@ -154,8 +198,8 @@ Prerequisites: To approve or reject a deployment to a protected environment using the UI: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. 1. Select the environment's name. 1. In the deployment's row, select **Approval options** (**{thumb-up}**). Before approving or rejecting the deployment, you can view the number of approvals granted and @@ -191,8 +235,8 @@ granted. To view the approval details of a deployment: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. 1. Select the environment's name. 1. In the deployment's row, select **Approval options** (**{thumb-up}**). @@ -207,8 +251,8 @@ The approval status details are shown: ### Using the UI -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. 1. Select the environment being deployed to. 1. Look for the `blocked` label. @@ -217,7 +261,7 @@ The approval status details are shown: Use the [Deployments API](../../api/deployments.md#get-a-specific-deployment) to see deployments. - The `status` field indicates if a deployment is blocked. -- When the [unified approval setting](#unified-approval-setting) is configured: +- When the [unified approval setting](#unified-approval-setting-deprecated) is configured: - The `pending_approval_count` field indicates how many approvals are remaining to run a deployment. - The `approvals` field contains the deployment's approvals. - When the [multiple approval rules](#multiple-approval-rules) is configured: diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md index fd2c85819fe..ce219e5d746 100644 --- a/doc/ci/environments/environments_dashboard.md +++ b/doc/ci/environments/environments_dashboard.md @@ -20,8 +20,9 @@ see which pipelines are green and which are red allowing you to diagnose if there is a block at a particular point, or if there's a more systemic problem you need to investigate. -You can access the dashboard on the top bar by selecting -**Main menu > Environments**. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Your work**. +1. Select **Environments**.  diff --git a/doc/ci/environments/img/multiple_approval_rules_v16_0.png b/doc/ci/environments/img/multiple_approval_rules_v16_0.png Binary files differnew file mode 100644 index 00000000000..2385bea6956 --- /dev/null +++ b/doc/ci/environments/img/multiple_approval_rules_v16_0.png diff --git a/doc/ci/environments/img/unified_approval_rules_v16_0.png b/doc/ci/environments/img/unified_approval_rules_v16_0.png Binary files differnew file mode 100644 index 00000000000..7f822aedff8 --- /dev/null +++ b/doc/ci/environments/img/unified_approval_rules_v16_0.png diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index f4d155369e9..f2fb8eaa27e 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -51,8 +51,8 @@ Deployments show up in this list only after a deployment job has created them. To search environments by name: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. 1. In the search bar, enter your search term. - The length of your **search term should be 3 or more characters**. - Matching applies from the beginning of the environment name. @@ -61,6 +61,12 @@ To search environments by name: - For example when the name is `review/test-app`, search term `test` matches `review/test-app`. - Also searching with the folder name prefixed like `review/test` matches `review/test-app`. +## CI/CD variables + +To customize your environments and deployments, you can use any of the +[predefined CI/CD variables](../../ci/variables/predefined_variables.md), +and define custom CI/CD variables. + ## Types of environments An environment is either static or dynamic: @@ -87,9 +93,9 @@ Prerequisites: To create a static environment in the UI: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. -1. Select **New environment**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. +1. Select **Create an environment**. 1. Complete the fields. 1. Select **Save**. @@ -123,7 +129,8 @@ deploy_staging: ### Create a dynamic environment -To create a dynamic environment, you use [CI/CD variables](../variables/index.md) that are unique to each pipeline. +To create a dynamic environment, you use [CI/CD variables](#cicd-variables) that are +unique to each pipeline. Prerequisites: @@ -158,6 +165,79 @@ deploy_review_app: - main ``` +#### Set a dynamic environment URL + +Some external hosting platforms generate a random URL for each deployment, for example: +`https://94dd65b.amazonaws.com/qa-lambda-1234567`. That makes it difficult to reference the URL in +the `.gitlab-ci.yml` file. + +To address this problem, you can configure a deployment job to report back a set of +variables. These variables include the URL that was dynamically generated by the external service. +GitLab supports the [dotenv (`.env`)](https://github.com/bkeepers/dotenv) file format, +and expands the `environment:url` value with variables defined in the `.env` file. + +To use this feature, specify the +[`artifacts:reports:dotenv`](../yaml/artifacts_reports.md#artifactsreportsdotenv) keyword in `.gitlab-ci.yml`. + +You can also specify a static part of the URL at `environment:url`, such as +`https://$DYNAMIC_ENVIRONMENT_URL`. If the value of `DYNAMIC_ENVIRONMENT_URL` is `example.com`, the +final result is `https://example.com`. + +The assigned URL for the `review/your-branch-name` environment is visible in the UI. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Set dynamic URLs after a job finished](https://youtu.be/70jDXtOf4Ig). + +In the following example a review app creates a new environment for each merge request: + +- The `review` job is triggered by every push, and creates or updates an environment named + `review/your-branch-name`. The environment URL is set to `$DYNAMIC_ENVIRONMENT_URL`. +- When the `review` job finishes, GitLab updates the `review/your-branch-name` environment's URL. + It parses the `deploy.env` report artifact, registers a list of variables as runtime-created, + expands the `environment:url: $DYNAMIC_ENVIRONMENT_URL` and sets it to the environment + URL. + +```yaml +review: + script: + - DYNAMIC_ENVIRONMENT_URL=$(deploy-script) # In script, get the environment URL. + - echo "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL" >> deploy.env # Add the value to a dotenv file. + artifacts: + reports: + dotenv: deploy.env # Report back dotenv file to rails. + environment: + name: review/$CI_COMMIT_REF_SLUG + url: $DYNAMIC_ENVIRONMENT_URL # and set the variable produced in script to `environment:url` + on_stop: stop_review + +stop_review: + script: + - ./teardown-environment + when: manual + environment: + name: review/$CI_COMMIT_REF_SLUG + action: stop +``` + +Note the following: + +- `stop_review` doesn't generate a dotenv report artifact, so it doesn't recognize the + `DYNAMIC_ENVIRONMENT_URL` environment variable. Therefore you shouldn't set `environment:url` in the + `stop_review` job. +- If the environment URL isn't valid (for example, the URL is malformed), the system doesn't update + the environment URL. +- If the script that runs in `stop_review` exists only in your repository and therefore can't use + `GIT_STRATEGY: none`, configure [merge request pipelines](../../ci/pipelines/merge_request_pipelines.md) + for these jobs. This ensures that runners can fetch the repository even after a feature branch is + deleted. For more information, see [Ref Specs for Runners](../pipelines/index.md#ref-specs-for-runners). + +NOTE: +For Windows runners, you should use the PowerShell `Add-Content` command to write to `.env` files. + +```powershell +Add-Content -Path deploy.env -Value "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL" +``` + ### Rename an environment > - Renaming an environment by using the UI was [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68550) in GitLab 14.3. @@ -220,153 +300,6 @@ The `when: manual` action: You can find the play button in the pipelines, environments, deployments, and jobs views. -## Configure Kubernetes deployments (deprecated) - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -WARNING: -This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -If you are deploying to a [Kubernetes cluster](../../user/infrastructure/clusters/index.md) -associated with your project, you can configure these deployments from your -`.gitlab-ci.yml` file. - -NOTE: -Kubernetes configuration isn't supported for Kubernetes clusters -[managed by GitLab](../../user/project/clusters/gitlab_managed_clusters.md). - -The following configuration options are supported: - -- [`namespace`](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) - -In the following example, the job deploys your application to the -`production` Kubernetes namespace. - -```yaml -deploy: - stage: deploy - script: - - echo "Deploy to production server" - environment: - name: production - url: https://example.com - kubernetes: - namespace: production - rules: - - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH -``` - -When you use the GitLab Kubernetes integration to deploy to a Kubernetes cluster, -you can view cluster and namespace information. On the deployment -job page, it's displayed above the job trace: - - - -### Configure incremental rollouts - -Learn how to release production changes to only a portion of your Kubernetes pods with -[incremental rollouts](../environments/incremental_rollouts.md). - -## CI/CD variables for environments and deployments - -When you create an environment, you specify the name and URL. - -If you want to use the name or URL in another job, you can use: - -- `$CI_ENVIRONMENT_NAME`. The name defined in the `.gitlab-ci.yml` file. -- `$CI_ENVIRONMENT_SLUG`. A "cleaned-up" version of the name, suitable for use in URL and DNS, for example. - This variable is guaranteed to be unique. -- `$CI_ENVIRONMENT_URL`. The environment's URL, which was specified in the - `.gitlab-ci.yml` file or automatically assigned. - -If you change the name of an existing environment, the: - -- `$CI_ENVIRONMENT_NAME` variable is updated with the new environment name. -- `$CI_ENVIRONMENT_SLUG` variable remains unchanged to prevent unintended side - effects. - -## Set dynamic environment URLs after a job finishes - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17066) in GitLab 12.9. - -In a job script, you can specify a static environment URL. -However, there may be times when you want a dynamic URL. For example, -if you deploy a Review App to an external hosting -service that generates a random URL per deployment, like `https://94dd65b.amazonaws.com/qa-lambda-1234567`. -In this case, you don't know the URL before the deployment script finishes. -If you want to use the environment URL in GitLab, you would have to update it manually. - -To address this problem, you can configure a deployment job to report back a set of -variables. These variables include the URL that was dynamically-generated by the external service. -GitLab supports the [dotenv (`.env`)](https://github.com/bkeepers/dotenv) file format, -and expands the `environment:url` value with variables defined in the `.env` file. - -To use this feature, specify the -[`artifacts:reports:dotenv`](../yaml/artifacts_reports.md#artifactsreportsdotenv) keyword in `.gitlab-ci.yml`. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see [Set dynamic URLs after a job finished](https://youtu.be/70jDXtOf4Ig). - -### Example of setting dynamic environment URLs - -The following example shows a Review App that creates a new environment -for each merge request. The `review` job is triggered by every push, and -creates or updates an environment named `review/your-branch-name`. -The environment URL is set to `$DYNAMIC_ENVIRONMENT_URL`: - -```yaml -review: - script: - - DYNAMIC_ENVIRONMENT_URL=$(deploy-script) # In script, get the environment URL. - - echo "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL" >> deploy.env # Add the value to a dotenv file. - artifacts: - reports: - dotenv: deploy.env # Report back dotenv file to rails. - environment: - name: review/$CI_COMMIT_REF_SLUG - url: $DYNAMIC_ENVIRONMENT_URL # and set the variable produced in script to `environment:url` - on_stop: stop_review - -stop_review: - script: - - ./teardown-environment - when: manual - environment: - name: review/$CI_COMMIT_REF_SLUG - action: stop -``` - -As soon as the `review` job finishes, GitLab updates the `review/your-branch-name` -environment's URL. -It parses the `deploy.env` report artifact, registers a list of variables as runtime-created, -uses it for expanding `environment:url: $DYNAMIC_ENVIRONMENT_URL` and sets it to the environment URL. -You can also specify a static part of the URL at `environment:url`, such as -`https://$DYNAMIC_ENVIRONMENT_URL`. If the value of `DYNAMIC_ENVIRONMENT_URL` is -`example.com`, the final result is `https://example.com`. - -The assigned URL for the `review/your-branch-name` environment is visible in the UI. - -Note the following: - -- `stop_review` doesn't generate a dotenv report artifact, so it doesn't recognize the - `DYNAMIC_ENVIRONMENT_URL` environment variable. Therefore you shouldn't set `environment:url` in the - `stop_review` job. -- If the environment URL isn't valid (for example, the URL is malformed), the system doesn't update - the environment URL. -- If the script that runs in `stop_review` exists only in your repository and therefore can't use - `GIT_STRATEGY: none`, configure [merge request pipelines](../../ci/pipelines/merge_request_pipelines.md) - for these jobs. This ensures that runners can fetch the repository even after a feature branch is - deleted. For more information, see [Ref Specs for Runners](../pipelines/index.md#ref-specs-for-runners). - -NOTE: -For Windows runners, using `echo` to write to `.env` files may fail. Using the PowerShell `Add-Content`command -helps in such cases. For example: - -```powershell -Add-Content -Path deploy.env -Value "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL" -``` - ## Track newly included merge requests per deployment GitLab can track newly included merge requests per deployment. @@ -412,8 +345,8 @@ If there is a problem with a deployment, you can retry it or roll it back. To retry or rollback a deployment: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. 1. Select the environment. 1. To the right of the deployment name: - To retry a deployment, select **Re-deploy to environment**. @@ -627,8 +560,8 @@ you can view its expiration date and time. To view an environment's expiration date and time: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. 1. Select the name of the environment. The expiration date and time is displayed in the upper-left corner, next to the environment's name. @@ -640,8 +573,8 @@ you can override its expiration. To override an environment's expiration: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. 1. Select the deployment name. 1. in the upper-right corner, select the thumbtack (**{thumbtack}**). @@ -652,7 +585,7 @@ manually. There may be times when you want to stop an environment without running the defined [`on_stop`](../yaml/index.md#environmenton_stop) action. For example, you want to delete many -environments without using CI/CD minutes. +environments without using [compute quota](../pipelines/cicd_minutes.md). To stop an environment without running the defined `on_stop` action, execute the [Stop an environment API](../../api/environments.md#stop-an-environment) with the parameter @@ -667,8 +600,8 @@ Environments view, the stop and deploy jobs must be in the same To stop an environment in the GitLab UI: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. 1. Next to the environment you want to stop, select **Stop**. 1. On the confirmation dialog box, select **Stop environment**. @@ -731,8 +664,8 @@ Prerequisites: To delete an environment: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. 1. Select the **Stopped** tab. 1. Next to the environment you want to delete, select **Delete environment**. 1. On the confirmation dialog box, select **Delete environment**. @@ -800,7 +733,7 @@ to get alerts when there are critical issues that need immediate attention. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214634) in GitLab 13.4. -If you [set up alerts for Prometheus metrics](../../operations/metrics/alerts.md), +If you [set up alerts for Prometheus metrics](../../operations/incident_management/integrations.md#configuration), alerts for environments are shown on the environments page. The alert with the highest severity is shown, so you can identify which environments need immediate attention. @@ -834,34 +767,20 @@ Limitations of GitLab Auto Rollback: GitLab Auto Rollback is turned off by default. To turn it on: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Automatic deployment rollbacks**. 1. Select the checkbox for **Enable automatic rollbacks**. 1. Select **Save changes**. -### Monitor environments - -To monitor the behavior of your app as it runs in each environment, -enable [Prometheus for monitoring system and response metrics](../../user/project/integrations/prometheus.md). -For the monitoring dashboard to appear, configure Prometheus to collect at least one -[supported metric](../../user/project/integrations/prometheus_library/index.md). - -All deployments to an environment are shown on the monitoring dashboard. -You can view changes in performance for each version of your application. - -GitLab attempts to retrieve [supported performance metrics](../../user/project/integrations/prometheus_library/index.md) -for any environment that has had a successful deployment. If monitoring data was -successfully retrieved, a **Monitoring** button appears for each environment. - -To view the last eight hours of performance data, select the **Monitoring** button. -It may take a minute or two for data to appear after initial deployment. +<!--- start_remove The following content will be removed on remove_date: '2023-09-22' --> - +### Monitor environments (removed) -#### Embed metrics in GitLab Flavored Markdown +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/10107) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. -Metric charts can be embedded in GitLab Flavored Markdown. See [Embedding Metrics in GitLab Flavored Markdown](../../operations/metrics/embed.md) for more details. +<!--- end_remove --> ### Web terminals (deprecated) @@ -990,14 +909,14 @@ the `review/feature-1` spec takes precedence over `review/*` and `*` specs. ## Related topics -- [Use GitLab CI to deploy to multiple environments (blog post)](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/) -- [Review Apps](../review_apps/index.md): Use dynamic environments to deploy your code for every branch. -- [Deploy boards](../../user/project/deploy_boards.md): View the status of your applications running on Kubernetes. -- [Protected environments](protected_environments.md): Determine who can deploy code to your environments. -- [Environments Dashboard](../environments/environments_dashboard.md): View a summary of each - environment's operational health. **(PREMIUM)** -- [Deployment safety](deployment_safety.md#restrict-write-access-to-a-critical-environment): Secure your deployments. -- [Track deployments of an external deployment tool](external_deployment_tools.md): Use an external deployment tool instead of built-in deployment solution. +- [Dashboard for Kubernetes](kubernetes_dashboard.md) +- [Deploy to multiple environments with GitLab CI/CD (blog post)](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/) +- [Review Apps](../review_apps/index.md) +- [Protected environments](protected_environments.md) +- [Environments Dashboard](../environments/environments_dashboard.md) +- [Deployment safety](deployment_safety.md#restrict-write-access-to-a-critical-environment) +- [Track deployments of an external deployment tool](external_deployment_tools.md) +- [Configure Kubernetes deployments (deprecated)](configure_kubernetes_deployments.md) ## Troubleshooting diff --git a/doc/ci/environments/kubernetes_dashboard.md b/doc/ci/environments/kubernetes_dashboard.md new file mode 100644 index 00000000000..7da48bed5d7 --- /dev/null +++ b/doc/ci/environments/kubernetes_dashboard.md @@ -0,0 +1,66 @@ +--- +stage: Deploy +group: Environments +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Dashboard for Kubernetes (Beta) **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390769) in GitLab 16.1, with [flags](../../administration/feature_flags.md) named `environment_settings_to_graphql`, `kas_user_access`, `kas_user_access_project`, and `expose_authorized_cluster_agents`. This feature is in [Beta](../../policy/experiment-beta-support.md#beta). + +Use the Dashboard for Kubernetes to understand the status of your clusters with an intuitive visual interface. +The dashboard works with every connected Kubernetes cluster, whether you deployed them +with CI/CD or GitOps. + +For Flux users, the synchronization status of a given environment is not displayed in the dashboard. +[Issue 391581](https://gitlab.com/gitlab-org/gitlab/-/issues/391581) proposes to add this functionality. + +## Configure a dashboard + +Configure a dashboard to use it for a given environment. +You can configure dashboard for an environment that already exists, or +add one when you create an environment. + +Prerequisite: + +- The agent for Kubernetes must be shared with the environment's project, or its parent group, using the [`user_access`](../../user/clusters/agent/user_access.md) keyword. + +### The environment already exists + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. +1. Select the environment to be associated with the Kubernetes. +1. Select **Edit**. +1. Select a GitLab agent for Kubernetes. +1. Select **Save**. + +### The environment doesn't exist + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. +1. Select **New environment**. +1. Complete the **Name** field. +1. Select a GitLab agent for Kubernetes. +1. Select **Save**. + +## View a dashboard + +To view a configured dashboard: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Operate > Environments**. +1. Expand the environment associated with GitLab agent for Kubernetes. +1. Expand **Kubernetes overview**. + +## Troubleshooting + +When working with the Dashboard for Kubernetes, you might encounter the following issues. + +### User cannot list resource in API group + +You might get an error that states `Error: services is forbidden: User "gitlab:user:<user-name>" cannot list resource "<resource-name>" in API group "" at the cluster scope`. + +This error happens when a user is not allowed to do the specified operation in the [Kubernetes RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/). + +To resolve, check your [RBAC configuration](../../user/clusters/agent/user_access.md#configure-kubernetes-access). If the RBAC is properly configured, contact your Kubernetes administrator. diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index f752e2179df..61b59ceedb2 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -30,8 +30,8 @@ Prerequisites: To protect an environment: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Protected environments**. 1. From the **Environment** list, select the environment you want to protect. 1. In the **Allowed to deploy** list, select the role, users, or groups you @@ -87,7 +87,7 @@ Alternatively, you can use the API to protect an environment: name: ${CI_JOB_NAME} ``` -1. Use the UI to [create a new group](../../user/group/manage.md#create-a-group). +1. Use the UI to [create a new group](../../user/group/index.md#create-a-group). For example, this group is called `protected-access-group` and has the group ID `9899826`. Note that the rest of the examples in these steps use this group. @@ -255,8 +255,8 @@ To protect a group-level environment, make sure your environments have the corre > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325249) in GitLab 15.1. -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Settings > CI/CD**. 1. Expand **Protected environments**. 1. From the **Environment** list, select the [deployment tier of environments](index.md#deployment-tier-of-environments) you want to protect. 1. In the **Allowed to deploy** list, select the [subgroups](../../user/group/subgroups/index.md) you want to give deploy access to. diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png Binary files differdeleted file mode 100644 index 16c15071dd7..00000000000 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png +++ /dev/null diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png Binary files differdeleted file mode 100644 index 7a92a3c2d50..00000000000 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png +++ /dev/null diff --git a/doc/ci/examples/deployment/index.md b/doc/ci/examples/deployment/index.md index 664ce84c488..a156cf1acb0 100644 --- a/doc/ci/examples/deployment/index.md +++ b/doc/ci/examples/deployment/index.md @@ -121,8 +121,8 @@ We also use two secure variables: To store API keys as secure variables: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Variables**. The variables defined in the project settings are sent along with the build script to the runner. diff --git a/doc/ci/examples/index.md b/doc/ci/examples/index.md index f2871e50617..a020f673fd7 100644 --- a/doc/ci/examples/index.md +++ b/doc/ci/examples/index.md @@ -23,11 +23,9 @@ The following table lists examples with step-by-step tutorials that are containe | Use case | Resource | |-------------------------------|----------| -| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../testing/browser_performance_testing.md). | | Deployment with Dpl | [Using `dpl` as deployment tool](deployment/index.md). | | GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | | End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | -| Load performance testing | [Load Performance Testing with the k6 container](../testing/load_performance_testing.md). | | Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | | npm with semantic-release | [Publish npm packages to the GitLab Package Registry using semantic-release](semantic-release.md). | | PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index 2146d76e4d6..00260199179 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -90,14 +90,14 @@ As part of publishing a package, semantic-release increases the version number i <!-- markdownlint-disable MD044 --> -1. On the top bar, in the upper-right corner, select your avatar. -1. On the left sidebar, select **Access Tokens**. +1. On the left sidebar, select your avatar. +1. Select **Preferences > Access Tokens**. 1. In the **Token name** box, enter a token name. 1. Under **select scopes**, select the **api** checkbox. 1. Select **Create project access token**. 1. Copy the value. -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Variables**. 1. Select **Add variable**. 1. In the **Key** box, enter `GITLAB_TOKEN`. diff --git a/doc/ci/img/environments_monitoring.png b/doc/ci/img/environments_monitoring.png Binary files differdeleted file mode 100644 index 63d272ae42a..00000000000 --- a/doc/ci/img/environments_monitoring.png +++ /dev/null diff --git a/doc/ci/index.md b/doc/ci/index.md index 65e6394b439..75e668290c8 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -82,8 +82,6 @@ GitLab CI/CD features, grouped by DevOps stage, include: | [ChatOps](chatops/index.md) | Trigger CI jobs from chat, with results sent back to the channel. | | [Connect to cloud services](cloud_services/index.md) | Connect to cloud providers using OpenID Connect (OIDC) to retrieve temporary credentials to access services or secrets. | | **Verify** | | -| [Browser Performance Testing](testing/browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | -| [Load Performance Testing](testing/load_performance_testing.md) | Quickly determine the server performance impact of pending code changes. | | [CI services](services/index.md) | Link Docker containers with your base image. | | [GitLab CI/CD for external repositories](ci_cd_for_external_repos/index.md) | Get the benefits of GitLab CI/CD combined with repositories in GitHub and Bitbucket Cloud. | | [Interactive Web Terminals](interactive_web_terminal/index.md) | Open an interactive web terminal to debug the running jobs. | diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index bf07af5e761..895aa8551d7 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -61,8 +61,7 @@ of the changes. ## Continuous Deployment -[Continuous Deployment](https://www.airpair.com/continuous-deployment/posts/continuous-deployment-for-practical-people) -is another step beyond Continuous Integration, similar to +Continuous Deployment is another step beyond Continuous Integration, similar to Continuous Delivery. The difference is that instead of deploying your application manually, you set it to be deployed automatically. Human intervention is not required. diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index 9cbf45a16e7..a60aaa04ea1 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -44,6 +44,8 @@ in a CI/CD job: git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.example.com/<namespace>/<project> ``` +You can't use a job token to push to a repository, but [issue 389060](https://gitlab.com/gitlab-org/gitlab/-/issues/389060) proposes to change this behavior. + ## GitLab CI/CD job token security To make sure that this token doesn't leak, GitLab: @@ -115,8 +117,8 @@ Prerequisite: To disable the job token scope allowlist: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Token Access**. 1. Toggle **Allow access to this project with a CI_JOB_TOKEN** to disabled. Enabled by default in new projects. @@ -136,8 +138,8 @@ Prerequisite: To add a project: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Token Access**. 1. Verify **Allow access to this project with a CI_JOB_TOKEN** is enabled. 1. Under **Allow CI job tokens from the following projects to access this project**, @@ -178,8 +180,8 @@ Prerequisite: To configure the job token scope: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Token Access**. 1. Toggle **Limit CI_JOB_TOKEN access** to enabled. 1. Optional. Add existing projects to the token's access scope. The user adding a diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index b9c2ee409b8..a0c0fc43502 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -48,8 +48,8 @@ Selecting an individual job shows you its job log, and allows you to: To view the full list of jobs that ran in a project: -1. On the top bar, select **Main menu > Projects** and find the project. -1. On the left sidebar, select **CI/CD > Jobs**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Jobs**. You can filter the list by [job status](#the-order-of-jobs-in-a-pipeline). @@ -342,7 +342,7 @@ In the example above: job log, but they are displayed in the raw job log. To see them, in the upper-right corner of the job log, select **Show complete raw** (**{doc-text}**). - `\r`: carriage return. - - `\e[0K`: clear line ANSI escape code. + - `\e[0K`: clear line ANSI escape sequence (`\e[K` does not work, the `0` must be included). Sample raw job log: diff --git a/doc/ci/jobs/job_artifacts.md b/doc/ci/jobs/job_artifacts.md index 0c9dd658ce2..95d939c7027 100644 --- a/doc/ci/jobs/job_artifacts.md +++ b/doc/ci/jobs/job_artifacts.md @@ -294,17 +294,13 @@ You can also delete individual artifacts from the [**Artifacts** page](#bulk-del ### Bulk delete artifacts -- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33348) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `ci_job_artifact_bulk_destroy`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `ci_job_artifact_bulk_destroy`. -The feature is not ready for production use. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33348) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `ci_job_artifact_bulk_destroy`. Disabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/398581) in GitLab 16.1. You can delete multiple artifacts at the same time: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **CI/CD > Artifacts**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Artifacts**. 1. Select the checkboxes next to the artifacts you want to delete. You can select up to 50 artifacts. 1. Select **Delete selected**. @@ -334,7 +330,7 @@ With this configuration, GitLab adds **artifact 1** as a link to `file.txt` to t By default artifacts are always kept for successful pipelines for the most recent commit on each ref. This means that the latest artifacts do not immediately expire according -to the `expire_in` specification. +to the `expire_in` configuration. If a pipeline for a new commit on the same ref completes successfully, the previous pipeline's artifacts are deleted according to the `expire_in` configuration. The artifacts @@ -345,11 +341,15 @@ Keeping the latest artifacts can use a large amount of storage space in projects with a lot of jobs or large artifacts. If the latest artifacts are not needed in a project, you can disable this behavior to save space: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Artifacts**. 1. Clear the **Keep artifacts from most recent successful jobs** checkbox. +After disabling this setting, all new artifacts expire according to the `expire_in` configuration. +Artifacts in old pipelines continue to be kept until a new pipeline runs for the same ref. +Then the artifacts in the earlier pipeline for that ref are allowed to expire too. + You can disable this behavior for all projects on a self-managed instance in the [instance's CI/CD settings](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index fa045a898fa..b17db47eef2 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -586,7 +586,7 @@ In blocking manual jobs: defined inside [`rules`](../yaml/index.md#rules). - The pipeline stops at the stage where the job is defined. To let the pipeline continue running, [run the manual job](#run-a-manual-job). -- Merge requests in projects with [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) +- Merge requests in projects with [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) enabled can't be merged with a blocked pipeline. - The pipeline shows a status of **blocked**. diff --git a/doc/ci/large_repositories/index.md b/doc/ci/large_repositories/index.md index f728e9b9e17..4b17f3354aa 100644 --- a/doc/ci/large_repositories/index.md +++ b/doc/ci/large_repositories/index.md @@ -245,19 +245,11 @@ concurrent = 4 This makes the cloning configuration to be part of the given runner and does not require us to update each `.gitlab-ci.yml`. -## Git fetch caching or pre-clone step - -For very active repositories with a large number of references and files, you can either (or both): - -- Consider using the [Gitaly pack-objects cache](../../administration/gitaly/configure_gitaly.md#pack-objects-cache) instead of a - pre-clone step. This is easier to set up and it benefits all repositories on your GitLab server, unlike the pre-clone step that - must be configured per-repository. The pack-objects cache also automatically works for forks. On GitLab.com, where the pack-objects cache is - enabled on all Gitaly servers, we found that we no longer need a pre-clone step for `gitlab-org/gitlab` development. -- Optimize your CI/CD jobs by seeding repository data in a pre-clone step with the - [`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) of GitLab Runner. See - [SaaS runners on Linux](../runners/saas/linux_saas_runner.md#pre-clone-script-deprecated) for details. - Besides speeding up pipelines in large and active projects, - seeding the repository data also helps avoid - `429 Too many requests` errors from Cloudflare. - This error can occur if you have many runners behind a single, - IP address using NAT, that pulls from GitLab.com. +## Git fetch caching step + +For very active repositories with a large number of references and files, consider using the +[Gitaly pack-objects cache](../../administration/gitaly/configure_gitaly.md#pack-objects-cache). +The pack-objects cache: + +- Benefits all repositories on your GitLab server. +- Automatically works for forks. diff --git a/doc/ci/lint.md b/doc/ci/lint.md index 119a0e5853e..f4c8a377c31 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -24,8 +24,8 @@ configuration added with the [`includes` keyword](yaml/index.md#include). To check CI/CD configuration with the CI lint tool: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **CI/CD > Pipelines**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Pipelines**. 1. In the upper-right corner, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. 1. Select **Validate**. @@ -45,8 +45,8 @@ Prerequisites: To simulate a pipeline: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **CI/CD > Pipelines**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Pipelines**. 1. In the upper-right corner, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. 1. Select **Simulate pipeline creation for the default branch**. diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index b50695e8e50..17cdb4f7e3e 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -291,8 +291,8 @@ Self-managed runners: GitLab.com shared runners: - Linux -- [Windows](../runners/saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta)). -- [macOS](../runners/saas/macos_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta)). +- [Windows](../runners/saas/windows_saas_runner.md) ([Beta](../../policy/experiment-beta-support.md#beta)). +- [macOS](../runners/saas/macos_saas_runner.md) ([Beta](../../policy/experiment-beta-support.md#beta)). ### Machine and specific build environments diff --git a/doc/ci/mobile_devops.md b/doc/ci/mobile_devops.md index 175a63dc3b9..a85de5e2a51 100644 --- a/doc/ci/mobile_devops.md +++ b/doc/ci/mobile_devops.md @@ -41,10 +41,9 @@ test: ### iOS build environments -GitLab SaaS runners on macOS are currently available in beta. Follow the [instructions to request access](../ci/runners/saas/macos_saas_runner.md#access-request-process) -for your project. +[GitLab SaaS runners on macOS](../ci/runners/saas/macos_saas_runner.md) are currently available in beta. -After you are granted access to the beta macOS runners, [choose an image](../ci/runners/saas/macos/environment.md#available-images) +After you are granted access to the beta macOS runners, [choose an image](../ci/runners/saas/macos_saas_runner.md#supported-macos-images) and add it to your `.gitlab-ci.yml` file. For example: @@ -271,7 +270,7 @@ For example: script: - fastlane build tags: - - shared-macos-amd64 + - saas-macos-medium-m1 ``` ## Distribution @@ -297,8 +296,8 @@ Use the [Google Play integration](../user/project/integrations/google_play.md), to configure your CI/CD pipelines to connect to the [Google Play Console](https://play.google.com/console) to build and release Android apps. To enable the integration: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Integrations**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > Integrations**. 1. Select **Google Play**. 1. In **Enable integration**, select the **Active** checkbox. 1. In **Package name**, enter the package name of the app. For example, `com.gitlab.app_name`. @@ -354,8 +353,8 @@ Use the [Apple App Store integration](../user/project/integrations/apple_app_sto to configure your CI/CD pipelines to connect to [App Store Connect](https://appstoreconnect.apple.com/) to build and release apps for iOS, iPadOS, macOS, tvOS, and watchOS. To enable the integration: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Integrations**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > Integrations**. 1. Select **Apple App Store**. 1. Turn on the **Active** toggle under **Enable Integration**. 1. Provide the Apple App Store Connect configuration information: diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index d920c34d90a..fcacc3b5d97 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -113,7 +113,7 @@ where: with the linked configuration. Using `!reference` tags can cause nested configuration that display with -multiple hyphens (`-`) in the expanded view. This behavior is expected, and the extra +multiple hyphens (`-`) at the start of the line in the expanded view. This behavior is expected, and the extra hyphens do not affect the job's execution. For example, this configuration and fully expanded version are both valid: @@ -124,11 +124,27 @@ fully expanded version are both valid: script: - pip install pyflakes + .rule-01: + rules: + - if: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ + when: manual + allow_failure: true + - if: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME + + .rule-02: + rules: + - if: $CI_COMMIT_BRANCH == "main" + when: manual + allow_failure: true + lint-python: image: python:latest script: - !reference [.python-req, script] - pyflakes python/ + rules: + - !reference [.rule-01, rules] + - !reference [.rule-02, rules] ``` - Expanded configuration in **Full configuration** tab: @@ -137,11 +153,30 @@ fully expanded version are both valid: ".python-req": script: - pip install pyflakes + ".rule-01": + rules: + - if: "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/" + when: manual + allow_failure: true + - if: "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME" + ".rule-02": + rules: + - if: $CI_COMMIT_BRANCH == "main" + when: manual + allow_failure: true lint-python: + image: python:latest script: - - - pip install pyflakes # <- The extra hyphens do not affect the job's execution. + - - pip install pyflakes # <- The extra hyphens do not affect the job's execution. - pyflakes python/ - image: python:latest + rules: + - - if: "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/" # <- The extra hyphens do not affect the job's execution. + when: manual + allow_failure: true + - if: "$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME" # <- No extra hyphen but aligned with previous rule + - - if: $CI_COMMIT_BRANCH == "main" # <- The extra hyphens do not affect the job's execution. + when: manual + allow_failure: true ``` ## Commit changes to CI configuration diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index ee3f0d8c539..c29d23cfff7 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -5,45 +5,47 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# CI/CD minutes quota **(PREMIUM)** +# Compute quota **(PREMIUM)** + +> [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" or "units of compute" in GitLab 16.1. NOTE: The term `CI/CD minutes` is being renamed to `units of compute`. During this transition, you might see references in the UI and documentation to `CI/CD minutes`, `CI minutes`, `pipeline minutes`, `CI pipeline minutes`, `pipeline minutes quota`, and `units of compute`. For more information, see [epic 2150](https://gitlab.com/groups/gitlab-com/-/epics/2150). Administrators can limit the amount of time that projects can use to run jobs on [shared runners](../runners/runners_scope.md#shared-runners) each month. This limit -is tracked with a quota of CI/CD minutes. +is tracked with a compute quota. By default, one minute of execution time by a single job uses -one CI/CD minute. The total amount of CI/CD minutes used by a pipeline is -[the sum of all its jobs' durations](#how-cicd-minute-usage-is-calculated). -Jobs can run concurrently, so the total CI/CD minute usage can be higher than the +one unit of compute. The total execution time for a pipeline is +[the sum of all its jobs' durations](#how-compute-usage-is-calculated). +Jobs can run concurrently, so the total usage can be higher than the end-to-end duration of a pipeline. On GitLab.com: -- CI/CD minutes quotas are enabled for all projects, but certain - projects [consume CI/CD minutes at a slower rate](#cost-factor). -- The base monthly CI/CD minutes quota for a GitLab.com [namespace](../../user/namespace/index.md) +- Compute quotas are enabled for all projects, but certain + projects [consume units of compute at a slower rate](#cost-factor). +- The base monthly compute quota for a GitLab.com [namespace](../../user/namespace/index.md) is determined by its [license tier](https://about.gitlab.com/pricing/). -- You can [purchase additional CI/CD minutes](#purchase-additional-cicd-minutes) - if you need more than the number of CI/CD minutes in your monthly quota. +- You can [purchase additional units of compute](#purchase-additional-units-of-compute) + if you need more than the amount of compute in your monthly quota. On self-managed GitLab instances: -- CI/CD minutes quotas are disabled by default. -- When enabled, CI/CD minutes quotas apply to private projects only. -- Administrators can [assign more CI/CD minutes](#set-the-quota-of-cicd-minutes-for-a-specific-namespace) - if a namespace uses all the CI/CD minutes in its monthly quota. +- Compute quotas are disabled by default. +- When enabled, compute quotas apply to private projects only. +- Administrators can [assign more units of compute](#set-the-compute-quota-for-a-specific-namespace) + if a namespace uses all its monthly quota. -[Project runners](../runners/runners_scope.md#project-runners) are not subject to a quota of CI/CD minutes. +[Project runners](../runners/runners_scope.md#project-runners) are not subject to a compute quota. -## Set the quota of CI/CD minutes for all namespaces +## Set the compute quota for all namespaces > [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. -By default, GitLab instances do not have a quota of CI/CD minutes. -The default value for the quota is `0`, which grants unlimited CI/CD minutes. +By default, GitLab instances do not have a compute quota. +The default value for the quota is `0`, which is unlimited. However, you can change this default value. Prerequisite: @@ -52,41 +54,43 @@ Prerequisite: To change the default quota that applies to all namespaces: -1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Continuous Integration and Deployment**. -1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. +1. In the **Compute quota** box, enter a limit. 1. Select **Save changes**. If a quota is already defined for a specific namespace, this value does not change that quota. -## Set the quota of CI/CD minutes for a specific namespace +## Set the compute quota for a specific namespace > [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. -You can override the global value and set a quota of CI/CD minutes +You can override the global value and set a compute quota for a specific namespace. Prerequisite: - You must be a GitLab administrator. -To set a quota of CI/CD minutes for a namespace: +To set a compute quota for a namespace: -1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. 1. On the left sidebar, select **Overview > Groups**. 1. For the group you want to update, select **Edit**. -1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. +1. In the **Compute quota** box, enter the maximum number of units of compute. 1. Select **Save changes**. You can also use the [update group API](../../api/groups.md#update-group) or the [update user API](../../api/users.md#user-modification) instead. NOTE: -You can set a quota of CI/CD minutes for only top-level groups or user namespaces. +You can set a compute quota for only top-level groups or user namespaces. If you set a quota for a subgroup, it is not used. -## View CI/CD minutes +## View compute usage Prerequisite: @@ -101,17 +105,16 @@ Prerequisite: - You must have the Owner role for the group. -To view CI/CD minutes being used for your group: +To view compute usage for your group: -1. On the top bar, select **Main menu > Groups** and find your group. The group must not be a subgroup. -1. On the left sidebar, select **Settings > Usage Quotas**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to + find your group. The group must not be a subgroup. +1. Select **Settings > Usage Quotas**. 1. Select the **Pipelines** tab. - - -The projects list shows projects with CI/CD minute usage or shared runners usage +The projects list shows projects with compute usage or shared runners usage in the current month only. The list includes all projects in the namespace and its -subgroups, sorted in descending order of CI/CD minute usage. +subgroups, sorted in descending order of compute usage. ### View Usage Quota reports for a personal namespace @@ -121,79 +124,79 @@ Prerequisite: - The namespace must be your personal namespace. -You can view the number of CI/CD minutes being used by a personal namespace: +You can view the compute usage for a personal namespace: -1. On the top bar, in the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. On the left sidebar, select **Usage Quotas**. The projects list shows [personal projects](../../user/project/working_with_projects.md#view-personal-projects) -with CI/CD minutes usage or shared runners usage in the current month only. The list -is sorted in descending order of CI/CD minute usage. +with compute usage or shared runners usage in the current month only. The list +is sorted in descending order of compute usage. -## Purchase additional CI/CD minutes **(FREE SAAS)** +## Purchase additional units of compute **(FREE SAAS)** -If you're using GitLab SaaS, you can purchase additional packs of CI/CD minutes. -These additional CI/CD minutes: +If you're using GitLab SaaS, you can purchase additional packs of units of compute. +These additional units of compute: - Are used only after the monthly quota included in your subscription runs out. - Are carried over to the next month, if any remain at the end of the month. -- Are valid for 12 months from date of purchase or until all minutes are consumed, whichever comes first. Expiry of minutes is not enforced. +- Are valid for 12 months from date of purchase or until all units of compute are consumed, whichever comes first. Expiry of units of compute is not enforced. For example, with a GitLab SaaS Premium license: -- You have `10,000` monthly minutes. -- You purchase an additional `5,000` minutes. -- Your total limit is `15,000` minutes. +- You have `10,000` monthly units of compute. +- You purchase an additional `5,000` units of compute. +- Your total limit is `15,000` units of compute. -If you use `13,000` minutes during the month, the next month your additional minutes become -`2,000`. If you use `9,000` minutes during the month, your additional minutes remain the same. +If you use `13,000` units of compute during the month, the next month your additional units of compute become +`2,000`. If you use `9,000` units of compute during the month, your additional units of compute remain the same. -If you bought additional CI/CD minutes while on a trial subscription, those minutes are available after the trial ends or you upgrade to a paid plan. +If you bought additional units of compute while on a trial subscription, those units of compute are available after the trial ends or you upgrade to a paid plan. -You can find pricing for additional CI/CD minutes on the +You can find pricing for additional units of compute on the [GitLab Pricing page](https://about.gitlab.com/pricing/). -### Purchase CI/CD minutes for a group **(FREE SAAS)** +### Purchase units of compute for a group **(FREE SAAS)** Prerequisite: - You must have the Owner role for the group. -You can purchase additional CI/CD minutes for your group. -You cannot transfer purchased CI/CD minutes from one group to another, +You can purchase additional units of compute for your group. +You cannot transfer purchased units of compute from one group to another, so be sure to select the correct group. -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Settings > Usage Quotas**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Settings > Usage Quotas**. 1. Select **Pipelines**. -1. Select **Buy additional minutes**. +1. Select **Buy additional units of compute**. 1. Complete the details of the transaction. -After your payment is processed, the additional CI/CD minutes are added to your group +After your payment is processed, the additional units of compute are added to your group namespace. -### Purchase CI/CD minutes for a personal namespace **(FREE SAAS)** +### Purchase units of compute for a personal namespace **(FREE SAAS)** Prerequisite: - The namespace must be your personal namespace. -To purchase additional minutes for your personal namespace: +To purchase additional units of compute for your personal namespace: -1. On the top bar, in the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. On the left sidebar, select **Usage Quotas**. -1. Select **Buy additional minutes**. GitLab redirects you to the Customers Portal. -1. Locate the subscription card that's linked to your personal namespace on GitLab SaaS, select **Buy more CI minutes**, +1. Select **Buy additional units of compute**. GitLab redirects you to the Customers Portal. +1. Locate the subscription card that's linked to your personal namespace on GitLab SaaS, select **Buy more units of compute**, and complete the details of the transaction. -After your payment is processed, the additional CI/CD minutes are added to your personal +After your payment is processed, the additional units of compute are added to your personal namespace. -## How CI/CD minute usage is calculated +## How compute usage is calculated -GitLab uses this formula to calculate the CI/CD minute usage of a job: +GitLab uses this formula to calculate the compute usage of a job: ```plaintext Job duration * Cost factor @@ -203,18 +206,18 @@ Job duration * Cost factor not including time spent in the `created` or `pending` statuses. - [**Cost factor**](#cost-factor): A number based on project visibility. -The value is transformed into minutes and added to the count of used CI/CD minutes +The value is transformed into units of compute and added to the count of used units in the job's top-level namespace. For example, if a user `alice` runs a pipeline: -- Under the `gitlab-org` namespace, the CI/CD minutes used by each job in the pipeline are +- Under the `gitlab-org` namespace, the units of compute used by each job in the pipeline are added to the overall consumption for the `gitlab-org` namespace, not the `alice` namespace. -- For one of the personal projects in their namespace, the CI/CD minutes are added +- For one of the personal projects in their namespace, the units of compute are added to the overall consumption for the `alice` namespace. -The CI/CD minutes used by one pipeline is the total CI/CD minutes used by all the jobs -that ran in the pipeline. Jobs can run concurrently, so the total CI/CD minutes usage +The compute used by one pipeline is the total units of compute used by all the jobs +that ran in the pipeline. Jobs can run concurrently, so the total compute usage can be higher than the end-to-end duration of a pipeline. ### Cost factor @@ -225,24 +228,24 @@ The cost factors for jobs running on shared runners on GitLab.com are: - Exceptions for public projects: - `0.5` for projects in the [GitLab for Open Source program](../../subscriptions/community_programs.md#gitlab-for-open-source). - `0.008` for forks of projects in the [GitLab for Open Source program](../../subscriptions/community_programs.md#gitlab-for-open-source). For every 125 minutes of job execution time, - you use 1 CI/CD minute. + you use 1 unit of compute. - Discounted dynamically for [community contributions to GitLab projects](#cost-factor-for-community-contributions-to-gitlab-projects). The cost factors on self-managed instances are: -- `0` for public projects, so they do not consume CI/CD minutes. +- `0` for public projects, so they do not consume units of compute. - `1` for internal and private projects. #### Cost factor for community contributions to GitLab projects Community contributors can use up to 300,000 minutes on shared runners when contributing to open source projects maintained by GitLab. The maximum of 300,000 minutes would only be possible if contributing exclusively to projects [part of the GitLab product](https://about.gitlab.com/handbook/engineering/metrics/#projects-that-are-part-of-the-product). The total number of minutes available on shared runners -is reduced by the CI/CD minutes used by pipelines from other projects. +is reduced by the units of compute used by pipelines from other projects. The 300,000 minutes applies to all SaaS tiers, and the cost factor calculation is: -- `Monthly minute quota / 300,000 job duration minutes = Cost factor` +- `Monthly compute quota / 300,000 job duration minutes = Cost factor` -For example, with the 10,000 CI/CD minutes per month in the Premium tier: +For example, with a monthly compute quota of 10,000 in the Premium tier: - 10,000 / 300,000 = 0.03333333333 cost factor. @@ -261,44 +264,46 @@ GitLab administrators can add a namespace to the reduced cost factor GitLab SaaS runners have different cost factors, depending on the runner type (Linux, Windows, macOS) and the virtual machine configuration. -| GitLab SaaS runner type | Machine Type | CI/CD minutes cost factor | -| :--------- | :------------------- | :--------- | -| Linux OS | Small |1| -| Linux OS | Medium |2| -| Linux OS | Large |3| -| Linux OS + GPU-enabled | Medium, GPU Standard |7| +| GitLab SaaS runner type | Machine Size | Cost factor | +|:-----------------------------|:---------------------|:------------| +| Linux OS amd64 | small | 1 | +| Linux OS amd64 | medium | 2 | +| Linux OS amd64 | large | 3 | +| Linux OS amd64 + GPU-enabled | medium, GPU standard | 7 | +| macOS M1 | Medium | 6 | +| Windows Server | - | 1 (Beta) | -### Monthly reset of CI/CD minutes +### Monthly reset of compute usage -On the first day of each calendar month, the accumulated usage of CI/CD minutes is reset to `0` +On the first day of each calendar month, the accumulated compute usage is reset to `0` for all namespaces that use shared runners. This means your full quota is available, and calculations start again from `0`. -For example, if you have a monthly quota of `10,000` CI/CD minutes: +For example, if you have a monthly quota of `10,000` units of compute: -- On **April 1**, you have `10,000` minutes. -- During April, you use only `6,000` of the `10,000` minutes. -- On **May 1**, the accumulated usage of minutes resets to `0`, and you have `10,000` minutes to use again +- On **April 1**, you have `10,000` units of compute. +- During April, you use only `6,000` of the `10,000` units of compute. +- On **May 1**, the accumulated compute usage resets to `0`, and you have `10,000` units of compute to use again during May. Usage data for the previous month is kept to show historical view of the consumption over time. -### Monthly rollover of purchased CI/CD minutes +### Monthly rollover of purchased units of compute -If you purchase additional CI/CD minutes and don't use the full amount, the remaining amount rolls over to +If you purchase additional units of compute and don't use the full amount, the remaining amount rolls over to the next month. For example: -- On **April 1**, you purchase `5,000` additional CI/CD minutes. -- During April, you use only `3,000` of the `5,000` additional minutes. -- On **May 1**, the unused minute roll over, so you have `2,000` additional minutes available for May. +- On **April 1**, you purchase `5,000` additional units of compute. +- During April, you use only `3,000` of the `5,000` additional units of compute. +- On **May 1**, the unused units of compute roll over, so you have `2,000` additional units of compute available for May. -Additional CI/CD minutes are a one-time purchase and do not renew or refresh each month. +Additional units of compute are a one-time purchase and do not renew or refresh each month. ## What happens when you exceed the quota -When the quota of CI/CD minutes is used for the current month, GitLab stops +When the compute quota is used for the current month, GitLab stops processing new jobs. - Any non-running job that should be picked by shared runners is automatically dropped. @@ -306,29 +311,29 @@ processing new jobs. - Any running job can be dropped at any point if the overall namespace usage goes over-quota by a grace period. -The grace period for running jobs is `1,000` CI/CD minutes. +The grace period for running jobs is `1,000` units of compute. -Jobs on project runners are not affected by the quota of CI/CD minutes. +Jobs on project runners are not affected by the compute quota. ### GitLab SaaS usage notifications On GitLab SaaS an email notification is sent to the namespace owners when: -- The available CI/CD minutes are below 30% of the quota. -- The available CI/CD minutes are below 5% of the quota. -- All CI/CD minutes have been used. +- The remaining units of compute is below 30% of the quota. +- The remaining units of compute is below 5% of the quota. +- All the compute quota has been used. ### Special quota limits In some cases, the quota limit is replaced by one of the following labels: -- **Unlimited minutes**: For namespaces with unlimited CI/CD minutes -- **Not supported minutes**: For namespaces where active shared runners are not enabled +- **Unlimited**: For namespaces with unlimited compute quota. +- **Not supported**: For namespaces where active shared runners are not enabled. -## Reduce consumption of CI/CD minutes +## Reduce compute quota usage -If your project consumes too many CI/CD minutes, there are some strategies you can -use to reduce your CI/CD minutes usage: +If your project consumes too much compute quota, there are some strategies you can +use to reduce your usage: - If you are using project mirrors, ensure that [pipelines for mirror updates](../../user/project/repository/mirror/pull.md#trigger-pipelines-for-mirror-updates) is disabled. @@ -342,23 +347,23 @@ use to reduce your CI/CD minutes usage: - If you are working from a fork and you submit a merge request to the parent project, you can ask a maintainer to run a pipeline [in the parent project](merge_request_pipelines.md#run-pipelines-in-the-parent-project). -If you manage an open source project, these improvements can also reduce CI/CD minutes +If you manage an open source project, these improvements can also reduce compute quota consumption for contributor fork projects, enabling more contributions. See our [pipeline efficiency guide](pipeline_efficiency.md) for more details. -## Reset CI/CD minutes used **(PREMIUM SELF)** +## Reset compute usage **(PREMIUM SELF)** -An administrator can reset the number of minutes used by a namespace for the current month. +An administrator can reset the compute usage for a namespace for the current month. -### Reset minutes for a personal namespace +### Reset usage for a personal namespace 1. Find the [user in the admin area](../../user/admin_area/index.md#administering-users). 1. Select **Edit**. -1. In **Limits**, select **Reset pipeline minutes**. +1. In **Limits**, select **Reset compute usage**. -### Reset minutes for a group namespace +### Reset usage for a group namespace 1. Find the [group in the admin area](../../user/admin_area/index.md#administering-groups). 1. Select **Edit**. -1. In **Permissions and group features**, select **Reset pipeline minutes**. +1. In **Permissions and group features**, select **Reset compute usage**. diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md index ffcfee98f05..fdc03daf7ad 100644 --- a/doc/ci/pipelines/downstream_pipelines.md +++ b/doc/ci/pipelines/downstream_pipelines.md @@ -389,7 +389,7 @@ to the right of the [pipeline graph](index.md#visualize-pipelines). In [pipeline mini graphs](index.md#pipeline-mini-graphs), the downstream pipeline displays to the right of the mini graph. -## Fetch artifacts from an upstream pipeline +## Fetch artifacts from an upstream pipeline **(PREMIUM)** Use [`needs:project`](../yaml/index.md#needsproject) to fetch artifacts from an upstream pipeline: @@ -661,6 +661,12 @@ For example, in a [multi-project pipeline](#multi-project-pipelines): artifacts: true ``` +### Control what type of variables to forward to downstream pipelines + +Use the [`trigger:forward` keyword](../yaml/index.md#triggerforward) to specify +what type of variables to forward to the downstream pipeline. Forwarded variables +are considered trigger variables, which have the [highest precedence](../variables/index.md#cicd-variable-precedence). + ## Troubleshooting ### Trigger job fails and does not create multi-project pipeline diff --git a/doc/ci/pipelines/img/group_cicd_minutes_quota.png b/doc/ci/pipelines/img/group_cicd_minutes_quota.png Binary files differdeleted file mode 100644 index 318527426bd..00000000000 --- a/doc/ci/pipelines/img/group_cicd_minutes_quota.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index b0c5f3a6a69..c99df5b15e6 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -109,10 +109,9 @@ Select a pipeline to open the **Pipeline Details** page and show the jobs that were run for that pipeline. From here you can cancel a running pipeline, retry jobs on a failed pipeline, or [delete a pipeline](#delete-a-pipeline). -[Starting in GitLab 12.3](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/50499), a link to the -latest pipeline for the last commit of a given branch is available at `/project/pipelines/[branch]/latest`. -Also, `/project/pipelines/latest` redirects you to the latest pipeline for the last commit -on the project's default branch. +A link to the latest pipeline for the last commit of a given branch is available at +`/project/-/pipelines/[branch]/latest`. Also, `/project/-/pipelines/latest` redirects +you to the latest pipeline for the last commit on the project's default branch. [Starting in GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/215367), you can filter the pipeline list by: @@ -140,8 +139,8 @@ operation of the pipeline. To execute a pipeline manually: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **CI/CD > Pipelines**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Pipelines**. 1. Select **Run pipeline**. 1. In the **Run for branch name or tag** field, select the branch or tag to run the pipeline for. 1. Enter any [CI/CD variables](../variables/index.md) required for the pipeline to run. @@ -189,6 +188,11 @@ In this example: - `DEPLOY_ENVIRONMENT` is pre-filled in the **Run pipeline** page with `canary` as the default value, and the message explains the other options. +NOTE: +Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/382857), projects that use [compliance pipelines](../../user/group/compliance_frameworks.md#compliance-pipelines) can have prefilled variables not appear +when running a pipeline manually. To workaround this issue, +[change the compliance pipeline configuration](../../user/group/compliance_frameworks.md#prefilled-variables-are-not-shown). + #### Configure a list of selectable prefilled variable values > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363660) in GitLab 15.5 [with a flag](../../administration/feature_flags.md) named `run_pipeline_graphql`. Disabled by default. @@ -345,8 +349,8 @@ Prerequisites: To trigger the pipeline when the upstream project is rebuilt: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Pipeline subscriptions**. 1. Enter the project you want to subscribe to, in the format `<namespace>/<project>`. For example, if the project is `https://gitlab.com/gitlab-org/gitlab`, use `gitlab-org/gitlab`. diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index d42f35627a2..045fa8dc16c 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -168,9 +168,7 @@ When you use merge request pipelines, you can use: - All the same [predefined variables](../variables/predefined_variables.md) that are available in branch pipelines. - [Additional predefined variables](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) - available only to jobs in merge request pipelines. These variables contain - information from the associated merge request, which can be when calling the - [GitLab Merge Request API endpoint](../../api/merge_requests.md) from a job. + available only to jobs in merge request pipelines. ## Troubleshooting @@ -202,8 +200,8 @@ It's possible to have both branch pipelines and merge request pipelines in the **Pipelines** tab of a single merge request. This might be [by configuration](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines), or [by accident](#two-pipelines-when-pushing-to-a-branch). -When using the [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) -feature and both pipelines types are present, the merge request pipelines are checked, +When the project has [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) enabled +and both pipelines types are present, the merge request pipelines are checked, not the branch pipelines. Therefore, the MR pipeline result is marked as unsuccessful if the diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index af29c8105ee..7bff13aa390 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -6,9 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Merge trains **(PREMIUM)** -FLAG: -In GitLab 15.11 and later, the **Start merge train** button is **Set to auto-merge** and the **Add to merge train** button is **Merge**. On self-managed GitLab, by default these changes are not available. To make them available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `auto_merge_labels_mr_widget`. On GitLab.com, this feature is not available. +NOTE: +[In GitLab 16.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/359057), the **Start merge train** +and **Start merge train when pipeline succeeds** buttons became **Set to auto-merge**. +**Remove from merge train** became **Cancel auto-merge**. Use merge trains to queue merge requests and verify their changes work together before they are merged to the target branch. @@ -29,14 +30,14 @@ For more information about: ## Merge train workflow A merge train starts when there are no merge requests waiting to merge and you -select [**Start merge train**](#start-a-merge-train). GitLab starts a merge train pipeline +select [**Merge**](#start-a-merge-train). GitLab starts a merge train pipeline that verifies that the changes can merge into the default branch. This first pipeline is the same as a [merged results pipeline](merged_results_pipelines.md), which runs on the changes of the source and target branches combined together. The author of the internal merged result commit is the user that initiated the merge. To queue a second merge request to merge immediately after the first pipeline completes, select -[**Add to merge train**](#add-a-merge-request-to-a-merge-train) and add it to the train. +[**Set to auto-merge**](#add-a-merge-request-to-a-merge-train) to add it to the train. This second merge train pipeline runs on the changes of _both_ merge requests combined with the target branch. Similarly, if you add a third merge request, that pipeline runs on the changes of all three merge requests merged with the target branch. The pipelines all run in parallel. @@ -100,8 +101,8 @@ Prerequisites: To enable merge trains: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Merge requests**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > Merge requests**. 1. In the **Merge method** section, verify that **Merge commit** is selected. 1. In the **Merge options** section: - In GitLab 13.6 and later, select **Enable merged results pipelines** and **Enable merge trains**. @@ -120,8 +121,8 @@ To start a merge train: 1. Visit a merge request. 1. Select: - - When no pipeline is running, **Start merge train**. - - When a pipeline is running, **Start merge train when pipeline succeeds**. + - When no pipeline is running, **Merge**. + - When a pipeline is running, **Set to auto-merge**. The merge request's merge train status displays under the pipeline widget with a message similar to `A new merge train has started and this merge request is the first of the queue.` @@ -138,8 +139,8 @@ To add a merge request to a merge train: 1. Visit a merge request. 1. Select: - - When no pipeline is running, **Add to merge train**. - - When a pipeline is running, **Add to merge train when pipeline succeeds**. + - When no pipeline is running, **Merge**. + - When a pipeline is running, **Set to auto-merge**. The merge request's merge train status displays under the pipeline widget with a message similar to `Added to the merge train. There are 2 merge requests waiting to be merged.` @@ -151,7 +152,7 @@ waiting to join the merge train. ## Remove a merge request from a merge train -To remove a merge request from a merge train, select **Remove from merge train**. +To remove a merge request from a merge train, select **Cancel auto-merge**. You can add the merge request to a merge train again later. When you remove a merge request from a merge train: @@ -191,8 +192,7 @@ can enable the feature flag to disable merge trains: Feature.enable(:disable_merge_trains) ``` -After you enable this feature flag, GitLab cancels existing merge trains and removes -the **Start/Add to merge train** option from merge requests. +After you enable this feature flag, GitLab cancels existing merge trains. To disable the feature flag, which enables merge trains again: @@ -209,18 +209,18 @@ the merge train drops your merge request automatically. For example, this could - Changing the merge request to a [draft](../../user/project/merge_requests/drafts.md). - A merge conflict. -- A new conversation thread that is unresolved, when [all threads must be resolved](../../user/discussions/index.md#prevent-merge-unless-all-threads-are-resolved) +- A new conversation thread that is unresolved, when [all threads must be resolved](../../user/project/merge_requests/index.md#prevent-merge-unless-all-threads-are-resolved) is enabled. You can find reason the merge request was dropped from the merge train in the system notes. Check the **Activity** section in the **Overview** tab for a message similar to: `User removed this merge request from the merge train because ...` -### Cannot use merge when pipeline succeeds +### Cannot use auto-merge -You cannot use [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) -when merge trains are enabled. See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/12267) -for more information. +You cannot use [auto-merge](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) +(formerly **Merge when pipeline succeeds**) to skip the merge train, when merge trains are enabled. +See [issue 12267](https://gitlab.com/gitlab-org/gitlab/-/issues/12267) for more information. ### Cannot retry merge train pipeline cannot @@ -241,7 +241,7 @@ You can: When [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge) is enabled, but the latest pipeline failed: -- The **Start/Add to merge train** option is not available. +- The **Set to auto-merge** or **Merge** options are not available. - The merge request displays `The pipeline for this merge request failed. Please retry the job or push a new commit to fix the failure.` Before you can re-add a merge request to a merge train, you can try to: diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md index 316d7819f55..16120924f37 100644 --- a/doc/ci/pipelines/merged_results_pipelines.md +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -41,8 +41,8 @@ To use merged results pipelines: To enable merged results pipelines in a project, you must have at least the Maintainer role: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Merge requests**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > Merge requests**. 1. In the **Merge options** section, select **Enable merged results pipelines**. 1. Select **Save changes**. diff --git a/doc/ci/pipelines/pipeline_artifacts.md b/doc/ci/pipelines/pipeline_artifacts.md index d8db79a54dc..79435c0276d 100644 --- a/doc/ci/pipelines/pipeline_artifacts.md +++ b/doc/ci/pipelines/pipeline_artifacts.md @@ -2,28 +2,13 @@ stage: Verify group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../testing/test_coverage_visualization.md' +remove_date: '2023-08-31' --- -# Pipeline artifacts **(FREE)** +This document was moved to [another location](../testing/test_coverage_visualization.md). -Pipeline artifacts are files created by GitLab after a pipeline finishes. Pipeline artifacts are -different to [job artifacts](../jobs/job_artifacts.md) because they are not explicitly managed by -`.gitlab-ci.yml` definitions. - -Pipeline artifacts are used by the [test coverage visualization feature](../testing/test_coverage_visualization.md) -to collect coverage information. - -## Storage - -Pipeline artifacts are saved to disk or object storage. They count towards a project's [storage usage quota](../../user/usage_quotas.md#storage-usage-quota). -The **Artifacts** on the Usage Quotas page is the sum of all job artifacts and pipeline artifacts. - -## When pipeline artifacts are deleted - -Pipeline artifacts from: - -- The latest pipeline are kept forever. -- Pipelines superseded by a newer pipeline are deleted seven days after their creation date. - -It can take up to two days for GitLab to delete pipeline artifacts from when they are due to be -deleted. +<!-- This redirect file can be deleted after <2023-08-31>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/pipelines/pipeline_security.md b/doc/ci/pipelines/pipeline_security.md new file mode 100644 index 00000000000..f035779e665 --- /dev/null +++ b/doc/ci/pipelines/pipeline_security.md @@ -0,0 +1,48 @@ +--- +stage: Verify +group: Pipeline Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Pipeline security + +## Secrets Management + +Secrets management is the systems that developers use to securely store sensitive data +in a secure environment with strict access controls. A **secret** is a sensitive credential +that should be kept confidential, and includes: + +- Passwords +- SSH keys +- Access tokens +- Other types of credentials + +## Secrets storage + +### Secrets management providers + +Secrets that are the most sensitive and under the strictest policies should be stored +in a separate secrets management provider such as [Vault](https://www.vaultproject.io). +The secrets are stored outside of the GitLab instance, which is the safest option. + +You can use the GitLab [Vault integration](../secrets/index.md#use-vault-secrets-in-a-ci-job) +to retrieve those secrets in CI/CD pipelines when they are needed. + +### CI/CD variables + +[CI/CD Variables](../variables/index.md) are a convenient way to store and use data +in a CI/CD pipeline, but variables are less secure than secrets management providers. +Variable values: + +- Are stored in the GitLab project, group, or instance settings. Users with access + to the settings have access to the variables. +- Can be [overridden](../variables/index.md#override-a-defined-cicd-variable), + making it hard to determine which value was used. +- Are more easily exposed by accidental pipeline misconfiguration. + +Sensitive data should be stored in a secrets management solution. If there is low +sensitivity data that you want to store in a CI/CD variable, be sure to always: + +- [Mask the variables](../variables/index.md#mask-a-cicd-variable). +- [Protect the variables](../variables/index.md#protect-a-cicd-variable) when possible. diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 3ff748644cf..49b129f11aa 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -14,7 +14,7 @@ Use scheduled pipelines to run GitLab CI/CD [pipelines](index.md) at regular int For a scheduled pipeline to run: - The schedule owner must have the Developer role. For pipelines on protected branches, - the schedule owner must be [allowed to merge](../../user/project/protected_branches.md#configure-a-protected-branch) + the schedule owner must be [allowed to merge](../../user/project/protected_branches.md#add-protection-to-existing-branches) to the branch. - The [CI/CD configuration](../yaml/index.md) must be valid. @@ -26,8 +26,8 @@ Otherwise, the pipeline is not created. No error message is displayed. To add a pipeline schedule: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **CI/CD > Schedules**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Pipeline schedules**. 1. Select **New schedule** and fill in the form. - **Interval Pattern**: Select one of the preconfigured intervals, or enter a custom interval in [cron notation](../../topics/cron/index.md). You can use any cron value, @@ -47,8 +47,8 @@ you must delete unused schedules before you can add another. The owner of a pipeline schedule can edit it: -1. On the top bar, select **Main menu > Projects** and find your project. -1. In the left sidebar, select **CI/CD > Schedules**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Pipeline schedules**. 1. Next to the schedule, select **Edit** (**{pencil}**) and fill in the form. The user must have the Developer role or above for the project. If the user is @@ -60,8 +60,8 @@ of the schedule. To trigger a pipeline schedule manually, so that it runs immediately instead of the next scheduled time: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **CI/CD > Schedules**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. On the left sidebar, select **Build > Pipeline schedules**. 1. On the right of the list, for the pipeline you want to run, select **Play** (**{play}**). @@ -76,11 +76,13 @@ including [protected environments](../environments/protected_environments.md) an To take ownership of a pipeline created by a different user: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **CI/CD > Schedules**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. On the left sidebar, select **Build > Pipeline schedules**. 1. On the right of the list, for the pipeline you want to become owner of, select **Take ownership**. +You need at least the Maintainer role to take ownership of a pipeline created by a different user. + ## Related topics - [Pipeline schedules API](../../api/pipeline_schedules.md) diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 3e9e6c50f64..38cdc5ed578 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -24,8 +24,8 @@ For public and internal projects, you can change who can see your: To change the visibility of your pipelines and related features: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Select or clear the **Public pipelines** checkbox. When it is selected, pipelines and related features are visible: @@ -56,8 +56,8 @@ This setting has no effect when: To change the pipeline visibility for non-project members: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > General**. 1. Expand **Visibility, project features, permissions**. 1. For **CI/CD**, choose: - **Only project members**: Only project members can view pipelines. @@ -72,8 +72,8 @@ is selected. You can set pending or running pipelines to cancel automatically when a pipeline for new changes runs on the same branch. You can enable this in the project settings: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **General Pipelines**. 1. Select the **Auto-cancel redundant pipelines** checkbox. 1. Select **Save changes**. @@ -94,8 +94,8 @@ newer one, which may not be what you want. To avoid this scenario: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Select the **Prevent outdated deployment jobs** checkbox. 1. Select **Save changes**. @@ -111,8 +111,8 @@ directory. However, you can specify an alternate filename path, including locati To customize the path: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **CI/CD configuration file** field, enter the filename. If the file: - Is not in the root directory, include the path. @@ -160,8 +160,8 @@ able to edit it. You can choose how your repository is fetched from GitLab when a job runs. -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Under **Git strategy**, select an option: - `git clone` is slower because it clones the repository from scratch @@ -181,8 +181,8 @@ in the `.gitlab-ci.yml` file. You can limit the number of changes that GitLab CI/CD fetches when it clones a repository. -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. Under **Git strategy**, under **Git shallow clone**, enter a value. The maximum value is `1000`. To disable shallow clone and make GitLab CI/CD @@ -198,8 +198,8 @@ in the `.gitlab-ci.yml` file. You can define how long a job can run before it times out. -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **Timeout** field, enter the number of minutes, or a human-readable value like `2 hours`. Must be 10 minutes or more, and less than one month. Default is 60 minutes. diff --git a/doc/ci/quick_start/tutorial.md b/doc/ci/quick_start/tutorial.md index 88d35bf56b0..acc47a07a02 100644 --- a/doc/ci/quick_start/tutorial.md +++ b/doc/ci/quick_start/tutorial.md @@ -36,7 +36,8 @@ Before adding the pipeline configuration, you must first set up a Docusaurus pro on GitLab.com: 1. Create a new project under your username (not a group): - 1. On the top bar, select **Main menu > Projects > View all projects**. + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **View all your projects**. 1. On the right of the page, select **New project**. 1. Select **Create blank project**. 1. Enter the project details: diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md index bae1bd9712b..74e3e493adb 100644 --- a/doc/ci/resource_groups/index.md +++ b/doc/ci/resource_groups/index.md @@ -75,7 +75,7 @@ The following modes are supported: - **Unordered:** This is the default process mode that limits the concurrency on running jobs. It's the easiest option to use when you don't care about the execution order - of the jobs. It starts processing the jobs whenever a job ready to run. + of the jobs. It starts processing the jobs whenever a job is ready to run. - **Oldest first:** This process mode limits the concurrency of the jobs. When a resource is free, it picks the first job from the list of upcoming jobs (`created`, `scheduled`, or `waiting_for_resource` state) that are sorted by pipeline ID in ascending order. diff --git a/doc/ci/review_apps/img/review_apps_preview_in_mr.png b/doc/ci/review_apps/img/review_apps_preview_in_mr.png Binary files differdeleted file mode 100644 index 3e6506a6a3a..00000000000 --- a/doc/ci/review_apps/img/review_apps_preview_in_mr.png +++ /dev/null diff --git a/doc/ci/review_apps/img/review_apps_preview_in_mr_v16_0.png b/doc/ci/review_apps/img/review_apps_preview_in_mr_v16_0.png Binary files differnew file mode 100644 index 00000000000..40345b9987f --- /dev/null +++ b/doc/ci/review_apps/img/review_apps_preview_in_mr_v16_0.png diff --git a/doc/ci/review_apps/img/view_on_env_blob.png b/doc/ci/review_apps/img/view_on_env_blob.png Binary files differdeleted file mode 100644 index acc457fbb38..00000000000 --- a/doc/ci/review_apps/img/view_on_env_blob.png +++ /dev/null diff --git a/doc/ci/review_apps/img/view_on_env_mr.png b/doc/ci/review_apps/img/view_on_env_mr.png Binary files differdeleted file mode 100644 index 0e61814a65d..00000000000 --- a/doc/ci/review_apps/img/view_on_env_mr.png +++ /dev/null diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 004bd9dfc5f..ddb8f0aaa61 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -16,7 +16,7 @@ Review apps: - Provide an automatic live preview of changes made in a feature branch by spinning up a dynamic environment for your merge requests. - Allow designers and product managers to see your changes without needing to check out your branch and run your changes in a sandbox environment. -- Are fully integrated with the [GitLab DevOps LifeCycle](../../index.md#the-entire-devops-lifecycle). +- Are fully integrated with the [GitLab DevOps LifeCycle](https://about.gitlab.com/stages-devops-lifecycle/). - Allow you to deploy your changes wherever you want.  @@ -35,7 +35,7 @@ Access to the review app is made available as a link on the [merge request](../. The following is an example of a merge request with an environment set dynamically. - + In this example, a branch was: @@ -76,8 +76,9 @@ Prerequisite: To use the review apps template: -1. On the top bar, select **Main menu > Projects** and find the project you want to create a review app job for. -1. On the left sidebar, select **Deployments > Environments**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to + find the project you want to create a review app job for. +1. Select **Build > Environments**. 1. Select **Enable review apps**. 1. Copy the provided code snippet and paste it into your `.gitlab-ci.yml` file: @@ -185,13 +186,9 @@ After you have the route mapping set up, it takes effect in the following locati  -- In the diff for a comparison or commit. +- In the diff for a comparison or commit, by selecting **View** (**{external-link}**) next to the file. -  - -- In the blob file view. - -  +- In the blob file view, by selecting **View** (**{external-link}**) next to the file. <!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> ## Visual Reviews (deprecated) **(PREMIUM)** diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index cf4b95f511d..c365cc934db 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -161,7 +161,8 @@ Prerequisite: To determine the IP address of a shared runner: -1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. 1. On the left sidebar, select **CI/CD > Runners**. 1. Find the runner in the table and view the **IP Address** column. @@ -955,8 +956,8 @@ You can clean up group runners that have been inactive for more than three month Group runners are those that were created at the group level. -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn on the **Enable stale runner cleanup** toggle. @@ -999,8 +1000,13 @@ The version of GitLab Runner used by your runners should be To determine which runners need to be upgraded: 1. View the list of runners: - - For a group, on the top bar, select **Main menu > Groups**, find your group, and on the left sidebar select **CI/CD > Runners**. - - For the instance, select **Main menu > Admin** and on the left sidebar, select **Runners**. + - For a group: + 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. + 1. Select **Build > Runners**. + - For the instance: + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. Select **CI/CD > Runners**. 1. Above the list of runners, view the status: - **Outdated - recommended**: The runner does not have the latest `PATCH` version, which may make it vulnerable @@ -1055,7 +1061,8 @@ Prerequisites: To automatically rotate runner authentication tokens: -1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**.. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Continuous Integration and Deployment** 1. Set a **Runners expiration** time for runners, leave empty for no expiration. diff --git a/doc/ci/runners/img/build_isolation.png b/doc/ci/runners/img/build_isolation.png Binary files differindex a363ef4709b..ca920999bbf 100644 --- a/doc/ci/runners/img/build_isolation.png +++ b/doc/ci/runners/img/build_isolation.png diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index 6883f0727dd..19c5be88c1b 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -7,34 +7,63 @@ type: reference # Runner SaaS **(FREE SAAS)** -If you use GitLab SaaS (GitLab.com), your [untagged](../../ci/runners/configure_runners.md#use-tags-to-control-which-jobs-a-runner-can-run) CI jobs automatically run in containers on the Linux Runners. -As long as shared runners are enabled for your project, no configuration is required. Your jobs can run on: +You can run your CI/CD jobs on GitLab.com using SaaS runners hosted by GitLab to seamlessly build, test and deploy +your application on different environments. +These runners fully integrated with GitLab.com and are enabled by default for all projects, with no configuration required. +Your jobs can run on: -- [Linux runners](saas/linux_saas_runner.md). -- [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta)). -- [macOS runners](saas/macos_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta)). +- [Linux runners](saas/linux_saas_runner.md) +- [GPU runners](saas/gpu_saas_runner.md) +- [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/experiment-beta-support.md#beta)) +- [macOS runners](saas/macos_saas_runner.md) ([Beta](../../policy/experiment-beta-support.md#beta)) -The number of minutes you can use on these runners depends on the -[maximum number of CI/CD minutes](../pipelines/cicd_minutes.md) +Refer to the Compute [cost factor](../../ci/pipelines/cicd_minutes.md#cost-factor) for the cost factor applied to the machine type based on size. +The number of minutes you can use on these runners depends on the [maximum number of units of compute](../pipelines/cicd_minutes.md) in your [subscription plan](https://about.gitlab.com/pricing/). -## Security for GitLab SaaS runners +[Untagged](../../ci/runners/configure_runners.md#use-tags-to-control-which-jobs-a-runner-can-run) jobs automatically run in containers +on the `small` Linux runners. -GitLab SaaS runners on Linux and Windows run on Google Compute Platform. The [Google Infrastructure Security Design Overview whitepaper](https://cloud.google.com/docs/security/infrastructure/design/resources/google_infrastructure_whitepaper_fa.pdf) provides an overview of how Google designs security into its technical infrastructure. The GitLab [Trust Center](https://about.gitlab.com/security/) and [GitLab Security Compliance Controls](https://about.staging.gitlab.com/handbook/engineering/security/security-assurance/security-compliance/sec-controls.html) pages provide an overview of the security and compliance controls that govern the GitLab SaaS runners. +The objective is to make 90% of CI jobs start executing in 120 seconds or less. The error rate should be less than 0.5%. -The runner that serves as a runner manager automatically initiates the creation and deletion of the virtual machines (VMs) used for CI jobs. When the runner manager picks up a GitLab SaaS CI job, it automatically executes that job on a new VM. There is no human or manual intervention in this process. The following section provides an overview of the additional built-in layers that harden the security of the GitLab Runner SaaS CI build environment. +## How SaaS runners work -### Security of CI job execution on GitLab Runner SaaS (Linux, Windows) +When you use SaaS runners: + +- Each of your jobs runs in a newly provisioned VM, which is dedicated to the specific job. +- The VM is active only for the duration of the job and immediately deleted. This means that any changes that your job makes to the virtual machine will not be available to a subsequent job. +- The virtual machine where your job runs has `sudo` access with no password. +- The storage is shared by the operating system, the image with pre-installed software, and a copy of your cloned repository. +This means that the available free disk space for your jobs to use is reduced. + +NOTE: +Jobs handled by SaaS runners on GitLab.com **time out after 3 hours**, regardless of the timeout configured in a project. + +## Security for SaaS runners + +GitLab SaaS runners on Linux and Windows run on Google Compute Platform. +The [Google Infrastructure Security Design Overview whitepaper](https://cloud.google.com/docs/security/infrastructure/design/resources/google_infrastructure_whitepaper_fa.pdf) +provides an overview of how Google designs security into its technical infrastructure. +The GitLab [Trust Center](https://about.gitlab.com/security/) and +[GitLab Security Compliance Controls](https://about.staging.gitlab.com/handbook/engineering/security/security-assurance/security-compliance/sec-controls.html) +pages provide an overview of the security and compliance controls that govern the GitLab SaaS runners. + +The following section provides an overview of the additional built-in layers that harden the security of the GitLab Runner SaaS CI build environment. + +### Security of CI job execution A dedicated temporary runner VM hosts and runs each CI job. On GitLab SaaS, two CI jobs never run on the same VM. +In this example, there are three jobs in the project's pipeline. Therefore, there are three temporary VMs used to run that pipeline, or one VM per job. +  -In this example, there are three jobs in the project's pipeline. Therefore, there are three temporary VMs used to run that pipeline, or one VM per job. +The build job ran on `runner-ns46nmmj-project-43717858`, test job on `f131a6a2runner-new2m-od-project-43717858` and deploy job on `runner-tmand5m-project-43717858`. -GitLab sends the command to remove the temporary runner VM to the Google Compute API immediately after the CI job completes. The [Google Compute Engine hypervisor](https://cloud.google.com/blog/products/gcp/7-ways-we-harden-our-kvm-hypervisor-at-google-cloud-security-in-plaintext) takes over the task of securely deleting the virtual machine and associated data. +GitLab sends the command to remove the temporary runner VM to the Google Compute API immediately after the CI job completes. The [Google Compute Engine hypervisor](https://cloud.google.com/blog/products/gcp/7-ways-we-harden-our-kvm-hypervisor-at-google-cloud-security-in-plaintext) +takes over the task of securely deleting the virtual machine and associated data. -### Network security of CI job virtual machines on GitLab Runner SaaS (Linux, Windows) +### Network security of CI job VMs - Firewall rules only allow outbound communication from the temporary VM to the public internet. - Inbound communication from the public internet to the temporary VM is not allowed. diff --git a/doc/ci/runners/new_creation_workflow.md b/doc/ci/runners/new_creation_workflow.md new file mode 100644 index 00000000000..0c43fd21d1d --- /dev/null +++ b/doc/ci/runners/new_creation_workflow.md @@ -0,0 +1,199 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Migrating to the new runner registration workflow **(FREE)** + +DISCLAIMER: +This page contains information related to upcoming products, features, and functionality. +It is important to note that the information presented is for informational purposes only. +Please do not rely on this information for purchasing or planning purposes. +As with all projects, the items mentioned on this page are subject to change or delay. +The development, release, and timing of any products, features, or functionality remain at the +sole discretion of GitLab Inc. + +In GitLab 16.0, we introduced a new runner creation workflow that uses authentication tokens to register +runners. The legacy workflow that uses registration tokens is deprecated and will be removed in GitLab 17.0. + +For information about the current development status of the new workflow, see [epic 7663](https://gitlab.com/groups/gitlab-org/-/epics/7663). + +For information about the technical design and reasons for the new architecture, see [Next GitLab Runner Token Architecture](../../architecture/blueprints/runner_tokens/index.md). + +## Feedback + +If you experience problems or have concerns about the new runner registration workflow, +or if the following information is not sufficient, +you can let us know in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387993). + +## The new runner registration workflow + +For the new runner registration workflow, you: + +1. [Create a runner](register_runner.md) directly in the GitLab UI. +1. Receive an authentication token. +1. Use the authentication token instead of the registration token when you register + a runner with this configuration. Runner managers registered in multiple hosts appear + under the same runner in the GitLab UI, but with an identifying system ID. + +The new runner registration workflow has the following benefits: + +- Preserved ownership records for runners, and minimized impact on users. +- The addition of a unique system ID ensures that you can reuse the same authentication token across +multiple runners. For more information, see [Reusing a GitLab Runner configuration](https://docs.gitlab.com/runner/fleet_scaling/#reusing-a-gitlab-runner-configuration). + +## Estimated time frame for planned changes + +- In GitLab 15.10 and later, you can use the new runner registration workflow. +- In GitLab 16.6, we plan to disable registration tokens. +- In GitLab 17.0, we plan to completely remove support for runner registration tokens. + +## Prevent your runner registration workflow from breaking + +Until GitLab 16.6, you can still use the legacy runner registration workflow. + +In GitLab 16.6, the legacy runner registration workflow will be disabled automatically. You will be able to manually re-enable the legacy runner registration workflow for a limited time. For more information, see +[Using registration tokens after GitLab 16.6](#using-registration-tokens-after-gitlab-166). + +If no action is taken before your GitLab instance is upgraded to GitLab 16.6, then your runner registration +workflow will break. + +To avoid a broken workflow, you must: + +1. [Create a shared runner](register_runner.md#for-a-shared-runner) and obtain the authentication token. +1. Replace the registration token in your runner registration workflow with the +authentication token. + +## Using registration tokens after GitLab 16.6 + +To continue using registration tokens after GitLab 16.6: + +- On GitLab.com, you can manually re-enable the legacy runner registration process in the top-level group settings until GitLab 16.8. +- On GitLab self-managed, you can manually re-enable the legacy runner registration process in the Admin Area settings until GitLab 17.0. + +Plans to implement a UI setting to re-enable registration tokens are proposed in [issue 411923](https://gitlab.com/gitlab-org/gitlab/-/issues/411923) + +## Changes to the `gitlab-runner register` command syntax + +The `gitlab-runner register` command will stop accepting registration tokens and instead accept new +authentication tokens generated in the GitLab runners administration page. +These authentication tokens are recognizable by their `glrt-` prefix. + +Example command for GitLab 15.9: + +```shell +gitlab-runner register + --non-interactive \ + --executor "shell" \ + --url "https://gitlab.com/" \ + --tag-list "shell,mac,gdk,test" \ + --run-untagged "false" \ + --locked "false" \ + --access-level "not_protected" \ + --registration-token "GR1348941C6YcZVddc8kjtdU-yWYD" +``` + +In GitLab 15.10 and later, you create the runner and some of the attributes in the UI, like the +tag list, locked status, and access level. +In GitLab 15.11 and later, these attributes are no longer accepted as arguments to `register`. + +The following example shows the new command: + +```shell +gitlab-runner register + --non-interactive \ + --executor "shell" \ + --url "https://gitlab.com/" \ + --token "glrt-2CR8_eVxiioB1QmzPZwa" +``` + +## Impact on autoscaling + +In autoscaling scenarios such as GitLab Runner Operator or GitLab Runner Helm Chart, the +registration token is replaced with the authentication token generated from the UI. +This means that the same runner configuration is reused across jobs, instead of creating a runner +for each job. +The specific runner can be identified by the unique system ID that is generated when the runner +process is started. + +## Impact on existing runners + +Existing runners will continue to work as usual. This change only affects registration of new runners. + +## Creating runners programmatically + +A new [POST /user/runners REST API](../../api/users.md#create-a-runner) was introduced in +GitLab 15.11, which allows a runner to be created in the context of an authenticated user. This should only be used in +scenarios where the runner configuration is dynamic, or not reusable. If the runner configuration is static, it is +preferable to reuse the authentication token of an existing runner. + +The following snippet shows how a group runner could be created and registered with a +[Group Access Token](../../user/group/settings/group_access_tokens.md) using the new creation flow. +The process is very similar when using [Project Access Tokens](../../user/project/settings/project_access_tokens.md) +or [Personal Access Tokens](../../user/profile/personal_access_tokens.md): + +```shell +# `GROUP_ID` contains the numerical ID of the group where the runner will be created +# `GITLAB_TOKEN` can be a Personal Access Token for a group owner, or a Group Access Token on the respective group +# created with `owner` access and `api` scope. +# +# The output will be parsed by `jq` to extract the token of the newly created runner +RUNNER_TOKEN=$(curl --silent --request POST "https://gitlab.com/api/v4/user/runners" \ + --header "private-token: $GITLAB_TOKEN" \ + --data runner_type=group_type --data group_id=$GROUP_ID --data 'description=My runner' --data 'tag_list=java,linux' \ + | jq -r '.token') + +gitlab-runner register \ + --non-interactive \ + --executor "shell" \ + --url "https://gitlab.com/" \ + --token "$RUNNER_TOKEN" +``` + +## Installing GitLab Runner with Helm chart + +Several runner configuration options cannot be set during runner registration. These options can only be configured: + +- When you create a runner in the UI. +- With the `user/runners` REST API endpoint. + +The following configuration options are no longer supported in [`values.yaml`](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/main/values.yaml): + +```yaml +## All these fields are DEPRECATED and the runner WILL FAIL TO START if you specify them +runnerRegistrationToken: "" +locked: true +tags: "" +maximumTimeout: "" +runUntagged: true +protected: true +``` + +If you store the runner token in `secrets`, you must also modify them. + +In the legacy runner registration workflow, fields were specified with: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: gitlab-runner-secret +type: Opaque +data: + runner-registration-token: "REDACTED" # DEPRECATED, set to "" + runner-token: "" +``` + +In the new runner registration workflow, you must use `runner-token` instead: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: gitlab-runner-secret +type: Opaque +data: + runner-registration-token: "" # need to leave as an empty string for compatibility reasons + runner-token: "REDACTED" +``` diff --git a/doc/ci/runners/register_runner.md b/doc/ci/runners/register_runner.md index f53297099d7..9c63850e38a 100644 --- a/doc/ci/runners/register_runner.md +++ b/doc/ci/runners/register_runner.md @@ -33,7 +33,8 @@ Prerequisites: To generate an authentication token for a shared runner: -1. On the top bar, select **Main menu > Admin**. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. 1. On the left sidebar, select **CI/CD > Runners**. 1. Select **New instance runner**. 1. Select a platform. @@ -41,6 +42,8 @@ To generate an authentication token for a shared runner: 1. Select **Submit**. 1. Follow the instructions to register the runner from the command line. +You can also [create a runner](../../api/users.md#create-a-runner) with the API to generate an authentication token. + ### For a group runner > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383143) in GitLab 15.10. Deployed behind the `create_runner_workflow_for_namespace` [flag](../../administration/feature_flags.md). Disabled by default. @@ -56,14 +59,16 @@ Prerequisites: To generate an authentication token for a group runner: -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **CI/CD > Runners**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Build > Runners**. 1. Select **New group runner**. 1. Select a platform. 1. Optional. Enter configurations for the runner. 1. Select **Submit**. 1. Follow the instructions to register the runner from the command line. +You can also [create a runner](../../api/users.md#create-a-runner) with the API to generate an authentication token. + ### For a project runner > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383143) in GitLab 15.10. Deployed behind the `create_runner_workflow_for_namespace` [flag](../../administration/feature_flags.md). Disabled by default. @@ -79,39 +84,45 @@ Prerequisites: To generate an authentication token for a project runner: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. +1. Expand the **Runners** section. 1. Select **New project runner**. 1. Select a platform. 1. Optional. Enter configurations for the runner. 1. Select **Submit**. 1. Follow the instructions to register the runner from the command line. +You can also [create a runner](../../api/users.md#create-a-runner) with the API to generate an authentication token. + ## Generate a registration token (deprecated) WARNING: The ability to pass a runner registration token, and support for certain configuration arguments was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6. Authentication tokens -should be used instead to register runners. Registration tokens, and support for certain configuration arguments -will be disabled behind a feature flag in GitLab 16.6 and removed in GitLab 17.0. The configuration arguments disabled for `glrt-` tokens are `--locked`, `--access-level`, `--run-untagged`, `--maximum-timeout`, `--paused`, `--tag-list`, and `--maintenance-note`. This change is a breaking +should be used instead to register runners. Registration tokens, and support for certain configuration +arguments, will be removed in GitLab 17.0. For more information, see [Migrating to the new runner registration workflow](new_creation_workflow.md). +The configuration arguments disabled for `glrt-` tokens will be `--locked`, `--access-level`, +`--run-untagged`, `--maximum-timeout`, `--paused`, `--tag-list`, and `--maintenance-note`. This change is a breaking change. ### For a shared runner -1. On the top bar, select **Main menu > Admin**. -1. On the left sidebar, select **CI/CD > Runners**. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **CI/CD > Runners**. 1. Select **Register an instance runner**. 1. Copy the registration token. ### For a group runner -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **CI/CD > Runners**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Build > Runners**. 1. Copy the registration token. ### For a project runner -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand the **Runners** section. 1. Copy the registration token. diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index 43204b463b3..e7b764025c9 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -29,12 +29,12 @@ If you are using a self-managed instance of GitLab: and selecting **Show runner installation instructions**. These instructions are also available [in the documentation](https://docs.gitlab.com/runner/install/index.html). - The administrator can also configure a maximum number of shared runner - [CI/CD minutes for each group](../pipelines/cicd_minutes.md#set-the-quota-of-cicd-minutes-for-a-specific-namespace). + [units of compute for each group](../pipelines/cicd_minutes.md#set-the-compute-quota-for-a-specific-namespace). If you are using GitLab.com: - You can select from a list of [shared runners that GitLab maintains](index.md). -- The shared runners consume the [CI/CD minutes](../pipelines/cicd_minutes.md) +- The shared runners consume the [units of compute](../pipelines/cicd_minutes.md) included with your account. ### Enable shared runners for a project @@ -51,8 +51,8 @@ For existing projects, an administrator must To enable shared runners for a project: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn on the **Enable shared runners for this project** toggle. @@ -60,8 +60,8 @@ To enable shared runners for a project: To enable shared runners for a group: -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn on the **Enable shared runners for this group** toggle. @@ -73,8 +73,8 @@ or group. To disable shared runners for a project: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Runners**. 1. In the **Shared runners** area, turn off the **Enable shared runners for this project** toggle. @@ -87,15 +87,15 @@ Shared runners are automatically disabled for a project: To disable shared runners for a group: -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Settings > CI/CD**. 1. Expand **Runners**. 1. Turn off the **Enable shared runners for this group** toggle. 1. Optional. To allow shared runners to be enabled for individual projects or subgroups, select **Allow projects and subgroups to override the group setting**. NOTE: -If you re-enable the shared runners for a group after you disable them, a user with the +If you re-enable the shared runners for a group after you disable them, a user with the Owner or Maintainer role must manually change this setting for each project subgroup or project. ### How shared runners pick jobs @@ -153,8 +153,8 @@ You must have the Owner role for the group. To create a group runner: 1. [Install GitLab Runner](https://docs.gitlab.com/runner/install/). -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **CI/CD > Runners**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Build > Runners**. 1. In the upper-right corner, select **Register a group runner**. 1. Select **Show runner installation and registration instructions**. These instructions include the token, URL, and a command to register a runner. @@ -170,8 +170,8 @@ You can view and manage all runners for a group, its subgroups, and projects. You can do this for your self-managed GitLab instance or for GitLab.com. You must have the Owner role for the group. -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **CI/CD > Runners**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Build > Runners**. From this page, you can edit, pause, and remove runners from the group, its subgroups, and projects. @@ -187,8 +187,8 @@ Prerequisites: To delete multiple runners in a single action in the group list: -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **CI/CD > Runners**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Build > Runners**. 1. To delete multiple runners, you can either: - Select the checkbox next to the runner. - Select the checkbox at the top of the runner list to select all runners in the list. @@ -207,8 +207,8 @@ By default, only those that are inherited are shown. To show all runners available in the instance, including shared runners and those in other groups: -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **CI/CD > Runners**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Build > Runners**. 1. Above the list, turn off the **Show only inherited** toggle. ### Pause or remove a group runner @@ -216,8 +216,8 @@ those in other groups: You can pause or remove a group runner for your self-managed GitLab instance or for GitLab.com. You must have the Owner role for the group. -1. On the top bar, select **Main menu > Groups** and find your group. -1. On the left sidebar, select **CI/CD > Runners**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Build > Runners**. 1. Select **Pause** or **Remove runner**. - If you pause a group runner that is used by multiple projects, the runner pauses for all projects. - From the group view, you cannot remove a runner that is assigned to more than one project. @@ -252,8 +252,9 @@ Prerequisite: To create a project runner: 1. [Install GitLab Runner](https://docs.gitlab.com/runner/install/). -1. On the top bar, select **Main menu > Projects** and find the project where you want to use the runner. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to + find the project where you want to use the runner. +1. Select **Settings > CI/CD**. 1. Expand **Runners**. 1. In the **Project runners** section, note the URL and token. 1. [Register the runner](https://docs.gitlab.com/runner/register/). @@ -273,8 +274,9 @@ You must have at least the Maintainer role for: To enable a project runner for a project: -1. On the top bar, select **Main menu > Projects** and find the project where you want to enable the runner. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to + find the project where you want to enable the runner. +1. Select **Settings > CI/CD**. 1. Expand **Runners**. 1. In the **Project runners** area, by the runner you want, select **Enable for this project**. @@ -292,8 +294,9 @@ but can also be changed later. To lock or unlock a project runner: -1. On the top bar, select **Main menu > Projects** and find the project where you want to enable the runner. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to + find the project where you want to enable the runner. +1. Select **Settings > CI/CD**. 1. Expand **Runners**. 1. Find the project runner you want to lock or unlock. Make sure it's enabled. You cannot lock shared or group runners. 1. Select **Edit** (**{pencil}**). diff --git a/doc/ci/runners/saas/gpu_saas_runner.md b/doc/ci/runners/saas/gpu_saas_runner.md new file mode 100644 index 00000000000..7b83f6593a0 --- /dev/null +++ b/doc/ci/runners/saas/gpu_saas_runner.md @@ -0,0 +1,46 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# GPU-enabled SaaS runners **(PREMIUM SAAS)** + +GitLab provides GPU-enabled SaaS runners to accelerate heavy compute workloads for ModelOps +or HPC such as the training or deployment of Large Language Models (LLMs) as part of ModelOps workloads. + +GitLab provides GPU-enabled runners only on Linux. For more information about how these runners work, see [SaaS runners on Linux](../saas/linux_saas_runner.md) + +## Machine types available for GPU-enabled runners + +The following machine types are available for GPU-enabled runners on Linux x86-64. + +| Runner Tag | vCPUs | Memory | Storage | GPU | +|----------------------------------------|-------|--------|---------|------------------------------------| +| `saas-linux-medium-amd64-gpu-standard` | 4 | 16 GB | 50 GB | 1 Nvidia Tesla T4 GPU (or similar) | + +## Container images with GPU drivers + +As with GitLab SaaS runners on Linux, your job runs in an isolated virtual machine (VM) +with a bring-your-own-image policy. GitLab mounts the GPU from the host VM into +your isolated environment. To use the GPU, you must use a Docker image with the +GPU driver installed. For Nvidia GPUs, you can use their [CUDA Toolkit](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda). + +## Example `.gitlab-ci.yml` file + +In the following example of the `.gitlab-ci.yml` file, the Nvidia CUDA base Ubuntu image is used. +In the `script:` section, you install Python. + +```yaml +gpu-job: + stage: build + tags: + - saas-linux-medium-amd64-gpu-standard + image: nvcr.io/nvidia/cuda:12.1.1-base-ubuntu22.04 + script: + - apt-get update + - apt-get install -y python3.10 + - python3.10 --version +``` + +If you don't want to install larger libraries such as Tensorflow or XGBoost each time you run a job, you can create your own image with all the required components pre-installed. diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md index 3a45e056643..d95fc701056 100644 --- a/doc/ci/runners/saas/linux_saas_runner.md +++ b/doc/ci/runners/saas/linux_saas_runner.md @@ -4,252 +4,93 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# SaaS runners on Linux +# SaaS runners on Linux **(FREE SAAS)** When you run jobs on SaaS runners on Linux, the runners are on auto-scaled ephemeral virtual machine (VM) instances. -Each VM uses the Google Container-Optimized OS (COS) and the latest version of Docker Engine. The default region for the VMs is `us-east1`. -## Machine types available for private projects (x86-64) +Each VM uses the Google Container-Optimized OS (COS) and the latest version of Docker Engine running the `docker+machine` +[executor](https://docs.gitlab.com/runner/executors/#docker-machine-executor). -For the SaaS runners on Linux, GitLab offers a range of machine types for use in private projects. -For Free, Premium, and Ultimate plan customers, jobs on these instances consume the CI/CD minutes allocated to your namespace. +## Machine types available for Linux (x86-64) -| | Small | Medium | Large | -|-------------------|---------------------------|---------------------------|--------------------------| -| Specs | 1 vCPU, 3.75 GB RAM | 2 vCPUs, 8 GB RAM | 4 vCPUs, 16 GB RAM | -| GitLab CI/CD tags | `saas-linux-small-amd64` | `saas-linux-medium-amd64` | `saas-linux-large-amd64` | -| Subscription | Free, Premium, Ultimate | Free, Premium, Ultimate | Premium, Ultimate | +For the SaaS runners on Linux, GitLab offers a range of machine types for use. +For Free, Premium, and Ultimate plan customers, jobs on these instances consume the compute quota allocated to your namespace. -The `small` machine type is the default. Your job runs on this machine type if you don't specify -a [tags:](../../yaml/index.md#tags) keyword in your `.gitlab-ci.yml` file. +| Runner Tag | vCPUs | Memory | Storage | +|-----------------------------------------------|-------|--------|---------| +| `saas-linux-small-amd64` | 2 | 8 GB | 25 GB | +| `saas-linux-medium-amd64` **(PREMIUM SAAS)** | 4 | 16 GB | 50 GB | +| `saas-linux-large-amd64` **(PREMIUM SAAS)** | 8 | 32 GB | 100 GB | -CI/CD jobs that run on `medium` and `large` machine types consumes CI minutes at a different rate than CI/CD jobs on the `small` machine type. +The `small` machine type is set as default. If no [tag](../../yaml/index.md#tags) keyword in your `.gitlab-ci.yml` file is specified, +the jobs will run on this default runner. -Refer to the CI/CD minutes [cost factor](../../../ci/pipelines/cicd_minutes.md#cost-factor) for the cost factor applied to the machine type based on size. +All SaaS runners on Linux currently run on +[`n2d-standard`](https://cloud.google.com/compute/docs/general-purpose-machines#n2d_machines) gerneral-purpose compute from GCP. +The machine type and underlying processor type can change. Jobs optimized for a specific processor design could behave inconsistently. -## GPU-enabled SaaS runners on Linux **(PREMIUM SAAS)** +## Container images -We offer GPU-enabled SaaS runners for heavy compute including ModelOps or HPC workloads. Available to Premium and Ultimate plan customers, jobs on these instances consume the CI/CD minutes allocated to your namespace. +As runners on Linux are using the `docker+machine` [executor](https://docs.gitlab.com/runner/executors/#docker-machine-executor), +you can choose any container image by defining the [`image`](../../../ci/yaml/index.md#image) in your `.gitlab-ci.yml` file. -| | Standard | -|-------------------|---------------------------| -| Specs | 4 vCPU, 16 GB RAM, 1 Nvidia Tesla T4 GPU (or similar) | -| GitLab CI/CD tags | `saas-linux-medium-gpu-standard` | +If no image is set, the default is `ruby:3.1`. -## Example of how to tag a job +## Docker in Docker support + +The runners are configured to run in `privileged` mode to support +[Docker in Docker](../../../ci/docker/using_docker_build.md#use-docker-in-docker) +to build Docker images natively or run multiple containers within your isolated job. + +## Caching on SaaS runners + +The SaaS runners share a [distributed cache](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) +stored in a Google Cloud Storage (GCS) bucket. Cache contents not updated in the last 14 days are automatically +removed, based on the [object lifecycle management policy](https://cloud.google.com/storage/docs/lifecycle). +The maximum size of an uploaded cache artifact can be 5 GB after the cache becomes a compressed archive. + +For more information about how caching works, see [Caching in GitLab CI/CD](../../caching/index.md). + +## Example `.gitlab-ci.yml` file To use a machine type other than `small`, add a `tags:` keyword to your job. For example: ```yaml -stages: - - Prebuild - - Build - - Unit Test - -job_001: - stage: Prebuild +job_small: script: - - echo "this job runs on the default (small) instance" + - echo "this job runs on the default (small) Linux instance" -job_002: - tags: [ saas-linux-medium-amd64 ] - stage: Build +job_medium: + tags: + - saas-linux-medium-amd64 script: - - echo "this job runs on the medium instance" - + - echo "this job runs on the medium Linux instance" -job_003: - tags: [ saas-linux-large-amd64 ] - stage: Unit Test +job_large: + tags: + - saas-linux-large-amd64 script: - - echo "this job runs on the large instance" - + - echo "this job runs on the large Linux instance" ``` -## SaaS runners for GitLab projects - -The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for -GitLab projects and related community forks. These runners are backed by a Google Compute -`n1-standard-2` machine type and do not run untagged jobs. Unlike the machine types used -for private projects, each virtual machine is re-used up to 40 times. +## SaaS runners for GitLab community contributions -## SaaS runners on Linux settings +If you want to [contribute to GitLab](https://about.gitlab.com/community/contribute/), jobs will be picked up by the +`gitlab-shared-runners-manager-X.gitlab.com` fleet of runners, dedicated for GitLab projects and related community forks. -Below are the settings for SaaS runners on Linux. +These runners are backed by the same machine type as our `small` runners. +Unlike the normal SaaS runners on Linux, each virtual machine is re-used up to 40 times. -| Setting | GitLab.com | Default | -|-------------------------------------------------------------------------|------------------|---------| -| Executor | `docker+machine` | - | -| Default Docker image | `ruby:3.1` | - | -| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | +As we want to encourage people to contribute, these runners are free of charge. -- **Cache**: These runners share a - [distributed cache](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) - that's stored in a Google Cloud Storage (GCS) bucket. Cache contents not updated in - the last 14 days are automatically removed, based on the - [object lifecycle management policy](https://cloud.google.com/storage/docs/lifecycle). +<!--- start_remove The following content will be removed on remove_date: '2023-08-22' --> -- **Timeout settings**: Jobs handled by the SaaS Runners on Linux - **time out after 3 hours**, regardless of the timeout configured in a - project. For details, see issues [#4010](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4010) - and [#4070](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/4070). +## Pre-clone script (removed) -NOTE: -SaaS runner instances are provisioned with a 25 GB storage volume. The underlying disk space of the storage volume -is shared by the operating system, the Docker image, and a copy of your cloned repository. -This means that the available free disk space that your jobs can use is **less than 25 GB**. - -## Pre-clone script (deprecated) - -WARNING: This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/391896) in GitLab 15.9 -and is planned for removal in 16.0. Use [`pre_get_sources_script`](../../../ci/yaml/index.md#hookspre_get_sources_script) instead. This change is a breaking change. -With SaaS runners on Linux, you can run commands in a CI/CD -job before the runner attempts to run `git init` and `git fetch` to -download a GitLab repository. The -[`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) -can be used for: - -- Seeding the build directory with repository data -- Sending a request to a server -- Downloading assets from a CDN -- Any other commands that must run before the `git init` - -To use this feature, define a [CI/CD variable](../../../ci/variables/index.md) called -`CI_PRE_CLONE_SCRIPT` that contains a bash script. - -NOTE: -The `CI_PRE_CLONE_SCRIPT` variable does not work on GitLab SaaS Windows or macOS runners. - -### Pre-clone script example - -This example was used in the `gitlab-org/gitlab` project until November 2021. -The project no longer uses this optimization because the -[pack-objects cache](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache) -lets Gitaly serve the full CI/CD fetch traffic. See [Git fetch caching](../../../development/pipelines/performance.md#git-fetch-caching). - -The `CI_PRE_CLONE_SCRIPT` was defined as a project CI/CD variable: - -```shell -( - echo "Downloading archived master..." - wget -O /tmp/gitlab.tar.gz https://storage.googleapis.com/gitlab-ci-git-repo-cache/project-278964/gitlab-master-shallow.tar.gz - - if [ ! -f /tmp/gitlab.tar.gz ]; then - echo "Repository cache not available, cloning a new directory..." - exit - fi - - rm -rf $CI_PROJECT_DIR - echo "Extracting tarball into $CI_PROJECT_DIR..." - mkdir -p $CI_PROJECT_DIR - cd $CI_PROJECT_DIR - tar xzf /tmp/gitlab.tar.gz - rm -f /tmp/gitlab.tar.gz - chmod a+w $CI_PROJECT_DIR -) -``` - -The first step of the script downloads `gitlab-master.tar.gz` from Google Cloud Storage. -There was a [GitLab CI/CD job named `cache-repo`](https://gitlab.com/gitlab-org/gitlab/-/blob/5fb40526c8c8aaafc5f92eab36d5bbddaca3893d/.gitlab/ci/cache-repo.gitlab-ci.yml) -that was responsible for keeping that archive up-to-date. Every two hours on a scheduled pipeline, -it did the following: - -1. Create a fresh clone of the `gitlab-org/gitlab` repository on GitLab.com. -1. Save the data as a `.tar.gz`. -1. Upload it into the Google Cloud Storage bucket. +and [removed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29405) in 16.0. +Use [`pre_get_sources_script`](../../../ci/yaml/index.md#hookspre_get_sources_script) instead. -When a job ran with this configuration, the output looked similar to: - -```shell -$ eval "$CI_PRE_CLONE_SCRIPT" -Downloading archived master... -Extracting tarball into /builds/gitlab-org/gitlab... -Fetching changes... -Reinitialized existing Git repository in /builds/gitlab-org/gitlab/.git/ -``` - -The `Reinitialized existing Git repository` message shows that -the pre-clone step worked. The runner runs `git init`, which -overwrites the Git configuration with the appropriate settings to fetch -from the GitLab repository. - -`CI_REPO_CACHE_CREDENTIALS` must contain the Google Cloud service account -JSON for uploading to the `gitlab-ci-git-repo-cache` bucket. - -This bucket should be located in the same continent as the -runner, or [you can incur network egress charges](https://cloud.google.com/storage/pricing). - -## `config.toml` - -The full contents of our `config.toml` are: - -NOTE: -Settings that are not public are shown as `X`. - -**Google Cloud Platform** - -```toml -concurrent = X -check_interval = 1 -metrics_server = "X" -sentry_dsn = "X" - -[[runners]] - name = "docker-auto-scale" - request_concurrency = X - url = "https://gitlab.com/" - token = "SHARED_RUNNER_TOKEN" - pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\"" - executor = "docker+machine" - environment = [ - "DOCKER_DRIVER=overlay2", - "DOCKER_TLS_CERTDIR=" - ] - limit = X - [runners.docker] - image = "ruby:3.1" - privileged = true - volumes = [ - "/certs/client", - "/dummy-sys-class-dmi-id:/sys/class/dmi/id:ro" # Make kaniko builds work on GCP. - ] - [runners.machine] - IdleCount = 50 - IdleTime = 3600 - MaxBuilds = 1 # For security reasons we delete the VM after job has finished so it's not reused. - MachineName = "srm-%s" - MachineDriver = "google" - MachineOptions = [ - "google-project=PROJECT", - "google-disk-size=25", - "google-machine-type=n1-standard-1", - "google-username=core", - "google-tags=gitlab-com,srm", - "google-use-internal-ip", - "google-zone=us-east1-d", - "engine-opt=mtu=1460", # Set MTU for container interface, for more information check https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3214#note_82892928 - "google-machine-image=PROJECT/global/images/IMAGE", - "engine-opt=ipv6", # This will create IPv6 interfaces in the containers. - "engine-opt=fixed-cidr-v6=fc00::/7", - "google-operation-backoff-initial-interval=2" # Custom flag from forked docker-machine, for more information check https://github.com/docker/machine/pull/4600 - ] - [[runners.machine.autoscaling]] - Periods = ["* * * * * sat,sun *"] - Timezone = "UTC" - IdleCount = 70 - IdleTime = 3600 - [[runners.machine.autoscaling]] - Periods = ["* 30-59 3 * * * *", "* 0-30 4 * * * *"] - Timezone = "UTC" - IdleCount = 700 - IdleTime = 3600 - [runners.cache] - Type = "gcs" - Shared = true - [runners.cache.gcs] - CredentialsFile = "/path/to/file" - BucketName = "bucket-name" -``` +<!--- end_remove --> diff --git a/doc/ci/runners/saas/macos/codesigning.md b/doc/ci/runners/saas/macos/codesigning.md index 809b8faf9df..7e70e984c7c 100644 --- a/doc/ci/runners/saas/macos/codesigning.md +++ b/doc/ci/runners/saas/macos/codesigning.md @@ -1,28 +1,11 @@ --- -stage: Verify -group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../macos_saas_runner.md' +remove_date: '2023-09-05' --- -# Code signing for SaaS runners on macOS +This document was moved to [another location](../macos_saas_runner.md). -Before you can integrate GitLab with Apple services, install to a device, or deploy to the Apple App Store, you must [code sign](https://developer.apple.com/support/code-signing/) your application. - -## Code signing iOS Projects with fastlane - -When you use SaaS runners on macOS, each job runs on a VM. Included in each VM is [fastlane](https://fastlane.tools/), -an open-source solution aimed at simplifying mobile app deployment. - -For information about how to set up code signing for your application, see the instructions in the [Mobile DevOps documentation](../../../../ci/mobile_devops.md#code-sign-ios-projects-with-fastlane). - -These instructions provide the minimal setup to use fastlane to code sign your application. For more information about using fastlane to handle code signing, see the following resources: - -- [fastlane getting started guide](https://docs.fastlane.tools/) -- [Best practices for integrating with GitLab CI](https://docs.fastlane.tools/best-practices/continuous-integration/gitlab/) -- [fastlane code signing getting started guide](https://docs.fastlane.tools/codesigning/getting-started/) - -## Related topics - -- [Apple Developer Support - Code Signing](https://developer.apple.com/support/code-signing/) -- [Code Signing Best Practice Guide](https://codesigning.guide/) -- [fastlane authentication with Apple Services guide](https://docs.fastlane.tools/getting-started/ios/authentication/) +<!-- This redirect file can be deleted after <2023-09-05>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
\ No newline at end of file diff --git a/doc/ci/runners/saas/macos/environment.md b/doc/ci/runners/saas/macos/environment.md index 7aa0f33fc59..7e70e984c7c 100644 --- a/doc/ci/runners/saas/macos/environment.md +++ b/doc/ci/runners/saas/macos/environment.md @@ -1,62 +1,11 @@ --- -stage: Verify -group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../macos_saas_runner.md' +remove_date: '2023-09-05' --- -# VM instances and images for SaaS runners on macOS **(PREMIUM SAAS)** +This document was moved to [another location](../macos_saas_runner.md). -When you use SaaS runners on macOS: - -- Each of your jobs runs in a newly provisioned VM, which is dedicated to the specific job. -- The VM is active only for the duration of the job and immediately deleted. This means that any changes that your job makes to the virtual machine will not be available to a subsequent job. -- The virtual machine where your job runs has `sudo` access with no password. - -NOTE: -Each time you run a job that requires tooling or dependencies not available in the base image, those items must be added to the newly provisioned build VM. That process will likely increase the total job duration. - -## VM types - -GitLab SaaS provides macOS build machines on Apple servers with Intel x86-64 processors. -The expectation is that virtual machines running on the Apple M1 chip will be available in the second half of 2022. - -At this time there is only one available machine type offered, `shared-macos-amd64`. - -| Instance type | vCPUS | Memory (GB) | -| --------- | --- | ------- | -| `shared-macos-amd64` | 4 | 10 | - -## VM images - -### Image update policy - -GitLab expects to release new images based on this cadence: - -macOS updates: - -- **For new OS versions:** When Apple releases a new macOS version to developers (like macOS `12`), GitLab will plan to release an image based on the OS within the next 30 business days. The image is considered `beta` and the contents of the image (including tool versions) are subject to change until the first patch release (`12.1`). The long-term name will not include `beta` (for example, `macos-12-xcode-13`), so customers are moved automatically out of beta over time. GitLab will try to minimize breaking changes between the first two minor versions but makes no guarantees. Tooling often gets critical bug fixes after the first public release of an OS version. - -- **After the first patch release (`12.1`):** - - The image moves to `maintenance` mode. The tools GitLab builds into the image with Homebrew and asdf are frozen. GitLab continues making Xcode updates, security updates, and any non-breaking changes deemed necessary. - - The image for the previous OS version (`11`) moves to `frozen` mode. GitLab then does only unavoidable changes: security updates, runner version upgrades, and setting the production password. - -Both macOS and Xcode follow a yearly release cadence. As time goes on, GitLab increments their versions synchronously (meaning we build macOS 11 with Xcode 12, macOS 12 with Xcode 13, and so on). - -### Available images - -You can execute your build on one of the following images. -You specify this image in your `.gitlab-ci.yml` file. - -Each image is running a specific version of macOS and Xcode. - -| VM image | Status | Included software | -|---------------------------|--------|--------------------| -| `macos-10.13-xcode-7` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | -| `macos-10.13-xcode-8` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | -| `macos-10.13-xcode-9` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/high-sierra.yml> | -| `macos-10.14-xcode-10` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/mojave.yml> | -| `macos-10.15-xcode-11` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/catalina.yml> | -| `macos-11-xcode-12` | `frozen` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/big-sur.yml> | -| `macos-12-xcode-13` | `maintenance` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/monterey.yml> | -| `macos-12-xcode-14` | `maintenance` | <https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/macstadium/orka/-/blob/main/toolchain/monterey.yml> | -| (none, awaiting macOS 13) | `beta` | | +<!-- This redirect file can be deleted after <2023-09-05>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
\ No newline at end of file diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index 20be2f2a147..836a14d7521 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -6,57 +6,72 @@ info: To determine the technical writer assigned to the Stage/Group associated w # SaaS runners on macOS (Beta) **(PREMIUM SAAS)** -SaaS runners on macOS are in [Beta](../../../policy/alpha-beta-support.md#beta) for approved open source programs and customers in Premium and Ultimate plans. +SaaS runners on macOS are in [Beta](../../../policy/experiment-beta-support.md#beta) for open source programs and customers in Premium and Ultimate plans. SaaS runners on macOS provide an on-demand macOS build environment integrated with GitLab SaaS [CI/CD](../../../ci/index.md). -Use these runners to build, test, and deploy apps for the Apple ecosystem (macOS, iOS, tvOS). You can take advantage +Use these runners to build, test, and deploy apps for the Apple ecosystem (macOS, iOS, watchOS, tvOS). You can take advantage of all the capabilities of the GitLab single DevOps platform and not have to manage or operate a -build environment. +build environment. Our [Mobile DevOps solution](../../../ci/mobile_devops.md#ios-build-environments) provides features, documentation, and guidance on building and deploying mobile applications for iOS. -Jobs handled by macOS shared runners on GitLab.com **time out after 3 hours**, regardless of the timeout configured in a project. +We want to keep iterating to get SaaS runners on macOS +[generally available](../../../policy/experiment-beta-support.md#generally-available-ga). +You can follow our work towards this goal in the +[related epic](https://gitlab.com/groups/gitlab-org/-/epics/8267). -## Access request process +## Machine types available for macOS -While in beta, to run CI jobs on the macOS runners, you must specify the GitLab SaaS customer personal or group [namespaces](../../../user/namespace/index.md) in the macOS `allow-list`. These are the namespaces that use the macOS runners. +GitLab SaaS provides macOS build machines on Apple silicon (M1) chips. +Intel x86-64 runners were deprecated in favor of Apple silicon. To build for an x86-64 target, use Rosetta 2 to emulate an Intel x86-64 build environment. -When you specify a personal or group namespace, the top level group is not added unless you specify it. +| Runner Tag | vCPUS | Memory | Storage | +| ---------------------- | ----- | ------ | ------- | +| `saas-macos-medium-m1` | 4 | 8 GB | 25 GB | -After you add your namespace, you can use the macOS runners for any projects under the namespace you included. +## Supported macOS images -To request access, open an [access request](https://gitlab.com/gitlab-com/runner-saas-macos-limited-availability/-/issues/new). -The expected turnaround for activation is two business days. +In comparison to our SaaS runners on Linux, where you can run any Docker image, +GitLab SaaS provides a set of VM images for macOS. -## Quickstart +You can execute your build in one of the following images, which you specify +in your `.gitlab-ci.yml` file. -To start using SaaS runners on macOS, you must be an active GitLab SaaS Premium or Ultimate customer. Participants in the GitLab Open Source program are also eligible to use the service. +Each image runs a specific version of macOS and Xcode. -### Configuring your pipeline +| VM image | Status | +|---------------------------|---------------| +| `macos-12-xcode-13` | `maintenance` | +| `macos-12-xcode-14` | `maintenance` | +| (none, awaiting macOS 13) | `beta` | -To start using the SaaS runners on macOS to run your CI jobs, you must configure your `.gitlab-ci.yml` file: +NOTE: +Each time you run a job that requires tooling or dependencies not available in the base image, those items must be added to the newly provisioned build VM increasing the total job duration. + +## Image update policy + +GitLab expects to release new images based on this cadence: + +macOS updates: + +- **For new OS versions:** When Apple releases a new macOS version to developers (like macOS `12`), GitLab will plan to release an image based on the OS within the next 30 business days. The image is considered `beta` and the contents of the image (including tool versions) are subject to change until the first patch release (`12.1`). The long-term name will not include `beta` (for example, `macos-12-xcode-13`), so customers are moved automatically out of beta over time. GitLab will try to minimize breaking changes between the first two minor versions but makes no guarantees. Tooling often gets critical bug fixes after the first public release of an OS version. -1. Add a `.gitlab-ci.yml` file to your project repository. -1. Specify the [image](macos/environment.md#vm-images) you want to use. -1. Commit a change to your repository. +- **After the first patch release (`12.1`):** + - The image moves to `maintenance` mode. The tools GitLab builds into the image with Homebrew and asdf are frozen. GitLab continues making Xcode updates, security updates, and any non-breaking changes deemed necessary. + - The image for the previous OS version (`11`) moves to `frozen` mode. GitLab then does only unavoidable changes: security updates, runner version upgrades, and setting the production password. -The runners automatically run your build. +Both macOS and Xcode follow a yearly release cadence. As time goes on, GitLab increments their versions synchronously (meaning we build macOS 11 with Xcode 12, macOS 12 with Xcode 13, and so on). -### Example `.gitlab-ci.yml` file +## Example `.gitlab-ci.yml` file The following sample `.gitlab-ci.yml` file shows how to start using the SaaS runners on macOS: ```yaml .macos_saas_runners: tags: - - shared-macos-amd64 - image: macos-11-xcode-12 - -stages: - - build - - test - -before_script: - - echo "started by ${GITLAB_USER_NAME}" + - saas-macos-medium-m1 + image: macos-12-xcode-14 + before_script: + - echo "started by ${GITLAB_USER_NAME}" build: extends: @@ -74,14 +89,39 @@ test: ``` NOTE: -You can specify a different Xcode image to run a job. To do so, replace the value for the `image` keyword with the value of the [virtual machine image name](macos/environment.md#vm-images) from the list of available images. +You can specify a different Xcode image to run a job. To do so, replace the value for the `image` keyword with the value of the [virtual machine image name](#supported-macos-images) from the list of available images. The default value is our latest image. + +## Code signing iOS Projects with fastlane + +Before you can integrate GitLab with Apple services, install to a device, or deploy to the Apple App Store, you must [code sign](https://developer.apple.com/support/code-signing/) your application. -## SaaS runners on macOS service level objective +Included in each runner on macOS VM image is [fastlane](https://fastlane.tools/), +an open-source solution aimed at simplifying mobile app deployment. -In SaaS runners on macOS, the objective is to make 90% of CI jobs start executing in 120 seconds or less. The error rate should be less than 0.5%. +For information about how to set up code signing for your application, see the instructions in the [Mobile DevOps documentation](../../../ci/mobile_devops.md#code-sign-ios-projects-with-fastlane). + +Related topics: + +- [Apple Developer Support - Code Signing](https://developer.apple.com/support/code-signing/) +- [Code Signing Best Practice Guide](https://codesigning.guide/) +- [fastlane authentication with Apple Services guide](https://docs.fastlane.tools/getting-started/ios/authentication/) ## Known Limitations and Usage Constraints - If the VM image does not include the specific software version you need for your job, then the job execution time will increase as the required software needs to be fetched and installed. - At this time, it is not possible to bring your own OS image. - The keychain for user `gitlab` is not publicly available. You must create a keychain instead. + +## Optimizing Homebrew + +By default, Homebrew checks for updates at the start of any operation. Homebrew has a +release cycle that may be more frequent than the GitLab MacOS image release cycle. This +difference in release cycles may cause steps that call `brew` to take extra time to complete +while Homebrew makes updates. + +To reduce build time due to unintended Homebrew updates, set the `HOMEBREW_NO_AUTO_UPDATE` variable in `.gitlab-ci.yml` : + +```yaml +variables: + HOMEBREW_NO_AUTO_UPDATE: 1 +``` diff --git a/doc/ci/runners/saas/windows_saas_runner.md b/doc/ci/runners/saas/windows_saas_runner.md index 6e4ffc036b0..fa981bddff3 100644 --- a/doc/ci/runners/saas/windows_saas_runner.md +++ b/doc/ci/runners/saas/windows_saas_runner.md @@ -6,88 +6,47 @@ info: To determine the technical writer assigned to the Stage/Group associated w # SaaS runners on Windows (Beta) **(FREE SAAS)** -SaaS runners on Windows are in [Beta](../../../policy/alpha-beta-support.md#beta) -and shouldn't be used for production workloads. - -During this beta period, the [shared runner quota for CI/CD minutes](../../pipelines/cicd_minutes.md) -applies for groups and projects in the same manner as Linux runners. This may -change when the beta period ends, as discussed in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30834). - -Windows runners on GitLab.com autoscale by launching virtual machines on +SaaS runner on Windows autoscale by launching virtual machines on the Google Cloud Platform. This solution uses an [autoscaling driver](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/-/blob/main/docs/README.md) developed by GitLab for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html). -Windows runners execute your CI/CD jobs on `n1-standard-2` instances with -2 vCPUs and 7.5 GB RAM. You can find a full list of available Windows packages in -the [package documentation](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/main/cookbooks/preinstalled-software/README.md). + +These SaaS runners are in [Beta](../../../policy/experiment-beta-support.md#beta) +and aren't recomended for production workloads. We want to keep iterating to get Windows runners in a stable state and -[generally available](../../../policy/alpha-beta-support.md#generally-available-ga). +[generally available](../../../policy/experiment-beta-support.md#generally-available-ga). You can follow our work towards this goal in the [related epic](https://gitlab.com/groups/gitlab-org/-/epics/2162). -## Configuration +## Machine types available for Windows + +| Runner Tag | vCPUs | Memory | Storage | +| ---------------------- | ----- | ------ | ------- | +| `shared-windows` | 2 | 7.5 GB | 75 GB | + +## Supported Windows versions + +The Windows runner virtual machine instances do not use the GitLab Docker executor. This means that you can't specify +[`image`](../../../ci/yaml/index.md#image) or [`services`](../../../ci/yaml/index.md#services) in your pipeline configuration. +Instead you have to select the [`tags`](../../../ci/yaml/index.md#tags) for the desired Windows version. -The full contents of our `config.toml` are: +You can execute your job in one of the following Windows versions: + +| Version tag | Status | +|----------------|---------------| +| `windows-1809` | `maintenance` | + +You can find a full list of available pre-installed software in +the [pre-installed software documentation](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/main/cookbooks/preinstalled-software/README.md). NOTE: -Settings that aren't public are shown as `X`. - -```toml -concurrent = X -check_interval = 3 - -[[runners]] - name = "windows-runner" - url = "https://gitlab.com/" - token = "TOKEN" - executor = "custom" - builds_dir = "C:\\GitLab-Runner\\builds" - cache_dir = "C:\\GitLab-Runner\\cache" - shell = "powershell" - [runners.custom] - config_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - config_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "config"] - prepare_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - prepare_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "prepare"] - run_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - run_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "run"] - cleanup_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - cleanup_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "cleanup"] -``` +Each time you run a job that requires tooling or dependencies not available in the base image, those components must be installed to the newly provisioned VM increasing the total job duration. -The full contents of our `autoscaler/config.toml` are: - -```toml -Provider = "gcp" -Executor = "winrm" -OS = "windows" -LogLevel = "info" -LogFormat = "text" -LogFile = "C:\\GitLab-Runner\\autoscaler\\autoscaler.log" -VMTag = "windows" - -[GCP] - ServiceAccountFile = "PATH" - Project = "some-project-df9323" - Zone = "us-east1-c" - MachineType = "n1-standard-2" - Image = "IMAGE" - DiskSize = 50 - DiskType = "pd-standard" - Subnetwork = "default" - Network = "default" - Tags = ["TAGS"] - Username = "gitlab_runner" - -[WinRM] - MaximumTimeout = 3600 - ExecutionMaxRetries = 0 - -[ProviderCache] - Enabled = true - Directory = "C:\\GitLab-Runner\\autoscaler\\machines" -``` +## Supported shell + +SaaS runners on Windows have PowerShell configured as the shell. +The `script` section of your `.gitlab-ci.yml` file therefore requires PowerShell commands. ## Example `.gitlab-ci.yml` file @@ -97,7 +56,6 @@ Below is a sample `.gitlab-ci.yml` file that shows how to start using the runner .shared_windows_runners: tags: - shared-windows - - windows - windows-1809 stages: @@ -126,7 +84,7 @@ test: ## Limitations and known issues -- All the limitations mentioned in our [Beta definition](../../../policy/alpha-beta-support.md#beta). +- All the limitations mentioned in our [Beta definition](../../../policy/experiment-beta-support.md#beta). - The average provisioning time for a new Windows VM is 5 minutes. This means that you may notice slower build start times on the Windows runner fleet during the beta. In a future @@ -136,17 +94,6 @@ test: follow along in the [related issue](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/-/issues/32). - The Windows runner fleet may be unavailable occasionally for maintenance or updates. -- The Windows runner virtual machine instances do not use the - GitLab Docker executor. This means that you can't specify - [`image`](../../../ci/yaml/index.md#image) or [`services`](../../../ci/yaml/index.md#services) in - your pipeline configuration. -- For the beta release, we have included a set of software packages in - the base VM image. If your CI job requires additional software that's - not included in this list, then you must add installation - commands to [`before_script`](../../../ci/yaml/index.md#before_script) or [`script`](../../../ci/yaml/index.md#script) to install the required - software. Note that each job runs on a new VM instance, so the - installation of additional software packages needs to be repeated for - each job in your pipeline. - The job may stay in a pending state for longer than the Linux runners. - There is the possibility that we introduce breaking changes which will diff --git a/doc/ci/secrets/id_token_authentication.md b/doc/ci/secrets/id_token_authentication.md index 1ff2a6efbcf..16b94fed4d3 100644 --- a/doc/ci/secrets/id_token_authentication.md +++ b/doc/ci/secrets/id_token_authentication.md @@ -60,6 +60,7 @@ The token also includes custom claims provided by GitLab: | `user_id` | Always | ID of the user executing the job. | | `user_login` | Always | Username of the user executing the job. | | `user_email` | Always | Email of the user executing the job. | +| `user_identities` | User Preference setting | List of the user's external identities ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387537) in GitLab 16.0). | | `pipeline_id` | Always | ID of the pipeline. | | `pipeline_source` | Always | [Pipeline source](../jobs/job_control.md#common-if-clauses-for-rules). | | `job_id` | Always | ID of the job. | @@ -73,6 +74,7 @@ The token also includes custom claims provided by GitLab: | `runner_id` | Always | ID of the runner executing the job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.0. | | `runner_environment` | Always | The type of runner used by the job. Can be either `gitlab-hosted` or `self-hosted`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.0. | | `sha` | Always | The commit SHA for the job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.0. | +| `ci_config_ref_uri` | Always | The ref path to the top-level pipeline definition, for example, `gitlab.example.com/my-group/my-project//.gitlab-ci.yml@refs/heads/main`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.1 behind the `ci_jwt_v2_ref_uri_claim` feature flag. This claim is `null` unless the pipeline definition is located in the same project. | ```json { @@ -83,6 +85,10 @@ The token also includes custom claims provided by GitLab: "user_id": "1", "user_login": "sample-user", "user_email": "sample-user@example.com", + "user_identities": [ + {"provider": "github", "extern_uid": "2435223452345"}, + {"provider": "bitbucket", "extern_uid": "john.smith"}, + ], "pipeline_id": "574", "pipeline_source": "push", "job_id": "302", @@ -96,6 +102,7 @@ The token also includes custom claims provided by GitLab: "runner_id": 1, "runner_environment": "self-hosted", "sha": "714a629c0b401fdce83e847fc9589983fc6f46bc", + "ci_config_ref_uri": "gitlab.example.com/my-group/my-project//.gitlab-ci.yml@refs/heads/main", "jti": "235b3a54-b797-45c7-ae9a-f72d7bc6ef5b", "iss": "https://gitlab.example.com", "iat": 1681395193, @@ -179,9 +186,40 @@ ID token authentication is now always available, and JSON Web Token access is al To enable automatic ID token authentication: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Token Access**. -1. Toggle **Limit JSON Web Token (JWT) access** to enabled. +1. Turn on the **Limit JSON Web Token (JWT) access** toggle. <!--- end_remove --> + +## Troubleshooting + +### `400: missing token` status code + +This error indicates that one or more basic components necessary for ID tokens are +either missing or not configured as expect. + +To find the problem, an administrator can look for more details in the instance's +`exceptions_json.log` for the specific method that failed. + +#### `GitLab::Ci::Jwt::NoSigningKeyError` + +This error in the `exceptions_json.log` file is likely because the signing key is +missing from the database and the token could not be generated. To verify this is the issue, +run the following query on the instance's PostgreSQL terminal: + +```sql +SELECT encrypted_ci_jwt_signing_key FROM application_settings; +``` + +If the returned value is empty, use the Rails snippet below to generate a new key +and replace it internally: + +```ruby + key = OpenSSL::PKey::RSA.new(2048).to_pem + + ApplicationSetting.find_each do |application_setting| + application_setting.update(ci_jwt_signing_key: key) + end +``` diff --git a/doc/ci/secure_files/index.md b/doc/ci/secure_files/index.md index 41e6fe06f84..b3ccf071996 100644 --- a/doc/ci/secure_files/index.md +++ b/doc/ci/secure_files/index.md @@ -29,9 +29,9 @@ tool. To add a secure file to a project: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. -1. In the **Secure Files** section, select **Expand**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. +1. Expand the **Secure Files** section. 1. Select **Upload File**. 1. Find the file to upload, select **Open**, and the file upload begins immediately. The file shows up in the list when the upload is complete. diff --git a/doc/ci/testing/code_coverage.md b/doc/ci/testing/code_coverage.md index bd1246d2f78..8ffd2cfb727 100644 --- a/doc/ci/testing/code_coverage.md +++ b/doc/ci/testing/code_coverage.md @@ -72,8 +72,8 @@ Use this regex for commonly used test tools. To see the evolution of your project code coverage over time, you can view a graph or download a CSV file with this data. -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Repository**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. On the left sidebar, select **Analyze > Repository analytics**. The historic data for each job is listed in the dropdown list above the graph. diff --git a/doc/ci/testing/code_quality.md b/doc/ci/testing/code_quality.md index 367777960b5..5f6af4cb8a9 100644 --- a/doc/ci/testing/code_quality.md +++ b/doc/ci/testing/code_quality.md @@ -227,7 +227,7 @@ Code Quality can be customized by defining available CI/CD variables: | CI/CD variable | Description | | --------------------------- | ----------- | | `SOURCE_CODE` | Path to the source code to scan. | -| `TIMEOUT_SECONDS` | Custom timeout for the `codeclimate analyze` command. | +| `TIMEOUT_SECONDS` | Custom timeout per engine container for the `codeclimate analyze` command, default is 900 seconds (15 minutes). | | `CODECLIMATE_DEBUG` | Set to enable [Code Climate debug mode](https://github.com/codeclimate/codeclimate#environment-variables) | | `CODECLIMATE_DEV` | Set to enable `--dev` mode which lets you run engines not known to the CLI. | | `REPORT_STDOUT` | Set to print the report to `STDOUT` instead of generating the usual report file. | @@ -492,6 +492,23 @@ Example: ] ``` +## Integrate multiple tools + +Code Quality combines the results from all jobs in a pipeline into a single `gl-code-quality-report.json` file. As a result, multiple individual tools can be used in a pipeline, either alongside, or in place of, the supported `Code-Quality.gitlab-ci.yml` template. + +Here is an example that returns ESLint output in the necessary format: + +```yaml +eslint: + image: node:18-alpine + script: + - npm ci + - npx eslint --format gitlab . + artifacts: + reports: + codequality: gl-code-quality-report.json +``` + ## Using Analysis Plugins Code Quality functionality can be extended by using Code Climate @@ -521,6 +538,12 @@ for more details. ## Troubleshooting +### The code cannot be found and the pipeline runs always with default configuration + +You are probably using a private runner with the Docker-in-Docker socket-binding configuration. +You should configure Code Quality checks to run on your worker as documented in section +"[Improve Code Quality performance with private runners](#improve-code-quality-performance-with-private-runners)". + ### Changing the default configuration has no effect A common issue is that the terms `Code Quality` (GitLab specific) and `Code Climate` @@ -549,12 +572,9 @@ This can be due to multiple reasons: ### Only a single Code Quality report is displayed, but more are defined -Starting [in GitLab 15.7](https://gitlab.com/gitlab-org/gitlab/-/issues/340525), Code Quality combines the results from all jobs in a pipeline. - -In previous versions, GitLab only uses the Code Quality artifact from the latest created job (with the largest job ID). -If multiple jobs in a pipeline generate a code quality artifact, those of earlier jobs are ignored. +Code Quality automatically [combines multiple reports](#integrate-multiple-tools). -To avoid confusion, configure only one job to generate a `gl-code-quality-report.json` file. +In GitLab 15.6 and earlier, Code Quality used only the artifact from the latest created job (with the largest job ID). Code Quality artifacts from earlier jobs were ignored. ### RuboCop errors diff --git a/doc/ci/testing/test_coverage_visualization.md b/doc/ci/testing/test_coverage_visualization.md index 31d1b7f3337..669157daa21 100644 --- a/doc/ci/testing/test_coverage_visualization.md +++ b/doc/ci/testing/test_coverage_visualization.md @@ -73,10 +73,11 @@ a [blocking manual job](../jobs/job_control.md#types-of-manual-jobs), the pipeline waits for the manual job before continuing and is not considered complete. The visualization cannot be displayed if the blocking manual job did not run. -### Artifact expiration +### Data expiration -By default, the [pipeline artifact](../pipelines/pipeline_artifacts.md#storage) used -to draw the visualization on the merge request expires **one week** after creation. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321323) in GitLab 13.12, the latest data is kept regardless of expiry time. + +By default, the data used to draw the visualization on the merge request expires **one week** after creation. ### Coverage report from child pipeline @@ -296,7 +297,7 @@ run tests: The following [`.gitlab-ci.yml`](../yaml/index.md) example for PHP uses [PHPUnit](https://phpunit.readthedocs.io/) to collect test coverage data and generate the report. -With a minimal [`phpunit.xml`](https://phpunit.readthedocs.io/en/9.5/configuration.html) file (you may reference +With a minimal [`phpunit.xml`](https://docs.phpunit.de/en/10.2/configuration.html) file (you may reference [this example repository](https://gitlab.com/yookoala/code-coverage-visualization-with-php/)), you can run the test and generate the `coverage.xml`: @@ -428,6 +429,8 @@ the coverage report itself and verify that: - The file you are viewing in the diff view is mentioned in the coverage report. - The `source` and `filename` nodes in the report follows the [expected structure](#automatic-class-path-correction) to match the files in your repository. +- The pipeline has completed. If the pipeline is [blocked on a manual job](../jobs/job_control.md#types-of-manual-jobs), + the pipeline is not considered complete. Report artifacts are not downloadable by default. If you want the report to be downloadable from the job details page, add your coverage report to the artifact `paths`: diff --git a/doc/ci/testing/unit_test_report_examples.md b/doc/ci/testing/unit_test_report_examples.md index c63e225a2a7..1b1600b3d99 100644 --- a/doc/ci/testing/unit_test_report_examples.md +++ b/doc/ci/testing/unit_test_report_examples.md @@ -260,7 +260,7 @@ test: This example uses [PHPUnit](https://phpunit.de/) with the `--log-junit` flag. You can also add this option using -[XML](https://phpunit.readthedocs.io/en/9.5/configuration.html#the-junit-element) +[XML](https://docs.phpunit.de/en/10.2/configuration.html#the-junit-element) in the `phpunit.xml` configuration file. ```yaml diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index b9e5dd87b24..412394a24e7 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -26,8 +26,8 @@ Prerequisite: To create a trigger token: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Pipeline triggers**. 1. Enter a description and select **Add trigger**. - You can view and copy the full token for all triggers you have created. @@ -153,8 +153,8 @@ users with the Owner and Maintainer role can view the values. To revoke a trigger token: -1. On the top bar, select **Main menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. 1. Expand **Pipeline triggers**. 1. To the left of the trigger token you want to revoke, select **Revoke** (**{remove}**). diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 973c6b90fc5..e94ccfa6b2b 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -246,15 +246,10 @@ After the pipeline is created, the message updates with the pipeline status. ### Merge request status messages -The merge request status widget shows the **Merge** button and whether or not a merge -request is ready to merge. If the merge request can't be merged, the reason for this -is displayed. +The merge request status widget shows: -If the pipeline is still running, **Merge** is replaced with the -**Merge when pipeline succeeds** button. - -If [**Merge Trains**](pipelines/merge_trains.md) -are enabled, the button is either **Add to merge train** or **Add to merge train when pipeline succeeds**. **(PREMIUM)** +- If the merge request is ready to merge. If the merge request can't be merged, the reason is displayed. +- **Merge**, if the pipeline is complete, or **Set to auto-merge** if the pipeline is still running. #### "A CI/CD pipeline must run and be successful before merge" message @@ -333,6 +328,23 @@ When you rerun a job, uses the same configuration each time. If you update confi including separate files added with [`include`](yaml/index.md#include), you must start a new pipeline to use the new configuration. +### Unable to pull image from another project + +When a runner tries to pull an image from a private project, the job could fail with the following error: + +```shell +WARNING: Failed to pull image with policy "always": Error response from daemon: pull access denied for registry.example.com/path/to/project, repository does not exist or may require 'docker login': denied: requested access to the resource is denied +``` + +This error can happen if the following are both true: + +- The **Allow access to this project with a CI_JOB_TOKEN** option is enabled in the private project + hosting the image. +- The job attempting to fetch the image is running for a project that is not listed in + the private project's allowlist. + +The recommended solution is to [add your project to the private project's job token scope allowlist](jobs/ci_job_token.md#add-a-project-to-the-job-token-scope-allowlist). + ## Pipeline warnings Pipeline configuration warnings are shown when you: @@ -387,9 +399,9 @@ on shared runners, which reduces system resource usage on the `jobs/request` end When enabled, jobs are processed in the order they were put in the system, instead of balanced across many projects. -### Disable CI/CD minutes quota enforcement +### Disable compute quota enforcement -To disable the enforcement of CI/CD minutes quotas on shared runners, you can temporarily +To disable the enforcement of [compute quotas](pipelines/cicd_minutes.md) on shared runners, you can temporarily enable the `ci_queueing_disaster_recovery_disable_quota` [feature flag](../administration/feature_flags.md). This flag reduces system resource usage on the `jobs/request` endpoint. @@ -401,7 +413,7 @@ Earlier jobs are already canceled by a periodic background worker (`StuckCiJobsW The following commands are run in the [rails console](../administration/operations/rails_console.md#starting-a-rails-console-session). WARNING: -Any command that changes data directly could be damaging if not run correctly, or under the right conditions. +Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case. ### Cancel stuck pending pipelines diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 64a0f1038ad..7b6ba36e35d 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -194,8 +194,9 @@ Prerequisite: To add an instance variable: -1. On the top bar, select **Main menu > Admin**. -1. On the left sidebar, select **Settings > CI/CD** and expand the **Variables** section. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD** and expand the **Variables** section. 1. Select **Add variable** and fill in the details: - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. - **Value**: In [GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/220028), @@ -353,9 +354,9 @@ kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$KU ``` WARNING: -Be careful when assigning the value of a file variable to another variable. The other -variable takes the content of the file as its value, **not** the path to the file. -[Issue 29407](https://gitlab.com/gitlab-org/gitlab/-/issues/29407) proposes to change this behavior. +Be careful when assigning the value of a file variable to another variable in GitLab 15.6 or older. +The other variable takes the content of the file as its value, **not** the path to the file. +In GitLab 15.7 and newer, this behavior [was fixed](https://gitlab.com/gitlab-org/gitlab/-/issues/29407) and the other variable now takes the path to the file as the value. #### Use a `.gitlab-ci.yml` variable as a file type variable @@ -718,6 +719,10 @@ use this setting for control over the environment the pipeline runs in. The example can be copied to your own group or instance for testing. More details on what other GitLab CI patterns are demonstrated are available at the project page. +- You can [pass CI/CD variables to downstream pipelines](../pipelines/downstream_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline). + Use [`trigger:forward` keyword](../yaml/index.md#triggerforward) to specify what type of variables + to pass to the downstream pipeline. + ## Troubleshooting ### List all variables diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 001a599776a..1656b2147d5 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -35,7 +35,7 @@ as it can cause the pipeline to behave unexpectedly. | `CI_COMMIT_DESCRIPTION` | 10.8 | all | The description of the commit. If the title is shorter than 100 characters, the message without the first line. | | `CI_COMMIT_MESSAGE` | 10.8 | all | The full commit message. | | `CI_COMMIT_REF_NAME` | 9.0 | all | The branch or tag name for which project is built. | -| `CI_COMMIT_REF_PROTECTED` | 11.11 | all | `true` if the job is running for a protected reference. | +| `CI_COMMIT_REF_PROTECTED` | 11.11 | all | `true` if the job is running for a protected reference, `false` otherwise. | | `CI_COMMIT_REF_SLUG` | 9.0 | all | `CI_COMMIT_REF_NAME` in lowercase, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | | `CI_COMMIT_SHA` | 9.0 | all | The commit revision the project is built for. | | `CI_COMMIT_SHORT_SHA` | 11.7 | all | The first eight characters of `CI_COMMIT_SHA`. | @@ -110,7 +110,7 @@ as it can cause the pipeline to behave unexpectedly. | `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to push containers to the project's GitLab Container Registry. Only available if the Container Registry is enabled for the project. This password value is the same as the `CI_JOB_TOKEN` and is valid only as long as the job is running. Use the `CI_DEPLOY_PASSWORD` for long-lived access to the registry | | `CI_REGISTRY_USER` | 9.0 | all | The username to push containers to the project's GitLab Container Registry. Only available if the Container Registry is enabled for the project. | | `CI_REGISTRY` | 8.10 | 0.5 | The address of the GitLab Container Registry. Only available if the Container Registry is enabled for the project. This variable includes a `:port` value if one is specified in the registry configuration. | -| `CI_REPOSITORY_URL` | 9.0 | all | The URL to clone the Git repository. | +| `CI_REPOSITORY_URL` | 9.0 | all | The full path to Git clone (HTTP) the repository with a [CI/CD job token](../jobs/ci_job_token.md), in the format `https://gitlab-ci-token:$CI_JOB_TOKEN@gitlab.example.com/my-group/my-project.git`. | | `CI_RUNNER_DESCRIPTION` | 8.10 | 0.5 | The description of the runner. | | `CI_RUNNER_EXECUTABLE_ARCH` | all | 10.6 | The OS/architecture of the GitLab Runner executable. Might not be the same as the environment of the executor. | | `CI_RUNNER_ID` | 8.10 | 0.5 | The unique ID of the runner being used. | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index c20d9be51e7..14dfc5dd551 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -28,10 +28,12 @@ There are two places defined variables can be used. On the: | [`artifacts:name`](../yaml/index.md#artifactsname) | yes | Runner | The variable expansion is made by GitLab Runner's shell environment. | | [`before_script`](../yaml/index.md#before_script) | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment) | | [`cache:key`](../yaml/index.md#cachekey) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| [`cache:policy`](../yaml/index.md#cachepolicy) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | | [`environment:name`](../yaml/index.md#environmentname) | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- `CI_ENVIRONMENT_*` variables.<br/>- [Persisted variables](#persisted-variables). | | [`environment:url`](../yaml/index.md#environmenturl) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | | [`environment:auto_stop_in`](../yaml/index.md#environmentauto_stop_in)| yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/> The value of the variable being substituted should be a period of time in a human readable natural language form. See [possible inputs](../yaml/index.md#environmentauto_stop_in) for more information.| | [`except:variables`](../yaml/index.md#onlyvariables--exceptvariables) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- `CI_ENVIRONMENT_*` variables, except `CI_ENVIRONMENT_NAME` which is supported.<br/>- [Persisted variables](#persisted-variables). | +| [`id_tokens:aud`](../yaml/index.md#id_tokens) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. Variable expansion [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414293) in GitLab 16.1. | | [`image`](../yaml/index.md#image) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | | [`include`](../yaml/index.md#include) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>See [Use variables with include](../yaml/includes.md#use-variables-with-include) for more information on supported variables. | | [`only:variables`](../yaml/index.md#onlyvariables--exceptvariables) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- `CI_ENVIRONMENT_*` variables, except `CI_ENVIRONMENT_NAME` which is supported.<br/>- [Persisted variables](#persisted-variables). | diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index 78c9e98c33f..37cb7efdf94 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -189,7 +189,7 @@ GitLab can display the results of one or more reports in: The `dotenv` report collects a set of environment variables as artifacts. The collected variables are registered as runtime-created variables of the job, -which you can use to [set dynamic environment URLs after a job finishes](../environments/index.md#set-dynamic-environment-urls-after-a-job-finishes). +which you can use to [set dynamic environment URLs after a job finishes](../environments/index.md#set-a-dynamic-environment-url). If duplicate environment variables are present in a `dotenv` report: diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index 35b302d0373..69595b62de2 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -418,6 +418,8 @@ these keywords: ### `include` with `rules:if` +> Support for `when: never` and `when:always` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348146) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `ci_support_include_rules_when_never`. Disabled by default. + Use [`rules:if`](index.md#rulesif) to conditionally include other configuration files based on the status of CI/CD variables. For example: @@ -425,6 +427,14 @@ based on the status of CI/CD variables. For example: include: - local: builds.yml rules: + - if: $DONT_INCLUDE_BUILDS == "true" + when: never + - local: builds.yml + rules: + - if: $ALWAYS_INCLUDE_BUILDS == "true" + when: always + - local: builds.yml + rules: - if: $INCLUDE_BUILDS == "true" - local: deploys.yml rules: @@ -437,6 +447,8 @@ test: ### `include` with `rules:exists` +> Support for `when: never` and `when:always` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348146) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `ci_support_include_rules_when_never`. Disabled by default. + Use [`rules:exists`](index.md#rulesexists) to conditionally include other configuration files based on the existence of files. For example: @@ -445,6 +457,16 @@ include: - local: builds.yml rules: - exists: + - exception-file.md + when: never + - local: builds.yml + rules: + - exists: + - important-file.md + when: always + - local: builds.yml + rules: + - exists: - file.md test: @@ -508,7 +530,7 @@ When the pipeline runs, GitLab: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/391331) in GitLab 15.11 as a Beta feature. FLAG: -`spec` and `with` are experimental [Open Beta features](../../policy/alpha-beta-support.md#beta) +`spec` and `with` are experimental [Open Beta features](../../policy/experiment-beta-support.md#beta) and subject to change without notice. ### Define input parameters with `spec:inputs` diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index ab5226c1c30..3dfc98c043a 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -739,7 +739,7 @@ attached to the job when it [succeeds, fails, or always](#artifactswhen). The artifacts are sent to GitLab after the job finishes. They are available for download in the GitLab UI if the size is smaller than the -the [maximum artifact size](../../user/gitlab_com/index.md#gitlab-cicd). +[maximum artifact size](../../user/gitlab_com/index.md#gitlab-cicd). By default, jobs in later stages automatically download all the artifacts created by jobs in earlier stages. You can control artifact download behavior in jobs with @@ -837,7 +837,6 @@ subdirectories of `binaries/`. > - [Made default behavior](https://gitlab.com/gitlab-org/gitlab/-/issues/229936) in GitLab 13.4. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241026) in GitLab 13.8, keeping latest job artifacts can be disabled at the project level. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276583) in GitLab 13.9, keeping latest job artifacts can be disabled instance-wide. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321323) in GitLab 13.12, the latest pipeline artifacts are kept regardless of expiry time. Use `expire_in` to specify how long [job artifacts](../jobs/job_artifacts.md) are stored before they expire and are deleted. The `expire_in` setting does not affect: @@ -845,9 +844,6 @@ they expire and are deleted. The `expire_in` setting does not affect: - Artifacts from the latest job, unless keeping the latest job artifacts is disabled [at the project level](../jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). or [instance-wide](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). -- [Pipeline artifacts](../pipelines/pipeline_artifacts.md). You can't specify an expiration date for - pipeline artifacts. See [When pipeline artifacts are deleted](../pipelines/pipeline_artifacts.md#when-pipeline-artifacts-are-deleted) - for more information. After their expiry, artifacts are deleted hourly by default (using a cron job), and are not accessible anymore. @@ -965,10 +961,13 @@ job: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223273) in GitLab 13.8 [with a flag](../../user/feature_flags.md) named `non_public_artifacts`, disabled by default. > - [Updated](https://gitlab.com/gitlab-org/gitlab/-/issues/322454) in GitLab 15.10. Artifacts created with `artifacts:public` before 15.10 are not guaranteed to remain private after this update. -FLAG: +WARNING: On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `non_public_artifacts`. On -GitLab.com, this feature is not available. +GitLab.com, this feature is not available. Due to [issue 413822](https://gitlab.com/gitlab-org/gitlab/-/issues/413822), +the keyword can be used when the feature flag is disabled, but the feature does not work. +Do not attempt to use this feature when the feature flag is disabled, and always test +with non-production data first. Use `artifacts:public` to determine whether the job artifacts should be publicly available. @@ -1453,6 +1452,7 @@ Must be used with `cache: paths`, or nothing is cached. - `pull` - `push` - `pull-push` (default) +- [CI/CD variables](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). **Example of `cache:policy`**: @@ -1480,6 +1480,10 @@ faster-test-job: - echo "Running tests..." ``` +**Related topics**: + +- You can [use a variable to control a job's cache policy](../caching/index.md#use-a-variable-to-control-a-jobs-cache-policy). + #### `cache:fallback_keys` Use `cache:fallback_keys` to specify a list of keys to try to restore cache from @@ -1846,7 +1850,7 @@ environment, using the `production` **Related topics**: -- [Available settings for `kubernetes`](../environments/index.md#configure-kubernetes-deployments-deprecated). +- [Available settings for `kubernetes`](../environments/configure_kubernetes_deployments.md). #### `environment:deployment_tier` @@ -2019,7 +2023,10 @@ JWTs created this way support OIDC authentication. The required `aud` sub-keywor **Possible inputs**: -- Token names with their `aud` claims. `aud` can be a single string or as an array of strings. +- Token names with their `aud` claims. `aud` supports: + - A single string. + - An array of strings. + - [CI/CD variables](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). **Example of `id_tokens`**: @@ -4265,7 +4272,7 @@ The keywords available for use in trigger jobs are: - For multi-project pipelines, the path to the downstream project. CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) in GitLab 15.3 and later, but not [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables). - Alternatively, use [`trigger:project](#triggerproject). + Alternatively, use [`trigger:project`](#triggerproject). - For child pipelines, use [`trigger:include`](#triggerinclude). **Example of `trigger`**: diff --git a/doc/ci/yaml/workflow.md b/doc/ci/yaml/workflow.md index 82144e55216..e88a96ae1f5 100644 --- a/doc/ci/yaml/workflow.md +++ b/doc/ci/yaml/workflow.md @@ -129,7 +129,7 @@ workflow: rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" - if: $CI_COMMIT_TAG - - if: $CI_COMMIT_REF_PROTECTED + - if: $CI_COMMIT_REF_PROTECTED == "true" ``` This example assumes that your long-lived branches are [protected](../../user/project/protected_branches.md). |
