diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-01-20 12:16:11 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-01-20 12:16:11 +0300 |
commit | edaa33dee2ff2f7ea3fac488d41558eb5f86d68c (patch) | |
tree | 11f143effbfeba52329fb7afbd05e6e2a3790241 /doc/ci/pipelines | |
parent | d8a5691316400a0f7ec4f83832698f1988eb27c1 (diff) |
Add latest changes from gitlab-org/gitlab@14-7-stable-eev14.7.0-rc42
Diffstat (limited to 'doc/ci/pipelines')
-rw-r--r-- | doc/ci/pipelines/cicd_minutes.md | 221 | ||||
-rw-r--r-- | doc/ci/pipelines/img/group_cicd_minutes_quota.png | bin | 0 -> 21010 bytes | |||
-rw-r--r-- | doc/ci/pipelines/img/pipeline_fork_v13_7.png (renamed from doc/ci/pipelines/img/pipeline-fork_v13_7.png) | bin | 15697 -> 15697 bytes | |||
-rw-r--r-- | doc/ci/pipelines/index.md | 109 | ||||
-rw-r--r-- | doc/ci/pipelines/job_artifacts.md | 24 | ||||
-rw-r--r-- | doc/ci/pipelines/merge_request_pipelines.md | 281 | ||||
-rw-r--r-- | doc/ci/pipelines/merge_trains.md | 8 | ||||
-rw-r--r-- | doc/ci/pipelines/pipelines_for_merged_results.md | 8 | ||||
-rw-r--r-- | doc/ci/pipelines/settings.md | 19 |
9 files changed, 450 insertions, 220 deletions
diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md new file mode 100644 index 00000000000..e0fb5b45986 --- /dev/null +++ b/doc/ci/pipelines/cicd_minutes.md @@ -0,0 +1,221 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference +--- + +# CI/CD minutes quota **(PREMIUM)** + +[Shared runners](../runners/runners_scope.md#shared-runners) are shared with every project and group in a GitLab instance. +When jobs run on shared runners, CI/CD minutes are used. + +You can set limits on the number of CI/CD minutes that are used each month. + +- On GitLab.com, the quota of CI/CD minutes is set for each [namespace](../../user/group/index.md#namespaces), + and is determined by [your license tier](https://about.gitlab.com/pricing/). +- On self-managed GitLab instances, the quota of CI/CD minutes for each namespace is set by administrators. + +In addition to the monthly quota, you can add more CI/CD minutes when needed. + +- On GitLab.com, you can [purchase additional CI/CD minutes](#purchase-additional-cicd-minutes). +- On self-managed GitLab instances, administrators can [assign more CI/CD minutes](#set-the-quota-of-cicd-minutes-for-a-specific-namespace). + +[Specific runners](../runners/runners_scope.md#specific-runners) +are not subject to a quota of CI/CD minutes. + +## Set the quota of CI/CD minutes for all namespaces + +> [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. + +By default, GitLab instances do not have a quota of CI/CD minutes. +The default value for the quota is `0`, which grants unlimited CI/CD minutes. +However, you can change this default value. + +Prerequisite: + +- You must be a GitLab administrator. + +To change the default quota that applies to all namespaces: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Continuous Integration and Deployment**. +1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. +1. Select **Save changes**. + +If a quota is already defined for a specific namespace, this value does not change that quota. + +## Set the quota of CI/CD minutes for a specific namespace + +> [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. + +You can override the global value and set a quota of CI/CD minutes +for a specific namespace. + +Prerequisite: + +- You must be a GitLab administrator. + +To set a quota of CI/CD minutes for a namespace: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Overview > Groups**. +1. For the group you want to update, select **Edit**. +1. In the **Quota of CI/CD minutes** box, enter the maximum number of CI/CD minutes. +1. Select **Save changes**. + +You can also use the [update group API](../../api/groups.md#update-group) or the +[update user API](../../api/users.md#user-modification) instead. + +NOTE: +You can set a quota of CI/CD minutes for only top-level groups or user namespaces. +If you set a quota for a subgroup, it is not used. + +## View CI/CD minutes used by a group + +You can view the number of CI/CD minutes being used by a group. + +Prerequisite: + +- You must have the Owner role for the group. + +To view CI/CD minutes being used for your group: + +1. On the top bar, select **Menu > Groups** and find your group. The group must not be a subgroup. +1. On the left sidebar, select **Settings > Usage Quotas**. +1. Select the **Pipelines** tab. + +![Group CI/CD minutes quota](img/group_cicd_minutes_quota.png) + +## View CI/CD minutes used by a personal namespace + +You can view the number of CI/CD minutes being used by a personal namespace: + +1. On the top bar, in the top right corner, select your avatar. +1. Select **Edit profile**. +1. On the left sidebar, select **Usage Quotas**. + +## Purchase additional CI/CD minutes **(FREE SAAS)** + +If you're using GitLab SaaS, you can purchase additional packs of CI/CD minutes. +These additional CI/CD minutes: + +- Are used only after the monthly quota included in your subscription runs out. +- Are carried over to the next month, if any remain at the end of the month. +- Don't expire. + +If you use more CI/CD minutes than your monthly quota, when you purchase more, +those CI/CD minutes are deducted from your quota. For example, with a GitLab SaaS +Premium license: + +- You have `10,000` monthly minutes. +- You purchase an additional `5,000` minutes. +- Your total limit is `15,000` minutes. + +If you use `13,000` minutes during the month, the next month your additional minutes become +`2,000`. If you use `9,000` minutes during the month, your additional minutes remain the same. + +You can find pricing for additional CI/CD minutes on the +[GitLab Pricing page](https://about.gitlab.com/pricing/). + +### Purchase CI/CD minutes for a group **(FREE SAAS)** + +You can purchase additional CI/CD minutes for your group. +You cannot transfer purchased CI/CD minutes from one group to another, +so be sure to select the correct group. + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > Usage Quotas**. +1. Select **Buy additional minutes**. +1. Complete the details of the transaction. + +After your payment is processed, the additional CI/CD minutes are added to your group +namespace. + +### Purchase CI/CD minutes for a personal namespace **(FREE SAAS)** + +To purchase additional minutes for your personal namespace: + +1. On the top bar, in the top right corner, select your avatar. +1. Select **Edit profile**. +1. On the left sidebar, select **Usage Quotas**. +1. Select **Buy additional minutes**. GitLab redirects you to the Customers Portal. +1. Locate the subscription card that's linked to your personal namespace on GitLab SaaS, select **Buy more CI minutes**, + and complete the details of the transaction. + +After your payment is processed, the additional CI/CD minutes are added to your personal +namespace. + +## How CI/CD minutes are calculated + +CI/CD minutes are calculated based on: + +- The duration the job runs. +- The visibility of the projects where the job runs. + +GitLab uses this formula to calculate CI/CD minutes consumed by a job: + +```plaintext +Job duration * Cost factor +``` + +- **Job duration**: The time, in seconds, that a job took to run on a shared runner. + It does not include time spent in `created` or `pending` status. +- **Cost factor**: A number based on project visibility. + +The number is transformed into minutes and added to the overall quota in the job's top-level namespace. + +For example: + +- A user, `alice`, runs a pipeline under the `gitlab-org` namespace. +- The CI/CD minutes consumed by each job in the pipeline are added to the + overall consumption for the `gitlab-org` namespace, not the `alice` namespace. +- If a pipeline runs for one of the personal projects for `alice`, the CI/CD minutes + are added to the overall consumption for the `alice` namespace. + +### Cost factor + +The cost factor for a job running on a shared runner is: + +- `0.008` for public projects on GitLab SaaS, if [created 2021-07-17 or later](https://gitlab.com/gitlab-org/gitlab/-/issues/332708). + (For every 125 minutes of job time, you accrue 1 CD/CD minute.) +- `0.008` for projects members of GitLab [Open Source program](../../subscriptions/index.md#gitlab-for-open-source). + (For every 125 minutes of job time, you accrue 1 CD/CD minute.) +- `0` for public projects on GitLab self-managed instances, and for GitLab SaaS public projects created before 2021-07-17. +- `1` for internal and private projects. + +### Additional costs on GitLab SaaS + +On GitLab SaaS, shared runners can have different cost factors depending on the cost involved +in executing the runner. For example, a high spec shared runner could be set to have a cost factor of `2`. +Conversely, a shared runner that executes jobs for public projects could have a low cost factor, like `0.008`. + +### Monthly reset of CI/CD minutes + +On the first day of each calendar month, the accumulated usage of CI/CD minutes is reset to `0` +for all namespaces that use shared runners. + +Usage data for the previous month is kept to show historical view of the consumption over time. + +## What happens when you exceed the quota + +When the quota of CI/CD minutes is used for the current month, GitLab stops +processing new jobs. + +- Any non-running job that should be picked by shared runners is automatically dropped. +- Any job being retried is automatically dropped. +- Any running job can be dropped at any point if the overall namespace usage goes over-quota + by a grace period. + +The grace period for running jobs is `1,000` CI/CD minutes. + +Jobs on specific runners are not affected by the quota of CI/CD minutes. + +### GitLab SaaS usage notifications + +On GitLab SaaS an email notification is sent to the namespace owners when: + +- The available CI/CD minutes are below 30% of the quota. +- The available CI/CD minutes are below 5% of the quota. +- All CI/CD minutes have been used. diff --git a/doc/ci/pipelines/img/group_cicd_minutes_quota.png b/doc/ci/pipelines/img/group_cicd_minutes_quota.png Binary files differnew file mode 100644 index 00000000000..318527426bd --- /dev/null +++ b/doc/ci/pipelines/img/group_cicd_minutes_quota.png diff --git a/doc/ci/pipelines/img/pipeline-fork_v13_7.png b/doc/ci/pipelines/img/pipeline_fork_v13_7.png Binary files differindex eb44290aa66..eb44290aa66 100644 --- a/doc/ci/pipelines/img/pipeline-fork_v13_7.png +++ b/doc/ci/pipelines/img/pipeline_fork_v13_7.png diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 24e518b1f69..b873b2ae30f 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -50,10 +50,6 @@ Pipelines can be configured in many different ways: followed by the next stage. - [Directed Acyclic Graph Pipeline (DAG) pipelines](../directed_acyclic_graph/index.md) are based on relationships between jobs and can run more quickly than basic pipelines. -- [Multi-project pipelines](multi_project_pipelines.md) combine pipelines for different projects together. -- [Parent-Child pipelines](parent_child_pipelines.md) break down complex pipelines - into one parent pipeline that can trigger multiple child sub-pipelines, which all - run in the same project and with the same SHA. This pipeline architecture is commonly used for mono-repos. - [Pipelines for Merge Requests](../pipelines/merge_request_pipelines.md) run for merge requests only (rather than for every commit). - [Pipelines for Merged Results](../pipelines/pipelines_for_merged_results.md) @@ -61,6 +57,44 @@ Pipelines can be configured in many different ways: already been merged into the target branch. - [Merge Trains](../pipelines/merge_trains.md) use pipelines for merged results to queue merges one after the other. +- [Parent-Child pipelines](parent_child_pipelines.md) break down complex pipelines + into one parent pipeline that can trigger multiple child sub-pipelines, which all + run in the same project and with the same SHA. This pipeline architecture is commonly used for mono-repos. +- [Multi-project pipelines](multi_project_pipelines.md) combine pipelines for different projects together. + +### How parent-child pipelines compare to multi-project pipelines + +Parent-child pipelines and multi-project pipelines can sometimes be used for similar +purposes, but there are some key differences: + +Parent-child pipelines: + +- Run under the same project, ref, and commit SHA as the parent pipeline. +- Affect the overall status of the ref the pipeline runs against. For example, + if a pipeline fails for the main branch, it's common to say that "main is broken". + The status of child pipelines don't directly affect the status of the ref, unless the child + pipeline is triggered with [`strategy:depend`](../yaml/index.md#triggerstrategy). +- Are automatically canceled if the pipeline is configured with [`interruptible`](../yaml/index.md#interruptible) + when a new pipeline is created for the same ref. +- Display only the parent pipelines in the pipeline index page. Child pipelines are + visible when visiting their parent pipeline's page. +- Are limited to 2 levels of nesting. A parent pipeline can trigger multiple child pipelines, + and those child pipeline can trigger multiple child pipelines (`A -> B -> C`). + +Multi-project pipelines: + +- Are triggered from another pipeline, but the upstream (triggering) pipeline does + not have much control over the downstream (triggered) pipeline. However, it can + choose the ref of the downstream pipeline, and pass CI/CD variables to it. +- Affect the overall status of the ref of the project it runs in, but does not + affect the status of the triggering pipeline's ref, unless it was triggered with + [`strategy:depend`](../yaml/index.md#triggerstrategy). +- Are not automatically canceled in the downstream project when using [`interruptible`](../yaml/index.md#interruptible) + if a new pipeline runs for the same ref in the upstream pipeline. They can be + automatically canceled if a new pipeline is triggered for the same ref on the downstream project. +- Multi-project pipelines are standalone pipelines because they are normal pipelines + that happened to be triggered by an external project. They are all visible on the pipeline index page. +- Are independent, so there are no nesting limits. ## Configure a pipeline @@ -257,14 +291,33 @@ WARNING: Deleting a pipeline expires all pipeline caches, and deletes all related objects, such as builds, logs, artifacts, and triggers. **This action cannot be undone.** -### Pipeline quotas +### Pipeline security on protected branches + +A strict security model is enforced when pipelines are executed on +[protected branches](../../user/project/protected_branches.md). + +The following actions are allowed on protected branches only if the user is +[allowed to merge or push](../../user/project/protected_branches.md) +on that specific branch: + +- Run manual pipelines (using the [Web UI](#run-a-pipeline-manually) or [pipelines API](#pipelines-api)). +- Run scheduled pipelines. +- Run pipelines using triggers. +- Run on-demand DAST scan. +- Trigger manual actions on existing pipelines. +- Retry or cancel existing jobs (using the Web UI or pipelines API). -Each user has a personal pipeline quota that tracks the usage of shared runners in all personal projects. -Each group has a [usage quota](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes) that tracks the usage of shared runners for all projects created within the group. +**Variables** marked as **protected** are accessible only to jobs that +run on protected branches, preventing untrusted users getting unintended access to +sensitive information like deployment credentials and tokens. -When a pipeline is triggered, regardless of who triggered it, the pipeline quota for the project owner's [namespace](../../user/group/index.md#namespaces) is used. In this case, the namespace can be the user or group that owns the project. +**Runners** marked as **protected** can run jobs only on protected +branches, preventing untrusted code from executing on the protected runner and +preserving deployment keys and other credentials from being unintentionally +accessed. In order to ensure that jobs intended to be executed on protected +runners do not use regular runners, they must be tagged accordingly. -#### How pipeline duration is calculated +### How pipeline duration is calculated Total running time for a given pipeline excludes retries and pending (queued) time. @@ -301,44 +354,6 @@ The union of A, B, and C is (1, 4) and (6, 7). Therefore, the total running time (4 - 1) + (7 - 6) => 4 ``` -#### How pipeline quota usage is calculated - -Pipeline quota usage is calculated as the sum of the duration of each individual job. This is slightly different to how pipeline _duration_ is [calculated](#how-pipeline-duration-is-calculated). Pipeline quota usage doesn't consider any overlap of jobs running in parallel. - -For example, a pipeline consists of the following jobs: - -- Job A takes 3 minutes. -- Job B takes 3 minutes. -- Job C takes 2 minutes. - -The pipeline quota usage is the sum of each job's duration. In this example, 8 runner minutes would be used, calculated as: 3 + 3 + 2. - -### Pipeline security on protected branches - -A strict security model is enforced when pipelines are executed on -[protected branches](../../user/project/protected_branches.md). - -The following actions are allowed on protected branches only if the user is -[allowed to merge or push](../../user/project/protected_branches.md) -on that specific branch: - -- Run manual pipelines (using the [Web UI](#run-a-pipeline-manually) or [pipelines API](#pipelines-api)). -- Run scheduled pipelines. -- Run pipelines using triggers. -- Run on-demand DAST scan. -- Trigger manual actions on existing pipelines. -- Retry or cancel existing jobs (using the Web UI or pipelines API). - -**Variables** marked as **protected** are accessible only to jobs that -run on protected branches, preventing untrusted users getting unintended access to -sensitive information like deployment credentials and tokens. - -**Runners** marked as **protected** can run jobs only on protected -branches, preventing untrusted code from executing on the protected runner and -preserving deployment keys and other credentials from being unintentionally -accessed. In order to ensure that jobs intended to be executed on protected -runners do not use regular runners, they must be tagged accordingly. - ## Visualize pipelines Pipelines can be complex structures with many sequential and parallel jobs. diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index e47b6dddc5f..1710c57b36b 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -405,3 +405,27 @@ generated. Check the job log for these messages. If you find no helpful messages, retry the failed job after activating [CI/CD debug logging](../variables/index.md#debug-logging). This logging should provide information to help you investigate further. + +### Error message `Missing /usr/bin/gitlab-runner-helper. Uploading artifacts is disabled.` + +There is a [known issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3068) where setting a CI/CD variable named `DEBUG` can cause artifact uploads to fail. + +To work around this, either use a different variable name or set it inline with `script`: + +```yaml +# This job might fail due to issue gitlab-org/gitlab-runner#3068 +failing_test_job: + variables: + DEBUG: true + script: bin/mycommand + artifacts: + paths: + - bin/results + +# This job does not define a CI/CD variable named `DEBUG` and is not affected by the issue +successful_test_job: + script: DEBUG=true bin/mycommand + artifacts: + paths: + - bin/results +``` diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 85e5b62b0c4..4d7ebc09e6f 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -2,214 +2,197 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, index -last_update: 2019-07-03 --- -# Pipelines for merge requests **(FREE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/15310) in GitLab 11.6. -In a [basic configuration](pipeline_architectures.md#basic-pipelines), GitLab runs a pipeline each time -changes are pushed to a branch. +# Pipelines for merge requests **(FREE)** -If you want the pipeline to run jobs **only** on commits associated with a merge request, -you can use *pipelines for merge requests*. +You can configure your [pipeline](index.md) to run every time you commit changes to a branch. +This type of pipeline is called a *branch pipeline*. -These pipelines are labeled as `detached` in the UI, and they do not have access to [protected variables](../variables/index.md#protect-a-cicd-variable). -Otherwise, these pipelines are the same as other pipelines. +Alternatively, you can configure your pipeline to run every time you make changes to the +source branch for a merge request. This type of pipeline is called a *pipeline for merge requests*. -Pipelines for merge requests can run when you: +Branch pipelines: -- Create a new merge request. -- Commit changes to the source branch for the merge request. -- Select the **Run pipeline** button from the **Pipelines** tab in the merge request. +- Run when you push a new commit to a branch. +- Are the default type of pipeline. +- Have access to [some predefined variables](../variables/predefined_variables.md). +- Have access to [protected variables](../variables/index.md#protect-a-cicd-variable). -If you use this feature with [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md), -pipelines for merge requests take precedence over other pipelines. +Pipelines for merge requests: -## Prerequisites +- Run when you: + - Create a new merge request. + - Push a new commit to the source branch for a merge request. + - Select **Run pipeline** from the **Pipelines** tab in a merge request. This option + is only available when pipelines for merge requests are configured for the pipeline. +- Do not run by default. The jobs in the CI/CD configuration file [must be configured](#prerequisites) + to run in pipelines for merge request. +- Have access to [more predefined variables](#available-predefined-variables). +- Do not have access to [protected variables](../variables/index.md#protect-a-cicd-variable). -To enable pipelines for merge requests: +Both of these types of pipelines can appear on the **Pipelines** tab of a merge request. -- Your repository must be a GitLab repository, not an - [external repository](../ci_cd_for_external_repos/index.md). -- You must have the Developer [role](../../user/permissions.md) - to run a pipeline for merge requests. +## Types of pipelines for merge requests -## Configure pipelines for merge requests +The three types of pipelines for merge requests are: -To configure pipelines for merge requests, you must configure your [CI/CD configuration file](../yaml/index.md). -To do this, you can use [`rules`](#use-rules-to-run-pipelines-for-merge-requests) or [`only/except`](#use-only-or-except-to-run-pipelines-for-merge-requests). +- Pipelines for merge requests, which run on the changes in the merge request's + source branch. These pipelines display a `detached` label to indicate that the + pipeline ran only on the contents of the source branch, ignoring the target branch. +- [Pipelines for merged results](pipelines_for_merged_results.md), which run on + the result of combining the source branch's changes with the target branch. +- [Merge trains](merge_trains.md), which run when merging multiple merge requests + at the same time. The changes from each merge request are combined into the + target branch with the changes in the earlier enqueued merge requests, to ensure + they all work together. -### Use `rules` to run pipelines for merge requests +## Prerequisites -GitLab recommends that you use the `rules` keyword, which is available in -[`workflow:rules` templates](../yaml/workflow.md#workflowrules-templates). +To use pipelines for merge requests: -### Use `only` or `except` to run pipelines for merge requests +- Your project's [CI/CD configuration file](../yaml/index.md) must be configured with + jobs that run in pipelines for merge requests. To do this, you can use: + - [`rules`](#use-rules-to-add-jobs). + - [`only/except`](#use-only-to-add-jobs). +- You must have at least the Developer [role](../../user/permissions.md) in the + source project to run a pipeline for merge requests. +- Your repository must be a GitLab repository, not an [external repository](../ci_cd_for_external_repos/index.md). -You can use the `only/except` keywords. However, with this method, you must specify `only: - merge_requests` for each job. +## Use `rules` to add jobs -In the following example, the pipeline contains a `test` job that is configured to run on merge requests. -The `build` and `deploy` jobs don't have the `only: - merge_requests` keyword, -so they don't run on merge requests. +You can use the [`rules`](../yaml/index.md#rules) keyword to configure jobs to run in +pipelines for merge requests. For example: ```yaml -build: - stage: build - script: ./build - only: - - main - -test: - stage: test - script: ./test - only: - - merge_requests - -deploy: - stage: deploy - script: ./deploy - only: - - main +job1: + script: + - echo "This job runs in pipelines for merge requests" + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' ``` -#### Exclude specific jobs - -When you use `only: [merge_requests]`, only jobs with -that keyword are run in the context of a merge request. No other jobs run. - -However, you can invert this behavior and have all of your jobs run except -for one or two. For example, you might have a pipeline with jobs `A`, `B`, and `C`, and you want: - -- All pipelines to always run `A` and `B`. -- `C` to run only for merge requests. - -To achieve this outcome, configure your `.gitlab-ci.yml` file as follows: +You can also use the [`workflow: rules`](../yaml/index.md#workflowrules) keyword +to configure the entire pipeline to run in pipelines for merge requests. For example: ```yaml -.only-default: &only-default - only: - - main - - merge_requests - - tags +workflow: + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' -A: - <<: *only-default +job1: script: - - ... + - echo "This job runs in pipelines for merge requests" -B: - <<: *only-default +job2: script: - - ... - -C: - script: - - ... - only: - - merge_requests + - echo "This job also runs in pipelines for merge requests" ``` -- `A` and `B` always run, because they get the `only` rule to execute in all cases. -- `C` only runs for merge requests. It doesn't run for any pipeline - except a merge request pipeline. - -In this example, you don't have to add the `only` rule to all of your jobs to make -them always run. You can use this format to set up a Review App, which helps to -save resources. +## Use `only` to add jobs -#### Exclude specific branches - -Branch refs use this format: `refs/heads/my-feature-branch`. -Merge request refs use this format: `refs/merge-requests/:iid/head`. - -Because of this difference, the following configuration does not work as expected: - -```yaml -# Does not exclude a branch named "docs-my-fix"! -test: - only: [merge_requests] - except: [/^docs-/] -``` - -Instead, use the -[`$CI_COMMIT_REF_NAME` predefined environment -variable](../variables/predefined_variables.md) in -combination with -[`only:variables`](../yaml/index.md#onlyvariables--exceptvariables) to -accomplish this behavior: +You can use the [`only`](../yaml/index.md#onlyrefs--exceptrefs) keyword with `merge_requests` +to configure jobs to run in pipelines for merge requests. ```yaml -test: - only: [merge_requests] - except: - variables: - - $CI_COMMIT_REF_NAME =~ /^docs-/ +job1: + script: + - echo "This job runs in pipelines for merge requests" + only: + - merge_requests ``` -## Run pipelines in the parent project for merge requests from a forked project **(PREMIUM)** +## Use with forked projects > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217451) in GitLab 13.3. > - [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9. -By default, external contributors who work in forks can't create pipelines in the -parent project. When a merge request that comes from a fork triggers a pipeline: +External contributors who work in forks can't create pipelines in the parent project. -- The pipeline is created and runs in the fork (source) project, not the parent (target) project. -- The pipeline uses the fork project's CI/CD configuration and resources. +A merge request from a fork that is submitted to the parent project triggers a +pipeline that: -If a pipeline runs in a fork, a **fork** badge appears for the pipeline in the merge request. +- Is created and runs in the fork (source) project, not the parent (target) project. +- Uses the fork project's CI/CD configuration, resources, and project CI/CD variables. -![Pipeline ran in fork](img/pipeline-fork_v13_7.png) +Pipelines for forks display with the **fork** badge in the parent project: -Sometimes parent project members want the pipeline to run in the parent -project. They may want to ensure that the post-merge pipeline passes in the parent project. -For example, a fork project could try to use a corrupted runner that doesn't execute -test scripts properly, but reports a passed pipeline. Reviewers in the parent project -could mistakenly trust the merge request because it passed a faked pipeline. +![Pipeline ran in fork](img/pipeline_fork_v13_7.png) -Parent project members with at least the [Developer role](../../user/permissions.md) -can create pipelines in the parent project for merge requests -from a forked project. In the merge request, go to the **Pipelines** tab and select -**Run pipeline**. +### Run pipelines in the parent project **(PREMIUM)** + +Project members in the parent project can choose to run a pipeline for merge requests +for a merge request submitted from a fork project. This pipeline: + +- Is created and runs in the parent (target) project, not the fork (source) project. +- Uses the CI/CD configuration present in the fork project's branch +- Uses the parent project's CI/CD configuration, resources, and project CI/CD variables. +- Uses the permissions of the parent project member that triggers the pipeline. + +Run pipelines in fork project MRs to ensure that the post-merge pipeline passes in +the parent project. Additionally, if you do not trust the fork project's runner, +running the pipeline in the parent project uses the parent project's trusted runners. WARNING: Fork merge requests can contain malicious code that tries to steal secrets in the -parent project when the pipeline runs, even before merge. As a reviewer, you must carefully +parent project when the pipeline runs, even before merge. As a reviewer, carefully check the changes in the merge request before triggering the pipeline. GitLab shows a warning that you must accept before you can trigger the pipeline. -## Predefined variables available for pipelines for merge requests +Parent project members with at least the [Developer role](../../user/permissions.md) +can create pipelines in the parent project for merge requests from a forked project: -When you use pipelines for merge requests, [additional predefined variables](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) are available to the CI/CD jobs. -These variables contain information from the associated merge request, so that you can -integrate your job with the [GitLab Merge Request API](../../api/merge_requests.md). +1. In the merge request, go to the **Pipelines** tab. +1. Select **Run pipeline**. You must accept the warning, or the pipeline does not run. -## Troubleshooting +## Available predefined variables -### Two pipelines created when pushing to a merge request +When you use pipelines for merge requests, you can use: + +- All the same [predefined variables](../variables/predefined_variables.md) that are + available in branch pipelines. +- [Additional predefined variables](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) + available only to jobs in pipelines for merge requests. These variables contain + information from the associated merge request, which can be when calling the + [GitLab Merge Request API endpoint](../../api/merge_requests.md) from a job. + +## Troubleshooting -If you are experiencing duplicated pipelines when using `rules`, take a look at -the [important differences between `rules` and `only`/`except`](../jobs/job_control.md#avoid-duplicate-pipelines), -which helps you get your starting configuration correct. +### Two pipelines when pushing to a branch -If you are seeing two pipelines when using `only/except`, please see the caveats -related to using `only/except` above (or, consider moving to `rules`). +If you get duplicate pipelines in merge requests, your pipeline might be configured +to run for both branches and merge requests at the same time. Adjust your pipeline +configuration to [avoid duplicate pipelines](../jobs/job_control.md#avoid-duplicate-pipelines). In [GitLab 13.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201845), -you can add `workflow:rules` to [switch from branch pipelines to merge request pipelines](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines). +you can add `workflow:rules` to [switch from branch pipelines to pipelines for merge requests](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines). After a merge request is open on the branch, the pipeline switches to a merge request pipeline. -### Two pipelines created when pushing an invalid CI configuration file +### Two pipelines when pushing an invalid CI/CD configuration file + +If you push an invalid CI/CD configuration to a merge request's branch, two failed +pipelines appear in the pipelines tab. One pipeline is a failed branch pipeline, +the other is a failed pipeline for merge requests. + +When the configuration syntax is fixed, no further failed pipelines should appear. +To find and fix the configuration problem, you can use: + +- The [pipeline editor](../pipeline_editor/index.md). +- The [CI lint tool](../lint.md). + +### The merge request's pipeline is marked as failed but the latest pipeline succeeded + +It's possible to have both branch pipelines and pipelines for merge requests in the +**Pipelines** tab of a single merge request. This might be [by configuration](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines), +or [by accident](#two-pipelines-when-pushing-to-a-branch). -Pushing to a branch with an invalid CI configuration file can trigger -the creation of two types of failed pipelines. One pipeline is a failed merge request -pipeline, and the other is a failed branch pipeline, but both are caused by the same -invalid configuration. +If both types of pipelines are in one merge request, the merge request's pipeline +is not considered successful if: -## Related topics +- The branch pipeline succeeds. +- The pipeline for merge request fails. -- [Pipelines for merged results](pipelines_for_merged_results.md). -- [Merge trains](merge_trains.md). +When using the [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) +feature and both pipelines types are present, the pipelines for merge requests are checked, +not the branch pipelines. diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index 593cdb68b3f..d47cbf5f47c 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -2,8 +2,6 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference -last_update: 2019-07-03 --- # Merge trains **(PREMIUM)** @@ -11,10 +9,6 @@ last_update: 2019-07-03 > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9186) in GitLab 12.0. > - [Squash and merge](../../user/project/merge_requests/squash_and_merge.md) support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13001) in GitLab 12.6. -INFO: -Get merge trains and more in GitLab Ultimate. -[Try a free 30-day trial now](https://about.gitlab.com/free-trial/index.html?glm_source=docs.gitlab.com&glm_content=p-ci-cd-external-docs). - For more information about why you might want to use merge trains, read [How merge trains keep your master green](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/). When [pipelines for merged results](pipelines_for_merged_results.md) are @@ -84,7 +78,7 @@ To enable merge trains: To enable merge trains for your project: 1. If you are on a self-managed GitLab instance, ensure the [feature flag](#merge-trains-feature-flag) is set correctly. -1. [Configure your CI/CD configuration file](merge_request_pipelines.md#configure-pipelines-for-merge-requests) +1. [Configure your CI/CD configuration file](merge_request_pipelines.md#prerequisites) so that the pipeline or individual jobs run for merge requests. 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. diff --git a/doc/ci/pipelines/pipelines_for_merged_results.md b/doc/ci/pipelines/pipelines_for_merged_results.md index 718519faf48..91a49a48882 100644 --- a/doc/ci/pipelines/pipelines_for_merged_results.md +++ b/doc/ci/pipelines/pipelines_for_merged_results.md @@ -2,18 +2,12 @@ stage: Verify group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference -last_update: 2019-07-03 --- # Pipelines for merged results **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7380) in GitLab 11.10. -INFO: -Get these pipelines and more in GitLab Ultimate. -[Try a free 30-day trial now](https://about.gitlab.com/free-trial/index.html?glm_source=docs.gitlab.com&glm_content=p-ci-cd-external-docs). - When you submit a merge request, you are requesting to merge changes from a source branch into a target branch. By default, the CI pipeline runs jobs against the source branch. @@ -61,7 +55,7 @@ To enable pipelines for merge results: To enable pipelines for merged results for your project: -1. [Configure your CI/CD configuration file](merge_request_pipelines.md#configure-pipelines-for-merge-requests) +1. [Configure your CI/CD configuration file](merge_request_pipelines.md#prerequisites) so that the pipeline or individual jobs run for merge requests. 1. Visit your project's **Settings > General** and expand **Merge requests**. 1. Check **Enable merged results pipelines**. diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index cf470836e32..85824dfb7c7 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -21,11 +21,7 @@ For public and internal projects, you can change who can see your: - Pipelines - Job output logs - Job artifacts -- [Pipeline security dashboard](../../user/application_security/security_dashboard/index.md#pipeline-security) - -However: - -- Job output logs and artifacts are [never visible for Guest users and non-project members](https://gitlab.com/gitlab-org/gitlab/-/issues/25649). +- [Pipeline security dashboard](../../user/application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline) To change the visibility of your pipelines and related features: @@ -41,8 +37,10 @@ To change the visibility of your pipelines and related features: When it is cleared: - - For **public** projects, pipelines are visible to everyone. Related features are visible - only to project members (Reporter or higher). + - For **public** projects, job logs, job artifacts, the pipeline security dashboard, + and the **CI/CD** menu items are visible only to project members (Reporter or higher). + Other users, including guest users, can only view the status of pipelines and jobs, and only + when viewing merge requests or commits. - For **internal** projects, pipelines are visible to all logged in users except [external users](../../user/permissions.md#external-users). Related features are visible only to project members (Reporter or higher). - For **private** projects, pipelines and related features are visible to project members (Reporter or higher) only. @@ -161,7 +159,8 @@ in the `.gitlab-ci.yml` file. ## Limit the number of changes fetched during clone -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/28919) in GitLab 12.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/28919) in GitLab 12.0. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77576) `git depth` value in GitLab 14.7. You can limit the number of changes that GitLab CI/CD fetches when it clones a repository. @@ -173,8 +172,8 @@ a repository. The maximum value is `1000`. To disable shallow clone and make GitLab CI/CD fetch all branches and tags each time, keep the value empty or set to `0`. -In GitLab 12.0 and later, newly created projects automatically have a default -`git depth` value of `50`. +In GitLab versions 14.7 and later, newly created projects have a default `git depth` +value of `20`. GitLab versions 14.6 and earlier have a default `git depth` value of `50`. This value can be overridden by the [`GIT_DEPTH` variable](../large_repositories/index.md#shallow-cloning) in the `.gitlab-ci.yml` file. |