diff options
Diffstat (limited to 'doc/ci/pipelines')
| -rw-r--r-- | doc/ci/pipelines/cicd_minutes.md | 6 | ||||
| -rw-r--r-- | doc/ci/pipelines/img/merged_result_pipeline.png | bin | 7374 -> 0 bytes | |||
| -rw-r--r-- | doc/ci/pipelines/index.md | 14 | ||||
| -rw-r--r-- | doc/ci/pipelines/job_artifacts.md | 10 | ||||
| -rw-r--r-- | doc/ci/pipelines/merge_request_pipelines.md | 68 | ||||
| -rw-r--r-- | doc/ci/pipelines/merge_trains.md | 43 | ||||
| -rw-r--r-- | doc/ci/pipelines/merged_results_pipelines.md | 78 | ||||
| -rw-r--r-- | doc/ci/pipelines/multi_project_pipelines.md | 2 | ||||
| -rw-r--r-- | doc/ci/pipelines/parent_child_pipelines.md | 2 | ||||
| -rw-r--r-- | doc/ci/pipelines/pipeline_artifacts.md | 3 | ||||
| -rw-r--r-- | doc/ci/pipelines/pipelines_for_merged_results.md | 134 | ||||
| -rw-r--r-- | doc/ci/pipelines/schedules.md | 24 | ||||
| -rw-r--r-- | doc/ci/pipelines/settings.md | 48 |
13 files changed, 224 insertions, 208 deletions
diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index e0fb5b45986..8b10a74bd78 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -149,7 +149,7 @@ namespace. ## How CI/CD minutes are calculated -CI/CD minutes are calculated based on: +CI/CD minutes for individual jobs are calculated based on: - The duration the job runs. - The visibility of the projects where the job runs. @@ -174,6 +174,10 @@ For example: - If a pipeline runs for one of the personal projects for `alice`, the CI/CD minutes are added to the overall consumption for the `alice` namespace. +The CI/CD minutes used by one pipeline is the total CI/CD minutes used by all the jobs +that ran in the pipeline. The CI/CD minute usage for a pipeline can be higher than +the duration of the pipeline if many jobs ran at the same time. + ### Cost factor The cost factor for a job running on a shared runner is: diff --git a/doc/ci/pipelines/img/merged_result_pipeline.png b/doc/ci/pipelines/img/merged_result_pipeline.png Binary files differdeleted file mode 100644 index 2584cd4d38d..00000000000 --- a/doc/ci/pipelines/img/merged_result_pipeline.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index b873b2ae30f..5a5fd2b5540 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -50,14 +50,14 @@ Pipelines can be configured in many different ways: followed by the next stage. - [Directed Acyclic Graph Pipeline (DAG) pipelines](../directed_acyclic_graph/index.md) are based on relationships between jobs and can run more quickly than basic pipelines. -- [Pipelines for Merge Requests](../pipelines/merge_request_pipelines.md) run for merge +- [Merge request pipelines](../pipelines/merge_request_pipelines.md) run for merge requests only (rather than for every commit). -- [Pipelines for Merged Results](../pipelines/pipelines_for_merged_results.md) +- [Merged results pipelines](../pipelines/merged_results_pipelines.md) are merge request pipelines that act as though the changes from the source branch have already been merged into the target branch. -- [Merge Trains](../pipelines/merge_trains.md) - use pipelines for merged results to queue merges one after the other. -- [Parent-Child pipelines](parent_child_pipelines.md) break down complex pipelines +- [Merge trains](../pipelines/merge_trains.md) + use merged results pipelines to queue merges one after the other. +- [Parent-child pipelines](parent_child_pipelines.md) break down complex pipelines into one parent pipeline that can trigger multiple child sub-pipelines, which all run in the same project and with the same SHA. This pipeline architecture is commonly used for mono-repos. - [Multi-project pipelines](multi_project_pipelines.md) combine pipelines for different projects together. @@ -123,7 +123,7 @@ This table lists the refspecs injected for each pipeline type: |--------------- |---------------------------------------- | | pipeline for branches | `+<sha>:refs/pipelines/<id>` and `+refs/heads/<name>:refs/remotes/origin/<name>` | | pipeline for tags | `+<sha>:refs/pipelines/<id>` and `+refs/tags/<name>:refs/tags/<name>` | -| [pipeline for merge requests](../pipelines/merge_request_pipelines.md) | `+<sha>:refs/pipelines/<id>` | +| [merge request pipeline](../pipelines/merge_request_pipelines.md) | `+<sha>:refs/pipelines/<id>` | The refs `refs/heads/<name>` and `refs/tags/<name>` exist in your project repository. GitLab generates the special ref `refs/pipelines/<id>` during a @@ -281,7 +281,7 @@ pipelines. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24851) in GitLab 12.7. -Users with the [Owner role](../../user/permissions.md) in a project can delete a pipeline +Users with the Owner role for a project can delete a pipeline by clicking on the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details** page, then using the **Delete** button. diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index 1710c57b36b..fa8041671dc 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -1,9 +1,8 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/job_artifacts.html' -type: reference, howto --- # Job artifacts **(FREE)** @@ -194,7 +193,8 @@ artifacts: ### Add untracked files to artifacts Use [`artifacts:untracked`](../yaml/index.md#artifactsuntracked) to add all Git untracked -files as artifacts (along with the paths defined in [`artifacts:paths`](../yaml/index.md#artifactspaths)). +files as artifacts (along with the paths defined in [`artifacts:paths`](../yaml/index.md#artifactspaths)). Untracked +files are those that haven't been added to the repository but exist in the repository checkout. Save all Git untracked files and files in `binaries`: @@ -259,7 +259,7 @@ You can delete a single job, which also removes the job's artifacts and log. You must be: - The owner of the job. -- A [maintainer](../../user/permissions.md#gitlab-cicd-permissions) of the project. +- A user with at least the Maintainer role for the project. To delete a job: @@ -381,7 +381,7 @@ to the `expire_in` specification. If a new pipeline for the same ref completes successfully, the previous pipeline's artifacts are deleted according to the `expire_in` configuration. The artifacts -of the new pipeline are kept automatically. +of the new pipeline are kept automatically. Keeping the latest artifacts can use a large amount of storage space in projects with a lot of jobs or large artifacts. If the latest artifacts are not needed in diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 4d7ebc09e6f..dcc3e7e6919 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -5,13 +5,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w --- -# Pipelines for merge requests **(FREE)** +# Merge request pipelines **(FREE)** + +> [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/351192) from `pipelines for merge requests` to `merge request pipelines` in GitLab 14.8. You can configure your [pipeline](index.md) to run every time you commit changes to a branch. This type of pipeline is called a *branch pipeline*. Alternatively, you can configure your pipeline to run every time you make changes to the -source branch for a merge request. This type of pipeline is called a *pipeline for merge requests*. +source branch for a merge request. This type of pipeline is called a *merge request pipeline*. Branch pipelines: @@ -20,28 +22,28 @@ Branch pipelines: - Have access to [some predefined variables](../variables/predefined_variables.md). - Have access to [protected variables](../variables/index.md#protect-a-cicd-variable). -Pipelines for merge requests: +Merge request pipelines: - Run when you: - Create a new merge request. - Push a new commit to the source branch for a merge request. - Select **Run pipeline** from the **Pipelines** tab in a merge request. This option - is only available when pipelines for merge requests are configured for the pipeline. + is only available when merge request pipelines are configured for the pipeline. - Do not run by default. The jobs in the CI/CD configuration file [must be configured](#prerequisites) - to run in pipelines for merge request. + to run in merge request pipelines. - Have access to [more predefined variables](#available-predefined-variables). - Do not have access to [protected variables](../variables/index.md#protect-a-cicd-variable). Both of these types of pipelines can appear on the **Pipelines** tab of a merge request. -## Types of pipelines for merge requests +## Types of merge request pipelines -The three types of pipelines for merge requests are: +The three types of merge request pipelines are: -- Pipelines for merge requests, which run on the changes in the merge request's +- Merge request pipelines, which run on the changes in the merge request's source branch. These pipelines display a `detached` label to indicate that the pipeline ran only on the contents of the source branch, ignoring the target branch. -- [Pipelines for merged results](pipelines_for_merged_results.md), which run on +- [Merged results pipelines](merged_results_pipelines.md), which run on the result of combining the source branch's changes with the target branch. - [Merge trains](merge_trains.md), which run when merging multiple merge requests at the same time. The changes from each merge request are combined into the @@ -50,31 +52,31 @@ The three types of pipelines for merge requests are: ## Prerequisites -To use pipelines for merge requests: +To use merge request pipelines: - Your project's [CI/CD configuration file](../yaml/index.md) must be configured with - jobs that run in pipelines for merge requests. To do this, you can use: + jobs that run in merge request pipelines. To do this, you can use: - [`rules`](#use-rules-to-add-jobs). - [`only/except`](#use-only-to-add-jobs). -- You must have at least the Developer [role](../../user/permissions.md) in the - source project to run a pipeline for merge requests. +- You must have at least the Developer role in the + source project to run a merge request pipeline. - Your repository must be a GitLab repository, not an [external repository](../ci_cd_for_external_repos/index.md). ## Use `rules` to add jobs You can use the [`rules`](../yaml/index.md#rules) keyword to configure jobs to run in -pipelines for merge requests. For example: +merge request pipelines. For example: ```yaml job1: script: - - echo "This job runs in pipelines for merge requests" + - echo "This job runs in merge request pipelines" rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' ``` You can also use the [`workflow: rules`](../yaml/index.md#workflowrules) keyword -to configure the entire pipeline to run in pipelines for merge requests. For example: +to configure the entire pipeline to run in merge request pipelines. For example: ```yaml workflow: @@ -83,22 +85,22 @@ workflow: job1: script: - - echo "This job runs in pipelines for merge requests" + - echo "This job runs in merge request pipelines" job2: script: - - echo "This job also runs in pipelines for merge requests" + - echo "This job also runs in merge request pipelines" ``` ## Use `only` to add jobs You can use the [`only`](../yaml/index.md#onlyrefs--exceptrefs) keyword with `merge_requests` -to configure jobs to run in pipelines for merge requests. +to configure jobs to run in merge request pipelines. ```yaml job1: script: - - echo "This job runs in pipelines for merge requests" + - echo "This job runs in merge request pipelines" only: - merge_requests ``` @@ -122,7 +124,7 @@ Pipelines for forks display with the **fork** badge in the parent project: ### Run pipelines in the parent project **(PREMIUM)** -Project members in the parent project can choose to run a pipeline for merge requests +Project members in the parent project can choose to run a merge request pipeline for a merge request submitted from a fork project. This pipeline: - Is created and runs in the parent (target) project, not the fork (source) project. @@ -140,20 +142,26 @@ parent project when the pipeline runs, even before merge. As a reviewer, careful check the changes in the merge request before triggering the pipeline. GitLab shows a warning that you must accept before you can trigger the pipeline. -Parent project members with at least the [Developer role](../../user/permissions.md) -can create pipelines in the parent project for merge requests from a forked project: +Prerequisites: + +- You must be a member of the parent project and have at least the [Developer role](../../user/permissions.md). +- The fork project must be [visible](../../public_access/public_access.md) to the + user running the pipeline. Otherwise, the **Pipelines** tab does not display + in the merge request. + +To run a pipeline in the parent project for a merge request from a fork project: 1. In the merge request, go to the **Pipelines** tab. 1. Select **Run pipeline**. You must accept the warning, or the pipeline does not run. ## Available predefined variables -When you use pipelines for merge requests, you can use: +When you use merge request pipelines, you can use: - All the same [predefined variables](../variables/predefined_variables.md) that are available in branch pipelines. - [Additional predefined variables](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) - available only to jobs in pipelines for merge requests. These variables contain + available only to jobs in merge request pipelines. These variables contain information from the associated merge request, which can be when calling the [GitLab Merge Request API endpoint](../../api/merge_requests.md) from a job. @@ -166,14 +174,14 @@ to run for both branches and merge requests at the same time. Adjust your pipeli configuration to [avoid duplicate pipelines](../jobs/job_control.md#avoid-duplicate-pipelines). In [GitLab 13.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201845), -you can add `workflow:rules` to [switch from branch pipelines to pipelines for merge requests](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines). +you can add `workflow:rules` to [switch from branch pipelines to merge request pipelines](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines). After a merge request is open on the branch, the pipeline switches to a merge request pipeline. ### Two pipelines when pushing an invalid CI/CD configuration file If you push an invalid CI/CD configuration to a merge request's branch, two failed pipelines appear in the pipelines tab. One pipeline is a failed branch pipeline, -the other is a failed pipeline for merge requests. +the other is a failed merge request pipeline. When the configuration syntax is fixed, no further failed pipelines should appear. To find and fix the configuration problem, you can use: @@ -183,7 +191,7 @@ To find and fix the configuration problem, you can use: ### The merge request's pipeline is marked as failed but the latest pipeline succeeded -It's possible to have both branch pipelines and pipelines for merge requests in the +It's possible to have both branch pipelines and merge request pipelines in the **Pipelines** tab of a single merge request. This might be [by configuration](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines), or [by accident](#two-pipelines-when-pushing-to-a-branch). @@ -191,8 +199,8 @@ If both types of pipelines are in one merge request, the merge request's pipelin is not considered successful if: - The branch pipeline succeeds. -- The pipeline for merge request fails. +- The merge request pipeline fails. When using the [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) -feature and both pipelines types are present, the pipelines for merge requests are checked, +feature and both pipelines types are present, the merge request pipelines are checked, not the branch pipelines. diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index d47cbf5f47c..ffcce06cfbd 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w For more information about why you might want to use merge trains, read [How merge trains keep your master green](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/). -When [pipelines for merged results](pipelines_for_merged_results.md) are +When [merged results pipelines](merged_results_pipelines.md) are enabled, the pipeline jobs run as if the changes from your source branch have already been merged into the target branch. @@ -67,7 +67,7 @@ branch](https://www.youtube.com/watch?v=D4qCqXgZkHQ). To enable merge trains: -- You must have the [Maintainer role](../../user/permissions.md). +- You must have the Maintainer role. - You must be using [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) 11.9 or later. - In GitLab 13.0 and later, you need [Redis](https://redis.io/) 5.0 or later. - Your repository must be a GitLab repository, not an @@ -93,7 +93,7 @@ In GitLab 13.5 and earlier, there is only one checkbox, named WARNING: If you select the check box but don't configure your CI/CD to use -pipelines for merge requests, your merge requests may become stuck in an +merge request pipelines, your merge requests may become stuck in an unresolved state or your pipelines may be dropped. ## Start a merge train @@ -145,8 +145,22 @@ This is the fastest option to get the change merged into the target branch.  WARNING: -Each time you merge a merge request immediately, the current merge train -is recreated and all pipelines restart. +Each time you merge a merge request immediately, the current merge train is recreated, +all pipelines restart, and [redundant pipelines are cancelled](#automatic-pipeline-cancellation). + +### Automatic pipeline cancellation + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12996) in GitLab 12.3. + +GitLab CI/CD can detect the presence of redundant pipelines, and cancels them +to conserve CI resources. + +When a user merges a merge request immediately in an ongoing merge +train, the train is reconstructed, because it recreates the expected +post-merge commit and pipeline. In this case, the merge train may already +have pipelines running against the previous expected post-merge commit. +These pipelines are considered redundant and are automatically +canceled. ## Troubleshooting @@ -179,8 +193,8 @@ for more information. ### Merge train pipeline cannot be retried -When a pipeline for merge trains fails the merge request is dropped from the train and the pipeline can't be retried. -Pipelines for merge trains run on the merged result of the changes in the merge request and +When a merge train pipeline fails, the merge request is dropped from the train and the pipeline can't be retried. +Merge train pipelines run on the merged result of the changes in the merge request and the changes from other merge requests already on the train. If the merge request is dropped from the train, the merged result is out of date and the pipeline can't be retried. @@ -197,11 +211,16 @@ is enabled in **Settings > General > Merge requests**. This option requires that run a new successful pipeline before you can re-add a merge request to a merge train. Merge trains ensure that each pipeline has succeeded before a merge happens, so -you can clear the **Pipelines must succeed** checkbox and keep -**Enable merge trains and pipelines for merged results** (merge trains) selected. +you can: + +- Clear the **Pipelines must succeed** checkbox. +- Select the **Enable merged results pipelines** and **Enable merge trains** checkboxes. + + In GitLab 13.5 and earlier, there is only one checkbox, named + **Enable merge trains and pipelines for merged results**. If you want to keep the **Pipelines must succeed** option selected along with merge -trains, create a new pipeline for merged results when this error occurs: +trains, create a new merged results pipeline when this error occurs: 1. On the **Pipelines** tab, select **Run pipeline**. 1. Select **Start/Add to merge train when pipeline succeeds**. @@ -215,8 +234,8 @@ In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/244831) you can [enable or disable merge trains in the project settings](#enable-merge-trains). In GitLab 13.5 and earlier, merge trains are automatically enabled when -[pipelines for merged results](pipelines_for_merged_results.md) are enabled. -To use pipelines for merged results without using merge trains, you can enable a +[merged results pipelines](merged_results_pipelines.md) are enabled. +To use merged results pipelines without using merge trains, you can enable a [feature flag](../../user/feature_flags.md) that blocks the merge trains feature. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md new file mode 100644 index 00000000000..4794107cc87 --- /dev/null +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -0,0 +1,78 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Merged results pipelines **(PREMIUM)** + +> [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/351192) from `pipelines for merged results` to `merged results pipelines` in GitLab 14.8. + +A *merged results pipeline* is a type of [merge request pipeline](merge_request_pipelines.md). It is a pipeline that runs against the results of the source and target branches merged together. + +GitLab creates an internal commit with the merged results, so the pipeline can run +against it. This commit does not exist in either branch, +but you can view it in the pipeline details. + +The pipeline runs against the target branch as it exists at the moment you run the pipeline. +Over time, while you're working in the source branch, the target branch might change. +Any time you want to be sure the merged results are accurate, you should re-run the pipeline. + +Merged results pipelines can't run when: + +- The target branch has changes that conflict with the changes in the source branch. +- The merge request is a [**Draft** merge request](../../user/project/merge_requests/drafts.md). + +In these cases, the pipeline runs as a [merge request pipeline](merge_request_pipelines.md) +and is labeled as `detached`. + +## Prerequisites + +To use merged results pipelines: + +- Your project's [CI/CD configuration file](../yaml/index.md) must be configured to + [run jobs in merge request pipelines](merge_request_pipelines.md#prerequisites). +- Your repository must be a GitLab repository, not an + [external repository](../ci_cd_for_external_repos/index.md). +- You must not be using [fast forward merges](../../user/project/merge_requests/fast_forward_merge.md). + [An issue exits](https://gitlab.com/gitlab-org/gitlab/-/issues/26996) to change this behavior. + +## Enable merged results pipelines + +To enable merged results pipelines in a project, you must have at least the +Maintainer role: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Merge requests**. +1. Select **Enable merged results pipelines**. +1. Select **Save changes**. + +WARNING: +If you select the checkbox but don't configure your pipeline to use +merge request pipelines, your merge requests may become stuck in an +unresolved state or your pipelines may be dropped. + +## Troubleshooting + +### Merged results pipelines are not created + +In GitLab 13.7 and earlier, merged results pipelines might not be created due +to a disabled [feature flag](../../user/feature_flags.md). This feature flag +[was removed](https://gitlab.com/gitlab-org/gitlab/-/issues/299115) in GitLab 13.8. +Upgrade to 13.8 or later, or make sure the `:merge_ref_auto_sync` +[feature flag is enabled](../../administration/feature_flags.md#check-if-a-feature-flag-is-enabled) +on your GitLab instance. + +### Pipelines fail intermittently with a `fatal: reference is not a tree:` error + +Merged results pipelines run on a merge ref for a merge request +(`refs/merge-requests/<iid>/merge`), so the Git reference could be overwritten at an +unexpected time. + +For example, when a source or target branch is advanced, the pipeline fails with +the `fatal: reference is not a tree:` error, which indicates that the checkout-SHA +is not found in the merge ref. + +This behavior was improved in GitLab 12.4 by introducing [persistent pipeline refs](../troubleshooting.md#fatal-reference-is-not-a-tree-error). +Upgrade to GitLab 12.4 or later to resolve the problem. diff --git a/doc/ci/pipelines/multi_project_pipelines.md b/doc/ci/pipelines/multi_project_pipelines.md index 8a83e7e31f4..2163527e3ca 100644 --- a/doc/ci/pipelines/multi_project_pipelines.md +++ b/doc/ci/pipelines/multi_project_pipelines.md @@ -283,7 +283,7 @@ tag in a different project. Prerequisites: - The upstream project must be [public](../../public_access/public_access.md). -- The user must have the [Developer role](../../user/permissions.md#project-members-permissions) +- The user must have the Developer role in the upstream project. To trigger the pipeline when the upstream project is rebuilt: diff --git a/doc/ci/pipelines/parent_child_pipelines.md b/doc/ci/pipelines/parent_child_pipelines.md index 5e4b707a38c..a3ded24e8c9 100644 --- a/doc/ci/pipelines/parent_child_pipelines.md +++ b/doc/ci/pipelines/parent_child_pipelines.md @@ -100,7 +100,7 @@ microservice_a: ## Merge request child pipelines -To trigger a child pipeline as a [Merge Request Pipeline](merge_request_pipelines.md) we need to: +To trigger a child pipeline as a [merge request pipeline](merge_request_pipelines.md) we need to: - Set the trigger job to run on merge requests: diff --git a/doc/ci/pipelines/pipeline_artifacts.md b/doc/ci/pipelines/pipeline_artifacts.md index b174b6af9f9..e9dd1b2a942 100644 --- a/doc/ci/pipelines/pipeline_artifacts.md +++ b/doc/ci/pipelines/pipeline_artifacts.md @@ -1,8 +1,7 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, howto --- # Pipeline artifacts **(FREE)** diff --git a/doc/ci/pipelines/pipelines_for_merged_results.md b/doc/ci/pipelines/pipelines_for_merged_results.md index 91a49a48882..457f5ce9c30 100644 --- a/doc/ci/pipelines/pipelines_for_merged_results.md +++ b/doc/ci/pipelines/pipelines_for_merged_results.md @@ -1,133 +1,9 @@ --- -stage: Verify -group: Pipeline Execution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'merged_results_pipelines.md' +remove_date: '2022-04-27' --- -# Pipelines for merged results **(PREMIUM)** +This document was moved to [another location](merged_results_pipelines.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7380) in GitLab 11.10. - -When you submit a merge request, you are requesting to merge changes from a -source branch into a target branch. By default, the CI pipeline runs jobs -against the source branch. - -With *pipelines for merged results*, the pipeline runs as if the changes from -the source branch have already been merged into the target branch. The commit shown for the pipeline does not exist on the source or target branches but represents the combined target and source branches. - - - -If the pipeline fails due to a problem in the target branch, you can wait until the -target is fixed and re-run the pipeline. -This new pipeline runs as if the source is merged with the updated target, and you -don't need to rebase. - -The pipeline does not automatically run when the target branch changes. Only changes -to the source branch trigger a new pipeline. If a long time has passed since the last successful -pipeline, you may want to re-run it before merge, to ensure that the source changes -can still be successfully merged into the target. - -When the merge request can't be merged, the pipeline runs against the source branch only. For example, when: - -- The target branch has changes that conflict with the changes in the source branch. -- The merge request is a [**Draft** merge request](../../user/project/merge_requests/drafts.md). - -In these cases, the pipeline runs as a [pipeline for merge requests](merge_request_pipelines.md) -and is labeled as `detached`. If these cases no longer exist, new pipelines -again run against the merged results. - -Any user who has developer [permissions](../../user/permissions.md) can run a -pipeline for merged results. - -## Prerequisites - -To enable pipelines for merge results: - -- You must have the [Maintainer role](../../user/permissions.md). -- You must be using [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) 11.9 or later. -- You must not be using - [fast forward merges](../../user/project/merge_requests/fast_forward_merge.md) yet. - To follow progress, see [#26996](https://gitlab.com/gitlab-org/gitlab/-/issues/26996). -- Your repository must be a GitLab repository, not an - [external repository](../ci_cd_for_external_repos/index.md). - -## Enable pipelines for merged results - -To enable pipelines for merged results for your project: - -1. [Configure your CI/CD configuration file](merge_request_pipelines.md#prerequisites) - so that the pipeline or individual jobs run for merge requests. -1. Visit your project's **Settings > General** and expand **Merge requests**. -1. Check **Enable merged results pipelines**. -1. Click **Save changes**. - -WARNING: -If you select the checkbox but don't configure your CI/CD to use -pipelines for merge requests, your merge requests may become stuck in an -unresolved state or your pipelines may be dropped. - -## Using Merge Trains - -When you enable [Pipelines for merged results](#pipelines-for-merged-results), -GitLab [automatically displays](merge_trains.md#add-a-merge-request-to-a-merge-train) -a **Start/Add Merge Train button**. - -Generally, this is a safer option than merging merge requests immediately, because your -merge request is evaluated with an expected post-merge result before the actual -merge happens. - -For more information, read the [documentation on Merge Trains](merge_trains.md). - -## Automatic pipeline cancellation - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12996) in GitLab 12.3. - -GitLab CI/CD can detect the presence of redundant pipelines, and cancels them -to conserve CI resources. - -When a user merges a merge request immediately in an ongoing merge -train, the train is reconstructed, because it recreates the expected -post-merge commit and pipeline. In this case, the merge train may already -have pipelines running against the previous expected post-merge commit. -These pipelines are considered redundant and are automatically -canceled. - -## Troubleshooting - -### Pipelines for merged results not created even with new change pushed to merge request - -Can be caused by some disabled feature flags. Please make sure that -the following feature flags are enabled on your GitLab instance: - -- `:merge_ref_auto_sync` - -To check and set these feature flag values, please ask an administrator to: - -1. Log into the Rails console of the GitLab instance: - - ```shell - sudo gitlab-rails console - ``` - -1. Check if the flags are enabled or not: - - ```ruby - Feature.enabled?(:merge_ref_auto_sync) - ``` - -1. If needed, enable the feature flags: - - ```ruby - Feature.enable(:merge_ref_auto_sync) - ``` - -### Intermittently pipelines fail by `fatal: reference is not a tree:` error - -Since pipelines for merged results are a run on a merge ref of a merge request -(`refs/merge-requests/<iid>/merge`), the Git reference could be overwritten at an -unexpected timing. For example, when a source or target branch is advanced. -In this case, the pipeline fails because of `fatal: reference is not a tree:` error, -which indicates that the checkout-SHA is not found in the merge ref. - -This behavior was improved at GitLab 12.4 by introducing [Persistent pipeline refs](../troubleshooting.md#fatal-reference-is-not-a-tree-error). -You should be able to create pipelines at any timings without concerning the error. +<!-- This redirect file can be deleted after 2022-04-27. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 692460120fe..fe9db3306cd 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -17,7 +17,9 @@ Pipeline schedules can be used to also run [pipelines](index.md) at specific int - Every other Sunday at 0900 hours (cron example: `0 9 * * sun%2`). - Once every day (cron example: `0 0 * * *`). -Schedule timing is configured with cron notation, parsed by [Fugit](https://github.com/floraison/fugit). +Schedule timing is configured with [cron notation](../../topics/cron/index.md). +You can use any cron value, but scheduled pipelines cannot run more frequently +than the instance's [maximum frequency for scheduled pipelines](#advanced-configuration). In addition to using the GitLab UI, pipeline schedules can be maintained using the [Pipeline schedules API](../../api/pipeline_schedules.md). @@ -82,20 +84,24 @@ job: ### Advanced configuration **(FREE SELF)** -The pipelines are not executed exactly on schedule because schedules are handled by -Sidekiq, which runs according to its interval. +Scheduled pipelines can be configured with any [cron value](../../topics/cron/index.md), +but they do not always run exactly when scheduled. An internal process, called the +_pipeline schedule worker_, queues all the scheduled pipelines, but does not +run continuously. The worker runs on its own schedule, and scheduled pipelines that +are ready to start are only queued the next time the worker runs. Scheduled pipelines +can't run more frequently than the worker. -For example, only two pipelines are created per day if: +The default frequency of the pipeline schedule worker is `3-59/10 * * * *` (every ten minutes, +starting with `0:03`, `0:13`, `0:23`, and so on). The default frequency for GitLab.com +is listed in the [GitLab.com settings](../../user/gitlab_com/index.md#gitlab-cicd). -- You set a schedule to create a pipeline every minute (`* * * * *`). -- The Sidekiq worker runs on 00:00 and 12:00 every day (`0 */12 * * *`). - -To change the Sidekiq worker's frequency: +To change the frequency of the pipeline schedule worker: 1. Edit the `gitlab_rails['pipeline_schedule_worker_cron']` value in your instance's `gitlab.rb` file. 1. [Reconfigure GitLab](../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -For GitLab.com, refer to the [dedicated settings page](../../user/gitlab_com/index.md#gitlab-cicd). +For example, to set the maximum frequency of pipelines to twice a day, set `pipeline_schedule_worker_cron` +to a cron value of `0 */12 * * *` (`00:00` and `12:00` every day). ## Working with scheduled pipelines diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 85824dfb7c7..e22746dbfa0 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -108,6 +108,10 @@ To customize the path: - Is on an external site, enter the full URL. 1. Select **Save changes**. +NOTE: +You cannot use your project's [pipeline editor](../pipeline_editor/index.md) to +edit CI/CD configuration files in other projects or on an external site. + ### Custom CI/CD configuration file examples If the CI/CD configuration file is not in the root directory, the path must be relative to it. @@ -191,21 +195,18 @@ Jobs that exceed the timeout are marked as failed. You can override this value [for individual runners](../runners/configure_runners.md#set-maximum-job-timeout-for-a-runner). -## Add test coverage results to a merge request +## Add test coverage results to a merge request (DEPRECATED) + +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 14.9. Replaced by [`coverage` keyword](../yaml/index.md#coverage). + +WARNING: +This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) +for use in GitLab 14.9, and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0. If you use test coverage in your code, you can use a regular expression to find coverage results in the job log. You can then include these results in the merge request in GitLab. -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. -1. Expand **General pipelines**. -1. In the **Test coverage parsing** field, enter a regular expression. - Leave blank to disable this feature. - -You can use <https://rubular.com> to test your regex. The regex returns the **last** -match found in the output. - If the pipeline succeeds, the coverage is shown in the merge request widget and in the jobs table. If multiple jobs in the pipeline have coverage reports, they are averaged. @@ -214,6 +215,31 @@ averaged.  +To define a coverage-parsing regular expression: + +- Using the project's `.gitlab-ci.yml`, provide a regular expression using the [`coverage`](../yaml/index.md#coverage) + keyword. Setting the regular expression this way takes precedence over the project's CI/CD settings. + +- Using the Project's CI/CD settings: + - Set using the GitLab UI: + + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Settings > CI/CD**. + 1. Expand **General pipelines**. + 1. In the **Test coverage parsing** field, enter a regular expression. Leave blank to disable this feature. + + - Set when [editing a project](../../api/projects.md#edit-project) or [creating a project](../../api/projects.md#create-project) + using the GitLab API with the `build_coverage_regex` attribute: + + ```shell + curl --request PUT --header "PRIVATE-TOKEN: <your-token>" \ + --url 'https://gitlab.com/api/v4/projects/<your-project-ID>' \ + --data "build_coverage_regex=<your-regular-expression>" + ``` + +You can use <https://rubular.com> to test your regular expression. The regular expression returns the **last** +match found in the output. + ### Test coverage examples Use this regex for commonly used test tools. @@ -338,7 +364,7 @@ https://gitlab.example.com/<namespace>/<project>/badges/<branch>/pipeline.svg?ig ### Test coverage report badge -You can define the regular expression for the [coverage report](#add-test-coverage-results-to-a-merge-request) +You can define the regular expression for the [coverage report](#add-test-coverage-results-to-a-merge-request-deprecated) that each job log is matched against. This means that each job in the pipeline can have the test coverage percentage value defined. |
