diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-04-10 09:09:41 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-04-10 09:09:41 +0300 |
commit | 187ee320b39af22929d74c5a2d9b0650bf50a09b (patch) | |
tree | ff04eab6c7914f6408c4f637f863fc07aa409cdc /doc/ci/pipelines | |
parent | a7dc052b7e01aee680d130274c79da9bfa459272 (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc/ci/pipelines')
-rw-r--r-- | doc/ci/pipelines/img/job_group_v12_10.png | bin | 0 -> 21609 bytes | |||
-rw-r--r-- | doc/ci/pipelines/index.md | 616 |
2 files changed, 304 insertions, 312 deletions
diff --git a/doc/ci/pipelines/img/job_group_v12_10.png b/doc/ci/pipelines/img/job_group_v12_10.png Binary files differnew file mode 100644 index 00000000000..0ce121ab572 --- /dev/null +++ b/doc/ci/pipelines/img/job_group_v12_10.png diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 5ce4e1649bc..91908d6c5e4 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -3,183 +3,193 @@ disqus_identifier: 'https://docs.gitlab.com/ee/ci/pipelines.html' type: reference --- -# Creating and using CI/CD pipelines +# CI/CD pipelines > Introduced in GitLab 8.8. NOTE: **Tip:** -Watch our +Watch the ["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) -webcast to see a comprehensive demo of GitLab CI/CD pipeline. +webcast to see a comprehensive demo of a GitLab CI/CD pipeline. Pipelines are the top-level component of continuous integration, delivery, and deployment. Pipelines comprise: -- Jobs that define what to run. For example, code compilation or test runs. -- Stages that define when and how to run. For example, that tests run only after code compilation. +- Jobs, which define *what* to do. For example, jobs that compile or test code. +- Stages, which define *when* to run the jobs. For example, stages that run tests after stages that compile the code. -Multiple jobs in the same stage are executed by [Runners](../runners/README.md) in parallel, if there are enough concurrent [Runners](../runners/README.md). +Jobs are executed by [Runners](../runners/README.md). Multiple jobs in the same stage are executed in parallel, +if there are enough concurrent runners. If all the jobs in a stage: - Succeed, the pipeline moves on to the next stage. - Fail, the next stage is not (usually) executed and the pipeline ends early. +In general, pipelines are executed automatically and require no intervention once created. However, there are +also times when you can manually interact with a pipeline. + +A typical pipeline might consist of four stages, executed in the following order: + +- A `build` stage, with a job called `compile`. +- A `test` stage, with two jobs called `test1` and `test2`. +- A `staging` stage, with a job called `deploy-to-stage`. +- A `production` stage, with a job called `deploy-to-prod`. + NOTE: **Note:** If you have a [mirrored repository that GitLab pulls from](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter), you may need to enable pipeline triggering in your project's **Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**. -## Simple pipeline example +## Types of pipelines -As an example, imagine a pipeline consisting of four stages, executed in the following order: +Pipelines can be configured in many different ways: -- `build`, with a job called `compile`. -- `test`, with two jobs called `test` and `test2`. -- `staging`, with a job called `deploy-to-stage`. -- `production`, with a job called `deploy-to-prod`. +- [Multi-project pipelines](../multi_project_pipelines.md) combine pipelines for different projects together. +- [Parent-Child pipelines](../parent_child_pipelines.md) break down complex pipelines + into one parent pipeline that can trigger multiple child sub-pipelines, which all + run in the same project and with the same SHA. +- [Pipelines for Merge Requests](../merge_request_pipelines/index.md) run for merge + requests only (rather than for every commit). +- [Pipelines for Merged Results](../merge_request_pipelines/pipelines_for_merged_results/index.md) + are merge request pipelines that act as though the changes from the source branch have + already been merged into the target branch. +- [Merge Trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md) + use pipelines for merged results to queue merges one after the other. -## Visualizing pipelines +## Configure a pipeline -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5742) in GitLab 8.11. +Pipelines and their component jobs and stages are defined in the CI/CD pipeline configuration file for each project. -Pipelines can be complex structures with many sequential and parallel jobs. - -To make it easier to understand the flow of a pipeline, GitLab has pipeline graphs for viewing pipelines -and their statuses. - -Pipeline graphs can be displayed in two different ways, depending on the page you -access the graph from. +- Jobs are the [basic configuration](../yaml/README.md#introduction) component. +- Stages are defined by using the [`stages`](../yaml/README.md#stages) keyword. -NOTE: **Note:** -GitLab capitalizes the stages' names when shown in the pipeline graphs (below). +For a list of configuration options in the CI pipeline file, see the [GitLab CI/CD Pipeline Configuration Reference](../yaml/README.md). -### Regular pipeline graphs +You can also configure specific aspects of your pipelines through the GitLab UI. For example: -Regular pipeline graphs show the names of the jobs of each stage. Regular pipeline graphs can -be found when you are on a [single pipeline page](#accessing-pipelines). For example: +- [Pipeline settings](settings.md) for each project. +- [Pipeline schedules](schedules.md). +- [Custom CI/CD variables](../variables/README.md#creating-a-custom-environment-variable). -![Pipelines example](img/pipelines.png) +### View pipelines -### Pipeline mini graphs +You can find the current and historical pipeline runs under your project's +**CI/CD > Pipelines** page. You can also access pipelines for a merge request by navigating +to its **Pipelines** tab. -Pipeline mini graphs take less space and can tell you at a -quick glance if all jobs passed or something failed. The pipeline mini graph can -be found when you navigate to: +![Pipelines index page](img/pipelines_index.png) -- The pipelines index page. -- A single commit page. -- A merge request page. +Clicking a pipeline will bring you to the **Pipeline Details** page and show +the jobs that were run for that pipeline. From here you can cancel a running pipeline, +retry jobs on a failed pipeline, or [delete a pipeline](#delete-a-pipeline). -Pipeline mini graphs allow you to see all related jobs for a single commit and the net result -of each stage of your pipeline. This allows you to quickly see what failed and -fix it. +[Starting in GitLab 12.3](https://gitlab.com/gitlab-org/gitlab-foss/issues/50499), a link to the +latest pipeline for the last commit of a given branch is available at `/project/pipelines/[branch]/latest`. +Also, `/project/pipelines/latest` will redirect you to the latest pipeline for the last commit +on the project's default branch. -Stages in pipeline mini graphs are collapsible. Hover your mouse over them and click to expand their jobs. +### Run a pipeline manually -| Mini graph | Mini graph expanded | -|:-------------------------------------------------------------|:---------------------------------------------------------------| -| ![Pipelines mini graph](img/pipelines_mini_graph_simple.png) | ![Pipelines mini graph extended](img/pipelines_mini_graph.png) | +Pipelines can be manually executed, with predefined or manually-specified [variables](../variables/README.md). -### Job ordering in pipeline graphs +You might do this if the results of a pipeline (for example, a code build) are required outside the normal +operation of the pipeline. -Job ordering depends on the type of pipeline graph. For [regular pipeline graphs](#regular-pipeline-graphs), jobs are sorted by name. +To execute a pipeline manually: -For [pipeline mini graphs](#pipeline-mini-graphs) ([introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/9760) -in GitLab 9.0), jobs are sorted by severity and then by name. +1. Navigate to your project's **CI/CD > Pipelines**. +1. Click on the **Run Pipeline** button. +1. On the **Run Pipeline** page: + 1. Select the branch to run the pipeline for in the **Create for** field. + 1. Enter any [environment variables](../variables/README.md) required for the pipeline run. + 1. Click the **Create pipeline** button. -The order of severity is: +The pipeline will execute the jobs as configured. -- failed -- warning -- pending -- running -- manual -- scheduled -- canceled -- success -- skipped -- created +### Run a pipeline by using a URL query string -For example: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24146) in GitLab 12.5. -![Pipeline mini graph sorting](img/pipelines_mini_graph_sorting.png) +You can use a query string to pre-populate the **Run Pipeline** page. For example, the query string +`.../pipelines/new?ref=my_branch&var[foo]=bar&file_var[file_foo]=file_bar` will pre-populate the +**Run Pipeline** page with: -### Expanding and collapsing job log sections +- **Run for** field: `my_branch`. +- **Variables** section: + - Variable: + - Key: `foo` + - Value: `bar` + - File: + - Key: `file_foo` + - Value: `file_bar` -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/14664) in GitLab 12.0. +The format of the `pipelines/new` URL is: -Job logs are divided into sections that can be collapsed or expanded. Each section will display -the duration. +```plaintext +.../pipelines/new?ref=<branch>&var[<variable_key>]=<value>&file_var[<file_key>]=<value> +``` -In the following example: +The following parameters are supported: -- Two sections are collapsed and can be expanded. -- Three sections are expanded and can be collapsed. +- `ref`: specify the branch to populate the **Run for** field with. +- `var`: specify a `Variable` variable. +- `file_var`: specify a `File` variable. -![Collapsible sections](img/collapsible_log_v12_6.png) +For each `var` or `file_var`, a key and value are required. -#### Custom collapsible sections +### Add manual interaction to your pipeline -You can create collapsible sections in job logs by manually outputting special codes -that GitLab will use to determine what sections to collapse: +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7931) in GitLab 8.15. -- Section start marker: `section_start:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` + `TEXT_OF_SECTION_HEADER` -- Section end marker: `section_end:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` +Manual actions, configured using the [`when:manual`](../yaml/README.md#whenmanual) parameter, +allow you to require manual interaction before moving forward in the pipeline. -You must add these codes to the script section of the CI configuration. For example, -using `echo`: +You can do this straight from the pipeline graph. Just click the play button +to execute that particular job. -```yaml -job1: - script: - - echo -e "section_start:`date +%s`:my_first_section\r\e[0KHeader of the 1st collapsible section" - - echo 'this line should be hidden when collapsed' - - echo -e "section_end:`date +%s`:my_first_section\r\e[0K" -``` +For example, your pipeline might start automatically, but it requires manual action to +[deploy to production](../environments.md#configuring-manual-deployments). In the example below, the `production` +stage has a job with a manual action. -In the example above: +![Pipelines example](img/pipelines.png) -- `date +%s`: The Unix timestamp (for example `1560896352`). -- `my_first_section`: The name given to the section. -- `\r\e[0K`: Prevents the section markers from displaying in the rendered (colored) - job log, but they are displayed in the raw job log. To see them, in the top right - of the job log, click **{doc-text}** (**Show complete raw**). - - `\r`: carriage return. - - `\e[0K`: clear line ANSI escape code. +#### Start multiple manual actions in a stage -Sample raw job log: +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/27188) in GitLab 11.11. -```plaintext -section_start:1560896352:my_first_section\r\e[0KHeader of the 1st collapsible section -this line should be hidden when collapsed -section_end:1560896353:my_first_section\r\e[0K -``` +Multiple manual actions in a single stage can be started at the same time using the "Play all manual" button. +Once the user clicks this button, each individual manual action will be triggered and refreshed +to an updated status. -### Pipeline success and duration charts +This functionality is only available: -> - Introduced in GitLab 3.1.1 as Commit Stats, and later renamed to Pipeline Charts. -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/issues/38318) to CI / CD Analytics in GitLab 12.8. +- For users with at least Developer access. +- If the stage contains [manual actions](#add-manual-interaction-to-your-pipeline). -GitLab tracks the history of your pipeline successes and failures, as well as how long each pipeline ran. To view this information, go to **Analytics > CI / CD Analytics**. +### Delete a pipeline -View successful pipelines: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24851) in GitLab 12.7. -![Successful pipelines](img/pipelines_success_chart.png) +Users with [owner permissions](../../user/permissions.md) in a project can delete a pipeline +by clicking on the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details** +page, then using the **Delete** button. -View pipeline duration history: +![Pipeline Delete Button](img/pipeline-delete.png) -![Pipeline duration](img/pipelines_duration_chart.png) +CAUTION: **Warning:** +Deleting a pipeline will expire all pipeline caches, and delete all related objects, +such as builds, logs, artifacts, and triggers. **This action cannot be undone.** -## Pipeline quotas +### Pipeline quotas Each user has a personal pipeline quota that tracks the usage of shared runners in all personal projects. Each group has a [usage quota](../../subscriptions/index.md#ci-pipeline-minutes) that tracks the usage of shared runners for all projects created within the group. When a pipeline is triggered, regardless of who triggered it, the pipeline quota for the project owner's [namespace](../../user/group/index.md#namespaces) is used. In this case, the namespace can be the user or group that owns the project. -### How pipeline duration is calculated +#### How pipeline duration is calculated Total running time for a given pipeline excludes retries and pending (queued) time. @@ -216,245 +226,285 @@ The union of A, B, and C is (1, 4) and (6, 7). Therefore, the total running time (4 - 1) + (7 - 6) => 4 ``` -## Configuring pipelines +### Pipeline security on protected branches -Pipelines, and their component jobs and stages, are defined in the [`.gitlab-ci.yml`](../yaml/README.md) file for each project. +A strict security model is enforced when pipelines are executed on +[protected branches](../../user/project/protected_branches.md). -In particular: +The following actions are allowed on protected branches only if the user is +[allowed to merge or push](../../user/project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings) +on that specific branch: -- Jobs are the [basic configuration](../yaml/README.md#introduction) component. -- Stages are defined using the [`stages`](../yaml/README.md#stages) keyword. +- Run manual pipelines (using the [Web UI](#run-a-pipeline-manually) or [pipelines API](#pipelines-api)). +- Run scheduled pipelines. +- Run pipelines using triggers. +- Trigger manual actions on existing pipelines. +- Retry or cancel existing jobs (using the Web UI or pipelines API). -For all available configuration options, see the [GitLab CI/CD Pipeline Configuration Reference](../yaml/README.md). +**Variables** marked as **protected** are accessible only to jobs that +run on protected branches, preventing untrusted users getting unintended access to +sensitive information like deployment credentials and tokens. -### Settings and schedules +**Runners** marked as **protected** can run jobs only on protected +branches, preventing untrusted code from executing on the protected runner and +preserving deployment keys and other credentials from being unintentionally +accessed. In order to ensure that jobs intended to be executed on protected +runners will not use regular runners, they must be tagged accordingly. -In addition to configuring jobs through `.gitlab-ci.yml`, additional configuration options are available -through the GitLab UI: +## View jobs in a pipeline -- Pipeline settings for each project. For more information, see [Pipeline settings](settings.md). -- Schedules for pipelines. For more information, see [Pipeline schedules](schedules.md). +When you access a pipeline, you can see the related jobs for that pipeline. -### Grouping jobs +Clicking an individual job will show you its job log, and allow you to: -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/6242) in GitLab 8.12. +- Cancel the job. +- Retry the job. +- Erase the job log. -If you have many similar jobs, your [pipeline graph](#visualizing-pipelines) becomes long and hard -to read. +### See why a job failed -For that reason, similar jobs can automatically be grouped together. -If the job names are formatted in certain ways, they will be collapsed into -a single group in regular pipeline graphs (not the mini graphs). +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17782) in GitLab 10.7. -You'll know when a pipeline has grouped jobs if you don't see the retry or -cancel button inside them. Hovering over them will show the number of grouped -jobs. Click to expand them. +When a pipeline fails or is allowed to fail, there are several places where you +can find the reason: -![Grouped pipelines](img/pipelines_grouped.png) +- In the [pipeline graph](#visualize-pipelines), on the pipeline detail view. +- In the pipeline widgets, in the merge requests and commit pages. +- In the job views, in the global and detailed views of a job. -#### Configuring grouping +In each place, if you hover over the failed job you can see the reason it failed. -In the pipeline [configuration file](../yaml/README.md), job names must include two numbers separated with one of -the following (you can even use them interchangeably): +![Pipeline detail](img/job_failure_reason.png) -- A space. -- A slash (`/`). -- A colon (`:`). +In [GitLab 10.8](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17814) and later, +you can also see the reason it failed on the Job detail page. -NOTE: **Note:** -More specifically, it uses [this](https://gitlab.com/gitlab-org/gitlab/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99) regular expression: `\d+[\s:\/\\]+\d+\s*`. +### The order of jobs in a pipeline -#### How grouping works +The order of jobs in a pipeline depends on the type of pipeline graph. -The jobs will be ordered by comparing those two numbers from left to right. You -usually want the first to be the index and the second the total. +- For [regular pipeline graphs](#regular-pipeline-graphs), jobs are sorted by name. +- For [pipeline mini graphs](#pipeline-mini-graphs), jobs are sorted by severity and then by name. -For example, the following jobs will be grouped under a job named `test`: +The order of severity is: -- `test 0 3` -- `test 1 3` -- `test 2 3` +- failed +- warning +- pending +- running +- manual +- scheduled +- canceled +- success +- skipped +- created -The following jobs will be grouped under a job named `test ruby`: +For example: -- `test 1:2 ruby` -- `test 2:2 ruby` +![Pipeline mini graph sorting](img/pipelines_mini_graph_sorting.png) -The following jobs will be grouped under a job named `test ruby` as well: +### Group jobs in a pipeline -- `1/3 test ruby` -- `2/3 test ruby` -- `3/3 test ruby` +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/6242) in GitLab 8.12. -### Pipelines for merge requests +If you have many similar jobs, your [pipeline graph](#visualize-pipelines) becomes long and hard +to read. -GitLab supports configuring pipelines that run only for merge requests. For more information, see -[Pipelines for merge requests](../merge_request_pipelines/index.md). +You can automatically group similar jobs together. If the job names are formatted in a certain way, +they will be collapsed into a single group in regular pipeline graphs (not the mini graphs). -### Badges +You'll know when a pipeline has grouped jobs if you don't see the retry or +cancel button inside them. Hovering over them will show the number of grouped +jobs. Click to expand them. -Pipeline status and test coverage report badges are available and configurable for each project. +![Grouped pipelines](img/pipelines_grouped.png) -For information on adding pipeline badges to projects, see [Pipeline badges](settings.md#pipeline-badges). +To create a group of jobs, in the [CI/CD pipeline configuration file](../yaml/README.md), +separate each job name with a number and one of the following: -## Multi-project pipelines +- A slash (`/`), for example, `test 1/3`, `test 2/3`, `test 3/3`. +- A colon (`:`), for example, `test 1:3`, `test 2:3`, `test 3:3`. +- A space, for example `test 0 3`, `test 1 3`, `test 2 3`. -Pipelines for different projects can be combined together into [Multi-project pipelines](../multi_project_pipelines.md). +You can use these symbols interchangeably. -[Multi-project pipeline graphs](../multi_project_pipelines.md#multi-project-pipeline-visualization-premium) help -you visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)** +For example, these three jobs will be in a group named `build ruby`: -## Parent-child pipelines +```yaml +build ruby 1/3: + stage: build + script: + - echo "ruby1" -Complex pipelines can be broken down into one parent pipeline that can trigger -multiple child sub-pipelines, which all run in the same project and with the same SHA. +build ruby 2/3: + stage: build + script: + - echo "ruby2" + +build ruby 3/3: + stage: build + script: + - echo "ruby3" +``` -For more information, see [Parent-Child pipelines](../parent_child_pipelines.md). +In the pipeline, the result is a group named `build ruby` with three jobs: -## Working with pipelines +![Job group](img/job_group_v12_10.png) -In general, pipelines are executed automatically and require no intervention once created. +The jobs will be ordered by comparing the numbers from left to right. You +usually want the first number to be the index and the second number to be the total. -However, there are instances where you'll need to interact with pipelines. These are documented below. +[This regular expression](https://gitlab.com/gitlab-org/gitlab/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99) +evaluates the job names: `\d+[\s:\/\\]+\d+\s*`. -### Manually executing pipelines +### Specifying variables when running manual jobs -Pipelines can be manually executed, with predefined or manually-specified [variables](../variables/README.md). +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30485) in GitLab 12.2. -You might do this if the results of a pipeline (for example, a code build) is required outside the normal -operation of the pipeline. +When running manual jobs you can supply additional job specific variables. -To execute a pipeline manually: +You can do this from the job page of the manual job you want to run with +additional variables. -1. Navigate to your project's **CI/CD > Pipelines**. -1. Click on the **Run Pipeline** button. -1. On the **Run Pipeline** page: - 1. Select the branch to run the pipeline for in the **Create for** field. - 1. Enter any [environment variables](../variables/README.md) required for the pipeline run. - 1. Click the **Create pipeline** button. +This is useful when you want to alter the execution of a job by using +environment variables. -The pipeline will execute the jobs as configured. +![Manual job variables](img/manual_job_variables.png) -#### Using a query string +### Delay a job -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24146) in GitLab 12.5. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/21767) in GitLab 11.4. -Variables on the **Run Pipeline** page can be pre-populated by passing variable keys and values -in a query string appended to the `pipelines/new` URL. The format is: +When you do not want to run a job immediately, you can use the [`when:delayed`](../yaml/README.md#whendelayed) parameter to +delay a job's execution for a certain period. -```plaintext -.../pipelines/new?ref=<branch>&var[<variable_key>]=<value>&file_var[<file_key>]=<value> -``` +This is especially useful for timed incremental rollout where new code is rolled out gradually. -The following parameters are supported: +For example, if you start rolling out new code and: -- `ref`: specify the branch to populate the **Run for** field with. -- `var`: specify a `Variable` variable. -- `file_var`: specify a `File` variable. +- Users do not experience trouble, GitLab can automatically complete the deployment from 0% to 100%. +- Users experience trouble with the new code, you can stop the timed incremental rollout by canceling the pipeline + and [rolling](../environments.md#retrying-and-rolling-back) back to the last stable version. -For each `var` or `file_var`, a key and value are required. +![Pipelines example](img/pipeline_incremental_rollout.png) -For example, the query string -`.../pipelines/new?ref=my_branch&var[foo]=bar&file_var[file_foo]=file_bar` will pre-populate the -**Run Pipeline** page as follows: +### Expand and collapse job log sections -- **Run for** field: `my_branch`. -- **Variables** section: - - Variable: - - Key: `foo` - - Value: `bar` - - File: - - Key: `file_foo` - - Value: `file_bar` +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/14664) in GitLab 12.0. -### Accessing pipelines +Job logs are divided into sections that can be collapsed or expanded. Each section will display +the duration. -You can find the current and historical pipeline runs under your project's -**CI/CD > Pipelines** page. You can also access pipelines for a merge request by navigating -to its **Pipelines** tab. +In the following example: -![Pipelines index page](img/pipelines_index.png) +- Two sections are collapsed and can be expanded. +- Three sections are expanded and can be collapsed. -Clicking on a pipeline will bring you to the **Pipeline Details** page and show -the jobs that were run for that pipeline. From here you can cancel a running pipeline, -retry jobs on a failed pipeline, or [delete a pipeline](#deleting-a-single-pipeline). +![Collapsible sections](img/collapsible_log_v12_6.png) -### Accessing individual jobs +#### Custom collapsible sections -When you access a pipeline, you can see the related jobs for that pipeline. +You can create collapsible sections in job logs by manually outputting special codes +that GitLab will use to determine what sections to collapse: -Clicking on an individual job will show you its job log, and allow you to: +- Section start marker: `section_start:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` + `TEXT_OF_SECTION_HEADER` +- Section end marker: `section_end:UNIX_TIMESTAMP:SECTION_NAME\r\e[0K` -- Cancel the job. -- Retry the job. -- Erase the job log. +You must add these codes to the script section of the CI configuration. For example, +using `echo`: -### Seeing the failure reason for jobs +```yaml +job1: + script: + - echo -e "section_start:`date +%s`:my_first_section\r\e[0KHeader of the 1st collapsible section" + - echo 'this line should be hidden when collapsed' + - echo -e "section_end:`date +%s`:my_first_section\r\e[0K" +``` -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17782) in GitLab 10.7. +In the example above: -When a pipeline fails or is allowed to fail, there are several places where you -can quickly check the reason it failed: +- `date +%s`: The Unix timestamp (for example `1560896352`). +- `my_first_section`: The name given to the section. +- `\r\e[0K`: Prevents the section markers from displaying in the rendered (colored) + job log, but they are displayed in the raw job log. To see them, in the top right + of the job log, click **{doc-text}** (**Show complete raw**). + - `\r`: carriage return. + - `\e[0K`: clear line ANSI escape code. -- In the pipeline graph, on the pipeline detail view. -- In the pipeline widgets, in the merge requests and commit pages. -- In the job views, in the global and detailed views of a job. +Sample raw job log: -In each place, if you hover over the failed job you can see the reason it failed. +```plaintext +section_start:1560896352:my_first_section\r\e[0KHeader of the 1st collapsible section +this line should be hidden when collapsed +section_end:1560896353:my_first_section\r\e[0K +``` -![Pipeline detail](img/job_failure_reason.png) +## Visualize pipelines -From [GitLab 10.8](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17814), -you can also see the reason it failed on the Job detail page. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5742) in GitLab 8.11. -### Manual actions from pipeline graphs +Pipelines can be complex structures with many sequential and parallel jobs. -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7931) in GitLab 8.15. +To make it easier to understand the flow of a pipeline, GitLab has pipeline graphs for viewing pipelines +and their statuses. -Manual actions, configured using the [`when:manual`](../yaml/README.md#whenmanual) parameter, -allow you to require manual interaction before moving forward in the pipeline. +Pipeline graphs can be displayed in two different ways, depending on the page you +access the graph from. -You can do this straight from the pipeline graph. Just click on the play button -to execute that particular job. +NOTE: **Note:** +GitLab capitalizes the stages' names in the pipeline graphs. -For example, your pipeline start automatically, but require manual action to -[deploy to production](../environments.md#configuring-manual-deployments). In the example below, the `production` -stage has a job with a manual action. +### Regular pipeline graphs + +Regular pipeline graphs show the names of the jobs in each stage. Regular pipeline graphs can +be found when you are on a [single pipeline page](#view-pipelines). For example: ![Pipelines example](img/pipelines.png) -### Specifying variables when running manual jobs +[Multi-project pipeline graphs](../multi_project_pipelines.md#multi-project-pipeline-visualization-premium) help +you visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30485) in GitLab 12.2. +### Pipeline mini graphs -When running manual jobs you can supply additional job specific variables. +Pipeline mini graphs take less space and can tell you at a +quick glance if all jobs passed or something failed. The pipeline mini graph can +be found when you navigate to: -You can do this from the job page of the manual job you want to run with -additional variables. +- The pipelines index page. +- A single commit page. +- A merge request page. -This is useful when you want to alter the execution of a job by using -environment variables. +Pipeline mini graphs allow you to see all related jobs for a single commit and the net result +of each stage of your pipeline. This allows you to quickly see what failed and +fix it. -![Manual job variables](img/manual_job_variables.png) +Stages in pipeline mini graphs are collapsible. Hover your mouse over them and click to expand their jobs. -### Delay a job in a pipeline graph +| Mini graph | Mini graph expanded | +|:-------------------------------------------------------------|:---------------------------------------------------------------| +| ![Pipelines mini graph](img/pipelines_mini_graph_simple.png) | ![Pipelines mini graph extended](img/pipelines_mini_graph.png) | -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/21767) in GitLab 11.4. +### Pipeline success and duration charts -When you do not want to run a job immediately, you can use the [`when:delayed`](../yaml/README.md#whendelayed) parameter to -delay a job's execution for a certain period. +> - Introduced in GitLab 3.1.1 as Commit Stats, and later renamed to Pipeline Charts. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/issues/38318) to CI / CD Analytics in GitLab 12.8. -This is especially useful for timed incremental rollout where new code is rolled out gradually. +GitLab tracks the history of your pipeline successes and failures, as well as how long each pipeline ran. To view this information, go to **Analytics > CI / CD Analytics**. -For example, if you start rolling out new code and: +View successful pipelines: -- Users do not experience trouble, GitLab can automatically complete the deployment from 0% to 100%. -- Users experience trouble with the new code, you can stop the timed incremental rollout by canceling the pipeline - and [rolling](../environments.md#retrying-and-rolling-back) back to the last stable version. +![Successful pipelines](img/pipelines_success_chart.png) -![Pipelines example](img/pipeline_incremental_rollout.png) +View pipeline duration history: + +![Pipeline duration](img/pipelines_duration_chart.png) + +### Pipeline badges + +Pipeline status and test coverage report badges are available and configurable for each project. +For information on adding pipeline badges to projects, see [Pipeline badges](settings.md#pipeline-badges). -### Using the API +## Pipelines API GitLab provides API endpoints to: @@ -464,65 +514,7 @@ GitLab provides API endpoints to: - [Triggering pipelines through the API](../triggers/README.md). - [Pipeline triggers API](../../api/pipeline_triggers.md). -### Start multiple manual actions in a stage - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/27188) in GitLab 11.11. - -Multiple manual actions in a single stage can be started at the same time using the "Play all manual" button. -Once the user clicks this button, each individual manual action will be triggered and refreshed -to an updated status. - -This functionality is only available: - -- For users with at least Developer access. -- If the stage contains [manual actions](#manual-actions-from-pipeline-graphs). - -### Deleting a single pipeline - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24851) in GitLab 12.7. - -Users with [owner permissions](../../user/permissions.md) in a project can delete a pipeline -by clicking on the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details** -page, then using the **Delete** button. - -![Pipeline Delete Button](img/pipeline-delete.png) - -CAUTION: **Warning:** -Deleting a pipeline will expire all pipeline caches, and delete all related objects, -such as builds, logs, artifacts, and triggers. **This action cannot be undone.** - -## Most Recent Pipeline - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/50499) in GitLab 12.3. - -There's a link to the latest pipeline for the last commit of a given branch at `/project/pipelines/[branch]/latest`. Also, `/project/pipelines/latest` will redirect you to the latest pipeline for the last commit on the project's default branch. - -## Security on protected branches - -A strict security model is enforced when pipelines are executed on -[protected branches](../../user/project/protected_branches.md). - -The following actions are allowed on protected branches only if the user is -[allowed to merge or push](../../user/project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings) -on that specific branch: - -- Run manual pipelines (using the [Web UI](#manually-executing-pipelines) or pipelines API). -- Run scheduled pipelines. -- Run pipelines using triggers. -- Trigger manual actions on existing pipelines. -- Retry or cancel existing jobs (using the Web UI or pipelines API). - -**Variables** marked as **protected** are accessible only to jobs that -run on protected branches, preventing untrusted users getting unintended access to -sensitive information like deployment credentials and tokens. - -**Runners** marked as **protected** can run jobs only on protected -branches, avoiding untrusted code to be executed on the protected runner and -preserving deployment keys and other credentials from being unintentionally -accessed. In order to ensure that jobs intended to be executed on protected -runners will not use regular runners, they must be tagged accordingly. - -## Persistent pipeline refs +## Troubleshooting `fatal: reference is not a tree:` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17043) in GitLab 12.4. |