diff options
Diffstat (limited to 'doc/development/pipelines.md')
| -rw-r--r-- | doc/development/pipelines.md | 423 |
1 files changed, 224 insertions, 199 deletions
diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index 39ca846c1cc..05b80cdb4a6 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -11,173 +11,15 @@ We're striving to [dogfood](https://about.gitlab.com/handbook/engineering/#dogfo GitLab [CI/CD features and best-practices](../ci/yaml/README.md) as much as possible. -## Stages +## Overview -The current stages are: - -- `sync`: This stage is used to synchronize changes from <https://gitlab.com/gitlab-org/gitlab> to - <https://gitlab.com/gitlab-org/gitlab-foss>. -- `prepare`: This stage includes jobs that prepare artifacts that are needed by - jobs in subsequent stages. -- `build-images`: This stage includes jobs that prepare docker images - that are needed by jobs in subsequent stages or downstream pipelines. -- `fixtures`: This stage includes jobs that prepare fixtures needed by frontend tests. -- `test`: This stage includes most of the tests, DB/migration jobs, and static analysis jobs. -- `post-test`: This stage includes jobs that build reports or gather data from - the `test` stage's jobs (e.g. coverage, Knapsack metadata etc.). -- `review-prepare`: This stage includes a job that build the CNG images that are - later used by the (Helm) Review App deployment (see - [Review Apps](testing_guide/review_apps.md) for details). -- `review`: This stage includes jobs that deploy the GitLab and Docs Review Apps. -- `qa`: This stage includes jobs that perform QA tasks against the Review App - that is deployed in the previous stage. -- `post-qa`: This stage includes jobs that build reports or gather data from - the `qa` stage's jobs (e.g. Review App performance report). -- `pages`: This stage includes a job that deploys the various reports as - GitLab Pages (e.g. <https://gitlab-org.gitlab.io/gitlab/coverage-ruby/>, - <https://gitlab-org.gitlab.io/gitlab/coverage-javascript/>, - <https://gitlab-org.gitlab.io/gitlab/webpack-report/>). - -## Default image - -The default image is defined in <https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml>. - -It includes Ruby, Go, Git, Git LFS, Chrome, Node, Yarn, PostgreSQL, and Graphics Magick. - -The images used in our pipelines are configured in the -[`gitlab-org/gitlab-build-images`](https://gitlab.com/gitlab-org/gitlab-build-images) -project, which is push-mirrored to <https://dev.gitlab.org/gitlab/gitlab-build-images> -for redundancy. - -The current version of the build images can be found in the -["Used by GitLab section"](https://gitlab.com/gitlab-org/gitlab-build-images/blob/master/.gitlab-ci.yml). - -## Default variables - -In addition to the [predefined variables](../ci/variables/predefined_variables.md), -each pipeline includes default variables defined in -<https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml>. - -## Common job definitions - -Most of the jobs [extend from a few CI definitions](../ci/yaml/README.md#extends) -defined in [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml) -that are scoped to a single [configuration parameter](../ci/yaml/README.md#configuration-parameters). - -| Job definitions | Description | -|------------------|-------------| -| `.default-tags` | Ensures a job has the `gitlab-org` tag to ensure it's using our dedicated runners. | -| `.default-retry` | Allows a job to [retry](../ci/yaml/README.md#retry) upon `unknown_failure`, `api_failure`, `runner_system_failure`, `job_execution_timeout`, or `stuck_or_timeout_failure`. | -| `.default-before_script` | Allows a job to use a default `before_script` definition suitable for Ruby/Rails tasks that may need a database running (e.g. tests). | -| `.default-cache` | Allows a job to use a default `cache` definition suitable for Ruby/Rails and frontend tasks. | -| `.use-pg11` | Allows a job to use the `postgres:11.6` and `redis:alpine` services. | -| `.use-pg11-ee` | Same as `.use-pg11` but also use the `docker.elastic.co/elasticsearch/elasticsearch:6.4.2` services. | -| `.use-kaniko` | Allows a job to use the `kaniko` tool to build Docker images. | -| `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` environment variable. | - -## `workflow:rules` - -We're using the [`workflow:rules` keyword](../ci/yaml/README.md#workflowrules) to -define default rules to determine whether or not a pipeline is created. - -These rules are defined in <https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml> -and are as follows: - -1. If `$FORCE_GITLAB_CI` is set, create a pipeline. -1. For merge requests, create a pipeline. -1. For `master` branch, create a pipeline (this includes on schedules, pushes, merges, etc.). -1. For tags, create a pipeline. -1. If `$GITLAB_INTERNAL` isn't set, don't create a pipeline. -1. For stable, auto-deploy, and security branches, create a pipeline. -1. For any other cases (e.g. when pushing a branch with no MR for it), no pipeline is created. - -## `rules`, `if:` conditions and `changes:` patterns - -We're using the [`rules` keyword](../ci/yaml/README.md#rules) extensively. - -All `rules` definitions are defined in -<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml>, -then included in individual jobs via [`extends`](../ci/yaml/README.md#extends). - -The `rules` definitions are composed of `if:` conditions and `changes:` patterns, -which are also defined in -<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml> -and included in `rules` definitions via [YAML anchors](../ci/yaml/README.md#anchors) - -### `if:` conditions - -| `if:` conditions | Description | Notes | -|------------------|-------------|-------| -| `if-not-canonical-namespace` | Matches if the project isn't in the canonical (`gitlab-org/`) or security (`gitlab-org/security`) namespace. | Use to create a job for forks (by using `when: on_success\|manual`), or **not** create a job for forks (by using `when: never`). | -| `if-not-ee` | Matches if the project isn't EE (i.e. project name isn't `gitlab` or `gitlab-ee`). | Use to create a job only in the FOSS project (by using `when: on_success|manual`), or **not** create a job if the project is EE (by using `when: never`). | -| `if-not-foss` | Matches if the project isn't FOSS (i.e. project name isn't `gitlab-foss`, `gitlab-ce`, or `gitlabhq`). | Use to create a job only in the EE project (by using `when: on_success|manual`), or **not** create a job if the project is FOSS (by using `when: never`). | -| `if-default-refs` | Matches if the pipeline is for `master`, `/^[\d-]+-stable(-ee)?$/` (stable branches), `/^\d+-\d+-auto-deploy-\d+$/` (auto-deploy branches), `/^security\//` (security branches), merge requests, and tags. | Note that jobs won't be created for branches with this default configuration. | -| `if-master-refs` | Matches if the current branch is `master`. | | -| `if-master-or-tag` | Matches if the pipeline is for the `master` branch or for a tag. | | -| `if-merge-request` | Matches if the pipeline is for a merge request. | | -| `if-nightly-master-schedule` | Matches if the pipeline is for a `master` scheduled pipeline with `$NIGHTLY` set. | | -| `if-dot-com-gitlab-org-schedule` | Limits jobs creation to scheduled pipelines for the `gitlab-org` group on GitLab.com. | | -| `if-dot-com-gitlab-org-master` | Limits jobs creation to the `master` branch for the `gitlab-org` group on GitLab.com. | | -| `if-dot-com-gitlab-org-merge-request` | Limits jobs creation to merge requests for the `gitlab-org` group on GitLab.com. | | -| `if-dot-com-gitlab-org-and-security-tag` | Limits job creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | | -| `if-dot-com-gitlab-org-and-security-merge-request` | Limit jobs creation to merge requests for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | | -| `if-dot-com-ee-schedule` | Limits jobs to scheduled pipelines for the `gitlab-org/gitlab` project on GitLab.com. | | -| `if-cache-credentials-schedule` | Limits jobs to scheduled pipelines with the `$CI_REPO_CACHE_CREDENTIALS` variable set. | | - -### `changes:` patterns - -| `changes:` patterns | Description | -|------------------------------|--------------------------------------------------------------------------| -| `ci-patterns` | Only create job for CI config-related changes. | -| `yaml-patterns` | Only create job for YAML-related changes. | -| `docs-patterns` | Only create job for docs-related changes. | -| `frontend-dependency-patterns` | Only create job when frontend dependencies are updated (i.e. `package.json`, and `yarn.lock`). changes. | -| `frontend-patterns` | Only create job for frontend-related changes. | -| `backstage-patterns` | Only create job for backstage-related changes (i.e. Danger, fixtures, RuboCop, specs). | -| `code-patterns` | Only create job for code-related changes. | -| `qa-patterns` | Only create job for QA-related changes. | -| `code-backstage-patterns` | Combination of `code-patterns` and `backstage-patterns`. | -| `code-qa-patterns` | Combination of `code-patterns` and `qa-patterns`. | -| `code-backstage-qa-patterns` | Combination of `code-patterns`, `backstage-patterns`, and `qa-patterns`. | - -## Interruptible jobs pipelines - -By default, all jobs are [interruptible](../ci/yaml/README.md#interruptible), except the -`dont-interrupt-me` job which runs automatically on `master`, and is `manual` -otherwise. - -If you want a running pipeline to finish even if you push new commits to a merge -request, be sure to start the `dont-interrupt-me` job before pushing. - -## PostgreSQL versions testing - -### Current versions testing - -| Where? | PostgreSQL version | -| ------ | ------ | -| MRs | 11 | -| `master` (non-scheduled pipelines) | 11 | -| 2-hourly scheduled pipelines | 11 | - -### Long-term plan - -We follow the [PostgreSQL versions shipped with Omnibus GitLab](https://docs.gitlab.com/omnibus/package-information/postgresql_versions.html): - -| PostgreSQL version | 12.10 (April 2020) | 13.0 (May 2020) | 13.1 (June 2020) | 13.2 (July 2020) | 13.3 (August 2020) | 13.4, 13.5 | 13.6 (November 2020) | 14.0 (May 2021?) | -| ------ | ------------------ | --------------- | ---------------- | ---------------- | ------------------ | ------------ | -------------------- | ---------------- | -| PG9.6 | MRs/`master`/`2-hour`/`nightly` | - | - | - | - | - | - | - | -| PG10 | `nightly` | - | - | - | - | - | - | - | -| PG11 | `master`/`2-hour` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | `nightly` | - | -| PG12 | - | - | - | - | `master`/`2-hour` | `master`/`2-hour` | MRs/`master`/`2-hour`/`nightly` | `master`/`2-hour` | -| PG13 | - | - | - | - | - | - | - | MRs/`master`/`2-hour`/`nightly` | - -## Pipeline types +### Pipeline types Since we use the [`rules:`](../ci/yaml/README.md#rules) and [`needs:`](../ci/yaml/README.md#needs) keywords extensively, we have four main pipeline types which are described below. Note that an MR that includes multiple types of changes would have a pipelines that include jobs from multiple types (e.g. a combination of docs-only and code-only pipelines). -### Docs-only MR pipeline +#### Docs-only MR pipeline Reference pipeline: <https://gitlab.com/gitlab-org/gitlab/pipelines/135236627> @@ -191,7 +33,7 @@ graph LR end ``` -### Code-only MR pipeline +#### Code-only MR pipeline Reference pipeline: <https://gitlab.com/gitlab-org/gitlab/pipelines/136295694> @@ -204,11 +46,11 @@ graph RL; click 1-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8100542&udv=0" 1-2["build-qa-image (3.4 minutes)"]; click 1-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914325&udv=0" - 1-3["compile-assets pull-cache (9.06 minutes)"]; + 1-3["compile-test-assets (9.06 minutes)"]; click 1-3 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914317&udv=0" - 1-4["compile-assets pull-cache as-if-foss (8.35 minutes)"]; + 1-4["compile-test-assets as-if-foss (8.35 minutes)"]; click 1-4 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356616&udv=0" - 1-5["gitlab:assets:compile pull-cache (22 minutes)"]; + 1-5["compile-production-assets (22 minutes)"]; click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0" 1-6["setup-test-env (8.22 minutes)"]; click 1-6 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914315&udv=0" @@ -224,6 +66,8 @@ graph RL; 1-18["kubesec-sast"]; 1-19["nodejs-scan-sast"]; 1-20["secrets-sast"]; + 1-21["static-analysis (17 minutes)"]; + click 1-21 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914471&udv=0" class 1-3 criticalPath; class 1-6 criticalPath; @@ -241,8 +85,6 @@ graph RL; 2_1-1 & 2_1-2 & 2_1-3 & 2_1-4 --> 1-6; end - 2_2-1["static-analysis (17 minutes)"]; - click 2_2-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914471&udv=0" 2_2-2["frontend-fixtures (17.2 minutes)"]; class 2_2-2 criticalPath; click 2_2-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=7910143&udv=0" @@ -252,13 +94,13 @@ graph RL; click 2_2-4 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356727&udv=0" 2_2-5["webpack-dev-server (6.1 minutes)"]; click 2_2-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8404303&udv=0" - subgraph "Needs `setup-test-env` & `compile-assets`"; - 2_2-1 & 2_2-2 & 2_2-4 & 2_2-5 --> 1-6 & 1-3; + subgraph "Needs `setup-test-env` & `compile-test-assets`"; + 2_2-2 & 2_2-4 & 2_2-5 --> 1-6 & 1-3; 2_2-3 --> 1-6 & 1-4; end 2_3-1["build-assets-image (2.5 minutes)"]; - subgraph "Needs `gitlab:assets:compile`"; + subgraph "Needs `compile-production-assets`"; 2_3-1 --> 1-5 end @@ -269,7 +111,7 @@ graph RL; end 2_5-1["rspec & db jobs (12-22 minutes)"]; - subgraph "Needs `compile-assets`, `setup-test-env`, & `retrieve-tests-metadata`"; + subgraph "Needs `compile-test-assets`, `setup-test-env`, & `retrieve-tests-metadata`"; 2_5-1 --> 1-3 & 1-6 & 1-14; class 2_5-1 criticalPath; click 2_5-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations" @@ -304,7 +146,7 @@ graph RL; end ``` -### Frontend-only MR pipeline +#### Frontend-only MR pipeline Reference pipeline: <https://gitlab.com/gitlab-org/gitlab/pipelines/134661039> @@ -317,11 +159,11 @@ graph RL; click 1-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8100542&udv=0" 1-2["build-qa-image (3.4 minutes)"]; click 1-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914325&udv=0" - 1-3["compile-assets pull-cache (9.06 minutes)"]; + 1-3["compile-test-assets (9.06 minutes)"]; click 1-3 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914317&udv=0" - 1-4["compile-assets pull-cache as-if-foss (8.35 minutes)"]; + 1-4["compile-test-assets as-if-foss (8.35 minutes)"]; click 1-4 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356616&udv=0" - 1-5["gitlab:assets:compile pull-cache (22 minutes)"]; + 1-5["compile-production-assets (22 minutes)"]; click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0" 1-6["setup-test-env (8.22 minutes)"]; click 1-6 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914315&udv=0" @@ -337,6 +179,8 @@ graph RL; 1-18["kubesec-sast"]; 1-19["nodejs-scan-sast"]; 1-20["secrets-sast"]; + 1-21["static-analysis (17 minutes)"]; + click 1-21 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914471&udv=0" class 1-3 criticalPath; class 1-5 criticalPath; @@ -355,8 +199,6 @@ graph RL; 2_1-1 & 2_1-2 & 2_1-3 & 2_1-4 --> 1-6; end - 2_2-1["static-analysis (17 minutes)"]; - click 2_2-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914471&udv=0" 2_2-2["frontend-fixtures (17.2 minutes)"]; class 2_2-2 criticalPath; click 2_2-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=7910143&udv=0" @@ -366,14 +208,14 @@ graph RL; click 2_2-4 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356727&udv=0" 2_2-5["webpack-dev-server (6.1 minutes)"]; click 2_2-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8404303&udv=0" - subgraph "Needs `setup-test-env` & `compile-assets`"; - 2_2-1 & 2_2-2 & 2_2-4 & 2_2-5 --> 1-6 & 1-3; + subgraph "Needs `setup-test-env` & `compile-test-assets`"; + 2_2-2 & 2_2-4 & 2_2-5 --> 1-6 & 1-3; 2_2-3 --> 1-6 & 1-4; end 2_3-1["build-assets-image (2.5 minutes)"]; class 2_3-1 criticalPath; - subgraph "Needs `gitlab:assets:compile`"; + subgraph "Needs `compile-production-assets`"; 2_3-1 --> 1-5 end @@ -384,7 +226,7 @@ graph RL; end 2_5-1["rspec & db jobs (12-22 minutes)"]; - subgraph "Needs `compile-assets`, `setup-test-env, & `retrieve-tests-metadata`"; + subgraph "Needs `compile-test-assets`, `setup-test-env, & `retrieve-tests-metadata`"; 2_5-1 --> 1-3 & 1-6 & 1-14; class 2_5-1 criticalPath; click 2_5-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations" @@ -444,7 +286,7 @@ graph RL; end ``` -### QA-only MR pipeline +#### QA-only MR pipeline Reference pipeline: <https://gitlab.com/gitlab-org/gitlab/pipelines/134645109> @@ -457,11 +299,11 @@ graph RL; click 1-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8100542&udv=0" 1-2["build-qa-image (3.4 minutes)"]; click 1-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914325&udv=0" - 1-3["compile-assets pull-cache (9.06 minutes)"]; + 1-3["compile-test-assets (9.06 minutes)"]; click 1-3 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914317&udv=0" - 1-4["compile-assets pull-cache as-if-foss (8.35 minutes)"]; + 1-4["compile-test-assets as-if-foss (8.35 minutes)"]; click 1-4 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356616&udv=0" - 1-5["gitlab:assets:compile pull-cache (22 minutes)"]; + 1-5["compile-production-assets (22 minutes)"]; click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0" 1-6["setup-test-env (8.22 minutes)"]; click 1-6 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914315&udv=0" @@ -477,6 +319,8 @@ graph RL; 1-18["kubesec-sast"]; 1-19["nodejs-scan-sast"]; 1-20["secrets-sast"]; + 1-21["static-analysis (17 minutes)"]; + click 1-21 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914471&udv=0" class 1-5 criticalPath; end @@ -487,14 +331,8 @@ graph RL; 2_1-1 --> 1-6; end - 2_2-1["static-analysis (17 minutes)"]; - click 2_2-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914471&udv=0" - subgraph "Needs `setup-test-env` & `compile-assets`"; - 2_2-1 --> 1-6 & 1-3; - end - 2_3-1["build-assets-image (2.5 minutes)"]; - subgraph "Needs `gitlab:assets:compile`"; + subgraph "Needs `compile-production-assets`"; 2_3-1 --> 1-5 class 2_3-1 criticalPath; end @@ -507,24 +345,61 @@ graph RL; end ``` -## Test jobs +### `workflow:rules` + +We're using the [`workflow:rules` keyword](../ci/yaml/README.md#workflowrules) to +define default rules to determine whether or not a pipeline is created. + +These rules are defined in <https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml> +and are as follows: + +1. If `$FORCE_GITLAB_CI` is set, create a pipeline. +1. For merge requests, create a pipeline. +1. For `master` branch, create a pipeline (this includes on schedules, pushes, merges, etc.). +1. For tags, create a pipeline. +1. If `$GITLAB_INTERNAL` isn't set, don't create a pipeline. +1. For stable, auto-deploy, and security branches, create a pipeline. +1. For any other cases (e.g. when pushing a branch with no MR for it), no pipeline is created. + +### PostgreSQL versions testing + +#### Current versions testing + +| Where? | PostgreSQL version | +| ------ | ------ | +| MRs | 11 | +| `master` (non-scheduled pipelines) | 11 | +| 2-hourly scheduled pipelines | 11 | + +#### Long-term plan + +We follow the [PostgreSQL versions shipped with Omnibus GitLab](https://docs.gitlab.com/omnibus/package-information/postgresql_versions.html): + +| PostgreSQL version | 12.10 (April 2020) | 13.0 (May 2020) | 13.1 (June 2020) | 13.2 (July 2020) | 13.3 (August 2020) | 13.4, 13.5 | 13.6 (November 2020) | 14.0 (May 2021?) | +| ------ | ------------------ | --------------- | ---------------- | ---------------- | ------------------ | ------------ | -------------------- | ---------------- | +| PG9.6 | MRs/`master`/`2-hour`/`nightly` | - | - | - | - | - | - | - | +| PG10 | `nightly` | - | - | - | - | - | - | - | +| PG11 | `master`/`2-hour` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | MRs/`master`/`2-hour`/`nightly` | `nightly` | - | +| PG12 | - | - | - | - | `master`/`2-hour` | `master`/`2-hour` | MRs/`master`/`2-hour`/`nightly` | `master`/`2-hour` | +| PG13 | - | - | - | - | - | - | - | MRs/`master`/`2-hour`/`nightly` | + +### Test jobs Consult [GitLab tests in the Continuous Integration (CI) context](testing_guide/ci.md) for more information. -## Review app jobs +### Review app jobs Consult the [Review Apps](testing_guide/review_apps.md) dedicated page for more information. -## As-if-FOSS jobs +### As-if-FOSS jobs The `* as-if-foss` jobs allows to run GitLab's test suite "as-if-FOSS", meaning as if the jobs would run in the context of the `gitlab-org/gitlab-foss` project. These jobs are only created in the following cases: -- `master` commits (pushes and scheduled pipelines). - `gitlab-org/security/gitlab` merge requests. - Merge requests which include `RUN AS-IF-FOSS` in their title. -- Merge requests that changes the CI config. +- Merge requests that changes the CI configuration. The `* as-if-foss` jobs have the `FOSS_ONLY='1'` variable set and gets their EE-specific folders removed before the tests start running. @@ -532,9 +407,41 @@ folders removed before the tests start running. The intent is to ensure that a change won't introduce a failure once the `gitlab-org/gitlab` project will be synced to the `gitlab-org/gitlab-foss` project. -## Pre-clone step +## Performance -The `gitlab-org/gitlab` project on GitLab.com uses a [pre-clone step](https://gitlab.com/gitlab-org/gitlab/issues/39134) +### Interruptible pipelines + +By default, all jobs are [interruptible](../ci/yaml/README.md#interruptible), except the +`dont-interrupt-me` job which runs automatically on `master`, and is `manual` +otherwise. + +If you want a running pipeline to finish even if you push new commits to a merge +request, be sure to start the `dont-interrupt-me` job before pushing. + +### Caching strategy + +1. All jobs must only pull caches by default. +1. All jobs must be able to pass with an empty cache. In other words, caches are only there to speed up jobs. +1. We currently have 6 different caches defined in + [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml), + with fixed keys: + - `.rails-cache`. + - `.static-analysis-cache`. + - `.qa-cache` + - `.yarn-cache`. + - `.assets-compile-cache` (the key includes `${NODE_ENV}` so it's actually two different caches). +1. Only 6 specific jobs, running in 2-hourly scheduled pipelines, are pushing (i.e. updating) to the caches: + - `update-rails-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/rails.gitlab-ci.yml). + - `update-static-analysis-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/rails.gitlab-ci.yml). + - `update-qa-cache`, defined in [`.gitlab/ci/qa.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/qa.gitlab-ci.yml). + - `update-assets-compile-production-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/frontend.gitlab-ci.yml). + - `update-assets-compile-test-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/frontend.gitlab-ci.yml). + - `update-yarn-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/frontend.gitlab-ci.yml). +1. These jobs will run in merge requests whose title include `UPDATE CACHE`. + +### Pre-clone step + +The `gitlab-org/gitlab` project on GitLab.com uses a [pre-clone step](https://gitlab.com/gitlab-org/gitlab/-/issues/39134) to seed the project with a recent archive of the repository. This is done for several reasons: @@ -597,6 +504,124 @@ credentials are stored in the 1Password GitLab.com Production vault. Note that this bucket should be located in the same continent as the runner, or [network egress charges will apply](https://cloud.google.com/storage/pricing). +## CI configuration internals + +### Stages + +The current stages are: + +- `sync`: This stage is used to synchronize changes from <https://gitlab.com/gitlab-org/gitlab> to + <https://gitlab.com/gitlab-org/gitlab-foss>. +- `prepare`: This stage includes jobs that prepare artifacts that are needed by + jobs in subsequent stages. +- `build-images`: This stage includes jobs that prepare Docker images + that are needed by jobs in subsequent stages or downstream pipelines. +- `fixtures`: This stage includes jobs that prepare fixtures needed by frontend tests. +- `test`: This stage includes most of the tests, DB/migration jobs, and static analysis jobs. +- `post-test`: This stage includes jobs that build reports or gather data from + the `test` stage's jobs (e.g. coverage, Knapsack metadata etc.). +- `review-prepare`: This stage includes a job that build the CNG images that are + later used by the (Helm) Review App deployment (see + [Review Apps](testing_guide/review_apps.md) for details). +- `review`: This stage includes jobs that deploy the GitLab and Docs Review Apps. +- `qa`: This stage includes jobs that perform QA tasks against the Review App + that is deployed in the previous stage. +- `post-qa`: This stage includes jobs that build reports or gather data from + the `qa` stage's jobs (e.g. Review App performance report). +- `pages`: This stage includes a job that deploys the various reports as + GitLab Pages (e.g. [`coverage-ruby`](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/), + [`coverage-javascript`](https://gitlab-org.gitlab.io/gitlab/coverage-javascript/), + [`webpack-report`](https://gitlab-org.gitlab.io/gitlab/webpack-report/). + +### Default image + +The default image is defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml). + +It includes Ruby, Go, Git, Git LFS, Chrome, Node, Yarn, PostgreSQL, and Graphics Magick. + +The images used in our pipelines are configured in the +[`gitlab-org/gitlab-build-images`](https://gitlab.com/gitlab-org/gitlab-build-images) +project, which is push-mirrored to [`gitlab/gitlab-build-images`](https://dev.gitlab.org/gitlab/gitlab-build-images) +for redundancy. + +The current version of the build images can be found in the +["Used by GitLab section"](https://gitlab.com/gitlab-org/gitlab-build-images/blob/master/.gitlab-ci.yml). + +### Default variables + +In addition to the [predefined variables](../ci/variables/predefined_variables.md), +each pipeline includes default variables defined in +<https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml>. + +### Common job definitions + +Most of the jobs [extend from a few CI definitions](../ci/yaml/README.md#extends) +defined in [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml) +that are scoped to a single [configuration parameter](../ci/yaml/README.md#configuration-parameters). + +| Job definitions | Description | +|------------------|-------------| +| `.default-tags` | Ensures a job has the `gitlab-org` tag to ensure it's using our dedicated runners. | +| `.default-retry` | Allows a job to [retry](../ci/yaml/README.md#retry) upon `unknown_failure`, `api_failure`, `runner_system_failure`, `job_execution_timeout`, or `stuck_or_timeout_failure`. | +| `.default-before_script` | Allows a job to use a default `before_script` definition suitable for Ruby/Rails tasks that may need a database running (e.g. tests). | +| `.rails-cache` | Allows a job to use a default `cache` definition suitable for Ruby/Rails tasks. | +| `.static-analysis-cache` | Allows a job to use a default `cache` definition suitable for static analysis tasks. | +| `.yarn-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that do a `yarn install`. | +| `.assets-compile-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that compile assets. | +| `.use-pg11` | Allows a job to use the `postgres:11.6` and `redis:alpine` services. | +| `.use-pg11-ee` | Same as `.use-pg11` but also use the `docker.elastic.co/elasticsearch/elasticsearch:6.4.2` services. | +| `.use-kaniko` | Allows a job to use the `kaniko` tool to build Docker images. | +| `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` environment variable. | + +### `rules`, `if:` conditions and `changes:` patterns + +We're using the [`rules` keyword](../ci/yaml/README.md#rules) extensively. + +All `rules` definitions are defined in +<https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml>, +then included in individual jobs via [`extends`](../ci/yaml/README.md#extends). + +The `rules` definitions are composed of `if:` conditions and `changes:` patterns, +which are also defined in +[`rules.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml) +and included in `rules` definitions via [YAML anchors](../ci/yaml/README.md#anchors) + +#### `if:` conditions + +| `if:` conditions | Description | Notes | +|------------------|-------------|-------| +| `if-not-canonical-namespace` | Matches if the project isn't in the canonical (`gitlab-org/`) or security (`gitlab-org/security`) namespace. | Use to create a job for forks (by using `when: on_success\|manual`), or **not** create a job for forks (by using `when: never`). | +| `if-not-ee` | Matches if the project isn't EE (i.e. project name isn't `gitlab` or `gitlab-ee`). | Use to create a job only in the FOSS project (by using `when: on_success|manual`), or **not** create a job if the project is EE (by using `when: never`). | +| `if-not-foss` | Matches if the project isn't FOSS (i.e. project name isn't `gitlab-foss`, `gitlab-ce`, or `gitlabhq`). | Use to create a job only in the EE project (by using `when: on_success|manual`), or **not** create a job if the project is FOSS (by using `when: never`). | +| `if-default-refs` | Matches if the pipeline is for `master`, `/^[\d-]+-stable(-ee)?$/` (stable branches), `/^\d+-\d+-auto-deploy-\d+$/` (auto-deploy branches), `/^security\//` (security branches), merge requests, and tags. | Note that jobs won't be created for branches with this default configuration. | +| `if-master-refs` | Matches if the current branch is `master`. | | +| `if-master-or-tag` | Matches if the pipeline is for the `master` branch or for a tag. | | +| `if-merge-request` | Matches if the pipeline is for a merge request. | | +| `if-nightly-master-schedule` | Matches if the pipeline is for a `master` scheduled pipeline with `$NIGHTLY` set. | | +| `if-dot-com-gitlab-org-schedule` | Limits jobs creation to scheduled pipelines for the `gitlab-org` group on GitLab.com. | | +| `if-dot-com-gitlab-org-master` | Limits jobs creation to the `master` branch for the `gitlab-org` group on GitLab.com. | | +| `if-dot-com-gitlab-org-merge-request` | Limits jobs creation to merge requests for the `gitlab-org` group on GitLab.com. | | +| `if-dot-com-gitlab-org-and-security-tag` | Limits job creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | | +| `if-dot-com-gitlab-org-and-security-merge-request` | Limit jobs creation to merge requests for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | | +| `if-dot-com-ee-schedule` | Limits jobs to scheduled pipelines for the `gitlab-org/gitlab` project on GitLab.com. | | +| `if-cache-credentials-schedule` | Limits jobs to scheduled pipelines with the `$CI_REPO_CACHE_CREDENTIALS` variable set. | | + +#### `changes:` patterns + +| `changes:` patterns | Description | +|------------------------------|--------------------------------------------------------------------------| +| `ci-patterns` | Only create job for CI config-related changes. | +| `yaml-patterns` | Only create job for YAML-related changes. | +| `docs-patterns` | Only create job for docs-related changes. | +| `frontend-dependency-patterns` | Only create job when frontend dependencies are updated (i.e. `package.json`, and `yarn.lock`). changes. | +| `frontend-patterns` | Only create job for frontend-related changes. | +| `backstage-patterns` | Only create job for backstage-related changes (i.e. Danger, fixtures, RuboCop, specs). | +| `code-patterns` | Only create job for code-related changes. | +| `qa-patterns` | Only create job for QA-related changes. | +| `code-backstage-patterns` | Combination of `code-patterns` and `backstage-patterns`. | +| `code-qa-patterns` | Combination of `code-patterns` and `qa-patterns`. | +| `code-backstage-qa-patterns` | Combination of `code-patterns`, `backstage-patterns`, and `qa-patterns`. | + --- [Return to Development documentation](README.md) |