diff options
Diffstat (limited to 'doc/development/pipelines.md')
| -rw-r--r-- | doc/development/pipelines.md | 43 |
1 files changed, 21 insertions, 22 deletions
diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index 4d931899da6..aa3f2e6791a 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -19,7 +19,7 @@ as much as possible. ## Overview -Pipelines for the GitLab project are created using the [`workflow:rules` keyword](../ci/yaml/README.md#workflowrules) +Pipelines for the GitLab project are created using the [`workflow:rules` keyword](../ci/yaml/README.md#workflow) feature of the GitLab CI/CD. Pipelines are always created for the following scenarios: @@ -29,7 +29,7 @@ Pipelines are always created for the following scenarios: - Tags. - Stable, `auto-deploy`, and security branches. -Pipeline creation is also affected by the following CI variables: +Pipeline creation is also affected by the following CI/CD variables: - If `$FORCE_GITLAB_CI` is set, pipelines are created. - If `$GITLAB_INTERNAL` is not set, pipelines are not created. @@ -414,10 +414,10 @@ The `rspec fail-fast` is a no-op if there are more than 10 test files related to Merge Request. This prevents `rspec fail-fast` duration from exceeding the average `rspec` job duration and defeating its purpose. -This number can be overridden by setting a CI variable named `RSPEC_FAIL_FAST_TEST_FILE_COUNT_THRESHOLD`. +This number can be overridden by setting a CI/CD variable named `RSPEC_FAIL_FAST_TEST_FILE_COUNT_THRESHOLD`. NOTE: -This experiment is only enabled when the CI variable `RSPEC_FAIL_FAST_ENABLED=true` is set. +This experiment is only enabled when the CI/CD variable `RSPEC_FAIL_FAST_ENABLED=true` is set. #### Determining related test files in a Merge Request @@ -426,7 +426,7 @@ We are using a custom mapping between source file to test files, maintained in t ### PostgreSQL versions testing -Even though [Omnibus defaults to PG12 for new installs and upgrades](https://docs.gitlab.com/omnibus/package-information/postgresql_versions.md), +Even though [Omnibus defaults to PG12 for new installs and upgrades](https://docs.gitlab.com/omnibus/package-information/postgresql_versions.html), our test suite is currently running against PG11, since GitLab.com still runs on PG11. We do run our test suite against PG12 on nightly scheduled pipelines as well as upon specific @@ -531,8 +531,7 @@ several reasons: The pre-clone step works by using the `CI_PRE_CLONE_SCRIPT` variable [defined by GitLab.com shared runners](../user/gitlab_com/index.md#pre-clone-script). -The `CI_PRE_CLONE_SCRIPT` is currently defined as a project CI/CD -variable: +The `CI_PRE_CLONE_SCRIPT` is currently defined as a project CI/CD variable: ```shell echo "Downloading archived master..." @@ -638,7 +637,7 @@ Some of the jobs are using images from Docker Hub, where we also use `${GITLAB_DEPENDENCY_PROXY}` as a prefix to the image path, so that we pull images from our [Dependency Proxy](../user/packages/dependency_proxy/index.md). -`${GITLAB_DEPENDENCY_PROXY}` is a group variable defined in +`${GITLAB_DEPENDENCY_PROXY}` is a group CI/CD variable defined in [`gitlab-org`](https://gitlab.com/gitlab-org) as `${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/`. This means when we use an image defined as: @@ -653,7 +652,7 @@ Docker Hub unless `${GITLAB_DEPENDENCY_PROXY}` is also defined there. ### Default variables -In addition to the [predefined variables](../ci/variables/predefined_variables.md), +In addition to the [predefined CI/CD variables](../ci/variables/predefined_variables.md), each pipeline includes default variables defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml). @@ -674,12 +673,12 @@ that are scoped to a single [configuration keyword](../ci/yaml/README.md#job-key | `.qa-cache` | Allows a job to use a default `cache` definition suitable for QA tasks. | | `.yarn-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that do a `yarn install`. | | `.assets-compile-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that compile assets. | -| `.use-pg11` | Allows a job to use the `postgres:11.6` and `redis:4.0-alpine` services. | -| `.use-pg11-ee` | Same as `.use-pg11` but also use the `docker.elastic.co/elasticsearch/elasticsearch:7.9.2` services. | -| `.use-pg12` | Allows a job to use the `postgres:12` and `redis:4.0-alpine` services. | -| `.use-pg12-ee` | Same as `.use-pg12` but also use the `docker.elastic.co/elasticsearch/elasticsearch:7.9.2` services. | +| `.use-pg11` | Allows a job to run the `postgres` 11 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). | +| `.use-pg11-ee` | Same as `.use-pg11` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). | +| `.use-pg12` | Allows a job to use the `postgres` 12 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). | +| `.use-pg12-ee` | Same as `.use-pg12` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). | | `.use-kaniko` | Allows a job to use the `kaniko` tool to build Docker images. | -| `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` environment variable. | +| `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` CI/CD variable. | | `.use-docker-in-docker` | Allows a job to use Docker in Docker. | ### `rules`, `if:` conditions and `changes:` patterns @@ -704,13 +703,13 @@ and included in `rules` definitions via [YAML anchors](../ci/yaml/README.md#anch | `if-not-canonical-namespace` | Matches if the project isn't in the canonical (`gitlab-org/`) or security (`gitlab-org/security`) namespace. | Use to create a job for forks (by using `when: on_success|manual`), or **not** create a job for forks (by using `when: never`). | | `if-not-ee` | Matches if the project isn't EE (i.e. project name isn't `gitlab` or `gitlab-ee`). | Use to create a job only in the FOSS project (by using `when: on_success|manual`), or **not** create a job if the project is EE (by using `when: never`). | | `if-not-foss` | Matches if the project isn't FOSS (i.e. project name isn't `gitlab-foss`, `gitlab-ce`, or `gitlabhq`). | Use to create a job only in the EE project (by using `when: on_success|manual`), or **not** create a job if the project is FOSS (by using `when: never`). | -| `if-default-refs` | Matches if the pipeline is for `master`, `/^[\d-]+-stable(-ee)?$/` (stable branches), `/^\d+-\d+-auto-deploy-\d+$/` (auto-deploy branches), `/^security\//` (security branches), merge requests, and tags. | Note that jobs aren't created for branches with this default configuration. | -| `if-master-refs` | Matches if the current branch is `master`. | | -| `if-master-push` | Matches if the current branch is `master` and pipeline source is `push`. | | -| `if-master-schedule-2-hourly` | Matches if the current branch is `master` and pipeline runs on a 2-hourly schedule. | | -| `if-master-schedule-nightly` | Matches if the current branch is `master` and pipeline runs on a nightly schedule. | | +| `if-default-refs` | Matches if the pipeline is for `master`, `main`, `/^[\d-]+-stable(-ee)?$/` (stable branches), `/^\d+-\d+-auto-deploy-\d+$/` (auto-deploy branches), `/^security\//` (security branches), merge requests, and tags. | Note that jobs aren't created for branches with this default configuration. | +| `if-master-refs` | Matches if the current branch is `master` or `main`. | | +| `if-master-push` | Matches if the current branch is `master` or `main` and pipeline source is `push`. | | +| `if-master-schedule-2-hourly` | Matches if the current branch is `master` or `main` and pipeline runs on a 2-hourly schedule. | | +| `if-master-schedule-nightly` | Matches if the current branch is `master` or `main` and pipeline runs on a nightly schedule. | | | `if-auto-deploy-branches` | Matches if the current branch is an auto-deploy one. | | -| `if-master-or-tag` | Matches if the pipeline is for the `master` branch or for a tag. | | +| `if-master-or-tag` | Matches if the pipeline is for the `master` or `main` branch or for a tag. | | | `if-merge-request` | Matches if the pipeline is for a merge request. | | | `if-merge-request-title-as-if-foss` | Matches if the pipeline is for a merge request and the MR title includes "RUN AS-IF-FOSS". | | | `if-merge-request-title-update-caches` | Matches if the pipeline is for a merge request and the MR title includes "UPDATE CACHE". | | @@ -719,14 +718,14 @@ and included in `rules` definitions via [YAML anchors](../ci/yaml/README.md#anch | `if-security-schedule` | Matches if the pipeline is for a security scheduled pipeline. | | | `if-nightly-master-schedule` | Matches if the pipeline is for a `master` scheduled pipeline with `$NIGHTLY` set. | | | `if-dot-com-gitlab-org-schedule` | Limits jobs creation to scheduled pipelines for the `gitlab-org` group on GitLab.com. | | -| `if-dot-com-gitlab-org-master` | Limits jobs creation to the `master` branch for the `gitlab-org` group on GitLab.com. | | +| `if-dot-com-gitlab-org-master` | Limits jobs creation to the `master` or `main` branch for the `gitlab-org` group on GitLab.com. | | | `if-dot-com-gitlab-org-merge-request` | Limits jobs creation to merge requests for the `gitlab-org` group on GitLab.com. | | | `if-dot-com-gitlab-org-and-security-tag` | Limits job creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | | | `if-dot-com-gitlab-org-and-security-merge-request` | Limit jobs creation to merge requests for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | | | `if-dot-com-gitlab-org-and-security-tag` | Limit jobs creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | | | `if-dot-com-ee-schedule` | Limits jobs to scheduled pipelines for the `gitlab-org/gitlab` project on GitLab.com. | | | `if-cache-credentials-schedule` | Limits jobs to scheduled pipelines with the `$CI_REPO_CACHE_CREDENTIALS` variable set. | | -| `if-rspec-fail-fast-disabled` | Limits jobs to pipelines with `$RSPEC_FAIL_FAST_ENABLED` variable not set to `"true"`. | | +| `if-rspec-fail-fast-disabled` | Limits jobs to pipelines with `$RSPEC_FAIL_FAST_ENABLED` CI/CD variable not set to `"true"`. | | | `if-rspec-fail-fast-skipped` | Matches if the pipeline is for a merge request and the MR title includes "SKIP RSPEC FAIL-FAST". | | | `if-security-pipeline-merge-result` | Matches if the pipeline is for a security merge request triggered by `@gitlab-release-tools-bot`. | | |