diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2024-01-02 18:10:19 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2024-01-02 18:10:19 +0300 |
commit | cef74ed0434b53fde0d7dcc2507b6f1639a63e7d (patch) | |
tree | 300bc30b6c4272cba3ab65297ede18bfffba2bd2 /doc/development/pipelines | |
parent | ecdd26856c46b1e9e0c500701b36b6ae338e18a0 (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc/development/pipelines')
-rw-r--r-- | doc/development/pipelines/index.md | 93 | ||||
-rw-r--r-- | doc/development/pipelines/internals.md | 19 |
2 files changed, 76 insertions, 36 deletions
diff --git a/doc/development/pipelines/index.md b/doc/development/pipelines/index.md index 072fb6382b4..aebb0f9076d 100644 --- a/doc/development/pipelines/index.md +++ b/doc/development/pipelines/index.md @@ -300,6 +300,23 @@ set and get the `ee/` folder removed before the tests start running. The intent is to ensure that a change doesn't introduce a failure after `gitlab-org/gitlab` is synced to `gitlab-org/gitlab-foss`. +#### As-if-FOSS cross project downstream pipeline + +As an alternative to the `* as-if-foss` jobs, we can also run a cross project +FOSS pipeline exactly in the `gitlab-org/gitlab-foss` project. We trigger it +in the following cases: + +- when the `pipeline:run-as-if-foss-cross-project` label is set on the merge request + +This is still working-in-progress to replace the `* as-if-foss` jobs. The +goal is to simplify pipeline rules and make it more clear about the intention. + +##### Tokens set in the project variables + +- `AS_IF_FOSS_TOKEN`: This is a [GitLab FOSS](https://gitlab.com/gitlab-org/gitlab-foss) + project token with `developer` role and `write_repository` permission, + to push generated `as-if-foss/*` branch. + ### As-if-JH cross project downstream pipeline #### What it is @@ -399,7 +416,8 @@ flowchart TD - `ADD_JH_FILES_TOKEN`: This is a [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab) project token with `read_api` permission, to be able to download JiHu files. - `AS_IF_JH_TOKEN`: This is a [GitLab JH validation](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation) - project token with `write_repository` permission, to push generated `as-if-jh/*` branch. + project token with `developer` role and `write_repository` permission, + to push generated `as-if-jh/*` branch. ##### How we generate the as-if-JH branch @@ -613,30 +631,30 @@ Exceptions to this general guideline should be motivated and documented. ### Ruby versions testing -We're running Ruby 3.0 on GitLab.com, as well as for the default branch. -To prepare for the next Ruby version, we run merge requests in Ruby 3.1. +We're running Ruby 3.1 on GitLab.com, as well as for the default branch. +To prepare for the next Ruby version, we will run merge requests in Ruby 3.2, +starting on February 2024. Please see the roadmap at +[Ruby 3.2 epic](https://gitlab.com/groups/gitlab-org/-/epics/9684#plan) +for more details. -This takes effects at the time when -[Run merge requests in Ruby 3.1 by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/134290) -is merged. See -[Ruby 3.1 epic](https://gitlab.com/groups/gitlab-org/-/epics/10034) -for the roadmap to fully make Ruby 3.1 the default. +To make sure all supported Ruby versions are working, we also run our test +suite on dedicated 2-hourly scheduled pipelines for each supported versions. -To make sure both Ruby versions are working, we also run our test suite -against both Ruby 3.0 and Ruby 3.1 on dedicated 2-hourly scheduled pipelines. +For merge requests, you can add the following labels to run the respective +Ruby version only: -For merge requests, you can add the `pipeline:run-in-ruby3_0` label to switch -the Ruby version to 3.0. When you do this, the test suite will no longer run -in Ruby 3.1 (default for merge requests). +- `pipeline:run-in-ruby3_0` +- `pipeline:run-in-ruby3_1` +- `pipeline:run-in-ruby3_2` -When the pipeline is running in a Ruby version not considered default, an -additional job `verify-default-ruby` will also run and always fail to remind -us to remove the label and run in default Ruby before merging the merge -request. At the moment both Ruby 3.0 and Ruby 3.1 are considered default. +Note that when you do this, the test suite will no longer run in the default +Ruby version for merge requests. In this case, an additional job +`verify-default-ruby` will also run and always fail to remind us to remove +the label and run in default Ruby before merging the merge request. This should let us: -- Test changes for Ruby 3.1 +- Test changes for any supported Ruby versions - Make sure it will not break anything when it's merged into the default branch ### PostgreSQL versions testing @@ -652,24 +670,27 @@ We also run our test suite against PostgreSQL 13 upon specific database library | Where? | PostgreSQL version | Ruby version | |--------------------------------------------------------------------------------------------------|-------------------------------------------------|-----------------------| -| Merge requests | 14 (default version), 13 for DB library changes | 3.1 | -| `master` branch commits | 14 (default version), 13 for DB library changes | 3.0 (default version) | -| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour) | 14 (default version), 13 for DB library changes | 3.0 (default version) | -| `maintenance` scheduled pipelines for the `ruby3_1` branch (every odd-numbered hour), see below. | 14 (default version), 13 for DB library changes | 3.1 | -| `nightly` scheduled pipelines for the `master` branch | 14 (default version), 13, 15 | 3.0 (default version) | - -There are 2 pipeline schedules used for testing Ruby 3.1. One is triggering a -pipeline in `ruby3_1-sync` branch, which updates the `ruby3_1` branch with latest -`master`, and no pipelines will be triggered by this push. The other schedule -is triggering a pipeline in `ruby3_1` 5 minutes after it, which is considered -the maintenance schedule to run test suites and update cache. - -The `ruby3_1` branch must not have any changes. The branch is only there to set -`RUBY_VERSION` to `3.1` in the maintenance pipeline schedule. - -The `gitlab` job in the `ruby3_1-sync` branch uses a `gitlab-org/gitlab` project -token with `write_repository` scope and `Maintainer` role with no expiration. -The token is stored in the `RUBY3_1_SYNC_TOKEN` variable in `gitlab-org/gitlab`. +| Merge requests | 14 (default version), 13 for DB library changes | 3.1 (default version) | +| `master` branch commits | 14 (default version), 13 for DB library changes | 3.1 (default version) | +| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour at XX:05) | 14 (default version), 13 for DB library changes | 3.1 (default version) | +| `maintenance` scheduled pipelines for the `ruby3_0` branch (every odd-numbered hour at XX:40) | 14 (default version), 13 for DB library changes | 3.0 | +| `maintenance` scheduled pipelines for the `ruby3_2` branch (every odd-numbered hour at XX:10) | 14 (default version), 13 for DB library changes | 3.2 | +| `nightly` scheduled pipelines for the `master` branch | 14 (default version), 13, 15 | 3.1 (default version) | + +For each current Ruby versions we're testing against with, we run +maintenance scheduled pipelines every 2 hours on their respective `ruby\d_\d` +branches. All these branches must not have any changes. These branches are +only there to run pipelines with their respective Ruby versions in the +scheduled maintenance pipelines. + +Additionally, we have scheduled pipelines running on `ruby-sync` branch also +every 2 hours, updating all the `ruby\d_\d` branches to be up-to-date with +the default branch `master`. No pipelines will be triggered by this push. + +The `gitlab` job in the `ruby-sync` branch uses a `gitlab-org/gitlab` project +token named `RUBY_SYNC` with `write_repository` scope and `Maintainer` role, +expiring on 2024-12-01. The token is stored in the `RUBY_SYNC_TOKEN` variable +in the pipeline schedule for `ruby-sync` branch. ### Redis versions testing diff --git a/doc/development/pipelines/internals.md b/doc/development/pipelines/internals.md index b3b4cbec02d..a81eb6f656b 100644 --- a/doc/development/pipelines/internals.md +++ b/doc/development/pipelines/internals.md @@ -48,6 +48,25 @@ from using `$FORCE_GITLAB_CI`. - [JiHu validation pipeline](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/jihu-validation-pipelines.html) - [Gitaly downstream GitLab pipeline](https://gitlab.com/gitlab-org/gitaly/-/issues/4615) +See the next section for how we can enable pipelines without using +`$FORCE_GITLAB_CI`. + +#### Alternative to `$FORCE_GITLAB_CI` + +Essentially, we use different variables to enable different pipelines. +An example doing this is `$START_AS_IF_FOSS`. When we want to trigger a +cross project FOSS pipeline, we set `$START_AS_IF_FOSS`, along with a set of +other variables like `$ENABLE_RSPEC_UNIT`, `$ENABLE_RSPEC_SYSTEM`, and so on +so forth to enable each jobs we want to run in the as-if-foss cross project +downstream pipeline. + +The advantage of this over `$FORCE_GITLAB_CI` is that we have full control +over how we want to run the pipeline because `$START_AS_IF_FOSS` is only used +for this purpose, and changing how the pipeline behaves under this variable +will not affect other types of pipelines, while using `$FORCE_GITLAB_CI` we +do not know what exactly the pipeline is because it's used for multiple +purposes. + ## Default image The default image is defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml). |