From cef74ed0434b53fde0d7dcc2507b6f1639a63e7d Mon Sep 17 00:00:00 2001 From: GitLab Bot Date: Tue, 2 Jan 2024 15:10:19 +0000 Subject: Add latest changes from gitlab-org/gitlab@master --- doc/administration/backup_restore/backup_gitlab.md | 3 +- doc/api/groups.md | 1 + doc/development/pipelines/index.md | 93 +++++++++++++--------- doc/development/pipelines/internals.md | 19 +++++ doc/update/index.md | 9 ++- doc/update/versions/gitlab_16_changes.md | 47 +++++++++-- doc/user/custom_roles.md | 4 +- 7 files changed, 126 insertions(+), 50 deletions(-) (limited to 'doc') diff --git a/doc/administration/backup_restore/backup_gitlab.md b/doc/administration/backup_restore/backup_gitlab.md index f38358810e3..707b209faf3 100644 --- a/doc/administration/backup_restore/backup_gitlab.md +++ b/doc/administration/backup_restore/backup_gitlab.md @@ -656,7 +656,8 @@ sudo -u git -H bundle exec rake gitlab:backup:create REPOSITORIES_STORAGES=stora #### Back up specific repositories -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88094) in GitLab 15.1. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88094) in GitLab 15.1. +> - [Skipping specific repositories added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121865) in GitLab 16.1. You can back up specific repositories using the `REPOSITORIES_PATHS` option. Similarly, you can use `SKIP_REPOSITORIES_PATHS` to skip certain repositories. diff --git a/doc/api/groups.md b/doc/api/groups.md index 938cd9dbab1..e168adef6e2 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -1704,6 +1704,7 @@ To delete the LDAP group link, provide either a `cn` or a `filter`, but not both > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/290367) in GitLab 15.3.0. > - `access_level` type [changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95607) from `string` to `integer` in GitLab 15.3.3. > - `member_role_id` type [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/417201) in GitLab 16.7 [with a flag](../administration/feature_flags.md) named `custom_roles_for_saml_group_links`. Disabled by default. +> - `member_role_id` type [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/417201) in GitLab 16.8. Feature flag `custom_roles_for_saml_group_links` removed. List, get, add, and delete SAML group links. diff --git a/doc/development/pipelines/index.md b/doc/development/pipelines/index.md index 072fb6382b4..aebb0f9076d 100644 --- a/doc/development/pipelines/index.md +++ b/doc/development/pipelines/index.md @@ -300,6 +300,23 @@ set and get the `ee/` folder removed before the tests start running. The intent is to ensure that a change doesn't introduce a failure after `gitlab-org/gitlab` is synced to `gitlab-org/gitlab-foss`. +#### As-if-FOSS cross project downstream pipeline + +As an alternative to the `* as-if-foss` jobs, we can also run a cross project +FOSS pipeline exactly in the `gitlab-org/gitlab-foss` project. We trigger it +in the following cases: + +- when the `pipeline:run-as-if-foss-cross-project` label is set on the merge request + +This is still working-in-progress to replace the `* as-if-foss` jobs. The +goal is to simplify pipeline rules and make it more clear about the intention. + +##### Tokens set in the project variables + +- `AS_IF_FOSS_TOKEN`: This is a [GitLab FOSS](https://gitlab.com/gitlab-org/gitlab-foss) + project token with `developer` role and `write_repository` permission, + to push generated `as-if-foss/*` branch. + ### As-if-JH cross project downstream pipeline #### What it is @@ -399,7 +416,8 @@ flowchart TD - `ADD_JH_FILES_TOKEN`: This is a [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab) project token with `read_api` permission, to be able to download JiHu files. - `AS_IF_JH_TOKEN`: This is a [GitLab JH validation](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation) - project token with `write_repository` permission, to push generated `as-if-jh/*` branch. + project token with `developer` role and `write_repository` permission, + to push generated `as-if-jh/*` branch. ##### How we generate the as-if-JH branch @@ -613,30 +631,30 @@ Exceptions to this general guideline should be motivated and documented. ### Ruby versions testing -We're running Ruby 3.0 on GitLab.com, as well as for the default branch. -To prepare for the next Ruby version, we run merge requests in Ruby 3.1. +We're running Ruby 3.1 on GitLab.com, as well as for the default branch. +To prepare for the next Ruby version, we will run merge requests in Ruby 3.2, +starting on February 2024. Please see the roadmap at +[Ruby 3.2 epic](https://gitlab.com/groups/gitlab-org/-/epics/9684#plan) +for more details. -This takes effects at the time when -[Run merge requests in Ruby 3.1 by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/134290) -is merged. See -[Ruby 3.1 epic](https://gitlab.com/groups/gitlab-org/-/epics/10034) -for the roadmap to fully make Ruby 3.1 the default. +To make sure all supported Ruby versions are working, we also run our test +suite on dedicated 2-hourly scheduled pipelines for each supported versions. -To make sure both Ruby versions are working, we also run our test suite -against both Ruby 3.0 and Ruby 3.1 on dedicated 2-hourly scheduled pipelines. +For merge requests, you can add the following labels to run the respective +Ruby version only: -For merge requests, you can add the `pipeline:run-in-ruby3_0` label to switch -the Ruby version to 3.0. When you do this, the test suite will no longer run -in Ruby 3.1 (default for merge requests). +- `pipeline:run-in-ruby3_0` +- `pipeline:run-in-ruby3_1` +- `pipeline:run-in-ruby3_2` -When the pipeline is running in a Ruby version not considered default, an -additional job `verify-default-ruby` will also run and always fail to remind -us to remove the label and run in default Ruby before merging the merge -request. At the moment both Ruby 3.0 and Ruby 3.1 are considered default. +Note that when you do this, the test suite will no longer run in the default +Ruby version for merge requests. In this case, an additional job +`verify-default-ruby` will also run and always fail to remind us to remove +the label and run in default Ruby before merging the merge request. This should let us: -- Test changes for Ruby 3.1 +- Test changes for any supported Ruby versions - Make sure it will not break anything when it's merged into the default branch ### PostgreSQL versions testing @@ -652,24 +670,27 @@ We also run our test suite against PostgreSQL 13 upon specific database library | Where? | PostgreSQL version | Ruby version | |--------------------------------------------------------------------------------------------------|-------------------------------------------------|-----------------------| -| Merge requests | 14 (default version), 13 for DB library changes | 3.1 | -| `master` branch commits | 14 (default version), 13 for DB library changes | 3.0 (default version) | -| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour) | 14 (default version), 13 for DB library changes | 3.0 (default version) | -| `maintenance` scheduled pipelines for the `ruby3_1` branch (every odd-numbered hour), see below. | 14 (default version), 13 for DB library changes | 3.1 | -| `nightly` scheduled pipelines for the `master` branch | 14 (default version), 13, 15 | 3.0 (default version) | - -There are 2 pipeline schedules used for testing Ruby 3.1. One is triggering a -pipeline in `ruby3_1-sync` branch, which updates the `ruby3_1` branch with latest -`master`, and no pipelines will be triggered by this push. The other schedule -is triggering a pipeline in `ruby3_1` 5 minutes after it, which is considered -the maintenance schedule to run test suites and update cache. - -The `ruby3_1` branch must not have any changes. The branch is only there to set -`RUBY_VERSION` to `3.1` in the maintenance pipeline schedule. - -The `gitlab` job in the `ruby3_1-sync` branch uses a `gitlab-org/gitlab` project -token with `write_repository` scope and `Maintainer` role with no expiration. -The token is stored in the `RUBY3_1_SYNC_TOKEN` variable in `gitlab-org/gitlab`. +| Merge requests | 14 (default version), 13 for DB library changes | 3.1 (default version) | +| `master` branch commits | 14 (default version), 13 for DB library changes | 3.1 (default version) | +| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour at XX:05) | 14 (default version), 13 for DB library changes | 3.1 (default version) | +| `maintenance` scheduled pipelines for the `ruby3_0` branch (every odd-numbered hour at XX:40) | 14 (default version), 13 for DB library changes | 3.0 | +| `maintenance` scheduled pipelines for the `ruby3_2` branch (every odd-numbered hour at XX:10) | 14 (default version), 13 for DB library changes | 3.2 | +| `nightly` scheduled pipelines for the `master` branch | 14 (default version), 13, 15 | 3.1 (default version) | + +For each current Ruby versions we're testing against with, we run +maintenance scheduled pipelines every 2 hours on their respective `ruby\d_\d` +branches. All these branches must not have any changes. These branches are +only there to run pipelines with their respective Ruby versions in the +scheduled maintenance pipelines. + +Additionally, we have scheduled pipelines running on `ruby-sync` branch also +every 2 hours, updating all the `ruby\d_\d` branches to be up-to-date with +the default branch `master`. No pipelines will be triggered by this push. + +The `gitlab` job in the `ruby-sync` branch uses a `gitlab-org/gitlab` project +token named `RUBY_SYNC` with `write_repository` scope and `Maintainer` role, +expiring on 2024-12-01. The token is stored in the `RUBY_SYNC_TOKEN` variable +in the pipeline schedule for `ruby-sync` branch. ### Redis versions testing diff --git a/doc/development/pipelines/internals.md b/doc/development/pipelines/internals.md index b3b4cbec02d..a81eb6f656b 100644 --- a/doc/development/pipelines/internals.md +++ b/doc/development/pipelines/internals.md @@ -48,6 +48,25 @@ from using `$FORCE_GITLAB_CI`. - [JiHu validation pipeline](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/jihu-validation-pipelines.html) - [Gitaly downstream GitLab pipeline](https://gitlab.com/gitlab-org/gitaly/-/issues/4615) +See the next section for how we can enable pipelines without using +`$FORCE_GITLAB_CI`. + +#### Alternative to `$FORCE_GITLAB_CI` + +Essentially, we use different variables to enable different pipelines. +An example doing this is `$START_AS_IF_FOSS`. When we want to trigger a +cross project FOSS pipeline, we set `$START_AS_IF_FOSS`, along with a set of +other variables like `$ENABLE_RSPEC_UNIT`, `$ENABLE_RSPEC_SYSTEM`, and so on +so forth to enable each jobs we want to run in the as-if-foss cross project +downstream pipeline. + +The advantage of this over `$FORCE_GITLAB_CI` is that we have full control +over how we want to run the pipeline because `$START_AS_IF_FOSS` is only used +for this purpose, and changing how the pipeline behaves under this variable +will not affect other types of pipelines, while using `$FORCE_GITLAB_CI` we +do not know what exactly the pipeline is because it's used for multiple +purposes. + ## Default image The default image is defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml). diff --git a/doc/update/index.md b/doc/update/index.md index c0a6b64a1ac..0c43af3b925 100644 --- a/doc/update/index.md +++ b/doc/update/index.md @@ -192,12 +192,13 @@ When upgrading: - GitLab 15: [`15.0.5`](versions/gitlab_15_changes.md#1500) > [`15.1.6`](versions/gitlab_15_changes.md#1510) (for GitLab instances with multiple web nodes) > [`15.4.6`](versions/gitlab_15_changes.md#1540) > [`15.11.13`](versions/gitlab_15_changes.md#15110). - - GitLab 16: [`16.0.x`](versions/gitlab_16_changes.md#1600) (only + - GitLab 16: [`16.0.8`](versions/gitlab_16_changes.md#1600) (only instances with [lots of users](versions/gitlab_16_changes.md#long-running-user-type-data-change) or [large pipeline variables history](versions/gitlab_16_changes.md#1610)) > - [`16.1`](versions/gitlab_16_changes.md#1610) (instances with NPM packages in their package registry) > - [`16.2.x`](versions/gitlab_16_changes.md#1620) (only instances with [large pipeline variables history](versions/gitlab_16_changes.md#1630)) > - [`16.3`](versions/gitlab_16_changes.md#1630) > [latest `16.Y.Z`](https://gitlab.com/gitlab-org/gitlab/-/releases). + [`16.1.5`](versions/gitlab_16_changes.md#1610) (instances with NPM packages in their package registry) > + [`16.2.8`](versions/gitlab_16_changes.md#1620) (only instances with [large pipeline variables history](versions/gitlab_16_changes.md#1630)) > + [`16.3.6`](versions/gitlab_16_changes.md#1630) > [`16.7.z`](versions/gitlab_16_changes.md#1670) + > [latest `16.Y.Z`](https://gitlab.com/gitlab-org/gitlab/-/releases). 1. Check for [required upgrade stops](#required-upgrade-stops). 1. Consult the [version-specific upgrade instructions](#version-specific-upgrading-instructions). diff --git a/doc/update/versions/gitlab_16_changes.md b/doc/update/versions/gitlab_16_changes.md index 7108f0eaa3f..a350ec88b0e 100644 --- a/doc/update/versions/gitlab_16_changes.md +++ b/doc/update/versions/gitlab_16_changes.md @@ -17,22 +17,55 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f ## Issues to be aware of when upgrading from 15.11 - [PostgreSQL 12 is not supported starting from GitLab 16](../../update/deprecations.md#postgresql-12-deprecated). Upgrade PostgreSQL to at least version 13.6 before upgrading to GitLab 16.0 or later. -- Some GitLab installations must upgrade to GitLab 16.0 before upgrading to any other version. For more information, see - [Long-running user type data change](#long-running-user-type-data-change). -- Other installations can skip 16.0, 16.1, and 16.2 as the first required stop on the upgrade path is 16.3. Review the notes for those intermediate - versions. - If your GitLab instance upgraded first to 15.11.0, 15.11.1, or 15.11.2 the database schema is incorrect. - Recommended: perform the workaround before upgrading to 16.x. - See [the details and workaround](#undefined-column-error-upgrading-to-162-or-later). -- Linux package installations must change Gitaly and Praefect configuration structure before upgrading to GitLab 16. + Perform the [workaround](#undefined-column-error-upgrading-to-162-or-later) before upgrading to 16.x. +- Most installations can skip 16.0, 16.1, and 16.2, as the first required stop on the upgrade path is 16.3. + In all cases, you should review the notes for those intermediate versions. + + Some GitLab installations must stop at those intermediate versions depending on which features are used + and the size of the environment: + + - 16.0.8: Instances with lots of records in the users table. + For more information, see [long-running user type data change](#long-running-user-type-data-change). + - [16.1.5](#1610): Instances that use the NPM package registry. + - [16.2.8](#1620): Instances with lots of pipeline variables (including historical pipelines). + + If your instance is affected and you skip these stops: + + - The upgrade can take hours to complete. + - The instance generates 500 errors until all the database changes are finished, after which + Puma and Sidekiq must restarted. + - For Linux package installations, a timeout occurs and a + [manual workaround to complete the migrations](../package/package_troubleshooting.md#mixlibshelloutcommandtimeout-rails_migrationgitlab-rails--command-timed-out-after-3600s) + is necessary. + +- GitLab 16.0 introduced changes around enforcing limits on project sizes. On self-managed, if you use + these limits, projects that have reached their limit causes error messages when pushing to unaffected Git + repositories in the same group. The errors often refer to exceeding a limit of zero bytes (`limit of 0 B`). + + The pushes succeed, but the errors imply otherwise and might cause issues with automation. + [Read more in the issue](https://gitlab.com/gitlab-org/gitlab/-/issues/416646). + The [bug is fixed in GitLab 16.5 and later](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/131122). + +### Linux package installations + +- Gitaly and Praefect configuration structure must be changed before upgrading to GitLab 16. **To avoid data loss** reconfigure Praefect first, and as part of the new configuration, disable metadata verification. Read more: - [Praefect configuration structure change](#praefect-configuration-structure-change). - [Gitaly configuration structure change](#gitaly-configuration-structure-change). +- If you reconfigure Gitaly to store Git data in a location other than `/var/opt/gitlab/git-data/repositories`, + packaged GitLab 16.0 and later does not automatically create the directory structure. + [Read the issue for more details and the workaround](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8320). + ## 16.7.0 +- GitLab 16.7 is a required upgrade stop. This ensures that all database changes introduced + in GitLab 16.7 and earlier have been implemented on all self-managed instances. Dependent changes can then be released + in GitLab 16.8 and later. [Issue 429611](https://gitlab.com/gitlab-org/gitlab/-/issues/429611) provides more details. + ### Linux package installations Specific information applies to Linux package installations: diff --git a/doc/user/custom_roles.md b/doc/user/custom_roles.md index 3ab4e31859c..99e8b8b5d52 100644 --- a/doc/user/custom_roles.md +++ b/doc/user/custom_roles.md @@ -223,8 +223,8 @@ To remove a custom role from a group member: 1. Select the **Max role** dropdown list for the member you want to remove a custom role from. 1. On the **Change role** dialog, select a static role. -You can update or remove a custom role from a group member also with the [Group and Project Members API endpoint](../api/members.md#edit-a-member-of-a-group-or-project). -and pass an empty `member_role_id` value: +You can also use the [Group and Project Members API endpoint](../api/members.md#edit-a-member-of-a-group-or-project) +to update or remove a custom role from a group member by passing an empty `member_role_id` value: ```shell # to update a project membership -- cgit v1.2.3