Add latest changes from gitlab-org/gitlab@master

3 files changed, 259 insertions, 29 deletions

diff --git a/doc/update/versions/gitlab_14_changes.md b/doc/update/versions/gitlab_14_changes.md
index 5cf3ebc5f9a..5e885ae1bd4 100644
--- a/doc/update/versions/gitlab_14_changes.md
+++ b/doc/update/versions/gitlab_14_changes.md
@@ -89,6 +89,17 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
   [check you have a PostgreSQL bug fix](../index.md#postgresql-segmentation-fault-issue)
   to avoid the database crashing.
 
+### Geo installations **(PREMIUM SELF)**
+
+- **Do not** upgrade to GitLab 14.9.0. Instead, use 14.9.1 or later.
+
+  We've discovered an issue with Geo's CI verification feature that may
+  [cause job traces to be lost](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/6664). This issue was fixed
+  in [the GitLab 14.9.1 patch release](https://about.gitlab.com/releases/2022/03/23/gitlab-14-9-1-released/).
+
+  If you have already upgraded to GitLab 14.9.0, you can disable the feature causing the issue by
+  [disabling the `geo_job_artifact_replication` feature flag](../../administration/feature_flags.md#how-to-enable-and-disable-features-behind-flags).
+
 ## 14.8.0
 
 - If upgrading from a version earlier than 14.6.5, 14.7.4, or 14.8.2, review the [Critical Security Release: 14.8.2, 14.7.4, and 14.6.5](https://about.gitlab.com/releases/2022/02/25/critical-security-release-gitlab-14-8-2-released/) blog post.
@@ -162,12 +173,40 @@ that may remain stuck permanently in a **pending** state.
   If your instance has Redis HA with Sentinel, follow the upgrade steps documented in
   [Redis HA (using Sentinel)](../zero_downtime.md#redis-ha-using-sentinel).
 
+### Geo installations **(PREMIUM SELF)**
+
+- There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467)
+  that affects Geo when the GitLab-managed object storage replication is used, causing blob object types to fail synchronization.
+
+  Since GitLab 14.2, verification failures result in synchronization failures and cause a resynchronization of these objects.
+
+  As verification is not yet implemented for files stored in object storage (see
+  [issue 13845](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) for more details), this
+  results in a loop that consistently fails for all objects stored in object storage.
+
+  For information on how to fix this, see
+  [Troubleshooting - Failed syncs with GitLab-managed object storage replication](../../administration/geo/replication/troubleshooting.md#failed-syncs-with-gitlab-managed-object-storage-replication).
+
 ## 14.6.0
 
 - See [LFS objects import and mirror issue in GitLab 14.6.0 to 14.7.2](../index.md#lfs-objects-import-and-mirror-issue-in-gitlab-1460-to-1472).
 - If upgrading from a version earlier than 14.6.5, 14.7.4, or 14.8.2, review the [Critical Security Release: 14.8.2, 14.7.4, and 14.6.5](https://about.gitlab.com/releases/2022/02/25/critical-security-release-gitlab-14-8-2-released/) blog post.
   Updating to 14.6.5 or later resets runner registration tokens for your groups and projects.
 
+### Geo installations **(PREMIUM SELF)**
+
+- There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467)
+  that affects Geo when the GitLab-managed object storage replication is used, causing blob object types to fail synchronization.
+
+  Since GitLab 14.2, verification failures result in synchronization failures and cause a resynchronization of these objects.
+
+  As verification is not yet implemented for files stored in object storage (see
+  [issue 13845](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) for more details), this
+  results in a loop that consistently fails for all objects stored in object storage.
+
+  For information on how to fix this, see
+  [Troubleshooting - Failed syncs with GitLab-managed object storage replication](../../administration/geo/replication/troubleshooting.md#failed-syncs-with-gitlab-managed-object-storage-replication).
+
 ## 14.5.0
 
 - When `make` is run, Gitaly builds are now created in `_build/bin` and no longer in the root directory of the source directory. If you
@@ -233,6 +272,20 @@ or [init scripts](../upgrading_from_source.md#configure-sysv-init-script) by [fo
   sudo -u git -H editor config/cable.yml
   ```
 
+### Geo installations **(PREMIUM SELF)**
+
+- There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467)
+  that affects Geo when the GitLab-managed object storage replication is used, causing blob object types to fail synchronization.
+
+  Since GitLab 14.2, verification failures result in synchronization failures and cause a resynchronization of these objects.
+
+  As verification is not yet implemented for files stored in object storage (see
+  [issue 13845](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) for more details), this
+  results in a loop that consistently fails for all objects stored in object storage.
+
+  For information on how to fix this, see
+  [Troubleshooting - Failed syncs with GitLab-managed object storage replication](../../administration/geo/replication/troubleshooting.md#failed-syncs-with-gitlab-managed-object-storage-replication).
+
 ## 14.4.4
 
 - For [zero-downtime upgrades](../zero_downtime.md) on a GitLab cluster with separate Web and API nodes, you must enable the `paginated_tree_graphql_query` [feature flag](../../administration/feature_flags.md#enable-or-disable-the-feature) _before_ upgrading GitLab Web nodes to 14.4.
@@ -242,6 +295,13 @@ or [init scripts](../upgrading_from_source.md#configure-sysv-init-script) by [fo
   bundle.esm.js:63 Uncaught (in promise) Error: GraphQL error: Field 'paginatedTree' doesn't exist on type 'Repository'
   ```
 
+## 14.4.1 and 14.4.2
+
+### Geo installations **(PREMIUM SELF)**
+
+- There is [an issue in GitLab 14.4.0 through 14.4.2](#1440) that can affect
+  Geo and other features that rely on cronjobs. We recommend upgrading to GitLab 14.4.3 or later.
+
 ## 14.4.0
 
 - Git 2.33.x and later is required. We recommend you use the
@@ -278,6 +338,23 @@ that may remain stuck permanently in a **pending** state when the instance lacks
   the downgrade. If the errors persist after a Grafana restart you may need to reset the Grafana db and re-add the
   customizations. The Grafana database can be reset with `sudo gitlab-ctl reset-grafana`.
 
+### Geo installations **(PREMIUM SELF)**
+
+- There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467)
+  that affects Geo when the GitLab-managed object storage replication is used, causing blob object types to fail synchronization.
+
+  Since GitLab 14.2, verification failures result in synchronization failures and cause a resynchronization of these objects.
+
+  As verification is not yet implemented for files stored in object storage (see
+  [issue 13845](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) for more details), this
+  results in a loop that consistently fails for all objects stored in object storage.
+
+  For information on how to fix this, see
+  [Troubleshooting - Failed syncs with GitLab-managed object storage replication](../../administration/geo/replication/troubleshooting.md#failed-syncs-with-gitlab-managed-object-storage-replication).
+
+- There is [an issue in GitLab 14.4.0 through 14.4.2](#1440) that can affect
+  Geo and other features that rely on cronjobs. We recommend upgrading to GitLab 14.4.3 or later.
+
 ## 14.3.0
 
 - [Instances running 14.0.0 - 14.0.4 should not upgrade directly to GitLab 14.2 or later](#upgrading-to-later-14y-releases).
@@ -415,6 +492,56 @@ for how to proceed.
 
 - If you encounter the error, `I18n::InvalidLocale: :en is not a valid locale`, when starting the application, follow the [patching](https://about.gitlab.com/handbook/support/workflows/patching_an_instance.html) process. Use [122978](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/122978) as the `mr_iid`.
 
+### Geo installations **(PREMIUM SELF)**
+
+- There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467)
+  that affects Geo when the GitLab-managed object storage replication is used, causing blob object types to fail synchronization.
+
+  Since GitLab 14.2, verification failures result in synchronization failures and cause a resynchronization of these objects.
+
+  As verification is not yet implemented for files stored in object storage (see
+  [issue 13845](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) for more details), this
+  results in a loop that consistently fails for all objects stored in object storage.
+
+  For information on how to fix this, see
+  [Troubleshooting - Failed syncs with GitLab-managed object storage replication](../../administration/geo/replication/troubleshooting.md#failed-syncs-with-gitlab-managed-object-storage-replication).
+
+- We found an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336013) where the Container Registry replication
+  wasn't fully working if you used multi-arch images. In case of a multi-arch image, only the primary architecture
+  (for example `amd64`) would be replicated to the secondary site. This has been [fixed in GitLab 14.3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67624) and was backported to 14.2 and 14.1, but manual steps are required to force a re-sync.
+
+  You can check if you are affected by running:
+
+  ```shell
+  docker manifest inspect <SECONDARY_IMAGE_LOCATION> | jq '.mediaType'
+  ```
+
+  Where `<SECONDARY_IMAGE_LOCATION>` is a container image on your secondary site.
+  If the output matches `application/vnd.docker.distribution.manifest.list.v2+json`
+  (there can be a `mediaType` entry at several levels, we only care about the top level entry),
+  then you don't need to do anything.
+
+  Otherwise, for each **secondary** site, on a Rails application node, open a [Rails console](../../administration/operations/rails_console.md), and run the following:
+
+   ```ruby
+   list_type = 'application/vnd.docker.distribution.manifest.list.v2+json'
+
+   Geo::ContainerRepositoryRegistry.synced.each do |gcr|
+     cr = gcr.container_repository
+     primary = Geo::ContainerRepositorySync.new(cr)
+     cr.tags.each do |tag|
+       primary_manifest = JSON.parse(primary.send(:client).repository_raw_manifest(cr.path, tag.name))
+       next unless primary_manifest['mediaType'].eql?(list_type)
+
+       cr.delete_tag_by_name(tag.name)
+     end
+     primary.execute
+   end
+   ```
+
+  If you are running a version prior to 14.1 and are using Geo and multi-arch containers in your Container Registry,
+  we recommend [upgrading](../../administration/geo/replication/upgrading_the_geo_sites.md) to at least GitLab 14.1.
+
 ## 14.2.0
 
 - [Instances running 14.0.0 - 14.0.4 should not upgrade directly to GitLab 14.2 or later](#upgrading-to-later-14y-releases).
@@ -456,6 +583,56 @@ for how to proceed.
 
 - If you encounter the error, `I18n::InvalidLocale: :en is not a valid locale`, when starting the application, follow the [patching](https://about.gitlab.com/handbook/support/workflows/patching_an_instance.html) process. Use [123476](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123476) as the `mr_iid`.
 
+### Geo installations **(PREMIUM SELF)**
+
+- There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467)
+  that affects Geo when the GitLab-managed object storage replication is used, causing blob object types to fail synchronization.
+
+  Since GitLab 14.2, verification failures result in synchronization failures and cause a resynchronization of these objects.
+
+  As verification is not yet implemented for files stored in object storage (see
+  [issue 13845](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) for more details), this
+  results in a loop that consistently fails for all objects stored in object storage.
+
+  For information on how to fix this, see
+  [Troubleshooting - Failed syncs with GitLab-managed object storage replication](../../administration/geo/replication/troubleshooting.md#failed-syncs-with-gitlab-managed-object-storage-replication).
+
+- We found an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336013) where the Container Registry replication
+  wasn't fully working if you used multi-arch images. In case of a multi-arch image, only the primary architecture
+  (for example `amd64`) would be replicated to the secondary site. This has been [fixed in GitLab 14.3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67624) and was backported to 14.2 and 14.1, but manual steps are required to force a re-sync.
+
+  You can check if you are affected by running:
+
+  ```shell
+  docker manifest inspect <SECONDARY_IMAGE_LOCATION> | jq '.mediaType'
+  ```
+
+  Where `<SECONDARY_IMAGE_LOCATION>` is a container image on your secondary site.
+  If the output matches `application/vnd.docker.distribution.manifest.list.v2+json`
+  (there can be a `mediaType` entry at several levels, we only care about the top level entry),
+  then you don't need to do anything.
+
+  Otherwise, for each **secondary** site, on a Rails application node, open a [Rails console](../../administration/operations/rails_console.md), and run the following:
+
+   ```ruby
+   list_type = 'application/vnd.docker.distribution.manifest.list.v2+json'
+
+   Geo::ContainerRepositoryRegistry.synced.each do |gcr|
+     cr = gcr.container_repository
+     primary = Geo::ContainerRepositorySync.new(cr)
+     cr.tags.each do |tag|
+       primary_manifest = JSON.parse(primary.send(:client).repository_raw_manifest(cr.path, tag.name))
+       next unless primary_manifest['mediaType'].eql?(list_type)
+
+       cr.delete_tag_by_name(tag.name)
+     end
+     primary.execute
+   end
+   ```
+
+  If you are running a version prior to 14.1 and are using Geo and multi-arch containers in your Container Registry,
+  we recommend [upgrading](../../administration/geo/replication/upgrading_the_geo_sites.md) to at least GitLab 14.1.
+
 ## 14.1.0
 
 - [Instances running 14.0.0 - 14.0.4 should not upgrade directly to GitLab 14.2 or later](#upgrading-to-later-14y-releases)
@@ -475,6 +652,50 @@ for how to proceed.
 
 - If you encounter the error, `I18n::InvalidLocale: :en is not a valid locale`, when starting the application, follow the [patching](https://about.gitlab.com/handbook/support/workflows/patching_an_instance.html) process. Use [123475](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123475) as the `mr_iid`.
 
+### Geo installations **(PREMIUM SELF)**
+
+- We found an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336013) where the Container Registry replication
+  wasn't fully working if you used multi-arch images. In case of a multi-arch image, only the primary architecture
+  (for example `amd64`) would be replicated to the secondary site. This has been [fixed in GitLab 14.3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67624) and was backported to 14.2 and 14.1, but manual steps are required to force a re-sync.
+
+  You can check if you are affected by running:
+
+  ```shell
+  docker manifest inspect <SECONDARY_IMAGE_LOCATION> | jq '.mediaType'
+  ```
+
+  Where `<SECONDARY_IMAGE_LOCATION>` is a container image on your secondary site.
+  If the output matches `application/vnd.docker.distribution.manifest.list.v2+json`
+  (there can be a `mediaType` entry at several levels, we only care about the top level entry),
+  then you don't need to do anything.
+
+  Otherwise, for each **secondary** site, on a Rails application node, open a [Rails console](../../administration/operations/rails_console.md), and run the following:
+
+   ```ruby
+   list_type = 'application/vnd.docker.distribution.manifest.list.v2+json'
+
+   Geo::ContainerRepositoryRegistry.synced.each do |gcr|
+     cr = gcr.container_repository
+     primary = Geo::ContainerRepositorySync.new(cr)
+     cr.tags.each do |tag|
+       primary_manifest = JSON.parse(primary.send(:client).repository_raw_manifest(cr.path, tag.name))
+       next unless primary_manifest['mediaType'].eql?(list_type)
+
+       cr.delete_tag_by_name(tag.name)
+     end
+     primary.execute
+   end
+   ```
+
+  If you are running a version prior to 14.1 and are using Geo and multi-arch containers in your Container Registry,
+  we recommend [upgrading](../../administration/geo/replication/upgrading_the_geo_sites.md) to at least GitLab 14.1.
+- We found an issue where [Primary sites cannot be removed from the UI](https://gitlab.com/gitlab-org/gitlab/-/issues/338231).
+
+  This bug only exists in the UI and does not block the removal of Primary sites using any other method.
+
+  If you are running an affected version and need to remove your Primary site, you can manually remove the Primary site
+  by using the [Geo Sites API](../../api/geo_nodes.md#delete-a-geo-node).
+
 ## 14.0.0
 
 Prerequisites:
@@ -579,6 +800,15 @@ Other issues:
   The following invalid roles have been set in 'roles': redis_slave_role
   ```
 
+### Geo installations **(PREMIUM SELF)**
+
+- We found an issue where [Primary sites cannot be removed from the UI](https://gitlab.com/gitlab-org/gitlab/-/issues/338231).
+
+  This bug only exists in the UI and does not block the removal of Primary sites using any other method.
+
+  If you are running an affected version and need to remove your Primary site, you can manually remove the Primary site
+  by using the [Geo Sites API](../../api/geo_nodes.md#delete-a-geo-node).
+
 ### Upgrading to later 14.Y releases
 
 - Instances running 14.0.0 - 14.0.4 should not upgrade directly to GitLab 14.2 or later,
diff --git a/doc/update/versions/gitlab_15_changes.md b/doc/update/versions/gitlab_15_changes.md
index 0bc1a4dba51..672c7b1bd81 100644
--- a/doc/update/versions/gitlab_15_changes.md
+++ b/doc/update/versions/gitlab_15_changes.md
@@ -21,7 +21,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - **Upgrade to patch release 15.11.3 or later**. This avoids [issue 408304](https://gitlab.com/gitlab-org/gitlab/-/issues/408304) when upgrading from 15.5.0 and earlier.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - Some project imports do not initialize wiki repositories on project creation. See
   [the details and workaround](#geo-wiki-repositories-not-initialized-on-project-creation).
@@ -99,7 +99,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
   For more information, see [issue 415724](https://gitlab.com/gitlab-org/gitlab/-/issues/415724).
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - `pg_upgrade` fails to upgrade the bundled PostregSQL database to version 13. See
   [the details and workaround](#geo-pg_upgrade-fails-to-upgrade-the-bundled-postregsql-database-to-version-13).
@@ -138,14 +138,14 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
   sudo apt install libkrb5-dev
   ```
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - `pg_upgrade` fails to upgrade the bundled PostregSQL database to version 13. See
   [the details and workaround](#geo-pg_upgrade-fails-to-upgrade-the-bundled-postregsql-database-to-version-13).
 
 ## 15.8.2
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
   - Affected versions: GitLab versions 15.6.x, 15.7.x, and 15.8.0 - 15.8.2.
@@ -155,7 +155,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
   - Affected versions: GitLab versions 15.6.x, 15.7.x, and 15.8.0 - 15.8.2.
@@ -166,7 +166,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 - Git 2.38.0 and later is required by Gitaly. For self-compiled installations, you should use the [Git version provided by Gitaly](../../install/installation.md#git).
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - `pg_upgrade` fails to upgrade the bundled PostregSQL database to version 13. See
   [the details and workaround](#geo-pg_upgrade-fails-to-upgrade-the-bundled-postregsql-database-to-version-13).
@@ -178,7 +178,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
   - Affected versions: GitLab versions 15.6.x, 15.7.x, and 15.8.0 - 15.8.2.
@@ -188,7 +188,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
   - Affected versions: GitLab versions 15.6.x, 15.7.x, and 15.8.0 - 15.8.2.
@@ -198,7 +198,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
   - Affected versions: GitLab versions 15.6.x, 15.7.x, and 15.8.0 - 15.8.2.
@@ -208,7 +208,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
   - Affected versions: GitLab versions 15.6.x, 15.7.x, and 15.8.0 - 15.8.2.
@@ -218,7 +218,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - [Container registry push events are rejected](https://gitlab.com/gitlab-org/gitlab/-/issues/386389) by the `/api/v4/container_registry_event/events` endpoint resulting in Geo secondary sites not being aware of updates to container registry images and subsequently not replicating the upgrades. Secondary sites may contain out of date container images after a failover as a consequence. This affects versions 15.6.0 - 15.6.6 and 15.7.0 - 15.7.2. If you're using Geo with container repositories, you are advised to upgrade to GitLab 15.6.7, 15.7.3, or 15.8.0 which contain a fix for this issue and avoid potential data loss after a failover.
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
@@ -229,7 +229,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - [Container registry push events are rejected](https://gitlab.com/gitlab-org/gitlab/-/issues/386389) by the `/api/v4/container_registry_event/events` endpoint resulting in Geo secondary sites not being aware of updates to container registry images and subsequently not replicating the updates. Secondary sites may contain out of date container images after a failover as a consequence. This affects versions 15.6.0 - 15.6.6 and 15.7.0 - 15.7.2. If you're using Geo with container repositories, you are advised to upgrade to GitLab 15.6.7, 15.7.3, or 15.8.0 which contain a fix for this issue and avoid potential data loss after a failover.
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
@@ -286,7 +286,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
   see [Understanding the file type variable expansion change in GitLab 15.7](https://about.gitlab.com/blog/2023/02/13/impact-of-the-file-type-variable-change-15-7/).
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - `pg_upgrade` fails to upgrade the bundled PostregSQL database to version 13. See
   [the details and workaround](#geo-pg_upgrade-fails-to-upgrade-the-bundled-postregsql-database-to-version-13).
@@ -299,7 +299,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
   - Affected versions: GitLab versions 15.6.x, 15.7.x, and 15.8.0 - 15.8.2.
@@ -309,7 +309,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - [Container registry push events are rejected](https://gitlab.com/gitlab-org/gitlab/-/issues/386389) by the `/api/v4/container_registry_event/events` endpoint resulting in Geo secondary sites not being aware of updates to container registry images and subsequently not replicating the updates. Secondary sites may contain out of date container images after a failover as a consequence. This affects versions 15.6.0 - 15.6.6 and 15.7.0 - 15.7.2. If you're using Geo with container repositories, you are advised to upgrade to GitLab 15.6.7, 15.7.3, or 15.8.0 which contain a fix for this issue and avoid potential data loss after a failover.
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
@@ -318,7 +318,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 ## 15.6.5
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - [Container registry push events are rejected](https://gitlab.com/gitlab-org/gitlab/-/issues/386389) by the `/api/v4/container_registry_event/events` endpoint resulting in Geo secondary sites not being aware of updates to container registry images and subsequently not replicating the updates. Secondary sites may contain out of date container images after a failover as a consequence. This affects versions 15.6.0 - 15.6.6 and 15.7.0 - 15.7.2. If you're using Geo with container repositories, you are advised to upgrade to GitLab 15.6.7, 15.7.3, or 15.8.0 which contain a fix for this issue and avoid potential data loss after a failover.
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
@@ -328,7 +328,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 ## 15.6.4
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - [Container registry push events are rejected](https://gitlab.com/gitlab-org/gitlab/-/issues/386389) by the `/api/v4/container_registry_event/events` endpoint resulting in Geo secondary sites not being aware of updates to container registry images and subsequently not replicating the updates. Secondary sites may contain out of date container images after a failover as a consequence. This affects versions 15.6.0 - 15.6.6, and 15.7.0 - 15.7.2. If you're using Geo with container repositories, you are advised to upgrade to GitLab 15.6.7, 15.7.3, or 15.8.0 which contain a fix for this issue and avoid potential data loss after a failover.
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
@@ -338,7 +338,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 ## 15.6.3
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - [Container registry push events are rejected](https://gitlab.com/gitlab-org/gitlab/-/issues/386389) by the `/api/v4/container_registry_event/events` endpoint resulting in Geo secondary sites not being aware of updates to container registry images and subsequently not replicating the updates. Secondary sites may contain out of date container images after a failover as a consequence. This affects versions 15.6.0 - 15.6.6 and 15.7.0 - 15.7.2. If you're using Geo with container repositories, you are advised to upgrade to GitLab 15.6.7, 15.7.3, or 15.8.0 which contain a fix for this issue and avoid potential data loss after a failover.
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
@@ -348,7 +348,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 ## 15.6.2
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - [Container registry push events are rejected](https://gitlab.com/gitlab-org/gitlab/-/issues/386389) by the `/api/v4/container_registry_event/events` endpoint resulting in Geo secondary sites not being aware of updates to container registry images and subsequently not replicating the updates. Secondary sites may contain out of date container images after a failover as a consequence. This affects versions 15.6.0 - 15.6.6 and 15.7.0 - 15.7.2. If you're using Geo with container repositories, you are advised to upgrade to GitLab 15.6.7, 15.7.3, or 15.8.0 which contain a fix for this issue and avoid potential data loss after a failover.
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
@@ -358,7 +358,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 ## 15.6.1
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - [Container registry push events are rejected](https://gitlab.com/gitlab-org/gitlab/-/issues/386389) by the `/api/v4/container_registry_event/events` endpoint resulting in Geo secondary sites not being aware of updates to container registry images and subsequently not replicating the updates. Secondary sites may contain out of date container images after a failover as a consequence. This affects versions 15.6.0 - 15.6.6 and 15.7.0 - 15.7.2. If you're using Geo with container repositories, you are advised to upgrade to GitLab 15.6.7, 15.7.3, or 15.8.0 which contain a fix for this issue and avoid potential data loss after a failover.
 - We discovered an issue where [replication and verification of projects and wikis was not keeping up](https://gitlab.com/gitlab-org/gitlab/-/issues/387980) on small number of Geo installations. Your installation may be affected if you see some projects and/or wikis persistently in the "Queued" state for verification. This can lead to data loss after a failover.
@@ -385,7 +385,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
   to GitLab 15.6.2. The issue can also be worked around:
   [read about how to create these indexes](https://gitlab.com/gitlab-org/gitlab/-/issues/378343#note_1199863087).
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - `pg_upgrade` fails to upgrade the bundled PostregSQL database to version 13. See
   [the details and workaround](#geo-pg_upgrade-fails-to-upgrade-the-bundled-postregsql-database-to-version-13).
@@ -451,7 +451,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
 
 - Due to [a bug introduced in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/390155), if one or more Git repositories in Gitaly Cluster is [unavailable](../../administration/gitaly/recovery.md#unavailable-repositories), then [Repository checks](../../administration/repository_checks.md#repository-checks) and [Geo replication and verification](../../administration/geo/index.md) stop running for all project or project wiki repositories in the affected Gitaly Cluster. The bug was fixed by [reverting the change in GitLab 15.9.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110823). Before upgrading to this version, check if you have any "unavailable" repositories. See [the bug issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390155) for more information.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - `pg_upgrade` fails to upgrade the bundled PostregSQL database to version 13. See
   [the details and workaround](#geo-pg_upgrade-fails-to-upgrade-the-bundled-postregsql-database-to-version-13).
@@ -525,7 +525,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
   Feature.disable(:restyle_login_page)
   ```
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - `pg_upgrade` fails to upgrade the bundled PostregSQL database to version 13. See
   [the details and workaround](#geo-pg_upgrade-fails-to-upgrade-the-bundled-postregsql-database-to-version-13).
@@ -584,7 +584,7 @@ A [license caching issue](https://gitlab.com/gitlab-org/gitlab/-/issues/376706)
     - 15.3.0 - 15.3.4 --> 15.3.5
     - 15.4.1 --> 15.4.3
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - `pg_upgrade` fails to upgrade the bundled PostregSQL database to version 13. See
   [the details and workaround](#geo-pg_upgrade-fails-to-upgrade-the-bundled-postregsql-database-to-version-13).
@@ -622,7 +622,7 @@ A [license caching issue](https://gitlab.com/gitlab-org/gitlab/-/issues/376706)
   1. Add `gitaly['runtime_dir'] = '<PATH_WITH_EXEC_PERM>'` to `/etc/gitlab/gitlab.rb` and specify a location without `noexec` set.
   1. Run `sudo gitlab-ctl reconfigure`.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - `pg_upgrade` fails to upgrade the bundled PostregSQL database to version 13. See
   [the details and workaround](#geo-pg_upgrade-fails-to-upgrade-the-bundled-postregsql-database-to-version-13).
@@ -664,7 +664,7 @@ A [license caching issue](https://gitlab.com/gitlab-org/gitlab/-/issues/376706)
   Before you upgrade to GitLab 15.1, add an [access token](../../api/rest/index.md#authentication) to your requests.
   The user creating the token must have [permission](../../user/permissions.md) to create pipelines in the project.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - [Geo proxying](../../administration/geo/secondary_proxy/index.md) was [enabled by default for different URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/346112) in 15.1. This may be a breaking change. If needed, you may [disable Geo proxying](../../administration/geo/secondary_proxy/index.md#disable-geo-proxying). If you are using SAML with different URLs, you must modify your SAML configuration and your Identity Provider configuration. For more information, see the [Geo with Single Sign-On (SSO) documentation](../../administration/geo/replication/single_sign_on.md).
 - LFS transfers can redirect to the primary from secondary site mid-session. See
@@ -727,7 +727,7 @@ A [license caching issue](https://gitlab.com/gitlab-org/gitlab/-/issues/376706)
       ...
   ```
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 - Incorrect object storage LFS files deletion on Geo secondary sites. See
   [the details and workaround](#geo-incorrect-object-storage-lfs-file-deletion-on-secondary-sites).
diff --git a/doc/update/versions/gitlab_16_changes.md b/doc/update/versions/gitlab_16_changes.md
index de6cb012aa6..10cc673da8c 100644
--- a/doc/update/versions/gitlab_16_changes.md
+++ b/doc/update/versions/gitlab_16_changes.md
@@ -141,7 +141,7 @@ Specific information applies to Linux package installations:
   [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118645). For more information, see the
   [`puma.rb.example`](https://gitlab.com/gitlab-org/gitlab/-/blob/16-0-stable-ee/config/puma.rb.example) file.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 Specific information applies to installations using Geo:
 
@@ -197,7 +197,7 @@ Specific information applies to Linux package installations:
 
   Workaround is to make use of a different key type, or upgrade the client OpenSSH to a version >= 8.7.
 
-### Geo installations
+### Geo installations **(PREMIUM SELF)**
 
 Specific information applies to installations using Geo: