Add latest changes from gitlab-org/gitlab@14-8-stable-eev14.8.0-rc42

author: GitLab Bot <gitlab-bot@gitlab.com> 2022-02-18 12:45:46 +0300
committer: GitLab Bot <gitlab-bot@gitlab.com> 2022-02-18 12:45:46 +0300
commit: a7b3560714b4d9cc4ab32dffcd1f74a284b93580 (patch)
tree: 7452bd5c3545c2fa67a28aa013835fb4fa071baf /doc/development
parent: ee9173579ae56a3dbfe5afe9f9410c65bb327ca7 (diff)
113 files changed, 3160 insertions, 1867 deletions
diff --git a/doc/development/agent/gitops.md b/doc/development/agent/gitops.md
index 3d59f5bd845..7c741408ae6 100644
--- a/doc/development/agent/gitops.md
+++ b/doc/development/agent/gitops.md
@@ -1,9 +1,9 @@
 ---
 redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/gitops.md'
-remove_date: '2022-06-24'
+remove_date: '2022-02-01'
 ---
 
 This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/gitops.md).
 
-<!-- This redirect file can be deleted after <2022-06-24>. -->
+<!-- This redirect file can be deleted after <2022-02-01>. -->
 <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
diff --git a/doc/development/agent/identity.md b/doc/development/agent/identity.md
index 67084a6d995..6caf108a32a 100644
--- a/doc/development/agent/identity.md
+++ b/doc/development/agent/identity.md
@@ -1,9 +1,9 @@
 ---
 redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md'
-remove_date: '2022-06-24'
+remove_date: '2022-02-01'
 ---
 
 This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md).
 
-<!-- This redirect file can be deleted after <2022-06-24>. -->
+<!-- This redirect file can be deleted after <2022-02-01>. -->
 <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
diff --git a/doc/development/agent/index.md b/doc/development/agent/index.md
index 2cb05e2dd8f..474f8a02933 100644
--- a/doc/development/agent/index.md
+++ b/doc/development/agent/index.md
@@ -1,9 +1,9 @@
 ---
 redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md'
-remove_date: '2022-06-24'
+remove_date: '2022-02-01'
 ---
 
 This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md).
 
-<!-- This redirect file can be deleted after <2022-06-24>. -->
+<!-- This redirect file can be deleted after <2022-02-01>. -->
 <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
diff --git a/doc/development/agent/local.md b/doc/development/agent/local.md
index 1fff5607de4..a4b29bea838 100644
--- a/doc/development/agent/local.md
+++ b/doc/development/agent/local.md
@@ -1,9 +1,9 @@
 ---
 redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/local.md'
-remove_date: '2022-06-24'
+remove_date: '2022-02-01'
 ---
 
 This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/local.md).
 
-<!-- This redirect file can be deleted after <2022-06-24>. -->
+<!-- This redirect file can be deleted after <2022-02-01>. -->
 <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
diff --git a/doc/development/agent/repository_overview.md b/doc/development/agent/repository_overview.md
index 43ea99889c5..8ea9dceb32a 100644
--- a/doc/development/agent/repository_overview.md
+++ b/doc/development/agent/repository_overview.md
@@ -1,9 +1,9 @@
 ---
 redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/repository_overview.md'
-remove_date: '2022-06-24'
+remove_date: '2022-02-01'
 ---
 
 This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/repository_overview.md).
 
-<!-- This redirect file can be deleted after <2022-06-24>. -->
+<!-- This redirect file can be deleted after <2022-02-01>. -->
 <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
diff --git a/doc/development/agent/routing.md b/doc/development/agent/routing.md
index 7792d6d56a4..364267a45fe 100644
--- a/doc/development/agent/routing.md
+++ b/doc/development/agent/routing.md
@@ -1,9 +1,9 @@
 ---
 redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/kas_request_routing.md'
-remove_date: '2022-06-24'
+remove_date: '2022-02-01'
 ---
 
 This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/kas_request_routing.md).
 
-<!-- This redirect file can be deleted after <2022-06-24>. -->
+<!-- This redirect file can be deleted after <2022-02-01>. -->
 <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
diff --git a/doc/development/agent/user_stories.md b/doc/development/agent/user_stories.md
index ce38064b31b..2ed4bbdc9f6 100644
--- a/doc/development/agent/user_stories.md
+++ b/doc/development/agent/user_stories.md
@@ -1,9 +1,9 @@
 ---
 redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/user_stories.md'
-remove_date: '2022-06-24'
+remove_date: '2022-02-01'
 ---
 
 This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/user_stories.md).
 
-<!-- This redirect file can be deleted after <2022-06-24>. -->
+<!-- This redirect file can be deleted after <2022-02-01>. -->
 <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
diff --git a/doc/development/application_slis/index.md b/doc/development/application_slis/index.md
index 87c0bcfede5..adb656761c5 100644
--- a/doc/development/application_slis/index.md
+++ b/doc/development/application_slis/index.md
@@ -57,31 +57,10 @@ Gitlab::Metrics::Sli.initialize_sli(:received_email, [
 ])
 ```
 
-Metrics must be initialized before they get
-scraped for the first time. This could be done at the start time of the
-process that will emit them, in which case we need to pay attention
-not to increase application's boot time too much. This is preferable
-if possible.
-
-Alternatively, if initializing would take too long, this can be done
-during the first scrape. We need to make sure we don't do it for every
-scrape. This can be done as follows:
-
-```ruby
-def initialize_request_slis_if_needed!
-  return if Gitlab::Metrics::Sli.initialized?(:rails_request_apdex)
-  Gitlab::Metrics::Sli.initialize_sli(:rails_request_apdex, possible_request_labels)
-end
-```
-
-Also pay attention to do it for the different metrics
-endpoints we have. Currently the
-[`WebExporter`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/metrics/exporter/web_exporter.rb)
-and the
-[`HealthController`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/controllers/health_controller.rb)
-for Rails and
-[`SidekiqExporter`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/metrics/exporter/sidekiq_exporter.rb)
-for Sidekiq.
+Metrics must be initialized before they get scraped for the first time.
+This currently happens during the `on_master_start` [life-cycle event](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/cluster/lifecycle_events.rb).
+Since this delays application readiness until metrics initialization returns, make sure the overhead
+this adds is understood and acceptable.
 
 ## Tracking operations for an SLI
 
diff --git a/doc/development/architecture.md b/doc/development/architecture.md
index 36d3655ba93..e8d04d68565 100644
--- a/doc/development/architecture.md
+++ b/doc/development/architecture.md
@@ -339,7 +339,7 @@ Component statuses are linked to configuration documentation for each component.
 
 ### Component list
 
-| Component                                             | Description                                                          | [Omnibus GitLab](https://docs.gitlab.com/omnibus/) | [GitLab Environment Toolkit (GET)](https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit) | [GitLab chart](https://docs.gitlab.com/charts/) | [minikube Minimal](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings) | [GitLab.com](https://gitlab.com) | [Source](../install/installation.md) | [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) |  [CE/EE](https://about.gitlab.com/install/ce-or-ee/)  |
+| Component                                             | Description                                                          | [Omnibus GitLab](https://docs.gitlab.com/omnibus/) | [GitLab Environment Toolkit (GET)](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) | [GitLab chart](https://docs.gitlab.com/charts/) | [minikube Minimal](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings) | [GitLab.com](https://gitlab.com) | [Source](../install/installation.md) | [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) |  [CE/EE](https://about.gitlab.com/install/ce-or-ee/)  |
 |-------------------------------------------------------|----------------------------------------------------------------------|:--------------:|:--------------:|:------------:|:----------------:|:----------:|:------:|:---:|:-------:|
 | [Certificate Management](#certificate-management)     | TLS Settings, Let's Encrypt                                          |       ✅       |       ✅        |      ✅       |        ⚙         |     ✅      |   ⚙    |  ⚙  | CE & EE |
 | [Consul](#consul)                                     | Database node discovery, failover                                    |       ⚙       |       ✅         |      ❌       |        ❌         |     ✅      |   ❌    |  ❌  | EE Only |
@@ -528,7 +528,7 @@ You can use it either for personal or business websites, such as portfolios, doc
 
 #### GitLab Runner
 
-- [Project page](https://gitlab.com/gitlab-org/gitlab-runner/blob/master/README.md)
+- [Project page](https://gitlab.com/gitlab-org/gitlab-runner/blob/main/README.md)
 - Configuration:
   - [Omnibus](https://docs.gitlab.com/runner/)
   - [Charts](https://docs.gitlab.com/runner/install/kubernetes.html)
diff --git a/doc/development/avoiding_downtime_in_migrations.md b/doc/development/avoiding_downtime_in_migrations.md
index 1d36bbf1212..1de96df327c 100644
--- a/doc/development/avoiding_downtime_in_migrations.md
+++ b/doc/development/avoiding_downtime_in_migrations.md
@@ -399,7 +399,7 @@ migrations:
 1. Change the index pattern to `pubsub-sidekiq-inf-gprd*`.
 1. Add filter for `json.queue: cronjob:database_batched_background_migration`.
 
-#### PostgerSQL slow queries log
+#### PostgreSQL slow queries log
 
 Slow queries log keeps track of low queries that took above 1 second to execute. To see them
 for batched background migration:
@@ -408,7 +408,7 @@ for batched background migration:
 1. Change the index pattern to `pubsub-postgres-inf-gprd*`.
 1. Add filter for `json.endpoint_id.keyword: Database::BatchedBackgroundMigrationWorker`.
 1. Optional. To see only updates, add a filter for `json.command_tag.keyword: UPDATE`.
-1. Optional. To see only failed statements, add a filter for `json.error_severiry.keyword: ERROR`.
+1. Optional. To see only failed statements, add a filter for `json.error_severity.keyword: ERROR`.
 1. Optional. Add a filter by table name.
 
 #### Grafana dashboards
diff --git a/doc/development/cached_queries.md b/doc/development/cached_queries.md
index 4f82d721164..492c8d13600 100644
--- a/doc/development/cached_queries.md
+++ b/doc/development/cached_queries.md
@@ -163,5 +163,5 @@ factors help improve the overall execution time:
 ## For more information
 
 - [Metrics that would help us detect the potential N+1 Cached SQL calls](https://gitlab.com/gitlab-org/gitlab/-/issues/259007)
-- [Merge Request performance guidelines for cached queries](merge_request_performance_guidelines.md#cached-queries)
+- [Merge request performance guidelines for cached queries](merge_request_performance_guidelines.md#cached-queries)
 - [Improvements for biggest offenders](https://gitlab.com/groups/gitlab-org/-/epics/4508)
diff --git a/doc/development/cascading_settings.md b/doc/development/cascading_settings.md
index f9c96e414d0..76ab2c6e693 100644
--- a/doc/development/cascading_settings.md
+++ b/doc/development/cascading_settings.md
@@ -1,6 +1,6 @@
 ---
 stage: Manage
-group: Authentication & Authorization
+group: Authentication and Authorization
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
diff --git a/doc/development/changelog.md b/doc/development/changelog.md
index 7c4a600e1fa..b51db69c2f7 100644
--- a/doc/development/changelog.md
+++ b/doc/development/changelog.md
@@ -101,7 +101,9 @@ EE: true
 - _Any_ contribution from a community member, no matter how small, **may** have
   a changelog entry regardless of these guidelines if the contributor wants one.
 - Any [GLEX experiment](experiment_guide/gitlab_experiment.md) changes **should not** have a changelog entry.
-- [Modifying](feature_flags/#changelog) a feature flag (flag removal, default-on setting).
+
+For more information, see
+[how to handle changelog entries with feature flags](feature_flags/index.md#changelog).
 
 ## Writing good changelog entries
 
diff --git a/doc/development/cicd/index.md b/doc/development/cicd/index.md
index 17a70393c96..2779b457fb9 100644
--- a/doc/development/cicd/index.md
+++ b/doc/development/cicd/index.md
@@ -27,7 +27,7 @@ On the left side we have the events that can trigger a pipeline based on various
 - A `git push` is the most common event that triggers a pipeline.
 - The [Web API](../../api/pipelines.md#create-a-new-pipeline).
 - A user clicking the "Run pipeline" button in the UI.
-- When a [merge request is created or updated](../../ci/pipelines/merge_request_pipelines.md#pipelines-for-merge-requests).
+- When a [merge request is created or updated](../../ci/pipelines/merge_request_pipelines.md).
 - When an MR is added to a [Merge Train](../../ci/pipelines/merge_trains.md#merge-trains).
 - A [scheduled pipeline](../../ci/pipelines/schedules.md#pipeline-schedules).
 - When project is [subscribed to an upstream project](../../ci/pipelines/multi_project_pipelines.md#trigger-a-pipeline-when-an-upstream-project-is-rebuilt).
diff --git a/doc/development/cicd/templates.md b/doc/development/cicd/templates.md
index b1252b86cc0..d7edad842b8 100644
--- a/doc/development/cicd/templates.md
+++ b/doc/development/cicd/templates.md
@@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 type: index, concepts, howto
 ---
 
-# Development guide for GitLab CI/CD templates
+# Development guide for GitLab CI/CD templates **(FREE)**
 
 This document explains how to develop [GitLab CI/CD templates](../../ci/examples/index.md).
 
@@ -18,6 +18,7 @@ Before submitting a merge request with a new or updated CI/CD template, you must
 - Name the template following the `*.gitlab-ci.yml` format.
 - Use valid [`.gitlab-ci.yml` syntax](../../ci/yaml/index.md). Verify it's valid
   with the [CI/CD lint tool](../../ci/lint.md).
+- [Add template metrics](#add-metrics).
 - Include [a changelog](../changelog.md) if the merge request introduces a user-facing change.
 - Follow the [template review process](#contribute-cicd-template-merge-requests).
 - (Optional but highly recommended) Test the template in an example GitLab project
@@ -382,6 +383,77 @@ you must:
 This information is important for users when [a stable template](#stable-version)
 is updated in a major version GitLab release.
 
+### Add metrics
+
+Every CI/CD template must also have metrics defined to track their use.
+
+To add a metric definition for a new template:
+
+1. Install and start the [GitLab GDK](https://gitlab.com/gitlab-org/gitlab-development-kit#installation).
+1. In the `gitlab` directory in your GDK, check out the branch that contains the new template.
+1. [Add the template inclusion event](../service_ping/implement.md#add-new-events)
+   with this Rake task:
+
+   ```shell
+   bin/rake gitlab:usage_data:generate_ci_template_events
+   ```
+
+   The task adds a section like the following to [`lib/gitlab/usage_data_counters/known_events/ci_templates.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/ci_templates.yml):
+
+   ```yaml
+   - name: p_ci_templates_my_template_name
+     category: ci_templates
+     redis_slot: ci_templates
+     aggregation: weekly
+   ```
+
+1. Copy the value of `name` from the new YAML section, and add it to the weekly
+   and monthly CI/CD template total count metrics:
+   - [`config/metrics/counts_7d/20210216184557_ci_templates_total_unique_counts_weekly.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216184557_ci_templates_total_unique_counts_weekly.yml)
+   - [`config/metrics/counts_28d/20210216184559_ci_templates_total_unique_counts_monthly.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216184559_ci_templates_total_unique_counts_monthly.yml)
+
+1. Use the same `name` as above as the last argument in the following command to
+  [add new metric definitions](../service_ping/metrics_dictionary.md#metrics-added-dynamic-to-service-ping-payload):
+
+   ```shell
+   bundle exec rails generate gitlab:usage_metric_definition:redis_hll ci_templates <template_metric_event_name>
+   ```
+
+   The output should look like:
+
+   ```shell
+   $ bundle exec rails generate gitlab:usage_metric_definition:redis_hll ci_templates p_ci_templates_my_template_name
+         create  config/metrics/counts_7d/20220120073740_p_ci_templates_my_template_name_weekly.yml
+         create  config/metrics/counts_28d/20220120073746_p_ci_templates_my_template_name_monthly.yml
+   ```
+
+1. Edit both newly generated files as follows:
+
+   - `name:` and `performance_indicator_type:`: Delete (not needed).
+   - `introduced_by_url:`: The URL of the MR adding the template.
+   - `data_source:`: Set to `redis_hll`.
+   - All other fields that have no values: Set to empty strings (`''`).
+   - Add the following to the end of each file:
+
+     ```yaml
+     options:
+       events:
+         - p_ci_templates_my_template_name
+     ```
+
+1. Commit and push the changes.
+
+For example, these are the metrics configuration files for the
+[5 Minute Production App template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/5-Minute-Production-App.gitlab-ci.yml):
+
+- The template inclusion event: [`lib/gitlab/usage_data_counters/known_events/ci_templates.yml#L438-L441`](https://gitlab.com/gitlab-org/gitlab/-/blob/dcddbf83c29d1ad0839d55362c1b43888304f453/lib/gitlab/usage_data_counters/known_events/ci_templates.yml#L438-L441)
+- The weekly and monthly metrics definitions:
+  - [`config/metrics/counts_7d/20210901223501_p_ci_templates_5_minute_production_app_weekly.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/1a6eceff3914f240864b2ca15ae2dc076ea67bf6/config/metrics/counts_7d/20210216184515_p_ci_templates_5_min_production_app_weekly.yml)
+  - [`config/metrics/counts_28d/20210901223505_p_ci_templates_5_minute_production_app_monthly.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216184517_p_ci_templates_5_min_production_app_monthly.yml)
+- The metrics count totals:
+  - [`config/metrics/counts_7d/20210216184557_ci_templates_total_unique_counts_weekly.yml#L19`](https://gitlab.com/gitlab-org/gitlab/-/blob/4e01ef2b094763943348655ef77008aba7a052ae/config/metrics/counts_7d/20210216184557_ci_templates_total_unique_counts_weekly.yml#L19)
+  - [`config/metrics/counts_28d/20210216184559_ci_templates_total_unique_counts_monthly.yml#L19`](https://gitlab.com/gitlab-org/gitlab/-/blob/4e01ef2b094763943348655ef77008aba7a052ae/config/metrics/counts_28d/20210216184559_ci_templates_total_unique_counts_monthly.yml#L19)
+
 ## Security
 
 A template could contain malicious code. For example, a template that contains the `export` shell command in a job
diff --git a/doc/development/code_review.md b/doc/development/code_review.md
index 1ed510c6ad7..3664ca7642a 100644
--- a/doc/development/code_review.md
+++ b/doc/development/code_review.md
@@ -285,7 +285,7 @@ Maintainers should check before merging if the merge request is approved by the
 required approvers. If still awaiting further approvals from others, remove yourself as a reviewer then `@` mention the author and explain why in a comment. Stay as reviewer if you're merging the code.
 
 Maintainers must check before merging if the merge request is introducing new
-vulnerabilities, by inspecting the list in the Merge Request
+vulnerabilities, by inspecting the list in the merge request
 [Security Widget](../user/application_security/index.md).
 When in doubt, a [Security Engineer](https://about.gitlab.com/company/team/) can be involved. The list of detected
 vulnerabilities must be either empty or containing:
@@ -296,8 +296,8 @@ vulnerabilities must be either empty or containing:
 Maintainers should **never** dismiss vulnerabilities to "empty" the list,
 without duly verifying them.
 
-Note that certain Merge Requests may target a stable branch. These are rare
-events. These types of Merge Requests cannot be merged by the Maintainer.
+Note that certain merge requests may target a stable branch. These are rare
+events. These types of merge requests cannot be merged by the Maintainer.
 Instead, these should be sent to the [Release Manager](https://about.gitlab.com/community/release-managers/).
 
 After merging, a maintainer should stay as the reviewer listed on the merge request.
@@ -479,7 +479,7 @@ WARNING:
     [very specific cases](https://about.gitlab.com/handbook/engineering/workflow/#criteria-for-merging-during-broken-master).
     For other cases, follow these [handbook instructions](https://about.gitlab.com/handbook/engineering/workflow/#merging-during-broken-master).
   - If the latest pipeline was created before the merge request was approved, start a new pipeline to ensure that full RSpec suite has been run. You may skip this step only if the merge request does not contain any backend change.
-  - If the **latest [Pipeline for Merged Results](../ci/pipelines/pipelines_for_merged_results.md)** finished less than 2 hours ago, you
+  - If the **latest [merged results pipeline](../ci/pipelines/merged_results_pipelines.md)** finished less than 2 hours ago, you
     may merge without starting a new pipeline as the merge request is close
     enough to `main`.
 - When you set the MR to "Merge When Pipeline Succeeds", you should take over
@@ -489,7 +489,7 @@ WARNING:
   the squashed commit's default commit message is taken from the merge request title.
   You're encouraged to [select a commit with a more informative commit message](../user/project/merge_requests/squash_and_merge.md) before merging.
 
-Thanks to **Pipeline for Merged Results**, authors no longer have to rebase their
+Thanks to **merged results pipelines**, authors no longer have to rebase their
 branch as frequently anymore (only when there are conflicts) because the Merge
 Results Pipeline already incorporate the latest changes from `main`.
 This results in faster review/merge cycles because maintainers don't have to ask
@@ -501,7 +501,7 @@ Merge Results against the latest `main` at the time of the pipeline creation.
 
 WARNING:
 **Review all changes thoroughly for malicious code before starting a
-[Pipeline for Merged Results](../ci/pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project).**
+[merged results pipeline](../ci/pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project).**
 
 When reviewing merge requests added by wider community contributors:
 
@@ -510,6 +510,9 @@ When reviewing merge requests added by wider community contributors:
   fetching of malicious packages.
 - Review links and images, especially in documentation MRs.
 - When in doubt, ask someone from `@gitlab-com/gl-security/appsec` to review the merge request **before manually starting any merge request pipeline**.
+- Only set the milestone when the merge request is likely to be included in
+  the current milestone. This is to avoid confusion around when it'll be
+  merged and avoid moving milestone too often when it's not yet ready.
 
 If the MR source branch is more than 1,000 commits behind the target branch:
 
@@ -599,7 +602,7 @@ Enterprise Edition instance. This has some implications:
       - [Background migrations](background_migrations.md) run in Sidekiq, and
         should only be done for migrations that would take an extreme amount of
         time at GitLab.com scale.
-1. **Sidekiq workers** [cannot change in a backwards-incompatible way](sidekiq_style_guide.md#sidekiq-compatibility-across-updates):
+1. **Sidekiq workers** [cannot change in a backwards-incompatible way](sidekiq/compatibility_across_updates.md):
    1. Sidekiq queues are not drained before a deploy happens, so there are
       workers in the queue from the previous version of GitLab.
    1. If you need to change a method signature, try to do so across two releases,
@@ -670,7 +673,7 @@ Properties of customer critical merge requests:
 - It is required that the reviewer(s) and maintainer(s) involved with a customer critical merge request are engaged as soon as this decision is made.
 - It is required to prioritize work for those involved on a customer critical merge request so that they have the time available necessary to focus on it.
 - It is required to adhere to GitLab [values](https://about.gitlab.com/handbook/values/) and processes when working on customer critical merge requests, taking particular note of family and friends first/work second, definition of done, iteration, and release when it's ready.
-- Customer critical merge requests are required to not reduce security, introduce data-loss risk, reduce availability, nor break existing functionality per the process for [prioritizing technical decisions](https://about.gitlab.com/handbook/engineering/#prioritizing-technical-decisions.md).
+- Customer critical merge requests are required to not reduce security, introduce data-loss risk, reduce availability, nor break existing functionality per the process for [prioritizing technical decisions](https://about.gitlab.com/handbook/engineering/principles/#prioritizing-technical-decisions).
 - On customer critical requests, it is _recommended_ that those involved _consider_ coordinating synchronously (Zoom, Slack) in addition to asynchronously (merge requests comments) if they believe this may reduce the elapsed time to merge even though this _may_ sacrifice [efficiency](https://about.gitlab.com/company/culture/all-remote/asynchronous/#evaluating-efficiency.md).
 - After a customer critical merge request is merged, a retrospective must be completed with the intention of reducing the frequency of future customer critical merge requests.
 
diff --git a/doc/development/contributing/design.md b/doc/development/contributing/design.md
index e85f5dd8349..efa8d4b0c41 100644
--- a/doc/development/contributing/design.md
+++ b/doc/development/contributing/design.md
@@ -60,7 +60,7 @@ Check visual design properties using your browser's _elements inspector_ ([Chrom
   guidelines.
 - _Optionally_ consider [dark mode](../../user/profile/preferences.md#dark-mode). [^1]
 
- [^1]: You're not required to design for [dark mode](../../user/profile/preferences.md#dark-mode) while the feature is in [alpha](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha). The [UX Foundations team](https://about.gitlab.com/direction/ecosystem/foundations/) plans to improve the dark mode in the future. Until we integrate [Pajamas](https://design.gitlab.com/) components into the product and the underlying design strategy is in place to support dark mode, we cannot guarantee that we won't introduce bugs and debt to this mode. At your discretion, evaluate the need to create dark mode patches.
+ [^1]: You're not required to design for [dark mode](../../user/profile/preferences.md#dark-mode) while the feature is in [alpha](../../policy/alpha-beta-support.md#alpha-features). The [UX Foundations team](https://about.gitlab.com/direction/ecosystem/foundations/) plans to improve the dark mode in the future. Until we integrate [Pajamas](https://design.gitlab.com/) components into the product and the underlying design strategy is in place to support dark mode, we cannot guarantee that we won't introduce bugs and debt to this mode. At your discretion, evaluate the need to create dark mode patches.
 
 ### States
 
diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md
index ec05f75c709..ea54f36a7e5 100644
--- a/doc/development/contributing/index.md
+++ b/doc/development/contributing/index.md
@@ -91,7 +91,7 @@ Sign up for the mailing list, answer GitLab questions on StackOverflow or respon
 If you would like to contribute to GitLab:
 
 - Issues with the
-  [`~Accepting merge requests` label](issue_workflow.md#label-for-community-contributors)
+  [`~Seeking community contributions` label](issue_workflow.md#label-for-community-contributors)
   are a great place to start.
 - Optimizing our tests is another great opportunity to contribute. You can use
   [RSpec profiling statistics](https://gitlab-org.gitlab.io/rspec_profiling_stats/) to identify
diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md
index 6a7bbb2e302..ad8403d242c 100644
--- a/doc/development/contributing/issue_workflow.md
+++ b/doc/development/contributing/issue_workflow.md
@@ -57,6 +57,10 @@ Most issues will have labels for at least one of the following:
 - Priority: `~"priority::1"`, `~"priority::2"`, `~"priority::3"`, `~"priority::4"`
 - Severity: ~`"severity::1"`, `~"severity::2"`, `~"severity::3"`, `~"severity::4"`
 
+Please add `~"breaking change"` label if the issue can be considered as a [breaking change](index.md#breaking-changes).
+
+Please add `~security` label if the issue is related to application security.
+
 All labels, their meaning and priority are defined on the
 [labels page](https://gitlab.com/gitlab-org/gitlab/-/labels).
 
@@ -282,12 +286,12 @@ Please refer to the issue triage [severity label](https://about.gitlab.com/handb
 
 ### Label for community contributors
 
-Issues that are beneficial to our users, 'nice to haves', that we currently do
-not have the capacity for or want to give the priority to, are labeled as
-~"Accepting merge requests", so the community can make a contribution.
+There are many issues that have a clear solution with uncontroversial benefit to GitLab users.
+However, GitLab might not have the capacity for all these proposals in the current roadmap.
+These issues are labeled ~"Seeking community contributions" because we welcome merge requests to resolve them.
 
 Community contributors can submit merge requests for any issue they want, but
-the ~"Accepting merge requests" label has a special meaning. It points to
+the ~"Seeking community contributions" label has a special meaning. It points to
 changes that:
 
 1. We already agreed on,
@@ -295,20 +299,24 @@ changes that:
 1. Are likely to get accepted by a maintainer.
 
 We want to avoid a situation when a contributor picks an
-~"Accepting merge requests" issue and then their merge request gets closed,
+~"Seeking community contributions" issue and then their merge request gets closed,
 because we realize that it does not fit our vision, or we want to solve it in a
 different way.
 
-We automatically add the ~"Accepting merge requests" label to issues
-that match the [triage policy](https://about.gitlab.com/handbook/engineering/quality/triage-operations/#accepting-merge-requests).
+We manually add the ~"Seeking community contributions" label to issues
+that fit the criteria described above.
+We do not automatically add this label, because it requires human evaluation.
 
 We recommend people that have never contributed to any open source project to
-look for issues labeled `~"Accepting merge requests"` with a [weight of 1](https://gitlab.com/groups/gitlab-org/-/issues?state=opened&label_name[]=Accepting+merge+requests&assignee_id=None&sort=weight&weight=1) or the `~"good for new contributors"` [label](https://gitlab.com/gitlab-org/gitlab/-/issues?scope=all&state=opened&label_name[]=good%20for%20new%20contributors&assignee_id=None) attached to it.
+look for issues labeled `~"Seeking community contributions"` with a
+[weight of 1](https://gitlab.com/groups/gitlab-org/-/issues?sort=created_date&state=opened&label_name[]=Seeking+community+contributions&assignee_id=None&weight=1) or the `~"good for new contributors"`
+[label](https://gitlab.com/gitlab-org/gitlab/-/issues?scope=all&state=opened&label_name[]=good%20for%20new%20contributors&assignee_id=None)
+attached to it.
 More experienced contributors are very welcome to tackle
-[any of them](https://gitlab.com/groups/gitlab-org/-/issues?state=opened&label_name[]=Accepting+merge+requests&assignee_id=None).
+[any of them](https://gitlab.com/groups/gitlab-org/-/issues?sort=created_date&state=opened&label_name[]=Seeking+community+contributions&assignee_id=None).
 
 For more complex features that have a weight of 2 or more and clear scope, we recommend looking at issues
-with the [label `~"Community Challenge"`](https://gitlab.com/gitlab-org/gitlab/-/issues?scope=all&state=opened&label_name[]=Accepting%20merge%20requests&label_name[]=Community%20challenge).
+with the [label `~"Community Challenge"`](https://gitlab.com/gitlab-org/gitlab/-/issues?sort=created_date&state=opened&label_name[]=Seeking+community+contributions&label_name[]=Community+challenge).
 If your MR for the `~"Community Challenge"` issue gets merged, you will also have a chance to win a custom
 GitLab merchandise.
 
@@ -319,7 +327,7 @@ members to further discuss scope, design, and technical considerations. This wil
 ensure that your contribution is aligned with the GitLab product and minimize
 any rework and delay in getting it merged into main.
 
-GitLab team members who apply the ~"Accepting merge requests" label to an issue
+GitLab team members who apply the ~"Seeking community contributions" label to an issue
 should update the issue description with a responsible product manager, inviting
 any potential community contributor to @-mention per above.
 
diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md
index cc6997e1a20..02148f2a717 100644
--- a/doc/development/contributing/merge_request_workflow.md
+++ b/doc/development/contributing/merge_request_workflow.md
@@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 We welcome merge requests from everyone, with fixes and improvements
 to GitLab code, tests, and documentation. The issues that are specifically suitable
-for community contributions are listed with the [`Accepting merge requests`](issue_workflow.md#label-for-community-contributors)
+for community contributions have the [`Seeking community contributions`](issue_workflow.md#label-for-community-contributors)
 label, but you are free to contribute to any issue you want.
 
 If an issue is marked for the current milestone at any time, even
@@ -18,7 +18,7 @@ in order to ensure the work is finished before the release date.
 
 If you want to add a new feature that is not labeled, it is best to first create
 an issue (if there isn't one already) and leave a comment asking for it
-to be marked as `Accepting merge requests`. See the [feature proposals](issue_workflow.md#feature-proposals)
+to be labeled as `Seeking community contributions`. See the [feature proposals](issue_workflow.md#feature-proposals)
 section.
 
 Merge requests should be submitted to the appropriate project at GitLab.com, for example
@@ -71,6 +71,9 @@ request is as follows:
    1. The MR must include *Before* and *After* screenshots if UI changes are made.
    1. Include any steps or setup required to ensure reviewers can view the changes you've made (for example, include any information about feature flags).
 1. If you're allowed to, set a relevant milestone and [labels](issue_workflow.md).
+   MR labels should generally match the corresponding issue (if there is one).
+   The group label should reflect the group that executed or coached the work,
+   not necessarily the group that owns the feature.
 1. UI changes should use available components from the GitLab Design System,
    [Pajamas](https://design.gitlab.com/).
 1. If the MR changes CSS classes, please include the list of affected pages, which
diff --git a/doc/development/contributing/style_guides.md b/doc/development/contributing/style_guides.md
index fdb6e99fdcd..da926005466 100644
--- a/doc/development/contributing/style_guides.md
+++ b/doc/development/contributing/style_guides.md
@@ -51,20 +51,10 @@ This should return a fully qualified path command with no other output.
 
 ### Lefthook configuration
 
-The current Lefthook configuration can be found in [`lefthook.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lefthook.yml).
+Lefthook is configured with a combination of:
 
-Before you push your changes, Lefthook automatically runs the following checks:
-
-- Danger: Runs a subset of checks that `danger-review` runs on your merge requests.
-- ES lint: Run `yarn run lint:eslint` checks (with the [`.eslintrc.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.eslintrc.yml) configuration) on the modified `*.{js,vue}` files. Tags: `frontend`, `style`.
-- HAML lint: Run `bundle exec haml-lint` checks (with the [`.haml-lint.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.haml-lint.yml) configuration) on the modified `*.html.haml` files. Tags: `view`, `haml`, `style`.
-- Markdown lint: Run `yarn markdownlint` checks on the modified `*.md` files. Tags: `documentation`, `style`.
-- SCSS lint: Run `yarn lint:stylelint` checks (with the [`.stylelintrc`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.stylelintrc) configuration) on the modified `*.scss{,.css}` files. Tags: `stylesheet`, `css`, `style`.
-- RuboCop: Run `bundle exec rubocop` checks (with the [`.rubocop.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.rubocop.yml) configuration) on the modified `*.rb` files. Tags: `backend`, `style`.
-- Vale: Run `vale` checks (with the [`.vale.ini`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.vale.ini) configuration) on the modified `*.md` files. Tags: `documentation`, `style`.
-- Documentation metadata: Run checks for the absence of [documentation metadata](../documentation/index.md#metadata).
-
-In addition to the default configuration, you can define a [local configuration](https://github.com/Arkweid/lefthook/blob/master/docs/full_guide.md#local-config).
+- Project configuration in [`lefthook.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lefthook.yml).
+- Any [local configuration](https://github.com/Arkweid/lefthook/blob/master/docs/full_guide.md#local-config).
 
 ### Disable Lefthook temporarily
 
diff --git a/doc/development/dangerbot.md b/doc/development/dangerbot.md
index 374e4e5de68..8da1f5700e5 100644
--- a/doc/development/dangerbot.md
+++ b/doc/development/dangerbot.md
@@ -66,7 +66,7 @@ continue to apply. However, there are a few things that deserve special emphasis
 Danger is a powerful tool and flexible tool, but not always the most appropriate
 way to solve a given problem or workflow.
 
-First, be aware of the GitLab [commitment to dogfooding](https://about.gitlab.com/handbook/engineering/#dogfooding).
+First, be aware of the GitLab [commitment to dogfooding](https://about.gitlab.com/handbook/engineering/principles/#dogfooding).
 The code we write for Danger is GitLab-specific, and it **may not** be most
 appropriate place to implement functionality that addresses a need we encounter.
 Our users, customers, and even our own satellite projects, such as [Gitaly](https://gitlab.com/gitlab-org/gitaly),
diff --git a/doc/development/database/dbcheck-migrations-job.md b/doc/development/database/dbcheck-migrations-job.md
new file mode 100644
index 00000000000..af72e28a875
--- /dev/null
+++ b/doc/development/database/dbcheck-migrations-job.md
@@ -0,0 +1,67 @@
+---
+stage: Enablement
+group: Database
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# db:check-migrations job
+
+This job runs on the test stage of a merge request pipeline. It checks:
+
+1. A schema dump comparison between the author's working branch and the target branch,
+   after executing a rollback of your new migrations. This check validates that the
+   schema properly resets to what it was before executing this new migration.
+1. A schema dump comparison between the author's working branch and the `db/structure.sql`
+   file that the author committed. This check validates that it contains all the expected changes
+   in the migration.
+1. A Git diff between the `db/schema_migrations` that the author committed and the
+   one that the script generated after running migrations. This check validates that everything
+   was properly committed.
+
+## Troubleshooting
+
+### False positives
+
+This job is allowed to fail, because it can throw some false positives.
+
+For example, when we drop a column and then roll back, this column is always
+re-added at the end of the list of columns. If the column was previously in
+the middle of the list, the rollback can't return the schema back exactly to
+its previous state. The job fails, but it's an acceptable situation.
+
+For a real-life example, refer to
+[this failed job](https://gitlab.com/gitlab-org/gitlab/-/jobs/2006544972#L138).
+Here, the author dropped the `position` column.
+
+### Schema dump comparison fails after rollback
+
+This failure often happens if your working branch is behind the target branch.
+A real scenario:
+
+```mermaid
+graph LR
+    Main((main<br>commit A)) ===> |remove constraint<br>fk_rails_dbebdaa8fe| MainB((main<br>commit B))
+    Main((main<br>commit A)) --> |checkout<br>dev| DevA((dev<br>commit A)):::dev
+    DevA((dev<br>commit A)) --> |add column<br>dependency_proxy_size| DevC((dev<br>commit C)):::dev
+    DevC -.-> |CI pipeline<br>executes| JOB-FAILED((JOB FAILED!)):::error
+
+    classDef main fill:#f4f0ff,stroke:#7b58cf
+    classDef dev fill:#e9f3fc,stroke:#1f75cb
+    classDef error fill:#f15146,stroke:#d4121a
+```
+
+1. You check out the `dev` working branch from the `main` target branch. At this point,
+   each branch has their `HEAD` at commit A.
+1. Someone works on the `main` branch and drops the `fk_rails_dbebdaa8fe` constraint,
+   thus creating commit B on `main`.
+1. You add column `dependency_proxy_size` to your `dev` branch.
+1. The `db:check-migrations` job fails for your `dev` branch's CI/CD pipeline, because
+   the `structure.sql` file did not rollback to its expected state.
+
+This happened because branch `dev` contained commits A and C, not B. Its database schema
+did not know about the removal of the `fk_rails_dbebdaa8fe` constraint. When comparing the two
+schemas, the `dev` branch contained this constraint while the `main` branch didn't.
+
+This example really happened. Read the [job failure logs](https://gitlab.com/gitlab-org/gitlab/-/jobs/1992050890).
+
+To fix these kind of issues, rebase the working branch onto the target branch to get the latest changes.
diff --git a/doc/development/database/efficient_in_operator_queries.md b/doc/development/database/efficient_in_operator_queries.md
index 76518a6f5e2..2503be826ea 100644
--- a/doc/development/database/efficient_in_operator_queries.md
+++ b/doc/development/database/efficient_in_operator_queries.md
@@ -368,7 +368,7 @@ then the query might perform worse than the non-optimized query. The `milestone_
 "index_issues_on_milestone_id" btree (milestone_id)
 ```
 
-Adding the `miletone_id = X` filter to the `scope` argument or to the optimized scope causes bad performance.
+Adding the `milestone_id = X` filter to the `scope` argument or to the optimized scope causes bad performance.
 
 Example (bad):
 
@@ -618,7 +618,7 @@ The following example shows the final `ORDER BY` clause:
 ORDER BY extract('epoch' FROM epics.closed_at - epics.created_at) DESC, epics.id DESC
 ```
 
-Snippet for loading records ordered by the calcualted duration:
+Snippet for loading records ordered by the calculated duration:
 
 ```ruby
 arel_table =  Epic.arel_table
@@ -641,7 +641,7 @@ records = Gitlab::Pagination::Keyset::InOperatorOptimization::QueryBuilder.new(
   array_mapping_scope: -> (id_expression) { Epic.where(Epic.arel_table[:group_id].eq(id_expression)) }
 ).execute.limit(20)
 
-puts records.pluck(:duration_in_seconds, :id) # other columnns are not available
+puts records.pluck(:duration_in_seconds, :id) # other columns are not available
 ```
 
 Building the query requires quite a bit of configuration. For the order configuration you
diff --git a/doc/development/database/index.md b/doc/development/database/index.md
index a7b752e14ef..efc48f72d00 100644
--- a/doc/development/database/index.md
+++ b/doc/development/database/index.md
@@ -19,6 +19,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 - [Understanding EXPLAIN plans](../understanding_explain_plans.md)
 - [explain.depesz.com](https://explain.depesz.com/) or [explain.dalibo.com](https://explain.dalibo.com/) for visualizing the output of `EXPLAIN`
 - [pgFormatter](https://sqlformat.darold.net/) a PostgreSQL SQL syntax beautifier
+- [db:check-migrations job](dbcheck-migrations-job.md)
 
 ## Migrations
 
diff --git a/doc/development/database/loose_foreign_keys.md b/doc/development/database/loose_foreign_keys.md
index 97d150b1a18..d08e90683fe 100644
--- a/doc/development/database/loose_foreign_keys.md
+++ b/doc/development/database/loose_foreign_keys.md
@@ -61,7 +61,7 @@ following information:
 - Child table name (`ci_pipelines`)
 - The data cleanup method (`async_delete` or `async_nullify`)
 
-The YAML file is located at `lib/gitlab/database/gitlab_loose_foreign_keys.yml`. The file groups
+The YAML file is located at `config/gitlab_loose_foreign_keys.yml`. The file groups
 foreign key definitions by the name of the child table. The child table can have multiple loose
 foreign key definitions, therefore we store them as an array.
 
diff --git a/doc/development/database_review.md b/doc/development/database_review.md
index 082c843a12c..8e217725a17 100644
--- a/doc/development/database_review.md
+++ b/doc/development/database_review.md
@@ -49,7 +49,7 @@ If new migrations are introduced, in the MR **you are required to provide**:
 
 Note that we have automated tooling for
 [GitLab](https://gitlab.com/gitlab-org/gitlab) (provided by the
-`db:check-migrations` pipeline job) that provides this output for migrations on
+[`db:check-migrations`](database/dbcheck-migrations-job.md) pipeline job) that provides this output for migrations on
 ~database merge requests. You do not need to provide this information manually
 if the bot can do it for you. The bot also checks that migrations are correctly
 reversible.
@@ -66,7 +66,7 @@ Refer to [Preparation when adding or modifying queries](#preparation-when-adding
 
 ### Roles and process
 
-A Merge Request **author**'s role is to:
+A merge request **author**'s role is to:
 
 - Decide whether a database review is needed.
 - If database review is needed, add the ~database label.
diff --git a/doc/development/dependencies.md b/doc/development/dependencies.md
index c81c6408211..329539f0cc2 100644
--- a/doc/development/dependencies.md
+++ b/doc/development/dependencies.md
@@ -51,6 +51,6 @@ This has certain benefits as outlined in our <a href="https://docs.gitlab.com/ee
 
 You might find that we do not currently update DEPENDENCY automatically, but we are planning to do so in [the near future](https://gitlab.com/gitlab-org/frontend/rfcs/-/issues/21).
 
-Thank you for understanding, I will close this Merge Request.
+Thank you for understanding, I will close this merge request.
 /close
 ```
diff --git a/doc/development/deprecation_guidelines/index.md b/doc/development/deprecation_guidelines/index.md
index 27c29a1ed7c..08e29e373f6 100644
--- a/doc/development/deprecation_guidelines/index.md
+++ b/doc/development/deprecation_guidelines/index.md
@@ -6,8 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # Deprecation guidelines
 
-This page includes information about how and when to remove or make breaking
-changes to GitLab features.
+This page includes information about how and when to remove or make [breaking
+changes](../contributing/index.md#breaking-changes) to GitLab features.
 
 ## Terminology
 
diff --git a/doc/development/diffs.md b/doc/development/diffs.md
index aaa3340af33..d61de740f15 100644
--- a/doc/development/diffs.md
+++ b/doc/development/diffs.md
@@ -35,7 +35,7 @@ have changed since then, it should still serve as a good introduction.
 
 ### Merge request diffs
 
-When refreshing a Merge Request (pushing to a source branch, force-pushing to target branch, or if the target branch now contains any commits from the MR)
+When refreshing a merge request (pushing to a source branch, force-pushing to target branch, or if the target branch now contains any commits from the MR)
 we fetch the comparison information using `Gitlab::Git::Compare`, which fetches `base` and `head` data using Gitaly and diff between them through
 `Gitlab::Git::Diff.between`.
 The diffs fetching process _limits_ single file diff sizes and the overall size of the whole diff through a series of constant values. Raw diff files are
@@ -45,7 +45,7 @@ Even though diffs larger than 10% of the value of `ApplicationSettings#diff_max_
 we still keep them on PostgreSQL. However, diff files larger than defined _safety limits_
 (see the [Diff limits section](#diff-limits)) are _not_ persisted in the database.
 
-In order to present diffs information on the Merge Request diffs page, we:
+In order to present diffs information on the merge request diffs page, we:
 
 1. Fetch all diff files from database `merge_request_diff_files`
 1. Fetch the _old_ and _new_ file blobs in batch to:
diff --git a/doc/development/directory_structure.md b/doc/development/directory_structure.md
index 8ba77bade2c..f909fda897f 100644
--- a/doc/development/directory_structure.md
+++ b/doc/development/directory_structure.md
@@ -38,7 +38,7 @@ end
 ### About namespace naming
 
 A good guideline for naming a top-level namespace (bounded context) is to use the related
-feature category. For example, `Continuous Integration` feature category maps to `Ci::` namespace.
+[feature category](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/categories.yml). For example, `Continuous Integration` feature category maps to `Ci::` namespace.
 
 Alternatively a new class could be added to `Projects::` or `Groups::` if it's either:
 
diff --git a/doc/development/distributed_tracing.md b/doc/development/distributed_tracing.md
index 1e85abf585c..680ac71f857 100644
--- a/doc/development/distributed_tracing.md
+++ b/doc/development/distributed_tracing.md
@@ -1,6 +1,6 @@
 ---
 stage: Monitor
-group: Monitor
+group: Respond
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md
index 776e5aefd00..5cf7bb74549 100644
--- a/doc/development/documentation/index.md
+++ b/doc/development/documentation/index.md
@@ -68,7 +68,7 @@ Adhere to the [Documentation Style Guide](styleguide/index.md). If a style stand
 
 ## Folder structure and files
 
-See the [Structure](styleguide/index.md#structure) section of the [Documentation Style Guide](styleguide/index.md).
+See the [Folder structure](site_architecture/folder_structure.md) page.
 
 ## Metadata
 
@@ -290,9 +290,7 @@ For example, [GitLab.com's `/help`](https://gitlab.com/help).
 
 ## Docs site architecture
 
-See the [Docs site architecture](site_architecture/index.md) page to learn
-how we build and deploy the site at <https://docs.gitlab.com> and
-to review all the assets and libraries in use.
+For information on how we build and deploy <https://docs.gitlab.com>, see [Docs site architecture](site_architecture/index.md).
 
 ### Global navigation
 
diff --git a/doc/development/documentation/review_apps.md b/doc/development/documentation/review_apps.md
index 4b58778a20c..8cb9e6437b8 100644
--- a/doc/development/documentation/review_apps.md
+++ b/doc/development/documentation/review_apps.md
@@ -41,7 +41,7 @@ the GitLab team to run the job.
 If you want to know the in-depth details, here's what's really happening:
 
 1. You manually run the `review-docs-deploy` job in a merge request.
-1. The job runs the [`scripts/trigger-build`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/trigger-build)
+1. The job runs the [`scripts/trigger-build.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/trigger-build.rb)
    script with the `docs deploy` flag, which triggers the "Triggered from `gitlab-org/gitlab` 'review-docs-deploy' job"
    pipeline trigger in the `gitlab-org/gitlab-docs` project for the `$DOCS_BRANCH` (defaults to `main`).
 1. The preview URL is shown both at the job output and in the merge request
@@ -61,7 +61,7 @@ The following GitLab features are used among others:
 - [Review Apps](../../ci/review_apps/index.md)
 - [Artifacts](../../ci/yaml/index.md#artifacts)
 - [Specific runner](../../ci/runners/runners_scope.md#prevent-a-specific-runner-from-being-enabled-for-other-projects)
-- [Pipelines for merge requests](../../ci/pipelines/merge_request_pipelines.md)
+- [Merge request pipelines](../../ci/pipelines/merge_request_pipelines.md)
 
 ## Troubleshooting review apps
 
diff --git a/doc/development/documentation/site_architecture/deployment_process.md b/doc/development/documentation/site_architecture/deployment_process.md
index 25bc699c9d4..5203ca52922 100644
--- a/doc/development/documentation/site_architecture/deployment_process.md
+++ b/doc/development/documentation/site_architecture/deployment_process.md
@@ -82,7 +82,7 @@ for the stable branch of the image to rebuild. You might do this:
 
 ## Latest documentation
 
-A Docker image (tagged `latest`) is built that contains:
+We build a Docker image (tagged `latest`) that contains:
 
 - The latest online version of the documentation.
 - The documentation from the stable branches of upstream projects.
diff --git a/doc/development/documentation/site_architecture/folder_structure.md b/doc/development/documentation/site_architecture/folder_structure.md
new file mode 100644
index 00000000000..e960a6491c7
--- /dev/null
+++ b/doc/development/documentation/site_architecture/folder_structure.md
@@ -0,0 +1,98 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Folder structure for documentation
+
+The documentation is separated by top-level audience folders [`user`](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/doc/user),
+[`administration`](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/doc/administration),
+and [`development`](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/doc/development)
+(contributing) folders.
+
+Beyond that, we primarily follow the structure of the GitLab user interface or
+API.
+
+Our goal is to have a clear hierarchical structure with meaningful URLs like
+`docs.gitlab.com/user/project/merge_requests/`. With this pattern, you can
+immediately tell that you are navigating to user-related documentation about
+Project features; specifically about Merge Requests. Our site's paths match
+those of our repository, so the clear structure also makes documentation easier
+to update.
+
+Put files for a specific product area into the related folder:
+
+| Directory             | Contents |
+|:----------------------|:------------------|
+| `doc/user/`           | Documentation for users. Anything that can be done in the GitLab user interface goes here, including usage of the `/admin` interface. |
+| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. Administrator settings in the GitLab user interface are under `doc/user/admin_area/`. |
+| `doc/api/`            | Documentation for the API. |
+| `doc/development/`    | Documentation related to the development of GitLab, whether contributing code or documentation. Related process and style guides should go here. |
+| `doc/legal/`          | Legal documents about contributing to GitLab. |
+| `doc/install/`        | Instructions for installing GitLab. |
+| `doc/update/`         | Instructions for updating GitLab. |
+| `doc/topics/`         | Indexes per topic (`doc/topics/topic_name/index.md`): all resources for that topic. |
+
+## Work with directories and files
+
+When working with directories and files:
+
+1. When you create a new directory, always start with an `index.md` file.
+   Don't use another filename and do not create `README.md` files.
+1. Do not use special characters and spaces, or capital letters in file
+   names, directory names, branch names, and anything that generates a path.
+1. When creating or renaming a file or directory and it has more than one word
+   in its name, use underscores (`_`) instead of spaces or dashes. For example,
+   proper naming would be `import_project/import_from_github.md`. This applies
+   to both image files and Markdown files.
+1. For image files, do not exceed 100KB.
+1. Do not upload video files to the product repositories.
+   [Link or embed videos](../styleguide/index.md#videos) instead.
+1. There are four main directories: `user`, `administration`, `api`, and
+   `development`.
+1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`,
+   `profile/`, `dashboard/` and `admin_area/`.
+   - `doc/user/project/` should contain all project related documentation.
+   - `doc/user/group/` should contain all group related documentation.
+   - `doc/user/profile/` should contain all profile related documentation.
+     Every page you would navigate under `/profile` should have its own document,
+     for example, `account.md`, `applications.md`, or `emails.md`.
+   - `doc/user/dashboard/` should contain all dashboard related documentation.
+   - `doc/user/admin_area/` should contain all administrator-related
+     documentation describing what can be achieved by accessing the GitLab
+     administrator interface (not to be confused with `doc/administration` where
+     server access is required).
+     - Every category under `/admin/application_settings/` should have its
+      own document located at `doc/user/admin_area/settings/`. For example,
+      the **Visibility and Access Controls** category should have a document
+      located at `doc/user/admin_area/settings/visibility_and_access_controls.md`.
+1. The `doc/topics/` directory holds topic-related technical content. Create
+   `doc/topics/topic_name/subtopic_name/index.md` when subtopics become necessary.
+   General user and administrator documentation should be placed accordingly.
+1. The `/university/` directory is *deprecated* and the majority of its documentation
+   has been moved.
+
+If you're unsure where to place a document or a content addition, this shouldn't
+stop you from authoring and contributing. Use your best judgment, and then ask
+the reviewer of your MR to confirm your decision. You can also ask a technical writer at
+any stage in the process. The technical writing team reviews all
+documentation changes, regardless, and can move content if there is a better
+place for it.
+
+## Avoid duplication
+
+Do not include the same information in multiple places.
+[Link to a single source of truth instead.](../styleguide/index.md#link-instead-of-repeating-text)
+
+## References across documents
+
+- Give each folder an `index.md` page that introduces the topic, and both introduces
+  and links to the child pages, including to the index pages of
+  any next-level sub-paths.
+- To ensure discoverability, ensure each new or renamed doc is linked from its
+  higher-level index page and other related pages.
+- When making reference to other GitLab products and features, link to their
+  respective documentation, at least on first mention.
+- When making reference to third-party products or technologies, link out to
+  their external sites, documentation, and resources.
diff --git a/doc/development/documentation/site_architecture/index.md b/doc/development/documentation/site_architecture/index.md
index 60894430d15..e7a915eab09 100644
--- a/doc/development/documentation/site_architecture/index.md
+++ b/doc/development/documentation/site_architecture/index.md
@@ -52,19 +52,20 @@ product, and all together are pulled to generate the docs website:
 - [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner/-/tree/main/docs)
 - [GitLab Chart](https://gitlab.com/charts/gitlab/tree/master/doc)
 
+Learn more about [the docs folder structure](folder_structure.md).
+
 ## Assets
 
 To provide an optimized site structure, design, and a search-engine friendly
 website, along with a discoverable documentation, we use a few assets for
 the GitLab Documentation website.
 
-### Libraries
+### External libraries
+
+GitLab Docs is built with a combination of external:
 
-- [Bootstrap 4.3.1 components](https://getbootstrap.com/docs/4.3/components/)
-- [Bootstrap 4.3.1 JS](https://getbootstrap.com/docs/4.3/getting-started/javascript/)
-- [jQuery](https://jquery.com/) 3.3.1
-- [Clipboard JS](https://clipboardjs.com/)
-- [Font Awesome 4.7.0](https://fontawesome.com/v4.7.0/icons/)
+- [JavaScript libraries](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/package.json).
+- [Ruby libraries](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/Gemfile).
 
 ### SEO
 
@@ -98,7 +99,7 @@ The pipeline in the `gitlab-docs` project:
 
 Once a week on Mondays, a scheduled pipeline runs and rebuilds the Docker images
 used in various pipeline jobs, like `docs-lint`. The Docker image configuration files are
-located in the [Dockerfiles directory](https://gitlab.com/gitlab-org/gitlab-docs/-/tree/master/dockerfiles).
+located in the [Dockerfiles directory](https://gitlab.com/gitlab-org/gitlab-docs/-/tree/main/dockerfiles).
 
 If you need to rebuild the Docker images immediately (must have maintainer level permissions):
 
diff --git a/doc/development/documentation/structure.md b/doc/development/documentation/structure.md
index 6ecffce01b4..21368098f39 100644
--- a/doc/development/documentation/structure.md
+++ b/doc/development/documentation/structure.md
@@ -5,10 +5,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 description: What to include in GitLab documentation pages.
 ---
 
-# Documentation topic types
+# Documentation topic types (CTRT)
 
 At GitLab, we have not traditionally used types for our content. However, we are starting to
-move in this direction, and we now use four primary topic types:
+move in this direction, and we now use four primary topic types (CTRT):
 
 - [Concept](#concept)
 - [Task](#task)
@@ -154,7 +154,7 @@ If multiple causes or workarounds exist, consider putting them into a table form
 ## Other types of content
 
 There are other types of content in the GitLab documentation that don't
-classify as one of the four primary [topic types](#documentation-topic-types).
+classify as one of the four primary [topic types](#documentation-topic-types-ctrt).
 These include:
 
 - [Tutorials](#tutorials)
@@ -174,7 +174,7 @@ In general, you might consider using a tutorial when:
 
 Tutorials are learning aids that complement our core documentation.
 They do not introduce new features.
-Always use the primary [topic types](#documentation-topic-types) to document new features.
+Always use the primary [topic types](#documentation-topic-types-ctrt) to document new features.
 
 Tutorials should be in this format:
 
diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md
index e57ffb90028..3e9c0177d48 100644
--- a/doc/development/documentation/styleguide/index.md
+++ b/doc/development/documentation/styleguide/index.md
@@ -181,115 +181,6 @@ included in backticks. For example:
   - `git clone` is a command, so it must be lowercase, while Git is the product,
     so it must have a capital G.
 
-## Structure
-
-We include concept and task topic types in the same larger topic.
-
-In general, we have one topic that's a landing page.
-Below that topic in the left nav are individual topics. Each of these include a concept
-and multiple related tasks, reference, and troubleshooting topics.
-
-### Folder structure overview
-
-The documentation is separated by top-level audience folders [`user`](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/doc/user),
-[`administration`](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/doc/administration),
-and [`development`](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/doc/development)
-(contributing) folders.
-
-Beyond that, we primarily follow the structure of the GitLab user interface or
-API.
-
-Our goal is to have a clear hierarchical structure with meaningful URLs like
-`docs.gitlab.com/user/project/merge_requests/`. With this pattern, you can
-immediately tell that you are navigating to user-related documentation about
-Project features; specifically about Merge Requests. Our site's paths match
-those of our repository, so the clear structure also makes documentation easier
-to update.
-
-Put files for a specific product area into the related folder:
-
-| Directory             | Contents |
-|:----------------------|:------------------|
-| `doc/user/`           | Documentation for users. Anything that can be done in the GitLab user interface goes here, including usage of the `/admin` interface. |
-| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. Administrator settings in the GitLab user interface are under `doc/user/admin_area/`. |
-| `doc/api/`            | Documentation for the API. |
-| `doc/development/`    | Documentation related to the development of GitLab, whether contributing code or documentation. Related process and style guides should go here. |
-| `doc/legal/`          | Legal documents about contributing to GitLab. |
-| `doc/install/`        | Instructions for installing GitLab. |
-| `doc/update/`         | Instructions for updating GitLab. |
-| `doc/topics/`         | Indexes per topic (`doc/topics/topic_name/index.md`): all resources for that topic. |
-
-### Work with directories and files
-
-When working with directories and files:
-
-1. When you create a new directory, always start with an `index.md` file.
-   Don't use another filename and do not create `README.md` files.
-1. Do not use special characters and spaces, or capital letters in file
-   names, directory names, branch names, and anything that generates a path.
-1. When creating or renaming a file or directory and it has more than one word
-   in its name, use underscores (`_`) instead of spaces or dashes. For example,
-   proper naming would be `import_project/import_from_github.md`. This applies
-   to both image files and Markdown files.
-1. For image files, do not exceed 100KB.
-1. Do not upload video files to the product repositories.
-   [Link or embed videos](#videos) instead.
-1. There are four main directories: `user`, `administration`, `api`, and
-   `development`.
-1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`,
-   `profile/`, `dashboard/` and `admin_area/`.
-   - `doc/user/project/` should contain all project related documentation.
-   - `doc/user/group/` should contain all group related documentation.
-   - `doc/user/profile/` should contain all profile related documentation.
-     Every page you would navigate under `/profile` should have its own document,
-     for example, `account.md`, `applications.md`, or `emails.md`.
-   - `doc/user/dashboard/` should contain all dashboard related documentation.
-   - `doc/user/admin_area/` should contain all administrator-related
-     documentation describing what can be achieved by accessing the GitLab
-     administrator interface (not to be confused with `doc/administration` where
-     server access is required).
-     - Every category under `/admin/application_settings/` should have its
-      own document located at `doc/user/admin_area/settings/`. For example,
-      the **Visibility and Access Controls** category should have a document
-      located at `doc/user/admin_area/settings/visibility_and_access_controls.md`.
-1. The `doc/topics/` directory holds topic-related technical content. Create
-   `doc/topics/topic_name/subtopic_name/index.md` when subtopics become necessary.
-   General user and administrator documentation should be placed accordingly.
-1. The `/university/` directory is *deprecated* and the majority of its documentation
-   has been moved.
-
-If you're unsure where to place a document or a content addition, this shouldn't
-stop you from authoring and contributing. Use your best judgment, and then ask
-the reviewer of your MR to confirm your decision. You can also ask a technical writer at
-any stage in the process. The technical writing team reviews all
-documentation changes, regardless, and can move content if there is a better
-place for it.
-
-### Avoid duplication
-
-Do not include the same information in multiple places.
-[Link to a single source of truth instead.](#link-instead-of-repeating-text)
-
-### References across documents
-
-- Give each folder an `index.md` page that introduces the topic, and both introduces
-  and links to the child pages, including to the index pages of
-  any next-level sub-paths.
-- To ensure discoverability, ensure each new or renamed doc is linked from its
-  higher-level index page and other related pages.
-- When making reference to other GitLab products and features, link to their
-  respective documentation, at least on first mention.
-- When making reference to third-party products or technologies, link out to
-  their external sites, documentation, and resources.
-
-### Structure in documents
-
-- Include any and all applicable subsections as described on the
-  [structure and template](../structure.md) page.
-- Structure content in alphabetical order in tables, lists, and so on, unless
-  there's a logical reason not to (for example, when mirroring the user
-  interface or an otherwise ordered sequence).
-
 ## Language
 
 GitLab documentation should be clear and easy to understand.
@@ -320,10 +211,11 @@ create an issue or an MR to propose a change to the user interface text.
 #### Feature names
 
 - Feature names are typically lowercase.
-- Some features are capitalized, typically nouns that name GitLab-specific
-  capabilities or tools.
-
-See the [word list](word_list.md) for details.
+- Some features require title case, typically nouns that name GitLab-specific capabilities or tools. Features requiring
+  title case should be:
+  - Added as a proper name to markdownlint [configuration](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.markdownlint.yml),
+    so that it can be consistently applied across all documentation.
+  - Added to the [word list](word_list.md).
 
 If the term is not in the word list, ask a GitLab Technical Writer for advice.
 
@@ -1151,7 +1043,8 @@ When you take screenshots:
 - **Review how the image renders on the page.** Preview the image locally or use the
 review app in the merge request. Make sure the image isn't blurry or overwhelming.
 - **Be consistent.** Coordinate screenshots with the other screenshots already on
-  a documentation page for a consistent reading experience.
+  a documentation page for a consistent reading experience. Ensure your navigation theme
+  is **Indigo** and the syntax highlighting theme is **Light**. These are the default preferences.
 
 ### Add callouts
 
@@ -1745,11 +1638,12 @@ If the content in a topic is not ready, use the disclaimer in the topic.
 
 ### Removing versions after each major release
 
-Whenever a major GitLab release occurs, we remove all version references
-to now-unsupported versions of GitLab. Note that this includes the removal of
-specific instructions for users of non-supported GitLab versions. For example,
-if GitLab versions 11.x and later are supported, special
-instructions for users of GitLab 10 should be removed.
+When a major GitLab release occurs, we remove all references
+to now-unsupported versions. This removal includes version-specific instructions. For example,
+if GitLab version 12.1 and later are supported,
+instructions for users of GitLab 11 should be removed.
+
+[View the list of supported versions](https://about.gitlab.com/support/statement-of-support.html#version-support).
 
 To view historical information about a feature, review GitLab
 [release posts](https://about.gitlab.com/releases/), or search for the issue or
diff --git a/doc/development/documentation/styleguide/word_list.md b/doc/development/documentation/styleguide/word_list.md
index dc3dcba0c95..2c435cdc69d 100644
--- a/doc/development/documentation/styleguide/word_list.md
+++ b/doc/development/documentation/styleguide/word_list.md
@@ -58,11 +58,7 @@ Capitalize these words when you refer to the UI. Otherwise use lowercase.
 
 ## administrator
 
-Use **administrator** instead of **admin** when talking about a user's access level.
-Use lowercase unless you are referring to the **Admin** access level you select in the UI.
-
-To view the administrator access level, in the GitLab UI, go to the Admin Area and select
-**Users**. Then select **New user**.
+Use **administrator access** instead of **admin** when talking about a user's access level.
 
 ![admin access level](img/admin_access_level.png)
 
@@ -71,7 +67,7 @@ An **administrator** is not a [role](#roles) or [permission](#permissions).
 Use:
 
 - To do this thing, you must be an administrator.
-- To do this thing, you must have the administrator access level.
+- To do this thing, you must have administrator access.
 
 Instead of:
 
@@ -82,6 +78,17 @@ Instead of:
 Use title case **Admin Area** to refer to the area of the UI that you access when you select **Menu > Admin**.
 This area of the UI says **Admin Area** at the top of the page and on the menu.
 
+## agent
+
+Use lowercase to refer to the [GitLab agent for Kubernetes](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent).
+For example:
+
+- To connect your cluster to GitLab, use the GitLab agent for Kubernetes.
+- Install the agent in your cluster.
+- Select an agent from the list.
+
+Do not use title case **GitLab Agent** or **GitLab Agent for Kubernetes**.
+
 ## allow, enable
 
 Try to avoid **allow** and **enable**, unless you are talking about security-related features.
@@ -102,7 +109,7 @@ This phrasing is more active and is from the user perspective, rather than the p
 
 Use uppercase for **Alpha**. For example: **The XYZ feature is in Alpha.** or **This Alpha release is ready to test.**
 
-You might also want to link to [this section](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga)
+You might also want to link to [this section](../../../policy/alpha-beta-support.md#alpha-features)
 in the handbook when writing about Alpha features.
 
 ## and/or
@@ -128,7 +135,7 @@ Try to avoid **below** when referring to an example or table in a documentation
 
 Use uppercase for **Beta**. For example: **The XYZ feature is in Beta.** or **This Beta release is ready to test.**
 
-You might also want to link to [this section](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga)
+You might also want to link to [this section](../../../policy/alpha-beta-support.md#beta-features)
 in the handbook when writing about Beta features.
 
 ## blacklist
@@ -194,6 +201,10 @@ CI/CD is always uppercase. No need to spell it out on first use.
 Use **CI/CD minutes** instead of **CI minutes**, **pipeline minutes**, **pipeline minutes quota**, or
 **CI pipeline minutes**. This decision was made in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342813).
 
+## CI/CD tunnel
+
+Use lowercase for **tunnel** in **CI/CD tunnel**.
+
 ## click
 
 Do not use **click**. Instead, use **select** with buttons, links, menu items, and lists.
@@ -312,6 +323,11 @@ Instead of:
 
 Use **expand** instead of **open** when you are talking about expanding or collapsing a section in the UI.
 
+## FAQ
+
+We want users to find information quickly, and they rarely search for the term **FAQ**.
+Information in FAQs belongs with other similar information, under an easily searchable topic title.
+
 ## field
 
 Use **box** instead of **field** or **text box**.
@@ -367,6 +383,11 @@ Use title case for **GitLab Runner**. This is the product you install. See also
 
 Use **GitLab self-managed** to refer to the product license for GitLab instances managed by customers themselves.
 
+## guide
+
+We want to speak directly to users. On `docs.gitlab.com`, do not use **guide** as part of a page title.
+For example, **Snowplow Guide**. Instead, speak about the feature itself, and how to use it. For example, **Use Snowplow to do xyz**.
+
 ## Guest
 
 When writing about the Guest role:
@@ -540,6 +561,8 @@ Do not use **navigate**. Use **go** instead. For example:
 - Go to this webpage.
 - Open a terminal and go to the `runner` directory.
 
+([Vale](../testing.md#vale) rule: [`SubstitutionSuggestions.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SubstitutionSuggestions.yml))
+
 ## need to, should
 
 Try to avoid **needs to**, because it's wordy. Avoid **should** when you can be more specific. If something is required, use **must**.
@@ -658,10 +681,22 @@ Use **press** when talking about keyboard keys. For example:
 
 Do not use profanity. Doing so may negatively affect other users and contributors, which is contrary to the GitLab value of [Diversity, Inclusion, and Belonging](https://about.gitlab.com/handbook/values/#diversity-inclusion).
 
+## provision
+
+Use the term **provision** when referring to provisioning cloud infrastructure. You provision the infrastructure, and then deploy applications to it.
+
+For example, you might write something like:
+
+- Provision an AWS EKS cluster and deploy your application to it.
+
 ## push rules
 
 Use lowercase for **push rules**.
 
+## register
+
+Use **register** instead of **sign up** when talking about creating an account.
+
 ## Reporter
 
 When writing about the Reporter role:
@@ -690,6 +725,9 @@ Roles are not the same as [**access levels**](#access-level).
 
 Use lowercase for **runners**. These are the agents that run CI/CD jobs. See also [GitLab Runner](#gitlab-runner) and [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/233529).
 
+When referring to runners, if you have to specify that the runners are installed on a customer's GitLab instance,
+use **self-managed** rather than **self-hosted**.
+
 ## (s)
 
 Do not use **(s)** to make a word optionally plural. It can slow down comprehension. For example:
@@ -735,6 +773,10 @@ Instead of:
 Use **select** with buttons, links, menu items, and lists. **Select** applies to more devices,
 while **click** is more specific to a mouse.
 
+## self-managed
+
+Use **self-managed** to refer to a customer's installation of GitLab. Do not use **self-hosted**.
+
 ## Service Desk
 
 Use title case for **Service Desk**.
@@ -752,6 +794,10 @@ Use **sign in** instead of **sign on** or **log on** or **log in**. If the user
 
 You can use **single sign-on**.
 
+## sign up
+
+Use **register** instead of **sign up** when talking about creating an account.
+
 ## simply, simple
 
 Do not use **simply** or **simple**. If the user doesn't find the process to be simple, we lose their trust. ([Vale](../testing.md#vale) rule: [`Simplicity.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/Simplicity.yml))
@@ -883,10 +929,6 @@ Instead of:
 Do not use **utilize**. Use **use** instead. It's more succinct and easier for non-native English speakers to understand.
 ([Vale](../testing.md#vale) rule: [`SubstitutionSuggestions.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SubstitutionSuggestions.yml))
 
-## Value Stream Analytics
-
-Use title case for **Value Stream Analytics**.
-
 ## via
 
 Do not use Latin abbreviations. Use **with**, **through**, or **by using** instead. ([Vale](../testing.md#vale) rule: [`LatinTerms.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/LatinTerms.yml))
diff --git a/doc/development/documentation/testing.md b/doc/development/documentation/testing.md
index 134183a0e75..905b44823c3 100644
--- a/doc/development/documentation/testing.md
+++ b/doc/development/documentation/testing.md
@@ -126,6 +126,24 @@ If you don't want to install all of the dependencies to test the links, you can:
 If you manually install `haml-lint` with this process, it does not update automatically
 and you should make sure your version matches the version used by GitLab.
 
+## Update linter configuration
+
+[Vale configuration](#vale) and [markdownlint configuration](#markdownlint) is under source control in each 
+project, so updates must be committed to each project individually.
+
+We consider the configuration in the `gitlab` project as the source of truth and that's where all updates should
+first be made.
+
+On a regular basis, the changes made in `gitlab` project to the Vale and markdownlint configuration should be
+synchronized to the other projects. In `omnibus-gitlab`, `gitlab-runner`, and `charts/gitlab`:
+
+1. Create a new branch.
+1. Copy the configuration files from the `gitlab` project into this branch, overwriting
+   the project's old configuration. Make sure no project-specific changes from the `gitlab`
+   project are included. For example, [`RelativeLinks.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/RelativeLinks.yml)
+   is hard coded for specific projects.
+1. Create a merge request and submit it to a technical writer for review and merge.
+
 ## Local linters
 
 To help adhere to the [documentation style guidelines](styleguide/index.md), and improve the content
@@ -181,7 +199,7 @@ You can find Vale configuration in the following projects:
 - [`gitlab-runner`](https://gitlab.com/gitlab-org/gitlab-runner/-/tree/main/docs/.vale/gitlab)
 - [`omnibus-gitlab`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/tree/master/doc/.vale/gitlab)
 - [`charts`](https://gitlab.com/gitlab-org/charts/gitlab/-/tree/master/doc/.vale/gitlab)
-- [`gitlab-development-kit`](https://gitlab.com/gitlab-org/gitlab-development-kit/-/tree/master/doc/.vale/gitlab)
+- [`gitlab-development-kit`](https://gitlab.com/gitlab-org/gitlab-development-kit/-/tree/main/doc/.vale/gitlab)
 
 This configuration is also used in build pipelines, where
 [error-level rules](#vale-result-types) are enforced.
@@ -293,23 +311,7 @@ To configure Vale in your editor, install one of the following as appropriate:
 
 - Sublime Text [`SublimeLinter-contrib-vale` package](https://packagecontrol.io/packages/SublimeLinter-contrib-vale).
 - Visual Studio Code [`errata-ai.vale-server` extension](https://marketplace.visualstudio.com/items?itemName=errata-ai.vale-server).
-  You can configure the plugin to
-  [display only a subset of alerts](#show-subset-of-vale-alerts).
-
-  In the extension's settings:
-
-  <!-- vale gitlab.Spelling = NO -->
-
-  - Select the **Use CLI** checkbox.
-  - In the  **Config** setting, enter an absolute
-    path to [`.vale.ini`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.vale.ini)
-    in one of the cloned GitLab repositories on your computer.
-
-  <!-- vale gitlab.Spelling = YES -->
-
-  - In the **Path** setting, enter the absolute path to the Vale binary. In most
-    cases, `vale` should work. To find the location, run `which vale` in a terminal.
-
+  You can configure the plugin to [display only a subset of alerts](#show-subset-of-vale-alerts).
 - Vim [ALE plugin](https://github.com/dense-analysis/ale).
 - Emacs [Flycheck extension](https://github.com/flycheck/flycheck).
   This requires some configuration:
@@ -344,7 +346,7 @@ To configure Vale in your editor, install one of the following as appropriate:
   In this setup the `markdownlint` checker is set as a "next" checker from the defined `vale` checker.
   Enabling this custom Vale checker provides error linting from both Vale and markdownlint.
 
-We don't use [Vale Server](https://errata-ai.github.io/vale/#using-vale-with-a-text-editor-or-another-third-party-application).
+We don't use [Vale Server](https://docs.errata.ai/vale-server/install).
 
 ### Configure pre-push hooks
 
diff --git a/doc/development/documentation/workflow.md b/doc/development/documentation/workflow.md
index 49ad51874e3..a12af51e436 100644
--- a/doc/development/documentation/workflow.md
+++ b/doc/development/documentation/workflow.md
@@ -79,7 +79,7 @@ A member of the Technical Writing team adds these labels:
 
 ### Reviewing and merging
 
-Anyone with the [Maintainer role](../../user/permissions.md) to the relevant GitLab project can
+Anyone with the Maintainer role to the relevant GitLab project can
 merge documentation changes. Maintainers must make a good-faith effort to ensure that the content:
 
 - Is clear and sufficiently easy for the intended audience to navigate and understand.
@@ -157,7 +157,7 @@ Ensure the following if skipping an initial Technical Writer review:
 - Specific [user permissions](../../user/permissions.md) are documented.
 - New documents are linked from higher-level indexes, for discoverability.
 - The style guide is followed:
-  - For [directories and files](styleguide/index.md#work-with-directories-and-files).
+  - For [directories and files](site_architecture/folder_structure.md).
   - For [images](styleguide/index.md#images).
 
 Merge requests that change the location of documentation must always be reviewed by a Technical
diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md
index 9f705f2c6f1..17e35d34ec7 100644
--- a/doc/development/ee_features.md
+++ b/doc/development/ee_features.md
@@ -11,8 +11,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 - **Write documentation.**: Add documentation to the `doc/` directory. Describe
   the feature and include screenshots, if applicable. Indicate [what editions](documentation/styleguide/index.md#product-tier-badges)
   the feature applies to.
-- **Submit a MR to the `www-gitlab-com` project.**: Add the new feature to the
+<!-- markdownlint-disable MD044 -->
+- **Submit a MR to the [`www-gitlab-com`](https://gitlab.com/gitlab-com/www-gitlab-com) project.**: Add the new feature to the
   [EE features list](https://about.gitlab.com/features/).
+<!-- markdownlint-enable MD044 -->
 
 ## Act as CE when unlicensed
 
diff --git a/doc/development/emails.md b/doc/development/emails.md
index 811619bb0ff..5361282334f 100644
--- a/doc/development/emails.md
+++ b/doc/development/emails.md
@@ -10,8 +10,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 A Sidekiq job is enqueued whenever `deliver_later` is called on an `ActionMailer`.
 If a mailer argument needs to be added or removed, it is important to ensure
-both backward and forward compatibility. Adhere to the Sidekiq Style Guide steps for
-[changing the arguments for a worker](sidekiq_style_guide.md#changing-the-arguments-for-a-worker).
+both backward and forward compatibility. Adhere to the Sidekiq steps for
+[changing the arguments for a worker](sidekiq/compatibility_across_updates.md#changing-the-arguments-for-a-worker).
 
 In the following example from [`NotificationService`](https://gitlab.com/gitlab-org/gitlab/-/blob/33ccb22e4fc271dbaac94b003a7a1a2915a13441/app/services/notification_service.rb#L74)
 adding or removing an argument in this mailer's definition may cause problems
@@ -89,6 +89,8 @@ See the [Rails guides](https://guides.rubyonrails.org/action_mailer_basics.html#
 
    As mentioned, the part after `+` is ignored, and this message is sent to the mailbox for `gitlab-incoming@gmail.com`.
 
+1. Read the [MailRoom Gem updates](#mailroom-gem-updates) section for more details before you proceed to make sure you have the right version of MailRoom installed. In summary, you need to update the `gitlab-mail_room` version in the `Gemfile` to the latest `gitlab-mail_room` temporarily and run `bundle install`. **Do not commit** this change as it's a temporary workaround.
+
 1. Run this command in the GitLab root directory to launch `mail_room`:
 
    ```shell
diff --git a/doc/development/event_store.md b/doc/development/event_store.md
index 7f2b9c86d27..c6d553f41cd 100644
--- a/doc/development/event_store.md
+++ b/doc/development/event_store.md
@@ -185,7 +185,7 @@ Changes to the schema require multiple rollouts. While the new version is being
 - Events get persisted in the Sidekiq queue as job arguments, so we could have 2 versions of the schema during deployments.
 
 As changing the schema ultimately impacts the Sidekiq arguments, please refer to our
-[Sidekiq style guide](sidekiq_style_guide.md#changing-the-arguments-for-a-worker) with regards to multiple rollouts.
+[Sidekiq style guide](sidekiq/compatibility_across_updates.md#changing-the-arguments-for-a-worker) with regards to multiple rollouts.
 
 #### Add properties
 
@@ -232,7 +232,6 @@ the event safely via the `handle_event` method. For example:
 ```ruby
 module MergeRequests
   class UpdateHeadPipelineWorker
-    include ApplicationWorker
     include Gitlab::EventStore::Subscriber
 
     def handle_event(event)
@@ -252,19 +251,20 @@ add a line like this to the `Gitlab::EventStore.configure!` method:
 ```ruby
 module Gitlab
   module EventStore
-    def self.configure!
-      Store.new.tap do |store|
-        # ...
+    def self.configure!(store)
+      # ...
 
-        store.subscribe ::MergeRequests::UpdateHeadPipelineWorker, to: ::Ci::PipelineCreatedEvent
+      store.subscribe ::MergeRequests::UpdateHeadPipelineWorker, to: ::Ci::PipelineCreatedEvent
 
-        # ...
-      end
+      # ...
     end
   end
 end
 ```
 
+A worker that is only defined in the EE codebase can subscribe to an event in the same way by 
+declaring the subscription in `ee/lib/ee/gitlab/event_store.rb`.
+
 Subscriptions are stored in memory when the Rails app is loaded and they are immediately frozen.
 It's not possible to modify subscriptions at runtime.
 
@@ -274,7 +274,7 @@ A subscription can specify a condition when to accept an event:
 
 ```ruby
 store.subscribe ::MergeRequests::UpdateHeadPipelineWorker,
-  to: ::Ci::PipelineCreatedEvent, 
+  to: ::Ci::PipelineCreatedEvent,
   if: -> (event) { event.data[:merge_request_id].present? }
 ```
 
diff --git a/doc/development/experiment_guide/gitlab_experiment.md b/doc/development/experiment_guide/gitlab_experiment.md
index 36d2a4f6fbf..369690ba86c 100644
--- a/doc/development/experiment_guide/gitlab_experiment.md
+++ b/doc/development/experiment_guide/gitlab_experiment.md
@@ -4,26 +4,24 @@ group: Adoption
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
-# Implementing an A/B/n experiment using GLEX
+# Implementing an A/B/n experiment
 
 ## Introduction
 
-`Gitlab::Experiment` (GLEX) is tightly coupled with the concepts provided by
-[Feature flags in development of GitLab](../feature_flags/index.md). Here, we refer
-to this layer as feature flags, and may also use the term Flipper, because we
-built our development and experiment feature flags atop it.
-
-You're strongly encouraged to read and understand the
-[Feature flags in development of GitLab](../feature_flags/index.md) portion of the
-documentation before considering running experiments. Experiments add additional
-concepts which may seem confusing or advanced without understanding the underpinnings
-of how GitLab uses feature flags in development. One concept: GLEX supports
-experiments with multiple variants, which are sometimes referred to as A/B/n tests.
-
-The [`gitlab-experiment` project](https://gitlab.com/gitlab-org/ruby/gems/gitlab-experiment)
-exists in a separate repository, so it can be shared across any GitLab property that uses
-Ruby. You should feel comfortable reading the documentation on that project as well
-if you want to dig into more advanced topics.
+Experiments in GitLab are tightly coupled with the concepts provided by
+[Feature flags in development of GitLab](../feature_flags/index.md). You're strongly encouraged
+to read and understand the [Feature flags in development of GitLab](../feature_flags/index.md)
+portion of the documentation before considering running experiments. Experiments add additional
+concepts which may seem confusing or advanced without understanding the underpinnings of how GitLab
+uses feature flags in development. One concept: experiments can be run with multiple variants,
+which are sometimes referred to as A/B/n tests.
+
+We use the [`gitlab-experiment` gem](https://gitlab.com/gitlab-org/ruby/gems/gitlab-experiment),
+sometimes referred to as GLEX, to run our experiments. The gem exists in a separate repository
+so it can be shared across any GitLab property that uses Ruby. You should feel comfortable reading
+the documentation on that project if you want to dig into more advanced topics or open issues. Be
+aware that the documentation there reflects what's in the main branch and may not be the same as
+the version being used within GitLab.
 
 ## Glossary of terms
 
@@ -35,41 +33,9 @@ when communicating about experiments:
 - `control`: The default, or "original" code path.
 - `candidate`: Defines an experiment with only one code path.
 - `variant(s)`: Defines an experiment with multiple code paths.
+- `behaviors`: Used to reference all possible code paths of an experiment, including the control.
 
-### How it works
-
-Use this decision tree diagram to understand how GLEX works. When an experiment runs,
-the following logic is executed to determine what variant should be provided,
-given how the experiment has been defined and using the provided context:
-
-```mermaid
-graph TD
-    GP[General Pool/Population] --> Running?
-    Running? -->|Yes| Cached?[Cached? / Pre-segmented?]
-    Running? -->|No| Excluded[Control / No Tracking]
-    Cached? -->|No| Excluded?
-    Cached? -->|Yes| Cached[Cached Value]
-    Excluded? -->|Yes| Excluded
-    Excluded? -->|No| Segmented?
-    Segmented? -->|Yes / Cached| VariantA
-    Segmented? -->|No| Included?[Experiment Group?]
-    Included? -->|Yes| Rollout
-    Included? -->|No| Control
-    Rollout -->|Cached| VariantA
-    Rollout -->|Cached| VariantB
-    Rollout -->|Cached| VariantC
-
-classDef included fill:#380d75,color:#ffffff,stroke:none
-classDef excluded fill:#fca121,stroke:none
-classDef cached fill:#2e2e2e,color:#ffffff,stroke:none
-classDef default fill:#fff,stroke:#6e49cb
-
-class VariantA,VariantB,VariantC included
-class Control,Excluded excluded
-class Cached cached
-```
-
-## Implement an experiment
+## Implementing an experiment
 
 [Examples](https://gitlab.com/gitlab-org/growth/growth/-/wikis/GLEX-Framework-code-examples)
 
@@ -87,9 +53,9 @@ experiment in code. An experiment implementation can be as simple as:
 
 ```ruby
 experiment(:pill_color, actor: current_user) do |e|
-  e.use { 'control' }
-  e.try(:red) { 'red' }
-  e.try(:blue) { 'blue' }
+  e.control { 'control' }
+  e.variant(:red) { 'red' }
+  e.variant(:blue) { 'blue' }
 end
 ```
 
@@ -146,11 +112,11 @@ We can also implement this experiment in a HAML file with HTML wrappings:
 ```haml
 #cta-interface
   - experiment(:pill_color, actor: current_user) do |e|
-    - e.use do
+    - e.control do
       .pill-button control
-    - e.try(:red) do
+    - e.variant(:red) do
       .pill-button.red red
-    - e.try(:blue) do
+    - e.variant(:blue) do
       .pill-button.blue blue
 ```
 
@@ -212,38 +178,30 @@ wouldn't be resolvable.
 
 ### Advanced experimentation
 
-GLEX allows for two general implementation styles:
+There are two ways to implement an experiment:
 
 1. The simple experiment style described previously.
-1. A more advanced style where an experiment class can be provided.
+1. A more advanced style where an experiment class is provided.
 
 The advanced style is handled by naming convention, and works similar to what you
 would expect in Rails.
 
 To generate a custom experiment class that can override the defaults in
-`ApplicationExperiment` (our base GLEX implementation), use the rails generator:
+`ApplicationExperiment` use the Rails generator:
 
 ```shell
 rails generate gitlab:experiment pill_color control red blue
 ```
 
 This generates an experiment class in `app/experiments/pill_color_experiment.rb`
-with the variants (or _behaviors_) we've provided to the generator. Here's an example
-of how that class would look after migrating the previous example into it:
+with the _behaviors_ we've provided to the generator. Here's an example
+of how that class would look after migrating our previous example into it:
 
 ```ruby
 class PillColorExperiment < ApplicationExperiment
-  def control_behavior
-    'control'
-  end
-
-  def red_behavior
-    'red'
-  end
-
-  def blue_behavior
-    'blue'
-  end
+  control { 'control' }
+  variant(:red) { 'red' }
+  variant(:blue) { 'blue' }
 end
 ```
 
@@ -254,13 +212,13 @@ providing the block we were initially providing, by explicitly calling `run`:
 experiment(:pill_color, actor: current_user).run
 ```
 
-The _behavior_ methods we defined in our experiment class represent the default
-implementation. You can still use the block syntax to override these _behavior_
-methods however, so the following would also be valid:
+The _behaviors_ we defined in our experiment class represent the default
+implementation. You can still use the block syntax to override these _behaviors_
+however, so the following would also be valid:
 
 ```ruby
 experiment(:pill_color, actor: current_user) do |e|
-  e.use { '<strong>control</strong>' }
+  e.control { '<strong>control</strong>' }
 end
 ```
 
@@ -279,11 +237,11 @@ variant, and any account older than 2 weeks old would be assigned the _blue_ var
 
 ```ruby
 class PillColorExperiment < ApplicationExperiment
+  # ...registered behaviors
+
   segment(variant: :red) { context.actor.first_name == 'Richard' }
   segment :old_account?, variant: :blue
 
-  # ...behaviors
-
   private
 
   def old_account?
@@ -315,9 +273,9 @@ be nothing - but no events are tracked in these cases as well.
 
 ```ruby
 class PillColorExperiment < ApplicationExperiment
-  exclude :old_account?, ->{ context.actor.first_name == 'Richard' }
+  # ...registered behaviors
 
-  # ...behaviors
+  exclude :old_account?, ->{ context.actor.first_name == 'Richard' }
 
   private
 
@@ -327,23 +285,11 @@ class PillColorExperiment < ApplicationExperiment
 end
 ```
 
-We can also do exclusion when we run the experiment. For instance,
-if we wanted to prevent the inclusion of non-administrators in an experiment, consider
-the following experiment. This type of logic enables us to do complex experiments
-while preventing us from passing things into our experiments, because
-we want to minimize passing things into our experiments:
-
-```ruby
-experiment(:pill_color, actor: current_user) do |e|
-  e.exclude! unless can?(current_user, :admin_project, project)
-end
-```
-
 You may also need to check exclusion in custom tracking logic by calling `should_track?`:
 
 ```ruby
 class PillColorExperiment < ApplicationExperiment
-  # ...behaviors
+  # ...registered behaviors
 
   def expensive_tracking_logic
     return unless should_track?
@@ -353,16 +299,11 @@ class PillColorExperiment < ApplicationExperiment
 end
 ```
 
-Exclusion rules aren't the best way to determine if an experiment is active. Override
-the `enabled?` method for a high-level way of determining if an experiment should
-run and track. Make the `enabled?` check as efficient as possible because it's the
-first early opt-out path an experiment can implement.
-
 ### Tracking events
 
 One of the most important aspects of experiments is gathering data and reporting on
-it. GLEX provides an interface that allows tracking events across an experiment.
-You can implement it consistently if you provide the same context between
+it. You can use the `track` method to track events across an experimental implementation.
+You can track events consistently to an experiment if you provide the same context between
 calls to your experiment. If you do not yet understand context, you should read
 about contexts now.
 
@@ -373,13 +314,13 @@ the arguments you would normally use when
 of tracking an event in Ruby would be:
 
 ```ruby
-experiment(:pill_color, actor: current_user).track(:created)
+experiment(:pill_color, actor: current_user).track(:clicked)
 ```
 
-When you run an experiment with any of these examples, an `:assigned` event
+When you run an experiment with any of the examples so far, an `:assigned` event
 is tracked automatically by default. All events that are tracked from an
 experiment have a special
-[experiment context](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_experiment/jsonschema/1-0-0)
+[experiment context](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_experiment/jsonschema/1-0-3)
 added to the event. This can be used - typically by the data team - to create a connection
 between the events on a given experiment.
 
@@ -390,28 +331,20 @@ event would be tracked at that time for them.
 
 NOTE:
 GitLab tries to be sensitive and respectful of our customers regarding tracking,
-so GLEX allows us to implement an experiment without ever tracking identifying
+so our experimentation library allows us to implement an experiment without ever tracking identifying
 IDs. It's not always possible, though, based on experiment reporting requirements.
 You may be asked from time to time to track a specific record ID in experiments.
 The approach is largely up to the PM and engineer creating the implementation.
 No recommendations are provided here at this time.
 
-## Test with RSpec
+## Testing with RSpec
 
-This gem provides some RSpec helpers and custom matchers. These are in flux as of GitLab 13.10.
-
-First, require the RSpec support file to mix in some of the basics:
-
-```ruby
-require 'gitlab/experiment/rspec'
-```
-
-You still need to include matchers and other aspects, which happens
-automatically for files in `spec/experiments`, but for other files and specs
-you want to include it in, you can specify the `:experiment` type:
+In the course of working with experiments, you'll probably want to utilize the RSpec
+tooling that's built in. This happens automatically for files in `spec/experiments`, but
+for other files and specs you want to include it in, you can specify the `:experiment` type:
 
 ```ruby
-it "tests", :experiment do
+it "tests experiments nicely", :experiment do
 end
 ```
 
@@ -421,28 +354,32 @@ You can stub experiments using `stub_experiments`. Pass it a hash using experime
 names as the keys, and the variants you want each to resolve to, as the values:
 
 ```ruby
-# Ensures the experiments named `:example` & `:example2` are both
-# "enabled" and that each will resolve to the given variant
-# (`:my_variant` & `:control` respectively).
+# Ensures the experiments named `:example` & `:example2` are both "enabled" and
+# that each will resolve to the given variant (`:my_variant` and `:control`
+# respectively).
 stub_experiments(example: :my_variant, example2: :control)
 
 experiment(:example) do |e|
   e.enabled? # => true
-  e.variant.name # => 'my_variant'
+  e.assigned.name # => 'my_variant'
 end
 
 experiment(:example2) do |e|
   e.enabled? # => true
-  e.variant.name # => 'control'
+  e.assigned.name # => 'control'
 end
 ```
 
-### Exclusion and segmentation matchers
+### Exclusion, segmentation, and behavior matchers
 
-You can also test the exclusion and segmentation matchers.
+You can also test things like the registered behaviors, the exclusions, and
+segmentations using the matchers.
 
 ```ruby
 class ExampleExperiment < ApplicationExperiment
+  control { }
+  candidate { '_candidate_' }
+    
   exclude { context.actor.first_name == 'Richard' }
   segment(variant: :candidate) { context.actor.username == 'jejacks0n' }
 end
@@ -450,6 +387,10 @@ end
 excluded = double(username: 'rdiggitty', first_name: 'Richard')
 segmented = double(username: 'jejacks0n', first_name: 'Jeremy')
 
+# register_behavior matcher
+expect(experiment(:example)).to register_behavior(:control)
+expect(experiment(:example)).to register_behavior(:candidate).with('_candidate_')
+
 # exclude matcher
 expect(experiment(:example)).to exclude(actor: excluded)
 expect(experiment(:example)).not_to exclude(actor: segmented)
@@ -495,35 +436,36 @@ expect(experiment(:example)).to track(:my_event, value: 1, property: '_property_
 experiment(:example, :variant_name, foo: :bar).track(:my_event, value: 1, property: '_property_')
 ```
 
-### Recording and assignment tracking
-
-To test assignment tracking and the `record!` method, you can use or adopt the following
-shared example: [tracks assignment and records the subject](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/support/shared_examples/lib/gitlab/experimentation_shared_examples.rb).
-
 ## Experiments in the client layer
 
-This is in flux as of GitLab 13.10, and can't be documented just yet.
+Any experiment that's been run in the request lifecycle surfaces in `window.gl.experiments`,
+and matches [this schema](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_experiment/jsonschema/1-0-3)
+so it can be used when resolving experimentation in the client layer.
 
-Any experiment that's been run in the request lifecycle surfaces in and `window.gl.experiments`,
-and matches [this schema](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_experiment/jsonschema/1-0-0)
-so you can use it when resolving some concepts around experimentation in the client layer.
+Given that we've defined a class for our experiment, and have defined the variants for it, we can publish that experiment in a couple ways.
 
-### Use experiments in Vue
+The first way is simply by running the experiment. Assuming the experiment has been run, it will surface in the client layer without having to do anything special.
 
-With the `gitlab-experiment` component, you can define slots that match the name of the
-variants pushed to `window.gl.experiments`. For example, if we alter the `pill_color`
-experiment to just use the default variants of `control` and `candidate` like so:
+The second way doesn't run the experiment and is intended to be used if the experiment only needs to surface in the client layer. To accomplish this we can simply `.publish` the experiment. This won't run any logic, but does surface the experiment details in the client layer so they can be utilized there.
+
+An example might be to publish an experiment in a `before_action` in a controller. Assuming we've defined the `PillColorExperiment` class, like we have above, we can surface it to the client by publishing it instead of running it: 
 
 ```ruby
-def show
-  experiment(:pill_color) do |e|
-    e.use { } # control
-    e.try { } # candidate
-  end.run
-end
+before_action -> { experiment(:pill_color).publish }, only: [:show]
 ```
 
-We can make use of the named slots `control` and `candidate` in the Vue component:
+You can then see this surface in the JavaScript console:
+
+```javascript
+window.gl.experiments // => { pill_color: { excluded: false, experiment: "pill_color", key: "ca63ac02", variant: "candidate" } } 
+```
+
+### Using experiments in Vue
+
+With the `gitlab-experiment` component, you can define slots that match the name of the
+variants pushed to `window.gl.experiments`.
+
+We can make use of the named slots in the Vue component, that match the behaviors defined in :
 
 ```vue
 <script>
@@ -540,23 +482,6 @@ export default {
       <button class="bg-default">Click default button</button>
     </template>
 
-    <template #candidate>
-      <button class="bg-red">Click red button</button>
-    </template>
-  </gitlab-experiment>
-</template>
-```
-
-When you're coding for an experiment with multiple variants, you can use the variant names.
-For example, the Vue component for the previously-defined `pill_color` experiment with `red` and `blue` variants would look like this:
-
-```vue
-<template>
-  <gitlab-experiment name="pill_color">
-    <template #control>
-      <button class="bg-default">Click default button</button>
-    </template>
-
     <template #red>
       <button class="bg-red">Click red button</button>
     </template>
@@ -596,7 +521,7 @@ NOTE:
 This method of stubbing in Jest specs will not automatically un-stub itself at the end of the test. We merge our stubbed experiment in with all the other global data in `window.gl`. If you need to remove the stubbed experiment(s) after your test or ensure a clean global object before your test, you'll need to manage the global object directly yourself:
 
 ```javascript
-desribe('tests that care about global state', () => {
+describe('tests that care about global state', () => {
   const originalObjects = [];
 
   beforeEach(() => {
@@ -628,7 +553,7 @@ is viewed as being either `on` or `off`, this isn't accurate for experiments.
 
 Generally, `off` means that when we ask if a feature flag is enabled, it will always
 return `false`, and `on` means that it will always return `true`. An interim state,
-considered `conditional`, also exists. GLEX takes advantage of this trinary state of
+considered `conditional`, also exists. We take advantage of this trinary state of
 feature flags. To understand this `conditional` aspect: consider that either of these
 settings puts a feature flag into this state:
 
@@ -638,7 +563,7 @@ settings puts a feature flag into this state:
 Conditional means that it returns `true` in some situations, but not all situations.
 
 When a feature flag is disabled (meaning the state is `off`), the experiment is
-considered _inactive_. You can visualize this in the [decision tree diagram](#how-it-works)
+considered _inactive_. You can visualize this in the [decision tree diagram](https://gitlab.com/gitlab-org/ruby/gems/gitlab-experiment#how-it-works)
 as reaching the first `Running?` node, and traversing the negative path.
 
 When a feature flag is rolled out to a `percentage_of_actors` or similar (meaning the
diff --git a/doc/development/experiment_guide/index.md b/doc/development/experiment_guide/index.md
index 8d62f92e0b9..254c136ef79 100644
--- a/doc/development/experiment_guide/index.md
+++ b/doc/development/experiment_guide/index.md
@@ -54,7 +54,7 @@ This uses [experiment](../feature_flags/index.md#experiment-type) feature flags.
 
 Some experiments may require you to add custom icons or illustrations to our codebase.
 This process is lengthy and at this stage, the outcome of the experiment uncertain.
-Therefore, you should postpone this effort until the [experiment cleanup process](https://about.gitlab.com/handbook/engineering/development/growth/#experiment-cleanup-issue).
+Therefore, you should postpone this effort until the [experiment cleanup process](https://about.gitlab.com/handbook/engineering/development/growth/experimentation/#experiment-cleanup-issue).
 
 We recommend the following workflow:
 
diff --git a/doc/development/export_csv.md b/doc/development/export_csv.md
index e2bbd0491d6..ff827023a50 100644
--- a/doc/development/export_csv.md
+++ b/doc/development/export_csv.md
@@ -11,8 +11,8 @@ This document lists the different implementations of CSV export in GitLab codeba
 | Export type | How it works | Advantages | Disadvantages | Existing examples |
 |---|---|---|---|---|
 | Streaming | - Query and yield data in batches to a response stream.<br>- Download starts immediately. | - Report available immediately. | - No progress indicator.<br>- Requires a reliable connection. | [Export Audit Event Log](../administration/audit_events.md#export-to-csv) |
-| Downloading | - Query and write data in batches to a temporary file.<br>- Loads the file into memory.<br>- Sends the file to the client. | - Report available immediately. | - Large amount of data might cause request timeout.<br>- Memory intensive.<br>- Request expires when user navigates to a different page. | [Export Chain of Custody Report](../user/compliance/compliance_report/#chain-of-custody-report) |
-| As email attachment | - Asynchronously process the query with background job.<br>- Email uses the export as an attachment. | - Asynchronous processing. | - Requires users use a different app (email) to download the CSV.<br>- Email providers may limit attachment size. | - [Export Issues](../user/project/issues/csv_export.md)<br>- [Export Merge Requests](../user/project/merge_requests/csv_export.md) |
+| Downloading | - Query and write data in batches to a temporary file.<br>- Loads the file into memory.<br>- Sends the file to the client. | - Report available immediately. | - Large amount of data might cause request timeout.<br>- Memory intensive.<br>- Request expires when user navigates to a different page. | - [Export Chain of Custody Report](../user/compliance/compliance_report/#chain-of-custody-report)<br>- [Export License Usage File](../subscriptions/self_managed/index.md#export-your-license-usage) |
+| As email attachment | - Asynchronously process the query with background job.<br>- Email uses the export as an attachment. | - Asynchronous processing. | - Requires users use a different app (email) to download the CSV.<br>- Email providers may limit attachment size. | - [Export issues](../user/project/issues/csv_export.md)<br>- [Export merge requests](../user/project/merge_requests/csv_export.md) |
 | As downloadable link in email (*) | - Asynchronously process the query with background job.<br>- Email uses an export link. | - Asynchronous processing.<br>- Bypasses email provider attachment size limit. | - Requires users use a different app (email).<br>- Requires additional storage and cleanup. | [Export User Permissions](https://gitlab.com/gitlab-org/gitlab/-/issues/1772) |
 | Polling (non-persistent state) | - Asynchronously processes the query with the background job.<br>- Frontend(FE) polls every few seconds to check if CSV file is ready. | - Asynchronous processing.<br>- Automatically downloads to local machine on completion.<br>- In-app solution. | - Non-persistable request - request expires when user navigates to a different page.<br>- API is processed for each polling request. | [Export Vulnerabilities](../user/application_security/vulnerability_report/#export-vulnerability-details) |
 | Polling (persistent state) (*) | - Asynchronously processes the query with background job.<br>- Backend (BE) maintains the export state<br>- FE polls every few seconds to check status.<br>- FE shows 'Download link' when export is ready.<br>- User can download or regenerate a new report. | - Asynchronous processing.<br>- No database calls made during the polling requests (HTTP 304 status is returned until export status changes).<br>- Does not require user to stay on page until export is complete.<br>- In-app solution.<br>- Can be expanded into a generic CSV feature (such as dashboard / CSV API). | - Requires to maintain export states in DB.<br>- Does not automatically download the CSV export to local machine, requires users to click 'Download' button. | [Export Merge Commits Report](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/43055) |
diff --git a/doc/development/fe_guide/design_anti_patterns.md b/doc/development/fe_guide/design_anti_patterns.md
index 9e602b1ea04..76825d6ff18 100644
--- a/doc/development/fe_guide/design_anti_patterns.md
+++ b/doc/development/fe_guide/design_anti_patterns.md
@@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 Anti-patterns may seem like good approaches at first, but it has been shown that they bring more ills than benefits. These should
 generally be avoided.
 
-Throughout the GitLab codebase, there may be historic uses of these anti-patterns. Please [use discretion](https://about.gitlab.com/handbook/engineering/#balance-refactoring-and-velocity)
+Throughout the GitLab codebase, there may be historic uses of these anti-patterns. Please [use discretion](https://about.gitlab.com/handbook/engineering/principles/#balance-refactoring-and-velocity)
 when figuring out whether or not to refactor, when touching code that uses one of these legacy patterns.
 
 NOTE:
diff --git a/doc/development/fe_guide/development_process.md b/doc/development/fe_guide/development_process.md
index cd82a7dadc3..9921b851344 100644
--- a/doc/development/fe_guide/development_process.md
+++ b/doc/development/fe_guide/development_process.md
@@ -64,6 +64,30 @@ Please use your best judgment when to use it and please contribute new points th
 - [ ] Follow up on issues that came out of the review. Create issues for discovered edge cases that should be covered in future iterations.
 ```
 
+### Code deletion checklist
+
+When your merge request deletes code, it's important to also delete all
+related code that is no longer used.
+When deleting Haml and Vue code, check whether it contains the following types of
+code that is unused:
+
+- CSS.
+
+  For example, we've deleted a Vue component that contained the `.mr-card` class, which is now unused.
+  The `.mr-card` CSS rule set should then be deleted from `merge_requests.scss`.
+
+- Ruby variables.
+
+  Deleting unused Ruby variables is important so we don't continue instantiating them with
+  potentially expensive code.
+
+  For example, we've deleted a Haml template that used the `@total_count` Ruby variable.
+  The `@total_count` variable was no longer used in the remaining templates for the page.
+  The instantiation of `@total_count` in `issues_controller.rb` should then be deleted so that we
+  don't make unnecessary database calls to calculate the count of issues.
+
+- Ruby methods.
+
 ### Merge Request Review
 
 With the purpose of being [respectful of others' time](https://about.gitlab.com/handbook/values/#be-respectful-of-others-time) please follow these guidelines when asking for a review:
diff --git a/doc/development/fe_guide/graphql.md b/doc/development/fe_guide/graphql.md
index ed71f612061..e79a473df9e 100644
--- a/doc/development/fe_guide/graphql.md
+++ b/doc/development/fe_guide/graphql.md
@@ -107,9 +107,9 @@ Default client accepts two parameters: `resolvers` and `config`.
 
 ### Multiple client queries for the same object
 
-If you are make multiple queries to the same Apollo client object you might encounter the following error: "Store error: the application attempted to write an object with no provided ID but the store already contains an ID of SomeEntity". [This error only should occur when you have made a query with an ID field for a portion, then made another that returns what would be the same object, but is missing the ID field.](https://github.com/apollographql/apollo-client/issues/2510#issue-271829009)
+If you are making multiple queries to the same Apollo client object you might encounter the following error: `Cache data may be lost when replacing the someProperty field of a Query object. To address this problem, either ensure all objects of SomeEntityhave an id or a custom merge function`. We are already checking `ID` presence for every GraphQL type that has an `ID`, so this shouldn't be the case. Most likely, the `SomeEntity` type doesn't have an `ID` property, and to fix this warning we need to define a custom merge function.
 
-This is being tracked in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/326101) and the documentation will be updated when this issue is resolved.
+We have some client-wide types with `merge: true` defined in the default client as [typePolicies](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/lib/graphql.js) (this means that Apollo will merge existing and incoming responses in the case of subsequent queries). Please consider adding `SomeEntity` there or defining a custom merge function for it.
 
 ## GraphQL Queries
 
@@ -667,9 +667,7 @@ apollo: {
 ```
 
 When we want to move to the next page, we use an Apollo `fetchMore` method, passing a
-new cursor (and, optionally, new variables) there. In the `updateQuery` hook, we have
-to return a result we want to see in the Apollo cache after fetching the next page.
-[`Immer`s `produce`](#immutability-and-cache-updates)-function can help us with the immutability here:
+new cursor (and, optionally, new variables) there.
 
 ```javascript
 fetchNextPage(endCursor) {
@@ -679,24 +677,114 @@ fetchNextPage(endCursor) {
       first: 10,
       after: endCursor,
     },
-    updateQuery(previousResult, { fetchMoreResult }) {
-      // Here we can implement the logic of adding new designs to existing ones
-      // (for example, if we use infinite scroll) or replacing old result
-      // with the new one if we use numbered pages
-
-      const { designs: previousDesigns } = previousResult.project.issue.designCollection;
-      const { designs: newDesigns } = fetchMoreResult.project.issue.designCollection
-
-      return produce(previousResult, draftData => {
-        // `produce` gives us a working copy, `draftData`, that we can modify
-        // as we please and from it will produce the next immutable result for us
-        draftData.project.issue.designCollection.designs = [...previousDesigns, ...newDesigns];
-      });
-    },
   });
 }
 ```
 
+##### Defining field merge policy
+
+We would also need to define a field policy to specify how do we want to merge the existing results with the incoming results. For example, if we have `Previous/Next` buttons, it makes sense to replace the existing result with the incoming one:
+
+```javascript
+const apolloProvider = new VueApollo({
+  defaultClient: createDefaultClient(
+    {},
+    {
+      cacheConfig: {
+        typePolicies: {
+          DesignCollection: {
+            fields: {
+              designs: {
+                merge(existing, incoming) {
+                  if (!incoming) return existing;
+                  if (!existing) return incoming;
+
+                  // We want to save only incoming nodes and replace existing ones
+                  return incoming
+                }
+              }
+            }
+          }
+        }
+      },
+    },
+  ),
+});
+```
+
+When we have an infinite scroll, it would make sense to add the incoming `designs` nodes to existing ones instead of replacing. In this case, merge function would be slightly different:
+
+```javascript
+const apolloProvider = new VueApollo({
+  defaultClient: createDefaultClient(
+    {},
+    {
+      cacheConfig: {
+        typePolicies: {
+          DesignCollection: {
+            fields: {
+              designs: {
+                merge(existing, incoming) {
+                  if (!incoming) return existing;
+                  if (!existing) return incoming;
+
+                  const { nodes, ...rest } = incoming;
+                  // We only need to merge the nodes array.
+                  // The rest of the fields (pagination) should always be overwritten by incoming
+                  let result = rest;
+                  result.nodes = [...existing.nodes, ...nodes];
+                  return result;
+                }
+              }
+            }
+          }
+        }
+      },
+    },
+  ),
+});
+```
+
+`apollo-client` [provides](https://github.com/apollographql/apollo-client/blob/212b1e686359a3489b48d7e5d38a256312f81fde/src/utilities/policies/pagination.ts)
+a few field policies to be used with paginated queries. Here's another way to achieve infinite
+scroll pagination with the `concatPagination` policy:
+
+```javascript
+import { concatPagination } from '@apollo/client/utilities';
+import Vue from 'vue';
+import VueApollo from 'vue-apollo';
+import createDefaultClient from '~/lib/graphql';
+
+Vue.use(VueApollo);
+
+export default new VueApollo({
+  defaultClient: createDefaultClient(
+    {},
+    {
+      cacheConfig: {
+        typePolicies: {
+          Project: {
+            fields: {
+              dastSiteProfiles: {
+                keyArgs: ['fullPath'], // You might need to set the keyArgs option to enforce the cache's integrity
+              },
+            },
+          },
+          DastSiteProfileConnection: {
+            fields: {
+              nodes: concatPagination(),
+            },
+          },
+        },
+      },
+    },
+  ),
+});
+```
+
+This is similar to the `DesignCollection` example above as new page results are appended to the
+previous ones.
+
 #### Using a recursive query in components
 
 When it is necessary to fetch all paginated data initially an Apollo query can do the trick for us.
@@ -816,7 +904,7 @@ const data = store.readQuery({
 });
 ```
 
-Read more about the `@connection` directive in [Apollo's documentation](https://www.apollographql.com/docs/react/v2/caching/cache-interaction/#the-connection-directive).
+Read more about the `@connection` directive in [Apollo's documentation](https://www.apollographql.com/docs/react/caching/advanced-topics/#the-connection-directive).
 
 ### Managing performance
 
@@ -1017,22 +1105,13 @@ apollo: {
           issuableId: convertToGraphQLId(this.issuableClass, this.issuableId),
         };
       },
-      // Describe how subscription should update the query
-      updateQuery(prev, { subscriptionData }) {
-        if (prev && subscriptionData?.data?.issuableAssigneesUpdated) {
-          const data = produce(prev, (draftData) => {
-            draftData.workspace.issuable.assignees.nodes =
-              subscriptionData.data.issuableAssigneesUpdated.assignees.nodes;
-          });
-          return data;
-        }
-        return prev;
-      },
     },
   },
 },
 ```
 
+We would need also to define a field policy similarly like we do it for the [paginated queries](#defining-field-merge-policy)
+
 ### Best Practices
 
 #### When to use (and not use) `update` hook in mutations
@@ -1081,55 +1160,6 @@ If you use the RubyMine IDE, and have marked the `tmp` directory as
 `gitlab/tmp/tests/graphql`. This will allow the **JS GraphQL** plugin to
 automatically find and index the schema.
 
-#### Mocking response as component data
-
-<!-- vale gitlab.Spelling = NO -->
-
-With [Vue Test Utils](https://vue-test-utils.vuejs.org/) one can quickly test components that
-fetch GraphQL queries. The simplest way is to use `shallowMount` and then set
-the data on the component:
-
-<!-- vale gitlab.Spelling = YES -->
-
-```javascript
-it('tests apollo component', () => {
-  const vm = shallowMount(App);
-
-  vm.setData({
-    ...mockData
-  });
-});
-```
-
-#### Testing loading state
-
-To test how a component renders when results from the GraphQL API are still loading, mock a loading state into respective Apollo queries/mutations:
-
-```javascript
-  function createComponent({
-    loading = false,
-  } = {}) {
-    const $apollo = {
-      queries: {
-        designs: {
-          loading,
-        },
-      },
-    };
-
-    wrapper = shallowMount(Index, {
-      sync: false,
-      mocks: { $apollo }
-    });
-  }
-
-  it('renders loading icon', () => {
-  createComponent({ loading: true });
-
-  expect(wrapper.element).toMatchSnapshot();
-})
-```
-
 #### Testing Apollo components
 
 If we use `ApolloQuery` or `ApolloMutation` in our components, in order to test their functionality we need to add a stub first:
@@ -1197,11 +1227,9 @@ it('calls mutation on submitting form ', () => {
 });
 ```
 
-### Testing with mocked Apollo Client
-
-To test the logic of Apollo cache updates, we might want to mock an Apollo Client in our unit tests. We use [`mock-apollo-client`](https://www.npmjs.com/package/mock-apollo-client) library to mock Apollo client and [`createMockApollo` helper](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/frontend/__helpers__/mock_apollo_helper.js) we created on top of it.
+#### Mocking Apollo Client
 
-To separate tests with mocked client from 'usual' unit tests, create an additional factory and pass the created `mockApollo` as an option to the `createComponent`-factory. This way we only create Apollo Client instance when it's necessary.
+To test the components with Apollo operations, we need to mock an Apollo Client in our unit tests. We use [`mock-apollo-client`](https://www.npmjs.com/package/mock-apollo-client) library to mock Apollo client and [`createMockApollo` helper](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/frontend/__helpers__/mock_apollo_helper.js) we created on top of it.
 
 We need to inject `VueApollo` into the Vue instance by calling `Vue.use(VueApollo)`. This will install `VueApollo` globally for all the tests in the file. It is recommended to call `Vue.use(VueApollo)` just after the imports.
 
@@ -1320,8 +1348,7 @@ it('renders designs list', async () => {
   const mockApollo = createMockApolloProvider();
   const wrapper = createComponent({ mockApollo });
 
-  jest.runOnlyPendingTimers();
-  await wrapper.vm.$nextTick();
+  await waitForPromises()
 
   expect(findDesigns()).toHaveLength(3);
 });
@@ -1342,8 +1369,7 @@ function createMockApolloProvider() {
 it('renders error if query fails', async () => {
   const wrapper = createComponent();
 
-  jest.runOnlyPendingTimers();
-  await wrapper.vm.$nextTick();
+  await waitForPromises()
 
   expect(wrapper.find('.test-error').exists()).toBe(true)
 })
@@ -1351,7 +1377,7 @@ it('renders error if query fails', async () => {
 
 Request handlers can also be passed to component factory as a parameter.
 
-Mutations could be tested the same way with a few additional `nextTick`s to get the updated result:
+Mutations could be tested the same way:
 
 ```javascript
 function createMockApolloProvider({
@@ -1391,7 +1417,7 @@ it('calls a mutation with correct parameters and reorders designs', async () =>
 
   expect(moveDesignHandler).toHaveBeenCalled();
 
-  await wrapper.vm.$nextTick();
+  await waitForPromises();
 
   expect(
     findDesigns()
@@ -1407,8 +1433,7 @@ To mock multiple query response states, success and failure, Apollo Client's nat
 describe('when query times out', () => {
   const advanceApolloTimers = async () => {
     jest.runOnlyPendingTimers();
-    await wrapper.vm.$nextTick();
-    await wrapper.vm.$nextTick();
+    await waitForPromises()
   };
 
   beforeEach(async () => {
@@ -1419,7 +1444,7 @@ describe('when query times out', () => {
       .mockResolvedValueOnce({ errors: [{ message: 'timeout' }] });
 
     createComponentWithApollo(failSucceedFail);
-    await wrapper.vm.$nextTick();
+    await waitForPromises();
   });
 
   it('shows correct errors and does not overwrite populated data when data is empty', async () => {
@@ -1862,6 +1887,16 @@ relative to `app/graphql/queries` folder: for example, if we need a
 `app/graphql/queries/repository/files.query.graphql` query, the path is
 `repository/files`.
 
+## Troubleshooting
+
+### Mocked client returns empty objects instead of mock response
+
+If your unit test is failing because response contains empty objects instead of mock data, you would need to add `__typename` field to the mocked response. This happens because mocked client (unlike the real one) does not populate the response with typenames and in some cases we need to do it manually so the client is able to recognize a GraphQL type.
+
+### Warning about losing cache data
+
+Sometimes you can see a warning in the console: `Cache data may be lost when replacing the someProperty field of a Query object. To address this problem, either ensure all objects of SomeEntityhave an id or a custom merge function`. Please check section about [multiple queries](#multiple-client-queries-for-the-same-object) to resolve an issue.
+
   ```yaml
   - current_route_path = request.fullpath.match(/-\/tree\/[^\/]+\/(.+$)/).to_a[1]
   - add_page_startup_graphql_call('repository/path_last_commit', { projectPath: @project.full_path, ref: current_ref, path: current_route_path || "" })
diff --git a/doc/development/fe_guide/style/javascript.md b/doc/development/fe_guide/style/javascript.md
index f22e3ea6395..4a0923ebe19 100644
--- a/doc/development/fe_guide/style/javascript.md
+++ b/doc/development/fe_guide/style/javascript.md
@@ -136,24 +136,40 @@ the class name with `js-`.
 
 ## ES Module Syntax
 
-Use ES module syntax to import modules:
+For most JavaScript files, use ES module syntax to import or export from modules. 
+Prefer named exports, as they improve name consistency.
 
 ```javascript
-// bad
-const SomeClass = require('some_class');
+// bad (with exceptions, see below)
+export default SomeClass;
+import SomeClass from 'file';
 
 // good
-import SomeClass from 'some_class';
+export { SomeClass };
+import { SomeClass } from 'file';
+```
+
+Using default exports is acceptable in a few particular circumstances:
+
+- Vue Single File Components (SFCs)
+- Vuex mutation files
+
+For more information, see [RFC 20](https://gitlab.com/gitlab-org/frontend/rfcs/-/issues/20).
 
+## CommonJS Module Syntax
+
+Our Node configuration requires CommonJS module syntax. Prefer named exports.
+
+```javascript
 // bad
 module.exports = SomeClass;
+const SomeClass = require('./some_class');
 
 // good
-export default SomeClass;
+module.exports = { SomeClass };
+const { SomeClass } = require('./some_class');
 ```
 
-We still use `require` in `scripts/` and `config/` files.
-
 ## Absolute vs relative paths for modules
 
 Use relative paths if the module you are importing is less than two levels up.
diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md
index 820b4bab802..af402713f6e 100644
--- a/doc/development/feature_flags/index.md
+++ b/doc/development/feature_flags/index.md
@@ -169,7 +169,7 @@ Each feature flag is defined in a separate YAML file consisting of a number of f
 | `name`              | yes      | Name of the feature flag.                                      |
 | `type`              | yes      | Type of feature flag.                                          |
 | `default_enabled`   | yes      | The default state of the feature flag that is strictly validated, with `default_enabled:` passed as an argument. |
-| `introduced_by_url` | no       | The URL to the Merge Request that introduced the feature flag. |
+| `introduced_by_url` | no       | The URL to the merge request that introduced the feature flag. |
 | `rollout_issue_url` | no       | The URL to the Issue covering the feature flag rollout.        |
 | `milestone`         | no       | Milestone in which the feature was added.                      |
 | `group`             | no       | The [group](https://about.gitlab.com/handbook/product/categories/#devops-stages) that owns the feature flag. |
@@ -513,7 +513,7 @@ Feature.remove(:feature_flag_name)
 
 - Any change behind a feature flag **disabled** by default **should not** have a changelog entry.
   - **Exception:** database migrations **should** have a changelog entry.
-- Any change related to a feature flag itself (flag removal, default-on setting) **should** have a changelog entry.
+- Any change related to a feature flag itself (flag removal, default-on setting) **should** have [a changelog entry](../changelog.md).
   Use the flowchart to determine the changelog entry type.
 
   ```mermaid
diff --git a/doc/development/features_inside_dot_gitlab.md b/doc/development/features_inside_dot_gitlab.md
index 73ba9cbd674..283a0d5d5fb 100644
--- a/doc/development/features_inside_dot_gitlab.md
+++ b/doc/development/features_inside_dot_gitlab.md
@@ -11,7 +11,7 @@ When implementing new features, please refer to these existing features to avoid
 
 - [Custom Dashboards](../operations/metrics/dashboards/index.md#add-a-new-dashboard-to-your-project): `.gitlab/dashboards/`.
 - [Issue Templates](../user/project/description_templates.md#create-an-issue-template): `.gitlab/issue_templates/`.
-- [Merge Request Templates](../user/project/description_templates.md#create-a-merge-request-template): `.gitlab/merge_request_templates/`.
+- [Merge request Templates](../user/project/description_templates.md#create-a-merge-request-template): `.gitlab/merge_request_templates/`.
 - [GitLab Agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/configuration_repository.md#layout): `.gitlab/agents/`.
 - [CODEOWNERS](../user/project/code_owners.md#set-up-code-owners): `.gitlab/CODEOWNERS`.
 - [Route Maps](../ci/review_apps/#route-maps): `.gitlab/route-map.yml`.
diff --git a/doc/development/img/architecture_simplified.png b/doc/development/img/architecture_simplified.png
index bd731758ddd..bab673feb4a 100644
--- a/doc/development/img/architecture_simplified.png
+++ b/doc/development/img/architecture_simplified.png
diff --git a/doc/development/import_project.md b/doc/development/import_project.md
index 9e236b4cfce..86e6e04347c 100644
--- a/doc/development/import_project.md
+++ b/doc/development/import_project.md
@@ -125,6 +125,14 @@ it fails with this error as `/` is not a valid character in a project name.
 
 A project with that name already exists.
 
+##### `Exception: Error importing repository into (namespace) - No space left on device`
+
+The disk has insufficient space to complete the import. 
+
+During import, the tarball is cached in your configured `shared_path` directory. Verify the
+disk has enough free space to accommodate both the cached tarball and the unpacked
+project files on disk. 
+
 ### Importing via the Rails console
 
 The last option is to import a project using a Rails console:
diff --git a/doc/development/index.md b/doc/development/index.md
index 197c7f48398..0501f70b818 100644
--- a/doc/development/index.md
+++ b/doc/development/index.md
@@ -221,10 +221,10 @@ the [reviewer values](https://about.gitlab.com/handbook/engineering/workflow/rev
 - [Geo development](geo.md)
 - [Redis guidelines](redis.md)
   - [Adding a new Redis instance](redis/new_redis_instance.md)
-- [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers
+- [Sidekiq guidelines](sidekiq/index.md) for working with Sidekiq workers
 - [Working with Gitaly](gitaly.md)
 - [Elasticsearch integration docs](elasticsearch.md)
-- [Working with Merge Request diffs](diffs.md)
+- [Working with merge request diffs](diffs.md)
 - [Approval Rules](approval_rules.md)
 - [Repository mirroring](repository_mirroring.md)
 - [File uploads](uploads.md)
diff --git a/doc/development/integrations/img/copy_cookies.png b/doc/development/integrations/img/copy_cookies.png
deleted file mode 100644
index 21575987173..00000000000
--- a/doc/development/integrations/img/copy_cookies.png
+++ /dev/null
diff --git a/doc/development/integrations/img/copy_curl.png b/doc/development/integrations/img/copy_curl.png
deleted file mode 100644
index 9fa871efbd5..00000000000
--- a/doc/development/integrations/img/copy_curl.png
+++ /dev/null
diff --git a/doc/development/integrations/jira_connect.md b/doc/development/integrations/jira_connect.md
index fc7204fdd5a..cfa1fdba699 100644
--- a/doc/development/integrations/jira_connect.md
+++ b/doc/development/integrations/jira_connect.md
@@ -65,27 +65,3 @@ If the app install failed, you might need to delete `jira_connect_installations`
 
 1. Open the [database console](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/postgresql.md#access-postgresql).
 1. Run `TRUNCATE TABLE jira_connect_installations CASCADE;`.
-
-## Add a namespace
-
-To add a [namespace](../../user/group/index.md#namespaces) to Jira:
-
-1. Make sure you are logged in on your GitLab development instance.
-1. On the GitLab app page in Jira, click **Get started**.
-1. Open your browser's developer tools and navigate to the **Network** tab.
-1. Try to add the namespace in Jira.
-1. If the request fails with 401 "not authorized", copy the request as a cURL command
-   and paste it in your terminal.
-
-   ![Example Vulnerability](img/copy_curl.png)
-
-1. Go to your development instance (usually at: <http://localhost:3000>), open developer
-   tools, navigate to the Network tab and reload the page.
-1. Copy all cookies from the first request.
-
-   ![Example Vulnerability](img/copy_cookies.png)
-
-1. Append the cookies to the cURL command in your terminal:
-   `--cookies "<cookies from the request>"`.
-1. Submit the cURL request.
-1. If the response is `{"success":true}`, the namespace was added.
diff --git a/doc/development/integrations/secure.md b/doc/development/integrations/secure.md
index 147d379cec7..27a166aebf9 100644
--- a/doc/development/integrations/secure.md
+++ b/doc/development/integrations/secure.md
@@ -10,7 +10,7 @@ Integrating a security scanner into GitLab consists of providing end users
 with a [CI job definition](../../ci/yaml/index.md)
 they can add to their CI configuration files to scan their GitLab projects.
 This CI job should then output its results in a GitLab-specified format. These results are then
-automatically presented in various places in GitLab, such as the Pipeline view, Merge Request
+automatically presented in various places in GitLab, such as the Pipeline view, merge request
 widget, and Security Dashboard.
 
 The scanning job is usually based on a [Docker image](https://docs.docker.com/)
diff --git a/doc/development/internal_api/index.md b/doc/development/internal_api/index.md
index 96910892022..db978253747 100644
--- a/doc/development/internal_api/index.md
+++ b/doc/development/internal_api/index.md
@@ -507,7 +507,7 @@ curl --request POST --header "Gitlab-Kas-Api-Request: <JWT token>" \
 
 Called from the GitLab Agent Server (`kas`) to create a security vulnerability
 from a Starboard vulnerability report. This request is idempotent. Multiple requests with the same data
-create a single vulnerability.
+create a single vulnerability. The response contains the UUID of the created vulnerability finding.
 
 | Attribute       | Type   | Required | Description |
 |:----------------|:-------|:---------|:------------|
@@ -553,6 +553,37 @@ curl --request PUT --header "Gitlab-Kas-Api-Request: <JWT token>" \
 }'
 ```
 
+Example response:
+
+```json
+{
+  "uuid": "4773b2ee-5ba5-5e9f-b48c-5f7a17f0faac"
+}
+```
+
+### Resolve Starboard vulnerabilities
+
+Called from the GitLab Agent Server (`kas`) to resolve Starboard security vulnerabilities.
+Accepts a list of finding UUIDs and marks all Starboard vulnerabilities not identified by
+the list as resolved.
+
+| Attribute | Type         | Required | Description                                                                                                                       |
+|:----------|:-------------|:---------|:----------------------------------------------------------------------------------------------------------------------------------|
+| `uuids`   | string array | yes      | UUIDs of detected vulnerabilities, as collected from [Create Starboard vulnerability](#create-starboard-vulnerability) responses. |
+
+```plaintext
+POST internal/kubernetes/modules/starboard_vulnerability/scan_result
+```
+
+Example Request:
+
+```shell
+curl --request POST --header "Gitlab-Kas-Api-Request: <JWT token>" \
+     --header "Authorization: Bearer <agent token>" --header "Content-Type: application/json" \
+     --url "http://localhost:3000/api/v4/internal/kubernetes/modules/starboard_vulnerability/scan_result" \
+     --data '{ "uuids": ["102e8a0a-fe29-59bd-b46c-57c3e9bc6411", "5eb12985-0ed5-51f4-b545-fd8871dc2870"] }'
+```
+
 ## Subscriptions
 
 The subscriptions endpoint is used by [CustomersDot](https://gitlab.com/gitlab-org/customers-gitlab-com) (`customers.gitlab.com`)
diff --git a/doc/development/issuable-like-models.md b/doc/development/issuable-like-models.md
index b3be2b1b2e6..42dfe2e0f2f 100644
--- a/doc/development/issuable-like-models.md
+++ b/doc/development/issuable-like-models.md
@@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 GitLab Rails codebase contains several models that hold common functionality and behave similarly to
 [Issues](../user/project/issues/index.md). Other examples of "issuables"
-are [Merge Requests](../user/project/merge_requests/index.md) and
+are [merge requests](../user/project/merge_requests/index.md) and
 [Epics](../user/group/epics/index.md).
 
 This guide accumulates guidelines on working with such Rails models.
diff --git a/doc/development/issue_types.md b/doc/development/issue_types.md
index 31fa50e1d97..e6047c62827 100644
--- a/doc/development/issue_types.md
+++ b/doc/development/issue_types.md
@@ -11,7 +11,7 @@ We are deprecating Issue Types as of GitLab 14.2 in favor of [Work Items and Wor
 
 Sometimes when a new resource type is added it's not clear if it should be only an
 "extension" of Issue (Issue Type) or if it should be a new first-class resource type
-(similar to Issue, Epic, Merge Request, Snippet).
+(similar to issue, epic, merge request, snippet).
 
 The idea of Issue Types was first proposed in [this
 issue](https://gitlab.com/gitlab-org/gitlab/-/issues/8767) and its usage was
diff --git a/doc/development/iterating_tables_in_batches.md b/doc/development/iterating_tables_in_batches.md
index 9ea7fd3dc29..38cdbdf5b79 100644
--- a/doc/development/iterating_tables_in_batches.md
+++ b/doc/development/iterating_tables_in_batches.md
@@ -142,7 +142,7 @@ database query:
 SELECT "users"."id" FROM "users" ORDER BY "users"."id" ASC LIMIT 1
 ```
 
-![Reading the start `id` value](img/each_batch_users_table_iteration_1_v13_7.png)
+![Reading the start ID value](img/each_batch_users_table_iteration_1_v13_7.png)
 
 Notice that the query only reads data from the index (`INDEX ONLY SCAN`), the table is not
 accessed. Database indexes are sorted so taking out the first item is a very cheap operation.
@@ -155,7 +155,7 @@ to get a "shifted" `id` value.
 SELECT "users"."id" FROM "users" WHERE "users"."id" >= 1 ORDER BY "users"."id" ASC LIMIT 1 OFFSET 5
 ```
 
-![Reading the end `id` value](img/each_batch_users_table_iteration_2_v13_7.png)
+![Reading the end ID value](img/each_batch_users_table_iteration_2_v13_7.png)
 
 Again, the query only looks into the index. The `OFFSET 5` takes out the sixth `id` value: this
 query reads a maximum of six items from the index regardless of the table size or the iteration
@@ -181,7 +181,7 @@ previous iteration in order to find out the next end `id` value.
 SELECT "users"."id" FROM "users" WHERE "users"."id" >= 302 ORDER BY "users"."id" ASC LIMIT 1 OFFSET 5
 ```
 
-![Reading the second end `id` value](img/each_batch_users_table_iteration_4_v13_7.png)
+![Reading the second end ID value](img/each_batch_users_table_iteration_4_v13_7.png)
 
 Now we can easily construct the `users` query for the second iteration.
 
diff --git a/doc/development/jh_features_review.md b/doc/development/jh_features_review.md
index 858048c1e7c..88830a80bf1 100644
--- a/doc/development/jh_features_review.md
+++ b/doc/development/jh_features_review.md
@@ -1,7 +1,7 @@
 ---
 stage: none
 group: unassigned
-info: https://gitlab.com/gitlab-jh/gitlab
+info: https://jihulab.com/gitlab-cn/gitlab
 ---
 
 # Guidelines for reviewing JiHu (JH) Edition related merge requests
@@ -21,13 +21,13 @@ We have two kinds of changes related to JH:
   - We will generalize this so both EE and JH can share the same mechanism,
     then we wouldn't have to treat them differently.
 
-If needed, review the corresponding JH merge request located at [JH repository](https://gitlab.com/gitlab-jh/gitlab)
+If needed, review the corresponding JH merge request located at [JH repository](https://jihulab.com/gitlab-cn/gitlab).
 
 ## When to merge files to the GitLab Inc. repository
 
-Files that are added to the `gitlab-jh` repository outside of `jh/` must be mirrored in the GitLab Inc. repository.
+Files that are added to the GitLab JH repository outside of `jh/` must be mirrored in the GitLab Inc. repository.
 
-If code that is added to the GitLab Inc. repository references (for example, `render_if_exists`) any `gitlab-jh` file that does not
+If code that is added to the GitLab Inc. repository references (for example, `render_if_exists`) any GitLab JH file that does not
 exist in the GitLab Inc. codebase, add a comment with a link to the JiHu merge request or file. This is to prevent
 the reference from being misidentified as a missing partial and subsequently deleted in the `gitlab` codebase.
 
@@ -49,7 +49,7 @@ This page is the single source of truth for JiHu-related processes.
 ## CI pipelines in a JH context
 
 EE repository does not have `jh/` directory therefore there is no way to run
-JH pipelines in the EE repository. All JH tests should go to [JH repository](https://gitlab.com/gitlab-jh/gitlab).
+JH pipelines in the EE repository. All JH tests should go to [JH repository](https://jihulab.com/gitlab-cn/gitlab).
 
 The top-level JH CI configuration is located at `jh/.gitlab-ci.yml` (which
 does not exist in EE repository) and it'll include EE CI configurations
@@ -89,7 +89,7 @@ Do not use methods such as `prepend`, `extend`, and `include`. Instead, use
 the relevant EE and JH modules by the name of the receiver module.
 
 If reviewing the corresponding JH file is needed, it should be found at
-[JH repository](https://gitlab.com/gitlab-jh/gitlab).
+[JH repository](https://jihulab.com/gitlab-cn/gitlab).
 
 ### General guidance for writing JH extensions
 
diff --git a/doc/development/logging.md b/doc/development/logging.md
index a4eda6ad02e..6a0b50d6970 100644
--- a/doc/development/logging.md
+++ b/doc/development/logging.md
@@ -1,6 +1,6 @@
 ---
 stage: Monitor
-group: Monitor
+group: Respond
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
@@ -398,3 +398,14 @@ end
 
 1. Be sure to update the [GitLab CE/EE documentation](../administration/logs.md) and the [GitLab.com
    runbooks](https://gitlab.com/gitlab-com/runbooks/blob/master/docs/logging/README.md).
+
+## Control logging visibility
+
+An increase in the logs can cause a growing backlog of unacknowledged messages. When adding new log messages, make sure they don't increase the overall volume of logging by more than 10%.
+
+### Deprecation notices
+
+If the expected volume of deprecation notices is large:
+
+- Only log them in the development environment.
+- If needed, log them in the testing environment.
diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md
index 106db862122..c8e99e8547f 100644
--- a/doc/development/merge_request_performance_guidelines.md
+++ b/doc/development/merge_request_performance_guidelines.md
@@ -255,15 +255,15 @@ It re-instantiates project object for each build, instead of using the same in-m
 In this particular case the workaround is fairly easy:
 
 ```ruby
+ActiveRecord::Associations::Preloader.new.preload(pipeline, [builds: :project])
+
 pipeline.builds.each do |build|
-  build.project = pipeline.project
   build.to_json(only: [:name], include: [project: { only: [:name]}])
 end
 ```
 
-We can assign `pipeline.project` to each `build.project`, since we know it should point to the same project.
-This allows us that each build point to the same in-memory project,
-avoiding the cached SQL query and re-instantiation of the project object for each build.
+`ActiveRecord::Associations::Preloader` uses the same in-memory object for the same project.
+This avoids the cached SQL query and also avoids re-instantiation of the project object for each build.
 
 ## Executing Queries in Loops
 
diff --git a/doc/development/multi_version_compatibility.md b/doc/development/multi_version_compatibility.md
index 27c4edf15f4..bdab92f5185 100644
--- a/doc/development/multi_version_compatibility.md
+++ b/doc/development/multi_version_compatibility.md
@@ -14,14 +14,14 @@ In a sense, these scenarios are all transient states. But they can often persist
 
 ### When modifying a Sidekiq worker
 
-For example when [changing arguments](sidekiq_style_guide.md#changing-the-arguments-for-a-worker):
+For example when [changing arguments](sidekiq/compatibility_across_updates.md#changing-the-arguments-for-a-worker):
 
 - Is it ok if jobs are being enqueued with the old signature but executed by the new monthly release?
 - Is it ok if jobs are being enqueued with the new signature but executed by the previous monthly release?
 
 ### When adding a new Sidekiq worker
 
-Is it ok if these jobs don't get executed for several hours because [Sidekiq nodes are not yet updated](sidekiq_style_guide.md#adding-new-workers)?
+Is it ok if these jobs don't get executed for several hours because [Sidekiq nodes are not yet updated](sidekiq/compatibility_across_updates.md#adding-new-workers)?
 
 ### When modifying JavaScript
 
@@ -89,7 +89,7 @@ Many users [skip some monthly releases](../update/index.md#upgrading-to-a-new-ma
 
 - 13.0 => 13.12
 
-These users accept some downtime during the update. Unfortunately we can't ignore this case completely. For example, 13.12 may execute Sidekiq jobs from 13.0, which illustrates why [we avoid removing arguments from jobs until a major release](sidekiq_style_guide.md#deprecate-and-remove-an-argument). The main question is: Will the deployment get to a good state after the update is complete?
+These users accept some downtime during the update. Unfortunately we can't ignore this case completely. For example, 13.12 may execute Sidekiq jobs from 13.0, which illustrates why [we avoid removing arguments from jobs until a major release](sidekiq/compatibility_across_updates.md#deprecate-and-remove-an-argument). The main question is: Will the deployment get to a good state after the update is complete?
 
 ## What kind of components can GitLab be broken down into?
 
@@ -180,7 +180,7 @@ coexists in production.
 
 ### Changing Sidekiq worker's parameters
 
-This topic is explained in detail in [Sidekiq Compatibility across Updates](sidekiq_style_guide.md#sidekiq-compatibility-across-updates).
+This topic is explained in detail in [Sidekiq Compatibility across Updates](sidekiq/compatibility_across_updates.md).
 
 When we need to add a new parameter to a Sidekiq worker class, we can split this into the following steps:
 
diff --git a/doc/development/new_fe_guide/modules/widget_extensions.md b/doc/development/new_fe_guide/modules/widget_extensions.md
index b833ba7c630..37712cb2cec 100644
--- a/doc/development/new_fe_guide/modules/widget_extensions.md
+++ b/doc/development/new_fe_guide/modules/widget_extensions.md
@@ -1,6 +1,6 @@
 ---
 stage: Create
-group: Source Code
+group: Code Review
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
@@ -11,27 +11,39 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 ## Summary
 
 Extensions in the merge request widget enable you to add new features
-into the widget that match the existing design and interaction as other extensions.
+into the merge request widget that match the design framework.
+With extensions we get a lot of benefits out of the box without much effort required, like:
+
+- A consistent look and feel.
+- Tracking when the extension is opened.
+- Virtual scrolling for performance.
 
 ## Usage
 
-To use extensions you need to first create a new extension object to fetch the
-data to render in the extension. See the example file in
-`app/assets/javascripts/vue_merge_request_widget/extensions/issues.js` for a working example.
+To use extensions you must first create a new extension object to fetch the
+data to render in the extension. For a working example, refer to the example file in
+`app/assets/javascripts/vue_merge_request_widget/extensions/issues.js`.
 
-The basic object structure is as below:
+The basic object structure:
 
 ```javascript
 export default {
-  name: '',
-  props: [],
+  name: '',       // Required: This helps identify the widget
+  props: [],      // Required: Props passed from the widget state
+  i18n: {         // Required: Object to hold i18n text
+    label: '',    // Required: Used for tooltips and aria-labels
+    loading: '',  // Required: Loading text for when data is loading
+  },
+  expandEvent: '',      // Optional: RedisHLL event name to track expanding content
+  enablePolling: false, // Optional: Tells extension to poll for data
   computed: {
-    summary() {},
-    statusIcon() {},
+    summary(data) {},     // Required: Level 1 summary text
+    statusIcon(data) {},  // Required: Level 1 status icon
+    tertiaryButtons() {}, // Optional: Level 1 action buttons
   },
   methods: {
-    fetchCollapsedData() {},
-    fetchFullData() {},
+    fetchCollapsedData(props) {}, // Required: Fetches data required for collapsed state
+    fetchFullData(props) {},      // Required: Fetches data for the full expanded content
   },
 };
 ```
@@ -39,10 +51,8 @@ export default {
 By following the same data structure, each extension can follow the same registering structure,
 but each extension can manage its data sources.
 
-After creating this structure you need to register it. Registering the extension can happen at any
-point _after_ the widget has been created.
-
-To register a extension the following can be done:
+After creating this structure, you must register it. You can register the extension at any
+point _after_ the widget has been created. To register a extension:
 
 ```javascript
 // Import the register method
@@ -55,18 +65,174 @@ import issueExtension from '~/vue_merge_request_widget/extensions/issues';
 registerExtension(issueExtension);
 ```
 
-## Fetching errors
+## Data fetching
+
+Each extension must fetch data. Fetching is handled when registering the extension,
+not by the core component itself. This approach allows for various different
+data fetching methods to be used, such as GraphQL or REST API calls.
+
+### API calls
+
+For performance reasons, it is best if the collapsed state fetches only the data required to
+render the collapsed state. This fetching happens within the `fetchCollapsedData` method.
+This method is called with the props as an argument, so you can easily access
+any paths set in the state.
+
+To allow the extension to set the data, this method **must** return the data. No
+special formatting is required. When the extension receives this data,
+it is set to `collapsedData`. You can access `collapsedData` in any computed property or
+method.
+
+When the user clicks **Expand**, the `fetchFullData` method is called. This method
+also gets called with the props as an argument. This method **must** also return
+the full data. However, this data needs to be correctly formatted to match the format
+mentioned in the data structure section.
+
+#### Technical debt
+
+For some of the current extensions, there is no split in data fetching. All the data
+is fetched through the `fetchCollapsedData` method. While less performant,
+it allows for faster iteration.
+
+To handle this the `fetchFullData` returns the data set through
+the `fetchCollapsedData` method call. In these cases, the `fetchFullData` must
+return a promise:
+
+```javascript
+fetchCollapsedData() {
+  return ['Some data'];
+},
+fetchFullData() {
+  return Promise.resolve(this.collapsedData)
+},
+```
+
+### Data structure
+
+The data returned from `fetchFullData` must match the format below. This format
+allows the core component to render the data in a way that matches
+the design framework. Any text properties can use the styling placeholders
+mentioned below:
+
+```javascript
+{
+  id: data.id,    // Required: ID used as a key for each row
+  header: 'Header' || ['Header', 'sub-header'], // Required: String or array can be used for the header text
+  text: '',       // Required: Main text for the row
+  subtext: '',    // Optional: Smaller sub-text to be displayed below the main text
+  icon: {         // Optional: Icon object
+    name: EXTENSION_ICONS.success, // Required: The icon name for the row
+  },
+  badge: {        // Optional: Badge displayed after text
+    text: '',     // Required: Text to be displayed inside badge
+    variant: '',  // Optional: GitLab UI badge variant, defaults to info
+  },
+  actions: [],    // Optional: Action button for row
+}
+```
+
+### Polling
+
+To enable polling for an extension, an options flag must be present in the extension:
+
+```javascript
+export default {
+  //...
+  enablePolling: true
+};
+```
+
+This flag tells the base component we should poll the `fetchCollapsedData()`
+defined in the extension. Polling stops if the response has data, or if an error is present.
+
+When writing the logic for `fetchCollapsedData()`, a complete Axios response must be returned
+from the method. The polling utility needs data like polling headers to work correctly:
+
+```javascript
+export default {
+  //...
+  enablePolling: true
+  methods: {
+    fetchCollapsedData() {
+      return axios.get(this.reportPath)
+    },
+  },
+};
+```
+
+Most of the time the data returned from the extension's endpoint is not in the format
+the UI needs. We must format the data before setting the collapsed data in the base component.
+
+If the computed property `summary` can rely on `collapsedData`, you can format the data
+when `fetchFullData` is invoked:
+
+```javascript
+export default {
+  //...
+  enablePolling: true
+  methods: {
+    fetchCollapsedData() {
+      return axios.get(this.reportPath)
+    },
+     fetchFullData() {
+      return Promise.resolve(this.prepareReports());
+    },
+    // custom method
+    prepareReports() {
+      // unpack values from collapsedData
+      const { new_errors, existing_errors, resolved_errors } = this.collapsedData;
+
+      // perform data formatting
+
+      return [...newErrors, ...existingErrors, ...resolvedErrors]
+    }
+  },
+};
+```
+
+If the extension relies on `collapsedData` being formatted before invoking `fetchFullData()`,
+then `fetchCollapsedData()` must return the Axios response as well as the formatted data:
+
+```javascript
+export default {
+  //...
+  enablePolling: true
+  methods: {
+    fetchCollapsedData() {
+      return axios.get(this.reportPath).then(res => {
+        const formattedData = this.prepareReports(res.data)
+
+        return {
+          ...res,
+          data: formattedData,
+        }
+      })
+    },
+    // Custom method
+    prepareReports() {
+      // Unpack values from collapsedData
+      const { new_errors, existing_errors, resolved_errors } = this.collapsedData;
+
+      // Perform data formatting
+
+      return [...newErrors, ...existingErrors, ...resolvedErrors]
+    }
+  },
+};
+```
+
+### Errors
 
 If `fetchCollapsedData()` or `fetchFullData()` methods throw an error:
 
-- The loading state of the extension is updated to `LOADING_STATES.collapsedError` and `LOADING_STATES.expandedError`
-  respectively.
+- The loading state of the extension is updated to `LOADING_STATES.collapsedError`
+  and `LOADING_STATES.expandedError` respectively.
 - The extensions header displays an error icon and updates the text to be either:
   - The text defined in `$options.i18n.error`.
   - "Failed to load" if `$options.i18n.error` is not defined.
 - The error is sent to Sentry to log that it occurred.
 
-To customise the error text, you need to add it to the `i18n` object in your extension:
+To customise the error text, add it to the `i18n` object in your extension:
 
 ```javascript
 export default {
@@ -77,3 +243,77 @@ export default {
   },
 };
 ```
+
+## Icons
+
+Level 1 and all subsequent levels can have their own status icons. To keep with
+the design framework, import the `EXTENSION_ICONS` constant
+from the `constants.js` file:
+
+```javascript
+import { EXTENSION_ICONS } from '~/vue_merge_request_widget/constants.js';
+```
+
+This constant has the below icons available for use. Per the design framework,
+only some of these icons should be used on level 1:
+
+- `failed`
+- `warning`
+- `success`
+- `neutral`
+- `error`
+- `notice`
+- `severityCritical`
+- `severityHigh`
+- `severityMedium`
+- `severityLow`
+- `severityInfo`
+- `severityUnknown`
+
+## Text styling
+
+Any area that has text can be styled with the placeholders below. This
+technique follows the same technique as `sprintf`. However, instead of specifying
+these through `sprintf`, the extension does this automatically.
+
+Every placeholder contains starting and ending tags. For example, `success` uses
+`Hello %{success_start}world%{success_end}`. The extension then
+adds the start and end tags with the correct styling classes.
+
+| Placeholder | Style |
+|---|---|
+| success | `gl-font-weight-bold gl-text-green-500` |
+| danger | `gl-font-weight-bold gl-text-red-500` |
+| critical | `gl-font-weight-bold gl-text-red-800` |
+| same | `gl-font-weight-bold gl-text-gray-700` |
+| strong | `gl-font-weight-bold` |
+| small | `gl-font-sm` |
+
+## Action buttons
+
+You can add action buttons to all level 1 and 2 in each extension. These buttons
+are meant as a way to provide links or actions for each row:
+
+- Action buttons for level 1 can be set through the `tertiaryButtons` computed property.
+  This property should return an array of objects for each action button.
+- Action buttons for level 2 can be set by adding the `actions` key to the level 2 rows object.
+  The value for this key must also be an array of objects for each action button.
+
+Links must follow this structure:
+
+```javascript
+{
+  text: 'Click me',
+  href: this.someLinkHref,
+  target: '_blank', // Optional
+}
+```
+
+For internal action buttons, follow this structure:
+
+```javascript
+{
+  text: 'Click me',
+  onClick() {}
+}
+```
diff --git a/doc/development/permissions.md b/doc/development/permissions.md
index 72e20d31cf8..a5d211a5d2e 100644
--- a/doc/development/permissions.md
+++ b/doc/development/permissions.md
@@ -1,6 +1,6 @@
 ---
 stage: Manage
-group: Authentication & Authorization
+group: Authentication and Authorization
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md
index a9c68905095..f3a4f47eb22 100644
--- a/doc/development/pipelines.md
+++ b/doc/development/pipelines.md
@@ -12,7 +12,7 @@ which itself includes files under
 [`.gitlab/ci/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/.gitlab/ci)
 for easier maintenance.
 
-We're striving to [dogfood](https://about.gitlab.com/handbook/engineering/#dogfooding)
+We're striving to [dogfood](https://about.gitlab.com/handbook/engineering/principles/#dogfooding)
 GitLab [CI/CD features and best-practices](../ci/yaml/index.md)
 as much as possible.
 
@@ -90,6 +90,13 @@ In addition, there are a few circumstances where we would always run the full Je
 - when any vendored JavaScript file is changed (i.e. `vendor/assets/javascripts/**/*`)
 - when any backend file is changed ([see the patterns list for details](https://gitlab.com/gitlab-org/gitlab/-/blob/3616946936c1adbd9e754c1bd06f86ba670796d8/.gitlab/ci/rules.gitlab-ci.yml#L205-216))
 
+### Fork pipelines
+
+We only run the minimal RSpec & Jest jobs for fork pipelines unless the `pipeline:run-all-rspec`
+label is set on the MR. The goal is to reduce the CI minutes consumed by fork pipelines.
+
+See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1170).
+
 ## Fail-fast job in merge request pipelines
 
 To provide faster feedback when a merge request breaks existing tests, we are experimenting with a
@@ -170,10 +177,23 @@ After that, the next pipeline uses the up-to-date `knapsack/report-master.json`
 
 ### Flaky tests
 
+#### Automatic skipping of flaky tests
+
 Tests that are [known to be flaky](testing_guide/flaky_tests.md#automatic-retries-and-flaky-tests-detection) are
 skipped unless the `$SKIP_FLAKY_TESTS_AUTOMATICALLY` variable is set to `false` or if the `~"pipeline:run-flaky-tests"`
 label is set on the MR.
 
+See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1069).
+
+#### Automatic retry of failing tests in a separate process
+
+When the `$RETRY_FAILED_TESTS_IN_NEW_PROCESS` variable is set to `true`, RSpec tests that failed are automatically retried once in a separate
+RSpec process. The goal is to get rid of most side-effects from previous tests that may lead to a subsequent test failure.
+
+We keep track of retried tests in the `$RETRIED_TESTS_REPORT_FILE` file saved as artifact by the `rspec:flaky-tests-report` job.
+
+See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1148).
+
 ### Monitoring
 
 The GitLab test suite is [monitored](performance.md#rspec-profiling) for the `main` branch, and any branch
@@ -194,7 +214,7 @@ If you want to force a Review App to be deployed regardless of your changes, you
 ## As-if-FOSS jobs
 
 The `* as-if-foss` jobs run the GitLab test suite "as if FOSS", meaning as if the jobs would run in the context
-of the `gitlab-org/gitlab-foss` project. These jobs are only created in the following cases:
+of `gitlab-org/gitlab-foss`. These jobs are only created in the following cases:
 
 - when the `pipeline:run-as-if-foss` label is set on the merge request
 - when the merge request is created in the `gitlab-org/security/gitlab` project
@@ -203,13 +223,12 @@ of the `gitlab-org/gitlab-foss` project. These jobs are only created in the foll
 The `* as-if-foss` jobs are run in addition to the regular EE-context jobs. They have the `FOSS_ONLY='1'` variable
 set and get the `ee/` folder removed before the tests start running.
 
-The intent is to ensure that a change doesn't introduce a failure after the `gitlab-org/gitlab` project is synced to
-the `gitlab-org/gitlab-foss` project.
+The intent is to ensure that a change doesn't introduce a failure after `gitlab-org/gitlab` is synced to `gitlab-org/gitlab-foss`.
 
 ## As-if-JH jobs
 
 The `* as-if-jh` jobs run the GitLab test suite "as if JiHu", meaning as if the jobs would run in the context
-of [the `gitlab-jh/gitlab` project](jh_features_review.md). These jobs are only created in the following cases:
+of [GitLab JH](jh_features_review.md). These jobs are only created in the following cases:
 
 - when the `pipeline:run-as-if-jh` label is set on the merge request
 - when the `pipeline:run-all-rspec` label is set on the merge request
@@ -218,16 +237,18 @@ of [the `gitlab-jh/gitlab` project](jh_features_review.md). These jobs are only
 
 The `* as-if-jh` jobs are run in addition to the regular EE-context jobs. The `jh/` folder is added before the tests start running.
 
-The intent is to ensure that a change doesn't introduce a failure after the `gitlab-org/gitlab` project is synced to
-the `gitlab-jh/gitlab` project.
+The intent is to ensure that a change doesn't introduce a failure after `gitlab-org/gitlab` is synced to [GitLab JH](https://jihulab.com/gitlab-cn/gitlab).
 
 ### Corresponding JH branch
 
-You can create a corresponding JH branch on the `gitlab-jh/gitlab` project by
+You can create a corresponding JH branch on [GitLab JH](https://jihulab.com/gitlab-cn/gitlab) by
 appending `-jh` to the branch name. If a corresponding JH branch is found,
 `* as-if-jh` jobs grab the `jh` folder from the respective branch,
 rather than from the default branch.
 
+NOTE:
+For now, CI will try to fetch the branch on the [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh/gitlab), so it might take some time for the new JH branch to propagate to the mirror.
+
 ## `undercover` RSpec test
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74859) in GitLab 14.6.
@@ -235,7 +256,7 @@ rather than from the default branch.
 The `rspec:undercoverage` job runs [`undercover`](https://rubygems.org/gems/undercover)
 to detect, and fail if any changes introduced in the merge request has zero coverage.
 
-The `rsepc:undercoverage` job obtains coverage data from the `rspec:coverage`
+The `rspec:undercoverage` job obtains coverage data from the `rspec:coverage`
 job.
 
 In the event of an emergency, or false positive from this job, add the
diff --git a/doc/development/policies.md b/doc/development/policies.md
index 61cc6e0edf5..c9e4fdb4350 100644
--- a/doc/development/policies.md
+++ b/doc/development/policies.md
@@ -1,6 +1,6 @@
 ---
 stage: Manage
-group: Authentication & Authorization
+group: Authentication and Authorization
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
@@ -111,7 +111,7 @@ Each line represents a rule that was evaluated. There are a few things to note:
 
 Here you can see that the first four rules were evaluated `false` for
 which user and subject. For example, you can see in the last line that
-the rule was activated because the user `john` had the Reporter [role](../user/permissions.md) on
+the rule was activated because the user `john` had the Reporter role on
 `Project/4`.
 
 When a policy is asked whether a particular ability is allowed
diff --git a/doc/development/product_qualified_lead_guide/index.md b/doc/development/product_qualified_lead_guide/index.md
new file mode 100644
index 00000000000..f9d18bacecd
--- /dev/null
+++ b/doc/development/product_qualified_lead_guide/index.md
@@ -0,0 +1,94 @@
+---
+stage: Growth
+group: Conversion
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Product Qualified Lead (PQL) development guide
+
+The Product Qualified Lead (PQL) funnel connects our users with our team members. Read more about [PQL product principles](https://about.gitlab.com/handbook/product/product-principles/#product-qualified-leads-pqls).
+
+A hand-raise PQL is a user who requests to speak to sales from within the product.
+
+## Embed a hand-raise lead form
+
+[HandRaiseLeadButton](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/assets/javascripts/hand_raise_leads/hand_raise_lead/components/hand_raise_lead_button.vue) is a reusable component that adds a button and a hand-raise modal to any screen.
+
+You can import a hand-raise lead button the following way.
+
+```javascript
+import HandRaiseLeadButton from 'ee/hand_raise_leads/hand_raise_lead/components/hand_raise_lead_button.vue';
+
+export default {
+  components: {
+    HandRaiseLeadButton,
+...
+</script>
+
+<template>
+
+<hand-raise-lead-button />
+
+```
+
+The hand-raise lead form accepts the following parameters via provide or inject.
+
+```javascript
+    provide: {
+      user: {
+        namespaceId,
+        userName,
+        firstName,
+        lastName,
+        companyName,
+        glmContent,
+      },
+    },
+```
+
+### Monitor the lead location
+
+When embedding a new hand raise form, use a unique `glmContent` or `glm_content` field that is different to any existing values.
+
+We currently use the following `glm content` values:
+
+| glm_content value | Notes |
+| ------ | ------ |
+| discover-group-security | This value is used in the group security feature discovery page. |
+| discover-group-security-pqltest | This value is used in the group security feature discovery page [experiment with 3 CTAs](https://gitlab.com/gitlab-org/gitlab/-/issues/349799). |
+| discover-project-security | This value is used in the project security feature discovery page. |
+| discover-project-security-pqltest | This value is used in the project security feature discovery page [experiment with 3 CTAs](https://gitlab.com/gitlab-org/gitlab/-/issues/349799). |
+| group-billing | This value is used in the group billing page. |
+| trial-status-show-group | This value is used in the top left nav when a namespace has an active trial. |
+
+### Test the component
+
+In a jest test, you may test the presence of the component.
+
+```javascript
+expect(wrapper.findComponent(HandRaiseLeadButton).exists()).toBe(true);
+```
+
+## PQL lead flow
+
+The flow of a PQL lead is as follows:
+
+1. A user triggers a [`HandRaiseLeadButton` component](#embed-a-hand-raise-lead-form) on `gitlab.com`.
+1. The `HandRaiseLeadButton` submits any information to the following API endpoint: `/-/trials/create_hand_raise_lead`.
+1. That endpoint reposts the form to the CustomersDot `trials/create_hand_raise_lead` endpoint.
+1. CustomersDot records the form data to the `leads` table and posts the form to [Platypus](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/integrations/platypus).
+1. Platypus posts the form to Workato (which is under the responsibility of the Business Operations team).
+1. Workato sends the form to Marketo.
+1. Marketo does scoring and sends the form to Salesforce.
+1. Our Sales team uses Salesforce to connect to the leads.
+
+## Monitor and manually test leads
+
+- Check the application and Sidekiq logs on `gitlab.com` and CustomersDot to monitor leads.
+- Check the `leads` table in CustomersDot.
+- Set up staging credentials for Platypus, and track the leads on the [Platypus Dashboard](https://staging.ci.nexus.gitlabenvironment.cloud/admin/queues/queue/new-lead-queue).
+- Ask for access to the Marketo Sandbox and validate the leads there.
+
+## Trials
+
+Trials follow the same flow as the PQL leads.
diff --git a/doc/development/profiling.md b/doc/development/profiling.md
index 789e0640933..deb743569c5 100644
--- a/doc/development/profiling.md
+++ b/doc/development/profiling.md
@@ -91,11 +91,6 @@ printer = RubyProf::CallStackPrinter.new(result)
 printer.print(File.open('/tmp/profile.html', 'w'))
 ```
 
-[GitLab-Profiler](https://gitlab.com/gitlab-com/gitlab-profiler) is a project
-that builds on this to add some additional niceties, such as allowing
-configuration with a single YAML file for multiple URLs, and uploading of the
-profile and log output to S3.
-
 ## Speedscope flamegraphs
 
 You can generate a flamegraph for a particular URL by selecting a flamegraph sampling mode button in the performance bar or by adding the `performance_bar=flamegraph` parameter to the request.
diff --git a/doc/development/prometheus_metrics.md b/doc/development/prometheus_metrics.md
index da6ba14cdd8..b3f259efc3d 100644
--- a/doc/development/prometheus_metrics.md
+++ b/doc/development/prometheus_metrics.md
@@ -1,6 +1,6 @@
 ---
 stage: Monitor
-group: Monitor
+group: Respond
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
diff --git a/doc/development/ruby_upgrade.md b/doc/development/ruby_upgrade.md
index 2102a256645..a208a93e300 100644
--- a/doc/development/ruby_upgrade.md
+++ b/doc/development/ruby_upgrade.md
@@ -272,4 +272,4 @@ and merged back independently.
 - **Give yourself enough time to fix problems ahead of a milestone release.** GitLab moves fast.
 As a Ruby upgrade requires many MRs to be sent and reviewed, make sure all changes are merged at least a week
 before the 22nd. This gives us extra time to act if something breaks. If in doubt, it is better to
-postpone the upgrade to the following month, as we [prioritize availability over velocity](https://about.gitlab.com/handbook/engineering/#prioritizing-technical-decisions).
+postpone the upgrade to the following month, as we [prioritize availability over velocity](https://about.gitlab.com/handbook/engineering/principles/#prioritizing-technical-decisions).
diff --git a/doc/development/scalability.md b/doc/development/scalability.md
index 7a3f3c7097d..fe7063be0e8 100644
--- a/doc/development/scalability.md
+++ b/doc/development/scalability.md
@@ -45,7 +45,7 @@ many groups or projects, and the access level (including guest, developer, or
 maintainer) to groups and projects determines what users can see and
 what they can access.
 
-Users with the administrator role can access all projects and even impersonate
+Users with administrator access can access all projects and even impersonate
 users.
 
 #### Sharding and partitioning
diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md
index 51d3338e5ed..d34b12c6361 100644
--- a/doc/development/secure_coding_guidelines.md
+++ b/doc/development/secure_coding_guidelines.md
@@ -184,13 +184,15 @@ and [possessive quantifiers](https://www.regular-expressions.info/possessive.htm
 - Avoid nested quantifiers if possible (for example `(a+)+`)
 - Try to be as precise as possible in your regex and avoid the `.` if there's an alternative
   - For example, Use `_[^_]+_` instead of `_.*_` to match `_text here_`
+- Use reasonable ranges (for example, `{1,10}`) for repeating patterns instead of unbounded `*` and `+` matchers
+- When possible, perform simple input validation such as maximum string length checks before using regular expressions
 - If in doubt, don't hesitate to ping `@gitlab-com/gl-security/appsec`
 
 #### Go
 
 Go's [`regexp`](https://pkg.go.dev/regexp) package uses `re2` and isn't vulnerable to backtracking issues.
 
-## Further Links
+### Further Links
 
 - [Rubular](https://rubular.com/) is a nice online tool to fiddle with Ruby Regexps.
 - [Runaway Regular Expressions](https://www.regular-expressions.info/catastrophic.html)
diff --git a/doc/development/service_ping/implement.md b/doc/development/service_ping/implement.md
index c32789740c3..3a1e4c6d87b 100644
--- a/doc/development/service_ping/implement.md
+++ b/doc/development/service_ping/implement.md
@@ -793,6 +793,8 @@ The Service Ping JSON payload for GitLab.com is shared in the
 You may also use the [Service Ping QA dashboard](https://app.periscopedata.com/app/gitlab/632033/Usage-Ping-QA) to check how well your metric performs.
 The dashboard allows filtering by GitLab version, by "Self-managed" and "SaaS", and shows you how many failures have occurred for each metric. Whenever you notice a high failure rate, you can re-optimize your metric.
 
+Use [Metrics Dictionary](https://metrics.gitlab.com/) [copy query to clipboard feature](https://www.youtube.com/watch?v=n4o65ivta48&list=PL05JrBw4t0Krg3mbR6chU7pXtMt_es6Pb) to get a query ready to run in Sisense for a specific metric.
+
 ## Set up and test Service Ping locally
 
 To set up Service Ping locally, you must:
diff --git a/doc/development/service_ping/index.md b/doc/development/service_ping/index.md
index 315ff2b090c..86e70cc8bbc 100644
--- a/doc/development/service_ping/index.md
+++ b/doc/development/service_ping/index.md
@@ -84,7 +84,7 @@ Registration is not yet required for participation, but will be added in a futur
 
 #### Enable Registration Features
 
-1. Sign in as a user with the [Administrator](../../user/permissions.md) role.
+1. Sign in as a user with administrator access.
 1. On the top bar, select **Menu > Admin**.
 1. On the left sidebar, select **Settings > Metrics and profiling**.
 1. Expand the **Usage statistics** section.
@@ -96,7 +96,7 @@ Registration is not yet required for participation, but will be added in a futur
 
 You can view the exact JSON payload sent to GitLab Inc. in the Admin Area. To view the payload:
 
-1. Sign in as a user with the [Administrator](../../user/permissions.md) role.
+1. Sign in as a user with administrator access.
 1. On the top bar, select **Menu > Admin**.
 1. On the left sidebar, select **Settings > Metrics and profiling**.
 1. Expand the **Usage statistics** section.
@@ -118,7 +118,7 @@ configuration file.
 
 To disable Service Ping in the GitLab UI:
 
-1. Sign in as a user with the [Administrator](../../user/permissions.md) role.
+1. Sign in as a user with administrator access.
 1. On the top bar, select **Menu > Admin**.
 1. On the left sidebar, select **Settings > Metrics and profiling**.
 1. Expand the **Usage statistics** section.
@@ -198,13 +198,32 @@ sequenceDiagram
 ## How Service Ping works
 
 1. The Service Ping [cron job](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/workers/gitlab_service_ping_worker.rb#L24) is set in Sidekiq to run weekly.
-1. When the cron job runs, it calls [`Gitlab::UsageData.to_json`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb#L49).
-1. `Gitlab::UsageData.to_json` [cascades down](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb) to ~400+ other counter method calls.
-1. The response of all methods calls are [merged together](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb#L68) into a single JSON payload in `Gitlab::UsageData.to_json`.
+1. When the cron job runs, it calls [`Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb).
+1. `Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)` [cascades down](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb) to ~400+ other counter method calls.
+1. The response of all methods calls are [merged together](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb#L68) into a single JSON payload.
 1. The JSON payload is then [posted to the Versions application](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb#L20)
    If a firewall exception is needed, the required URL depends on several things. If
    the hostname is `version.gitlab.com`, the protocol is `TCP`, and the port number is `443`,
    the required URL is <https://version.gitlab.com/>.
+1. In case of an error, it will be reported to the Version application along with following pieces of information:
+
+- `uuid` - GitLab instance unique identifier
+- `hostname` - GitLab instance hostname
+- `version` - GitLab instance current versions 
+- `elapsed` - Amount of time which passed since Service Ping report process started and moment of error occurrence
+- `message` - Error message
+
+<pre>
+<code>
+{
+  "uuid"=>"02333324-1cd7-4c3b-a45b-a4993f05fb1d", 
+  "hostname"=>"127.0.0.1", 
+  "version"=>"14.7.0-pre", 
+  "elapsed"=>0.006946, 
+  "message"=>'PG::UndefinedColumn: ERROR:  column \"non_existent_attribute\" does not exist\nLINE 1: SELECT COUNT(non_existent_attribute) FROM \"issues\" /*applica...'
+}
+</code>
+</pre>
 
 ### On a Geo secondary site
 
@@ -510,7 +529,7 @@ To generate Service Ping, use [Teleport](https://goteleport.com/docs/) or a deta
 
 ### Verification (After approx 30 hours)
 
-#### Verify with a detached screen session
+#### Verify with Teleport
 
 1. Follow [the steps](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/Teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to request a new access to the required environment and connect to the Rails console
 1. Check the last payload in `raw_usage_data` table: `RawUsageData.last.payload`
@@ -557,6 +576,28 @@ skip_db_write:
 ServicePing::SubmitService.new(skip_db_write: true).execute
 ```
 
+## Manually upload Service Ping payload 
+
+> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7388) in GitLab 14.8 with a flag named `admin_application_settings_service_usage_data_center`. Disabled by default.
+
+Service Ping payload can be uploaded to GitLab even if your application instance doesn't have access to the internet,
+or you don't have Service Ping [cron job](#how-service-ping-works) enabled.
+
+To upload payload manually:
+
+1. Sign in as a user with administrator access.
+1. On the top bar, select **Menu > Admin**.
+1. On the left sidebar, select **Settings > Service** usage data.
+1. Select **Download payload**.
+1. Save the JSON file.
+1. Visit [Service usage data center](https://version.gitlab.com/usage_data/new).
+1. Select **Choose file** and choose the file from p5.
+1. Select **Upload**.
+
+## Monitoring
+
+Service Ping reporting process state is monitored with [internal SiSense dashboard](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health). 
+
 ## Troubleshooting
 
 ### Cannot disable Service Ping using the configuration file
diff --git a/doc/development/service_ping/metrics_dictionary.md b/doc/development/service_ping/metrics_dictionary.md
index 808c5064cf3..93eec4efabd 100644
--- a/doc/development/service_ping/metrics_dictionary.md
+++ b/doc/development/service_ping/metrics_dictionary.md
@@ -6,8 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # Metrics Dictionary Guide
 
-[Service Ping](index.md) metrics are defined in the
-[Metrics Dictionary](https://metrics.gitlab.com/index.html).
+[Service Ping](index.md) metrics are defined in individual YAML files definitions from which the
+[Metrics Dictionary](https://metrics.gitlab.com/) is built.
 This guide describes the dictionary and how it's implemented.
 
 ## Metrics Definition and validation
@@ -39,7 +39,7 @@ Each metric is defined in a separate YAML file consisting of a number of fields:
 | `product_group`     | yes      | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the metric. |
 | `product_category`  | no       | The [product category](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/categories.yml) for the metric. |
 | `value_type`        | yes      | `string`; one of [`string`, `number`, `boolean`, `object`](https://json-schema.org/understanding-json-schema/reference/type.html).                                                     |
-| `status`            | yes      | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `deprecated`, `removed`, `broken`. |
+| `status`            | yes      | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `removed`, `broken`. |
 | `time_frame`        | yes      | `string`; may be set to a value like `7d`, `28d`, `all`, `none`. |
 | `data_source`       | yes      | `string`; may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `system`. |
 | `data_category`     | yes      | `string`; [categories](#data-category) of the metric, may be set to `operational`, `optional`, `subscription`, `standard`. The default value is `optional`.|
@@ -49,7 +49,7 @@ Each metric is defined in a separate YAML file consisting of a number of fields:
 | `tier`              | yes      | `array`; may contain one or a combination of `free`, `premium` or `ultimate`. The [tier]( https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/) where the tracked feature is available. This should be verbose and contain all tiers where a metric is available. |
 | `milestone`         | no       | The milestone when the metric is introduced. |
 | `milestone_removed` | no       | The milestone when the metric is removed. |
-| `introduced_by_url` | no       | The URL to the Merge Request that introduced the metric. |
+| `introduced_by_url` | no       | The URL to the merge request that introduced the metric. |
 | `repair_issue_url`  | no       | The URL of the issue that was created to repair a metric with a `broken` status. |
 | `options`           | no       | `object`: options information needed to calculate the metric value. |
 | `skip_validation`   | no       | This should **not** be set. [Used for imported metrics until we review, update and make them valid](https://gitlab.com/groups/gitlab-org/-/epics/5425). |
@@ -60,7 +60,6 @@ Metric definitions can have one of the following statuses:
 
 - `active`: Metric is used and reports data.
 - `broken`: Metric reports broken data (for example, -1 fallback), or does not report data at all. A metric marked as `broken` must also have the `repair_issue_url` attribute.
-- `deprecated`: Metric is deprecated and possibly planned to be removed.
 - `removed`: Metric was removed, but it may appear in Service Ping payloads sent from instances running on older versions of GitLab.
 
 ### Metric value_type
@@ -194,7 +193,7 @@ tier:
 - ultimate
 ```
 
-## Create a new metric definition
+### Create a new metric definition
 
 The GitLab codebase provides a dedicated [generator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_definition_generator.rb) to create new metric definitions.
 
@@ -229,7 +228,7 @@ bundle exec rails generate gitlab:usage_metric_definition counts.issues --ee --d
 create  ee/config/metrics/counts_7d/issues.yml
 ```
 
-## Metrics added dynamic to Service Ping payload
+### Metrics added dynamic to Service Ping payload
 
 The [Redis HLL metrics](implement.md#known-events-are-added-automatically-in-service-data-payload) are added automatically to Service Ping payload.
 
@@ -250,3 +249,13 @@ bundle exec rails generate gitlab:usage_metric_definition:redis_hll issues users
 create  config/metrics/counts_7d/i_closed_weekly.yml
 create  config/metrics/counts_28d/i_closed_monthly.yml
 ```
+
+## Metrics Dictionary
+
+[Metrics Dictionary is a separate application](https://gitlab.com/gitlab-org/growth/product-intelligence/metric-dictionary).
+
+All metrics available in Service Ping are in the [Metrics Dictionary](https://metrics.gitlab.com/).
+
+### Copy query to clipboard
+
+To check if a metric has data in Sisense, use the copy query to clipboard feature. This copies a query that's ready to use in Sisense. The query gets the last five service ping data for GitLab.com for a given metric. For information about how to check if a Service Ping metric has data in Sisense, see this [demo](https://www.youtube.com/watch?v=n4o65ivta48).
diff --git a/doc/development/service_ping/metrics_lifecycle.md b/doc/development/service_ping/metrics_lifecycle.md
index ebfab6341e9..a7ecf15a493 100644
--- a/doc/development/service_ping/metrics_lifecycle.md
+++ b/doc/development/service_ping/metrics_lifecycle.md
@@ -14,6 +14,11 @@ Follow the [Implement Service Ping](implement.md) guide.
 
 ## Change an existing metric
 
+See [this video tutorial](https://youtu.be/bYf3c01KCls) for help with the update of metric attributes.
+
+NOTE:
+The `key_path` attribute represents the location of the metric in Service Ping payload and must not be changed.
+
 Because we do not control when customers update their self-managed instances of GitLab,
 we **STRONGLY DISCOURAGE** changes to the logic used to calculate any metric.
 Any such changes lead to inconsistent reports from multiple GitLab instances.
@@ -85,10 +90,9 @@ To remove a metric:
 1. Verify that removing the metric from the Service Ping payload does not cause
    errors in [Version App](https://gitlab.com/gitlab-services/version-gitlab-com)
    when the updated payload is collected and processed. Version App collects
-   and persists all Service Ping reports. To do that you can modify
-   [fixtures](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/spec/support/usage_data_helpers.rb#L540)
-   used to test
-   [`UsageDataController#create`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/3760ef28/spec/controllers/usage_data_controller_spec.rb#L75)
+   and persists all Service Ping reports. To verify Service Ping processing in your local development environment, follow this [guide](https://www.youtube.com/watch?v=FS5emplabRU).
+   Alternatively, you can modify [fixtures](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/spec/support/usage_data_helpers.rb#L540)
+   used to test the [`UsageDataController#create`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/3760ef28/spec/controllers/usage_data_controller_spec.rb#L75)
    endpoint, and assure that test suite does not fail when metric that you wish to remove is not included into test payload.
 
 1. Create an issue in the
diff --git a/doc/development/service_ping/review_guidelines.md b/doc/development/service_ping/review_guidelines.md
index eb64d460b5a..137e11608cf 100644
--- a/doc/development/service_ping/review_guidelines.md
+++ b/doc/development/service_ping/review_guidelines.md
@@ -14,7 +14,7 @@ general best practices for code reviews, refer to our [code review guide](../cod
 ## Resources for reviewers
 
 - [Service Ping Guide](index.md)
-- [Metrics Dictionary](https://metrics.gitlab.com/index.html)
+- [Metrics Dictionary](https://metrics.gitlab.com/)
 
 ## Review process
 
diff --git a/doc/development/service_ping/troubleshooting.md b/doc/development/service_ping/troubleshooting.md
new file mode 100644
index 00000000000..770b6650764
--- /dev/null
+++ b/doc/development/service_ping/troubleshooting.md
@@ -0,0 +1,31 @@
+---
+stage: Growth
+group: Product Intelligence
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Troubleshooting
+
+## Service Ping Payload drop
+
+### Symptoms
+
+You will be alerted by a [Sisense alert](https://app.periscopedata.com/app/gitlab/alert/Service-ping-payload-drop-Alert/5a5b8766b8af43b4a75d6e9c684a365f/edit) that is sent to `#g_product_intelligence` Slack channel
+
+### Locating the problem
+
+First you need to identify at which stage in Service Ping data pipeline the drop is occurring.
+
+Start at [Service Ping Health Dashboard](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health) on Sisense.
+
+The alert compares the current daily value with the daily value from previous week, excluding the last 48 hours as this is the interval where we export the data in GCP and get it into DWH.
+
+You can use [this query](https://gitlab.com/gitlab-org/gitlab/-/issues/347298#note_836685350) as an example, to start detecting when the drop started.
+
+### Troubleshooting GitLab application layer
+
+For results about an investigation conducted into an unexpected drop in Service ping Payload events volume, see [this issue](https://gitlab.com/gitlab-data/analytics/-/issues/11071).
+
+### Troubleshooting data warehouse layer
+
+Reach out to the [Data team](https://about.gitlab.com/handbook/business-technology/data-team) to ask about current state of data warehouse. On their handbook page there is a [section with contact details](https://about.gitlab.com/handbook/business-technology/data-team/#how-to-connect-with-us).
diff --git a/doc/development/sidekiq/compatibility_across_updates.md b/doc/development/sidekiq/compatibility_across_updates.md
new file mode 100644
index 00000000000..919f6935139
--- /dev/null
+++ b/doc/development/sidekiq/compatibility_across_updates.md
@@ -0,0 +1,159 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Sidekiq Compatibility across Updates
+
+The arguments for a Sidekiq job are stored in a queue while it is
+scheduled for execution. During a online update, this could lead to
+several possible situations:
+
+1. An older version of the application publishes a job, which is executed by an
+   upgraded Sidekiq node.
+1. A job is queued before an upgrade, but executed after an upgrade.
+1. A job is queued by a node running the newer version of the application, but
+   executed on a node running an older version of the application.
+
+## Adding new workers
+
+On GitLab.com, we [do not currently have a Sidekiq deployment in the
+canary stage](https://gitlab.com/gitlab-org/gitlab/-/issues/19239). This
+means that a new worker than can be scheduled from an HTTP endpoint may
+be scheduled from canary but not run on Sidekiq until the full
+production deployment is complete. This can be several hours later than
+scheduling the job. For some workers, this will not be a problem. For
+others - particularly [latency-sensitive
+jobs](worker_attributes.md#latency-sensitive-jobs) - this will result in a poor user
+experience.
+
+This only applies to new worker classes when they are first introduced.
+As we recommend [using feature flags](../feature_flags/) as a general
+development process, it's best to control the entire change (including
+scheduling of the new Sidekiq worker) with a feature flag.
+
+## Changing the arguments for a worker
+
+Jobs need to be backward and forward compatible between consecutive versions
+of the application. Adding or removing an argument may cause problems
+during deployment before all Rails and Sidekiq nodes have the updated code.
+
+### Deprecate and remove an argument
+
+**Before you remove arguments from the `perform_async` and `perform` methods.**, deprecate them. The
+following example deprecates and then removes `arg2` from the `perform_async` method:
+
+1. Provide a default value (usually `nil`) and use a comment to mark the
+   argument as deprecated in the coming minor release. (Release M)
+
+    ```ruby
+    class ExampleWorker
+      # Keep arg2 parameter for backwards compatibility.
+      def perform(object_id, arg1, arg2 = nil)
+        # ...
+      end
+    end
+    ```
+
+1. One minor release later, stop using the argument in `perform_async`. (Release M+1)
+
+    ```ruby
+    ExampleWorker.perform_async(object_id, arg1)
+    ```
+
+1. At the next major release, remove the value from the worker class. (Next major release)
+
+    ```ruby
+    class ExampleWorker
+      def perform(object_id, arg1)
+        # ...
+      end
+    end
+    ```
+
+### Add an argument
+
+There are two options for safely adding new arguments to Sidekiq workers:
+
+1. Set up a [multi-step deployment](#multi-step-deployment) in which the new argument is first added to the worker.
+1. Use a [parameter hash](#parameter-hash) for additional arguments. This is perhaps the most flexible option.
+
+#### Multi-step deployment
+
+This approach requires multiple releases.
+
+1. Add the argument to the worker with a default value (Release M).
+
+    ```ruby
+    class ExampleWorker
+      def perform(object_id, new_arg = nil)
+        # ...
+      end
+    end
+    ```
+
+1. Add the new argument to all the invocations of the worker (Release M+1).
+
+    ```ruby
+    ExampleWorker.perform_async(object_id, new_arg)
+    ```
+
+1. Remove the default value (Release M+2).
+
+    ```ruby
+    class ExampleWorker
+      def perform(object_id, new_arg)
+        # ...
+      end
+    end
+    ```
+
+#### Parameter hash
+
+This approach doesn't require multiple releases if an existing worker already
+uses a parameter hash.
+
+1. Use a parameter hash in the worker to allow future flexibility.
+
+    ```ruby
+    class ExampleWorker
+      def perform(object_id, params = {})
+        # ...
+      end
+    end
+    ```
+
+## Removing workers
+
+Try to avoid removing workers and their queues in minor and patch
+releases.
+
+During online update instance can have pending jobs and removing the queue can
+lead to those jobs being stuck forever. If you can't write migration for those
+Sidekiq jobs, please consider removing the worker in a major release only.
+
+## Renaming queues
+
+For the same reasons that removing workers is dangerous, care should be taken
+when renaming queues.
+
+When renaming queues, use the `sidekiq_queue_migrate` helper migration method
+in a **post-deployment migration**:
+
+```ruby
+class MigrateTheRenamedSidekiqQueue < Gitlab::Database::Migration[1.0]
+  def up
+    sidekiq_queue_migrate 'old_queue_name', to: 'new_queue_name'
+  end
+
+  def down
+    sidekiq_queue_migrate 'new_queue_name', to: 'old_queue_name'
+  end
+end
+
+```
+
+You must rename the queue in a post-deployment migration not in a normal
+migration. Otherwise, it runs too early, before all the workers that
+schedule these jobs have stopped running. See also [other examples](../post_deployment_migrations.md#use-cases).
diff --git a/doc/development/sidekiq/idempotent_jobs.md b/doc/development/sidekiq/idempotent_jobs.md
new file mode 100644
index 00000000000..4b201e22ca9
--- /dev/null
+++ b/doc/development/sidekiq/idempotent_jobs.md
@@ -0,0 +1,208 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Sidekiq idempotent jobs
+
+It's known that a job can fail for multiple reasons. For example, network outages or bugs.
+In order to address this, Sidekiq has a built-in retry mechanism that is
+used by default by most workers within GitLab.
+
+It's expected that a job can run again after a failure without major side-effects for the
+application or users, which is why Sidekiq encourages
+jobs to be [idempotent and transactional](https://github.com/mperham/sidekiq/wiki/Best-Practices#2-make-your-job-idempotent-and-transactional).
+
+As a general rule, a worker can be considered idempotent if:
+
+- It can safely run multiple times with the same arguments.
+- Application side-effects are expected to happen only once
+  (or side-effects of a second run do not have an effect).
+
+A good example of that would be a cache expiration worker.
+
+A job scheduled for an idempotent worker is [deduplicated](#deduplication) when
+an unstarted job with the same arguments is already in the queue.
+
+## Ensuring a worker is idempotent
+
+Make sure the worker tests pass using the following shared example:
+
+```ruby
+include_examples 'an idempotent worker' do
+  it 'marks the MR as merged' do
+    # Using subject inside this block will process the job multiple times
+    subject
+
+    expect(merge_request.state).to eq('merged')
+  end
+end
+```
+
+Use the `perform_multiple` method directly instead of `job.perform` (this
+helper method is automatically included for workers).
+
+## Declaring a worker as idempotent
+
+```ruby
+class IdempotentWorker
+  include ApplicationWorker
+
+  # Declares a worker is idempotent and can
+  # safely run multiple times.
+  idempotent!
+
+  # ...
+end
+```
+
+It's encouraged to only have the `idempotent!` call in the top-most worker class, even if
+the `perform` method is defined in another class or module.
+
+If the worker class isn't marked as idempotent, a cop fails. Consider skipping
+the cop if you're not confident your job can safely run multiple times.
+
+## Deduplication
+
+When a job for an idempotent worker is enqueued while another
+unstarted job is already in the queue, GitLab drops the second
+job. The work is skipped because the same work would be
+done by the job that was scheduled first; by the time the second
+job executed, the first job would do nothing.
+
+### Strategies
+
+GitLab supports two deduplication strategies:
+
+- `until_executing`
+- `until_executed`
+
+More [deduplication strategies have been
+suggested](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/195). If
+you are implementing a worker that could benefit from a different
+strategy, please comment in the issue.
+
+#### Until Executing
+
+This strategy takes a lock when a job is added to the queue, and removes that lock before the job starts.
+
+For example, `AuthorizedProjectsWorker` takes a user ID. When the
+worker runs, it recalculates a user's authorizations. GitLab schedules
+this job each time an action potentially changes a user's
+authorizations. If the same user is added to two projects at the
+same time, the second job can be skipped if the first job hasn't
+begun, because when the first job runs, it creates the
+authorizations for both projects.
+
+```ruby
+module AuthorizedProjectUpdate
+  class UserRefreshOverUserRangeWorker
+    include ApplicationWorker
+
+    deduplicate :until_executing
+    idempotent!
+
+    # ...
+  end
+end
+```
+
+#### Until Executed
+
+This strategy takes a lock when a job is added to the queue, and removes that lock after the job finishes.
+It can be used to prevent jobs from running simultaneously multiple times.
+
+```ruby
+module Ci
+  class BuildTraceChunkFlushWorker
+    include ApplicationWorker
+
+    deduplicate :until_executed
+    idempotent!
+
+    # ...
+  end
+end
+```
+
+Also, you can pass `if_deduplicated: :reschedule_once` option to re-run a job once after
+the currently running job finished and deduplication happened at least once.
+This ensures that the latest result is always produced even if a race condition
+happened. See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342123) for more information.
+
+### Scheduling jobs in the future
+
+GitLab doesn't skip jobs scheduled in the future, as we assume that
+the state has changed by the time the job is scheduled to
+execute. Deduplication of jobs scheduled in the feature is possible
+for both `until_executed` and `until_executing` strategies.
+
+If you do want to deduplicate jobs scheduled in the future,
+this can be specified on the worker by passing `including_scheduled: true` argument
+when defining deduplication strategy:
+
+```ruby
+module AuthorizedProjectUpdate
+  class UserRefreshOverUserRangeWorker
+    include ApplicationWorker
+
+    deduplicate :until_executing, including_scheduled: true
+    idempotent!
+
+    # ...
+  end
+end
+```
+
+## Setting the deduplication time-to-live (TTL)
+
+Deduplication depends on an idempotency key that is stored in Redis. This is normally
+cleared by the configured deduplication strategy.
+
+However, the key can remain until its TTL in certain cases like:
+
+1. `until_executing` is used but the job was never enqueued or executed after the Sidekiq
+   client middleware was run.
+
+1. `until_executed` is used but the job fails to finish due to retry exhaustion, gets
+   interrupted the maximum number of times, or gets lost.
+
+The default value is 6 hours. During this time, jobs won't be enqueued even if the first
+job never executed or finished.
+
+The TTL can be configured with:
+
+```ruby
+class ProjectImportScheduleWorker
+  include ApplicationWorker
+
+  idempotent!
+  deduplicate :until_executing, ttl: 5.minutes
+end
+```
+
+Duplicate jobs can happen when the TTL is reached, so make sure you lower this only for jobs
+that can tolerate some duplication.
+
+### Preserve the latest WAL location for idempotent jobs
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69372) in GitLab 14.3.
+> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/338350) in GitLab 14.4.
+> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/338350) in GitLab 14.6.
+
+The deduplication always take into account the latest binary replication pointer, not the first one.
+This happens because we drop the same job scheduled for the second time and the Write-Ahead Log (WAL) is lost.
+This could lead to comparing the old WAL location and reading from a stale replica.
+
+To support both deduplication and maintaining data consistency with load balancing,
+we are preserving the latest WAL location for idempotent jobs in Redis.
+This way we are always comparing the latest binary replication pointer,
+making sure that we read from the replica that is fully caught up.
+
+FLAG:
+On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to
+[disable the feature flag](../../administration/feature_flags.md) named `preserve_latest_wal_locations_for_idempotent_jobs`.
+
+This feature flag is related to GitLab development and is not intended to be used by GitLab administrators, though.
+On GitLab.com, this feature is available.
diff --git a/doc/development/sidekiq/index.md b/doc/development/sidekiq/index.md
new file mode 100644
index 00000000000..c9906c4c768
--- /dev/null
+++ b/doc/development/sidekiq/index.md
@@ -0,0 +1,190 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Sidekiq guides
+
+We use [Sidekiq](https://github.com/mperham/sidekiq) as our background
+job processor. These guides are for writing jobs that will work well on
+GitLab.com and be consistent with our existing worker classes. For
+information on administering GitLab, see [configuring Sidekiq](../../administration/sidekiq.md).
+
+There are pages with additional detail on the following topics:
+
+1. [Compatibility across updates](compatibility_across_updates.md)
+1. [Job idempotency and job deduplication](idempotent_jobs.md)
+1. [Limited capacity worker: continuously performing work with a specified concurrency](limited_capacity_worker.md)
+1. [Logging](logging.md)
+1. [Worker attributes](worker_attributes.md)
+    1. **Job urgency** specifies queuing and execution SLOs
+    1. **Resource boundaries** and **external dependencies** for describing the workload
+    1. **Feature categorization**
+    1. **Database load balancing**
+
+## ApplicationWorker
+
+All workers should include `ApplicationWorker` instead of `Sidekiq::Worker`,
+which adds some convenience methods and automatically sets the queue based on
+the [routing rules](../../administration/operations/extra_sidekiq_routing.md#queue-routing-rules).
+
+## Retries
+
+Sidekiq defaults to using [25
+retries](https://github.com/mperham/sidekiq/wiki/Error-Handling#automatic-job-retry),
+with back-off between each retry. 25 retries means that the last retry
+would happen around three weeks after the first attempt (assuming all 24
+prior retries failed).
+
+For most workers - especially [idempotent workers](idempotent_jobs.md) -
+the default of 25 retries is more than sufficient. Many of our older
+workers declare 3 retries, which used to be the default within the
+GitLab application. 3 retries happen over the course of a couple of
+minutes, so the jobs are prone to failing completely.
+
+A lower retry count may be applicable if any of the below apply:
+
+1. The worker contacts an external service and we do not provide
+   guarantees on delivery. For example, webhooks.
+1. The worker is not idempotent and running it multiple times could
+   leave the system in an inconsistent state. For example, a worker that
+   posts a system note and then performs an action: if the second step
+   fails and the worker retries, the system note will be posted again.
+1. The worker is a cronjob that runs frequently. For example, if a cron
+   job runs every hour, then we don't need to retry beyond an hour
+   because we don't need two of the same job running at once.
+
+Each retry for a worker is counted as a failure in our metrics. A worker
+which always fails 9 times and succeeds on the 10th would have a 90%
+error rate.
+
+## Sidekiq Queues
+
+Previously, each worker had its own queue, which was automatically set based on the
+worker class name. For a worker named `ProcessSomethingWorker`, the queue name
+would be `process_something`. You can now route workers to a specific queue using
+[queue routing rules](../../administration/operations/extra_sidekiq_routing.md#queue-routing-rules).
+In GDK, new workers are routed to a queue named `default`.
+
+If you're not sure what queue a worker uses,
+you can find it using `SomeWorker.queue`. There is almost never a reason to
+manually override the queue name using `sidekiq_options queue: :some_queue`.
+
+After adding a new worker, run `bin/rake
+gitlab:sidekiq:all_queues_yml:generate` to regenerate
+`app/workers/all_queues.yml` or `ee/app/workers/all_queues.yml` so that
+it can be picked up by
+[`sidekiq-cluster`](../../administration/operations/extra_sidekiq_processes.md)
+in installations that don't use routing rules. To learn more about potential changes,
+read [Use routing rules by default and deprecate queue selectors for self-managed](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/596).
+
+Additionally, run
+`bin/rake gitlab:sidekiq:sidekiq_queues_yml:generate` to regenerate
+`config/sidekiq_queues.yml`.
+
+## Queue Namespaces
+
+While different workers cannot share a queue, they can share a queue namespace.
+
+Defining a queue namespace for a worker makes it possible to start a Sidekiq
+process that automatically handles jobs for all workers in that namespace,
+without needing to explicitly list all their queue names. If, for example, all
+workers that are managed by `sidekiq-cron` use the `cronjob` queue namespace, we
+can spin up a Sidekiq process specifically for these kinds of scheduled jobs.
+If a new worker using the `cronjob` namespace is added later on, the Sidekiq
+process also picks up jobs for that worker (after having been restarted),
+without the need to change any configuration.
+
+A queue namespace can be set using the `queue_namespace` DSL class method:
+
+```ruby
+class SomeScheduledTaskWorker
+  include ApplicationWorker
+
+  queue_namespace :cronjob
+
+  # ...
+end
+```
+
+Behind the scenes, this sets `SomeScheduledTaskWorker.queue` to
+`cronjob:some_scheduled_task`. Commonly used namespaces have their own
+concern module that can easily be included into the worker class, and that may
+set other Sidekiq options besides the queue namespace. `CronjobQueue`, for
+example, sets the namespace, but also disables retries.
+
+`bundle exec sidekiq` is namespace-aware, and listens on all
+queues in a namespace (technically: all queues prefixed with the namespace name)
+when a namespace is provided instead of a simple queue name in the `--queue`
+(`-q`) option, or in the `:queues:` section in `config/sidekiq_queues.yml`.
+
+Note that adding a worker to an existing namespace should be done with care, as
+the extra jobs take resources away from jobs from workers that were already
+there, if the resources available to the Sidekiq process handling the namespace
+are not adjusted appropriately.
+
+## Versioning
+
+Version can be specified on each Sidekiq worker class.
+This is then sent along when the job is created.
+
+```ruby
+class FooWorker
+  include ApplicationWorker
+
+  version 2
+
+  def perform(*args)
+    if job_version == 2
+      foo = args.first['foo']
+    else
+      foo = args.first
+    end
+  end
+end
+```
+
+Under this schema, any worker is expected to be able to handle any job that was
+enqueued by an older version of that worker. This means that when changing the
+arguments a worker takes, you must increment the `version` (or set `version 1`
+if this is the first time a worker's arguments are changing), but also make sure
+that the worker is still able to handle jobs that were queued with any earlier
+version of the arguments. From the worker's `perform` method, you can read
+`self.job_version` if you want to specifically branch on job version, or you
+can read the number or type of provided arguments.
+
+## Job size
+
+GitLab stores Sidekiq jobs and their arguments in Redis. To avoid
+excessive memory usage, we compress the arguments of Sidekiq jobs
+if their original size is bigger than 100KB.
+
+After compression, if their size still exceeds 5MB, it raises an
+[`ExceedLimitError`](https://gitlab.com/gitlab-org/gitlab/-/blob/f3dd89e5e510ea04b43ffdcb58587d8f78a8d77c/lib/gitlab/sidekiq_middleware/size_limiter/exceed_limit_error.rb#L8)
+error when scheduling the job.
+
+If this happens, rely on other means of making the data
+available in Sidekiq. There are possible workarounds such as:
+
+- Rebuild the data in Sidekiq with data loaded from the database or
+  elsewhere.
+- Store the data in [object storage](../file_storage.md#object-storage)
+  before scheduling the job, and retrieve it inside the job.
+
+## Job weights
+
+Some jobs have a weight declared. This is only used when running Sidekiq
+in the default execution mode - using
+[`sidekiq-cluster`](../../administration/operations/extra_sidekiq_processes.md)
+does not account for weights.
+
+As we are [moving towards using `sidekiq-cluster` in
+Free](https://gitlab.com/gitlab-org/gitlab/-/issues/34396), newly-added
+workers do not need to have weights specified. They can use the
+default weight, which is 1.
+
+## Tests
+
+Each Sidekiq worker must be tested using RSpec, just like any other class. These
+tests should be placed in `spec/workers`.
diff --git a/doc/development/sidekiq/limited_capacity_worker.md b/doc/development/sidekiq/limited_capacity_worker.md
new file mode 100644
index 00000000000..5b86e01781c
--- /dev/null
+++ b/doc/development/sidekiq/limited_capacity_worker.md
@@ -0,0 +1,84 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Sidekiq limited capacity worker
+
+It is possible to limit the number of concurrent running jobs for a worker class
+by using the `LimitedCapacity::Worker` concern.
+
+The worker must implement three methods:
+
+- `perform_work`: The concern implements the usual `perform` method and calls
+  `perform_work` if there's any available capacity.
+- `remaining_work_count`: Number of jobs that have work to perform.
+- `max_running_jobs`: Maximum number of jobs allowed to run concurrently.
+
+```ruby
+class MyDummyWorker
+  include ApplicationWorker
+  include LimitedCapacity::Worker
+
+  def perform_work(*args)
+  end
+
+  def remaining_work_count(*args)
+    5
+  end
+
+  def max_running_jobs
+    25
+  end
+end
+```
+
+Additional to the regular worker, a cron worker must be defined as well to
+backfill the queue with jobs. the arguments passed to `perform_with_capacity`
+are passed to the `perform_work` method.
+
+```ruby
+class ScheduleMyDummyCronWorker
+  include ApplicationWorker
+  include CronjobQueue
+
+  def perform(*args)
+    MyDummyWorker.perform_with_capacity(*args)
+  end
+end
+```
+
+## How many jobs are running?
+
+It runs `max_running_jobs` at almost all times.
+
+The cron worker checks the remaining capacity on each execution and it
+schedules at most `max_running_jobs` jobs. Those jobs on completion
+re-enqueue themselves immediately, but not on failure. The cron worker is in
+charge of replacing those failed jobs.
+
+## Handling errors and idempotence
+
+This concern disables Sidekiq retries, logs the errors, and sends the job to the
+dead queue. This is done to have only one source that produces jobs and because
+the retry would occupy a slot with a job to perform in the distant future.
+
+We let the cron worker enqueue new jobs, this could be seen as our retry and
+back off mechanism because the job might fail again if executed immediately.
+This means that for every failed job, we run at a lower capacity
+until the cron worker fills the capacity again. If it is important for the
+worker not to get a backlog, exceptions must be handled in `#perform_work` and
+the job should not raise.
+
+The jobs are deduplicated using the `:none` strategy, but the worker is not
+marked as `idempotent!`.
+
+## Metrics
+
+This concern exposes three Prometheus metrics of gauge type with the worker class
+name as label:
+
+- `limited_capacity_worker_running_jobs`
+- `limited_capacity_worker_max_running_jobs`
+- `limited_capacity_worker_remaining_work_count`
diff --git a/doc/development/sidekiq/logging.md b/doc/development/sidekiq/logging.md
new file mode 100644
index 00000000000..015376b0fc6
--- /dev/null
+++ b/doc/development/sidekiq/logging.md
@@ -0,0 +1,155 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Sidekiq logging
+
+## Worker context
+
+> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/9) in GitLab 12.8.
+
+To have some more information about workers in the logs, we add
+[metadata to the jobs in the form of an
+`ApplicationContext`](../logging.md#logging-context-metadata-through-rails-or-grape-requests).
+In most cases, when scheduling a job from a request, this context is already
+deducted from the request and added to the scheduled job.
+
+When a job runs, the context that was active when it was scheduled
+is restored. This causes the context to be propagated to any job
+scheduled from within the running job.
+
+All this means that in most cases, to add context to jobs, we don't
+need to do anything.
+
+There are however some instances when there would be no context
+present when the job is scheduled, or the context that is present is
+likely to be incorrect. For these instances, we've added Rubocop rules
+to draw attention and avoid incorrect metadata in our logs.
+
+As with most our cops, there are perfectly valid reasons for disabling
+them. In this case it could be that the context from the request is
+correct. Or maybe you've specified a context already in a way that
+isn't picked up by the cops. In any case, leave a code comment
+pointing to which context to use when disabling the cops.
+
+When you do provide objects to the context, make sure that the
+route for namespaces and projects is pre-loaded. This can be done by using
+the `.with_route` scope defined on all `Routable`s.
+
+### Cron workers
+
+The context is automatically cleared for workers in the cronjob queue
+(`include CronjobQueue`), even when scheduling them from
+requests. We do this to avoid incorrect metadata when other jobs are
+scheduled from the cron worker.
+
+Cron workers themselves run instance wide, so they aren't scoped to
+users, namespaces, projects, or other resources that should be added to
+the context.
+
+However, they often schedule other jobs that _do_ require context.
+
+That is why there needs to be an indication of context somewhere in
+the worker. This can be done by using one of the following methods
+somewhere within the worker:
+
+1. Wrap the code that schedules jobs in the `with_context` helper:
+
+   ```ruby
+     def perform
+       deletion_cutoff = Gitlab::CurrentSettings
+                           .deletion_adjourned_period.days.ago.to_date
+       projects = Project.with_route.with_namespace
+                    .aimed_for_deletion(deletion_cutoff)
+
+       projects.find_each(batch_size: 100).with_index do |project, index|
+         delay = index * INTERVAL
+
+         with_context(project: project) do
+           AdjournedProjectDeletionWorker.perform_in(delay, project.id)
+         end
+       end
+     end
+   ```
+
+1. Use the a batch scheduling method that provides context:
+
+   ```ruby
+     def schedule_projects_in_batch(projects)
+       ProjectImportScheduleWorker.bulk_perform_async_with_contexts(
+         projects,
+         arguments_proc: -> (project) { project.id },
+         context_proc: -> (project) { { project: project } }
+       )
+     end
+   ```
+
+   Or, when scheduling with delays:
+
+   ```ruby
+     diffs.each_batch(of: BATCH_SIZE) do |diffs, index|
+       DeleteDiffFilesWorker
+         .bulk_perform_in_with_contexts(index *  5.minutes,
+                                        diffs,
+                                        arguments_proc: -> (diff) { diff.id },
+                                        context_proc: -> (diff) { { project: diff.merge_request.target_project } })
+     end
+   ```
+
+### Jobs scheduled in bulk
+
+Often, when scheduling jobs in bulk, these jobs should have a separate
+context rather than the overarching context.
+
+If that is the case, `bulk_perform_async` can be replaced by the
+`bulk_perform_async_with_context` helper, and instead of
+`bulk_perform_in` use `bulk_perform_in_with_context`.
+
+For example:
+
+```ruby
+    ProjectImportScheduleWorker.bulk_perform_async_with_contexts(
+      projects,
+      arguments_proc: -> (project) { project.id },
+      context_proc: -> (project) { { project: project } }
+    )
+```
+
+Each object from the enumerable in the first argument is yielded into 2
+blocks:
+
+- The `arguments_proc` which needs to return the list of arguments the
+  job needs to be scheduled with.
+
+- The `context_proc` which needs to return a hash with the context
+  information for the job.
+
+## Arguments logging
+
+As of GitLab 13.6, Sidekiq job arguments are logged by default, unless [`SIDEKIQ_LOG_ARGUMENTS`](../../administration/troubleshooting/sidekiq.md#log-arguments-to-sidekiq-jobs)
+is disabled.
+
+By default, the only arguments logged are numeric arguments, because
+arguments of other types could contain sensitive information. To
+override this, use `loggable_arguments` inside a worker with the indexes
+of the arguments to be logged. (Numeric arguments do not need to be
+specified here.)
+
+For example:
+
+```ruby
+class MyWorker
+  include ApplicationWorker
+
+  loggable_arguments 1, 3
+
+  # object_id will be logged as it's numeric
+  # string_a will be logged due to the loggable_arguments call
+  # string_b will be filtered from logs
+  # string_c will be logged due to the loggable_arguments call
+  def perform(object_id, string_a, string_b, string_c)
+  end
+end
+```
diff --git a/doc/development/sidekiq/worker_attributes.md b/doc/development/sidekiq/worker_attributes.md
new file mode 100644
index 00000000000..d681e17a053
--- /dev/null
+++ b/doc/development/sidekiq/worker_attributes.md
@@ -0,0 +1,325 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Sidekiq worker attributes
+
+## Job urgency
+
+Jobs can have an `urgency` attribute set, which can be `:high`,
+`:low`, or `:throttled`. These have the below targets:
+
+| **Urgency**  | **Queue Scheduling Target** | **Execution Latency Requirement**  |
+|--------------|-----------------------------|------------------------------------|
+| `:high`      | 10 seconds                  | p50 of 1 second, p99 of 10 seconds |
+| `:low`       | 1 minute                    | Maximum run time of 5 minutes      |
+| `:throttled` | None                        | Maximum run time of 5 minutes      |
+
+To set a job's urgency, use the `urgency` class method:
+
+```ruby
+class HighUrgencyWorker
+  include ApplicationWorker
+
+  urgency :high
+
+  # ...
+end
+```
+
+### Latency sensitive jobs
+
+If a large number of background jobs get scheduled at once, queueing of jobs may
+occur while jobs wait for a worker node to be become available. This is normal
+and gives the system resilience by allowing it to gracefully handle spikes in
+traffic. Some jobs, however, are more sensitive to latency than others.
+
+In general, latency-sensitive jobs perform operations that a user could
+reasonably expect to happen synchronously, rather than asynchronously in a
+background worker. A common example is a write following an action. Examples of
+these jobs include:
+
+1. A job which updates a merge request following a push to a branch.
+1. A job which invalidates a cache of known branches for a project after a push
+   to the branch.
+1. A job which recalculates the groups and projects a user can see after a
+   change in permissions.
+1. A job which updates the status of a CI pipeline after a state change to a job
+   in the pipeline.
+
+When these jobs are delayed, the user may perceive the delay as a bug: for
+example, they may push a branch and then attempt to create a merge request for
+that branch, but be told in the UI that the branch does not exist. We deem these
+jobs to be `urgency :high`.
+
+Extra effort is made to ensure that these jobs are started within a very short
+period of time after being scheduled. However, in order to ensure throughput,
+these jobs also have very strict execution duration requirements:
+
+1. The median job execution time should be less than 1 second.
+1. 99% of jobs should complete within 10 seconds.
+
+If a worker cannot meet these expectations, then it cannot be treated as a
+`urgency :high` worker: consider redesigning the worker, or splitting the
+work between two different workers, one with `urgency :high` code that
+executes quickly, and the other with `urgency :low`, which has no
+execution latency requirements (but also has lower scheduling targets).
+
+### Changing a queue's urgency
+
+On GitLab.com, we run Sidekiq in several
+[shards](https://dashboards.gitlab.net/d/sidekiq-shard-detail/sidekiq-shard-detail),
+each of which represents a particular type of workload.
+
+When changing a queue's urgency, or adding a new queue, we need to take
+into account the expected workload on the new shard. Note that, if we're
+changing an existing queue, there is also an effect on the old shard,
+but that always reduces work.
+
+To do this, we want to calculate the expected increase in total execution time
+and RPS (throughput) for the new shard. We can get these values from:
+
+- The [Queue Detail
+  dashboard](https://dashboards.gitlab.net/d/sidekiq-queue-detail/sidekiq-queue-detail)
+  has values for the queue itself. For a new queue, we can look for
+  queues that have similar patterns or are scheduled in similar
+  circumstances.
+- The [Shard Detail
+  dashboard](https://dashboards.gitlab.net/d/sidekiq-shard-detail/sidekiq-shard-detail)
+  has Total Execution Time and Throughput (RPS). The Shard Utilization
+  panel displays if there is currently any excess capacity for this
+  shard.
+
+We can then calculate the RPS * average runtime (estimated for new jobs)
+for the queue we're changing to see what the relative increase in RPS and
+execution time we expect for the new shard:
+
+```ruby
+new_queue_consumption = queue_rps * queue_duration_avg
+shard_consumption = shard_rps * shard_duration_avg
+
+(new_queue_consumption / shard_consumption) * 100
+```
+
+If we expect an increase of **less than 5%**, then no further action is needed.
+
+Otherwise, please ping `@gitlab-org/scalability` on the merge request and ask
+for a review.
+
+## Jobs with External Dependencies
+
+Most background jobs in the GitLab application communicate with other GitLab
+services. For example, PostgreSQL, Redis, Gitaly, and Object Storage. These are considered
+to be "internal" dependencies for a job.
+
+However, some jobs are dependent on external services in order to complete
+successfully. Some examples include:
+
+1. Jobs which call web-hooks configured by a user.
+1. Jobs which deploy an application to a k8s cluster configured by a user.
+
+These jobs have "external dependencies". This is important for the operation of
+the background processing cluster in several ways:
+
+1. Most external dependencies (such as web-hooks) do not provide SLOs, and
+   therefore we cannot guarantee the execution latencies on these jobs. Since we
+   cannot guarantee execution latency, we cannot ensure throughput and
+   therefore, in high-traffic environments, we need to ensure that jobs with
+   external dependencies are separated from high urgency jobs, to ensure
+   throughput on those queues.
+1. Errors in jobs with external dependencies have higher alerting thresholds as
+   there is a likelihood that the cause of the error is external.
+
+```ruby
+class ExternalDependencyWorker
+  include ApplicationWorker
+
+  # Declares that this worker depends on
+  # third-party, external services in order
+  # to complete successfully
+  worker_has_external_dependencies!
+
+  # ...
+end
+```
+
+A job cannot be both high urgency and have external dependencies.
+
+## CPU-bound and Memory-bound Workers
+
+Workers that are constrained by CPU or memory resource limitations should be
+annotated with the `worker_resource_boundary` method.
+
+Most workers tend to spend most of their time blocked, waiting on network responses
+from other services such as Redis, PostgreSQL, and Gitaly. Since Sidekiq is a
+multi-threaded environment, these jobs can be scheduled with high concurrency.
+
+Some workers, however, spend large amounts of time _on-CPU_ running logic in
+Ruby. Ruby MRI does not support true multi-threading - it relies on the
+[GIL](https://thoughtbot.com/blog/untangling-ruby-threads#the-global-interpreter-lock)
+to greatly simplify application development by only allowing one section of Ruby
+code in a process to run at a time, no matter how many cores the machine
+hosting the process has. For IO bound workers, this is not a problem, since most
+of the threads are blocked in underlying libraries (which are outside of the
+GIL).
+
+If many threads are attempting to run Ruby code simultaneously, this leads
+to contention on the GIL which has the effect of slowing down all
+processes.
+
+In high-traffic environments, knowing that a worker is CPU-bound allows us to
+run it on a different fleet with lower concurrency. This ensures optimal
+performance.
+
+Likewise, if a worker uses large amounts of memory, we can run these on a
+bespoke low concurrency, high memory fleet.
+
+Note that memory-bound workers create heavy GC workloads, with pauses of
+10-50ms. This has an impact on the latency requirements for the
+worker. For this reason, `memory` bound, `urgency :high` jobs are not
+permitted and fail CI. In general, `memory` bound workers are
+discouraged, and alternative approaches to processing the work should be
+considered.
+
+If a worker needs large amounts of both memory and CPU time, it should
+be marked as memory-bound, due to the above restriction on high urgency
+memory-bound workers.
+
+## Declaring a Job as CPU-bound
+
+This example shows how to declare a job as being CPU-bound.
+
+```ruby
+class CPUIntensiveWorker
+  include ApplicationWorker
+
+  # Declares that this worker will perform a lot of
+  # calculations on-CPU.
+  worker_resource_boundary :cpu
+
+  # ...
+end
+```
+
+## Determining whether a worker is CPU-bound
+
+We use the following approach to determine whether a worker is CPU-bound:
+
+- In the Sidekiq structured JSON logs, aggregate the worker `duration` and
+  `cpu_s` fields.
+- `duration` refers to the total job execution duration, in seconds
+- `cpu_s` is derived from the
+  [`Process::CLOCK_THREAD_CPUTIME_ID`](https://www.rubydoc.info/stdlib/core/Process:clock_gettime)
+  counter, and is a measure of time spent by the job on-CPU.
+- Divide `cpu_s` by `duration` to get the percentage time spend on-CPU.
+- If this ratio exceeds 33%, the worker is considered CPU-bound and should be
+  annotated as such.
+- Note that these values should not be used over small sample sizes, but
+  rather over fairly large aggregates.
+
+## Feature category
+
+All Sidekiq workers must define a known [feature category](../feature_categorization/index.md#sidekiq-workers).
+
+## Job data consistency strategies
+
+In GitLab 13.11 and earlier, Sidekiq workers would always send database queries to the primary
+database node,
+both for reads and writes. This ensured that data integrity
+is both guaranteed and immediate, since in a single-node scenario it is impossible to encounter
+stale reads even for workers that read their own writes.
+If a worker writes to the primary, but reads from a replica, however, the possibility
+of reading a stale record is non-zero due to replicas potentially lagging behind the primary.
+
+When the number of jobs that rely on the database increases, ensuring immediate data consistency
+can put unsustainable load on the primary database server. We therefore added the ability to use
+[Database Load Balancing for Sidekiq workers](../../administration/postgresql/database_load_balancing.md).
+By configuring a worker's `data_consistency` field, we can then allow the scheduler to target read replicas
+under several strategies outlined below.
+
+## Trading immediacy for reduced primary load
+
+We require Sidekiq workers to make an explicit decision around whether they need to use the
+primary database node for all reads and writes, or whether reads can be served from replicas. This is
+enforced by a RuboCop rule, which ensures that the `data_consistency` field is set.
+
+When setting this field, consider the following trade-off:
+
+- Ensure immediately consistent reads, but increase load on the primary database.
+- Prefer read replicas to add relief to the primary, but increase the likelihood of stale reads that have to be retried.
+
+To maintain the same behavior compared to before this field was introduced, set it to `:always`, so
+database operations will only target the primary. Reasons for having to do so include workers
+that mostly or exclusively perform writes, or workers that read their own writes and who might run
+into data consistency issues should a stale record be read back from a replica. **Try to avoid
+these scenarios, since `:always` should be considered the exception, not the rule.**
+
+To allow for reads to be served from replicas, we added two additional consistency modes: `:sticky` and `:delayed`.
+
+When you declare either `:sticky` or `:delayed` consistency, workers become eligible for database
+load-balancing. 
+
+In both cases, if the replica is not up-to-date and the time from scheduling the job was less than the minimum delay interval,
+ the jobs sleep up to the minimum delay interval (0.8 seconds). This gives the replication process time to finish.
+The difference is in what happens when there is still replication lag after the delay: `sticky` workers
+switch over to the primary right away, whereas `delayed` workers fail fast and are retried once.
+If they still encounter replication lag, they also switch to the primary instead.
+**If your worker never performs any writes, it is strongly advised to apply one of these consistency settings,
+since it will never need to rely on the primary database node.**
+
+The table below shows the `data_consistency` attribute and its values, ordered by the degree to which
+they prefer read replicas and will wait for replicas to catch up:
+
+| **Data Consistency**  | **Description**  |
+|--------------|-----------------------------|
+| `:always`    | The job is required to use the primary database (default). It should be used for workers that primarily perform writes or that have strict requirements around data consistency when reading their own writes. |
+| `:sticky`    | The job prefers replicas, but switches to the primary for writes or when encountering replication lag. It should be used for jobs that require to be executed as fast as possible but can sustain a small initial queuing delay.  |
+| `:delayed`   | The job prefers replicas, but switches to the primary for writes. When encountering replication lag before the job starts, the job is retried once. If the replica is still not up to date on the next retry, it switches to the primary. It should be used for jobs where delaying execution further typically does not matter, such as cache expiration or web hooks execution. |
+
+In all cases workers read either from a replica that is fully caught up,
+or from the primary node, so data consistency is always ensured.
+
+To set a data consistency for a worker, use the `data_consistency` class method:
+
+```ruby
+class DelayedWorker
+  include ApplicationWorker
+
+  data_consistency :delayed
+
+  # ...
+end
+```
+
+### `feature_flag` property
+
+The `feature_flag` property allows you to toggle a job's `data_consistency`,
+which permits you to safely toggle load balancing capabilities for a specific job.
+When `feature_flag` is disabled, the job defaults to `:always`, which means that the job will always use the primary database.
+
+The `feature_flag` property does not allow the use of
+[feature gates based on actors](../feature_flags/index.md).
+This means that the feature flag cannot be toggled only for particular
+projects, groups, or users, but instead, you can safely use [percentage of time rollout](../feature_flags/index.md).
+Note that since we check the feature flag on both Sidekiq client and server, rolling out a 10% of the time,
+will likely results in 1% (`0.1` `[from client]*0.1` `[from server]`) of effective jobs using replicas.
+
+Example:
+
+```ruby
+class DelayedWorker
+  include ApplicationWorker
+
+  data_consistency :delayed, feature_flag: :load_balancing_for_delayed_worker
+
+  # ...
+end
+```
+
+### Data consistency with idempotent jobs
+
+For [idempotent jobs](idempotent_jobs.md) that declare either `:sticky` or `:delayed` data consistency, we are
+[preserving the latest WAL location](idempotent_jobs.md#preserve-the-latest-wal-location-for-idempotent-jobs) while deduplicating,
+ensuring that we read from the replica that is fully caught up.
diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md
index af994e7138d..3756dd6b598 100644
--- a/doc/development/sidekiq_style_guide.md
+++ b/doc/development/sidekiq_style_guide.md
@@ -1,1101 +1,9 @@
 ---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+redirect_to: 'sidekiq/index.md'
+remove_date: '2022-04-13'
 ---
 
-# Sidekiq Style Guide
+This document was moved to [another location](sidekiq/index.md).
 
-This document outlines various guidelines that should be followed when adding or
-modifying Sidekiq workers.
-
-## ApplicationWorker
-
-All workers should include `ApplicationWorker` instead of `Sidekiq::Worker`,
-which adds some convenience methods and automatically sets the queue based on
-the [routing rules](../administration/operations/extra_sidekiq_routing.md#queue-routing-rules).
-
-## Retries
-
-Sidekiq defaults to using [25
-retries](https://github.com/mperham/sidekiq/wiki/Error-Handling#automatic-job-retry),
-with back-off between each retry. 25 retries means that the last retry
-would happen around three weeks after the first attempt (assuming all 24
-prior retries failed).
-
-For most workers - especially [idempotent workers](#idempotent-jobs) -
-the default of 25 retries is more than sufficient. Many of our older
-workers declare 3 retries, which used to be the default within the
-GitLab application. 3 retries happen over the course of a couple of
-minutes, so the jobs are prone to failing completely.
-
-A lower retry count may be applicable if any of the below apply:
-
-1. The worker contacts an external service and we do not provide
-   guarantees on delivery. For example, webhooks.
-1. The worker is not idempotent and running it multiple times could
-   leave the system in an inconsistent state. For example, a worker that
-   posts a system note and then performs an action: if the second step
-   fails and the worker retries, the system note will be posted again.
-1. The worker is a cronjob that runs frequently. For example, if a cron
-   job runs every hour, then we don't need to retry beyond an hour
-   because we don't need two of the same job running at once.
-
-Each retry for a worker is counted as a failure in our metrics. A worker
-which always fails 9 times and succeeds on the 10th would have a 90%
-error rate.
-
-## Sidekiq Queues
-
-Previously, each worker had its own queue, which was automatically set based on the
-worker class name. For a worker named `ProcessSomethingWorker`, the queue name
-would be `process_something`. You can now route workers to a specific queue using
-[queue routing rules](../administration/operations/extra_sidekiq_routing.md#queue-routing-rules).
-In GDK, new workers are routed to a queue named `default`.
-
-If you're not sure what queue a worker uses,
-you can find it using `SomeWorker.queue`. There is almost never a reason to
-manually override the queue name using `sidekiq_options queue: :some_queue`.
-
-After adding a new worker, run `bin/rake
-gitlab:sidekiq:all_queues_yml:generate` to regenerate
-`app/workers/all_queues.yml` or `ee/app/workers/all_queues.yml` so that
-it can be picked up by
-[`sidekiq-cluster`](../administration/operations/extra_sidekiq_processes.md)
-in installations that don't use routing rules. To learn more about potential changes,
-read [Use routing rules by default and deprecate queue selectors for self-managed](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/596).
-
-Additionally, run
-`bin/rake gitlab:sidekiq:sidekiq_queues_yml:generate` to regenerate
-`config/sidekiq_queues.yml`.
-
-## Queue Namespaces
-
-While different workers cannot share a queue, they can share a queue namespace.
-
-Defining a queue namespace for a worker makes it possible to start a Sidekiq
-process that automatically handles jobs for all workers in that namespace,
-without needing to explicitly list all their queue names. If, for example, all
-workers that are managed by `sidekiq-cron` use the `cronjob` queue namespace, we
-can spin up a Sidekiq process specifically for these kinds of scheduled jobs.
-If a new worker using the `cronjob` namespace is added later on, the Sidekiq
-process also picks up jobs for that worker (after having been restarted),
-without the need to change any configuration.
-
-A queue namespace can be set using the `queue_namespace` DSL class method:
-
-```ruby
-class SomeScheduledTaskWorker
-  include ApplicationWorker
-
-  queue_namespace :cronjob
-
-  # ...
-end
-```
-
-Behind the scenes, this sets `SomeScheduledTaskWorker.queue` to
-`cronjob:some_scheduled_task`. Commonly used namespaces have their own
-concern module that can easily be included into the worker class, and that may
-set other Sidekiq options besides the queue namespace. `CronjobQueue`, for
-example, sets the namespace, but also disables retries.
-
-`bundle exec sidekiq` is namespace-aware, and listens on all
-queues in a namespace (technically: all queues prefixed with the namespace name)
-when a namespace is provided instead of a simple queue name in the `--queue`
-(`-q`) option, or in the `:queues:` section in `config/sidekiq_queues.yml`.
-
-Note that adding a worker to an existing namespace should be done with care, as
-the extra jobs take resources away from jobs from workers that were already
-there, if the resources available to the Sidekiq process handling the namespace
-are not adjusted appropriately.
-
-## Versioning
-
-Version can be specified on each Sidekiq worker class.
-This is then sent along when the job is created.
-
-```ruby
-class FooWorker
-  include ApplicationWorker
-
-  version 2
-
-  def perform(*args)
-    if job_version == 2
-      foo = args.first['foo']
-    else
-      foo = args.first
-    end
-  end
-end
-```
-
-Under this schema, any worker is expected to be able to handle any job that was
-enqueued by an older version of that worker. This means that when changing the
-arguments a worker takes, you must increment the `version` (or set `version 1`
-if this is the first time a worker's arguments are changing), but also make sure
-that the worker is still able to handle jobs that were queued with any earlier
-version of the arguments. From the worker's `perform` method, you can read
-`self.job_version` if you want to specifically branch on job version, or you
-can read the number or type of provided arguments.
-
-## Idempotent Jobs
-
-It's known that a job can fail for multiple reasons. For example, network outages or bugs.
-In order to address this, Sidekiq has a built-in retry mechanism that is
-used by default by most workers within GitLab.
-
-It's expected that a job can run again after a failure without major side-effects for the
-application or users, which is why Sidekiq encourages
-jobs to be [idempotent and transactional](https://github.com/mperham/sidekiq/wiki/Best-Practices#2-make-your-job-idempotent-and-transactional).
-
-As a general rule, a worker can be considered idempotent if:
-
-- It can safely run multiple times with the same arguments.
-- Application side-effects are expected to happen only once
-  (or side-effects of a second run do not have an effect).
-
-A good example of that would be a cache expiration worker.
-
-A job scheduled for an idempotent worker is [deduplicated](#deduplication) when
-an unstarted job with the same arguments is already in the queue.
-
-### Ensuring a worker is idempotent
-
-Make sure the worker tests pass using the following shared example:
-
-```ruby
-include_examples 'an idempotent worker' do
-  it 'marks the MR as merged' do
-    # Using subject inside this block will process the job multiple times
-    subject
-
-    expect(merge_request.state).to eq('merged')
-  end
-end
-```
-
-Use the `perform_multiple` method directly instead of `job.perform` (this
-helper method is automatically included for workers).
-
-### Declaring a worker as idempotent
-
-```ruby
-class IdempotentWorker
-  include ApplicationWorker
-
-  # Declares a worker is idempotent and can
-  # safely run multiple times.
-  idempotent!
-
-  # ...
-end
-```
-
-It's encouraged to only have the `idempotent!` call in the top-most worker class, even if
-the `perform` method is defined in another class or module.
-
-If the worker class isn't marked as idempotent, a cop fails. Consider skipping
-the cop if you're not confident your job can safely run multiple times.
-
-### Deduplication
-
-When a job for an idempotent worker is enqueued while another
-unstarted job is already in the queue, GitLab drops the second
-job. The work is skipped because the same work would be
-done by the job that was scheduled first; by the time the second
-job executed, the first job would do nothing.
-
-#### Strategies
-
-GitLab supports two deduplication strategies:
-
-- `until_executing`
-- `until_executed`
-
-More [deduplication strategies have been
-suggested](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/195). If
-you are implementing a worker that could benefit from a different
-strategy, please comment in the issue.
-
-##### Until Executing
-
-This strategy takes a lock when a job is added to the queue, and removes that lock before the job starts.
-
-For example, `AuthorizedProjectsWorker` takes a user ID. When the
-worker runs, it recalculates a user's authorizations. GitLab schedules
-this job each time an action potentially changes a user's
-authorizations. If the same user is added to two projects at the
-same time, the second job can be skipped if the first job hasn't
-begun, because when the first job runs, it creates the
-authorizations for both projects.
-
-```ruby
-module AuthorizedProjectUpdate
-  class UserRefreshOverUserRangeWorker
-    include ApplicationWorker
-
-    deduplicate :until_executing
-    idempotent!
-
-    # ...
-  end
-end
-```
-
-##### Until Executed
-
-This strategy takes a lock when a job is added to the queue, and removes that lock after the job finishes.
-It can be used to prevent jobs from running simultaneously multiple times.
-
-```ruby
-module Ci
-  class BuildTraceChunkFlushWorker
-    include ApplicationWorker
-
-    deduplicate :until_executed
-    idempotent!
-
-    # ...
-  end
-end
-```
-
-Also, you can pass `if_deduplicated: :reschedule_once` option to re-run a job once after
-the currently running job finished and deduplication happened at least once.
-This ensures that the latest result is always produced even if a race condition
-happened. See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342123) for more information.
-
-#### Scheduling jobs in the future
-
-GitLab doesn't skip jobs scheduled in the future, as we assume that
-the state has changed by the time the job is scheduled to
-execute. Deduplication of jobs scheduled in the feature is possible
-for both `until_executed` and `until_executing` strategies.
-
-If you do want to deduplicate jobs scheduled in the future,
-this can be specified on the worker by passing `including_scheduled: true` argument
-when defining deduplication strategy:
-
-```ruby
-module AuthorizedProjectUpdate
-  class UserRefreshOverUserRangeWorker
-    include ApplicationWorker
-
-    deduplicate :until_executing, including_scheduled: true
-    idempotent!
-
-    # ...
-  end
-end
-```
-
-### Setting the deduplication time-to-live (TTL)
-
-Deduplication depends on an idempotency key that is stored in Redis. This is normally
-cleared by the configured deduplication strategy.
-
-However, the key can remain until its TTL in certain cases like:
-
-1. `until_executing` is used but the job was never enqueued or executed after the Sidekiq
-   client middleware was run.
-
-1. `until_executed` is used but the job fails to finish due to retry exhaustion, gets
-   interrupted the maximum number of times, or gets lost.
-
-The default value is 6 hours. During this time, jobs won't be enqueued even if the first
-job never executed or finished.
-
-The TTL can be configured with:
-
-```ruby
-class ProjectImportScheduleWorker
-  include ApplicationWorker
-
-  idempotent!
-  deduplicate :until_executing, ttl: 5.minutes
-end
-```
-
-Duplicate jobs can happen when the TTL is reached, so make sure you lower this only for jobs
-that can tolerate some duplication.
-
-### Deduplication with load balancing
-
-> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6763) in GitLab 14.4.
-
-Jobs that declare either `:sticky` or `:delayed` data consistency
-are eligible for database load-balancing.
-In both cases, jobs are [scheduled in the future](#scheduling-jobs-in-the-future) with a short delay (1 second).
-This minimizes the chance of replication lag after a write.
-
-If you really want to deduplicate jobs eligible for load balancing,
-specify `including_scheduled: true` argument when defining deduplication strategy:
-
-```ruby
-class DelayedIdempotentWorker
-  include ApplicationWorker
-  data_consistency :delayed
-
-  deduplicate :until_executing, including_scheduled: true
-  idempotent!
-
-  # ...
-end
-```
-
-#### Preserve the latest WAL location for idempotent jobs
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69372) in GitLab 14.3.
-> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/338350) in GitLab 14.4.
-> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/338350) in GitLab 14.6.
-
-The deduplication always take into account the latest binary replication pointer, not the first one.
-This happens because we drop the same job scheduled for the second time and the Write-Ahead Log (WAL) is lost.
-This could lead to comparing the old WAL location and reading from a stale replica.
-
-To support both deduplication and maintaining data consistency with load balancing,
-we are preserving the latest WAL location for idempotent jobs in Redis.
-This way we are always comparing the latest binary replication pointer,
-making sure that we read from the replica that is fully caught up.
-
-FLAG:
-On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to
-[disable the feature flag](../administration/feature_flags.md) named `preserve_latest_wal_locations_for_idempotent_jobs`.
-
-This feature flag is related to GitLab development and is not intended to be used by GitLab administrators, though.
-On GitLab.com, this feature is available.
-
-## Limited capacity worker
-
-It is possible to limit the number of concurrent running jobs for a worker class
-by using the `LimitedCapacity::Worker` concern.
-
-The worker must implement three methods:
-
-- `perform_work`: The concern implements the usual `perform` method and calls
-  `perform_work` if there's any available capacity.
-- `remaining_work_count`: Number of jobs that have work to perform.
-- `max_running_jobs`: Maximum number of jobs allowed to run concurrently.
-
-```ruby
-class MyDummyWorker
-  include ApplicationWorker
-  include LimitedCapacity::Worker
-
-  def perform_work(*args)
-  end
-
-  def remaining_work_count(*args)
-    5
-  end
-
-  def max_running_jobs
-    25
-  end
-end
-```
-
-Additional to the regular worker, a cron worker must be defined as well to
-backfill the queue with jobs. the arguments passed to `perform_with_capacity`
-are passed to the `perform_work` method.
-
-```ruby
-class ScheduleMyDummyCronWorker
-  include ApplicationWorker
-  include CronjobQueue
-
-  def perform(*args)
-    MyDummyWorker.perform_with_capacity(*args)
-  end
-end
-```
-
-### How many jobs are running?
-
-It runs `max_running_jobs` at almost all times.
-
-The cron worker checks the remaining capacity on each execution and it
-schedules at most `max_running_jobs` jobs. Those jobs on completion
-re-enqueue themselves immediately, but not on failure. The cron worker is in
-charge of replacing those failed jobs.
-
-### Handling errors and idempotence
-
-This concern disables Sidekiq retries, logs the errors, and sends the job to the
-dead queue. This is done to have only one source that produces jobs and because
-the retry would occupy a slot with a job to perform in the distant future.
-
-We let the cron worker enqueue new jobs, this could be seen as our retry and
-back off mechanism because the job might fail again if executed immediately.
-This means that for every failed job, we run at a lower capacity
-until the cron worker fills the capacity again. If it is important for the
-worker not to get a backlog, exceptions must be handled in `#perform_work` and
-the job should not raise.
-
-The jobs are deduplicated using the `:none` strategy, but the worker is not
-marked as `idempotent!`.
-
-### Metrics
-
-This concern exposes three Prometheus metrics of gauge type with the worker class
-name as label:
-
-- `limited_capacity_worker_running_jobs`
-- `limited_capacity_worker_max_running_jobs`
-- `limited_capacity_worker_remaining_work_count`
-
-## Job urgency
-
-Jobs can have an `urgency` attribute set, which can be `:high`,
-`:low`, or `:throttled`. These have the below targets:
-
-| **Urgency**  | **Queue Scheduling Target** | **Execution Latency Requirement**  |
-|--------------|-----------------------------|------------------------------------|
-| `:high`      | 10 seconds                  | p50 of 1 second, p99 of 10 seconds |
-| `:low`       | 1 minute                    | Maximum run time of 5 minutes      |
-| `:throttled` | None                        | Maximum run time of 5 minutes      |
-
-To set a job's urgency, use the `urgency` class method:
-
-```ruby
-class HighUrgencyWorker
-  include ApplicationWorker
-
-  urgency :high
-
-  # ...
-end
-```
-
-### Latency sensitive jobs
-
-If a large number of background jobs get scheduled at once, queueing of jobs may
-occur while jobs wait for a worker node to be become available. This is normal
-and gives the system resilience by allowing it to gracefully handle spikes in
-traffic. Some jobs, however, are more sensitive to latency than others.
-
-In general, latency-sensitive jobs perform operations that a user could
-reasonably expect to happen synchronously, rather than asynchronously in a
-background worker. A common example is a write following an action. Examples of
-these jobs include:
-
-1. A job which updates a merge request following a push to a branch.
-1. A job which invalidates a cache of known branches for a project after a push
-   to the branch.
-1. A job which recalculates the groups and projects a user can see after a
-   change in permissions.
-1. A job which updates the status of a CI pipeline after a state change to a job
-   in the pipeline.
-
-When these jobs are delayed, the user may perceive the delay as a bug: for
-example, they may push a branch and then attempt to create a merge request for
-that branch, but be told in the UI that the branch does not exist. We deem these
-jobs to be `urgency :high`.
-
-Extra effort is made to ensure that these jobs are started within a very short
-period of time after being scheduled. However, in order to ensure throughput,
-these jobs also have very strict execution duration requirements:
-
-1. The median job execution time should be less than 1 second.
-1. 99% of jobs should complete within 10 seconds.
-
-If a worker cannot meet these expectations, then it cannot be treated as a
-`urgency :high` worker: consider redesigning the worker, or splitting the
-work between two different workers, one with `urgency :high` code that
-executes quickly, and the other with `urgency :low`, which has no
-execution latency requirements (but also has lower scheduling targets).
-
-### Changing a queue's urgency
-
-On GitLab.com, we run Sidekiq in several
-[shards](https://dashboards.gitlab.net/d/sidekiq-shard-detail/sidekiq-shard-detail),
-each of which represents a particular type of workload.
-
-When changing a queue's urgency, or adding a new queue, we need to take
-into account the expected workload on the new shard. Note that, if we're
-changing an existing queue, there is also an effect on the old shard,
-but that always reduces work.
-
-To do this, we want to calculate the expected increase in total execution time
-and RPS (throughput) for the new shard. We can get these values from:
-
-- The [Queue Detail
-  dashboard](https://dashboards.gitlab.net/d/sidekiq-queue-detail/sidekiq-queue-detail)
-  has values for the queue itself. For a new queue, we can look for
-  queues that have similar patterns or are scheduled in similar
-  circumstances.
-- The [Shard Detail
-  dashboard](https://dashboards.gitlab.net/d/sidekiq-shard-detail/sidekiq-shard-detail)
-  has Total Execution Time and Throughput (RPS). The Shard Utilization
-  panel displays if there is currently any excess capacity for this
-  shard.
-
-We can then calculate the RPS * average runtime (estimated for new jobs)
-for the queue we're changing to see what the relative increase in RPS and
-execution time we expect for the new shard:
-
-```ruby
-new_queue_consumption = queue_rps * queue_duration_avg
-shard_consumption = shard_rps * shard_duration_avg
-
-(new_queue_consumption / shard_consumption) * 100
-```
-
-If we expect an increase of **less than 5%**, then no further action is needed.
-
-Otherwise, please ping `@gitlab-org/scalability` on the merge request and ask
-for a review.
-
-## Job size
-
-GitLab stores Sidekiq jobs and their arguments in Redis. To avoid
-excessive memory usage, we compress the arguments of Sidekiq jobs
-if their original size is bigger than 100KB.
-
-After compression, if their size still exceeds 5MB, it raises an
-[`ExceedLimitError`](https://gitlab.com/gitlab-org/gitlab/-/blob/f3dd89e5e510ea04b43ffdcb58587d8f78a8d77c/lib/gitlab/sidekiq_middleware/size_limiter/exceed_limit_error.rb#L8)
-error when scheduling the job.
-
-If this happens, rely on other means of making the data
-available in Sidekiq. There are possible workarounds such as:
-
-- Rebuild the data in Sidekiq with data loaded from the database or
-  elsewhere.
-- Store the data in [object storage](file_storage.md#object-storage)
-  before scheduling the job, and retrieve it inside the job.
-
-## Job data consistency strategies
-
-In GitLab 13.11 and earlier, Sidekiq workers would always send database queries to the primary
-database node,
-both for reads and writes. This ensured that data integrity
-is both guaranteed and immediate, since in a single-node scenario it is impossible to encounter
-stale reads even for workers that read their own writes.
-If a worker writes to the primary, but reads from a replica, however, the possibility
-of reading a stale record is non-zero due to replicas potentially lagging behind the primary.
-
-When the number of jobs that rely on the database increases, ensuring immediate data consistency
-can put unsustainable load on the primary database server. We therefore added the ability to use
-[Database Load Balancing for Sidekiq workers](../administration/postgresql/database_load_balancing.md).
-By configuring a worker's `data_consistency` field, we can then allow the scheduler to target read replicas
-under several strategies outlined below.
-
-## Trading immediacy for reduced primary load
-
-We require Sidekiq workers to make an explicit decision around whether they need to use the
-primary database node for all reads and writes, or whether reads can be served from replicas. This is
-enforced by a RuboCop rule, which ensures that the `data_consistency` field is set.
-
-When setting this field, consider the following trade-off:
-
-- Ensure immediately consistent reads, but increase load on the primary database.
-- Prefer read replicas to add relief to the primary, but increase the likelihood of stale reads that have to be retried.
-
-To maintain the same behavior compared to before this field was introduced, set it to `:always`, so
-database operations will only target the primary. Reasons for having to do so include workers
-that mostly or exclusively perform writes, or workers that read their own writes and who might run
-into data consistency issues should a stale record be read back from a replica. **Try to avoid
-these scenarios, since `:always` should be considered the exception, not the rule.**
-
-To allow for reads to be served from replicas, we added two additional consistency modes: `:sticky` and `:delayed`.
-
-When you declare either `:sticky` or `:delayed` consistency, workers become eligible for database
-load-balancing. In both cases, jobs are enqueued with a short delay.
-This minimizes the likelihood of replication lag after a write.
-
-The difference is in what happens when there is replication lag after the delay: `sticky` workers
-switch over to the primary right away, whereas `delayed` workers fail fast and are retried once.
-If they still encounter replication lag, they also switch to the primary instead.
-**If your worker never performs any writes, it is strongly advised to apply one of these consistency settings,
-since it will never need to rely on the primary database node.**
-
-The table below shows the `data_consistency` attribute and its values, ordered by the degree to which
-they prefer read replicas and will wait for replicas to catch up:
-
-| **Data Consistency**  | **Description**  |
-|--------------|-----------------------------|
-| `:always`    | The job is required to use the primary database (default). It should be used for workers that primarily perform writes or that have strict requirements around data consistency when reading their own writes. |
-| `:sticky`    | The job prefers replicas, but switches to the primary for writes or when encountering replication lag. It should be used for jobs that require to be executed as fast as possible but can sustain a small initial queuing delay.  |
-| `:delayed`   | The job prefers replicas, but switches to the primary for writes. When encountering replication lag before the job starts, the job is retried once. If the replica is still not up to date on the next retry, it switches to the primary. It should be used for jobs where delaying execution further typically does not matter, such as cache expiration or web hooks execution. |
-
-In all cases workers read either from a replica that is fully caught up,
-or from the primary node, so data consistency is always ensured.
-
-To set a data consistency for a worker, use the `data_consistency` class method:
-
-```ruby
-class DelayedWorker
-  include ApplicationWorker
-
-  data_consistency :delayed
-
-  # ...
-end
-```
-
-### `feature_flag` property
-
-The `feature_flag` property allows you to toggle a job's `data_consistency`,
-which permits you to safely toggle load balancing capabilities for a specific job.
-When `feature_flag` is disabled, the job defaults to `:always`, which means that the job will always use the primary database.
-
-The `feature_flag` property does not allow the use of
-[feature gates based on actors](../development/feature_flags/index.md).
-This means that the feature flag cannot be toggled only for particular
-projects, groups, or users, but instead, you can safely use [percentage of time rollout](../development/feature_flags/index.md).
-Note that since we check the feature flag on both Sidekiq client and server, rolling out a 10% of the time,
-will likely results in 1% (`0.1` `[from client]*0.1` `[from server]`) of effective jobs using replicas.
-
-Example:
-
-```ruby
-class DelayedWorker
-  include ApplicationWorker
-
-  data_consistency :delayed, feature_flag: :load_balancing_for_delayed_worker
-
-  # ...
-end
-```
-
-### Data consistency with idempotent jobs
-
-For [idempotent jobs](#idempotent-jobs) that declare either `:sticky` or `:delayed` data consistency, we are
-[preserving the latest WAL location](#preserve-the-latest-wal-location-for-idempotent-jobs) while deduplicating,
-ensuring that we read from the replica that is fully caught up.
-
-## Jobs with External Dependencies
-
-Most background jobs in the GitLab application communicate with other GitLab
-services. For example, PostgreSQL, Redis, Gitaly, and Object Storage. These are considered
-to be "internal" dependencies for a job.
-
-However, some jobs are dependent on external services in order to complete
-successfully. Some examples include:
-
-1. Jobs which call web-hooks configured by a user.
-1. Jobs which deploy an application to a k8s cluster configured by a user.
-
-These jobs have "external dependencies". This is important for the operation of
-the background processing cluster in several ways:
-
-1. Most external dependencies (such as web-hooks) do not provide SLOs, and
-   therefore we cannot guarantee the execution latencies on these jobs. Since we
-   cannot guarantee execution latency, we cannot ensure throughput and
-   therefore, in high-traffic environments, we need to ensure that jobs with
-   external dependencies are separated from high urgency jobs, to ensure
-   throughput on those queues.
-1. Errors in jobs with external dependencies have higher alerting thresholds as
-   there is a likelihood that the cause of the error is external.
-
-```ruby
-class ExternalDependencyWorker
-  include ApplicationWorker
-
-  # Declares that this worker depends on
-  # third-party, external services in order
-  # to complete successfully
-  worker_has_external_dependencies!
-
-  # ...
-end
-```
-
-A job cannot be both high urgency and have external dependencies.
-
-## CPU-bound and Memory-bound Workers
-
-Workers that are constrained by CPU or memory resource limitations should be
-annotated with the `worker_resource_boundary` method.
-
-Most workers tend to spend most of their time blocked, waiting on network responses
-from other services such as Redis, PostgreSQL, and Gitaly. Since Sidekiq is a
-multi-threaded environment, these jobs can be scheduled with high concurrency.
-
-Some workers, however, spend large amounts of time _on-CPU_ running logic in
-Ruby. Ruby MRI does not support true multi-threading - it relies on the
-[GIL](https://thoughtbot.com/blog/untangling-ruby-threads#the-global-interpreter-lock)
-to greatly simplify application development by only allowing one section of Ruby
-code in a process to run at a time, no matter how many cores the machine
-hosting the process has. For IO bound workers, this is not a problem, since most
-of the threads are blocked in underlying libraries (which are outside of the
-GIL).
-
-If many threads are attempting to run Ruby code simultaneously, this leads
-to contention on the GIL which has the effect of slowing down all
-processes.
-
-In high-traffic environments, knowing that a worker is CPU-bound allows us to
-run it on a different fleet with lower concurrency. This ensures optimal
-performance.
-
-Likewise, if a worker uses large amounts of memory, we can run these on a
-bespoke low concurrency, high memory fleet.
-
-Note that memory-bound workers create heavy GC workloads, with pauses of
-10-50ms. This has an impact on the latency requirements for the
-worker. For this reason, `memory` bound, `urgency :high` jobs are not
-permitted and fail CI. In general, `memory` bound workers are
-discouraged, and alternative approaches to processing the work should be
-considered.
-
-If a worker needs large amounts of both memory and CPU time, it should
-be marked as memory-bound, due to the above restriction on high urgency
-memory-bound workers.
-
-## Declaring a Job as CPU-bound
-
-This example shows how to declare a job as being CPU-bound.
-
-```ruby
-class CPUIntensiveWorker
-  include ApplicationWorker
-
-  # Declares that this worker will perform a lot of
-  # calculations on-CPU.
-  worker_resource_boundary :cpu
-
-  # ...
-end
-```
-
-## Determining whether a worker is CPU-bound
-
-We use the following approach to determine whether a worker is CPU-bound:
-
-- In the Sidekiq structured JSON logs, aggregate the worker `duration` and
-  `cpu_s` fields.
-- `duration` refers to the total job execution duration, in seconds
-- `cpu_s` is derived from the
-  [`Process::CLOCK_THREAD_CPUTIME_ID`](https://www.rubydoc.info/stdlib/core/Process:clock_gettime)
-  counter, and is a measure of time spent by the job on-CPU.
-- Divide `cpu_s` by `duration` to get the percentage time spend on-CPU.
-- If this ratio exceeds 33%, the worker is considered CPU-bound and should be
-  annotated as such.
-- Note that these values should not be used over small sample sizes, but
-  rather over fairly large aggregates.
-
-## Feature category
-
-All Sidekiq workers must define a known [feature category](feature_categorization/index.md#sidekiq-workers).
-
-## Job weights
-
-Some jobs have a weight declared. This is only used when running Sidekiq
-in the default execution mode - using
-[`sidekiq-cluster`](../administration/operations/extra_sidekiq_processes.md)
-does not account for weights.
-
-As we are [moving towards using `sidekiq-cluster` in
-Free](https://gitlab.com/gitlab-org/gitlab/-/issues/34396), newly-added
-workers do not need to have weights specified. They can use the
-default weight, which is 1.
-
-## Worker context
-
-> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/9) in GitLab 12.8.
-
-To have some more information about workers in the logs, we add
-[metadata to the jobs in the form of an
-`ApplicationContext`](logging.md#logging-context-metadata-through-rails-or-grape-requests).
-In most cases, when scheduling a job from a request, this context is already
-deducted from the request and added to the scheduled job.
-
-When a job runs, the context that was active when it was scheduled
-is restored. This causes the context to be propagated to any job
-scheduled from within the running job.
-
-All this means that in most cases, to add context to jobs, we don't
-need to do anything.
-
-There are however some instances when there would be no context
-present when the job is scheduled, or the context that is present is
-likely to be incorrect. For these instances, we've added Rubocop rules
-to draw attention and avoid incorrect metadata in our logs.
-
-As with most our cops, there are perfectly valid reasons for disabling
-them. In this case it could be that the context from the request is
-correct. Or maybe you've specified a context already in a way that
-isn't picked up by the cops. In any case, leave a code comment
-pointing to which context to use when disabling the cops.
-
-When you do provide objects to the context, make sure that the
-route for namespaces and projects is pre-loaded. This can be done by using
-the `.with_route` scope defined on all `Routable`s.
-
-### Cron workers
-
-The context is automatically cleared for workers in the cronjob queue
-(`include CronjobQueue`), even when scheduling them from
-requests. We do this to avoid incorrect metadata when other jobs are
-scheduled from the cron worker.
-
-Cron workers themselves run instance wide, so they aren't scoped to
-users, namespaces, projects, or other resources that should be added to
-the context.
-
-However, they often schedule other jobs that _do_ require context.
-
-That is why there needs to be an indication of context somewhere in
-the worker. This can be done by using one of the following methods
-somewhere within the worker:
-
-1. Wrap the code that schedules jobs in the `with_context` helper:
-
-   ```ruby
-     def perform
-       deletion_cutoff = Gitlab::CurrentSettings
-                           .deletion_adjourned_period.days.ago.to_date
-       projects = Project.with_route.with_namespace
-                    .aimed_for_deletion(deletion_cutoff)
-
-       projects.find_each(batch_size: 100).with_index do |project, index|
-         delay = index * INTERVAL
-
-         with_context(project: project) do
-           AdjournedProjectDeletionWorker.perform_in(delay, project.id)
-         end
-       end
-     end
-   ```
-
-1. Use the a batch scheduling method that provides context:
-
-   ```ruby
-     def schedule_projects_in_batch(projects)
-       ProjectImportScheduleWorker.bulk_perform_async_with_contexts(
-         projects,
-         arguments_proc: -> (project) { project.id },
-         context_proc: -> (project) { { project: project } }
-       )
-     end
-   ```
-
-   Or, when scheduling with delays:
-
-   ```ruby
-     diffs.each_batch(of: BATCH_SIZE) do |diffs, index|
-       DeleteDiffFilesWorker
-         .bulk_perform_in_with_contexts(index *  5.minutes,
-                                        diffs,
-                                        arguments_proc: -> (diff) { diff.id },
-                                        context_proc: -> (diff) { { project: diff.merge_request.target_project } })
-     end
-   ```
-
-### Jobs scheduled in bulk
-
-Often, when scheduling jobs in bulk, these jobs should have a separate
-context rather than the overarching context.
-
-If that is the case, `bulk_perform_async` can be replaced by the
-`bulk_perform_async_with_context` helper, and instead of
-`bulk_perform_in` use `bulk_perform_in_with_context`.
-
-For example:
-
-```ruby
-    ProjectImportScheduleWorker.bulk_perform_async_with_contexts(
-      projects,
-      arguments_proc: -> (project) { project.id },
-      context_proc: -> (project) { { project: project } }
-    )
-```
-
-Each object from the enumerable in the first argument is yielded into 2
-blocks:
-
-- The `arguments_proc` which needs to return the list of arguments the
-  job needs to be scheduled with.
-
-- The `context_proc` which needs to return a hash with the context
-  information for the job.
-
-## Arguments logging
-
-As of GitLab 13.6, Sidekiq job arguments are logged by default, unless [`SIDEKIQ_LOG_ARGUMENTS`](../administration/troubleshooting/sidekiq.md#log-arguments-to-sidekiq-jobs)
-is disabled.
-
-By default, the only arguments logged are numeric arguments, because
-arguments of other types could contain sensitive information. To
-override this, use `loggable_arguments` inside a worker with the indexes
-of the arguments to be logged. (Numeric arguments do not need to be
-specified here.)
-
-For example:
-
-```ruby
-class MyWorker
-  include ApplicationWorker
-
-  loggable_arguments 1, 3
-
-  # object_id will be logged as it's numeric
-  # string_a will be logged due to the loggable_arguments call
-  # string_b will be filtered from logs
-  # string_c will be logged due to the loggable_arguments call
-  def perform(object_id, string_a, string_b, string_c)
-  end
-end
-```
-
-## Tests
-
-Each Sidekiq worker must be tested using RSpec, just like any other class. These
-tests should be placed in `spec/workers`.
-
-## Sidekiq Compatibility across Updates
-
-Keep in mind that the arguments for a Sidekiq job are stored in a queue while it
-is scheduled for execution. During a online update, this could lead to several
-possible situations:
-
-1. An older version of the application publishes a job, which is executed by an
-   upgraded Sidekiq node.
-1. A job is queued before an upgrade, but executed after an upgrade.
-1. A job is queued by a node running the newer version of the application, but
-   executed on a node running an older version of the application.
-
-### Adding new workers
-
-On GitLab.com, we [do not currently have a Sidekiq deployment in the
-canary stage](https://gitlab.com/gitlab-org/gitlab/-/issues/19239). This
-means that a new worker than can be scheduled from an HTTP endpoint may
-be scheduled from canary but not run on Sidekiq until the full
-production deployment is complete. This can be several hours later than
-scheduling the job. For some workers, this will not be a problem. For
-others - particularly [latency-sensitive
-jobs](#latency-sensitive-jobs) - this will result in a poor user
-experience.
-
-This only applies to new worker classes when they are first introduced.
-As we recommend [using feature flags](feature_flags/) as a general
-development process, it's best to control the entire change (including
-scheduling of the new Sidekiq worker) with a feature flag.
-
-### Changing the arguments for a worker
-
-Jobs need to be backward and forward compatible between consecutive versions
-of the application. Adding or removing an argument may cause problems
-during deployment before all Rails and Sidekiq nodes have the updated code.
-
-#### Deprecate and remove an argument
-
-**Before you remove arguments from the `perform_async` and `perform` methods.**, deprecate them. The
-following example deprecates and then removes `arg2` from the `perform_async` method:
-
-1. Provide a default value (usually `nil`) and use a comment to mark the
-   argument as deprecated in the coming minor release. (Release M)
-
-    ```ruby
-    class ExampleWorker
-      # Keep arg2 parameter for backwards compatibility.
-      def perform(object_id, arg1, arg2 = nil)
-        # ...
-      end
-    end
-    ```
-
-1. One minor release later, stop using the argument in `perform_async`. (Release M+1)
-
-    ```ruby
-    ExampleWorker.perform_async(object_id, arg1)
-    ```
-
-1. At the next major release, remove the value from the worker class. (Next major release)
-
-    ```ruby
-    class ExampleWorker
-      def perform(object_id, arg1)
-        # ...
-      end
-    end
-    ```
-
-#### Add an argument
-
-There are two options for safely adding new arguments to Sidekiq workers:
-
-1. Set up a [multi-step deployment](#multi-step-deployment) in which the new argument is first added to the worker.
-1. Use a [parameter hash](#parameter-hash) for additional arguments. This is perhaps the most flexible option.
-
-##### Multi-step deployment
-
-This approach requires multiple releases.
-
-1. Add the argument to the worker with a default value (Release M).
-
-    ```ruby
-    class ExampleWorker
-      def perform(object_id, new_arg = nil)
-        # ...
-      end
-    end
-    ```
-
-1. Add the new argument to all the invocations of the worker (Release M+1).
-
-    ```ruby
-    ExampleWorker.perform_async(object_id, new_arg)
-    ```
-
-1. Remove the default value (Release M+2).
-
-    ```ruby
-    class ExampleWorker
-      def perform(object_id, new_arg)
-        # ...
-      end
-    end
-    ```
-
-##### Parameter hash
-
-This approach doesn't require multiple releases if an existing worker already
-uses a parameter hash.
-
-1. Use a parameter hash in the worker to allow future flexibility.
-
-    ```ruby
-    class ExampleWorker
-      def perform(object_id, params = {})
-        # ...
-      end
-    end
-    ```
-
-### Removing workers
-
-Try to avoid removing workers and their queues in minor and patch
-releases.
-
-During online update instance can have pending jobs and removing the queue can
-lead to those jobs being stuck forever. If you can't write migration for those
-Sidekiq jobs, please consider removing the worker in a major release only.
-
-### Renaming queues
-
-For the same reasons that removing workers is dangerous, care should be taken
-when renaming queues.
-
-When renaming queues, use the `sidekiq_queue_migrate` helper migration method
-in a **post-deployment migration**:
-
-```ruby
-class MigrateTheRenamedSidekiqQueue < Gitlab::Database::Migration[1.0]
-  def up
-    sidekiq_queue_migrate 'old_queue_name', to: 'new_queue_name'
-  end
-
-  def down
-    sidekiq_queue_migrate 'new_queue_name', to: 'old_queue_name'
-  end
-end
-
-```
-
-You must rename the queue in a post-deployment migration not in a normal
-migration. Otherwise, it runs too early, before all the workers that
-schedule these jobs have stopped running. See also [other examples](post_deployment_migrations.md#use-cases).
+<!-- This redirect file can be deleted after <2022-04-13>. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
diff --git a/doc/development/snowplow/implementation.md b/doc/development/snowplow/implementation.md
index 439485c9e73..d35413cfd5f 100644
--- a/doc/development/snowplow/implementation.md
+++ b/doc/development/snowplow/implementation.md
@@ -18,21 +18,27 @@ to track custom events.
 
 For the recommended frontend tracking implementation, see [Usage recommendations](#usage-recommendations).
 
+Structured events and page views include the [`gitlab_standard`](schemas.md#gitlab_standard)
+context, using the `window.gl.snowplowStandardContext` object which includes
+[default data](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/views/layouts/_snowplow.html.haml)
+as base. This object can be modified for any subsequent structured event fired,
+although it's not recommended.
+
 Tracking implementations must have an `action` and a `category`. You can provide additional
-categories from the [structured event taxonomy](index.md#structured-event-taxonomy) with an `extra` object
-that accepts key-value pairs.
+properties from the [structured event taxonomy](index.md#structured-event-taxonomy), in
+addition to an `extra` object that accepts key-value pairs.
 
-| Field      | Type   | Default value              | Description                                                                                                                                                                                                    |
+| Property      | Type   | Default value              | Description                                                                                                                                                                                                    |
 |:-----------|:-------|:---------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | `category` | string | `document.body.dataset.page` | Page or subsection of a page in which events are captured.                                                                                                                                            |
-| `action`   | string | generic                  | Action the user is taking. Clicks must be `click` and activations must be `activate`. For example, focusing a form field is `activate_form_input`, and clicking a button is `click_button`. |
-| `data`     | object | `{}`                         | Additional data such as `label`, `property`, `value`, `context` as described in [Structured event taxonomy](index.md#structured-event-taxonomy), and `extra` (key-value pairs object). |
+| `action`   | string | `'generic'`                  | Action the user is taking. Clicks must be `click` and activations must be `activate`. For example, focusing a form field is `activate_form_input`, and clicking a button is `click_button`. |
+| `data`     | object | `{}`                         | Additional data such as `label`, `property`, `value` as described in [Structured event taxonomy](index.md#structured-event-taxonomy), `context` for custom contexts, and `extra` (key-value pairs object). |
 
 ### Usage recommendations
 
 - Use [data attributes](#implement-data-attribute-tracking) on HTML elements that emit `click`, `show.bs.dropdown`, or `hide.bs.dropdown` events.
-- Use the [Vue mixin](#implement-vue-component-tracking) for tracking custom events, or if the supported events for data attributes are not propagating.
-- Use the [tracking class](#implement-raw-javascript-tracking) when tracking raw JavaScript files.
+- Use the [Vue mixin](#implement-vue-component-tracking) for tracking custom events, or if the supported events for data attributes are not propagating. For example, clickable components that don't emit `click`.
+- Use the [tracking class](#implement-raw-javascript-tracking) when tracking in vanilla JavaScript files.
 
 ### Implement data attribute tracking
 
@@ -42,6 +48,9 @@ The following example shows `data-track-*` attributes assigned to a button:
 
 ```haml
 %button.btn{ data: { track: { action: "click_button", label: "template_preview", property: "my-template" } } }
+
+// or
+// %button.btn{ data: { track_action: "click_button", track_label: "template_preview", track_property: "my-template" } }
 ```
 
 ```html
@@ -62,7 +71,7 @@ The following example shows `data-track-*` attributes assigned to a button:
 | `data-track-property` | false    | Any additional property of the element, or object being acted on. |
 | `data-track-value`    | false    | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. If omitted, this is the element's `value` property or `undefined`. For checkboxes, the default value is the element's checked attribute or `0` when unchecked. |
 | `data-track-extra` | false    | A key-value pair object passed as a valid JSON string. This attribute is added to the `extra` property in our [`gitlab_standard`](schemas.md#gitlab_standard) schema. |
-| `data-track-context`  | false    | The `context` as described in our [Structured event taxonomy](index.md#structured-event-taxonomy). |
+| `data-track-context`  | false    | To append a custom context object, passed as a valid JSON string. |
 
 #### Event listeners
 
@@ -74,12 +83,19 @@ If click events stop propagating, you must implement listeners and [Vue componen
 
 #### Helper methods
 
-Use the following Ruby helper:
+You can use the following Ruby helper:
 
 ```ruby
 tracking_attrs(label, action, property) # { data: { track_label... } }
+```
 
+You can also use it on HAML templates:
+
+```haml
 %button{ **tracking_attrs('main_navigation', 'click_button', 'navigation') }
+
+// When adding additional data
+// %button{ data: { platform: "...", **tracking_attrs('main_navigation', 'click_button', 'navigation') } }
 ```
 
 If you use the GitLab helper method [`nav_link`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/helpers/tab_helper.rb#L76), you must wrap `html_options` under the `html_options` keyword argument. If you
@@ -91,7 +107,7 @@ use the `ActionView` helper method [`link_to`](https://api.rubyonrails.org/v5.2.
 track_action: "click_button" })
 
 # Good
-= nav_link(controller: ['dashboard/groups', 'explore/groups'], html_options: { data: { track_label: 
+= nav_link(controller: ['dashboard/groups', 'explore/groups'], html_options: { data: { track_label:
 "explore_groups", track_action: "click_button" } })
 
 # Good (other helpers)
@@ -101,63 +117,64 @@ track_action: "click_button" })
 
 ### Implement Vue component tracking
 
-For custom event tracking, use a Vue `mixin` in components. Vue `mixin` exposes the `Tracking.event`
-static method and the `track` method. You can specify tracking options in `data` or `computed`.
-These options override any defaults and allow the values to be dynamic from props or based on state.
-
-Several default options are passed when an event is tracked from the component:
+For custom event tracking, use the [Vue mixin](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/tracking.js#L207). It exposes `Tracking.event` as the `track` method.
+You can specify tracking options by creating a `tracking` data object or
+computed property, and as a second parameter: `this.track('click_button', opts)`.
+These options override any defaults and allow the values to be dynamic from props or based on state:
 
-- `category`: If you don't specify, by default `document.body.dataset.page` is used.
-- `label`
-- `property`
-- `value`
+| Property | Type | Default | Example |
+| -- | -- | -- | -- |
+| `category` | string | `document.body.dataset.page` | `'code_quality_walkthrough'` |
+| `label` | string | `''` | `'process_start_button'` |
+| `property` | string | `''` | `'asc'` or `'desc'` |
+| `value` | integer | `undefined` | `0`, `1`, `500` |
+| `extra` | object | `{}` | `{ selectedVariant: this.variant }` |
 
 To implement Vue component tracking:
 
-1. Import the `Tracking` library and request a `mixin`:
+1. Import the `Tracking` library and call the `mixin` method:
 
     ```javascript
     import Tracking from '~/tracking';
-    const trackingMixin = Tracking.mixin;
-    ```
 
-1. Provide categories to track the event from the component. For example, to track all events in a
-component with a label, use the `label` category:
+    const trackingMixin = Tracking.mixin();
 
-    ```javascript
-    import Tracking from '~/tracking';
-    const trackingMixin = Tracking.mixin({ label: 'right_sidebar' });
+    // Optionally provide default properties
+    // const trackingMixin = Tracking.mixin({ label: 'right_sidebar' });
     ```
 
-1. In the component, declare the Vue `mixin`:
+1. Use the mixin in the component:
 
     ```javascript
     export default {
       mixins: [trackingMixin],
-      // ...[component implementation]...
+      // Or
+      // mixins: [Tracking.mixin()],
+      // mixins: [Tracking.mixin({ label: 'right_sidebar' })],
+
       data() {
         return {
           expanded: false,
-          tracking: {
-            label: 'left_sidebar',
-          },
         };
       },
     };
     ```
 
-1. To receive event data as a tracking object or computed property:
-   - Declare it in the `data` function. Use a `tracking` object when default event properties are dynamic or provided at runtime:
+1. You can specify tracking options in by creating a `tracking` data object
+or computed property:
 
       ```javascript
       export default {
         name: 'RightSidebar',
+
         mixins: [Tracking.mixin()],
+
         data() {
           return {
+            expanded: false,
+            variant: '',
             tracking: {
               label: 'right_sidebar',
-              // category: '',
               // property: '',
               // value: '',
               // experiment: '',
@@ -165,18 +182,28 @@ component with a label, use the `label` category:
             },
           };
         },
+
+        // Or
+        // computed: {
+        //   tracking() {
+        //     return {
+        //       property: this.variant,
+        //       extra: { expanded: this.expanded },
+        //     };
+        //   },
+        // },
       };
       ```
 
-   - Declare it in the event data in the `track` function. This object merges with any previously provided options:
+1. Call the `track` method. Tracking options can be passed as the second parameter:
 
-      ```javascript
-      this.track('click_button', {
-        label: 'right_sidebar',
-      });
-      ```
+    ```javascript
+    this.track('click_button', {
+      label: 'right_sidebar',
+    });
+    ```
 
-1. Optional. Use the `track` method in a template:
+    Or use the `track` method in the template:
 
     ```html
     <template>
@@ -185,55 +212,67 @@ component with a label, use the `label` category:
 
         <div v-if="expanded">
           <p>Hello world!</p>
-          <button @click="track('click_action')">Track another event</button>
+          <button @click="track('click_button')">Track another event</button>
         </div>
       </div>
     </template>
     ```
 
-The following example shows an implementation of Vue component tracking:
+#### Testing example
 
 ```javascript
 export default {
-  name: 'RightSidebar',
-  mixins: [Tracking.mixin({ label: 'right_sidebar' })],
+  name: 'CountDropdown',
+
+  mixins: [Tracking.mixin({ label: 'count_dropdown' })],
+
   data() {
     return {
-      expanded: false,
+      variant: 'counter',
+      count: 0,
     };
   },
+
   methods: {
-    toggle() {
-      this.expanded = !this.expanded;
-      // Additional data will be merged, like `value` below
-      this.track('click_toggle', { value: Number(this.expanded) });
-    }
-  }
+    handleChange({ target }) {
+      const { variant } = this;
+
+      this.count = Number(target.value);
+
+      this.track('change_value', {
+        value: this.count,
+        extra: { variant }
+      });
+    },
+  },
 };
 ```
 
-#### Testing example
-
 ```javascript
 import { mockTracking } from 'helpers/tracking_helper';
 // mockTracking(category, documentOverride, spyMethod)
 
-describe('RightSidebar.vue', () => {
+describe('CountDropdown.vue', () => {
   let trackingSpy;
   let wrapper;
 
+  ...
+
   beforeEach(() => {
     trackingSpy = mockTracking(undefined, wrapper.element, jest.spyOn);
   });
 
-  const findToggle = () => wrapper.find('[data-testid="toggle"]');
+  const findDropdown = () => wrapper.find('[data-testid="dropdown"]');
 
-  it('tracks turning off toggle', () => {
-    findToggle().trigger('click');
+  it('tracks change event', () => {
+    const dropdown = findDropdown();
+    dropdown.element.value = 30;
+    dropdown.trigger('change');
 
-    expect(trackingSpy).toHaveBeenCalledWith(undefined, 'click_toggle', {
-      label: 'right_sidebar',
-      value: 0,
+    expect(trackingSpy).toHaveBeenCalledWith(undefined, 'change_value', {
+      value: 30,
+      label: 'count_dropdown',
+      extra: { variant: 'counter' },
     });
   });
 });
@@ -241,7 +280,8 @@ describe('RightSidebar.vue', () => {
 
 ### Implement raw JavaScript tracking
 
-To call custom event tracking and instrumentation directly from the JavaScript file, call the `Tracking.event` static function.
+To track from a vanilla JavaScript file, use the `Tracking.event` static function
+(calls [`dispatchSnowplowEvent`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/dispatch_snowplow_event.js)).
 
 The following example demonstrates tracking a click on a button by manually calling `Tracking.event`.
 
@@ -251,7 +291,7 @@ import Tracking from '~/tracking';
 const button = document.getElementById('create_from_template_button');
 
 button.addEventListener('click', () => {
-  Tracking.event('dashboard:projects:index', 'click_button', {
+  Tracking.event(undefined, 'click_button', {
     label: 'create_from_template',
     property: 'template_preview',
     extra: {
diff --git a/doc/development/snowplow/index.md b/doc/development/snowplow/index.md
index dbc7a25075f..29f4514a21e 100644
--- a/doc/development/snowplow/index.md
+++ b/doc/development/snowplow/index.md
@@ -72,6 +72,7 @@ sequenceDiagram
       GitLab.com Snowplow Collector ->> GitLab.com Snowplow Collector: Write to disk
     end
     GitLab.com Snowplow Collector ->> S3 Bucket: Kinesis Firehose
+    Note over GitLab.com Snowplow Collector, S3 Bucket: Pseudonymization
     S3 Bucket->>Snowflake DW: Import data
     Snowflake DW->>Snowflake DW: Transform data using dbt
     Snowflake DW->>Sisense Dashboards: Data available for querying
@@ -166,11 +167,24 @@ LIMIT 100
 
 Snowplow JavaScript adds [web-specific parameters](https://docs.snowplowanalytics.com/docs/collecting-data/collecting-from-own-applications/snowplow-tracker-protocol/#Web-specific_parameters) to all web events by default.
 
+## Snowplow monitoring
+
+For different stages in the processing pipeline, there are several tools that monitor Snowplow events tracking:
+
+- [Product Intelligence Grafana dashboard](https://dashboards.gitlab.net/d/product-intelligence-main/product-intelligence-product-intelligence?orgId=1) monitors backend events sent from GitLab.com instance to collectors fleet. This dashboard provides information about:
+  - The number of events that successfully reach Snowplow collectors.
+  - The number of events that failed to reach Snowplow collectors.
+  - The number of backend events that were sent.
+- [AWS CloudWatch dashboard](https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#dashboards:name=SnowPlow;start=P3D) monitors the state of the events processing pipeline. The pipeline starts from Snowplow collectors, through to enrichers and pseudonymization, and up to persistence on S3 bucket from which events are imported to Snowflake Data Warehouse. To view this dashboard AWS access is required, follow this [instruction](https://gitlab.com/gitlab-org/growth/product-intelligence/snowplow-pseudonymization#monitoring) if you are interested in getting one.
+- [SiSense dashboard](https://app.periscopedata.com/app/gitlab/417669/Snowplow-Summary-Dashboard) provides information about the number of good and bad events imported into the Data Warehouse, in addition to the total number of imported Snowplow events.
+
+For more information, see this [video walk-through](https://www.youtube.com/watch?v=NxPS0aKa_oU).
+
 ## Related topics
 
 - [Snowplow data structure](https://docs.snowplowanalytics.com/docs/understanding-your-pipeline/canonical-event/)
 - [Our Iglu schema registry](https://gitlab.com/gitlab-org/iglu)
-- [List of events used in our codebase (Event Dictionary)](https://metrics.gitlab.com/snowplow.html)
+- [List of events used in our codebase (Event Dictionary)](https://metrics.gitlab.com/snowplow/)
 - [Product Intelligence Guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)
 - [Service Ping Guide](../service_ping/index.md)
 - [Product Intelligence Direction](https://about.gitlab.com/direction/product-intelligence/)
diff --git a/doc/development/snowplow/review_guidelines.md b/doc/development/snowplow/review_guidelines.md
index 69fad1794e2..0359636380b 100644
--- a/doc/development/snowplow/review_guidelines.md
+++ b/doc/development/snowplow/review_guidelines.md
@@ -14,7 +14,7 @@ general best practices for code reviews, refer to our [code review guide](../cod
 ## Resources for reviewers
 
 - [Snowplow Guide](index.md)
-- [Event Dictionary](https://metrics.gitlab.com/snowplow.html)
+- [Event Dictionary](https://metrics.gitlab.com/snowplow/)
 
 ## Review process
 
diff --git a/doc/development/snowplow/schemas.md b/doc/development/snowplow/schemas.md
index eb57e7d98a5..63864c9329b 100644
--- a/doc/development/snowplow/schemas.md
+++ b/doc/development/snowplow/schemas.md
@@ -12,19 +12,23 @@ This page provides Snowplow schema reference for GitLab events.
 
 We are including the [`gitlab_standard` schema](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_standard/jsonschema/) with every event. See [Standardize Snowplow Schema](https://gitlab.com/groups/gitlab-org/-/epics/5218) for details.
 
-The [`StandardContext`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/tracking/standard_context.rb) class represents this schema in the application.
+The [`StandardContext`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/tracking/standard_context.rb)
+class represents this schema in the application. Some properties are automatically populated for [frontend](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/views/layouts/_snowplow.html.haml)
+events.
 
-| Field Name     | Required            | Type                  | Description                                                                                 |
-|----------------|---------------------|-----------------------|---------------------------------------------------------------------------------------------|
-| `project_id`   | **{dotted-circle}** | integer               |                                                                 |
-| `namespace_id` | **{dotted-circle}** | integer               |                                                               |
-| `user_id`      | **{dotted-circle}** | integer               | User database record ID attribute. This file undergoes a pseudonymization process at the collector level. |
-| `context_generated_at` | **{dotted-circle}** | string (date time format) | Timestamp indicating when context was generated.                                                                |
-| `environment`  | **{check-circle}**  | string (max 32 chars) | Name of the source environment, such as `production` or `staging`             |
-| `source`       | **{check-circle}**  | string (max 32 chars) | Name of the source application, such as  `gitlab-rails` or `gitlab-javascript` |
-| `plan`         | **{dotted-circle}**  | string (max 32 chars) | Name of the plan for the namespace, such as `free`, `premium`, or `ultimate`. Automatically picked from the `namespace`. |
-| `google_analytics_id` | **{dotted-circle}**  | string (max 32 chars) | Google Analytics ID, present when set from our marketing sites. |
-| `extra`        | **{dotted-circle}**  | JSON                  | Any additional data associated with the event, in the form of key-value pairs |
+| Field Name     | Required            | Default value | Type                  | Description                                                                                 |
+|----------------|:-------------------:|-----------------------|--|---------------------------------------------------------------------------------------------|
+| `project_id`   | **{dotted-circle}** | Current project ID * | integer               |                                                                 |
+| `namespace_id` | **{dotted-circle}** | Current group/namespace ID * | integer               |                                                               |
+| `user_id`      | **{dotted-circle}** | Current user ID * | integer               | User database record ID attribute. This file undergoes a pseudonymization process at the collector level. |
+| `context_generated_at` | **{dotted-circle}** | Current timestamp | string (date time format) | Timestamp indicating when context was generated.                                                                |
+| `environment`  | **{check-circle}**  | Current environment | string (max 32 chars) | Name of the source environment, such as `production` or `staging`             |
+| `source`       | **{check-circle}**  | Event source | string (max 32 chars) | Name of the source application, such as  `gitlab-rails` or `gitlab-javascript` |
+| `plan`         | **{dotted-circle}**  | Current namespace plan * | string (max 32 chars) | Name of the plan for the namespace, such as `free`, `premium`, or `ultimate`. Automatically picked from the `namespace`. |
+| `google_analytics_id` | **{dotted-circle}**  | GA ID value * | string (max 32 chars) | Google Analytics ID, present when set from our marketing sites. |
+| `extra`        | **{dotted-circle}**  |  | JSON                  | Any additional data associated with the event, in the form of key-value pairs |
+
+_\* Default value present for frontend events only_
 
 ## Default Schema
 
diff --git a/doc/development/snowplow/troubleshooting.md b/doc/development/snowplow/troubleshooting.md
new file mode 100644
index 00000000000..75c8b306a67
--- /dev/null
+++ b/doc/development/snowplow/troubleshooting.md
@@ -0,0 +1,50 @@
+---
+stage: Growth
+group: Product Intelligence
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Troubleshooting
+
+## Good events drop
+
+### Symptoms
+
+You will be alarmed via a [Sisense alert](https://app.periscopedata.com/app/gitlab/alert/Volume-of-Snowplow-Good-events/5a5f80ef34fe450da5ebb84eaa84067f/edit) that is sent to `#g_product_intelligence` Slack channel
+
+### Locating the problem
+
+First you need to identify at which stage in Snowplow the data pipeline the drop is occurring.
+Start at [Snowplow dashboard](https://console.aws.amazon.com/systems-manager/resource-groups/cloudwatch?dashboard=SnowPlow&region=us-east-1#) on CloudWatch,
+if you do not have access to CloudWatch you need to create an [access request issue](https://gitlab.com/gitlab-com/team-member-epics/access-requests/-/issues/9730) first.
+While on CloudWatch dashboard set time range to last 4 weeks, to get better picture of system characteristics over time. Than visit following charts:
+
+1. `ELB New Flow Count` and `Collector Auto Scaling Group Network In/Out` - they show in order: number of connections to collectors via load balancers and data volume (in bytes) processed by collectors. If there is drop visible there, it means less events were fired from the GitLab application. Proceed to [application layer guide](#troubleshooting-gitlab-application-layer) for more details
+1. `Firehose Records to S3` - it shows how many event records were saved to S3 bucket, if there was drop on this chart but not on the charts from 1. it means that problem is located at AWS infrastructure layer, please refer to [AWS layer guide](#troubleshooting-aws-layer)
+1. If drop wasn't visible on any of previous charts it means that probelm is at data warehouse layer, please refer to [data warehouse layer guide](#troubleshooting-data-warehouse-layer)
+
+### Troubleshooting GitLab application layer
+
+Drop occurring at application layer can be symptom of some issue, but it might be also a result of normal application lifecycle, intended changes done to product intelligence or experiments tracking
+or even a result of a public holiday in some regions of the world with a larger user-base. To verify if there is an underlying problem to solve, you can check following things:
+
+1. Check `about.gitlab.com` website traffic on [Google Analytics](https://analytics.google.com/) to verify if some public holiday might impact overall use of GitLab system
+   1. You may require to open an access request for Google Analytics access first eg: [access request internal issue](https://gitlab.com/gitlab-com/team-member-epics/access-requests/-/issues/1772)
+1. Plot `select date(dvce_created_tstamp) , event , count(*) from legacy.snowplow_unnested_events_90 where dvce_created_tstamp > '2021-06-15' and dvce_created_tstamp < '2021-07-10' group by 1 , 2 order by 1 , 2` in SiSense to see what type of events was responsible for drop
+1. Plot `select date(dvce_created_tstamp) ,se_category , count(*) from legacy.snowplow_unnested_events_90 where dvce_created_tstamp > '2021-06-15' and dvce_created_tstamp < '2021-07-31' and event = 'struct' group by  1 , 2 order by  1, 2` what events recorded the biggest drops in suspected category
+1. Check if there was any MR merged that might cause reduction in reported events, pay an attention to ~"product intelligence" and ~"growth experiment" labeled MRs
+1. Check (via [Grafana explore tab](https://dashboards.gitlab.net/explore) ) following Prometheus counters `gitlab_snowplow_events_total`, `gitlab_snowplow_failed_events_total` and `gitlab_snowplow_successful_events_total` to see how many events were fired correctly from GitLab.com. Example query to use `sum(rate(gitlab_snowplow_successful_events_total{env="gprd"}[5m])) / sum(rate(gitlab_snowplow_events_total{env="gprd"}[5m]))` would chart rate at which number of good events rose in comparison to events sent in total. If number drops from 1 it means that problem might be in communication between GitLab and AWS collectors fleet.
+1. Check [logs in Kibana](https://log.gprd.gitlab.net/app/discover#) and filter with `{ "query": { "match_phrase": { "json.message": "failed to be reported to collector at" } } }` if there are some failed events logged
+
+For results about an investigation conducted into an unexpected drop in snowplow events volume, see [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/335206).
+
+### Troubleshooting AWS layer
+
+Already conducted investigations:
+
+- [Steep decrease of Snowplow page views](https://gitlab.com/gitlab-org/gitlab/-/issues/268009)
+- [`snowplow.trx.gitlab.net` unreachable](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5073)
+
+### Troubleshooting data warehouse layer
+
+Reach out to [Data team](https://about.gitlab.com/handbook/business-technology/data-team) to ask about current state of data warehouse. On their handbook page there is a [section with contact details](https://about.gitlab.com/handbook/business-technology/data-team/#how-to-connect-with-us)
diff --git a/doc/development/stage_group_dashboards.md b/doc/development/stage_group_dashboards.md
index 88e9141574e..744d049f72d 100644
--- a/doc/development/stage_group_dashboards.md
+++ b/doc/development/stage_group_dashboards.md
@@ -64,8 +64,8 @@ component can have 2 indicators:
    [customize the request Apdex](application_slis/rails_request_apdex.md), this new Apdex
    measurement is not yet part of the error budget.
 
-   For Sidekiq job execution, the threshold depends on the [job
-   urgency](sidekiq_style_guide.md#job-urgency). It is
+   For Sidekiq job execution, the threshold depends on the
+   [job urgency](sidekiq/worker_attributes.md#job-urgency). It is
    [currently](https://gitlab.com/gitlab-com/runbooks/-/blob/f22f40b2c2eab37d85e23ccac45e658b2c914445/metrics-catalog/services/lib/sidekiq-helpers.libsonnet#L25-38)
    **10 seconds** for high-urgency jobs and **5 minutes** for other
    jobs.
diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md
index 543feaa967c..405ff40ef6a 100644
--- a/doc/development/testing_guide/end_to_end/best_practices.md
+++ b/doc/development/testing_guide/end_to_end/best_practices.md
@@ -363,11 +363,11 @@ after(:all) do
 end
 ```
 
-## Tag tests that require the Administrator role
+## Tag tests that require administrator access
 
-We don't run tests that require the Administrator role against our Production environments.
+We don't run tests that require administrator access against our Production environments.
 
-When you add a new test that requires the Administrator role, apply the RSpec metadata `:requires_admin` so that the test will not be included in the test suites executed against Production and other environments on which we don't want to run those tests.
+When you add a new test that requires administrator access, apply the RSpec metadata `:requires_admin` so that the test will not be included in the test suites executed against Production and other environments on which we don't want to run those tests.
 
 When running tests locally or configuring a pipeline, the environment variable `QA_CAN_TEST_ADMIN_FEATURES` can be set to `false` to skip tests that have the `:requires_admin` tag.
 
diff --git a/doc/development/testing_guide/end_to_end/execution_context_selection.md b/doc/development/testing_guide/end_to_end/execution_context_selection.md
index 8456cfa8314..0fdcf0c8c3b 100644
--- a/doc/development/testing_guide/end_to_end/execution_context_selection.md
+++ b/doc/development/testing_guide/end_to_end/execution_context_selection.md
@@ -120,3 +120,5 @@ Similarly to specifying that a test should only run against a specific environme
 test only when it runs against a specific environment. The syntax is exactly the same, except that the `only: { ... }`
 hash is nested in the [`quarantine: { ... }`](https://about.gitlab.com/handbook/engineering/quality/guidelines/debugging-qa-test-failures/#quarantining-tests) hash.
 For example, `quarantine: { only: { subdomain: :staging } }` only quarantines the test when run against `staging`.
+
+The quarantine feature can be explicitly disabled with the `DISABLE_QUARANTINE` environment variable. This can be useful when running tests locally.
diff --git a/doc/development/testing_guide/end_to_end/feature_flags.md b/doc/development/testing_guide/end_to_end/feature_flags.md
index 48157a961e1..238b1ccd8f9 100644
--- a/doc/development/testing_guide/end_to_end/feature_flags.md
+++ b/doc/development/testing_guide/end_to_end/feature_flags.md
@@ -144,6 +144,14 @@ end
 After the feature flag is removed, clean up the resource class and delete the variable.
 All methods should use the condition procedures of the now-default state.
 
+## Managing flakiness due to caching
+
+All application settings, and all feature flags, are cached inside GitLab for one minute.
+All caching is disabled during testing, except on static environments.
+
+When a test changes a feature flag, it can cause flaky behavior if elements are visible only with an
+active feature flag. To circumvent this behavior, add a wait for elements behind a feature flag.
+
 ## Running a scenario with a feature flag enabled
 
 It's also possible to run an entire scenario with a feature flag enabled, without having to edit
diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md
index 1fc9bc8258a..dc989acbdcc 100644
--- a/doc/development/testing_guide/end_to_end/index.md
+++ b/doc/development/testing_guide/end_to_end/index.md
@@ -101,19 +101,19 @@ This is due to technical limitations in the GitLab permission model: the ability
 against a protected branch is controlled by the ability to push/merge to this branch.
 This means that for developers to be able to trigger a pipeline for the default branch in
 `gitlab-org/omnibus-gitlab`/`gitlab-org/gitlab-qa`, they would need to have the
-[Maintainer role](../../../user/permissions.md) for those projects.
+Maintainer role for those projects.
 For security reasons and implications, we couldn't open up the default branch to all the Developers.
 Hence we created these mirrors where Developers and Maintainers are allowed to push/merge to the default branch.
 This problem was discovered in <https://gitlab.com/gitlab-org/gitlab-qa/-/issues/63#note_107175160> and the "mirror"
 work-around was suggested in <https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4717>.
 A feature proposal to segregate access control regarding running pipelines from ability to push/merge was also created at <https://gitlab.com/gitlab-org/gitlab/-/issues/24585>.
 
-#### With Pipeline for Merged Results
+#### With merged results pipelines
 
-In a Pipeline for Merged Results, the pipeline runs on a new ref that contains the merge result of the source and target branch.
+In a merged results pipeline, the pipeline runs on a new ref that contains the merge result of the source and target branch.
 However, this ref is not available to the `gitlab-qa-mirror` pipeline.
 
-For this reason, the end-to-end tests on a Pipeline for Merged Results would use the head of the merge request source branch.
+For this reason, the end-to-end tests on a merged results pipeline would use the head of the merge request source branch.
 
 ```mermaid
 graph LR
@@ -126,7 +126,7 @@ A --> C
 B --> C
 
 A --> E["E2E tests"]
-C --> D["Pipeline for merged results"]
+C --> D["Merged results pipeline"]
  ```
 
 ##### Running custom tests
diff --git a/doc/development/testing_guide/end_to_end/resources.md b/doc/development/testing_guide/end_to_end/resources.md
index abba1d51f07..e2b005d8a1b 100644
--- a/doc/development/testing_guide/end_to_end/resources.md
+++ b/doc/development/testing_guide/end_to_end/resources.md
@@ -431,11 +431,32 @@ module QA
       def validate_reuse_preconditions
         raise ResourceReuseError unless reused_name_valid?
       end
+
+      # Internally we identify an instance of a reusable resource by a unique value of `@reuse_as`, but in GitLab the
+      # resource has one or more attributes that must also be unique. This method lists those attributes and allows the
+      # test framework to check that each instance of a reusable resource has values that match the associated values
+      # in Gitlab.
+      def unique_identifiers
+        [:name, :path]
+      end
     end
   end
 end
 ```
 
+Reusable resources aren't removed immediately when `remove_via_api!` is called. Instead, they're removed after the test
+suite completes. To do so each class must be registered with `QA::Resource::ReusableCollection` in `qa/spec/spec_helper.rb`
+as in the example below. Registration allows `QA::Resource::ReusableCollection` to keep track of each instance of each
+registered class, and to delete them all in the `config.after(:suite)` hook.
+
+```ruby
+config.before(:suite) do |suite|
+  QA::Resource::ReusableCollection.register_resource_classes do |collection|
+    QA::Resource::ReusableProject.register(collection)
+  end
+end
+```
+
 Consider some examples of how a reusable resource is used:
 
 ```ruby
@@ -488,6 +509,65 @@ end
 # match the name specified when the project was first fabricated.
 ```
 
+### Validating reusable resources
+
+Reusable resources can speed up test suites by avoiding the cost of creating the same resource again and again. However,
+that can cause problems if a test makes changes to a resource that prevent it from being reused as expected by later
+tests. That can lead to order-dependent test failures that can be difficult to troubleshoot.
+
+For example, the default project created by `QA::Resource::ReusableProject` has `auto_devops_enabled` set to `false`
+(inherited from `QA::Resource::Project`). If a test reuses that project and enables Auto DevOps, subsequent tests that reuse
+the project will fail if they expect Auto DevOps to be disabled.
+
+We try to avoid that kind of trouble by validating reusable resources after a test suite. If the environment variable
+`QA_VALIDATE_RESOURCE_REUSE` is set to `true` the test framework will check each reusable resource to verify that none
+of the attributes they were created with have been changed. It does that by creating a new resource using the same
+attributes that were used to create the original resource. It then compares the new resource to the original and raises
+an error if any attributes don't match.
+
+#### Implementation
+
+When you implement a new type of reusable resource there are two `private` methods you must implement so the resource
+can be validated. They are:
+
+- `reference_resource`: creates a new instance of the resource that can be compared with the one that was used during the tests.
+- `unique_identifiers`: returns an array of attributes that allow the resource to be identified (e.g., name) and that are therefore
+expected to differ when comparing the reference resource with the resource reused in the tests.
+
+The following example shows the implementation of those two methods in `QA::Resource::ReusableProject`.
+
+```ruby
+# Creates a new project that can be compared to a reused project, using the attributes of the original.
+#
+# @return [QA::Resource] a new instance of Resource::ReusableProject that should be a copy of the original resource
+def reference_resource
+  # These are the attributes that the reused resource was created with
+  attributes = self.class.resources[reuse_as][:attributes]
+
+  # Two projects can't have the same path, and since we typically use the same value for the name and path, we assign
+  # a unique name and path to the reference resource.
+  name = "reference_resource_#{SecureRandom.hex(8)}_for_#{attributes.delete(:name)}"
+
+  Project.fabricate_via_api! do |project|
+    self.class.resources[reuse_as][:attributes].each do |attribute_name, attribute_value|
+      project.instance_variable_set("@#{attribute_name}", attribute_value) if attribute_value
+    end
+    project.name = name
+    project.path = name
+    project.path_with_namespace = "#{project.group.full_path}/#{project.name}"
+  end
+end
+
+# The attributes of the resource that should be the same whenever a test wants to reuse a project.
+#
+# @return [Array<Symbol>] the attribute names.
+def unique_identifiers
+  # As noted above, path must be unique, and since we typically use the same value for both, we treat name and path
+  # as unique. These attributes are ignored when we compare the reference and reused resources.
+  [:name, :path]
+end
+```
+
 ## Where to ask for help?
 
 If you need more information, ask for help on `#quality` channel on Slack
diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
index bb214976926..07f7e9582ee 100644
--- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
+++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
@@ -20,6 +20,7 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec
 | `:github`         | The test requires a GitHub personal access token. |
 | `:group_saml`     | The test requires a GitLab instance that has SAML SSO enabled at the group level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. |
 | `:instance_saml`  | The test requires a GitLab instance that has SAML SSO enabled at the instance level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. |
+| `:integrations`   | This aims to test the available [integrations](../../../user/project/integrations/overview.md#integrations-listing). The test requires Docker to be installed in the run context. It will provision the containers and can be run against a local instance or using the `gitlab-qa` scenario `Test::Integration::Integrations` | 
 | `:service_ping_disabled`  | The test interacts with the GitLab configuration service ping at the instance level to turn admin setting service ping checkbox on or off. This tag will have the test run only in the `service_ping_disabled` job and must be paired with the `:orchestrated` and `:requires_admin` tags. |
 | `:jira`           | The test requires a Jira Server. [GitLab-QA](https://gitlab.com/gitlab-org/gitlab-qa) provisions the Jira Server in a Docker container when the `Test::Integration::Jira` test scenario is run.
 | `:kubernetes`     | The test includes a GitLab instance that is configured to be run behind an SSH tunnel, allowing a TLS-accessible GitLab. This test also includes provisioning of at least one Kubernetes cluster to test against. _This tag is often be paired with `:orchestrated`._ |
diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
index ef3e0624395..7fb95769fc2 100644
--- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
+++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
@@ -141,6 +141,34 @@ docker stop gitlab-gitaly-cluster praefect postgres gitaly3 gitaly2 gitaly1
 docker rm gitlab-gitaly-cluster praefect postgres gitaly3 gitaly2 gitaly1
 ```
 
+## Tests that require a runner
+
+To execute tests that use a runner without errors, while creating the GitLab Docker instance the `--hostname` parameter in the Docker `run` command should be given a specific interface IP address or a non-loopback hostname accessible from the runner container. Having `localhost` (or `127.0.0.1`) as the GitLab hostname won't work (unless the GitLab Runner is created with the Docker network as `host`)
+
+Examples of tests which require a runner:
+
+- `qa/qa/specs/features/ee/browser_ui/13_secure/create_merge_request_with_secure_spec.rb`
+- `qa/qa/specs/features/browser_ui/4_verify/runner/register_runner_spec.rb`
+
+Example:
+
+```shell
+docker run \ 
+  --detach \
+  --hostname interface_ip_address \
+  --publish 80:80 \
+  --name gitlab \
+  --restart always \
+  --volume ~/ee_volume/config:/etc/gitlab \
+  --volume ~/ee_volume/logs:/var/log/gitlab \
+  --volume ~/ee_volume/data:/var/opt/gitlab \
+  --shm-size 256m \
+  gitlab/gitlab-ee:latest
+```
+
+Where `interface_ip_address` is your local network's interface IP, which you can find with the `ifconfig` command.
+The same would apply to GDK running with the instance address as `localhost` too.
+
 ## Guide to run and debug Monitor tests
 
 ### How to set up
diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md
index d2e68ea7715..09fc8f4d33e 100644
--- a/doc/development/testing_guide/flaky_tests.md
+++ b/doc/development/testing_guide/flaky_tests.md
@@ -14,14 +14,14 @@ eventually.
 ## Quarantined tests
 
 When a test frequently fails in `main`,
-[a ~"master:broken" issue](https://about.gitlab.com/handbook/engineering/workflow/#broken-master)
-should be created.
+create [a ~"failure::flaky-test" issue](https://about.gitlab.com/handbook/engineering/workflow/#broken-master).
+
 If the test cannot be fixed in a timely fashion, there is an impact on the
-productivity of all the developers, so it should be placed in quarantine by
-assigning the `:quarantine` metadata with the issue URL.
+productivity of all the developers, so it should be quarantined by
+assigning the `:quarantine` metadata with the issue URL, and add the `~"quarantined test"` label to the issue.
 
 ```ruby
-it 'should succeed', quarantine: 'https://gitlab.com/gitlab-org/gitlab/-/issues/12345' do
+it 'succeeds', quarantine: 'https://gitlab.com/gitlab-org/gitlab/-/issues/12345' do
   expect(response).to have_gitlab_http_status(:ok)
 end
 ```
@@ -32,23 +32,13 @@ This means it is skipped unless run with `--tag quarantine`:
 bin/rspec --tag quarantine
 ```
 
-**Before putting a test in quarantine, you should make sure that a
-~"master:broken" issue exists for it so it doesn't stay in quarantine forever.**
-
 Once a test is in quarantine, there are 3 choices:
 
-- Should the test be fixed (that is, get rid of its flakiness)?
-- Should the test be moved to a lower level of testing?
-- Should the test be removed entirely (for example, because there's already a
+- Fix the test (that is, get rid of its flakiness).
+- Move the test to a lower level of testing.
+- Remove the test entirely (for example, because there's already a
   lower-level test, or it's duplicating another same-level test, or it's testing
-  too much etc.)?
-
-### Quarantine tests on the CI
-
-Quarantined tests are run on the CI in dedicated jobs that are allowed to fail:
-
-- `rspec-pg-quarantine` (CE & EE)
-- `rspec-pg-quarantine-ee` (EE only)
+  too much etc.).
 
 ## Automatic retries and flaky tests detection
 
@@ -116,6 +106,7 @@ reproduction.
   - [Lazy loaded images can cause Capybara to mis-click](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18713)
 - [Triggering JS events before the event handlers are set up](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18742)
 - [Wait for the image to be lazy-loaded when asserting on a Markdown image's `src` attribute](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25408)
+- [Avoid asserting against flash notice banners](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79432)
 
 #### Capybara viewport size related issues
 
diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md
index eb35227a50c..d03a4976a8c 100644
--- a/doc/development/testing_guide/frontend_testing.md
+++ b/doc/development/testing_guide/frontend_testing.md
@@ -748,6 +748,8 @@ Consult the [official Jest docs](https://jestjs.io/docs/jest-object#mock-modules
 
 ## Running Frontend Tests
 
+Before generating fixtures, make sure you have a running GDK instance.
+
 For running the frontend tests, you need the following commands:
 
 - `rake frontend:fixtures` (re-)generates [fixtures](#frontend-test-fixtures). Make sure that
diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md
index aef83109b9b..27d5ae70ed7 100644
--- a/doc/development/testing_guide/review_apps.md
+++ b/doc/development/testing_guide/review_apps.md
@@ -15,6 +15,7 @@ For any of the following scenarios, the `start-review-app-pipeline` job would be
 - for merge requests with CI config changes
 - for merge requests with frontend changes
 - for merge requests with changes to `{,ee/,jh/}{app/controllers}/**/*`
+- for merge requests with changes to `{,ee/,jh/}{app/models}/**/*`
 - for merge requests with QA changes
 - for scheduled pipelines
 - the MR has the `pipeline:run-review-app` label set
@@ -30,6 +31,8 @@ You can also manually start the `review-qa-all`: it runs the full QA suite.
 After the end-to-end test runs have finished, [Allure reports](https://github.com/allure-framework/allure2) are generated and published by
 the `allure-report-qa-smoke`, `allure-report-qa-reliable`, and `allure-report-qa-all` jobs. A comment with links to the reports are added to the merge request.
 
+Errors can be found in the `gitlab-review-apps` Sentry project and [filterable by Review App URL](https://sentry.gitlab.net/gitlab/gitlab-review-apps/?query=url%3A%22https%3A%2F%2Fgitlab-review-require-ve-u92nn2.gitlab-review.app%2F%22) or [commit SHA](https://sentry.gitlab.net/gitlab/gitlab-review-apps/releases/6095b501da7/all-events/).
+
 ## Performance Metrics
 
 On every [pipeline](https://gitlab.com/gitlab-org/gitlab/pipelines/125315730) in the `qa` stage, the
diff --git a/doc/development/transient/prevention-patterns.md b/doc/development/transient/prevention-patterns.md
index c517a6bcd54..93712b104e6 100644
--- a/doc/development/transient/prevention-patterns.md
+++ b/doc/development/transient/prevention-patterns.md
@@ -120,7 +120,7 @@ When there are 2 jobs being worked on at the same time, it is possible that the
 In this example, `Worker B` is meant to set the updated status. But `Worker A` calls `#update_state` a little too late.
 
 This can be avoided by utilizing either database locks or `Gitlab::ExclusiveLease`. This way, jobs will be
-worked on one at a time. This also allows them to be marked as [idempotent](../sidekiq_style_guide.md#idempotent-jobs).
+worked on one at a time. This also allows them to be marked as [idempotent](../sidekiq/idempotent_jobs.md).
 
 ### Retry mechanism handling
 
diff --git a/doc/development/uploads.md b/doc/development/uploads.md
index 6d8b951be83..14ecf38e21c 100644
--- a/doc/development/uploads.md
+++ b/doc/development/uploads.md
@@ -346,3 +346,85 @@ object using `UploadedFile#from_params`! This method can be unsafe to use depend
 passed. Instead, use the [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb)
 object that [`multipart.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/middleware/multipart.rb)
 builds automatically for you.
+
+### Document Object Storage buckets and CarrierWave integration
+
+When using Object Storage, GitLab expects each kind of upload to maintain its own bucket in the respective
+Object Storage destination. Moreover, the integration with CarrierWave is not used all the time.
+The [Object Storage Working Group](https://about.gitlab.com/company/team/structure/working-groups/object-storage/)
+is investigating an approach that unifies Object Storage buckets into a single one and removes CarrierWave
+so as to simplify implementation and administration of uploads.
+
+Therefore, document new uploads here by slotting them into the following tables:
+
+- [Feature bucket details](#feature-bucket-details)
+- [CarrierWave integration](#carrierwave-integration)
+
+#### Feature bucket details
+
+| Feature                                  | Upload technology | Uploader              | Bucket structure                                                                                          |
+|------------------------------------------|-------------------|-----------------------|-----------------------------------------------------------------------------------------------------------|
+| Job artifacts                            | `direct upload`     | `workhorse`             | `/artifacts/<proj_id_hash>/<date>/<job_id>/<artifact_id>`                                                 |
+| Pipeline artifacts                       | `carrierwave`       | `sidekiq`               | `/artifacts/<proj_id_hash>/pipelines/<pipeline_id>/artifacts/<artifact_id>`                               |
+| Live job traces                          | `fog`               | `sidekiq`               | `/artifacts/tmp/builds/<job_id>/chunks/<chunk_index>.log`                                                 |
+| Job traces archive                       | `carrierwave`       | `sidekiq`               | `/artifacts/<proj_id_hash>/<date>/<job_id>/<artifact_id>/job.log`                                         |
+| Autoscale runner caching                 | N/A               | `gitlab-runner`         | `/gitlab-com-[platform-]runners-cache/???`                                                                |
+| Backups                                  | N/A               | `s3cmd`, `awscli`, or `gcs` | `/gitlab-backups/???`                                                                                     |
+| Git LFS                                  | `direct upload`     | `workhorse`             | `/lsf-objects/<lfs_obj_oid[0:2]>/<lfs_obj_oid[2:2]>`                                                      |
+| Design management files                  | `disk buffering`    | `rails controller`      | `/lsf-objects/<lfs_obj_oid[0:2]>/<lfs_obj_oid[2:2]>`                                                      |
+| Design management thumbnails             | `carrierwave`       | `sidekiq`               | `/uploads/design_management/action/image_v432x230/<model_id>`                                             |
+| Generic file uploads                     | `direct upload`     | `workhorse`             | `/uploads/@hashed/[0:2]/[2:4]/<hash1>/<hash2>/file`                                                       |
+| Generic file uploads - personal snippets | `direct upload`     | `workhorse`             | `/uploads/personal_snippet/<snippet_id>/<filename>`                                                       |
+| Global appearance settings               | `disk buffering`    | `rails controller`      | `/uploads/appearance/...`                                                                                 |
+| Topics                                   | `disk buffering`    | `rails controller`      | `/uploads/projects/topic/...`                                                                             |
+| Avatar images                            | `direct upload`     | `workhorse`             | `/uploads/[user,group,project]/avatar/<model_id>`                                                         |
+| Import/export                            | `direct upload`     | `workhorse`             | `/uploads/import_export_upload/???`                                                                       |
+| GitLab Migration                         | `carrierwave`       | `sidekiq`               | `/uploads/bulk_imports/???`                                                                               |
+| MR diffs                                 | `carrierwave`       | `sidekiq`               | `/external-diffs/merge_request_diffs/mr-<mr_id>/diff-<diff_id>`                                           |
+| Package manager archives                 | `direct upload`     | `sidekiq`               | `/packages/<proj_id_hash>/packages/<pkg_segment>/files/<pkg_file_id>`                                     |
+| Package manager archives                 | `direct upload`     | `sidekiq`               | `/packages/<container_id_hash>/debian_*_component_file/<component_file_id>`                               |
+| Package manager archives                 | `direct upload`     | `sidekiq`               | `/packages/<container_id_hash>/debian_*_distribution/<distribution_file_id>`                              |
+| Container image cache (?)                | `direct upload`     | `workhorse`             | `/dependency-proxy/<group_id_hash>/dependency_proxy/<group_id>/files/<proxy_id>/<blob_id or manifest_id>` |
+| Terraform state files                    | `carrierwave`       | `rails controller`      | `/terraform/<proj_id_hash>/<terraform_state_id>`                                                          |
+| Pages content archives                   | `carrierwave`       | `sidekiq`               | `/gitlab-gprd-pages/<proj_id_hash>/pages_deployments/<deployment_id>/`                                    |
+| Secure Files                             | `carrierwave`       | `sidekiq`               | `/ci-secure-files/<proj_id_hash>/secure_files/<secure_file_id>/`                                    |
+
+#### CarrierWave integration
+
+| File                                                    | Carrierwave usage                                                                | Categorized         |
+|---------------------------------------------------------|----------------------------------------------------------------------------------|---------------------|
+| `app/models/project.rb`                                 | `include Avatarable`                                                             | :white_check_mark:  |
+| `app/models/projects/topic.rb`                          | `include Avatarable`                                                             | :white_check_mark:  |
+| `app/models/group.rb`                                   | `include Avatarable`                                                             | :white_check_mark:  |
+| `app/models/user.rb`                                    | `include Avatarable`                                                             | :white_check_mark:  |
+| `app/models/terraform/state_version.rb`                 | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `app/models/ci/job_artifact.rb`                         | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `app/models/ci/pipeline_artifact.rb`                    | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `app/models/pages_deployment.rb`                        | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `app/models/lfs_object.rb`                              | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `app/models/dependency_proxy/blob.rb`                   | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `app/models/dependency_proxy/manifest.rb`               | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `app/models/packages/composer/cache_file.rb`            | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `app/models/packages/package_file.rb`                   | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `app/models/concerns/packages/debian/component_file.rb` | `include FileStoreMounter`                                                       | :white_check_mark:  |
+| `ee/app/models/issuable_metric_image.rb`                | `include FileStoreMounter`                                                       |                     |
+| `ee/app/models/vulnerabilities/remediation.rb`          | `include FileStoreMounter`                                                       |                     |
+| `ee/app/models/vulnerabilities/export.rb`               | `include FileStoreMounter`                                                       |                     |
+| `app/models/packages/debian/project_distribution.rb`    | `include Packages::Debian::Distribution`                                         | :white_check_mark:  |
+| `app/models/packages/debian/group_distribution.rb`      | `include Packages::Debian::Distribution`                                         | :white_check_mark:  |
+| `app/models/packages/debian/project_component_file.rb`  | `include Packages::Debian::ComponentFile`                                        | :white_check_mark:  |
+| `app/models/packages/debian/group_component_file.rb`    | `include Packages::Debian::ComponentFile`                                        | :white_check_mark:  |
+| `app/models/merge_request_diff.rb`                      | `mount_uploader :external_diff, ExternalDiffUploader`                            | :white_check_mark:  |
+| `app/models/note.rb`                                    | `mount_uploader :attachment, AttachmentUploader`                                 | :white_check_mark:  |
+| `app/models/appearance.rb`                              | `mount_uploader :logo,         AttachmentUploader`                               | :white_check_mark:  |
+| `app/models/appearance.rb`                              | `mount_uploader :header_logo,  AttachmentUploader`                               | :white_check_mark:  |
+| `app/models/appearance.rb`                              | `mount_uploader :favicon,      FaviconUploader`                                  | :white_check_mark:  |
+| `app/models/project.rb`                                 | `mount_uploader :bfg_object_map, AttachmentUploader`                             |                     |
+| `app/models/import_export_upload.rb`                    | `mount_uploader :import_file, ImportExportUploader`                              | :white_check_mark:  |
+| `app/models/import_export_upload.rb`                    | `mount_uploader :export_file, ImportExportUploader`                              | :white_check_mark:  |
+| `app/models/ci/deleted_object.rb`                       | `mount_uploader :file, DeletedObjectUploader`                                    |                     |
+| `app/models/design_management/action.rb`                | `mount_uploader :image_v432x230, DesignManagement::DesignV432x230Uploader`       | :white_check_mark:  |
+| `app/models/concerns/packages/debian/distribution.rb`   | `mount_uploader :signed_file, Packages::Debian::DistributionReleaseFileUploader` | :white_check_mark:  |
+| `app/models/bulk_imports/export_upload.rb`              | `mount_uploader :export_file, ExportUploader`                                    | :white_check_mark:  |
+| `ee/app/models/user_permission_export_upload.rb`        | `mount_uploader :file, AttachmentUploader`                                       |                     |
+| `app/models/ci/secure_file.rb`                          | `include FileStoreMounter`                                       |                     |
diff --git a/doc/development/value_stream_analytics.md b/doc/development/value_stream_analytics.md
index 6b2442f1c32..0d545fa8e3f 100644
--- a/doc/development/value_stream_analytics.md
+++ b/doc/development/value_stream_analytics.md
@@ -4,11 +4,11 @@ group: Optimize
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
 ---
 
-# Value Stream Analytics development guide
+# Value stream analytics development guide
 
 Value stream analytics calculates the time between two arbitrary events recorded on domain objects and provides aggregated statistics about the duration.
 
-For information on how to configure Value Stream Analytics in GitLab, see our [analytics documentation](../user/analytics/value_stream_analytics.md).
+For information on how to configure value stream analytics in GitLab, see our [analytics documentation](../user/analytics/value_stream_analytics.md).
 
 ## Stage
 
diff --git a/doc/development/value_stream_analytics/img/object_hierarchy_example_V14_10.png b/doc/development/value_stream_analytics/img/object_hierarchy_example_V14_10.png
new file mode 100644
index 00000000000..0e3c760fa3c
--- /dev/null
+++ b/doc/development/value_stream_analytics/img/object_hierarchy_example_V14_10.png
diff --git a/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md b/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md
new file mode 100644
index 00000000000..aef85107cd9
--- /dev/null
+++ b/doc/development/value_stream_analytics/value_stream_analytics_aggregated_backend.md
@@ -0,0 +1,330 @@
+---
+stage: Manage
+group: Optimize
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Aggregated Value Stream Analytics
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335391) in GitLab 14.7.
+
+DISCLAIMER:
+This page contains information related to upcoming products, features, and functionality.
+It is important to note that the information presented is for informational purposes only.
+Please do not rely on this information for purchasing or planning purposes.
+As with all projects, the items mentioned on this page are subject to change or delay.
+The development, release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This page provides a high-level overview of the aggregated backend for
+Value Stream Analytics (VSA).
+
+## Current Status
+
+As of 14.8 the aggregated VSA backend is used only in the `gitlab-org` group, for testing purposes
+. We plan to gradually roll it out in the next major release (15.0) for the rest of the groups.
+
+## Motivation
+
+The aggregated backend aims to solve the performance limitations of the VSA feature and set it up
+for long-term growth.
+
+Our main database is not prepared for analytical workloads. Executing long-running queries can
+affect the reliability of the application. For large groups, the current
+implementation (old backend) is slow and, in some cases, doesn't even load due to the configured
+statement timeout (15s).
+
+The database queries in the old backend use the core domain models directly through
+`IssuableFinders` classes: ([MergeRequestsFinder](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/finders/merge_requests_finder.rb) and [IssuesFinder](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/finders/issues_finder.rb)).
+With the requested change of the [date range filters](https://gitlab.com/groups/gitlab-org/-/epics/6046),
+this approach was no longer viable from the performance point of view.
+
+Benefits of the aggregated VSA backend:
+
+- Simpler database queries (fewer JOINs).
+- Faster aggregations, only a single table is accessed.
+- Possibility to introduce further aggregations for improving the first page load time.
+- Better performance for large groups (with many sub-groups, projects, issues and, merge requests).
+- Ready for database decomposition. The VSA related database tables could live in a separate
+database with a minimal development effort.
+- Ready for keyset pagination which can be useful for exporting the data.
+- Possibility to implement more complex event definitions.
+  - For example, the start event can be two timestamp columns where the earliest value would be
+  used by the system.
+  - Example: `MIN(issues.created_at, issues.updated_at)`
+
+## How does Value Stream Analytics work?
+
+Value Stream Analytics calculates the duration between two timestamp columns or timestamp
+expressions and runs various aggregations on the data.
+
+Examples:
+
+- Duration between the Merge Request creation time and Merge Request merge time.
+- Duration between the Issue creation time and Issue close time.
+
+This duration is exposed in various ways:
+
+- Aggregation: median, average
+- Listing: list the duration for individual Merge Request and Issue records
+
+Apart from the durations, we expose the record count within a stage.
+
+### Stages
+
+A stage represents an event pair (start and end events) with additional metadata, such as the name
+of the stage. Stages are configurable by the user within the pairing rules defined in the backend.
+
+**Example stage: Code Review**
+
+- Start event identifier: Merge Request creation time
+- Start event column: uses the `merge_requests.created_at` timestamp column.
+- End event identifier: Merge Request merge time
+- End event column: uses the `merge_request_metrics.merged_at` timestamp column.
+- Stage event hash ID: a calculated hash for the pair of start and end event identifiers.
+  - If two stages have the same configuration of start and end events, then their stage event hash
+  IDs are identical.
+  - The stage event hash ID is later used to store the aggregated data in partitioned database tables.
+
+### Value streams
+
+Value streams are container objects for the stages. There can be multiple value streams per group
+or project focusing on different aspects of the Dev Ops lifecycle.
+
+### Example configuration
+
+![vsa object hierarchy example](img/object_hierarchy_example_V14_10.png)
+
+In this example, there are two independent value streams set up for two teams that are using
+different development workflows within the `Test Group` (top-level namespace).
+
+The first value stream uses standard timestamp-based events for defining the stages. The second
+value stream uses label events.
+
+Each value stream and stage item from the example will be persisted in the database. Notice that
+the `Deployment` stage is identical for both value streams; that means that the underlying
+`stage_event_hash_id` is the same for both stages. The `stage_event_hash_id` reduces
+the amount of data the backend collects and plays a vital role in database partitioning.
+
+We expect value streams and stages to be rarely changed. When stages (start and end events) are
+changed, the aggregated data gets stale. This is fixed by the periodical aggregation occurring
+every day.
+
+### Feature availability
+
+The aggregated VSA feature is available on the group and project level however, the aggregated
+backend is only available for Premium and Ultimate customers due to data storage and data
+computation costs. Storing de-normalized, aggregated data requires significant disk space.
+
+## Aggregated value stream analytics architecture
+
+The main idea behind the aggregated VSA backend is separation: VSA database tables and queries do
+not use the core domain models directly (Issue, MergeRequest). This allows us to scale and
+optimize VSA independently from the other parts of the application.
+
+The architecture consists of two main mechanisms:
+
+- Periodical data collection and loading (happens in the background).
+- Querying the collected data (invoked by the user).
+
+### Data loading
+
+The aggregated nature of VSA comes from the periodical data loading. The system queries the core
+domain models to collect the stage and timestamp data. This data is periodically inserted into the
+VSA database tables.
+
+High-level overview for each top-level namespace with Premium or Ultimate license:
+
+1. Load all stages in the group.
+1. Iterate over the issues and merge requests records.
+1. Based on the stage configurations (start and end event identifiers) collect the timestamp data.
+1. `INSERT` or `UPDATE` the data into the VSA database tables.
+
+The data loading is implemented within the [`Analytics::CycleAnalytics::DataLoaderService`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/services/analytics/cycle_analytics/data_loader_service.rb)
+class. There are groups containing a lot of data, so to avoid overloading the primary database,
+the service performs operations in batches and enforces strict application limits:
+
+- Load records in batches.
+- Insert records in batches.
+- Stop processing when a limit is reached, schedule a background job to continue the processing
+later.
+- Continue processing data from a specific point.
+
+As of GitLab 14.7, the data loading is done manually. Once the feature is ready, the service will
+be invoked periodically by the system via a cron job (this part is not implemented yet).
+
+#### Record iteration
+
+The batched iteration is implemented with the
+[efficient IN operator](../database/efficient_in_operator_queries.md). The background job scans
+all issues and merge request records in the group hierarchy ordered by the `updated_at` and the
+`id` columns. For already aggregated groups, the `DataLoaderService` continues the aggregation
+from a specific point which saves time.
+
+Collecting the timestamp data happens on every iteration. The `DataLoaderService` determines which
+stage events are configured within the group hierarchy and builds a query that selects the
+required timestamps. The stage record knows which events are configured and the events know how to
+select the timestamp columns.
+
+Example for collected stage events: merge request merged, merge request created, merge request
+closed
+
+Generated SQL query for loading the timestamps:
+
+```sql
+SELECT
+  -- the list of columns depends on the configured stages
+  "merge_request_metrics"."merged_at",
+  "merge_requests"."created_at",
+  "merge_request_metrics"."latest_closed_at"
+  FROM "merge_requests"
+  LEFT OUTER JOIN "merge_request_metrics" ON "merge_request_metrics"."merge_request_id" = "merge_requests"."id"
+  WHERE "merge_requests"."id" IN (1, 2, 3, 4) -- ids are coming from the batching query
+```
+
+The `merged_at` column is located in a separate table (`merge_request_metrics`). The
+`Gitlab::Analytics::CycleAnalytics::StagEvents::MergeRequestMerged` class adds itself to a scope
+for loading the timestamp data without affecting the number of rows (uses `LEFT JOIN`). This
+behavior is implemented for each `StageEvent` class with the `include_in` method.
+
+The data collection query works on the event level. It extracts the event timestamps from the
+stages and ensures that we don't collect the same data multiple times. The events mentioned above
+could come from the following stage configuration:
+
+- merge request created - merge request merged
+- merge request created - merge request closed
+
+Other combinations might be also possible, but we prevent the ones that make no sense, for example:
+
+- merge request merged - merge request created
+
+Creation time always happens first, so this stage always reports negative duration.
+
+#### Data scope
+
+The data collection scans and processes all issues and merge requests records in the group
+hierarchy, starting from the top-level group. This means that if a group only has one value stream
+in a sub-group, we nevertheless collect data of all issues and merge requests in the hierarchy of
+this group. This aims to simplify the data collection mechanism. Moreover, data research shows
+that most group hierarchies have their stages configured on the top level.
+
+During the data collection process, the collected timestamp data is transformed into rows. For
+each configured stage, if the start event timestamp is present, the system inserts or updates one
+event record. This allows us to determine the upper limit of the inserted rows per group by
+counting all issues and merge requests and multiplying the sum by the stage count.
+
+#### Data consistency concerns
+
+Due to the async nature of the data collection, data consistency issues are bound to happen. This
+is a trade-off that makes the query performance significantly faster. We think that for analytical
+workload a slight lag in the data is acceptable.
+
+Before the rollout we plan to implement some indicators on the VSA page that shows the most
+recent backend activities. For example, indicators that show the last data collection timestamp
+and the last consistency check timestamp.
+
+#### Database structure
+
+VSA collects data for the following domain models: `Issue` and `MergeRequest`. To keep the
+aggregated data separated, we use two additional database tables:
+
+- `analytics_cycle_analytics_issue_stage_events`
+- `analytics_cycle_analytics_merge_request_stage_events`
+
+Both tables are hash partitioned by the `stage_event_hash_id`. Each table uses 32 partitions. It's
+an arbitrary number and it could be changed. Important is to keep the partitions under 100GB in
+size (which gives the feature a lot of headroom).
+
+|Column|Description|
+|-|-|
+|`stage_event_hash_id`|partitioning key|
+|`merge_request_id` or `issue_id`|reference to the domain record (Issuable)|
+|`group_id`|reference to the group (de-normalization)|
+|`project_id`|reference to the project|
+|`milestone_id`|duplicated data from the domain record table|
+|`author_id`|duplicated data from the domain record table|
+|`state_id`|duplicated data from the domain record table|
+|`start_event_timestamp`|timestamp derived from the stage configuration|
+|`end_event_timestamp`|timestamp derived from the stage configuration|
+
+With accordance to the data separation requirements, the table doesn't have any foreign keys. The
+consistency is ensured by a background job (eventually consistent).
+
+### Data querying
+
+The base query always includes the following filters:
+
+- `stage_event_hash_id` - partition key
+- `project_id` or `group_id` - depending if it's a project or group level query
+- `end_event_timestamp` - date range filter (last 30 days)
+
+Example: Selecting review stage duration for the GitLab project
+
+```sql
+SELECT end_event_timestamp - start_event_timestamp
+FROM analytics_cycle_analytics_merge_request_stage_events
+WHERE
+stage_event_hash_id = 16 AND -- hits a specific partition
+project_id = 278964 AND
+end_event_timestamp > '2022-01-01' AND end_event_timestamp < '2022-01-30'
+```
+
+#### Query generation
+
+The query backend is hidden behind the same interface that the old backend implementation uses.
+Thanks to this, we can easily switch between the old and new query backends.
+
+- `DataCollector`: entrypoint for querying VSA data
+  - `BaseQueryBuilder`: provides the base `ActiveRecord` scope (filters are applied here).
+  - `average`: average aggregation.
+  - `median`: median aggregation.
+  - `count`: row counting.
+  - `records`: list of issue or merge request records.
+
+#### Filters
+
+VSA supports various filters on the base query. Most of the filters require no additional JOINs:
+
+|Filter name|Description|
+|-|-|
+|`milestone_title`|The backend translates it to `milestone_id` filter|
+|`author_username`|The backend translates it to `author_id` filter|
+|`project_ids`|Only used on the group-level|
+
+Exceptions: these filters are applied on other tables which means we `JOIN` them.
+
+|Filter name|Description|
+|-|-|
+|`label_name`|Array filter, using the `label_links` table|
+|`assignee_username`|Array filter, using the `*_assignees` table|
+
+To fully decompose the database, the required ID values would need to be replicated in the VSA
+database tables. This change could be implemented using array columns.
+
+### Endpoints
+
+The feature uses private JSON APIs for delivering the data to the frontend. On the first page load
+, the following requests are invoked:
+
+- Initial HTML page load which is mostly empty. Some configuration data is exposed via `data`
+attributes.
+- `value_streams` - Load the available value streams for the given group.
+- `stages` - Load the stages for the currently selected value stream.
+- `median` - For each stage, request the median duration.
+- `count` - For each stage, request the number of items in the stage (this is a
+[limit count](../merge_request_performance_guidelines.md#badge-counters), maximum 1000 rows).
+- `average_duration_chart` - Data for the duration chart.
+- `summary`, `time_summary` - Top-level aggregations, most of the metrics are using different APIs/
+finders and not invoking the aggregated backend.
+
+When clicking on a specific stage, the `records` endpoint is invoked, which returns the related
+records (paginated) for the chosen stage in a specific order.
+
+### Database decomposition
+
+By separating the query logic from the main application code, the feature is ready for database
+decomposition. If we decide that VSA requires a separate database instance, then moving the
+aggregated tables can be accomplished with little effort.
+
+A different database technology could also be used to further improve the performance of the
+feature, for example [Timescale DB](https://www.timescale.com).
author	GitLab Bot <gitlab-bot@gitlab.com>	2022-02-18 12:45:46 +0300
committer	GitLab Bot <gitlab-bot@gitlab.com>	2022-02-18 12:45:46 +0300
commit	a7b3560714b4d9cc4ab32dffcd1f74a284b93580 (patch)
tree	7452bd5c3545c2fa67a28aa013835fb4fa071baf /doc/development
parent	ee9173579ae56a3dbfe5afe9f9410c65bb327ca7 (diff)