Add latest changes from gitlab-org/gitlab@16-1-stable-eev16.1.0-rc42

135 files changed, 6543 insertions, 4872 deletions

diff --git a/doc/development/ai_architecture.md b/doc/development/ai_architecture.md
index e9994c8a6f4..ac62f50baf5 100644
--- a/doc/development/ai_architecture.md
+++ b/doc/development/ai_architecture.md
@@ -84,7 +84,7 @@ Web --> AIF
 
 GitLab currently operates a cloud-hosted AI architecture. We are exploring how self-managed instances integrate with it.
 
-There are two primary reasons for this: the best AI models are cloud-based as they often depend on specialized hardware designed for this purpose, and operating self-managed infrastructure capable of AI at-scale and with appropriate performance is a significant undertaking. We are actively [tracking self-managed customers interested in AI](https://gitlab.com/gitlab-org/gitlab/-/issues/409183). 
+There are two primary reasons for this: the best AI models are cloud-based as they often depend on specialized hardware designed for this purpose, and operating self-managed infrastructure capable of AI at-scale and with appropriate performance is a significant undertaking. We are actively [tracking self-managed customers interested in AI](https://gitlab.com/gitlab-org/gitlab/-/issues/409183).
 
 ## Supported technologies
 
diff --git a/doc/development/ai_features.md b/doc/development/ai_features.md
index 6ed1d59c3e0..c46117c4afd 100644
--- a/doc/development/ai_features.md
+++ b/doc/development/ai_features.md
@@ -66,7 +66,9 @@ All AI features are experimental.
    1. Enable **Experiment features**
    1. Enable **Third-party AI services**
 1. Enable the specific feature flag for the feature you want to test
-1. Set either the required access token `OpenAi` or `Vertex`. Ask in [`#ai_enablement_team`](https://gitlab.slack.com/archives/C051K31F30R) to receive an access token.
+1. Set the required access token. To receive an access token:
+   1. For Vertex, follow the [instructions below](#configure-gcp-vertex-access).
+   1. For all other providers, like Anthropic or OpenAI, create an access request where `@m_gill`, `@wayne`, and `@timzallmann` are the tech stack owners.
 
 ### Set up the embedding database
 
@@ -82,28 +84,38 @@ For features that use the embedding database, additional setup is needed.
 1. Run `gdk reconfigure`
 1. Run database migrations to create the embedding database
 
-### Setup for GitLab chat
+### Setup for GitLab documentation chat (legacy chat)
 
 To populate the embedding database for GitLab chat:
 
 1. Open a rails console
 1. Run [this script](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/10588#note_1373586079) to populate the embedding database
 
+### Debugging
+
+To gather more insights about the full request, use the `Gitlab::Llm::Logger` file to debug logs.
+To follow the debugging messages related to the AI requests on the abstraction layer, you can use:
+
+```shell
+tail -f log/llm.log
+```
+
 ### Configure GCP Vertex access
 
 In order to obtain a GCP service key for local development, please follow the steps below:
 
-- Create a sandbox GCP environment by visiting [this page](https://about.gitlab.com/handbook/infrastructure-standards/#individual-environment) and following the instructions
+- Create a sandbox GCP environment by visiting [this page](https://about.gitlab.com/handbook/infrastructure-standards/#individual-environment) and following the instructions, or by requesting access to our existing group environment by using [this template](https://gitlab.com/gitlab-com/it/infra/issue-tracker/-/issues/new?issuable_template=gcp_group_account_iam_update_request). At this time, access to any endpoints outside of `text-bison` or `chat-bison` must be made through the group environment.
 - In the GCP console, go to `IAM & Admin` > `Service Accounts` and click on the "Create new service account" button
 - Name the service account something specific to what you're using it for. Select Create and Continue. Under `Grant this service account access to project`, select the role `Vertex AI User`. Select `Continue` then `Done`
-- Select your new service account and `Manage keys` > `Add Key` > `Create new key`. This will download the **private** JSON credentials for your service account. Your full settings should then be:
+- Select your new service account and `Manage keys` > `Add Key` > `Create new key`. This will download the **private** JSON credentials for your service account.
+- Open the Rails console. Update the settings to:
 
 ```ruby
-Gitlab::CurrentSettings.update(tofa_credentials: File.read('/YOUR_FILE.json'))
+Gitlab::CurrentSettings.update(vertex_ai_credentials: File.read('/YOUR_FILE.json'))
 
 # Note: These credential examples will not work locally for all models
-Gitlab::CurrentSettings.update(tofa_host: "<root-domain>") # Example: us-central1-aiplatform.googleapis.com
-Gitlab::CurrentSettings.update(tofa_url: "<full-api-endpoint>") # Example: https://ROOT-DOMAIN/v1/projects/MY-COOL-PROJECT/locations/us-central1/publishers/google/models/MY-SPECIAL-MODEL:predict
+Gitlab::CurrentSettings.update(vertex_ai_host: "<root-domain>") # Example: us-central1-aiplatform.googleapis.com
+Gitlab::CurrentSettings.update(vertex_ai_project: "<project-id>") # Example: cloud-large-language-models
 ```
 
 Internal team members can [use this snippet](https://gitlab.com/gitlab-com/gl-infra/production/-/snippets/2541742) for help configuring these endpoints.
@@ -308,7 +320,6 @@ We recommend to use [policies](policies.md) to deal with authorization for a fea
 1. Feature specific feature flag is enabled
 1. The namespace has the required license for the feature
 1. User is a member of the group/project
-1. Resource is allowed to be sent (see `send_to_ai?` method)
 1. `experiment_features_enabled` and `third_party_ai_features_enabled` flags are set on the `Namespace`
 
 For our example, we need to implement the `allowed?(:amazing_new_ai_feature)` call. As an example, you can look at the [Issue Policy for the summarize comments feature](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/policies/ee/issue_policy.rb). In our example case, we want to implement the feature for Issues as well:
@@ -322,8 +333,7 @@ module EE
     prepended do
       with_scope :subject
       condition(:ai_available) do
-        ::Feature.enabled?(:openai_experimentation) &&
-          @subject.send_to_ai?
+        ::Feature.enabled?(:openai_experimentation)
       end
 
       with_scope :subject
@@ -340,19 +350,6 @@ module EE
 end
 ```
 
-### Implement `send_to_ai?`
-
-To make sure we only send data that is allowed to be sent, we have the `send_to_ai?` method. It checks if the resource is not confidential and public data.
-Some resources already implement `send_to_ai?`. Make sure yours does as well. In our case, `Issue` is already covered with the `Issuable` concern. This is an example how it could look like:
-
-```ruby
-# ee/app/models/concerns/ee
-
-def send_to_ai?
-  !try(:confidential) && resource_parent.public?
-end
-```
-
 ### Check if feature is allowed for this resource based on namespace settings
 
 There are two settings allowed on root namespace level that restrict the use of AI features:
@@ -444,6 +441,13 @@ module Gitlab
 end
 ```
 
+Because we support multiple AI providers, you may also use those providers for the same example:
+
+```ruby
+Gitlab::Llm::VertexAi::Client.new(user)
+Gitlab::Llm::Anthropic::Client.new(user)
+```
+
 ### Add Ai Action to GraphQL
 
 TODO
diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md
index 15b587e3b1e..d6492b7814e 100644
--- a/doc/development/api_graphql_styleguide.md
+++ b/doc/development/api_graphql_styleguide.md
@@ -792,7 +792,7 @@ The documentation mentions that the old Global ID style is now deprecated.
 ## Mark schema items as Alpha
 
 You can mark GraphQL schema items (fields, arguments, enum values, and mutations) as
-[Alpha](../policy/alpha-beta-support.md#experiment).
+[Alpha](../policy/experiment-beta-support.md#experiment).
 
 An item marked as Alpha is [exempt from the deprecation process](#breaking-change-exemptions) and can be removed
 at any time without notice. Mark an item as Alpha when it is
diff --git a/doc/development/api_styleguide.md b/doc/development/api_styleguide.md
index 0652b88617b..566267b97f1 100644
--- a/doc/development/api_styleguide.md
+++ b/doc/development/api_styleguide.md
@@ -303,7 +303,7 @@ from the server to the platform if we identify invalid parameters at the beginni
 If you need to add a custom validator, it would be added to
 it's own file in the [`validators`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/validations/validators) directory.
 Since we use [Grape](https://github.com/ruby-grape/grape) to add our API
-we inherit from the `Grape::Validations::Base` class in our validator class.
+we inherit from the `Grape::Validations::Validators::Base` class in our validator class.
 Now, all you have to do is define the `validate_param!` method which takes
 in two parameters: the `params` hash and the `param` name to validate.
 
diff --git a/doc/development/bulk_import.md b/doc/development/bulk_import.md
index b1fa4726b65..304aab9e3b3 100644
--- a/doc/development/bulk_import.md
+++ b/doc/development/bulk_import.md
@@ -8,9 +8,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2771) in GitLab 13.7.
 
-WARNING:
-This feature is [under construction](https://gitlab.com/groups/gitlab-org/-/epics/2771) and its API/Architecture might change in the future.
-
 [Group migration by direct transfer](../user/group/import/index.md#migrate-groups-by-direct-transfer-recommended) is the
 evolution of migrating groups and projects using file exports. The goal is to have an easier way for the user to migrate a whole group,
 including projects, from one GitLab instance to another.
diff --git a/doc/development/caching.md b/doc/development/caching.md
index dff94b68ca0..c3385c8499d 100644
--- a/doc/development/caching.md
+++ b/doc/development/caching.md
@@ -267,7 +267,7 @@ All the time!
 
 #### Possible downsides
 
-- Adding new attributes to a cached object using `Gitlab::JsonCache`
+- Adding new attributes to a cached object using `Gitlab::Cache::JsonCache`
   and `Gitlab::SafeRequestStore`, for example, can lead to stale data issues
   where the cache data doesn't have the appropriate value for the new attribute
   (see this past [incident](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/6372)).
diff --git a/doc/development/chatops_on_gitlabcom.md b/doc/development/chatops_on_gitlabcom.md
index 22f57e8ba43..002470f7325 100644
--- a/doc/development/chatops_on_gitlabcom.md
+++ b/doc/development/chatops_on_gitlabcom.md
@@ -1,6 +1,6 @@
 ---
-stage: Manage
-group: Import and Integrate
+stage: Deploy
+group: Environments
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
 ---
 
diff --git a/doc/development/cicd/cicd_tables.md b/doc/development/cicd/cicd_tables.md
index b246328a817..8cfb0faca00 100644
--- a/doc/development/cicd/cicd_tables.md
+++ b/doc/development/cicd/cicd_tables.md
@@ -29,7 +29,7 @@ Here is an example on how to use database helpers to create a new table and fore
     end
 
     add_concurrent_partitioned_foreign_key(
-      :p_ci_examples, :ci_builds,
+      :p_ci_examples, :p_ci_builds,
       column: [:partition_id, :build_id],
       target_column: [:partition_id, :id],
       on_update: :cascade,
@@ -51,7 +51,7 @@ When creating the routing table:
 - The table name must start with the `p_` prefix. There are analyzers in place to ensure that all queries go
   through the routing tables and do not access the partitions directly.
 - Each new table needs a `partition_id` column and its value must equal
-  the value from the related association. In this example, that is `ci_builds`. All resources
+  the value from the related association. In this example, that is `p_ci_builds`. All resources
   belonging to a pipeline share the same `partition_id` value.
 - The primary key must have the columns ordered this way to allow efficient
   search only by `id`.
@@ -74,7 +74,7 @@ the application runs:
   def up
     with_lock_retries do
       connection.execute(<<~SQL)
-        LOCK TABLE ci_builds IN SHARE UPDATE EXCLUSIVE MODE;
+        LOCK TABLE p_ci_builds IN SHARE ROW EXCLUSIVE MODE;
         LOCK TABLE ONLY p_ci_examples IN ACCESS EXCLUSIVE MODE;
       SQL
 
@@ -92,7 +92,7 @@ Partitions are created in `gitlab_partitions_dynamic` schema.
 When creating a partition, remember:
 
 - Partition names do not use the `p_` prefix.
-- The default value for `partition_id` is `100`.
+- The starting value for `partition_id` is `100`.
 
 ## Cascade the partition value
 
diff --git a/doc/development/cicd/index.md b/doc/development/cicd/index.md
index 2cc8fca02dd..b186c37f933 100644
--- a/doc/development/cicd/index.md
+++ b/doc/development/cicd/index.md
@@ -185,12 +185,14 @@ We have a few inconsistencies in our codebase that should be refactored.
 For example, `CommitStatus` should be `Ci::Job` and `Ci::JobArtifact` should be `Ci::BuildArtifact`.
 See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/16111) for the full refactoring plan.
 
-## CI/CD Minutes
+## Compute quota
 
-This diagram shows how the [CI/CD minutes](../../ci/pipelines/cicd_minutes.md)
+> [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" and "units of compute" in GitLab 16.1.
+
+This diagram shows how the [Compute quota](../../ci/pipelines/cicd_minutes.md)
 feature and its components work.
 
-![CI/CD minutes architecture](img/ci_minutes.png)
+![compute quota architecture](img/ci_minutes.png)
 <!-- Editable diagram available at https://app.diagrams.net/?libs=general;flowchart#G1XjLPvJXbzMofrC3eKRyDEk95clV6ypOb -->
 
 Watch a walkthrough of this feature in details in the video below.
diff --git a/doc/development/cicd/templates.md b/doc/development/cicd/templates.md
index 1bf4a780e26..77e529867af 100644
--- a/doc/development/cicd/templates.md
+++ b/doc/development/cicd/templates.md
@@ -424,7 +424,7 @@ To add a metric definition for a new template:
 
 1. Install and start the [GitLab GDK](https://gitlab.com/gitlab-org/gitlab-development-kit#installation).
 1. In the `gitlab` directory in your GDK, check out the branch that contains the new template.
-1. [Add the template inclusion event](../service_ping/implement.md#add-new-events)
+1. [Add the template inclusion event](../internal_analytics/service_ping/implement.md#add-new-events)
    with this Rake task:
 
    ```shell
@@ -445,7 +445,7 @@ To add a metric definition for a new template:
    - [`config/metrics/counts_28d/20210216184559_ci_templates_total_unique_counts_monthly.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216184559_ci_templates_total_unique_counts_monthly.yml)
 
 1. Use the same `name` as above as the last argument in the following command to
-  [add new metric definitions](../service_ping/metrics_dictionary.md#metrics-added-dynamic-to-service-ping-payload):
+  [add new metric definitions](../internal_analytics/service_ping/metrics_dictionary.md#metrics-added-dynamic-to-service-ping-payload):
 
    ```shell
    bundle exec rails generate gitlab:usage_metric_definition:redis_hll ci_templates <template_metric_event_name>
@@ -466,7 +466,7 @@ To add a metric definition for a new template:
    - `data_source:`: Set to `redis_hll`.
    - `description`: Add a short description of what this metric counts, for example: `Count of pipelines using the latest Auto Deploy template`
    - `product_*`: Set to [section, stage, group, and feature category](https://about.gitlab.com/handbook/product/categories/#devops-stages)
-     as per the [metrics dictionary guide](../service_ping/metrics_dictionary.md#metrics-definition-and-validation).
+     as per the [metrics dictionary guide](../internal_analytics/service_ping/metrics_dictionary.md#metrics-definition-and-validation).
      If you are unsure what to use for these keywords, you can ask for help in the merge request.
    - Add the following to the end of each file:
 
diff --git a/doc/development/code_review.md b/doc/development/code_review.md
index f2edc882d91..e9f00b640d9 100644
--- a/doc/development/code_review.md
+++ b/doc/development/code_review.md
@@ -28,12 +28,23 @@ The reviewer can:
 - Give you a second opinion on the chosen solution and implementation.
 - Help look for bugs, logic problems, or uncovered edge cases.
 
-If the merge request is trivial to review (for example, fixing a typo or a tiny refactor that doesn't change the behavior or any data),
-you can skip the reviewer step and directly ask a [maintainer](https://about.gitlab.com/handbook/engineering/workflow/code-review/#maintainer).
-Otherwise, a merge request should always be first reviewed by a reviewer in each
-[category (e.g. backend, database)](#approval-guidelines)
-the MR touches, as maintainers may not have the relevant domain knowledge, and
-also to spread the workload.
+If the merge request is small and straightforward to review, you can skip the reviewer step and
+directly ask a
+[maintainer](https://about.gitlab.com/handbook/engineering/workflow/code-review/#maintainer).
+
+What constitutes "small and straightforward" is a gray area. Here are
+some examples of small and straightforward changes:
+
+- Fixing a typo or making small copy changes ([example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121337#note_1399406719)).
+- A tiny refactor that doesn't change any behavior or data.
+- Removing references to a feature flag that has been default enabled for > 1 month.
+- Removing unused methods or classes.
+- A well-understood logic change that requires changes to < 5 lines of code.
+
+Otherwise, a merge request should be first reviewed by a reviewer in each
+[category (for example: backend, database)](#approval-guidelines)
+the MR touches, as maintainers may not have the relevant domain knowledge. This
+also helps to spread the workload.
 
 For assistance with security scans or comments, include the Application Security Team (`@gitlab-com/gl-security/appsec`).
 
@@ -47,6 +58,14 @@ The **Approved** button is in the merge request widget.
 Getting your merge request **merged** also requires a maintainer. If it requires
 more than one approval, the last maintainer to review and approve merges it.
 
+Some domain areas (like `Verify`) require an approval from a domain expert, based on
+CODEOWNERS rules. Because CODEOWNERS sections are independent approval rules, we could have certain
+rules (for example `Verify`) that may be a subset of other more generic approval rules (for example `backend`).
+For a more efficient process, authors should look for domain-specific approvals before generic approvals.
+Domain-specific approvers may also be maintainers, and if so they should review
+the domain specifics and broader change at the same time and approve once for
+both roles.
+
 Read more about [author responsibilities](#the-responsibility-of-the-merge-request-author) below.
 
 ### Domain experts
@@ -150,35 +169,38 @@ please use in your personal YAML file entry: `- reviewer database local` instead
 
 As described in the section on the responsibility of the maintainer below, you
 are recommended to get your merge request approved and merged by maintainers
-with [domain expertise](#domain-experts).
+with [domain expertise](#domain-experts). The optional approval of the first
+reviewer is not covered here. However, your merge request should be reviewed
+by a reviewer before passing it to a maintainer as described in the
+[overview](#getting-your-merge-request-reviewed-approved-and-merged) section.
 
 | If your merge request includes  | It must be approved by a |
 | ------------------------------- | ------------------------ |
-| `~backend` changes (*1*)        | [Backend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_backend). |
-| `~database` migrations or changes to expensive queries (*2*) | [Database maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_database). Refer to the [database review guidelines](database_review.md) for more details. |
+| `~backend` changes <sup>1</sup>        | [Backend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_backend). |
+| `~database` migrations or changes to expensive queries <sup>2</sup> | [Database maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_database). Refer to the [database review guidelines](database_review.md) for more details. |
 | `~workhorse` changes | [Workhorse maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_workhorse). |
-| `~frontend` changes (*1*)       | [Frontend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_frontend). |
-| `~UX` user-facing changes (*3*) | [Product Designer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_UX). Refer to the [design and user interface guidelines](contributing/design.md) for details. |
-| Adding a new JavaScript library (*1*) | - [Frontend foundations member](https://about.gitlab.com/direction/manage/foundations/) if the library significantly increases the [bundle size](https://gitlab.com/gitlab-org/frontend/playground/webpack-memory-metrics/-/blob/master/doc/report.md).<br/>- A [legal department member](https://about.gitlab.com/handbook/legal/) if the license used by the new library hasn't been approved for use in GitLab.<br/><br/>More information about license compatibility can be found in our [GitLab Licensing and Compatibility documentation](licensing.md). |
+| `~frontend` changes <sup>1</sup>       | [Frontend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_frontend). |
+| `~UX` user-facing changes <sup>3</sup> | [Product Designer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_UX). Refer to the [design and user interface guidelines](contributing/design.md) for details. |
+| Adding a new JavaScript library <sup>1</sup> | - [Frontend foundations member](https://about.gitlab.com/direction/manage/foundations/) if the library significantly increases the [bundle size](https://gitlab.com/gitlab-org/frontend/playground/webpack-memory-metrics/-/blob/master/doc/report.md).<br/>- A [legal department member](https://about.gitlab.com/handbook/legal/) if the license used by the new library hasn't been approved for use in GitLab.<br/><br/>More information about license compatibility can be found in our [GitLab Licensing and Compatibility documentation](licensing.md). |
 | A new dependency or a file system change | - [Distribution team member](https://about.gitlab.com/company/team/). See how to work with the [Distribution team](https://about.gitlab.com/handbook/engineering/development/enablement/systems/distribution/#how-to-work-with-distribution) for more details.<br/>- For RubyGems, request an [AppSec review](gemfile.md#request-an-appsec-review). |
 | `~documentation` or `~UI text` changes | [Technical writer](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments) based on assignments in the appropriate [DevOps stage group](https://about.gitlab.com/handbook/product/categories/#devops-stages). |
 | Changes to development guidelines | Follow the [review process](development_processes.md#development-guidelines-review) and get the approvals accordingly. |
-| End-to-end **and** non-end-to-end changes (*4*) | [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors). |
-| Only End-to-end changes (*4*) **or** if the MR author is a [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors) | [Quality maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_qa). |
+| End-to-end **and** non-end-to-end changes <sup>4</sup> | [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors). |
+| Only End-to-end changes <sup>4</sup> **or** if the MR author is a [Software Engineer in Test](https://about.gitlab.com/handbook/engineering/quality/#individual-contributors) | [Quality maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_qa). |
 | A new or updated [application limit](https://about.gitlab.com/handbook/product/product-processes/#introducing-application-limits) | [Product manager](https://about.gitlab.com/company/team/). |
-| Analytics Instrumentation (telemetry or analytics) changes | [Analytics Instrumentation engineer](https://gitlab.com/gitlab-org/analytics-section/product-intelligence/engineers). |
+| Analytics Instrumentation (telemetry or analytics) changes | [Analytics Instrumentation engineer](https://gitlab.com/gitlab-org/analytics-section/analytics-instrumentation/engineers). |
 | An addition of, or changes to a [Feature spec](testing_guide/testing_levels.md#frontend-feature-tests) | [Quality maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_qa) or [Quality reviewer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_qa). |
 | A new service to GitLab (Puma, Sidekiq, Gitaly are examples) | [Product manager](https://about.gitlab.com/company/team/). See the [process for adding a service component to GitLab](adding_service_component.md) for details. |
 | Changes related to authentication or authorization | [Manage:Authentication and Authorization team member](https://about.gitlab.com/company/team/). Check the [code review section on the group page](https://about.gitlab.com/handbook/engineering/development/dev/manage/authentication-and-authorization/#additional-considerations) for more details. Patterns for files known to require review from the team are listed in the in the `Authentication and Authorization` section of the [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS) file, and the team will be listed in the approvers section of all merge requests that modify these files. |
 
-- (*1*): Specs other than JavaScript specs are considered `~backend` code. Haml markup is considered `~frontend` code. However, Ruby code within Haml templates is considered `~backend` code. When in doubt, request both a frontend and backend review.
-- (*2*): We encourage you to seek guidance from a database maintainer if your merge
-  request is potentially introducing expensive queries. It is most efficient to comment
-  on the line of code in question with the SQL queries so they can give their advice.
-- (*3*): User-facing changes include both visual changes (regardless of how minor),
-  and changes to the rendered DOM which impact how a screen reader may announce
-  the content.
-- (*4*): End-to-end changes include all files within the `qa` directory.
+1. Specs other than JavaScript specs are considered `~backend` code. Haml markup is considered `~frontend` code. However, Ruby code in Haml templates is considered `~backend` code. When in doubt, request both a frontend and backend review.
+1. We encourage you to seek guidance from a database maintainer if your merge
+   request is potentially introducing expensive queries. It is most efficient to comment
+   on the line of code in question with the SQL queries so they can give their advice.
+1. User-facing changes include both visual changes (regardless of how minor),
+   and changes to the rendered DOM which impact how a screen reader may announce
+   the content.
+1. End-to-end changes include all files in the `qa` directory.
 
 #### Acceptance checklist
 
@@ -210,6 +232,7 @@ See the [test engineering process](https://about.gitlab.com/handbook/engineering
 1. You have considered the availability and reliability risks of this change.
 1. You have considered the scalability risk based on future predicted growth.
 1. You have considered the performance, reliability, and availability impacts of this change on large customers who may have significantly more data than the average customer.
+1. You have considered the performance, reliability, and availability impacts of this change on customers who may run GitLab on the [minimum system](../install/requirements.md).
 
 ##### Observability instrumentation
 
@@ -372,7 +395,7 @@ codebase, and not that of any specific domain, they can review, approve, and mer
 merge requests from any team and in any product area.
 
 Maintainers are the DRI of assuring that the acceptance criteria of a merge request are reasonably met.
-In general, [quality is everyone’s responsibility](https://about.gitlab.com/handbook/engineering/quality/),
+In general, [quality is everyone's responsibility](https://about.gitlab.com/handbook/engineering/quality/),
 but maintainers of an MR are held responsible for **ensuring** that an MR meets those general quality standards.
 
 If a maintainer feels that an MR is substantial enough, or requires a [domain expert](#domain-experts),
@@ -405,10 +428,10 @@ On March 18th 2021, an updated process was put in place aimed at efficiently and
 
 Here is a summary of the changes, also reflected in this section above.
 
-- Merge request authors and DRIs stay as Assignees
-- Authors request a review from Reviewers when they are expected to review
-- Reviewers remove themselves after they're done reviewing/approving
-- The last approver stays as Reviewer upon merging
+- Merge request authors and DRIs stay as Assignees.
+- Authors request a review by assigning users as Reviewers.
+- Reviewers unassign themselves after they're done reviewing and approving.
+- The last approver (who merges the MR) stays assigned as Reviewer.
 
 ## Best practices
 
@@ -541,28 +564,37 @@ Before taking the decision to merge:
 - If the MR contains both Quality and non-Quality-related changes, the MR should be merged by the relevant maintainer for user-facing changes (backend, frontend, or database) after the Quality related changes are approved by a Software Engineer in Test.
 
 At least one maintainer must approve an MR before it can be merged. MR authors and
-people who add commits to an MR are not authorized to approve or merge the MR and
-must seek a maintainer who has not contributed to the MR to approve and merge it.
+people who add commits to an MR are not authorized to approve the MR and
+must seek a maintainer who has not contributed to the MR to approve it. In
+general, the final required approver should merge the MR.
+
+Scenarios in which the final approver might not merge an MR:
+
+- Approver forgets to set auto-merge after approving.
+- Approver doesn't realize that they are the final approver.
+- Approver sets auto-merge but it is un-set by GitLab.
+
+If any of these scenarios occurs, an MR author may merge their own MR if it
+has all required approvals and they have merge rights to the repository.
+This is also in line with the GitLab [bias for action](https://handbook.gitlab.com/handbook/values/#bias-for-action) value.
 
 This policy is in place to satisfy the CHG-04 control of the GitLab
 [Change Management Controls](https://about.gitlab.com/handbook/security/change-management-policy.html).
 
-<!-- Or should it link to: https://about.gitlab.com/handbook/engineering/infrastructure/change-management/ ? -->
-
 To implement this policy in `gitlab-org/gitlab`, we have enabled the following
 settings to ensure MRs get an approval from a top-level CODEOWNERS maintainer:
 
 - [Prevent approval by author](../user/project/merge_requests/approvals/settings.md#prevent-approval-by-author).
 - [Prevent approvals by users who add commits](../user/project/merge_requests/approvals/settings.md#prevent-approvals-by-users-who-add-commits).
 - [Prevent editing approval rules in merge requests](../user/project/merge_requests/approvals/settings.md#prevent-editing-approval-rules-in-merge-requests).
-- [Remove all approvals when commits are added to the source branch](../user/project/merge_requests/approvals/settings.md#remove-all-approvals-when-commits-are-added-to-the-source-branch)
+- [Remove all approvals when commits are added to the source branch](../user/project/merge_requests/approvals/settings.md#remove-all-approvals-when-commits-are-added-to-the-source-branch).
 
 To update the code owners in the `CODEOWNERS` file for `gitlab-org/gitlab`, follow
 the process explained in the [code owners approvals handbook section](https://about.gitlab.com/handbook/engineering/workflow/code-review/#code-owner-approvals).
 
-  There are scenarios such as rebasing locally or applying suggestions that are considered
-  the same as adding a commit and could reset existing approvals. Approvals are not removed
-  when rebasing from the UI or with the [`/rebase` quick action](../user/project/quick_actions.md).
+Some actions, such as rebasing locally or applying suggestions, are considered
+the same as adding a commit and could reset existing approvals. Approvals are not removed
+when rebasing from the UI or with the [`/rebase` quick action](../user/project/quick_actions.md).
 
 When ready to merge:
 
@@ -576,8 +608,7 @@ WARNING:
   messy commit history, it will be more efficient to squash commits instead of
   circling back with the author about that. Otherwise, if the MR only has a few commits, we'll
   be respecting the author's setting by not squashing them.
-- Start a new merge request pipeline with the `Run pipeline` button in the merge
-  request's "Pipelines" tab, and enable "Merge When Pipeline Succeeds" (MWPS).
+- Go to the merge request's **Pipelines** tab, and select **Run pipeline**. Then, on the **Overview** tab, enable **Auto-merge**.
   Note that:
   - If **[the default branch is broken](https://about.gitlab.com/handbook/engineering/workflow/#broken-master),
     do not merge the merge request** except for
@@ -587,7 +618,7 @@ WARNING:
   - If the **latest [merged results pipeline](../ci/pipelines/merged_results_pipelines.md)** was **created less than 6 hours ago**, and **finished less than 2 hours ago**, you
     may merge without starting a new pipeline as the merge request is close
     enough to `main`.
-- When you set the MR to "Merge When Pipeline Succeeds", you should take over
+- When you set the MR to auto-merge, you should take over
   subsequent revisions for anything that would be spotted after that.
 - For merge requests that have had [Squash and merge](../user/project/merge_requests/squash_and_merge.md#squash-and-merge) set,
   the squashed commit's default commit message is taken from the merge request title.
@@ -597,7 +628,7 @@ Thanks to **merged results pipelines**, authors no longer have to rebase their
 branch as frequently anymore (only when there are conflicts) because the Merge
 Results Pipeline already incorporate the latest changes from `main`.
 This results in faster review/merge cycles because maintainers don't have to ask
-for a final rebase: instead, they only have to start a MR pipeline and set MWPS.
+for a final rebase: instead, they only have to start a MR pipeline and set auto-merge.
 This step brings us very close to the actual Merge Trains feature by testing the
 Merge Results against the latest `main` at the time of the pipeline creation.
 
@@ -733,7 +764,7 @@ A merge request may benefit from being considered a customer critical priority b
 
 Properties of customer critical merge requests:
 
-- The [VP of Development](https://about.gitlab.com/job-families/engineering/development/management/vp/) ([@clefelhocz1](https://gitlab.com/clefelhocz1)) is the DRI for deciding if a merge request qualifies as customer critical.
+- The [VP of Development](https://about.gitlab.com/job-families/engineering/development/management/vp/) ([@clefelhocz1](https://gitlab.com/clefelhocz1)) is the approver for deciding if a merge request qualifies as customer critical. Also, if two of his direct reports approve, that can also serve as approval.
 - The DRI applies the `customer-critical-merge-request` label to the merge request.
 - It is required that the reviewers and maintainers involved with a customer critical merge request are engaged as soon as this decision is made.
 - It is required to prioritize work for those involved on a customer critical merge request so that they have the time available necessary to focus on it.
diff --git a/doc/development/contributing/design.md b/doc/development/contributing/design.md
index d68bc194266..58ec0de6074 100644
--- a/doc/development/contributing/design.md
+++ b/doc/development/contributing/design.md
@@ -73,7 +73,7 @@ Check visual design properties using your browser's _elements inspector_ ([Chrom
   guidelines.
 - _Optionally_ consider [dark mode](../../user/profile/preferences.md#dark-mode). [^1]
 
- [^1]: You're not required to design for [dark mode](../../user/profile/preferences.md#dark-mode) while the feature is an [Experiment](../../policy/alpha-beta-support.md#experiment). The [UX Foundations team](https://about.gitlab.com/direction/manage/foundations/) plans to improve the dark mode in the future. Until we integrate [Pajamas](https://design.gitlab.com/) components into the product and the underlying design strategy is in place to support dark mode, we cannot guarantee that we won't introduce bugs and debt to this mode. At your discretion, evaluate the need to create dark mode patches.
+ [^1]: You're not required to design for [dark mode](../../user/profile/preferences.md#dark-mode) while the feature is an [Experiment](../../policy/experiment-beta-support.md#experiment). The [UX Foundations team](https://about.gitlab.com/direction/manage/foundations/) plans to improve the dark mode in the future. Until we integrate [Pajamas](https://design.gitlab.com/) components into the product and the underlying design strategy is in place to support dark mode, we cannot guarantee that we won't introduce bugs and debt to this mode. At your discretion, evaluate the need to create dark mode patches.
 
 ### States
 
@@ -110,8 +110,8 @@ Check accessibility using your browser's _accessibility inspector_ ([Chrome](htt
 
 When the design is ready, _before_ starting its implementation:
 
-- Share design specifications in the related issue, preferably through a [Figma link](https://help.figma.com/hc/en-us/articles/360040531773-Share-Files-with-anyone-using-Link-Sharing#copy-link)
-  link or [GitLab Designs feature](../../user/project/issues/design_management.md).
+- Share design specifications in the related issue, preferably through a [Figma link](https://help.figma.com/hc/en-us/articles/360040531773-Share-Files-with-anyone-using-Link-Sharing#Copy_link)
+  or [GitLab Designs feature](../../user/project/issues/design_management.md).
   See [when you should use each tool](https://about.gitlab.com/handbook/product/ux/product-designer/#deliver).
 - Document user flow and states (for example, using [Mermaid flowcharts in Markdown](../../user/markdown.md#mermaid)).
 - Document animations and transitions.
diff --git a/doc/development/contributing/first_contribution.md b/doc/development/contributing/first_contribution.md
index 0c4b5b21171..3cf7f222fe4 100644
--- a/doc/development/contributing/first_contribution.md
+++ b/doc/development/contributing/first_contribution.md
@@ -190,19 +190,19 @@ You can check that you were successful:
 ### Update the translation files
 
 English UI strings are localized into many languages.
-These strings are saved in a `.pot` file, which you must update
+These strings are saved in a `.pot` file, which must be regenerated
 any time you update UI text.
 
-To generate the localization file:
+To automatically regenerate the localization file:
 
 1. Ensure you are in the `gitlab-development-kit/gitlab` directory.
 1. Run the following command:
 
    ```shell
-   bin/rake gettext:compile
+   tooling/bin/gettext_extractor locale/gitlab.pot
    ```
 
-   After several minutes, a `.pot` file is generated in the `/locale` directory.
+   The `.pot` file will be generated in the `/locale` directory.
 
 Now, in the `gitlab-development-kit/gitlab` directory, if you type `git status`
 you should have both files listed:
diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md
index 82a08246503..eb26ddbd9e9 100644
--- a/doc/development/contributing/index.md
+++ b/doc/development/contributing/index.md
@@ -36,13 +36,8 @@ open a [new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issue%5Bmil
 Select the appropriate template, and add all the necessary information about the work you are planning on doing.
 That way you can get more guidance and support from GitLab team members.
 
-If you're not sure what to work on, you can:
-
-- View issues with the
-  [`~Seeking community contributions` label](../labels/index.md#label-for-community-contributors).
-- Optimize tests. Use [RSpec profiling statistics](https://gitlab-org.gitlab.io/rspec_profiling_stats/)
-  to identify the slowest tests. These tests are good candidates for improving and checking if any
-  [best practices](../testing_guide/best_practices.md) can speed them up.
+If you're not sure what to work on, you can [view issues](https://gitlab.com/groups/gitlab-org/-/issues/?sort=updated_desc&state=opened&label_name%5B%5D=quick%20win&label_name%5B%5D=Seeking%20community%20contributions&first_page_size=100) with the
+  `~Seeking community contributions` and `~quick win` label.
 
 When you find an issue, leave a comment on the issue you want to work on.
 This helps the GitLab team and members of the wider GitLab community know that you will be working on that issue.
@@ -65,8 +60,9 @@ To write and test your code, you will use the GitLab Development Kit.
      consider doing so with an empty rails app and port it to GDK after.
 
    - To run a pre-configured GDK instance in the cloud, use [GDK with Gitpod](../../integration/gitpod.md).
-     From a project's repository, select the caret (angle-down) next to **Web IDE**,
-     and select **Gitpod** from the list.
+     From a project repository:
+       1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project.
+       1. In the upper right, select **Edit > Gitpod**.
 1. If you want to contribute to the [website](https://about.gitlab.com/) or the [handbook](https://about.gitlab.com/handbook/),
    go to the footer of any page and select **Edit in Web IDE** to open the [Web IDE](../../user/project/web_ide/index.md).
 
@@ -89,7 +85,7 @@ For details, see the [merge request workflow](merge_request_workflow.md).
 1. When you create a merge request, the [`@gitlab-bot`](https://gitlab.com/gitlab-bot) automatically applies
    the ["~Community contribution"](https://about.gitlab.com/handbook/engineering/quality/triage-operations/#ensure-quick-feedback-for-community-contributions) label.
 1. In the 24-48 hours after you create the merge request, a
-   [Merge Request Coach](https://about.gitlab.com/handbook/marketing/community-relations/contributor-success/merge-request-coach-lifecycle.html)
+   [Merge Request Coach](https://about.gitlab.com/handbook/marketing/developer-relations/contributor-success/merge-request-coach-lifecycle.html)
    will review your merge request and apply stage, group, and type labels.
 1. If a merge request was not automatically assigned, ask for a review by typing `@gitlab-bot ready` in a comment.
    If your code has not been assigned a reviewer within two working days of its initial submission, you can ask
@@ -144,15 +140,10 @@ Lastly, keep the following in mind when submitting merge requests:
 - For the criteria for closing issues, see [the Issue Triage handbook page](https://about.gitlab.com/handbook/engineering/quality/issue-triage/#outdated-issues).
 - For the criteria for closing merge requests, see [the Merge Request Workflow](merge_request_workflow.md).
 
-## Getting an Enterprise Edition license
-
-GitLab has two development platforms:
-
-- GitLab Community Edition (CE), our free and open source edition.
-- GitLab Enterprise Edition (EE), which is our commercial edition.
+## Contributing to Premium/Ultimate features with an Enterprise Edition license
 
-If you need a license for contributing to an EE-feature, see
-[relevant information](https://about.gitlab.com/handbook/marketing/community-relations/contributor-success/community-contributors-workflows.html#contributing-to-the-gitlab-enterprise-edition-ee).
+If you would like to work on GitLab features that are within a paid tier, also known as the code that lives in the [EE folder](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee), it requires a GitLab Enterprise Edition license.
+Please request an Enterprise Edition Developers License according to the [documented process](https://about.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows.html#contributing-to-the-gitlab-enterprise-edition-ee).
 
 ## Get help
 
diff --git a/doc/development/database/adding_database_indexes.md b/doc/development/database/adding_database_indexes.md
index 7b29b1b14de..23a12413975 100644
--- a/doc/development/database/adding_database_indexes.md
+++ b/doc/development/database/adding_database_indexes.md
@@ -429,7 +429,7 @@ Use the asynchronous index helpers on your local environment to test changes for
 For very large tables, index destruction can be a challenge to manage.
 While `remove_concurrent_index` removes indexes in a way that does not block
 ordinary traffic, it can still be problematic if index destruction runs for
-during `autovacuum`. Necessary database operations like `autovacuum` cannot run, and
+many hours. Necessary database operations like `autovacuum` cannot run, and
 the deployment process on GitLab.com is blocked while waiting for index
 destruction to finish.
 
diff --git a/doc/development/database/batched_background_migrations.md b/doc/development/database/batched_background_migrations.md
index 6a6b43e52a0..3e54a78757a 100644
--- a/doc/development/database/batched_background_migrations.md
+++ b/doc/development/database/batched_background_migrations.md
@@ -42,7 +42,159 @@ Background migrations can help when:
 - You should use the [generator](#generator) to create batched background migrations,
   so that required files are created by default.
 
-## Isolation
+## How it works
+
+Batched background migrations (BBM) are subclasses of
+`Gitlab::BackgroundMigration::BatchedMigrationJob` that define a `perform` method.
+As the first step, a regular migration creates a `batched_background_migrations`
+record with the BBM class and the required arguments. By default,
+`batched_background_migrations` is in an active state, and those are picked up
+by the Sidekiq worker to execute the actual batched migration.
+
+All migration classes must be defined in the namespace `Gitlab::BackgroundMigration`. Place the files
+in the directory `lib/gitlab/background_migration/`.
+
+### Execution mechanism
+
+Batched background migrations are picked from the queue in the order they are enqueued. Multiple migrations are fetched
+and executed in parallel, as long they are in active state and do not target the same database table.
+The default number of migrations processed in parallel is 2, for GitLab.com this limit is configured to 4.
+Once migration is picked for execution, a job is created for the specific batch. After each job execution, migration's
+batch size may be increased or decreased, based on the performance of the last 20 jobs.
+
+```plantuml
+@startuml
+hide empty description
+skinparam ConditionEndStyle hline
+left to right direction
+rectangle "Batched Background Migration Queue" as migrations {
+  rectangle "Migration N (active)" as migrationn
+  rectangle "Migration 1 (completed)" as migration1
+  rectangle "Migration 2 (active)" as migration2
+  rectangle "Migration 3 (on hold)" as migration3
+  rectangle "Migration 4 (active)" as migration4
+  migration1 -[hidden]> migration2
+  migration2 -[hidden]> migration3
+  migration3 -[hidden]> migration4
+  migration4 -[hidden]> migrationn
+}
+rectangle "Execution Workers" as workers {
+ rectangle "Execution Worker 1 (busy)" as worker1
+ rectangle "Execution Worker 2 (available)" as worker2
+ worker1 -[hidden]> worker2
+}
+migration2 --> [Scheduling Worker]
+migration4 --> [Scheduling Worker]
+[Scheduling Worker] --> worker2
+@enduml
+```
+
+Soon as a worker is available, the BBM is processed by the runner.
+
+```plantuml
+@startuml
+hide empty description
+start
+rectangle Runner {
+  :Migration;
+  if (Have reached batching bounds?) then (Yes)
+    if (Have jobs to retry?) then (Yes)
+      :Fetch the batched job;
+    else (No)
+      :Finish active migration;
+      stop
+    endif
+  else (No)
+    :Create a batched job;
+  endif
+  :Execute batched job;
+  :Evaluate DB health;
+  note right: Checks for table autovacuum, Patroni Apdex, Write-ahead logging
+  if (Evaluation signs to stop?) then (Yes)
+    :Put migration on hold;
+  else (No)
+    :Optimize migration;
+  endif
+}
+@enduml
+```
+
+### Idempotence
+
+Batched background migrations are executed in a context of a Sidekiq process.
+The usual Sidekiq rules apply, especially the rule that jobs should be small
+and idempotent. Make sure that in case that your migration job is retried, data
+integrity is guaranteed.
+
+See [Sidekiq best practices guidelines](https://github.com/mperham/sidekiq/wiki/Best-Practices)
+for more details.
+
+### Migration optimization
+
+After each job execution, a verification takes place to check if the migration can be optimized.
+The optimization underlying mechanic is based on the concept of time efficiency. It calculates
+the exponential moving average of time efficiencies for the last N jobs and updates the batch
+size of the batched background migration to its optimal value.
+
+### Job retry mechanism
+
+The batched background migrations retry mechanism ensures that a job is executed again in case of failure.
+The following diagram shows the different stages of our retry mechanism:
+
+```plantuml
+@startuml
+hide empty description
+note as N1
+  can_split?:
+  the failure is due to a query timeout
+end note
+    [*] --> Running
+Running --> Failed
+note on link
+  if number of retries <= MAX_ATTEMPTS
+end note
+Running --> Succeeded
+Failed --> Running
+note on link
+  if number of retries > MAX_ATTEMPTS
+  and can_split? == true
+  then two jobs with smaller
+  batch size will be created
+end note
+Failed --> [*]
+Succeeded --> [*]
+@enduml
+```
+
+- `MAX_ATTEMPTS` is defined in the [`Gitlab::Database::BackgroundMigration`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/database/background_migration/batched_job.rb)
+  class.
+- `can_split?` is defined in the [`Gitlab::Database::BatchedJob`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/database/background_migration/batched_job.rb) class.
+
+### Failed batched background migrations
+
+The whole batched background migration is marked as `failed`
+(`/chatops run batched_background_migrations status MIGRATION_ID` shows
+the migration as `failed`) if any of the following is true:
+
+- There are no more jobs to consume, and there are failed jobs.
+- More than [half of the jobs failed since the background migration was started](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/database/background_migration/batched_migration.rb#L160).
+
+### Throttling batched migrations
+
+Because batched migrations are update heavy and there were few incidents in the past because of the heavy load from migrations while the database was underperforming, a throttling mechanism exists to mitigate them.
+
+These database indicators are checked to throttle a migration. On getting a
+stop signal, the migration is paused for a set time (10 minutes):
+
+- WAL queue pending archival crossing a threshold.
+- Active autovacuum on the tables on which the migration works on.
+- Patroni apdex SLI dropping below the SLO.
+
+It's an ongoing effort to add more indicators to further enhance the
+database health check framework. For more details, see
+[epic 7594](https://gitlab.com/groups/gitlab-org/-/epics/7594).
+
+### Isolation
 
 Batched background migrations must be isolated and cannot use application code (for example,
 models defined in `app/models` except the `ApplicationRecord` classes).
@@ -96,16 +248,6 @@ ApplicationRecord.connection.execute("SELECT * FROM projects")
 ActiveRecord::Base.connection.execute("SELECT * FROM projects")
 ```
 
-## Idempotence
-
-Batched background migrations are executed in a context of a Sidekiq process.
-The usual Sidekiq rules apply, especially the rule that jobs should be small
-and idempotent. Make sure that in case that your migration job is retried, data
-integrity is guaranteed.
-
-See [Sidekiq best practices guidelines](https://github.com/mperham/sidekiq/wiki/Best-Practices)
-for more details.
-
 ## Batched background migrations for EE-only features
 
 All the background migration classes for EE-only features should be present in GitLab FOSS.
@@ -118,12 +260,6 @@ Background migration classes for EE-only features that use job arguments should
 in the GitLab FOSS class. This is required to prevent job arguments validation from failing when
 migration is scheduled in GitLab FOSS context.
 
-Batched Background migrations are simple classes that define a `perform` method. A
-Sidekiq worker then executes such a class, passing any arguments to it. All
-migration classes must be defined in the namespace
-`Gitlab::BackgroundMigration`. Place the files in the directory
-`lib/gitlab/background_migration/`.
-
 ## Queueing
 
 Queueing a batched background migration should be done in a post-deployment
@@ -148,49 +284,6 @@ Make sure the newly-created data is either migrated, or
 saved in both the old and new version upon creation. Removals in
 turn can be handled by defining foreign keys with cascading deletes.
 
-### Job retry mechanism
-
-The batched background migrations retry mechanism ensures that a job is executed again in case of failure.
-The following diagram shows the different stages of our retry mechanism:
-
-```plantuml
-@startuml
-hide empty description
-note as N1
-  can_split?:
-  the failure is due to a query timeout
-end note
-[*] --> Running
-Running --> Failed
-note on link
-  if number of retries <= MAX_ATTEMPTS
-end note
-Running --> Succeeded
-Failed --> Running
-note on link
-  if number of retries > MAX_ATTEMPTS
-  and can_split? == true
-  then two jobs with smaller
-  batch size will be created
-end note
-Failed --> [*]
-Succeeded --> [*]
-@enduml
-```
-
-- `MAX_ATTEMPTS` is defined in the [`Gitlab::Database::BackgroundMigration`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/database/background_migration/batched_job.rb)
-class.
-- `can_split?` is defined in the [`Gitlab::Database::BatchedJob`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/database/background_migration/batched_job.rb) class.
-
-### Failed batched background migrations
-
-The whole batched background migration is marked as `failed`
-(`/chatops run batched_background_migrations status MIGRATION_ID` will show
-the migration as `failed`) if any of the following are true:
-
-- There are no more jobs to consume, and there are failed jobs.
-- More than [half of the jobs failed since the background migration was started](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/database/background_migration/batched_migration.rb).
-
 ### Requeuing batched background migrations
 
 If one of the batched background migrations contains a bug that is fixed in a patch
@@ -831,6 +924,7 @@ Let's assume that a batched background migration failed on a particular batch on
 Fortunately you can leverage our [database migration pipeline](database_migration_pipeline.md) to rerun a particular batch with additional logging and/or fix to see if it solves the problem.
 
 <!-- vale gitlab.Substitutions = NO -->
+
 For an example see [Draft: Test PG::CardinalityViolation fix](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110910) but make sure to read the entire section.
 
 To do that, you need to:
@@ -872,7 +966,7 @@ end
 
 #### 3. Apply a workaround for our migration helpers (optional)
 
-If your batched background migration touches tables from a schema other than the one you specified by using `restrict_gitlab_migration` helper (example: the scheduling migration has `restrict_gitlab_migration gitlab_schema: :gitlab_main` but the background job uses tables from the `:gitlab_ci` schema) then the migration will fail. To prevent that from happening you'll have to monkey patch database helpers so they don't fail the testing pipeline job:
+If your batched background migration touches tables from a schema other than the one you specified by using `restrict_gitlab_migration` helper (example: the scheduling migration has `restrict_gitlab_migration gitlab_schema: :gitlab_main` but the background job uses tables from the `:gitlab_ci` schema) then the migration will fail. To prevent that from happening you must to monkey patch database helpers so they don't fail the testing pipeline job:
 
 1. Add the schema names to [`RestrictGitlabSchema`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb#L57)
 
diff --git a/doc/development/database/clickhouse/index.md b/doc/development/database/clickhouse/index.md
index 032e4f5f6ee..8ca6240e0f1 100644
--- a/doc/development/database/clickhouse/index.md
+++ b/doc/development/database/clickhouse/index.md
@@ -117,7 +117,7 @@ Files: `config.xml`
 | Topic | Security Requirement | Reason |
 | ----- | -------------------- | ------ |
 | Permissions | ClickHouse runs by default with the `clickhouse` user. Running as `root` is never needed. Use the principle of least privileges for the folders: `/etc/clickhouse-server`, `/var/lib/clickhouse`, `/var/log/clickhouse-server`. These folders must belong to the `clickhouse` user and group, and no other system user must have access to them. | Default passwords, ports and rules are "open doors". ([Fail securely & use secure defaults](https://about.gitlab.com/handbook/security/architecture/#fail-securely--use-secure-defaults) principle) |
-| Encryption | Use an encrypted storage for logs and data if RED data is processed. On Kubernetes, the [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) used must be encrypted. | Encrypt data at rest. ([Defense in depth](https://about.gitlab.com/handbook/security/architecture/#implement-defense-in-depth)) |
+| Encryption | Use an encrypted storage for logs and data if RED data is processed. On Kubernetes, the [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) used must be encrypted. [GKE](https://cloud.google.com/blog/products/containers-kubernetes/exploring-container-security-use-your-own-keys-to-protect-your-data-on-gke) and [EKS](https://aws.github.io/aws-eks-best-practices/security/docs/data/) encrypt all data at rest already. In this case, using your own key is best but not required. | Encrypt data at rest. ([Defense in depth](https://about.gitlab.com/handbook/security/architecture/#implement-defense-in-depth)) |
 
 ### Logging
 
diff --git a/doc/development/database/database_dictionary.md b/doc/development/database/database_dictionary.md
index 84b76ddc34c..70691d8746c 100644
--- a/doc/development/database/database_dictionary.md
+++ b/doc/development/database/database_dictionary.md
@@ -18,7 +18,7 @@ For the `geo` database, the dictionary files are stored under `ee/db/geo/docs/`.
 ## Example dictionary file
 
 ```yaml
-----
+---
 table_name: terraform_states
 classes:
 - Terraform::State
diff --git a/doc/development/database/foreign_keys.md b/doc/development/database/foreign_keys.md
index 25b3d815d7a..5dda3dd55a3 100644
--- a/doc/development/database/foreign_keys.md
+++ b/doc/development/database/foreign_keys.md
@@ -195,5 +195,5 @@ end
 ```
 
 Using a foreign key as primary key saves space but can make
-[batch counting](../service_ping/implement.md#batch-counters) in [Service Ping](../service_ping/index.md) less efficient.
+[batch counting](../internal_analytics/service_ping/implement.md#batch-counters) in [Service Ping](../service_ping/index.md) less efficient.
 Consider using a regular `id` column if the table is relevant for Service Ping.
diff --git a/doc/development/database/query_performance.md b/doc/development/database/query_performance.md
index 10ab726940a..77067e2979d 100644
--- a/doc/development/database/query_performance.md
+++ b/doc/development/database/query_performance.md
@@ -22,7 +22,7 @@ When you are optimizing your SQL queries, there are two dimensions to pay attent
 | Concurrent operations in a migration      | `5min`             | Concurrent operations do not block the database, but they block the GitLab update. This includes operations such as `add_concurrent_index` and `add_concurrent_foreign_key`.                                                                                                                                               |
 | Concurrent operations in a post migration | `20min`            | Concurrent operations do not block the database, but they block the GitLab post update process. This includes operations such as `add_concurrent_index` and `add_concurrent_foreign_key`. If index creation exceeds 20 minutes, consider [async index creation](adding_database_indexes.md#create-indexes-asynchronously). |
 | Background migrations                     | `1s`               |                                                                                                                                                                                                                                                                                                                            |
-| Service Ping                              | `1s`               | See the [Service Ping docs](../service_ping/implement.md) for more details.                                                                                                                                                                                                                                                |
+| Service Ping                              | `1s`               | See the [Service Ping docs](../internal_analytics/service_ping/implement.md) for more details.                                                                                                                                                                                                                                                |
 
 - When analyzing your query's performance, pay attention to if the time you are seeing is on a [cold or warm cache](#cold-and-warm-cache). These guidelines apply for both cache types.
 - When working with batched queries, change the range and batch size to see how it effects the query timing and caching.
diff --git a/doc/development/database_review.md b/doc/development/database_review.md
index 0e34e550098..76e9add215b 100644
--- a/doc/development/database_review.md
+++ b/doc/development/database_review.md
@@ -27,7 +27,7 @@ A database review is required for:
   database review.
 - Changes in Service Data metrics that use `count`, `distinct_count`, `estimate_batch_distinct_count` and `sum`.
   These metrics could have complex queries over large tables.
-  See the [Product Intelligence Guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)
+  See the [Analytics Instrumentation Guide](https://about.gitlab.com/handbook/product/analytics-instrumentation-guide/)
   for implementation details.
 
 A database reviewer is expected to look out for overly complex
diff --git a/doc/development/deprecation_guidelines/index.md b/doc/development/deprecation_guidelines/index.md
index bc14b0f127c..e81bed07da9 100644
--- a/doc/development/deprecation_guidelines/index.md
+++ b/doc/development/deprecation_guidelines/index.md
@@ -9,54 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 This page includes information about how and when to remove or make breaking changes
 to GitLab features.
 
-## Terminology
-
-<!--
-If updating these definitions, be sure to update them in the handbook as well:
-https://about.gitlab.com/handbook/product/gitlab-the-product/#definitions
--->
-
-**Deprecation**:
-
-- Required before ending support for a feature or removing a feature.
-- Feature not recommended for use.
-- Development restricted to Priority 1 / Severity 1 bug fixes.
-- Will be removed in a future major release.
-- Begins after a deprecation announcement outlining an end-of-support or removal date.
-- Ends after the end-of-support date or removal date has passed.
-
-**End of Support**:
-
-- Optional step before removal.
-- Feature usage strongly discouraged.
-- No support or fixes provided.
-- No longer tested internally.
-- Will be removed in a future major release.
-- Begins after an end-of-support date has passed.
-
-[Announcing an End of Support period](https://about.gitlab.com/handbook/marketing/blog/release-posts/#announcing-an-end-of-support-period)
-should only be used in special circumstances and is not recommended for general use.
-Most features should be deprecated and then removed.
-
-**Removal**:
-
-- Feature usage impossible.
-- Feature no longer supported (if End of Support period hasn't already been announced).
-- Happens in a major release in line with our
-  [semantic versioning policy](../../policy/maintenance.md).
-- Begins after removal date has passed.
-
-**Breaking change**:
-
-A "breaking change" is any change that requires users to make a corresponding change to their code, settings, or workflow. "Users" might be humans, API clients, or even code classes that "use" another class. Examples of breaking changes include:
-
-- Removing a user-facing feature without a replacement/workaround.
-- Changing the definition of an existing API (by doing things like re-naming query parameters or changing routes).
-- Removing a public method from a code class.
-
-A breaking change can be considered major if it affects many users, or represents a significant change in behavior.
-
-![Deprecation, End of Support, Removal process](img/deprecation_removal_process.png)
+For details about the terms used on this page, see [the terminology](../../update/terminology.md).
 
 ## When can a feature be deprecated?
 
@@ -68,6 +21,8 @@ Do not include the deprecation announcement in the merge request that introduces
 Use a separate MR to create a deprecation entry. For steps to create a deprecation entry, see
 [Deprecations](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations).
 
+![Deprecation, End of Support, Removal process](img/deprecation_removal_process.png)
+
 ## How are Community Contributions to a deprecated feature handled?
 
 Development on deprecated features is restricted to Priority 1 / Severity 1 bug fixes. Any community contributions to deprecated features are unlikely to be prioritized during milestone planning.
diff --git a/doc/development/documentation/alpha_beta.md b/doc/development/documentation/alpha_beta.md
index d421aae3cb9..a6f756f93ae 100644
--- a/doc/development/documentation/alpha_beta.md
+++ b/doc/development/documentation/alpha_beta.md
@@ -7,7 +7,7 @@ group: unassigned
 # Documenting Experiment and Beta features
 
 Some features are not generally available and are instead considered
-[Experiment or Beta](../../policy/alpha-beta-support.md).
+[Experiment or Beta](../../policy/experiment-beta-support.md).
 
 When you document a feature in one of these three statuses:
 
@@ -25,7 +25,7 @@ For example:
 ```markdown
 ## Great new feature (Experiment)
 
-> [Introduced](link) in GitLab 15.10. This feature is an [Experiment](<link_to>/policy/alpha-beta-support.md).
+> [Introduced](link) in GitLab 15.10. This feature is an [Experiment](<link_to>/policy/experiment-beta-support.md).
 
 FLAG:
 On self-managed GitLab, by default this feature is not available.
@@ -34,7 +34,7 @@ On GitLab.com, this feature is not available. This feature is not ready for prod
 
 Use this great new feature when you need to do this new thing.
 
-This feature is an [Experiment](<link_to>/policy/alpha-beta-support.md). To join
+This feature is an [Experiment](<link_to>/policy/experiment-beta-support.md). To join
 the list of users testing this feature, do this thing. If you find a bug,
 [open an issue](link).
 ```
diff --git a/doc/development/documentation/contribute.md b/doc/development/documentation/contribute.md
index 8b08743c6e9..d3b7d9a4d93 100644
--- a/doc/development/documentation/contribute.md
+++ b/doc/development/documentation/contribute.md
@@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # Contribute to the GitLab documentation
 
-Everyone is welcome to update the GitLab documentation!
+Everyone is welcome to update the GitLab documentation.
 
 ## Work without an issue
 
@@ -50,9 +50,7 @@ When you are ready to update the documentation:
 1. In your fork, find the documentation page in the `\doc` directory.
 1. If you know Git, make your changes and open a merge request.
    If not, follow these steps:
-   1. In the upper-right corner, select **Edit** if it is visible.
-      If it is not, select the down arrow (**{chevron-lg-down}**) next to
-      **Open in Web IDE** or **Gitpod**, and select **Edit**.
+   1. In the upper right, select **Edit > Edit single file**.
    1. In the **Commit message** text box, enter a commit message.
       Use 3-5 words, start with a capital letter, and do not end with a period.
    1. Select **Commit changes**.
diff --git a/doc/development/documentation/feature_flags.md b/doc/development/documentation/feature_flags.md
index 854faa839ef..5cee300481b 100644
--- a/doc/development/documentation/feature_flags.md
+++ b/doc/development/documentation/feature_flags.md
@@ -17,7 +17,7 @@ When the state of a feature flag changes, the developer who made the change
 
 Every feature introduced to the codebase, even if it's behind a disabled feature flag,
 must be documented. For more information, see
-[the discussion that led to this decision](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/47917#note_459984428). [Experiment or Beta](../../policy/alpha-beta-support.md) features are usually behind a feature flag, and must also be documented. For more information, see [Document Experiment or Beta features](alpha_beta.md).
+[the discussion that led to this decision](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/47917#note_459984428). [Experiment or Beta](../../policy/experiment-beta-support.md) features are usually behind a feature flag, and must also be documented. For more information, see [Document Experiment or Beta features](alpha_beta.md).
 
 When the feature is [implemented in multiple merge requests](../feature_flags/index.md#feature-flags-in-gitlab-development),
 discuss the plan with your technical writer.
diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md
index 6811b3d6584..761dde839c1 100644
--- a/doc/development/documentation/index.md
+++ b/doc/development/documentation/index.md
@@ -259,8 +259,8 @@ is inside `_()` so it can be translated:
   link:
 
   ```haml
-  - link_start = '<a href="%{url}" target="_blank" rel="noopener noreferrer">'.html_safe % { url: help_page_path('user/permissions') }
-  %p= _("This is a text describing the option/feature in a sentence. %{link_start}Learn more.%{link_end}").html_safe % { link_start: link_start, link_end: '</a>'.html_safe }
+  - link = link_to('', help_page_path('user/permissions'), target: '_blank', rel: 'noopener noreferrer')
+  %p= safe_format(_("This is a text describing the option/feature in a sentence. %{link_start}Learn more.%{link_end}"), tag_pair(link, :link_start, :link_end))
   ```
 
 - Using a button link. Useful in places where text would be out of context with
@@ -290,7 +290,7 @@ be translated:
 
 ```ruby
 docs_link = link_to _('Learn more.'), help_page_url('user/permissions', anchor: 'anchor-link'), target: '_blank', rel: 'noopener noreferrer'
-_('This is a text describing the option/feature in a sentence. %{docs_link}').html_safe % { docs_link: docs_link.html_safe }
+safe_format(_('This is a text describing the option/feature in a sentence. %{docs_link}'), docs_link: docs_link)
 ```
 
 In cases where you need to generate a link from outside of views/helpers, where the `link_to` and `help_page_url` methods are not available, use the following code block
@@ -298,7 +298,7 @@ as a guide where the methods are fully qualified:
 
 ```ruby
 docs_link = ActionController::Base.helpers.link_to _('Learn more.'), Rails.application.routes.url_helpers.help_page_url('user/permissions', anchor: 'anchor-link'), target: '_blank', rel: 'noopener noreferrer'
-_('This is a text describing the option/feature in a sentence. %{docs_link}').html_safe % { docs_link: docs_link.html_safe }
+safe_format(_('This is a text describing the option/feature in a sentence. %{docs_link}'), docs_link: docs_link)
 ```
 
 Do not use `include ActionView::Helpers::UrlHelper` just to make the `link_to` method available as you might see in some existing code. Read more in
diff --git a/doc/development/documentation/restful_api_styleguide.md b/doc/development/documentation/restful_api_styleguide.md
index 93afbf7e6dd..c39c7c02108 100644
--- a/doc/development/documentation/restful_api_styleguide.md
+++ b/doc/development/documentation/restful_api_styleguide.md
@@ -80,7 +80,8 @@ response attributes:
 Example request:
 
 ```shell
-curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/endpoint?parameters"
+curl --header "PRIVATE-TOKEN: <your_access_token>" \
+  --url "https://gitlab.example.com/api/v4/endpoint?parameters"
 ```
 
 Example response:
@@ -201,9 +202,13 @@ For information about writing attribute descriptions, see the [GraphQL API descr
 - Wherever needed use this personal access token: `<your_access_token>`.
 - Always put the request first. `GET` is the default so you don't have to
   include it.
-- Wrap the URL in double quotes (`"`).
+- Use long option names (`--header` instead of `-H`) for legibility. (Tested in
+  [`scripts/lint-doc.sh`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/lint-doc.sh).)
+- Declare URLs with the `--url` parameter, and wrap the URL in double quotes (`"`).
 - Prefer to use examples using the personal access token and don't pass data of
   username and password.
+- For legibility, use the <code>&#92;</code> character and indentation to break long single-line
+  commands apart into multiple lines.
 
 | Methods                                         | Description                                            |
 |:------------------------------------------------|:-------------------------------------------------------|
@@ -227,7 +232,8 @@ relevant style guide sections on [Fake user information](styleguide/index.md#fak
 Get the details of a group:
 
 ```shell
-curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/gitlab-org"
+curl --header "PRIVATE-TOKEN: <your_access_token>" \
+  --url "https://gitlab.example.com/api/v4/groups/gitlab-org"
 ```
 
 ### cURL example with parameters passed in the URL
@@ -235,7 +241,8 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a
 Create a new project under the authenticated user's namespace:
 
 ```shell
-curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects?name=foo"
+curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
+  --url "https://gitlab.example.com/api/v4/projects?name=foo"
 ```
 
 ### Post data using cURL's `--data`
@@ -245,7 +252,9 @@ can use cURL's `--data` option. The example below will create a new project
 `foo` under the authenticated user's namespace.
 
 ```shell
-curl --data "name=foo" --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects"
+curl --data "name=foo" \
+  --header "PRIVATE-TOKEN: <your_access_token>" \
+  --url "https://gitlab.example.com/api/v4/projects"
 ```
 
 ### Post data using JSON content
@@ -254,20 +263,23 @@ This example creates a new group. Be aware of the use of single (`'`) and double
 (`"`) quotes.
 
 ```shell
-curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --header "Content-Type: application/json" \
-     --data '{"path": "my-group", "name": "My group"}' "https://gitlab.example.com/api/v4/groups"
+curl --request POST \
+  --header "PRIVATE-TOKEN: <your_access_token>" \
+  --header "Content-Type: application/json" \
+  --data '{"path": "my-group", "name": "My group"}' \
+  --url "https://gitlab.example.com/api/v4/groups"
 ```
 
 For readability, you can also set up the `--data` by using the following format:
 
 ```shell
 curl --request POST \
---url "https://gitlab.example.com/api/v4/groups" \
---header "content-type: application/json" \
---header "PRIVATE-TOKEN: <your_access_token>" \
---data '{
-  "path": "my-group",
-  "name": "My group"
+  --url "https://gitlab.example.com/api/v4/groups" \
+  --header "content-type: application/json" \
+  --header "PRIVATE-TOKEN: <your_access_token>" \
+  --data '{
+    "path": "my-group",
+    "name": "My group"
 }'
 ```
 
@@ -277,8 +289,11 @@ Instead of using JSON or URL-encoding data, you can use `multipart/form-data` wh
 properly handles data encoding:
 
 ```shell
-curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --form "title=ssh-key" \
-     --form "key=ssh-rsa AAAAB3NzaC1yc2EA..." "https://gitlab.example.com/api/v4/users/25/keys"
+curl --request POST \
+  --header "PRIVATE-TOKEN: <your_access_token>" \
+  --form "title=ssh-key" \
+  --form "key=ssh-rsa AAAAB3NzaC1yc2EA..." \
+  --url "https://gitlab.example.com/api/v4/users/25/keys"
 ```
 
 The above example is run by and administrator and will add an SSH public key
@@ -292,7 +307,9 @@ contains spaces in its title. Observe how spaces are escaped using the `%20`
 ASCII code.
 
 ```shell
-curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20GitLab"
+curl --request POST \
+  --header "PRIVATE-TOKEN: <your_access_token>" \
+  --url "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20GitLab"
 ```
 
 Use `%2F` for slashes (`/`).
@@ -304,6 +321,9 @@ exclude specific users when requesting a list of users for a project, you would
 do something like this:
 
 ```shell
-curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" --data "skip_users[]=<user_id>" \
-     --data "skip_users[]=<user_id>" "https://gitlab.example.com/api/v4/projects/<project_id>/users"
+curl --request PUT \
+  --header "PRIVATE-TOKEN: <your_access_token>"
+  --data "skip_users[]=<user_id>" \
+  --data "skip_users[]=<user_id>" \
+  --url "https://gitlab.example.com/api/v4/projects/<project_id>/users"
 ```
diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md
index 00307583be4..6a56d824e35 100644
--- a/doc/development/documentation/styleguide/index.md
+++ b/doc/development/documentation/styleguide/index.md
@@ -522,6 +522,11 @@ When using code block style:
   [list of supported languages and lexers](https://github.com/rouge-ruby/rouge/wiki/List-of-supported-languages-and-lexers)
   for available syntax highlighters. Use `plaintext` if no better hint is available.
 
+#### cURL commands in code blocks
+
+See [cURL commands](../restful_api_styleguide.md#curl-commands) for information
+about styling cURL commands.
+
 ## Lists
 
 - Do not use a period if the phrase is not a full sentence.
@@ -640,15 +645,27 @@ To keep tables accessible and scannable, tables should not have any
 empty cells. If there is no otherwise meaningful value for a cell, consider entering
 **N/A** for 'not applicable' or **None**.
 
-To help tables be easier to maintain, consider adding additional spaces to the
-column widths to make them consistent. For example:
+To help keep tables easier to maintain, you can:
 
-```markdown
-| App name | Description          | Requirements   |
-|:---------|:---------------------|:---------------|
-| App 1    | Description text 1.  | Requirements 1 |
-| App 2    | Description text 2.  | None           |
-```
+- Add additional spaces to make the column widths consistent. For example:
+
+  ```markdown
+  | App name | Description         | Requirements   |
+  |----------|---------------------|----------------|
+  | App 1    | Description text 1. | Requirements 1 |
+  | App 2    | Description text 2. | None           |
+  ```
+
+- Skip the additional spaces in the rightmost column for tables that are very wide.
+  For example:
+
+  ```markdown
+  | Setting   | Default | Description |
+  |-----------|---------|-------------|
+  | Setting 1 | `1000`  | A short description. |
+  | Setting 2 | `2000`  | A long description that would make the table too wide and add too much whitespace if every cell in this column was aligned. |
+  | Setting 3 | `0`     | Another short description. |
+  ```
 
 Consider installing a plugin or extension in your editor for formatting tables:
 
@@ -656,6 +673,16 @@ Consider installing a plugin or extension in your editor for formatting tables:
 - [Markdown Table Formatter](https://packagecontrol.io/packages/Markdown%20Table%20Formatter) for Sublime Text
 - [Markdown Table Formatter](https://atom.io/packages/markdown-table-formatter) for Atom
 
+### Updates to existing tables
+
+When you add or edit rows in an existing table, the cells in the new rows might be wider.
+If you realign the columns to account for the width, the diff becomes difficult to read,
+because the entire table shows as modified.
+
+Markdown tables naturally fall out of alignment over time, but still render correctly
+on `docs.gitlab.com`. The technical writing team can realign cells the next time
+the page is refactored.
+
 ### Table headings
 
 Use sentence case for table headings. For example, `Keyword value` or `Project name`.
@@ -681,10 +708,10 @@ For the footnotes below the table, use a bold number followed by a sentence.
 For example:
 
 ```markdown
-| App name | Description                      |
-|:---------|:---------------------------------|
-| App A    | Description text. <sup>1</sup>   |
-| App B    | Description text. <sup>2</sup>   |
+| App name | Description                    |
+|:---------|:-------------------------------|
+| App A    | Description text. <sup>1</sup> |
+| App B    | Description text. <sup>2</sup> |
 
 1. This is the footnote.
 1. This is the other footnote.
@@ -692,10 +719,10 @@ For example:
 
 This text renders this output:
 
-| App name | Description                      |
-|:---------|:---------------------------------|
-| App A    | Description text. <sup>1</sup>   |
-| App B    | Description text. <sup>2</sup>   |
+| App name | Description                    |
+|:---------|:-------------------------------|
+| App A    | Description text. <sup>1</sup> |
+| App B    | Description text. <sup>2</sup> |
 
 1. This is the footnote.
 1. This is the other footnote.
@@ -834,7 +861,7 @@ Sometimes they are more precise and will be maintained more actively.
 
 For each external link you add, weigh the customer benefit with the maintenance difficulties.
 
-### Links requiring permissions
+### Links that require permissions
 
 Don't link directly to:
 
@@ -842,23 +869,27 @@ Don't link directly to:
 - Project features that require [special permissions](../../../user/permissions.md)
   to view.
 
-These fail for:
+These links fail for:
 
 - Those without sufficient permissions.
 - Automated link checkers.
 
-Instead:
+If you must use one of these links:
 
-- To reduce confusion, mention in the text that the information is either:
-  - Contained in a confidential issue.
-  - Requires special permission to a project to view.
-- Provide a link in back ticks (`` ` ``) so that those with access to the issue
-  can navigate to it.
+- If the link is to a confidential issue, mention that the issue is visible only to GitLab team members, as in the first example.
+- If the link requires a specific role or permissions, mention that information, as in the second example.
+- Put the link in backticks, so that it does not cause link checkers to fail.
 
-Example:
+Examples:
 
 ```markdown
-For more information, see the [confidential issue](../../../user/project/issues/confidential_issues.md) `https://gitlab.com/gitlab-org/gitlab-foss/-/issues/<issue_number>`.
+GitLab team members can view more information in this confidential issue:
+`https://gitlab.com/gitlab-org/gitlab/-/issues/<issue_number>`
+```
+
+```markdown
+Users with the Maintainer role for the project can use the pipeline editor:
+`https://gitlab.com/gitlab-org/gitlab/-/ci/editor`
 ```
 
 ### Link to specific lines of code
@@ -894,10 +925,10 @@ When documenting how to navigate through the GitLab UI:
 Use these terms when referring to the main GitLab user interface
 elements:
 
-- **Top bar**: This is the top bar that spans the width of the user interface.
-  It includes the menu, the GitLab logo, search field, counters, and the user's avatar.
 - **Left sidebar**: This is the navigation sidebar on the left of the user
-  interface, specific to the project or group.
+  interface.
+  - Do not use the phrase `context switcher` or `switch contexts`. Instead, try to direct the user to the exact location with a set of repeatable steps.
+  - Do not use the phrase `the **Explore** menu` or `the **Your work** sidebar`. Instead, use `the left sidebar`.
 - **Right sidebar**: This is the navigation sidebar on the right of the user
   interface, specific to the open issue, merge request, or epic.
 
@@ -913,51 +944,57 @@ To be consistent, use these templates when you write navigation steps in a task
 To open project settings:
 
 ```markdown
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Settings > CI/CD**.
+1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project.
+1. Select **Settings > CI/CD**.
 1. Expand **General pipelines**.
 ```
 
 To open group settings:
 
 ```markdown
-1. On the top bar, select **Main menu > Groups** and find your group.
-1. On the left sidebar, select **Settings > CI/CD**.
+1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group.
+1. Select **Settings > CI/CD**.
 1. Expand **General pipelines**.
 ```
 
 To open either project or group settings:
 
 ```markdown
-1. On the top bar, select **Main menu**, and:
-   - For a project, select **Projects** and find your project.
-   - For a group, select **Groups** and find your group.
-1. On the left sidebar, select **Settings > CI/CD**.
+1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project or group.
+1. Select **Settings > CI/CD**.
 1. Expand **General pipelines**.
 ```
 
 To create a project:
 
 ```markdown
-1. On the top bar, select **Create new... > New project**.
+1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**.
 ```
 
 To create a group:
 
 ```markdown
-1. On the top bar, select **Create new... > New group**.
+1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New group**.
 ```
 
 To open the Admin Area:
 
 ```markdown
-1. On the top bar, select **Main menu > Admin**.
+1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+1. Select **Admin Area**.
+```
+
+To open the **Your work** menu item:
+
+```markdown
+1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+1. Select **Your work**.
 ```
 
 To select your avatar:
 
 ```markdown
-1. On the top bar, in the upper-right corner, select your avatar.
+1. On the left sidebar, select your avatar.
 ```
 
 To save the selection in some dropdown lists:
@@ -969,6 +1006,20 @@ To save the selection in some dropdown lists:
 1. Select any area outside the dropdown list.
 ```
 
+To view all your projects:
+
+```markdown
+1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+1. Select **View all your projects**.
+```
+
+To view all your groups:
+
+```markdown
+1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+1. Select **View all your groups**.
+```
+
 ### Optional steps
 
 If a step is optional, start the step with the word `Optional` followed by a period.
@@ -998,8 +1049,8 @@ Use the phrase **Complete the fields**.
 
 For example:
 
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Settings > Repository**.
+1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project.
+1. Select **Settings > Repository**.
 1. Expand **Push rules**.
 1. Complete the fields.
 
diff --git a/doc/development/documentation/styleguide/word_list.md b/doc/development/documentation/styleguide/word_list.md
index 094de2bd724..0bbd679abe5 100644
--- a/doc/development/documentation/styleguide/word_list.md
+++ b/doc/development/documentation/styleguide/word_list.md
@@ -128,6 +128,10 @@ The token generated when you create an agent for Kubernetes. Use **agent access
 - secret token
 - authentication token
 
+## AI, artificial intelligence
+
+Use **AI**. Do not spell out **artificial intelligence**.
+
 ## air gap, air-gapped
 
 Use **offline environment** to describe installations that have physical barriers or security policies that prevent or limit internet access. Do not use **air gap**, **air gapped**, or **air-gapped**. For example:
@@ -208,7 +212,7 @@ Try to avoid **below** when referring to an example or table in a documentation
 
 Use uppercase for **Beta**. For example: **The XYZ feature is in Beta.** or **This Beta release is ready to test.**
 
-You might also want to link to [this section](../../../policy/alpha-beta-support.md#beta)
+You might also want to link to [this section](../../../policy/experiment-beta-support.md#beta)
 in the handbook when writing about Beta features.
 
 ## blacklist
@@ -294,6 +298,10 @@ This version is different than the larger, more monolithic **Linux package** tha
 
 You can also use **cloud-native GitLab** for short. It should be hyphenated and lowercase.
 
+## Code Suggestions
+
+Use title case for **Code Suggestions**.
+
 ## collapse
 
 Use **collapse** instead of **close** when you are talking about expanding or collapsing a section in the UI.
@@ -304,6 +312,16 @@ Use **From the command line** to introduce commands.
 
 Hyphenate when using as an adjective. For example, **a command-line tool**.
 
+## compute
+
+Use **compute** for the resources used by runners to run CI/CD jobs.
+
+Related terms:
+
+- [**units of compute**](#units-of-compute): How compute usage is calculated. For example, `400 units of compute`.
+- [**compute quota**](../../../ci/pipelines/cicd_minutes.md): The limit of units of compute that a namespace can use each month.
+- **compute usage**: The number of units of compute that the namespace has used from the monthly quota.
+
 ## confirmation dialog
 
 Use **confirmation dialog** to describe the dialog box that asks you to confirm your action. For example:
@@ -484,7 +502,7 @@ Use **expand** instead of **open** when you are talking about expanding or colla
 
 Use uppercase for **Experiment**. For example: **The XYZ feature is an Experiment.** or **This Experiment is ready to test.**
 
-You might also want to link to [this section](../../../policy/alpha-beta-support.md#experiment)
+You might also want to link to [this section](../../../policy/experiment-beta-support.md#experiment)
 in the handbook when writing about Experiment features.
 
 ## FAQ
@@ -507,8 +525,8 @@ Instead of:
 However, you can make an exception when you are writing a task and you need to refer to all
 of the fields at once. For example:
 
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Settings > CI/CD**.
+1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project.
+1. Select **Settings > CI/CD**.
 1. Expand **General pipelines**.
 1. Complete the fields.
 
@@ -672,6 +690,12 @@ Do not use Latin abbreviations. Use **that is** instead. ([Vale](../testing.md#v
 
 Do not use **in order to**. Use **to** instead. ([Vale](../testing.md#vale) rule: [`Wordy.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/Wordy.yml))
 
+## indexes, indices
+
+For the plural of **index**, use **indexes**.
+
+However, for ElasticSearch, use [**indices**](https://www.elastic.co/blog/what-is-an-elasticsearch-index).
+
 ## Installation from source
 
 When referring to the installation method using the self-compiled code, refer to it
@@ -1225,7 +1249,7 @@ to the GitLab [reference architectures](../../../administration/reference_archit
 
 ## search
 
-When you search, you type a string in the search box on the top bar.
+When you search, you type a string in the search box on the left sidebar.
 The search results are displayed on a search page.
 
 Searching is different from [filtering](#filter).
@@ -1454,9 +1478,8 @@ Use **units of compute** instead of these (or similar) terms:
 - **CI minutes**
 - **pipeline minutes**
 - **CI pipeline minutes**
-- **pipeline minutes quota**
+- **pipeline minutes**
 
-This language is still being standardized in the documentation and UI beginning in March, 2023.
 For more information, see [issue 5218](https://gitlab.com/gitlab-com/Product/-/issues/5218).
 
 ## units of measurement
diff --git a/doc/development/documentation/testing.md b/doc/development/documentation/testing.md
index 4a7e206da20..ab0fe7b20a9 100644
--- a/doc/development/documentation/testing.md
+++ b/doc/development/documentation/testing.md
@@ -25,8 +25,11 @@ in the relevant projects:
 - <https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/.gitlab-ci.yml>
 - <https://gitlab.com/gitlab-org/cloud-native/gitlab-operator/-/blob/master/.gitlab-ci.yml>
 
-We also run some documentation tests in the GitLab Development Kit project:
-<https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/.gitlab/ci/test.gitlab-ci.yml>.
+We also run some documentation tests in the:
+
+- GitLab Development Kit project:
+  <https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/.gitlab/ci/test.gitlab-ci.yml>.
+- Gitaly project: <https://gitlab.com/gitlab-org/gitaly/-/blob/master/.gitlab-ci.yml>.
 
 ## Run tests locally
 
@@ -48,6 +51,24 @@ To run tests locally, it's important to:
 
 ### Lint checks
 
+Merge requests containing changes to Markdown (`.md`) files run a `docs-lint markdown`
+job. This job runs `markdownlint`, and a set of tests from
+[`/scripts/lint-doc.sh`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/lint-doc.sh)
+that look for page content problems that Vale and markdownlint cannot test for.
+The job fails if any of these tests fail:
+
+- Curl (`curl`) commands must use long-form options (`--header`) instead of short options, like `-h`.
+- Documentation pages must contain front matter indicating ownership of the page.
+- Non-standard Unicode space characters (NBSP, NNBSP, ZWSP) must not be used in documentation,
+  because they can cause irregularities in search indexing and grepping.
+- `CHANGELOG.md` must not contain duplicate versions.
+- No files in the `doc/` directory may be executable.
+- Use `index.md` instead of `README.md`.
+- Directories and file names must use underscores instead of dashes.
+- Directories and file names must be in lower case.
+
+#### Run lint checks locally
+
 Lint checks are performed by the [`lint-doc.sh`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/lint-doc.sh)
 script and can be executed with the help of a Rake task as follows:
 
diff --git a/doc/development/documentation/topic_types/index.md b/doc/development/documentation/topic_types/index.md
index 37ed876fc59..381f70914c6 100644
--- a/doc/development/documentation/topic_types/index.md
+++ b/doc/development/documentation/topic_types/index.md
@@ -23,7 +23,7 @@ The acronym refers to the first letter of each topic type.
 
 In addition to the four primary topic types, you can use the following:
 
-- Page types: [Tutorials](tutorial.md) and [Get started](#get-started)
+- Page type: [Tutorials](tutorial.md)
 - Topic type: [Related topics](#related-topics)
 - Page or topic type: [Glossaries](glossary.md)
 
@@ -36,6 +36,7 @@ You should avoid:
 - Topics that have one or two sentences only. In these cases:
   - Incorporate the information in another topic.
   - If the sentence links to another page, use a [Related topics](#related-topics) link instead.
+- Get started topics. To document a procedure for a single feature, use a [task](task.md). For a set of steps, use a [tutorial](tutorial.md).
 
 ## Topic title guidelines
 
@@ -63,27 +64,3 @@ full sentences, and so should not end in a period.
 - [CI/CD variables](link-to-topic)
 - [Environment variables](link-to-topic)
 ```
-
-## Get started
-
-A get started page is a set of steps to help a user get set up
-quickly to use a single GitLab feature or tool.
-It consists of more than one task.
-
-Get started pages should be in this format:
-
-```markdown
-# Title ("Get started with <feature>")
-
-Complete the following steps to ... .
-
-1. First step.
-1. Another step.
-1. Another step.
-
-If you need to add more than one task,
-consider using subsections for each distinct task.
-```
-
-In the left nav, use `Get started` as the text. On the page itself, spell out
-the full name. For example, `Get started with application security`.
diff --git a/doc/development/documentation/topic_types/task.md b/doc/development/documentation/topic_types/task.md
index 2ddfd841ec3..87ce4d770f5 100644
--- a/doc/development/documentation/topic_types/task.md
+++ b/doc/development/documentation/topic_types/task.md
@@ -43,13 +43,13 @@ Prerequisites:
 
 To create an issue:
 
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Issues > List**.
+1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project.
+1. Select **Plan > Issues**.
 1. In the upper-right corner, select **New issue**.
 1. Complete the fields. (If you have reference content that lists each field, link to it here.)
 1. Select **Create issue**.
 
-The issue is created. You can view it by going to **Issues > List**.
+The issue is created. You can view it by going to **Plan > Issues**.
 ```
 
 ## Task topic titles
diff --git a/doc/development/documentation/versions.md b/doc/development/documentation/versions.md
index 859e7265a2a..dadae134f4c 100644
--- a/doc/development/documentation/versions.md
+++ b/doc/development/documentation/versions.md
@@ -120,8 +120,8 @@ To deprecate a page or topic:
    You can add any additional context-specific details that might help users.
 
 1. Add the following HTML comments above and below the content.
-   For the `remove_date`, set a date three months after the release where it
-   was deprecated.
+   For `remove_date`, set a date three months after the release where it
+   will be removed.
 
    ```markdown
    <!--- start_remove The following content will be removed on remove_date: 'YYYY-MM-DD' -->
@@ -181,7 +181,7 @@ To remove a topic:
    including the version history items and the word `WARNING:`.
 1. Add `(removed)` after the title.
 1. Add the following HTML comments above and below the topic.
-   For the `remove_date`, set a date three months after the release where it was removed.
+   For `remove_date`, set a date three months after the release where it was removed.
 
    ```markdown
    <!--- start_remove The following content will be removed on remove_date: 'YYYY-MM-DD' -->
@@ -201,8 +201,8 @@ This content is removed from the documentation as part of the Technical Writing
 ## Which versions are removed
 
 GitLab supports the current major version and two previous major versions.
-For example, if 15.0 is the current major version, all major and minor releases of
-GitLab 15.0, 14.0, and 13.0 are supported.
+For example, if 16.0 is the current major version, all major and minor releases of
+GitLab 16.0, 15.0, and 14.0 are supported.
 
 [View the list of supported versions](https://about.gitlab.com/support/statement-of-support/#version-support).
 
diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md
index bc997c37e66..aeb9739ecb3 100644
--- a/doc/development/ee_features.md
+++ b/doc/development/ee_features.md
@@ -49,13 +49,16 @@ version of the product:
 1. Enable **Allow use of licensed EE features** to make licensed EE features available to projects
    only if the project namespace's plan includes the feature.
 
-   1. Visit **Admin > Settings > General**.
+   1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+   1. Select **Admin Area**.
+   1. On the left sidebar, select **Settings > General**.
    1. Expand **Account and limit**.
    1. Select the **Allow use of licensed EE features** checkbox.
    1. Select **Save changes**.
 
 1. Ensure the group you want to test the EE feature for is actually using an EE plan:
-   1. On the top bar, select **Main menu > Admin**.
+   1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+   1. Select **Admin Area**.
    1. On the left sidebar, select **Overview > Groups**.
    1. Identify the group you want to modify, and select **Edit**.
    1. Scroll to **Permissions and group features**. For **Plan**, select `Ultimate`.
@@ -548,6 +551,9 @@ EE-specific models should `extend EE::Model`.
 For example, if EE has a specific `Tanuki` model, you would
 place it in `ee/app/models/ee/tanuki.rb`.
 
+ActiveRecord `enums` should be entirely
+[defined in FOSS](database/creating_enums.md#all-of-the-keyvalue-pairs-should-be-defined-in-foss).
+
 ### Code in `app/views/`
 
 It's a very frequent problem that EE is adding some specific view code in a CE
diff --git a/doc/development/fe_guide/customizable_dashboards.md b/doc/development/fe_guide/customizable_dashboards.md
index 9e45c660745..ac8b0b8a1ab 100644
--- a/doc/development/fe_guide/customizable_dashboards.md
+++ b/doc/development/fe_guide/customizable_dashboards.md
@@ -6,27 +6,160 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # Customizable dashboards
 
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98610) in GitLab 15.5 as an [Experiment](../../policy/alpha-beta-support.md#experiment).
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98610) in GitLab 15.5 as an [Experiment](../../policy/experiment-beta-support.md#experiment).
 
-Customizable dashboards provide a dashboard structure that allows users to create
-their own dashboards and commit the structure to a repository.
+Customizable dashboards provide a configuation-based [dashboard](https://design.gitlab.com/patterns/dashboards)
+structure, which is used to render and modify dashboard configurations created by GitLab or users.
 
-This feature is available for Premium and Ultimate subscriptions.
+The dashboard structure does not provide the means to save and version control
+user configuration files on a repository. This functionality is provided by [Analytics dashboards](../../user/analytics/analytics_dashboards.md)
+which uses the customizable dashboard component.
+
+NOTE:
+Customizable dashboards is intended for Premium and Ultimate subscriptions.
+
+## Overview
+
+Customizable dashboard can be broken down into 3 logical components:
+
+- Dashboard
+- Panels
+- Visualizations
+
+A dashboard consists of a list of panels to render, each panel references one
+visualization, and a single visualization can be shared by many panels.
+
+A typical dashboard structure looks like this:
+
+```plaintext
+dashboard
+├── panelA
+│  └── visualizationX
+├── panelB
+│  └── visualizationY
+├── panelC
+│  └── visualizationY
+```
 
 ## Usage
 
 To use customizable dashboards:
 
-1. Create your dashboard component.
-1. Render an instance of `CustomizableDashboard`.
-1. Pass a list of panels to render.
+1. Create a new Vue component for your dashboard.
+1. Create a [visualization configuration](#visualization-configuration).
+1. Create your [dashboard configuration](#dashboard-configuration).
+1. Render an instance of `CustomizableDashboard` and pass it your [dashboard configuration](#using-the-component).
+
+### Visualization configuration
+
+Each visualization is a graphical representation of query results fetched from a data source.
+
+```javascript
+// visualizations.js
+
+export const pageViewsOverTime = {
+  // The name of the Vue component used to render the query.
+  type: 'LineChart',
+  // Chart options defined by the charting library being used by the panel.
+  options: {
+    xAxis: { name: __('Time'), type: 'time' },
+    yAxis: { name: __('Counts'), type: 'time' },
+  },
+  // The data to query
+  data: {
+    // The data source to query. Here it is Product Analytics.
+    type: 'cube_analytics',
+    // The query to run on the data source. Here in Cube.js format.
+    query: {
+      dimensions: [],
+      filters: [
+        {
+          member: 'SnowplowTrackedEvents.event',
+          operator: 'equals',
+          values: ['page_view']
+        }
+      ],
+      measures: ['SnowplowTrackedEvents.pageViewsCount'],
+      timeDimensions: [
+        {
+          dimension: 'SnowplowTrackedEvents.derivedTstamp',
+          granularity: 'day',
+        },
+      ],
+      limit: 100,
+      timezone: 'UTC',
+    },
+  },
+};
+```
+
+#### Adding a new visualization render type
+
+To add a new visualization render type:
+
+1. Create a new Vue component that accepts `data` and `options` properties.
+See [`line_chart.vue`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/assets/javascripts/analytics/analytics_dashboards/components/visualizations/line_chart.vue) as an example.
+1. Add your component to the list of conditional imports in [`panel_base.vue`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/assets/javascripts/vue_shared/components/customizable_dashboard/panels_base.vue#L13).
+
+#### Adding a new visualization data source
+
+To add a new data source:
+
+1. Create a new JavaScript module that exports a `fetch` method. See [`cube_analytics.js`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/assets/javascripts/analytics/analytics_dashboards/data_sources/cube_analytics.js#L122) as an example.
+1. Add your module to the list exports in [`data_sources/index.js`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/assets/javascripts/analytics/analytics_dashboards/data_sources/index.js).
+
+NOTE:
+Your data source must respect the filters so that all panels show data for
+the same date range.
+
+### Dashboard configuration
+
+Here is an example dashboard configuration:
 
-For example, a customizable dashboard for users over time:
+```javascript
+// constants.js
+import { pageViewsOverTime } from './visualizations';
+
+export const dashboard = {
+  slug: 'my_dashboard', // Used to set the URL path for the dashboard.
+  title: 'My dashboard title', // The title to display.
+  // Each dashboard consists of an array of panels to display.
+  panels: [
+    {
+      id: 1,
+      title: 'Page views over time', // The panel title to display.
+      // The visualization configuration. This can be shared by many panels.
+      visualization: pageViewsOverTime,
+      // Gridstack settings based upon https://github.com/gridstack/gridstack.js/tree/master/doc#item-options.
+      // All values are grid row/column numbers up to 12.
+      // We use the default 12 column grid https://github.com/gridstack/gridstack.js#change-grid-columns.
+      gridAttributes: {
+        yPos: 1,
+        xPos: 0,
+        width: 6,
+        height: 5,
+      },
+      // Optional overrides for the values in `visualization.data.query`.
+      // Here we override the Cube.js query to get page views per week instead of days.
+      queryOverrides: {
+        timeDimensions: {
+          dimension: 'SnowplowTrackedEvents.derivedTstamp',
+          granularity: 'week',
+        },
+      },
+    },
+  ],
+};
+```
+
+### Using the component
+
+Here is an example component that renders a customizable dashboard:
 
 ```vue
 <script>
 import CustomizableDashboard from 'ee/vue_shared/components/customizable_dashboard/customizable_dashboard.vue';
-import { s__ } from '~/locale';
+import { dashboard } from './constants';
 
 export default {
   name: 'AnalyticsDashboard',
@@ -35,63 +168,107 @@ export default {
   },
   data() {
     return {
-      panels: [
-        {
-          component: 'CubeLineChart', // The name of the panel component.
-          title: s__('ProductAnalytics|Users / Time'), // The title shown on the panel component.
-          // Gridstack settings based upon https://github.com/gridstack/gridstack.js/tree/master/doc#item-options.
-          // All values are grid row/column numbers up to 12.
-          // We use the default 12 column grid https://github.com/gridstack/gridstack.js#change-grid-columns.
-          gridAttributes: {
-            height: 4,
-            width: 6,
-            minHeight: 4,
-            minWidth: 6,
-            xPos: 0,
-            yPos: 0,
-          },
-          // Options that are used to set bespoke values for each panel.
-          // Available customizations are determined by the panel itself.
-          customizations: {},
-          // Chart options defined by the charting library being used by the panel.
-          chartOptions: {
-            xAxis: { name: __('Time'), type: 'time' },
-            yAxis: { name: __('Counts') },
-          },
-          // The data for the panel.
-          // This could be imported or in this case, a query passed to be used by the panels API.
-          // Each panel type determines how it handles this property.
-          data: {
-            query: {
-              users: {
-                measures: ['TrackedEvents.count'],
-                dimensions: ['TrackedEvents.eventType'],
-              },
-            },
-          },
-        },
-      ]
+      // We keep the original (default) dashboard object to track changes.
+      dashboard: {
+        ...dashboard,
+        default: { ...dashboard, }
+      },
+      // Optional dashboard filters. Currently the only availble filter is date range.
+      defaultFilters: {
+        dateRangeOption: 'last_7_days' // 'custom', 'today', 'last_7_days', 'last_30_days'
+        endDate: new Date(2023, 06, 14),
+        startDate: new Date(2023, 06, 7),
+      },
+      // Set to true to sync the filter object with the URL query string.
+      syncUrlFilters: true,
+      // Set to true to show the date range filter.
+      showDateRangeFilter: true,
+      // The maximum size of the date range allowed in days. 0 for unlimited.
+      dateRangeLimit: 0,
     };
   },
 };
 </script>
 
 <template>
-  <h1>{{ s__('ProductAnalytics|Analytics dashboard') }}</h1>
-  <customizable-dashboard :panels="panels" />
+  <customizable-dashboard
+    :initial-dashboard="dashboard"
+    :default-filters="defaultFilters"
+    :sync-url-filters="syncUrlFilters"
+    :show-date-range-filter="showDateRangeFilter"
+    :date-range-limit="dateRangeLimit"
+  />
 </template>
 ```
 
-The panels data can be retrieved from a file or API request, or imported through HTML data attributes.
+## Dashboard designer
 
-For each panel, a `component` is defined. Each `component` is a component declaration and should be included in
-[`vue_shared/components/customizable_dashboard/panels_base.vue`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/app/assets/javascripts/vue_shared/components/customizable_dashboard/panels_base.vue)
-as a dynamic import, to keep the memory usage down until it is used.
+> Introduced in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `combined_analytics_dashboards_editor`. Disabled by default.
 
-For example:
+The dashboard designer provides a graphical interface for users to modify the
+panels and add new ones on user-defined dashboards. Is is not available on
+GitLab hardcoded dashboards.
 
-```javascript
-components: {
-  CubeLineChart: () => import('ee/product_analytics/dashboards/components/panels/cube_line_chart.vue')
+NOTE:
+The dashboard designer is in the early experimental stage and subject to
+change.
+
+```vue
+<script>
+import { s__ } from '~/locale';
+
+export const I18N_MY_NEW_CATEGORY = s__('Namespace|My data source');
+
+export default {
+  name: 'AnalyticsDashboard',
+  data() {
+    return {
+      ...,
+      // Set to true to render the dashboard saving state.
+      isSaving: false,
+      // A list of availble visualizations categorized by feature.
+      availableVisualizations: {
+        // The visualization category title to display.
+        [I18N_MY_NEW_CATEGORY]: {
+          // Set to true when asynchronously loading the visualization IDs
+          loading: false,
+          // List of availble visualization IDs for the user to add to the dashboard.
+          visualizationIds: [
+            'page_views_over_time',
+            'events_over_time',
+          ],
+        },
+      }
+    };
+  },
+  methods: {
+    /**
+     * Event handler for when a user saves changes made to the current dashboard.
+     * @param  {String} dashboardId The current dashboard ID.
+     * @param  {String} newDashboardObject The newly modified dashboard object.
+     */
+    saveDashboard(dashboardId, newDashboardObject) {
+      // Save changes and modify `this.dashboard`.
+    },
+    /**
+     * Event handler for when a user adds a visualization in a new panel.
+     * @param  {String} visualizationId The ID (usually filename) of the visualization.
+     * @param  {String} visualizationSource The source to get the new visualization config.
+     */
+    addNewPanel(visualizationId, visualizationSource) {
+      // Load the visualization and push a new panel onto `this.dashboard.panels`.
+    },
+  },
 }
+</script>
+
+<template>
+  <customizable-dashboard
+    ...
+    :available-visualizations="availableVisualizations"
+    :is-saving="isSaving"
+    @save="handleSave"
+    @add-panel="handleAddPanel"
+  />
+</template>
 ```
diff --git a/doc/development/fe_guide/haml.md b/doc/development/fe_guide/haml.md
index 9dc5b265783..1dc8cf63de9 100644
--- a/doc/development/fe_guide/haml.md
+++ b/doc/development/fe_guide/haml.md
@@ -42,7 +42,7 @@ For example:
         = f.check_box :prevent_sharing_groups_outside_hierarchy, disabled: !can_change_prevent_sharing_groups_outside_hierarchy?(@group), class: 'custom-control-input'
         = f.label :prevent_sharing_groups_outside_hierarchy, class: 'custom-control-label' do
           %span
-            = s_('GroupSettings|Prevent members from sending invitations to groups outside of %{group} and its subgroups.').html_safe % { group: link_to_group(@group) }
+            = safe_format(s_('GroupSettings|Prevent members from sending invitations to groups outside of %{group} and its subgroups.'), group: link_to_group(@group))
           %p.help-text= prevent_sharing_groups_outside_hierarchy_help_text(@group)
 
     .form-group.gl-mb-3
@@ -61,16 +61,16 @@ For example:
   = gitlab_ui_form_for @group do |f|
     .form-group.gl-mb-3
       = f.gitlab_ui_checkbox_component :prevent_sharing_groups_outside_hierarchy,
-          s_('GroupSettings|Prevent members from sending invitations to groups outside of %{group} and its subgroups.').html_safe % { group: link_to_group(@group) },
+          safe_format(s_('GroupSettings|Prevent members from sending invitations to groups outside of %{group} and its subgroups.'), group: link_to_group(@group)),
           help_text: prevent_sharing_groups_outside_hierarchy_help_text(@group),
           checkbox_options: { disabled: !can_change_prevent_sharing_groups_outside_hierarchy?(@group) }
 
     .form-group.gl-mb-3
       = f.gitlab_ui_checkbox_component :lfs_enabled, checkbox_options: { checked: @group.lfs_enabled? } do |c|
-        = c.label do
+        - c.with_label do
           = _('Allow projects within this group to use Git LFS')
           = link_to sprite_icon('question-o'), help_page_path('topics/git/lfs/index')
-        = c.help_text do
+        - c.with_help_text do
           = _('This setting can be overridden in each project.')
   ```
 
diff --git a/doc/development/fe_guide/source_editor.md b/doc/development/fe_guide/source_editor.md
index 45ec3ba1464..943ac2969f3 100644
--- a/doc/development/fe_guide/source_editor.md
+++ b/doc/development/fe_guide/source_editor.md
@@ -210,7 +210,7 @@ export default {
 
 In the code example, `this` refers to the instance. By referring to the instance,
 we can access the complete underlying
-[Monaco editor API](https://microsoft.github.io/monaco-editor/api/),
+[Monaco editor API](https://microsoft.github.io/monaco-editor/docs.html),
 which includes functions like `getValue()`.
 
 Now let's use our extension:
diff --git a/doc/development/fe_guide/storybook.md b/doc/development/fe_guide/storybook.md
index 8e0814ad96b..eaa8f8b4068 100644
--- a/doc/development/fe_guide/storybook.md
+++ b/doc/development/fe_guide/storybook.md
@@ -53,8 +53,85 @@ To add a story:
    If the component is located in the `ee/` directory, make sure to prefix the story's title with `ee/` as well.
    This will ensure the Storybook navigation maps closely to our internal directory structure.
 
-## Mock backend APIs
+## Using GitLab REST and GraphQL APIs
 
-The GitLab Storybook uses [MirajeJS](https://miragejs.com/) to mock REST and GraphQL APIs. Storybook shares the MirajeJS server
-with the [frontend integration tests](../testing_guide/testing_levels.md#frontend-integration-tests). You can find the MirajeJS
-configuration files in `spec/frontend_integration/mock_server`.
+You can write stories for components that use either GitLab’s [REST](../../api/rest/index.md) or
+[GraphQL](../../api/graphql/index.md) APIs.
+
+### Set up API access token and GitLab instance URL
+
+To add a story with API access:
+
+1. Create a [personal access token](../../user/profile/personal_access_tokens.md) in your GitLab instance.
+
+   NOTE:
+   If you test against `gitlab.com`, make sure to use a token with `read_api` if possible and to make the token short-lived.
+
+1. Create an `.env` file in the `storybook` directory. Use the `storybook/.env.template` file as
+a starting point.
+
+1. Set the `API_ACCESS_TOKEN` variable to the access token that you created.
+
+1. Set the `GITLAB_URL` variable to the GitLab instance’s domain URL, for example: `http://gdk.test:3000`.
+
+1. Start or restart your storybook.
+
+You can also use the GitLab API Access panel in the Storybook UI to set the GitLab instance URL and access token.
+
+### Set up API access in your stories
+
+You should apply the `withGitLabAPIAccess` decorator to the stories that will consume GitLab’s APIs. This decorator
+will display a badge indicating that the story won't work without providing the API access parameters:
+
+```javascript
+import { withGitLabAPIAccess } from 'storybook_addons/gitlab_api_access';
+import Api from '~/api';
+import { ContentEditor } from './index';
+
+export default {
+  component: ContentEditor,
+  title: 'ce/content_editor/content_editor',
+  decorators: [withGitLabAPIAccess],
+};
+```
+
+#### Using REST API
+
+The Storybook sets up `~/lib/utils/axios_utils` in `storybook/config/preview.js`. Components that use the REST API
+should work out of the box as long as you provide a valid GitLab instance URL and access token.
+
+#### Using GraphQL
+
+To write a story for a component that uses the GraphQL API, use the `createVueApollo` method provided in
+the Story context.
+
+```javascript
+import Vue from 'vue';
+import VueApollo from 'vue-apollo';
+import { withGitLabAPIAccess } from 'storybook_addons/gitlab_api_access';
+import WorkspacesList from './list.vue';
+
+Vue.use(VueApollo);
+
+const Template = (_, { argTypes, createVueApollo }) => {
+  return {
+    components: { WorkspacesList },
+    apolloProvider: createVueApollo(),
+    provide: {
+      emptyStateSvgPath: '',
+    },
+    props: Object.keys(argTypes),
+    template: '<workspaces-list />',
+  };
+};
+
+export default {
+  component: WorkspacesList,
+  title: 'ee/remote_development/workspaces_list',
+  decorators: [withGitLabAPIAccess],
+};
+
+export const Default = Template.bind({});
+
+Default.args = {};
+```
diff --git a/doc/development/fe_guide/view_component.md b/doc/development/fe_guide/view_component.md
index 0245110ec75..cfd78597501 100644
--- a/doc/development/fe_guide/view_component.md
+++ b/doc/development/fe_guide/view_component.md
@@ -66,7 +66,7 @@ In its simplest form the banner component looks like this:
 ```haml
 = render Pajamas::BannerComponent.new(button_text: 'Learn more', button_link: example_path,
   svg_path: 'illustrations/example.svg') do |c|
-  - c.title { 'Hello world!' }
+  - c.with_title { 'Hello world!' }
   %p Content of your banner goes here...
 ```
 
@@ -75,11 +75,11 @@ instead of `svg_path` and the `primary_action` slot instead of `button_text` and
 
 ```haml
 = render Pajamas::BannerComponent.new do |c|
-  - c.illustration do
+  - c.with_illustration do
     = custom_icon('my_inline_svg')
-  - c.title do
+  - c.with_title do
     Hello world!
-  - c.primary_action do
+  - c.with_primary_action do
     = render 'my_button_in_a_partial'
 ```
 
@@ -133,12 +133,12 @@ The card has one mandatory `body` slot and optional `header` and `footer` slots:
 
 ```haml
 = render Pajamas::CardComponent.new do |c|
-  - c.header do
+  - c.with_header do
     I'm the header.
-  - c.body do
+  - c.with_body do
     %p Multiple line
     %p body content.
-  - c.footer do
+  - c.with_footer do
     Footer goes here.
 ```
 
@@ -164,9 +164,9 @@ For example:
 ```haml
 = render Pajamas::CheckboxTagComponent.new(name: 'project[initialize_with_sast]',
   checkbox_options: { data: { qa_selector: 'initialize_with_sast_checkbox', track_label: track_label, track_action: 'activate_form_input', track_property: 'init_with_sast' } }) do |c|
-  = c.label do
+  - c.with_label do
     = s_('ProjectsNew|Enable Static Application Security Testing (SAST)')
-  = c.help_text do
+  - c.with_help_text do
     = s_('ProjectsNew|Analyze your source code for known security vulnerabilities.')
     = link_to _('Learn more.'), help_page_path('user/application_security/sast/index'), target: '_blank', rel: 'noopener noreferrer', data: { track_action: 'followed' }
 ```
@@ -219,11 +219,11 @@ Many of the settings pages use a layout where the title and description are on t
 
 ```haml
 = render ::Layouts::HorizontalSectionComponent.new(options: { class: 'gl-mb-6' }) do |c|
-  = c.title { _('Naming, visibility') }
-  = c.description do
+  - c.with_title { _('Naming, visibility') }
+  - c.with_description do
     = _('Update your group name, description, avatar, and visibility.')
     = link_to _('Learn more about groups.'), help_page_path('user/group/index')
-  = c.body do
+  - c.with_body do
     .form-group.gl-form-group
       = f.label :name, _('New group name')
       = f.text_field :name
diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md
index 7ba774392a1..1a43084245e 100644
--- a/doc/development/fe_guide/vue.md
+++ b/doc/development/fe_guide/vue.md
@@ -28,10 +28,24 @@ To better explain this, let's imagine the page that has one toggle, and toggling
 - when you have to maintain any form of application state and share it between tags/elements;
 - when you expect complex logic to be added in the future - it's easier to start with basic Vue application than having to rewrite JS/HAML to Vue on the next step.
 
+## Avoid multiple Vue applications on the page
+
+In the past, we added interactivity to the page piece-by-piece, adding multiple small Vue applications to different parts of the rendered HAML page. However, this approach led us to multiple complications:
+
+- in most cases, these applications don't share state and perform API requests independently which grows a number of requests;
+- we have to provide data from Rails to Vue using multiple endpoints;
+- we cannot render Vue applications dynamically after page load, so the page structure becomes rigid;
+- we cannot fully leverage client-side routing to replace Rails routing;
+- multiple applications lead to unpredictable user experience, increased page complexity, harder debugging process;
+- the way apps communicate with each other affects Web Vitals numbers.
+
+Because of these reasons, we want to be cautious about adding new Vue applications to the pages where another Vue application is already present (this does not include old or new navigation). Before adding a new app, please make sure that it is absolutely impossible to extend an existing application to achieve a desired functionality. When in doubt, please feel free to ask for the architectural advise on `#frontend` or `#frontend-maintainers` Slack channel.
+
+If you still need to add a new application, please make sure it shares local state with existing applications (preferably via Apollo Client, or Vuex if we use REST API)
+
 ## Vue architecture
 
-All new features built with Vue.js must follow a [Flux architecture](https://facebookarchive.github.io/flux/).
-The main goal we are trying to achieve is to have only one data flow, and only one data entry.
+The main goal we are trying to achieve with Vue architecture is to have only one data flow, and only one data entry.
 To achieve this goal we use [Vuex](#vuex) or [Apollo Client](graphql.md#libraries)
 
 You can also read about this architecture in Vue documentation about
diff --git a/doc/development/feature_development.md b/doc/development/feature_development.md
index 15058134092..c4a5c996ab1 100644
--- a/doc/development/feature_development.md
+++ b/doc/development/feature_development.md
@@ -160,9 +160,9 @@ The following integration guides are internal. Some integrations require access
 - [Externalization](i18n/externalization.md)
 - [Translation](i18n/translation.md)
 
-## Product Intelligence guides
+## Analytics Instrumentation guides
 
-- [Product Intelligence guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)
+- [Analytics Instrumentation guide](https://about.gitlab.com/handbook/product/analytics-instrumentation-guide/)
 - [Service Ping guide](service_ping/index.md)
 - [Snowplow guide](snowplow/index.md)
 
diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md
index 87d2da016d6..30837ac8f3f 100644
--- a/doc/development/feature_flags/index.md
+++ b/doc/development/feature_flags/index.md
@@ -144,6 +144,14 @@ An `experiment` feature flag should conform to the same standards as a `developm
 although the interface has some differences. An experiment feature flag should have a rollout issue,
 created using the [Experiment Tracking template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Experiment%20Rollout.md). More information can be found in the [experiment guide](../experiment_guide/index.md).
 
+### `worker` type
+
+`worker` feature flags are used for controlling Sidekiq workers behavior, such as deferring Sidekiq jobs.
+
+`worker` feature flags likely do not have any YAML definition as the name could be dynamically generated using
+the worker name itself, e.g. `defer_sidekiq_jobs_AuthorizedProjectsWorker`. Some examples for using `worker` type feature
+flags can be found in [deferring Sidekiq jobs](#deferring-sidekiq-jobs).
+
 ## Feature flag definition and validation
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229161) in GitLab 13.3.
@@ -698,3 +706,40 @@ test is written to enable/disable a feature flag explicitly.
 When a feature flag is changed on Staging or on GitLab.com, a Slack message will be posted to the `#qa-staging` or `#qa-production` channels to inform
 the pipeline triage DRI so that they can more easily determine if any failures are related to a feature flag change. However, if you are working on a change you can
 help to avoid unexpected failures by [confirming that the end-to-end tests pass with a feature flag enabled.](../testing_guide/end_to_end/feature_flags.md#confirming-that-end-to-end-tests-pass-with-a-feature-flag-enabled)
+
+## Controlling Sidekiq worker behavior with feature flags
+
+Feature flags with [`worker` type](#worker-type) can be used to control the behavior of a Sidekiq worker.
+
+### Deferring Sidekiq jobs
+
+Feature flags with the format of `defer_sidekiq_jobs_{WorkerName}` delay the execution of the worker
+by scheduling the job at a later time.
+Deferring jobs can be useful during an incident where contentious behavior from
+worker instances are saturating infrastructure resources (such as database and database connection pool).
+The implementation can be found at [DeferJobs Sidekiq server middleware](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/sidekiq_middleware/defer_jobs.rb).
+
+NOTE:
+Jobs are deferred indefinitely as long as the feature flag is enabled. It is important to disable the
+feature flag after the worker is deemed safe to continue processing.
+
+When set to true, 100% of the jobs are deferred. When you want processing to resume, you can
+use a **percentage of time** rollout. For example:
+
+```shell
+# defer 100% of the jobs
+/chatops run feature set defer_sidekiq_jobs_SlowRunningWorker true
+
+# defer 99% of the jobs, only letting 1% processed
+/chatops run feature set defer_sidekiq_jobs_SlowRunningWorker 99
+
+# defer 50% of the jobs
+/chatops run feature set defer_sidekiq_jobs_SlowRunningWorker 50
+
+# stop deferring the jobs, jobs are being processed normally
+/chatops run feature set defer_sidekiq_jobs_SlowRunningWorker false
+```
+
+NOTE:
+The percentage of time value denotes the percentage of time the jobs are being deferred (instead of being processed).
+For example, setting to `99` means only 1% of the jobs are being processed at random.
diff --git a/doc/development/gemfile.md b/doc/development/gemfile.md
index add93e37024..e6275068ea8 100644
--- a/doc/development/gemfile.md
+++ b/doc/development/gemfile.md
@@ -112,71 +112,7 @@ ourselves, or because we think it would benefit the wider community.
 Extracting code to a gem also means that we can be sure that the gem
 does not contain any hidden dependencies on our application code.
 
-In general, we want to think carefully before doing this as there are
-also disadvantages:
-
-### Potential disadvantages
-
-1. Gems - even those maintained by GitLab - do not necessarily go
-   through the same [code review process](code_review.md) as the main
-   Rails application.
-1. Extracting the code into a separate project means that we need a
-   minimum of two merge requests to change functionality: one in the gem
-   to make the functional change, and one in the Rails app to bump the
-   version.
-1. Our needs for our own usage of the gem may not align with the wider
-   community's needs. In general, if we are not using the latest version
-   of our own gem, that might be a warning sign.
-
-### Create and publish a Ruby gem
-
-In the case where we do want to extract some library code we've written
-to a gem, go through these steps:
-
-1. Determine a suitable name for the gem. If it's a GitLab-owned gem, prefix
-   the gem name with `gitlab-`. For example, `gitlab-sidekiq-fetcher`.
-1. Create the gem or fork as necessary.
-1. Ensure the `gitlab_rubygems` group is an owner of the new gem by running:
-
-   ```shell
-   gem owner <gem-name> --add gitlab_rubygems
-   ```
-
-1. [Publish the gem to rubygems.org](https://guides.rubygems.org/publishing/#publishing-to-rubygemsorg)
-1. Visit `https://rubygems.org/gems/<gem-name>` and verify that the gem published
-   successfully and `gitlab_rubygems` is also an owner.
-1. Start with the code in the Rails application. Here it's fine to have
-   the code in `lib/` and loaded automatically. We can skip this step if
-   the step below makes more sense initially.
-1. Before extracting to its own project, move the gem to `vendor/gems` and
-   load it in the `Gemfile` using the `path` option. This gives us a gem
-   that can be published to RubyGems.org, with its own test suite and
-   isolated set of dependencies, that is still in our main code tree and
-   goes through the standard code review process.
-   - For an example, see the [merge request !57805](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/57805).
-1. Once the gem is stable - we have been using it in production for a
-   while with few, if any, changes - extract to its own project under
-   the [`gitlab-org/ruby/gems` namespace](https://gitlab.com/gitlab-org/ruby/gems/).
-
-   - To create this project:
-       1. Follow the [instructions for new projects](https://about.gitlab.com/handbook/engineering/gitlab-repositories/#creating-a-new-project).
-       1. Follow the instructions for setting up a [CI/CD configuration](https://about.gitlab.com/handbook/engineering/gitlab-repositories/#cicd-configuration).
-       1. Follow the instructions for [publishing a project](https://about.gitlab.com/handbook/engineering/gitlab-repositories/#publishing-a-project).
-   - See [issue #325463](https://gitlab.com/gitlab-org/gitlab/-/issues/325463)
-     for an example.
-   - In some cases we may want to move a gem to its own namespace. Some
-     examples might be that it will naturally have more than one project
-     (say, something that has plugins as separate libraries), or that we
-     expect non-GitLab-team-members to be maintainers on this project as
-     well as GitLab team members.
-
-     The latter situation (maintainers from outside GitLab) could also
-     apply if someone who currently works at GitLab wants to maintain
-     the gem beyond their time working at GitLab.
-
-When publishing a gem to RubyGems.org, also note the section on
-[gem owners](https://about.gitlab.com/handbook/developer-onboarding/#ruby-gems)
-in the handbook.
+Read more about [Gems development guidelines](gems.md).
 
 ## Upgrade Rails
 
diff --git a/doc/development/gems.md b/doc/development/gems.md
new file mode 100644
index 00000000000..709dfa105bf
--- /dev/null
+++ b/doc/development/gems.md
@@ -0,0 +1,321 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Gems development guidelines
+
+GitLab uses Gems as a tool to improve code reusability and modularity
+in a monolithic codebase.
+
+Sometimes we create libraries within our codebase that we want to
+extract, either because their functionality is highly isolated,
+we want to use them in other applications
+ourselves, or we think it would benefit the wider community.
+Extracting code to a gem also means that we can be sure that the gem
+does not contain any hidden dependencies on our application code.
+
+## When to use Gems
+
+Gems should be used always when implementing functions that can be considered isolated,
+that are decoupled from the business logic of GitLab and can be developed separately. Consider the
+following examples where Gem logic could be placed:
+
+The best example where we can look for opportunities to introduce new gems
+is the [lib/](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/) folder.
+
+The **lib/** folder is a mix of code that is generic/universal, GitLab-specific, and tightly integrated with the rest of the codebase.
+
+If you cannot find a good place for your code in **lib/** you should strongly
+consider creating the new Gem [In the same repo](#in-the-same-repo).
+
+## In the same repo
+
+**Our GitLab Gems should be always put in `gems/` of GitLab monorepo.**
+
+That gives us the advantages of gems (modular code, quicker to run tests in development).
+and prevents complexity (coordinating changes across repos, new permissions, multiple projects, etc.).
+
+Gems stored in the same repo should be referenced in `Gemfile` with the `path:` syntax.
+They should not be published to RubyGems.
+
+### Advantages
+
+Using Gems can provide several benefits for code maintenance:
+
+- Code Reusability - Gems are isolated libraries that serve single purpose. When using Gems, a common functions
+  can be isolated in a simple package, that is well documented, tested, and re-used in different applications.
+
+- Modularity - Gems help to create isolation by encapsulating specific functionality within self-contained library.
+  This helps to better organize code, better define who is owner of a given module, makes it easier to maintain
+  or update specific gems.
+
+- Small - Gems by design due to implementing isolated set of functions are small. Small projects are much easier
+  to comprehend, extend and maintain.
+
+- Testing - Using Gems since they are small makes much faster to run all tests, or be very through with testing of the gem.
+  Since the gem is packaged, not changed too often, it also allows us to run those tests less frequently improving
+  CI testing time.
+
+### To Do
+
+#### Desired use cases
+
+The `gitlab-utils` is a Gem containing as of set of class that implement common intrisic functions
+used by GitLab developers, like `strong_memoize` or `Gitlab::Utils.to_boolean`.
+
+The `gitlab-database-schema-migrations` is a potential Gem containing our extensions to Rails
+framework improving how database migrations are stored in repository. This builds on top of Rails
+and is not specific to GitLab the application, and could be generally used for other projects
+or potentially be upstreamed.
+
+The `gitlab-database-load-balancing` similar to previous is a potential Gem to implement GitLab specific
+load balancing to Rails database handling. Since this is rather complex and highly specific code
+maintaing it's complexity in a isolated and well tested Gem would help with removing this complexity
+from a big monolithic codebase.
+
+The `gitlab-flipper` is another potential Gem implementing all our custom extensions to support feature
+flags in a codebase. Over-time the monolithic codebase did grow with the check for feature flags
+usage, adding consistency checks and various helpers to track owners of feature flags added. This is
+not really part of GitLab business logic and could be used to better track our implementation
+of Flipper and possibly much easier change it to dogfood [GitLab Feature Flags](../operations/feature_flags.md).
+
+The `gitlab-ci-reports-parsers` is a potential Gem that could implement all various parsers for various formats.
+The parsed output would be transformed into objects that could then be used by GitLab the application
+to store it in the database. This functionality could be an additional Gem since it is isolated,
+rarely changed, and GitLab Rails only consumes the data.
+
+The same pattern could be applied to all other type of parsers, like security vulnerabilities, or any
+other complex structures that need to be transformed into a form that is consumed by GitLab Rails.
+
+The `gitlab-active_record` is a gem adding GitLab specific Active Record patches.
+It is very well desired for such to be managed separately to isolate complexity.
+
+#### Other potential use cases
+
+The `gitlab-ci-config` is a potential Gem containing all our CI code used to parse `.gitlab-ci.yml`.
+This code is today lightly interlocked with GitLab the application due to lack of proper abstractions.
+However, moving this to dedicated Gem could allow us to build various adapters to handle integration
+with GitLab the application. The interface would for example define an adapter to resolve `includes:`.
+Once we would have a `gitlab-ci-config` Gem it could be used within GitLab and outside of GitLab Rails
+and [GitLab CLI](https://gitlab.com/gitlab-org/cli).
+
+### Create and use a new Gem
+
+You can see example adding new Gem: [!121676](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121676).
+
+1. Create a new Ruby Gem in `gems/gitlab-<name-of-gem>` with `bundle gem gems/gitlab-<name-of-gem> --no-exe --no-coc --no-ext --no-mit`.
+1. Edit or remove `gitlab-<name-of-gem>/README.md` to provide a simple one paragraph description of the Gem.
+1. Edit `gitlab-<name-of-gem>/gitlab-<name-of-gem>.gemspec` and fill the details about the Gem as in the following example:
+
+    ```ruby
+    Gem::Specification.new do |spec|
+      spec.name = "gitlab-<name-of-gem>"
+      spec.version = Gitlab::NameOfGem::VERSION
+      spec.authors = ["group::tenant-scale"]
+      spec.email = ["engineering@gitlab.com"]
+
+      spec.summary = "GitLab's RSpec extensions"
+      spec.description = "A set of useful helpers to configure RSpec with various stubs and CI configs."
+      spec.homepage = "https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-<name-of-gem>"
+      spec.required_ruby_version = ">= 2.6.0"
+    end
+    ```
+
+1. Update `gems/gitlab-<name-of-gem>/.rubocop` with:
+
+    ```yaml
+    inherit_from:
+      - ../../.rubocop.yml
+
+    CodeReuse/ActiveRecord:
+      Enabled: false
+
+    AllCops:
+      TargetRubyVersion: 3.0
+
+    Naming/FileName:
+      Exclude:
+        - spec/**/*.rb
+    ```
+
+1. Configure CI for a newly added Gem:
+
+- Add `gems/gitlab-<name-of-gem>/.gitlab-ci.yml`:
+
+    ```yaml
+    workflow:
+      rules:
+        - if: $CI_MERGE_REQUEST_ID
+
+    rspec:
+      image: "ruby:${RUBY_VERSION}"
+      cache:
+        key: gitlab-<name-of-gem>
+        paths:
+          - gitlab-<name-of-gem>/vendor/ruby
+      before_script:
+        - cd vendor/gems/bundler-checksum
+        - ruby -v                                   # Print out ruby version for debugging
+        - gem install bundler --no-document         # Bundler is not installed with the image
+        - bundle config set --local path 'vendor'   # Install dependencies into ./vendor/ruby
+        - bundle config set with 'development'
+        - bundle config set --local frozen 'true'   # Disallow Gemfile.lock changes on CI
+        - bundle config                             # Show bundler configuration
+        - bundle install -j $(nproc)
+      script:
+        - bundle exec rspec
+      parallel:
+        matrix:
+          - RUBY_VERSION: ["2.7", "3.0", "3.1", "3.2"]
+    ```
+
+- To `.gitlab/ci/rules.gitlab-ci.yml` add:
+
+    ```yaml
+    .gems:rules:gitlab-<name-of-gem>:
+      rules:
+        - <<: *if-merge-request
+          changes: ["gems/gitlab-<name-of-gem>/**/*"]
+    ```
+
+- To `.gitlab/ci/gitlab-gems.gitlab-ci.yml` add:
+
+    ```yaml
+    gems gitlab-<name-of-gem>:
+      extends:
+        - .gems:rules:gitlab-<name-of-gem>
+      needs: []
+      trigger:
+        include: gems/gitlab-<name-of-gem>/.gitlab-ci.yml
+        strategy: depend
+    ```
+
+1. Reference Gem in `Gemfile` with:
+
+    ```ruby
+    gem 'gitlab-<name-of-gem>', path: 'gems/gitlab-<name-of-gem>'
+    ```
+
+## In the external repo
+
+In general, we want to think carefully before doing this as there are
+severe disadvantages.
+
+Gems stored in the external repo MUST be referenced in `Gemfile` with `version` syntax.
+They MUST be always published to RubyGems.
+
+### Examples
+
+At GitLab we use a number of external gems:
+
+- [LabKit Ruby](https://gitlab.com/gitlab-org/labkit-ruby)
+- [GitLab Ruby Gems](https://gitlab.com/gitlab-org/ruby/gems)
+
+### Potential disadvantages
+
+- Gems - even those maintained by GitLab - do not necessarily go
+  through the same [code review process](code_review.md) as the main
+  Rails application. This is particularly critical for Application Security.
+- Requires setting up CI/CD from scratch, including tools like Danger that
+  support consistent code review standards.
+- Extracting the code into a separate project means that we need a
+  minimum of two merge requests to change functionality: one in the gem
+  to make the functional change, and one in the Rails app to bump the
+  version.
+- Integration with `gitlab-rails` requiring a second MR means integration problems
+  may be discovered late.
+- With a smaller pool of reviewers and maintainers compared to `gitlab-rails`,
+  it may take longer to get code reviewed and the impact of "bus factor" increases.
+- Inconsistent workflows for how a new gem version is released. It is currently at
+  the discretion of library maintainers to decide how it works.
+- Promotes knowledge silos because code has less visibility and exposure than `gitlab-rails`.
+- We have a well defined process for promoting GitLab reviewers to maintainers.
+  This is not true for extracted libraries, increasing the risk of lowering the bar for code reviews,
+  and increasing the risk of shipping a change.
+- Our needs for our own usage of the gem may not align with the wider
+  community's needs. In general, if we are not using the latest version
+  of our own gem, that might be a warning sign.
+
+### Potential advantages
+
+- Faster feedback loops, since CI/CD runs against smaller repositories.
+- Ability to expose the project to the wider community and benefit from external contributions.
+- Repository owners are most likely the best audience to review a change, which reduces
+  the necessity of finding the right reviewers in `gitlab-rails`.
+
+### Create and publish a Ruby gem
+
+The project for a new Gem should always be created in [`gitlab-org/ruby/gems` namespace](https://gitlab.com/gitlab-org/ruby/gems/):
+
+1. Determine a suitable name for the gem. If it's a GitLab-owned gem, prefix
+   the gem name with `gitlab-`. For example, `gitlab-sidekiq-fetcher`.
+1. Create the gem or fork as necessary.
+1. Ensure the `gitlab_rubygems` group is an owner of the new gem by running:
+
+   ```shell
+   gem owner <gem-name> --add gitlab_rubygems
+   ```
+
+1. [Publish the gem to rubygems.org](https://guides.rubygems.org/publishing/#publishing-to-rubygemsorg)
+1. Visit `https://rubygems.org/gems/<gem-name>` and verify that the gem published
+   successfully and `gitlab_rubygems` is also an owner.
+1. Create a project in [`gitlab-org/ruby/gems` namespace](https://gitlab.com/gitlab-org/ruby/gems/).
+
+   - To create this project:
+       1. Follow the [instructions for new projects](https://about.gitlab.com/handbook/engineering/gitlab-repositories/#creating-a-new-project).
+       1. Follow the instructions for setting up a [CI/CD configuration](https://about.gitlab.com/handbook/engineering/gitlab-repositories/#cicd-configuration).
+       1. Follow the instructions for [publishing a project](https://about.gitlab.com/handbook/engineering/gitlab-repositories/#publishing-a-project).
+   - See [issue #325463](https://gitlab.com/gitlab-org/gitlab/-/issues/325463)
+     for an example.
+   - In some cases we may want to move a gem to its own namespace. Some
+     examples might be that it will naturally have more than one project
+     (say, something that has plugins as separate libraries), or that we
+     expect users outside GitLab to be maintainers on this project as
+     well as GitLab team members.
+
+     The latter situation (maintainers from outside GitLab) could also
+     apply if someone who currently works at GitLab wants to maintain
+     the gem beyond their time working at GitLab.
+
+When publishing a gem to RubyGems.org, also note the section on
+[gem owners](https://about.gitlab.com/handbook/developer-onboarding/#ruby-gems)
+in the handbook.
+
+## The `vendor/gems/`
+
+The purpose of `vendor/` is to pull into GitLab monorepo external dependencies,
+which do have external repositories, but for the sake of simplicity we want
+to store them in monorepo:
+
+- The `vendor/gems/` MUST ONLY be used if we are pulling from external repository either via script, or manually.
+- The `vendor/gems/` MUST NOT be used for storing in-house gems.
+- The `vendor/gems/` MAY accept fixes to make them buildable with GitLab monorepo
+- The `gems/` MUST be used for storing all in-house gems that are part of GitLab monorepo.
+- The **RubyGems** MUST be used for all externally stored dependencies that are not in `gems/` in GitLab monorepo.
+
+### Handling of an existing gems in `vendor/gems`
+
+- For in-house Gems that do not have external repository and are currently stored in `vendor/gems/`:
+
+  - For Gems that are used by other repositories:
+
+    - We will migrate it into its own repository.
+    - We will start or continue publishing them via RubyGems.
+    - Those Gems will be referenced via version in `Gemfile` and fetched from RubyGems.
+
+  - For Gems that are only used by monorepo:
+
+    - We will stop publishing new versions to RubyGems.
+    - We will not pull from RubyGems already published versions since there might
+      be applications depedent on those.
+    - We will move those gems to `gems/`.
+    - Those Gems will be referenced via `path:` in `Gemfile`.
+
+- For `vendor/gems/` that are external and vendored in monorepo:
+
+  - We will maintain them in the repository if they require some fixes that cannot be or are not yet upstreamed.
+  - It is expected that vendored gems might be published by third-party.
+  - Those Gems will not be published by us to RubyGems.
+  - Those Gems will be referenced via `path:` in `Gemfile`, since we cannot depend on RubyGems.
diff --git a/doc/development/github_importer.md b/doc/development/github_importer.md
index 73497c22b65..4a24279043d 100644
--- a/doc/development/github_importer.md
+++ b/doc/development/github_importer.md
@@ -149,10 +149,10 @@ comments.
 This worker imports note attachments that are linked inside Markdown.
 For each entity with Markdown text in the project, we schedule a job of:
 
-- `Gitlab::GithubImport::ImportReleaseAttachmentsWorker` for every release.
-- `Gitlab::GithubImport::ImportNoteAttachmentsWorker` for every note.
-- `Gitlab::GithubImport::ImportIssueAttachmentsWorker` for every issue.
-- `Gitlab::GithubImport::ImportMergeRequestAttachmentsWorker` for every merge request.
+- `Gitlab::GithubImport::Importer::Attachments::ReleasesImporter` for every release.
+- `Gitlab::GithubImport::Importer::Attachments::NotesImporter` for every note.
+- `Gitlab::GithubImport::Importer::Attachments::IssuesImporter` for every issue.
+- `Gitlab::GithubImport::Importer::Attachments::MergeRequestsImporter` for every merge request.
 
 Each job:
 
@@ -205,7 +205,7 @@ also reduces pressure on the system as a whole.
 
 GitLab includes a worker called `Gitlab::Import::StuckProjectImportJobsWorker`
 that periodically runs and marks project imports as failed if they have been
-running for more than 15 hours. For GitHub projects, this poses a bit of a
+running for more than 24 hours. For GitHub projects, this poses a bit of a
 problem: importing large projects could take several hours depending on how
 often we hit the GitHub rate limit (more on this below), but we don't want
 `Gitlab::Import::StuckProjectImportJobsWorker` to mark our import as failed because of this.
diff --git a/doc/development/gitlab_shell/index.md b/doc/development/gitlab_shell/index.md
index 0663341f806..ef034761a1f 100644
--- a/doc/development/gitlab_shell/index.md
+++ b/doc/development/gitlab_shell/index.md
@@ -218,5 +218,5 @@ sequenceDiagram
 ## Related topics
 
 - [LICENSE](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/LICENSE)
-- [PROCESS.md](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/PROCESS.md)
+- [Processes](process.md)
 - [Using the GitLab Shell chart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/)
diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md
index ac14b1b5ea2..12ef454d234 100644
--- a/doc/development/i18n/externalization.md
+++ b/doc/development/i18n/externalization.md
@@ -561,7 +561,7 @@ To include formatting in the translated string, you can do the following:
 - In Ruby/HAML:
 
   ```ruby
-  safe_format(_('Some %{strongOpen}bold%{strongClose} text.'), strongOpen: '<strong>'.html_safe, strongClose: '</strong>'.html_safe)
+  safe_format(_('Some %{strongOpen}bold%{strongClose} text.'), tag_pair(tag.strong, :strongOpen, :strongClose))
   # => 'Some <strong>bold</strong> text.'
   ```
 
@@ -589,7 +589,7 @@ instead:
 - In Ruby/HAML:
 
    ```ruby
-   html_escape_once(_('In &lt; 1 hour')).html_safe
+   safe_format(_('In &lt; 1 hour'))
 
    # => 'In < 1 hour'
    ```
@@ -800,8 +800,8 @@ translatable in certain languages.
 
   ```haml
   - zones_link_url = 'https://cloud.google.com/compute/docs/regions-zones/regions-zones'
-  - zones_link_start = safe_format('<a href="%{url}" target="_blank" rel="noopener noreferrer">', url: zones_link_url)
-  = safe_format(s_('ClusterIntegration|Learn more about %{zones_link_start}zones%{zones_link_end}'), zones_link_start: zones_link_start, zones_link_end: '</a>'.html_safe)
+  - zones_link = link_to('', zones_link_url, target: '_blank', rel: 'noopener noreferrer')
+  = safe_format(s_('ClusterIntegration|Learn more about %{zones_link_start}zones%{zones_link_end}'), tag_pair(zones_link, :zones_link_start, :zones_link_end))
   ```
 
 - In Vue, instead of:
diff --git a/doc/development/i18n/index.md b/doc/development/i18n/index.md
index 0f9688ec26d..b7e385a774a 100644
--- a/doc/development/i18n/index.md
+++ b/doc/development/i18n/index.md
@@ -31,7 +31,7 @@ See [Externalization for GitLab](externalization.md).
 
 ## Translate strings
 
-The translation process is managed at [https://translate.gitlab.com](https://translate.gitlab.com)
+The translation process is managed at [https://crowdin.com/project/gitlab-ee](https://crowdin.com/project/gitlab-ee)
 using [Crowdin](https://crowdin.com/).
 You must create a Crowdin account before you can submit translations. Once you are signed in, select
 the language you wish to contribute translations to.
diff --git a/doc/development/identity_verification.md b/doc/development/identity_verification.md
new file mode 100644
index 00000000000..a0593905879
--- /dev/null
+++ b/doc/development/identity_verification.md
@@ -0,0 +1,110 @@
+---
+stage: Data Science
+group: Anti-Abuse
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Identity verification development
+
+For information on this feature that are not development-specific, see the [feature documentation](../security/identity_verification.md).
+
+## Feature flags
+
+Because of the many registration paths and multiple verification stages, identity verification has several feature flags.
+
+Before you enable these features, ensure [hard email confirmation](../security/user_email_confirmation.md) is enabled and [Arkose](../integration/arkose.md#configuration) is configured properly.
+
+| Feature flag name | Description |
+|---------|-------------|
+| `identity_verification` | Turns on email verification for all registration paths |
+| `identity_verification_phone_number` | Turns on phone verification for medium risk users for all flows (the Arkose challenge flag for the specific flow and the `identity_verification` flag must be enabled for this to have effect) |
+| `identity_verification_credit_card` | Turns on credit card verification for high risk users for all flows (the Arkose challenge flag for the specific flow and the `identity_verification` flag must be enabled for this to have effect) |
+| `arkose_labs_signup_challenge` | Enables Arkose challenge for all flows, except the Trial and OAuth flows |
+| `arkose_labs_trial_signup_challenge` | Enables Arkose challenge for the Trial flow (the `arkose_labs_signup_challenge` flag must be enabled as well for this to have effect) |
+| `arkose_labs_oauth_signup_challenge` | Enables Arkose challenge for the OAuth flow |
+
+## Logging
+
+You can triage and debug issues raised by identity verification with the [GitLab production logs](https://log.gprd.gitlab.net).
+
+### View logs associated to a user and email verification
+
+To view logs associated to the [email stage](../security/identity_verification.md#email-verification) for a user:
+
+- Query the GitLab production logs with the following KQL:
+
+  ```plaintext
+  KQL: json.controller:"IdentityVerificationController" AND json.username:replace_username_here
+  ```
+
+Valuable debugging information can be found in the `json.action` and `json.location` columns.
+
+### View logs associated to a user and phone verification
+
+To view logs associated to the [phone stage](../security/identity_verification.md#phone-number-verification) for a user:
+
+- Query the GitLab production logs with the following KQL:
+
+  ```plaintext
+  KQL: json.message: "IdentityVerification::Phone" AND json.username:replace_username_here
+  ```
+
+On rows where `json.event` is `Failed Attempt`, you can find valuable debugging information in the `json.reason` column such as:
+
+| Reason  | Description |
+|---------|-------------|
+ | `invalid_phone_number` | Either there was a typo in the phone number, or the user used a VOIP number. GitLab does not allow users to sign up with non-mobile phone numbers. |
+| `invalid_code` | The user entered an incorrect verification code. |
+| `rate_limited` | The user had 10 or more failed attempts, so they were rate-limited for one hour. |
+| `related_to_banned_user` | The user tried a phone number already related to a banned user. |
+
+### View logs associated to a user and credit card verification
+
+To view logs associated to the [credit card stage](../security/identity_verification.md#credit-card-verification) for a user:
+
+- Query the GitLab production logs with the following KQL:
+
+  ```plaintext
+  KQL: json.message: "IdentityVerification::CreditCard" AND json.username:replace_username_here
+  ```
+
+On rows where `json.event` is `Failed Attempt`, you can find valuable debugging information in the `json.reason` column such as:
+
+| Reason  | Description |
+|---------|-------------|
+| `rate_limited` | The user had 10 or more failed attempts, so they were rate-limited for one hour. |
+| `related_to_banned_user` | The user tried a credit card number already related to a banned user. |
+
+### View logs associated with high-risk users
+
+To view logs associated with the [credit card stage](../security/identity_verification.md#credit-card-verification) for high-risk users:
+
+- Query the GitLab production logs with the following KQL:
+
+  ```plaintext
+  json.controller:"SubscriptionsController" AND json.action:"payment_form" AND json.params.value:"cc_registration_validation"
+  ```
+
+## Code walkthrough
+
+<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
+For a walkthrough and high level explanation of the code, see [Identity Verification - Code walkthrough](https://www.youtube.com/watch?v=DIsnMiNzND8).
+
+## QA Integration
+
+For end-to-end production and staging tests to function properly, GitLab [allows QA users to bypass identity verification](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/117633).
+
+## Additional resources
+
+<!-- markdownlint-disable MD044 -->
+The [Anti-abuse team](https://about.gitlab.com/handbook/engineering/development/data-science/anti-abuse/#team-members) owns identity verification. You can join our channel on Slack: [#g_anti-abuse](https://gitlab.slack.com/archives/C03EH5HCLPR).
+<!-- markdownlint-enable MD044 -->
+
+For help with Telesign:
+
+<!-- markdownlint-disable MD044 -->
+- Telesign/GitLab collaboration channel on Slack: [#gitlab-telesign-support](https://gitlab.slack.com/archives/C052EAXB6BY)
+<!-- markdownlint-enable MD044 -->
+- Telesign support contact: `support@telesign.com`
+- [Telesign portal](https://teleportal.telesign.com/)
+- [Telesign documentation](https://developer.telesign.com/enterprise/docs/get-started-with-docs)
diff --git a/doc/development/import_project.md b/doc/development/import_project.md
index ed5854f8833..0be17ea5873 100644
--- a/doc/development/import_project.md
+++ b/doc/development/import_project.md
@@ -52,6 +52,12 @@ There is also an option to [import the project via GitHub](../user/project/impor
 
 This method takes longer to import than the other methods and depends on several factors. It's recommended to use the other methods.
 
+To test importing from GitHub Enterprise (GHE) to GitLab, you need a GHE instance. You can request a
+[GitHub Enterprise Server trial](https://docs.github.com/en/get-started/signing-up-for-github/setting-up-a-trial-of-github-enterprise-server) and install it on Google Cloud Platform.
+
+- GitLab team members can use [Sandbox Cloud Realm](https://about.gitlab.com/handbook/infrastructure-standards/realms/sandbox/) for this purpose.
+- Others can request a [Google Cloud Platforms free trial](https://cloud.google.com/free).
+
 ### Import by using a Rake task
 
 To import the test project by using a Rake task, see
diff --git a/doc/development/integrations/index.md b/doc/development/integrations/index.md
index 4c66dbfa1a4..b51a2799088 100644
--- a/doc/development/integrations/index.md
+++ b/doc/development/integrations/index.md
@@ -10,7 +10,7 @@ description: "GitLab's development guidelines for Integrations"
 This page provides development guidelines for implementing [GitLab integrations](../../user/project/integrations/index.md),
 which are part of our [main Rails project](https://gitlab.com/gitlab-org/gitlab).
 
-Also see our [direction page](https://about.gitlab.com/direction/manage/integrations/) for an overview of our strategy around integrations.
+Also see our [direction page](https://about.gitlab.com/direction/manage/import_and_integrate/integrations/) for an overview of our strategy around integrations.
 
 This guide is a work in progress. You're welcome to ping `@gitlab-org/manage/integrations`
 if you need clarification or spot any outdated information.
@@ -300,6 +300,40 @@ All UI strings should be prepared for translation by following our [internationa
 
 The strings should use the integration name as [namespace](../i18n/externalization.md#namespaces), for example, `s_('FooBarIntegration|My string')`.
 
+## Deprecate and remove an integration
+
+To remove an integration, you must first deprecate the integration. For more information,
+see the [feature deprecation guidelines](../../development/deprecation_guidelines/index.md).
+
+### Deprecate an integration
+
+You must announce any deprecation [no later than the third milestone preceding intended removal](../../development/deprecation_guidelines/index.md#when-can-a-feature-be-deprecated).
+To deprecate an integration:
+
+- [Add a deprecation entry](../../development/deprecation_guidelines/index.md#update-the-deprecations-and-removals-documentation-pages).
+- [Mark the integration documentation as deprecated](../../development/documentation/versions.md#deprecate-a-page-or-topic).
+- Optional. To prevent any new project-level records from
+  being created, add the integration to `Project#disabled_integrations` (see [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114835)).
+
+### Remove an integration
+
+To safely remove an integration, you must stage the removal across two milestones.
+
+In the major milestone of intended removal (M.0), disable the integration and delete the records from the database:
+
+- Remove the integration from `Integration::INTEGRATION_NAMES`.
+- Delete the integration model's `#execute` and `#test` methods (if defined), but keep the model.
+- Add a post-migration to delete the integration records from PostgreSQL (see [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114721)).
+- [Add a removal entry](../../development/deprecation_guidelines/index.md#update-the-deprecations-and-removals-documentation-pages).
+- [Mark the integration documentation as removed](../../development/documentation/versions.md#remove-a-page).
+- [Update the integration API documentation](../../api/integrations.md).
+
+In the next minor release (M.1):
+
+- Remove the integration's model and any remaining code.
+- Close any issues, merge requests, and epics that have the integration's label (`~Integration::<name>`).
+- Delete the integration's label (`~Integration::<name>`) from `gitlab-org`.
+
 ## Ongoing migrations and refactorings
 
 Developers should be aware that the Integrations team is in the process of
diff --git a/doc/development/integrations/jenkins.md b/doc/development/integrations/jenkins.md
index 43487486f97..65194a04a62 100644
--- a/doc/development/integrations/jenkins.md
+++ b/doc/development/integrations/jenkins.md
@@ -24,7 +24,8 @@ brew services start jenkins
 GitLab does not allow requests to localhost or the local network by default. When running Jenkins on your local machine, you need to enable local access.
 
 1. Log into your GitLab instance as an administrator.
-1. On the top bar, select **Main menu > Admin**.
+1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+1. Select **Admin Area**.
 1. On the left sidebar, select **Settings > Network**.
 1. Expand **Outbound requests**, and select the following checkboxes:
 
diff --git a/doc/development/internal_analytics/index.md b/doc/development/internal_analytics/index.md
new file mode 100644
index 00000000000..8abea4c2b2f
--- /dev/null
+++ b/doc/development/internal_analytics/index.md
@@ -0,0 +1,12 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Internal analytics
+
+Learn how to implement internal analytics using:
+
+- [Service Ping](service_ping/index.md)
+- [Snowplow](snowplow/index.md)
diff --git a/doc/development/internal_analytics/service_ping/implement.md b/doc/development/internal_analytics/service_ping/implement.md
new file mode 100644
index 00000000000..0dfc3806712
--- /dev/null
+++ b/doc/development/internal_analytics/service_ping/implement.md
@@ -0,0 +1,882 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Implement Service Ping
+
+Service Ping consists of two kinds of data:
+
+- **Counters**: Track how often a certain event happened over time, such as how many CI/CD pipelines have run.
+  They are monotonic and usually trend up.
+- **Observations**: Facts collected from one or more GitLab instances and can carry arbitrary data.
+  There are no general guidelines for how to collect those, due to the individual nature of that data.
+
+To implement a new metric in Service Ping, follow these steps:
+
+1. [Implement the required counter](#types-of-counters)
+1. [Name and place the metric](metrics_dictionary.md#metric-key_path)
+1. [Test counters manually using your Rails console](#test-counters-manually-using-your-rails-console)
+1. [Generate the SQL query](#generate-the-sql-query)
+1. [Optimize queries with Database Lab](#optimize-queries-with-database-lab)
+1. [Add the metric definition to the Metrics Dictionary](#add-the-metric-definition)
+1. [Add the metric to the Versions Application](#add-the-metric-to-the-versions-application)
+1. [Create a merge request](#create-a-merge-request)
+1. [Verify your metric](#verify-your-metric)
+1. [Set up and test Service Ping locally](#set-up-and-test-service-ping-locally)
+
+## Instrumentation classes
+
+NOTE:
+Implementing metrics directly in `usage_data.rb` is deprecated.
+When you add or change a Service Ping Metric, you must migrate metrics to [instrumentation classes](metrics_instrumentation.md).
+For information about the progress on migrating Service Ping metrics, see this [epic](https://gitlab.com/groups/gitlab-org/-/epics/5547).
+
+For example, we have the following instrumentation class:
+`lib/gitlab/usage/metrics/instrumentations/count_boards_metric.rb`.
+
+You should add it to `usage_data.rb` as follows:
+
+```ruby
+boards: add_metric('CountBoardsMetric', time_frame: 'all'),
+```
+
+## Types of counters
+
+There are several types of counters for metrics:
+
+- **[Batch counters](#batch-counters)**: Used for counts, sums, and averages.
+- **[Redis counters](#redis-counters):** Used for in-memory counts.
+- **[Alternative counters](#alternative-counters):** Used for settings and configurations.
+
+NOTE:
+Only use the provided counter methods. Each counter method contains a built-in fail-safe mechanism that isolates each counter to avoid breaking the entire Service Ping process.
+
+### Batch counters
+
+For large tables, PostgreSQL can take a long time to count rows due to MVCC [(Multi-version Concurrency Control)](https://en.wikipedia.org/wiki/Multiversion_concurrency_control). Batch counting is a counting method where a single large query is broken into multiple smaller queries. For example, instead of a single query querying 1,000,000 records, with batch counting, you can execute 100 queries of 10,000 records each. Batch counting is useful for avoiding database timeouts as each batch query is significantly shorter than one single long running query.
+
+For GitLab.com, there are extremely large tables with 15 second query timeouts, so we use batch counting to avoid encountering timeouts. Here are the sizes of some GitLab.com tables:
+
+| Table                        | Row counts in millions |
+|------------------------------|------------------------|
+| `merge_request_diff_commits` | 2280                   |
+| `ci_build_trace_sections`    | 1764                   |
+| `merge_request_diff_files`   | 1082                   |
+| `events`                     | 514                    |
+
+Batch counting requires indexes on columns to calculate max, min, and range queries. In some cases,
+you must add a specialized index on the columns involved in a counter.
+
+#### Ordinary batch counters
+
+Create a new [database metrics](metrics_instrumentation.md#database-metrics) instrumentation class with `count` operation for a given `ActiveRecord_Relation`
+
+Method:
+
+```ruby
+add_metric('CountIssuesMetric', time_frame: 'all')
+```
+
+Examples:
+
+Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md).
+
+#### Distinct batch counters
+
+Create a new [database metrics](metrics_instrumentation.md#database-metrics) instrumentation class with `distinct_count` operation for a given `ActiveRecord_Relation`.
+
+Method:
+
+```ruby
+add_metric('CountUsersAssociatingMilestonesToReleasesMetric', time_frame: 'all')
+```
+
+WARNING:
+Counting over non-unique columns can lead to performance issues. For more information, see the [iterating tables in batches](../../database/iterating_tables_in_batches.md) guide.
+
+Examples:
+
+Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md).
+
+#### Sum batch operation
+
+Sum the values of a given ActiveRecord_Relation on given column and handles errors.
+Handles the `ActiveRecord::StatementInvalid` error
+
+Method:
+
+```ruby
+add_metric('JiraImportsTotalImportedIssuesCountMetric')
+```
+
+#### Average batch operation
+
+Average the values of a given `ActiveRecord_Relation` on given column and handles errors.
+
+Method:
+
+```ruby
+add_metric('CountIssuesWeightAverageMetric')
+```
+
+Examples:
+
+Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md).
+
+#### Grouping and batch operations
+
+The `count`, `distinct_count` and `sum` batch counters can accept an `ActiveRecord::Relation`
+object, which groups by a specified column. With a grouped relation, the methods do batch counting,
+handle errors, and returns a hash table of key-value pairs.
+
+Examples:
+
+```ruby
+count(Namespace.group(:type))
+# returns => {nil=>179, "Group"=>54}
+
+distinct_count(Project.group(:visibility_level), :creator_id)
+# returns => {0=>1, 10=>1, 20=>11}
+
+sum(Issue.group(:state_id), :weight))
+# returns => {1=>3542, 2=>6820}
+```
+
+#### Add operation
+
+Sum the values given as parameters. Handles the `StandardError`.
+Returns `-1` if any of the arguments are `-1`.
+
+Method:
+
+```ruby
+add(*args)
+```
+
+Examples:
+
+```ruby
+project_imports = distinct_count(::Project.where.not(import_type: nil), :creator_id)
+bulk_imports = distinct_count(::BulkImport, :user_id)
+
+ add(project_imports, bulk_imports)
+```
+
+#### Estimated batch counters
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/48233) in GitLab 13.7.
+
+Estimated batch counter functionality handles `ActiveRecord::StatementInvalid` errors
+when used through the provided `estimate_batch_distinct_count` method.
+Errors return a value of `-1`.
+
+WARNING:
+This functionality estimates a distinct count of a specific ActiveRecord_Relation in a given column,
+which uses the [HyperLogLog](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40671.pdf) algorithm.
+As the HyperLogLog algorithm is probabilistic, the **results always include error**.
+The highest encountered error rate is 4.9%.
+
+When correctly used, the `estimate_batch_distinct_count` method enables efficient counting over
+columns that contain non-unique values, which cannot be assured by other counters.
+
+##### `estimate_batch_distinct_count` method
+
+Method:
+
+```ruby
+estimate_batch_distinct_count(relation, column = nil, batch_size: nil, start: nil, finish: nil)
+```
+
+The [method](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/utils/usage_data.rb#L63)
+includes the following arguments:
+
+- `relation`: The ActiveRecord_Relation to perform the count.
+- `column`: The column to perform the distinct count. The default is the primary key.
+- `batch_size`: From `Gitlab::Database::PostgresHll::BatchDistinctCounter::DEFAULT_BATCH_SIZE`. Default value: 10,000.
+- `start`: The custom start of the batch count, to avoid complex minimum calculations.
+- `finish`: The custom end of the batch count to avoid complex maximum calculations.
+
+The method includes the following prerequisites:
+
+- The supplied `relation` must include the primary key defined as the numeric column.
+  For example: `id bigint NOT NULL`.
+- The `estimate_batch_distinct_count` can handle a joined relation. To use its ability to
+  count non-unique columns, the joined relation **must not** have a one-to-many relationship,
+  such as `has_many :boards`.
+- Both `start` and `finish` arguments should always represent primary key relationship values,
+  even if the estimated count refers to another column, for example:
+
+  ```ruby
+    estimate_batch_distinct_count(::Note, :author_id, start: ::Note.minimum(:id), finish: ::Note.maximum(:id))
+  ```
+
+Examples:
+
+1. Simple execution of estimated batch counter, with only relation provided,
+   returned value represents estimated number of unique values in `id` column
+   (which is the primary key) of `Project` relation:
+
+   ```ruby
+     estimate_batch_distinct_count(::Project)
+   ```
+
+1. Execution of estimated batch counter, where provided relation has applied
+   additional filter (`.where(time_period)`), number of unique values estimated
+   in custom column (`:author_id`), and parameters: `start` and `finish` together
+   apply boundaries that defines range of provided relation to analyze:
+
+   ```ruby
+     estimate_batch_distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::Note.minimum(:id), finish: ::Note.maximum(:id))
+   ```
+
+When instrumenting metric with usage of estimated batch counter please add
+`_estimated` suffix to its name, for example:
+
+```ruby
+  "counts": {
+    "ci_builds_estimated": estimate_batch_distinct_count(Ci::Build),
+    ...
+```
+
+### Redis counters
+
+Handles `::Redis::CommandError` and `Gitlab::UsageDataCounters::BaseCounter::UnknownEvent`.
+Returns -1 when a block is sent or hash with all values and -1 when a `counter(Gitlab::UsageDataCounters)` is sent.
+The different behavior is due to 2 different implementations of the Redis counter.
+
+Method:
+
+```ruby
+redis_usage_data(counter, &block)
+```
+
+Arguments:
+
+- `counter`: a counter from `Gitlab::UsageDataCounters`, that has `fallback_totals` method implemented
+- or a `block`: which is evaluated
+
+#### Ordinary Redis counters
+
+Example of implementation: [`Gitlab::UsageDataCounters::WikiPageCounter`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/wiki_page_counter.rb), using Redis methods [`INCR`](https://redis.io/commands/incr/) and [`GET`](https://redis.io/commands/get/).
+
+Events are handled by counter classes in the `Gitlab::UsageDataCounters` namespace, inheriting from `BaseCounter`, that are either:
+
+1. Listed in [`Gitlab::UsageDataCounters::COUNTERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters.rb#L5) to be then included in `Gitlab::UsageData`.
+
+1. Specified in the metric definition using the `RedisMetric` instrumentation class by their `prefix` option to be picked up using the [metric instrumentation](metrics_instrumentation.md) framework. Refer to the [Redis metrics](metrics_instrumentation.md#redis-metrics) documentation for an example implementation.
+
+Inheriting classes are expected to override `KNOWN_EVENTS` and `PREFIX` constants to build event names and associated metrics. For example, for prefix `issues` and events array `%w[create, update, delete]`, three metrics will be added to the Service Ping payload: `counts.issues_create`, `counts.issues_update` and `counts.issues_delete`.
+
+##### `UsageData` API
+
+You can use the `UsageData` API to track events.
+To track events, the `usage_data_api` feature flag must
+be enabled (set to `default_enabled: true`).
+Enabled by default in GitLab 13.7 and later.
+
+##### UsageData API tracking
+
+1. Track events using the [`UsageData` API](#usagedata-api).
+
+   Increment event count using an ordinary Redis counter, for a given event name.
+
+   API requests are protected by checking for a valid CSRF token.
+
+   ```plaintext
+   POST /usage_data/increment_counter
+   ```
+
+   | Attribute | Type | Required | Description |
+   | :-------- | :--- | :------- | :---------- |
+   | `event` | string | yes | The event name to track. |
+
+   Response:
+
+   - `200` if the event was tracked.
+   - `400 Bad request` if the event parameter is missing.
+   - `401 Unauthorized` if the user is not authenticated.
+   - `403 Forbidden` if an invalid CSRF token is provided.
+
+1. Track events using the JavaScript/Vue API helper which calls the [`UsageData` API](#usagedata-api).
+
+   To track events, `usage_data_api` and `usage_data_#{event_name}` must be enabled.
+
+   ```javascript
+   import api from '~/api';
+
+   api.trackRedisCounterEvent('my_already_defined_event_name'),
+   ```
+
+#### Redis HLL counters
+
+WARNING:
+HyperLogLog (HLL) is a probabilistic algorithm and its **results always includes some small error**. According to [Redis documentation](https://redis.io/commands/pfcount/), data from
+used HLL implementation is "approximated with a standard error of 0.81%".
+
+NOTE:
+ A user's consent for `usage_stats` (`User.single_user&.requires_usage_stats_consent?`) is not checked during the data tracking stage due to performance reasons. Keys corresponding to those counters are present in Redis even if `usage_stats_consent` is still required. However, no metric is collected from Redis and reported back to GitLab as long as `usage_stats_consent` is required.
+
+With `Gitlab::UsageDataCounters::HLLRedisCounter` we have available data structures used to count unique values.
+
+Implemented using Redis methods [PFADD](https://redis.io/commands/pfadd/) and [PFCOUNT](https://redis.io/commands/pfcount/).
+
+##### Add new events
+
+1. Define events in [`known_events`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/).
+
+   Example event:
+
+   ```yaml
+   - name: users_creating_epics
+     aggregation: weekly
+   ```
+
+   Keys:
+
+   - `name`: unique event name.
+
+     Name format for Redis HLL events `{hll_counters}_<name>`
+
+     Example names: `users_creating_epics`, `users_triggering_security_scans`.
+
+   - `aggregation`: may be set to a `:daily` or `:weekly` key. Defines how counting data is stored in Redis.
+     Aggregation on a `daily` basis does not pull more fine grained data.
+
+1. Use one of the following methods to track the event:
+
+   - In the controller using the `ProductAnalyticsTracking` module and the following format:
+
+     ```ruby
+     track_event(*controller_actions, name:, action:, label:, conditions: nil, destinations: [:redis_hll], &block)
+     ```
+
+     Arguments:
+
+     - `controller_actions`: the controller actions to track.
+     - `name`: the event name.
+     - `action`: required if destination is `:snowplow. Action name for the triggered event. See [event schema](../snowplow/index.md#event-schema) for more details.
+     - `label`: required if destination is `:snowplow. Label for the triggered event. See [event schema](../snowplow/index.md#event-schema) for more details.
+     - `conditions`: optional custom conditions. Uses the same format as Rails callbacks.
+     - `destinations`: optional list of destinations. Currently supports `:redis_hll` and `:snowplow`. Default: `:redis_hll`.
+     - `&block`: optional block that computes and returns the `custom_id` that we want to track. This overrides the `visitor_id`.
+
+     Example:
+
+     ```ruby
+     # controller
+     class ProjectsController < Projects::ApplicationController
+       include ProductAnalyticsTracking
+
+       skip_before_action :authenticate_user!, only: :show
+       track_event :index, :show,
+         name: 'users_visiting_projects',
+         action: 'user_perform_visit',
+         label: 'redis_hll_counters.users_visiting_project_monthly',
+         destinations: %i[redis_hll snowplow]
+
+       def index
+         render html: 'index'
+       end
+
+      def new
+        render html: 'new'
+      end
+
+      def show
+        render html: 'show'
+      end
+     end
+     ```
+
+   - In the API using the `increment_unique_values(event_name, values)` helper method.
+
+     Arguments:
+
+     - `event_name`: the event name.
+     - `values`: the values counted. Can be one value or an array of values.
+
+     Example:
+
+     ```ruby
+     get ':id/registry/repositories' do
+       repositories = ContainerRepositoriesFinder.new(
+         user: current_user, subject: user_group
+       ).execute
+
+       increment_unique_values('users_listing_repositories', current_user.id)
+
+       present paginate(repositories), with: Entities::ContainerRegistry::Repository, tags: params[:tags], tags_count: params[:tags_count]
+     end
+     ```
+
+   - Using `track_usage_event(event_name, values)` in services and GraphQL.
+
+     Increment unique values count using Redis HLL, for a given event name.
+
+     Examples:
+
+     - [Track usage event for an incident in a service](https://gitlab.com/gitlab-org/gitlab/-/blob/v13.8.3-ee/app/services/issues/update_service.rb#L66)
+     - [Track usage event for an incident in GraphQL](https://gitlab.com/gitlab-org/gitlab/-/blob/v13.8.3-ee/app/graphql/mutations/alert_management/update_alert_status.rb#L16)
+
+     ```ruby
+       track_usage_event(:incident_management_incident_created, current_user.id)
+     ```
+
+   - Using the [`UsageData` API](#usagedata-api).
+
+     Increment unique users count using Redis HLL, for a given event name.
+
+     API requests are protected by checking for a valid CSRF token.
+
+     ```plaintext
+     POST /usage_data/increment_unique_users
+     ```
+
+     | Attribute | Type | Required | Description |
+     | :-------- | :--- | :------- | :---------- |
+     | `event` | string | yes | The event name to track |
+
+     Response:
+
+     - `200` if the event was tracked, or if tracking failed for any reason.
+     - `400 Bad request` if an event parameter is missing.
+     - `401 Unauthorized` if the user is not authenticated.
+     - `403 Forbidden` if an invalid CSRF token is provided.
+
+   - Using the JavaScript/Vue API helper, which calls the [`UsageData` API](#usagedata-api).
+
+     Example for an existing event already defined in [known events](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/):
+
+     ```javascript
+     import api from '~/api';
+
+     api.trackRedisHllUserEvent('my_already_defined_event_name'),
+     ```
+
+1. Get event data using `Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names:, start_date:, end_date:, context: '')`.
+
+   Arguments:
+
+   - `event_names`: the list of event names.
+   - `start_date`: start date of the period for which we want to get event data.
+   - `end_date`: end date of the period for which we want to get event data.
+   - `context`: context of the event. Allowed values are `default`, `free`, `bronze`, `silver`, `gold`, `starter`, `premium`, `ultimate`.
+
+1. Testing tracking and getting unique events
+
+Trigger events in rails console by using `track_event` method
+
+   ```ruby
+   Gitlab::UsageDataCounters::HLLRedisCounter.track_event('users_viewing_compliance_audit_events', values: 1)
+   Gitlab::UsageDataCounters::HLLRedisCounter.track_event('users_viewing_compliance_audit_events', values: [2, 3])
+   ```
+
+Next, get the unique events for the current week.
+
+   ```ruby
+   # Get unique events for metric for current_week
+   Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'users_viewing_compliance_audit_events',
+   start_date: Date.current.beginning_of_week, end_date: Date.current.next_week)
+   ```
+
+##### Recommendations
+
+We have the following recommendations for [adding new events](#add-new-events):
+
+- Event aggregation: weekly.
+- When adding new metrics, use a [feature flag](../../../operations/feature_flags.md) to control the impact.
+It's recommended to disable the new feature flag by default (set `default_enabled: false`).
+- Events can be triggered using the `UsageData` API, which helps when there are > 10 events per change
+
+##### Enable or disable Redis HLL tracking
+
+We can disable tracking completely by using the global flag:
+
+```shell
+/chatops run feature set redis_hll_tracking true
+/chatops run feature set redis_hll_tracking false
+```
+
+##### Known events are added automatically in Service Data payload
+
+Service Ping adds all events [`known_events/*.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events) to Service Data generation under the `redis_hll_counters` key. This column is stored in [version-app as a JSON](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/db/schema.rb#L209).
+For each event we add metrics for the weekly and monthly time frames, and totals for each where applicable:
+
+- `#{event_name}_weekly`: Data for 7 days for daily [aggregation](#add-new-events) events and data for the last complete week for weekly [aggregation](#add-new-events) events.
+- `#{event_name}_monthly`: Data for 28 days for daily [aggregation](#add-new-events) events and data for the last 4 complete weeks for weekly [aggregation](#add-new-events) events.
+
+Example of `redis_hll_counters` data:
+
+```ruby
+{:redis_hll_counters=>
+  {"compliance"=>
+    {"users_viewing_compliance_dashboard_weekly"=>0,
+     "users_viewing_compliance_dashboard_monthly"=>0,
+     "users_viewing_compliance_audit_events_weekly"=>0,
+     "users_viewing_audit_events_monthly"=>0,
+     "compliance_total_unique_counts_weekly"=>0,
+     "compliance_total_unique_counts_monthly"=>0},
+ "analytics"=>
+    {"users_viewing_analytics_group_devops_adoption_weekly"=>0,
+     "users_viewing_analytics_group_devops_adoption_monthly"=>0,
+     "analytics_total_unique_counts_weekly"=>0,
+     "analytics_total_unique_counts_monthly"=>0},
+   "ide_edit"=>
+    {"users_editing_by_web_ide_weekly"=>0,
+     "users_editing_by_web_ide_monthly"=>0,
+     "users_editing_by_sfe_weekly"=>0,
+     "users_editing_by_sfe_monthly"=>0,
+     "ide_edit_total_unique_counts_weekly"=>0,
+     "ide_edit_total_unique_counts_monthly"=>0}
+ }
+}
+```
+
+Example:
+
+```ruby
+# Redis Counters
+redis_usage_data(Gitlab::UsageDataCounters::WikiPageCounter)
+
+# Define events in common.yml https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/common.yml
+
+# Tracking events
+Gitlab::UsageDataCounters::HLLRedisCounter.track_event('users_expanding_vulnerabilities', values: visitor_id)
+
+# Get unique events for metric
+redis_usage_data { Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'users_expanding_vulnerabilities', start_date: 28.days.ago, end_date: Date.current) }
+```
+
+### Alternative counters
+
+Handles `StandardError` and fallbacks into -1 this way not all measures fail if we encounter one exception.
+Mainly used for settings and configurations.
+
+Method:
+
+```ruby
+alt_usage_data(value = nil, fallback: -1, &block)
+```
+
+Arguments:
+
+- `value`: a static value in which case the value is returned.
+- or a `block`: which is evaluated
+- `fallback: -1`: the common value used for any metrics that are failing.
+
+Example:
+
+```ruby
+alt_usage_data { Gitlab::VERSION }
+alt_usage_data { Gitlab::CurrentSettings.uuid }
+alt_usage_data(999)
+```
+
+### Add counters to build new metrics
+
+When adding the results of two counters, use the `add` Service Data method that
+handles fallback values and exceptions. It also generates a valid [SQL export](index.md#export-service-ping-data).
+
+Example:
+
+```ruby
+add(User.active, User.bot)
+```
+
+### Prometheus queries
+
+In those cases where operational metrics should be part of Service Ping, a database or Redis query is unlikely
+to provide useful data. Instead, Prometheus might be more appropriate, because most GitLab architectural
+components publish metrics to it that can be queried back, aggregated, and included as Service Data.
+
+NOTE:
+Prometheus as a data source for Service Ping is only available for single-node Omnibus installations
+that are running the [bundled Prometheus](../../../administration/monitoring/prometheus/index.md) instance.
+
+To query Prometheus for metrics, a helper method is available to `yield` a fully configured
+`PrometheusClient`, given it is available as per the note above:
+
+```ruby
+with_prometheus_client do |client|
+  response = client.query('<your query>')
+  ...
+end
+```
+
+Refer to [the `PrometheusClient` definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/prometheus_client.rb)
+for how to use its API to query for data.
+
+### Fallback values for Service Ping
+
+We return fallback values in these cases:
+
+| Case                        | Value |
+|-----------------------------|-------|
+| Deprecated Metric ([Removed with version 14.3](https://gitlab.com/gitlab-org/gitlab/-/issues/335894)) | -1000 |
+| Timeouts, general failures  | -1    |
+| Standard errors in counters | -2    |
+| Histogram metrics failure   | { '-1' => -1 } |
+
+## Test counters manually using your Rails console
+
+```ruby
+# count
+Gitlab::UsageData.count(User.active)
+Gitlab::UsageData.count(::Clusters::Cluster.aws_installed.enabled, :cluster_id)
+
+# count distinct
+Gitlab::UsageData.distinct_count(::Project, :creator_id)
+Gitlab::UsageData.distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id))
+```
+
+## Generate the SQL query
+
+Your Rails console returns the generated SQL queries. For example:
+
+```ruby
+pry(main)> Gitlab::UsageData.count(User.active)
+   (2.6ms)  SELECT "features"."key" FROM "features"
+   (15.3ms)  SELECT MIN("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4))
+   (2.4ms)  SELECT MAX("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4))
+   (1.9ms)  SELECT COUNT("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4)) AND "users"."id" BETWEEN 1 AND 100000
+```
+
+## Optimize queries with Database Lab
+
+[Database Lab](../../database/database_lab.md) is a service that uses a production clone to test queries.
+
+- GitLab.com's production database has a 15 second timeout.
+- Any single query must stay below the [1 second execution time](../../database/query_performance.md#timing-guidelines-for-queries) with cold caches.
+- Add a specialized index on columns involved to reduce the execution time.
+
+To understand the query's execution, we add the following information
+to a merge request description:
+
+- For counters that have a `time_period` test, we add information for both:
+  - `time_period = {}` for all time periods.
+  - `time_period = { created_at: 28.days.ago..Time.current }` for the last 28 days.
+- Execution plan and query time before and after optimization.
+- Query generated for the index and time.
+- Migration output for up and down execution.
+
+For more details, see the [database review guide](../../database_review.md#preparation-when-adding-or-modifying-queries).
+
+### Optimization recommendations and examples
+
+- Use specialized indexes. For examples, see these merge requests:
+  - [Example 1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26871)
+  - [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26445)
+- Use defined `start` and `finish`. These values can be memoized and reused, as in this
+  [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37155).
+- Avoid joins and unnecessary complexity in your queries. See this
+  [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36316) as an example.
+- Set a custom `batch_size` for `distinct_count`, as in this [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38000).
+
+## Add the metric definition
+
+See the [Metrics Dictionary guide](metrics_dictionary.md) for more information.
+
+## Add the metric to the Versions Application
+
+Check if the new metric must be added to the Versions Application. See the `usage_data` [schema](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/db/schema.rb#L147) and Service Data [parameters accepted](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/app/services/usage_ping.rb). Any metrics added under the `counts` key are saved in the `stats` column.
+
+## Create a merge request
+
+Create a merge request for the new Service Ping metric, and do the following:
+
+- Add the `feature` label to the merge request. A metric is a user-facing change and is part of expanding the Service Ping feature.
+- Add a changelog entry that complies with the [changelog entries guide](../../changelog.md).
+- Ask for an Analytics Instrumentation review.
+  On GitLab.com, we have DangerBot set up to monitor Analytics Instrumentation related files and recommend a [Analytics Instrumentation review](review_guidelines.md).
+
+## Verify your metric
+
+On GitLab.com, the Product Intelligence team regularly [monitors Service Ping](https://gitlab.com/groups/gitlab-org/-/epics/6000).
+They may alert you that your metrics need further optimization to run quicker and with greater success.
+
+The Service Ping JSON payload for GitLab.com is shared in the
+[#g_product_intelligence](https://gitlab.slack.com/archives/CL3A7GFPF) Slack channel every week.
+
+You may also use the [Service Ping QA dashboard](https://app.periscopedata.com/app/gitlab/632033/Usage-Ping-QA) to check how well your metric performs.
+The dashboard allows filtering by GitLab version, by "Self-managed" and "SaaS", and shows you how many failures have occurred for each metric. Whenever you notice a high failure rate, you can re-optimize your metric.
+
+Use [Metrics Dictionary](https://metrics.gitlab.com/) [copy query to clipboard feature](https://www.youtube.com/watch?v=n4o65ivta48&list=PL05JrBw4t0Krg3mbR6chU7pXtMt_es6Pb) to get a query ready to run in Sisense for a specific metric.
+
+## Set up and test Service Ping locally
+
+To set up Service Ping locally, you must:
+
+1. [Set up local repositories](#set-up-local-repositories).
+1. [Test local setup](#test-local-setup).
+1. Optional. [Test Prometheus-based Service Ping](#test-prometheus-based-service-ping).
+
+### Set up local repositories
+
+1. Clone and start [GitLab](https://gitlab.com/gitlab-org/gitlab-development-kit).
+1. Clone and start [Versions Application](https://gitlab.com/gitlab-services/version-gitlab-com).
+   Make sure you run `docker-compose up` to start a PostgreSQL and Redis instance.
+1. Point GitLab to the Versions Application endpoint instead of the default endpoint:
+   1. Open [service_ping/submit_service.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb#L5) locally and modify `STAGING_BASE_URL`.
+   1. Set it to the local Versions Application URL: `http://localhost:3000`.
+
+### Test local setup
+
+1. Using the `gitlab` Rails console, manually trigger Service Ping:
+
+   ```ruby
+   GitlabServicePingWorker.new.perform('triggered_from_cron' => false)
+   ```
+
+1. Use the `versions` Rails console to check the Service Ping was successfully received,
+   parsed, and stored in the Versions database:
+
+   ```ruby
+   UsageData.last
+   ```
+
+## Test Prometheus-based Service Ping
+
+If the data submitted includes metrics [queried from Prometheus](#prometheus-queries)
+you want to inspect and verify, you must:
+
+- Ensure that a Prometheus server is running locally.
+- Ensure the respective GitLab components are exporting metrics to the Prometheus server.
+
+If you do not need to test data coming from Prometheus, no further action
+is necessary. Service Ping should degrade gracefully in the absence of a running Prometheus server.
+
+Three kinds of components may export data to Prometheus, and are included in Service Ping:
+
+- [`node_exporter`](https://github.com/prometheus/node_exporter): Exports node metrics
+  from the host machine.
+- [`gitlab-exporter`](https://gitlab.com/gitlab-org/gitlab-exporter): Exports process metrics
+  from various GitLab components.
+- Other various GitLab services, such as Sidekiq and the Rails server, which export their own metrics.
+
+### Test with an Omnibus container
+
+This is the recommended approach to test Prometheus-based Service Ping.
+
+To verify your change, build a new Omnibus image from your code branch using CI/CD, download the image,
+and run a local container instance:
+
+1. From your merge request, select the `qa` stage, then trigger the `e2e:package-and-test` job. This job triggers an Omnibus
+   build in a [downstream pipeline of the `omnibus-gitlab-mirror` project](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/-/pipelines).
+1. In the downstream pipeline, wait for the `gitlab-docker` job to finish.
+1. Open the job logs and locate the full container name including the version. It takes the following form: `registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>`.
+1. On your local machine, make sure you are signed in to the GitLab Docker registry. You can find the instructions for this in
+   [Authenticate to the GitLab Container Registry](../../../user/packages/container_registry/authenticate_with_container_registry.md).
+1. Once signed in, download the new image by using `docker pull registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>`
+1. For more information about working with and running Omnibus GitLab containers in Docker, refer to [GitLab Docker images](../../../install/docker.md) documentation.
+
+### Test with GitLab development toolkits
+
+This is the less recommended approach, because it comes with a number of difficulties when emulating a real GitLab deployment.
+
+The [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) is not set up to run a Prometheus server or `node_exporter` alongside other GitLab components. If you would
+like to do so, [Monitoring the GDK with Prometheus](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/prometheus/index.md#monitoring-the-gdk-with-prometheus) is a good start.
+
+The [GCK](https://gitlab.com/gitlab-org/gitlab-compose-kit) has limited support for testing Prometheus based Service Ping.
+By default, it comes with a fully configured Prometheus service that is set up to scrape a number of components.
+However, it has the following limitations:
+
+- It does not run a `gitlab-exporter` instance, so several `process_*` metrics from services such as Gitaly may be missing.
+- While it runs a `node_exporter`, `docker-compose` services emulate hosts, meaning that it normally reports itself as not associated
+  with any of the other running services. That is not how node metrics are reported in a production setup, where `node_exporter`
+  always runs as a process alongside other GitLab components on any given node. For Service Ping, none of the node data would therefore
+  appear to be associated to any of the services running, because they all appear to be running on different hosts. To alleviate this problem, the `node_exporter` in GCK was arbitrarily "assigned" to the `web` service, meaning only for this service `node_*` metrics appears in Service Ping.
+
+## Aggregated metrics
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/45979) in GitLab 13.6.
+
+WARNING:
+This feature is intended solely for internal GitLab use.
+
+The aggregated metrics feature provides insight into the data attributes in a collection of Service Ping metrics.
+This aggregation allows you to count data attributes in events without counting each occurrence of the same data attribute in multiple events.
+For example, you can aggregate the number of users who perform several actions, such as creating a new issue and opening a new merge request.
+You can then count each user that performed any combination of these actions.
+
+### Defining aggregated metric via metric YAML definition
+
+To add data for aggregated metrics to the Service Ping payload,
+create metric YAML definition file following [Aggregated metric instrumentation guide](metrics_instrumentation.md#aggregated-metrics).
+
+### Redis sourced aggregated metrics
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/45979) in GitLab 13.6.
+
+To declare the aggregate of events collected with [Redis HLL Counters](#redis-hll-counters),
+you must fulfill the following requirements:
+
+1. All events listed at `events` attribute must come from
+   [`known_events/*.yml`](#known-events-are-added-automatically-in-service-data-payload) files.
+1. All events listed at `events` attribute must have the same `aggregation` attribute.
+1. `time_frame` does not include `all` value, which is unavailable for Redis sourced aggregated metrics.
+
+While it is possible to aggregate EE-only events together with events that occur in all GitLab editions, it's important to remember that doing so may produce high variance between data collected from EE and CE GitLab instances.
+
+### Database sourced aggregated metrics
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52784) in GitLab 13.9.
+
+To declare an aggregate of metrics based on events collected from database, follow
+these steps:
+
+1. [Persist the metrics for aggregation](#persist-metrics-for-aggregation).
+1. [Add new aggregated metric definition](#add-new-aggregated-metric-definition).
+
+#### Persist metrics for aggregation
+
+Only metrics calculated with [Estimated Batch Counters](#estimated-batch-counters)
+can be persisted for database sourced aggregated metrics. To persist a metric,
+inject a Ruby block into the
+[`estimate_batch_distinct_count`](#estimate_batch_distinct_count-method) method.
+This block should invoke the
+`Gitlab::Usage::Metrics::Aggregates::Sources::PostgresHll.save_aggregated_metrics`
+[method](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage/metrics/aggregates/sources/postgres_hll.rb#L21),
+which stores `estimate_batch_distinct_count` results for future use in aggregated metrics.
+
+The `Gitlab::Usage::Metrics::Aggregates::Sources::PostgresHll.save_aggregated_metrics`
+method accepts the following arguments:
+
+- `metric_name`: The name of metric to use for aggregations. Should be the same
+  as the key under which the metric is added into Service Ping.
+- `recorded_at_timestamp`: The timestamp representing the moment when a given
+  Service Ping payload was collected. You should use the convenience method `recorded_at`
+  to fill `recorded_at_timestamp` argument, like this: `recorded_at_timestamp: recorded_at`
+- `time_period`: The time period used to build the `relation` argument passed into
+  `estimate_batch_distinct_count`. To collect the metric with all available historical
+  data, set a `nil` value as time period: `time_period: nil`.
+- `data`: HyperLogLog buckets structure representing unique entries in `relation`.
+  The `estimate_batch_distinct_count` method always passes the correct argument
+  into the block, so `data` argument must always have a value equal to block argument,
+  like this: `data: result`
+
+Example metrics persistence:
+
+```ruby
+class UsageData
+  def count_secure_pipelines(time_period)
+    ...
+    relation = ::Security::Scan.by_scan_types(scan_type).where(time_period)
+
+    pipelines_with_secure_jobs['dependency_scanning_pipeline'] = estimate_batch_distinct_count(relation, :pipeline_id, batch_size: 1000, start: start_id, finish: finish_id) do |result|
+      ::Gitlab::Usage::Metrics::Aggregates::Sources::PostgresHll
+        .save_aggregated_metrics(metric_name: 'dependency_scanning_pipeline', recorded_at_timestamp: recorded_at, time_period: time_period, data: result)
+    end
+  end
+end
+```
+
+#### Add new aggregated metric definition
+
+After all metrics are persisted, you can add an aggregated metric definition following [Aggregated metric instrumentation guide](metrics_instrumentation.md#aggregated-metrics).
+To declare the aggregate of metrics collected with [Estimated Batch Counters](#estimated-batch-counters),
+you must fulfill the following requirements:
+
+- Metrics names listed in the `events:` attribute, have to use the same names you passed in the `metric_name` argument while persisting metrics in previous step.
+- Every metric listed in the `events:` attribute, has to be persisted for **every** selected `time_frame:` value.
diff --git a/doc/development/internal_analytics/service_ping/index.md b/doc/development/internal_analytics/service_ping/index.md
new file mode 100644
index 00000000000..69d37f0dae2
--- /dev/null
+++ b/doc/development/internal_analytics/service_ping/index.md
@@ -0,0 +1,509 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Service Ping development guidelines
+
+> - Introduced in GitLab Ultimate 11.2, more statistics.
+> - In GitLab 14.1, [renamed from Usage Ping to Service Ping](https://gitlab.com/groups/gitlab-org/-/epics/5990). In 14.0 and earlier, use the Usage Ping documentation for the Rails commands appropriate to your version.
+
+Service Ping is a GitLab process that collects and sends a weekly payload to GitLab.
+The payload provides important high-level data that helps our product, support,
+and sales teams understand how GitLab is used. The data helps to:
+
+- Compare counts month over month (or week over week) to get a rough sense for how an instance uses
+  different product features.
+- Collect other facts that help us classify and understand GitLab installations.
+- Calculate our stage monthly active users (SMAU), which helps to measure the success of our stages
+  and features.
+
+Service Ping information is not anonymous. It's linked to the instance's hostname, but does
+not contain project names, usernames, or any other specific data.
+
+Service Ping is enabled by default. However, you can [disable](../../../user/admin_area/settings/usage_statistics.md#enable-or-disable-usage-statistics) it on any self-managed instance. When Service Ping is enabled, GitLab gathers data from the other instances and can show your instance's usage statistics to your users.
+
+## Service Ping terminology
+
+We use the following terminology to describe the Service Ping components:
+
+- **Service Ping**: the process that collects and generates a JSON payload.
+- **Service Data**: the contents of the Service Ping JSON payload. This includes metrics.
+- **Metrics**: primarily made up of row counts for different tables in an instance's database. Each
+  metric has a corresponding [metric definition](metrics_dictionary.md#metrics-definition-and-validation)
+  in a YAML file.
+- **MAU**: monthly active users.
+- **WAU**: weekly active users.
+
+### Limitations
+
+- Service Ping does not track frontend events things like page views, link clicks, or user sessions.
+- Service Ping focuses only on aggregated backend events.
+
+Because of these limitations we recommend you:
+
+- Instrument your products with Snowplow for more detailed analytics on GitLab.com.
+- Use Service Ping to track aggregated backend events on self-managed instances.
+
+## Service Ping request flow
+
+The following example shows a basic request/response flow between a GitLab instance, the Versions Application, the License Application, Salesforce, the GitLab S3 Bucket, the GitLab Snowflake Data Warehouse, and Sisense:
+
+```mermaid
+sequenceDiagram
+    participant GitLab Instance
+    participant Versions Application
+    participant Licenses Application
+    participant Salesforce
+    participant S3 Bucket
+    participant Snowflake DW
+    participant Sisense Dashboards
+    GitLab Instance->>Versions Application: Send Service Ping
+    loop Process usage data
+        Versions Application->>Versions Application: Parse usage data
+        Versions Application->>Versions Application: Write to database
+        Versions Application->>Versions Application: Update license ping time
+    end
+    loop Process data for Salesforce
+        Versions Application-xLicenses Application: Request Zuora subscription id
+        Licenses Application-xVersions Application: Zuora subscription id
+        Versions Application-xSalesforce: Request Zuora account id  by Zuora subscription id
+        Salesforce-xVersions Application: Zuora account id
+        Versions Application-xSalesforce: Usage data for the Zuora account
+    end
+    Versions Application->>S3 Bucket: Export Versions database
+    S3 Bucket->>Snowflake DW: Import data
+    Snowflake DW->>Snowflake DW: Transform data using dbt
+    Snowflake DW->>Sisense Dashboards: Data available for querying
+    Versions Application->>GitLab Instance: DevOps Score (Conversational Development Index)
+```
+
+## How Service Ping works
+
+1. The Service Ping [cron job](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/workers/gitlab_service_ping_worker.rb#L24) is set in Sidekiq to run weekly.
+1. When the cron job runs, it calls [`Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb).
+1. `Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)` [cascades down](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb) to ~400+ other counter method calls.
+1. The response of all methods calls are [merged together](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb#L68) into a single JSON payload.
+1. The JSON payload is then [posted to the Versions application](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb#L20)
+   If a firewall exception is needed, the required URL depends on several things. If
+   the hostname is `version.gitlab.com`, the protocol is `TCP`, and the port number is `443`,
+   the required URL is <https://version.gitlab.com/>.
+1. In case of an error, it will be reported to the Version application along with following pieces of information:
+
+   - `uuid` - GitLab instance unique identifier
+   - `hostname` - GitLab instance hostname
+   - `version` - GitLab instance current versions
+   - `elapsed` - Amount of time which passed since Service Ping report process started and moment of error occurrence
+   - `message` - Error message
+
+   <pre>
+   <code>
+   {
+     "uuid"=>"02333324-1cd7-4c3b-a45b-a4993f05fb1d",
+     "hostname"=>"127.0.0.1",
+     "version"=>"14.7.0-pre",
+     "elapsed"=>0.006946,
+     "message"=>'PG::UndefinedColumn: ERROR:  column \"non_existent_attribute\" does not exist\nLINE 1: SELECT COUNT(non_existent_attribute) FROM \"issues\" /*applica...'
+   }
+   </code>
+   </pre>
+
+1. Finally, the timing metadata information that is used for diagnostic purposes is submitted to the Versions application. It consists of a list of metric identifiers and the time it took to calculate the metrics:
+
+   > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37911) in GitLab 15.0 [with a flag](../../../user/feature_flags.md), enabled by default.
+   > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/295289) in GitLab 15.2. [Feature flag `measure_service_ping_metric_collection`](https://gitlab.com/gitlab-org/gitlab/-/issues/358128) removed.
+
+```ruby
+    {
+      "metadata"=>
+      {
+        "uuid"=>"0000000-0000-0000-0000-000000000000",
+        "metrics"=>
+        [{"name"=>"version", "time_elapsed"=>1.1811964213848114e-05},
+         {"name"=>"installation_type", "time_elapsed"=>0.00017242692410945892},
+         {"name"=>"license_billable_users", "time_elapsed"=>0.009520471096038818},
+         ....
+         {"name"=>"counts.clusters_platforms_eks",
+          "time_elapsed"=>0.05638605775311589},
+         {"name"=>"counts.clusters_platforms_gke",
+          "time_elapsed"=>0.40995341585949063},
+         {"name"=>"counts.clusters_platforms_user",
+          "time_elapsed"=>0.06410990096628666},
+         {"name"=>"counts.clusters_management_project",
+          "time_elapsed"=>0.24020783510059118}
+        ]
+      }
+    }
+```
+
+### On a Geo secondary site
+
+We also collect metrics specific to [Geo](../../../administration/geo/index.md) secondary sites to send with Service Ping.
+
+1. The [Geo secondary service ping cron job](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/workers/geo/secondary_usage_data_cron_worker.rb) is set in Sidekiq to run weekly.
+1. When the cron job runs, it calls [`SecondaryUsageData.update_metrics!`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/geo/secondary_usage_data.rb#L33). This collects the relevant metrics from Prometheus and stores the data in the Geo secondary tracking database for transmission to the primary site during a [Geo node status update](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/geo_node_status.rb#L105).
+1. Geo node status data is sent with the JSON payload in the process described above. The following is an example of the payload where each object in the array represents a Geo node:
+
+   ```json
+   [
+     {
+       "repository_verification_enabled"=>true,
+       "repositories_replication_enabled"=>true,
+       "repositories_synced_count"=>24,
+       "repositories_failed_count"=>0,
+       "git_fetch_event_count_weekly"=>nil,
+       "git_push_event_count_weekly"=>nil,
+       ... other geo node status fields
+     }
+   ]
+   ```
+
+## Implementing Service Ping
+
+See the [implement Service Ping](implement.md) guide.
+
+## Example Service Ping payload
+
+The following is example content of the Service Ping payload.
+
+```json
+{
+  "uuid": "0000000-0000-0000-0000-000000000000",
+  "hostname": "example.com",
+  "version": "12.10.0-pre",
+  "installation_type": "omnibus-gitlab",
+  "active_user_count": 999,
+  "recorded_at": "2020-04-17T07:43:54.162+00:00",
+  "edition": "EEU",
+  "license_md5": "00000000000000000000000000000000",
+  "license_sha256": "0000000000000000000000000000000000000000000000000000000000000000",
+  "license_id": null,
+  "historical_max_users": 999,
+  "licensee": {
+    "Name": "ABC, Inc.",
+    "Email": "email@example.com",
+    "Company": "ABC, Inc."
+  },
+  "license_user_count": 999,
+  "license_starts_at": "2020-01-01",
+  "license_expires_at": "2021-01-01",
+  "license_plan": "ultimate",
+  "license_add_ons": {
+  },
+  "license_trial": false,
+  "counts": {
+    "assignee_lists": 999,
+    "boards": 999,
+    "ci_builds": 999,
+    ...
+  },
+  "container_registry_enabled": true,
+  "dependency_proxy_enabled": false,
+  "gitlab_shared_runners_enabled": true,
+  "gravatar_enabled": true,
+  "influxdb_metrics_enabled": true,
+  "ldap_enabled": false,
+  "mattermost_enabled": false,
+  "omniauth_enabled": true,
+  "prometheus_enabled": false,
+  "prometheus_metrics_enabled": false,
+  "reply_by_email_enabled": "incoming+%{key}@incoming.gitlab.com",
+  "signup_enabled": true,
+  "projects_with_expiration_policy_disabled": 999,
+  "projects_with_expiration_policy_enabled": 999,
+  ...
+  "elasticsearch_enabled": true,
+  "license_trial_ends_on": null,
+  "geo_enabled": false,
+  "git": {
+    "version": {
+      "major": 2,
+      "minor": 26,
+      "patch": 1
+    }
+  },
+  "gitaly": {
+    "version": "12.10.0-rc1-93-g40980d40",
+    "servers": 56,
+    "clusters": 14,
+    "filesystems": [
+      "EXT_2_3_4"
+    ]
+  },
+  "gitlab_pages": {
+    "enabled": true,
+    "version": "1.17.0"
+  },
+  "container_registry_server": {
+    "vendor": "gitlab",
+    "version": "2.9.1-gitlab"
+  },
+  "database": {
+    "adapter": "postgresql",
+    "version": "9.6.15",
+    "pg_system_id": 6842684531675334351,
+    "flavor": "Cloud SQL for PostgreSQL"
+  },
+  "analytics_unique_visits": {
+    "g_analytics_contribution": 999,
+    ...
+  },
+  "usage_activity_by_stage": {
+    "configure": {
+      "project_clusters_enabled": 999,
+      ...
+    },
+    "create": {
+      "merge_requests": 999,
+      ...
+    },
+    "manage": {
+      "events": 999,
+      ...
+    },
+    "monitor": {
+      "clusters": 999,
+      ...
+    },
+    "package": {
+      "projects_with_packages": 999
+    },
+    "plan": {
+      "issues": 999,
+      ...
+    },
+    "release": {
+      "deployments": 999,
+      ...
+    },
+    "secure": {
+      "user_container_scanning_jobs": 999,
+      ...
+    },
+    "verify": {
+      "ci_builds": 999,
+      ...
+    }
+  },
+  "usage_activity_by_stage_monthly": {
+    "configure": {
+      "project_clusters_enabled": 999,
+      ...
+    },
+    "create": {
+      "merge_requests": 999,
+      ...
+    },
+    "manage": {
+      "events": 999,
+      ...
+    },
+    "monitor": {
+      "clusters": 999,
+      ...
+    },
+    "package": {
+      "projects_with_packages": 999
+    },
+    "plan": {
+      "issues": 999,
+      ...
+    },
+    "release": {
+      "deployments": 999,
+      ...
+    },
+    "secure": {
+      "user_container_scanning_jobs": 999,
+      ...
+    },
+    "verify": {
+      "ci_builds": 999,
+      ...
+    }
+  },
+  "topology": {
+    "duration_s": 0.013836685999194742,
+    "application_requests_per_hour": 4224,
+    "query_apdex_weekly_average": 0.996,
+    "failures": [],
+    "nodes": [
+      {
+        "node_memory_total_bytes": 33269903360,
+        "node_memory_utilization": 0.35,
+        "node_cpus": 16,
+        "node_cpu_utilization": 0.2,
+        "node_uname_info": {
+          "machine": "x86_64",
+          "sysname": "Linux",
+          "release": "4.19.76-linuxkit"
+        },
+        "node_services": [
+          {
+            "name": "web",
+            "process_count": 16,
+            "process_memory_pss": 233349888,
+            "process_memory_rss": 788220927,
+            "process_memory_uss": 195295487,
+            "server": "puma"
+          },
+          {
+            "name": "sidekiq",
+            "process_count": 1,
+            "process_memory_pss": 734080000,
+            "process_memory_rss": 750051328,
+            "process_memory_uss": 731533312
+          },
+          ...
+        ],
+        ...
+      },
+      ...
+    ]
+  }
+}
+```
+
+## Notable changes
+
+In GitLab 14.6, [`flavor`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75587) was added to try to detect the underlying managed database variant.
+Possible values are "Amazon Aurora PostgreSQL", "PostgreSQL on Amazon RDS", "Cloud SQL for PostgreSQL",
+"Azure Database for PostgreSQL - Flexible Server", or "null".
+
+In GitLab 13.5, `pg_system_id` was added to send the [PostgreSQL system identifier](https://www.2ndquadrant.com/en/blog/support-for-postgresqls-system-identifier-in-barman/).
+
+## Export Service Ping data
+
+Rake tasks exist to export Service Ping data in different formats.
+
+- The Rake tasks export the raw SQL queries for `count`, `distinct_count`, `sum`.
+- The Rake tasks export the Redis counter class or the line of the Redis block for `redis_usage_data`.
+- The Rake tasks calculate the `alt_usage_data` metrics.
+
+In the home directory of your local GitLab installation run the following Rake tasks for the YAML and JSON versions respectively:
+
+```shell
+# for YAML export of SQL queries
+bin/rake gitlab:usage_data:dump_sql_in_yaml
+
+# for JSON export of SQL queries
+bin/rake gitlab:usage_data:dump_sql_in_json
+
+# for JSON export of Non SQL data
+bin/rake gitlab:usage_data:dump_non_sql_in_json
+
+# You may pipe the output into a file
+bin/rake gitlab:usage_data:dump_sql_in_yaml > ~/Desktop/usage-metrics-2020-09-02.yaml
+```
+
+## Generate Service Ping
+
+To generate Service Ping, use [Teleport](https://goteleport.com/docs/) or a detached screen session on a remote server.
+
+### Triggering
+
+#### Trigger Service Ping with Teleport
+
+1. Request temporary [access](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to the required environment.
+1. After your approval is issued, [access the Rails console](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/teleport/Connect_to_Rails_Console_via_Teleport.md#access-approval).
+1. Run `GitlabServicePingWorker.new.perform('triggered_from_cron' => false)`.
+
+#### Trigger Service Ping with a detached screen session
+
+1. Connect to bastion with agent forwarding:
+
+   ```shell
+   ssh -A lb-bastion.gprd.gitlab.com
+   ```
+
+1. Create named screen:
+
+   ```shell
+   screen -S <username>_usage_ping_<date>
+   ```
+
+1. Connect to console host:
+
+   ```shell
+   ssh $USER-rails@console-01-sv-gprd.c.gitlab-production.internal
+   ```
+
+1. Run:
+
+   ```shell
+   GitlabServicePingWorker.new.perform('triggered_from_cron' => false)
+   ```
+
+1. To detach from screen, press `ctrl + A`, `ctrl + D`.
+1. Exit from bastion:
+
+   ```shell
+   exit
+   ```
+
+1. Get the metrics duration from logs:
+
+Search in Google Console logs for `time_elapsed`. [Query example](https://cloudlogging.app.goo.gl/nWheZvD8D3nWazNe6).
+
+### Verification (After approx 30 hours)
+
+#### Verify with Teleport
+
+1. Follow [the steps](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to request a new access to the required environment and connect to the Rails console
+1. Check the last payload in `raw_usage_data` table: `RawUsageData.last.payload`
+1. Check the when the payload was sent: `RawUsageData.last.sent_at`
+
+#### Verify using detached screen session
+
+1. Reconnect to bastion:
+
+   ```shell
+   ssh -A lb-bastion.gprd.gitlab.com
+   ```
+
+1. Find your screen session:
+
+   ```shell
+   screen -ls
+   ```
+
+1. Attach to your screen session:
+
+   ```shell
+   screen -x 14226.mwawrzyniak_usage_ping_2021_01_22
+   ```
+
+1. Check the last payload in `raw_usage_data` table:
+
+   ```shell
+   RawUsageData.last.payload
+   ```
+
+1. Check the when the payload was sent:
+
+   ```shell
+   RawUsageData.last.sent_at
+   ```
+
+### Skip database write operations
+
+To skip database write operations, DevOps report creation, and storage of usage data payload, pass an optional argument:
+
+```shell
+skip_db_write:
+GitlabServicePingWorker.new.perform('triggered_from_cron' => false, 'skip_db_write' => true)
+```
+
+## Monitoring
+
+Service Ping reporting process state is monitored with [internal SiSense dashboard](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health).
+
+## Related topics
+
+- [Product Intelligence Guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)
+- [Snowplow Guide](../snowplow/index.md)
+- [Product Intelligence Direction](https://about.gitlab.com/direction/analytics/product-intelligence/)
+- [Data Analysis Process](https://about.gitlab.com/handbook/business-technology/data-team/#data-analysis-process/)
+- [Data for Product Managers](https://about.gitlab.com/handbook/business-technology/data-team/programs/data-for-product-managers/)
+- [Data Infrastructure](https://about.gitlab.com/handbook/business-technology/data-team/platform/infrastructure/)
diff --git a/doc/development/internal_analytics/service_ping/metrics_dictionary.md b/doc/development/internal_analytics/service_ping/metrics_dictionary.md
new file mode 100644
index 00000000000..d1f1c0b595a
--- /dev/null
+++ b/doc/development/internal_analytics/service_ping/metrics_dictionary.md
@@ -0,0 +1,334 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Metrics Dictionary Guide
+
+[Service Ping](index.md) metrics are defined in individual YAML files definitions from which the
+[Metrics Dictionary](https://metrics.gitlab.com/) is built. Currently, the metrics dictionary is built automatically once a day. When a change to a metric is made in a YAML file, you can see the change in the dictionary within 24 hours.
+This guide describes the dictionary and how it's implemented.
+
+## Metrics Definition and validation
+
+We are using [JSON Schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/schema.json) to validate the metrics definition.
+
+This process is meant to ensure consistent and valid metrics defined for Service Ping. All metrics *must*:
+
+- Comply with the defined [JSON schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/schema.json).
+- Have a unique `key_path` .
+- Have an owner.
+
+All metrics are stored in YAML files:
+
+- [`config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/metrics)
+
+WARNING:
+Only metrics with a metric definition YAML and whose status is not `removed` are added to the Service Ping JSON payload.
+
+Each metric is defined in a separate YAML file consisting of a number of fields:
+
+| Field               | Required | Additional information                                         |
+|---------------------|----------|----------------------------------------------------------------|
+| `key_path`          | yes      | JSON key path for the metric, location in Service Ping payload.  |
+| `name` (deprecated)  | no       | Metric name suggestion. Does not have any impact on the Service Ping payload, only serves as documentation. |
+| `description`       | yes      |                                                                |
+| `product_section`   | yes      | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml). |
+| `product_stage`     | yes       | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the metric. |
+| `product_group`     | yes      | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the metric. |
+| `value_type`        | yes      | `string`; one of [`string`, `number`, `boolean`, `object`](https://json-schema.org/understanding-json-schema/reference/type.html).                                                     |
+| `status`            | yes      | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `removed`, `broken`. |
+| `time_frame`        | yes      | `string`; may be set to a value like `7d`, `28d`, `all`, `none`. |
+| `data_source`       | yes      | `string`; may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `system`, `license`. |
+| `data_category`     | yes      | `string`; [categories](#data-category) of the metric, may be set to `operational`, `optional`, `subscription`, `standard`. The default value is `optional`.|
+| `instrumentation_class` | yes   | `string`; [the class that implements the metric](metrics_instrumentation.md).  |
+| `distribution`      | yes      | `array`; may be set to one of `ce, ee` or `ee`. The [distribution](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available.  |
+| `performance_indicator_type`  | no      | `array`; may be set to one of [`gmau`, `smau`, `paid_gmau`, `umau` or `customer_health_score`](https://about.gitlab.com/handbook/business-technology/data-team/data-catalog/xmau-analysis/). |
+| `tier`              | yes      | `array`; may contain one or a combination of `free`, `premium` or `ultimate`. The [tier](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. This should be verbose and contain all tiers where a metric is available. |
+| `milestone`         | yes       | The milestone when the metric is introduced and when it's available to self-managed instances with the official GitLab release. |
+| `milestone_removed` | no       | The milestone when the metric is removed. |
+| `introduced_by_url` | no       | The URL to the merge request that introduced the metric to be available for self-managed instances. |
+| `removed_by_url`    | no       | The URL to the merge request that removed the metric. |
+| `repair_issue_url`  | no       | The URL of the issue that was created to repair a metric with a `broken` status. |
+| `options`           | no       | `object`: options information needed to calculate the metric value. |
+| `skip_validation`   | no       | This should **not** be set. [Used for imported metrics until we review, update and make them valid](https://gitlab.com/groups/gitlab-org/-/epics/5425). |
+
+### Metric `key_path`
+
+The `key_path` of the metric is the location in the JSON Service Ping payload.
+
+The `key_path` could be composed from multiple parts separated by `.` and it must be unique.
+
+We recommend to add the metric in one of the top-level keys:
+
+- `settings`: for settings related metrics.
+- `counts_weekly`: for counters that have data for the most recent 7 days.
+- `counts_monthly`: for counters that have data for the most recent 28 days.
+- `counts`: for counters that have data for all time.
+
+NOTE:
+We can't control what the metric's `key_path` is, because some of them are generated dynamically in `usage_data.rb`.
+For example, see [Redis HLL metrics](implement.md#redis-hll-counters).
+
+### Metric name (deprecated)
+
+WARNING:
+This feature was deprecated in GitLab 16.1
+and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/411602) in 16.2.
+
+To improve metric discoverability by a wider audience, each metric with
+instrumentation added at an appointed `key_path` receives a `name` attribute
+filled with the name suggestion, corresponding to the metric `data_source` and instrumentation.
+Metric name suggestions can contain two types of elements:
+
+1. **User input prompts**: enclosed by angle brackets (`< >`), these pieces should be replaced or
+   removed when you create a metrics YAML file.
+1. **Fixed suggestion**: plaintext parts generated according to well-defined algorithms.
+   They are based on underlying instrumentation, and must not be changed.
+
+For a metric name to be valid, it must not include any prompt, and fixed suggestions
+must not be changed.
+
+#### Generate a metric name suggestion (deprecated)
+
+WARNING:
+This feature was deprecated in GitLab 16.1
+and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/411602) in 16.2.
+
+The metric YAML generator can suggest a metric name for you.
+To generate a metric name suggestion, first instrument the metric at the provided `key_path`.
+Then, generate the metric's YAML definition and
+return to the instrumentation and update it.
+
+1. Add the metric instrumentation class to `lib/gitlab/usage/metrics/instrumentations/`.
+1. Add the metric logic in the instrumentation class.
+1. Run the [metrics YAML generator](metrics_dictionary.md#create-a-new-metric-definition).
+1. Use the metric name suggestion to select a suitable metric name.
+1. Update the metric's YAML definition with the correct `key_path`.
+
+### Metric statuses
+
+Metric definitions can have one of the following statuses:
+
+- `active`: Metric is used and reports data.
+- `broken`: Metric reports broken data (for example, -1 fallback), or does not report data at all. A metric marked as `broken` must also have the `repair_issue_url` attribute.
+- `removed`: Metric was removed, but it may appear in Service Ping payloads sent from instances running on older versions of GitLab.
+
+### Metric `value_type`
+
+Metric definitions can have one of the following values for `value_type`:
+
+- `boolean`
+- `number`
+- `string`
+- `object`: A metric with `value_type: object` must have `value_json_schema` with a link to the JSON schema for the object.
+In general, we avoid complex objects and prefer one of the `boolean`, `number`, or `string` value types.
+An example of a metric that uses `value_type: object` is `topology` (`/config/metrics/settings/20210323120839_topology.yml`),
+which has a related schema in `/config/metrics/objects_schemas/topology_schema.json`.
+
+### Metric `time_frame`
+
+A metric's time frame is calculated based on the `time_frame` field and the `data_source` of the metric.
+For `redis_hll` metrics, the type of aggregation is also taken into consideration. In this context, the term "aggregation" refers to [chosen events data storage interval](implement.md#add-new-events), and is **NOT** related to the Aggregated Metrics feature.
+For more information about the aggregation type of each feature, see the [`common.yml` file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/common.yml). Weeks run from Monday to Sunday.
+
+| data_source            | time_frame | aggregation    | Description                                     |
+|------------------------|------------|----------------|-------------------------------------------------|
+| any                    | `none`     | not applicable | A type of data that’s not tracked over time, such as settings and configuration information |
+| `database`             | `all`      | not applicable | The whole time the metric has been active (all-time interval) |
+| `database`             | `7d`       | not applicable | 9 days ago to 2 days ago |
+| `database`             | `28d`      | not applicable | 30 days ago to 2 days ago |
+| `redis`                | `all`      | not applicable | The whole time the metric has been active (all-time interval) |
+| `redis_hll`            | `7d`       | `daily`        | Most recent 7 complete days |
+| `redis_hll`            | `7d`       | `weekly`       | Most recent complete week |
+| `redis_hll`            | `28d`      | `daily`        | Most recent 28 complete days |
+| `redis_hll`            | `28d`      | `weekly`       | Most recent 4 complete weeks |
+
+### Data category
+
+We use the following categories to classify a metric:
+
+- `operational`: Required data for operational purposes.
+- `optional`: Default value for a metric. Data that is optional to collect. This can be [enabled or disabled](../../../user/admin_area/settings/usage_statistics.md#enable-or-disable-usage-statistics) in the Admin Area.
+- `subscription`: Data related to licensing.
+- `standard`: Standard set of identifiers that are included when collecting data.
+
+An aggregate metric is a metric that is the sum of two or more child metrics. Service Ping uses the data category of
+the aggregate metric to determine whether or not the data is included in the reported Service Ping payload.
+
+### Metric name suggestion examples (deprecated)
+
+WARNING:
+This feature was deprecated in GitLab 16.1
+and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/411602) in 16.2.
+
+#### Metric with `data_source: database`
+
+For a metric instrumented with SQL:
+
+```sql
+SELECT COUNT(DISTINCT user_id) FROM clusters WHERE clusters.management_project_id IS NOT NULL
+```
+
+- **Suggested name**: `count_distinct_user_id_from_<adjective describing: '(clusters.management_project_id IS NOT NULL)'>_clusters`
+- **Prompt**: `<adjective describing: '(clusters.management_project_id IS NOT NULL)'>`
+  should be replaced with an adjective that best represents filter conditions, such as `project_management`
+- **Final metric name**: For example, `count_distinct_user_id_from_project_management_clusters`
+
+For metric instrumented with SQL:
+
+```sql
+SELECT COUNT(DISTINCT clusters.user_id)
+FROM clusters_applications_helm
+INNER JOIN clusters ON clusters.id = clusters_applications_helm.cluster_id
+WHERE clusters_applications_helm.status IN (3, 5)
+```
+
+- **Suggested name**: `count_distinct_user_id_from_<adjective describing: '(clusters_applications_helm.status IN (3, 5))'>_clusters_<with>_<adjective describing: '(clusters_applications_helm.status IN (3, 5))'>_clusters_applications_helm`
+- **Prompt**: `<adjective describing: '(clusters_applications_helm.status IN (3, 5))'>`
+  should be replaced with an adjective that best represents filter conditions
+- **Final metric name**: `count_distinct_user_id_from_clusters_with_available_clusters_applications_helm`
+
+In the previous example, the prompt is irrelevant, and user can remove it. The second
+occurrence corresponds with the `available` scope defined in `Clusters::Concerns::ApplicationStatus`.
+It can be used as the right adjective to replace prompt.
+
+The `<with>` represents a suggested conjunction for the suggested name of the joined relation.
+The person documenting the metric can use it by either:
+
+- Removing the surrounding `<>`.
+- Using a different conjunction, such as `having` or `including`.
+
+#### Metric with `data_source: redis` or `redis_hll`
+
+For metrics instrumented with a Redis-based counter, the suggested name includes
+only the single prompt to be replaced by the person working with metrics YAML.
+
+- **Prompt**: `<please fill metric name, suggested format is: {subject}_{verb}{ing|ed}_{object} eg: users_creating_epics or merge_requests_viewed_in_single_file_mode>`
+- **Final metric name**: We suggest the metric name should follow the format of
+  `{subject}_{verb}{ing|ed}_{object}`, such as `user_creating_epics`, `users_triggering_security_scans`,
+  or `merge_requests_viewed_in_single_file_mode`
+
+#### Metric with `data_source: prometheus` or `system`
+
+For metrics instrumented with Prometheus or coming from the operating system,
+the suggested name includes only the single prompt by person working with metrics YAML.
+
+- **Prompt**: `<please fill metric name>`
+- **Final metric name**: Due to the variety of cases that can apply to this kind of metric,
+  no naming convention exists. Each person instrumenting a metric should use their
+  best judgment to come up with a descriptive name.
+
+### Example YAML metric definition
+
+The linked [`uuid`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/license/uuid.yml)
+YAML file includes an example metric definition, where the `uuid` metric is the GitLab
+instance unique identifier.
+
+```yaml
+key_path: uuid
+description: GitLab instance unique identifier
+product_section: analytics
+product_stage: analytics
+product_group: analytics_instrumentation
+value_type: string
+status: active
+milestone: 9.1
+instrumentation_class: UuidMetric
+introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1521
+time_frame: none
+data_source: database
+distribution:
+- ce
+- ee
+tier:
+- free
+- premium
+- ultimate
+```
+
+### Create a new metric definition
+
+The GitLab codebase provides a dedicated [generator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_definition_generator.rb) to create new metric definitions.
+
+For uniqueness, the generated files include a timestamp prefix in ISO 8601 format.
+
+The generator takes a list of key paths and 3 options as arguments. It creates metric YAML definitions in the corresponding location:
+
+- `--ee`, `--no-ee` Indicates if metric is for EE.
+- `--dir=DIR` Indicates the metric directory. It must be one of: `counts_7d`, `7d`, `counts_28d`, `28d`, `counts_all`, `all`, `settings`, `license`.
+- `--class_name=CLASS_NAME` Indicates the instrumentation class. For example `UsersCreatingIssuesMetric`, `UuidMetric`
+
+**Single metric example**
+
+```shell
+bundle exec rails generate gitlab:usage_metric_definition counts.issues --dir=7d --class_name=CountIssues
+// Creates 1 file
+// create  config/metrics/counts_7d/issues.yml
+```
+
+**Multiple metrics example**
+
+```shell
+bundle exec rails generate gitlab:usage_metric_definition counts.issues counts.users --dir=7d --class_name=CountUsersCreatingIssues
+// Creates 2 files
+// create  config/metrics/counts_7d/issues.yml
+// create  config/metrics/counts_7d/users.yml
+```
+
+NOTE:
+To create a metric definition used in EE, add the `--ee` flag.
+
+```shell
+bundle exec rails generate gitlab:usage_metric_definition counts.issues --ee --dir=7d --class_name=CountUsersCreatingIssues
+// Creates 1 file
+// create  ee/config/metrics/counts_7d/issues.yml
+```
+
+### Metrics added dynamic to Service Ping payload
+
+The [Redis HLL metrics](implement.md#known-events-are-added-automatically-in-service-data-payload) are added automatically to Service Ping payload.
+
+A YAML metric definition is required for each metric. A dedicated generator is provided to create metric definitions for Redis HLL events.
+
+The generator takes `category` and `events` arguments, as the root key is `redis_hll_counters`, and creates two metric definitions for each of the events (for weekly and monthly time frames):
+
+**Single metric example**
+
+```shell
+bundle exec rails generate gitlab:usage_metric_definition:redis_hll issues count_users_closing_issues
+// Creates 2 files
+// create  config/metrics/counts_7d/count_users_closing_issues_weekly.yml
+// create  config/metrics/counts_28d/count_users_closing_issues_monthly.yml
+```
+
+**Multiple metrics example**
+
+```shell
+bundle exec rails generate gitlab:usage_metric_definition:redis_hll issues count_users_closing_issues count_users_reopening_issues
+// Creates 4 files
+// create  config/metrics/counts_7d/count_users_closing_issues_weekly.yml
+// create  config/metrics/counts_28d/count_users_closing_issues_monthly.yml
+// create  config/metrics/counts_7d/count_users_reopening_issues_weekly.yml
+// create  config/metrics/counts_28d/count_users_reopening_issues_monthly.yml
+```
+
+To create a metric definition used in EE, add the `--ee` flag.
+
+```shell
+bundle exec rails generate gitlab:usage_metric_definition:redis_hll issues users_closing_issues --ee
+// Creates 2 files
+// create  config/metrics/counts_7d/i_closed_weekly.yml
+// create  config/metrics/counts_28d/i_closed_monthly.yml
+```
+
+## Metrics Dictionary
+
+[Metrics Dictionary is a separate application](https://gitlab.com/gitlab-org/analytics-section/analytics-instrumentation/metric-dictionary).
+
+All metrics available in Service Ping are in the [Metrics Dictionary](https://metrics.gitlab.com/).
+
+### Copy query to clipboard
+
+To check if a metric has data in Sisense, use the copy query to clipboard feature. This copies a query that's ready to use in Sisense. The query gets the last five service ping data for GitLab.com for a given metric. For information about how to check if a Service Ping metric has data in Sisense, see this [demo](https://www.youtube.com/watch?v=n4o65ivta48).
diff --git a/doc/development/internal_analytics/service_ping/metrics_instrumentation.md b/doc/development/internal_analytics/service_ping/metrics_instrumentation.md
new file mode 100644
index 00000000000..b6ca773a572
--- /dev/null
+++ b/doc/development/internal_analytics/service_ping/metrics_instrumentation.md
@@ -0,0 +1,478 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Metrics instrumentation guide
+
+This guide describes how to develop Service Ping metrics using metrics instrumentation.
+
+<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
+For a video tutorial, see the [Adding Service Ping metric via instrumentation class](https://youtu.be/p2ivXhNxUoY).
+
+## Nomenclature
+
+- **Instrumentation class**:
+  - Inherits one of the metric classes: `DatabaseMetric`, `RedisMetric`, `RedisHLLMetric`, `NumbersMetric` or `GenericMetric`.
+  - Implements the logic that calculates the value for a Service Ping metric.
+
+- **Metric definition**
+  The Service Data metric YAML definition.
+
+- **Hardening**:
+  Hardening a method is the process that ensures the method fails safe, returning a fallback value like -1.
+
+## How it works
+
+A metric definition has the [`instrumentation_class`](metrics_dictionary.md) field, which can be set to a class.
+
+The defined instrumentation class should inherit one of the existing metric classes: `DatabaseMetric`, `RedisMetric`, `RedisHLLMetric`, `NumbersMetric` or `GenericMetric`.
+
+The current convention is that a single instrumentation class corresponds to a single metric. On rare occasions, there are exceptions to that convention like [Redis metrics](#redis-metrics). To use a single instrumentation class for more than one metric, please reach out to one of the `@gitlab-org/analytics-section/analytics-instrumentation/engineers` members to consult about your case.
+
+Using the instrumentation classes ensures that metrics can fail safe individually, without breaking the entire
+ process of Service Ping generation.
+
+We have built a domain-specific language (DSL) to define the metrics instrumentation.
+
+## Database metrics
+
+You can use database metrics to track data kept in the database, for example, a count of issues that exist on a given instance.
+
+- `operation`: Operations for the given `relation`, one of `count`, `distinct_count`, `sum`, and `average`.
+- `relation`: Assigns lambda that returns the `ActiveRecord::Relation` for the objects we want to perform the `operation`. The assigned lambda can accept up to one parameter. The parameter is hashed and stored under the `options` key in the metric definition.
+- `start`: Specifies the start value of the batch counting, by default is `relation.minimum(:id)`.
+- `finish`: Specifies the end value of the batch counting, by default is `relation.maximum(:id)`.
+- `cache_start_and_finish_as`: Specifies the cache key for `start` and `finish` values and sets up caching them. Use this call when `start` and `finish` are expensive queries that should be reused between different metric calculations.
+- `available?`: Specifies whether the metric should be reported. The default is `true`.
+- `timestamp_column`: Optionally specifies timestamp column for metric used to filter records for time constrained metrics. The default is `created_at`.
+
+[Example of a merge request that adds a database metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60022).
+
+```ruby
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+        class CountIssuesMetric < DatabaseMetric
+          operation :count
+
+          relation ->(options) { Issue.where(confidential: options[:confidential]) }
+        end
+      end
+    end
+  end
+end
+```
+
+### Ordinary batch counters Example
+
+```ruby
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+        class CountIssuesMetric < DatabaseMetric
+          operation :count
+
+          start { Issue.minimum(:id) }
+          finish { Issue.maximum(:id) }
+
+          relation { Issue }
+        end
+      end
+    end
+  end
+end
+```
+
+### Distinct batch counters Example
+
+```ruby
+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+        class CountUsersAssociatingMilestonesToReleasesMetric < DatabaseMetric
+          operation :distinct_count, column: :author_id
+
+          relation { Release.with_milestones }
+
+          start { Release.minimum(:author_id) }
+          finish { Release.maximum(:author_id) }
+        end
+      end
+    end
+  end
+end
+```
+
+### Sum Example
+
+```ruby
+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+        class JiraImportsTotalImportedIssuesCountMetric < DatabaseMetric
+          operation :sum, column: :imported_issues_count
+
+          relation { JiraImportState.finished }
+        end
+      end
+    end
+  end
+end
+```
+
+### Average Example
+
+```ruby
+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+        class CountIssuesWeightAverageMetric < DatabaseMetric
+          operation :average, column: :weight
+
+          relation { Issue }
+        end
+      end
+    end
+  end
+end
+```
+
+## Redis metrics
+
+You can use Redis metrics to track events not kept in the database, for example, a count of how many times the search bar has been used.
+
+[Example of a merge request that adds `Redis` metrics](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103455).
+
+The `RedisMetric` class can only be used as the `instrumentation_class` for Redis metrics with simple counters classes (classes that only inherit `BaseCounter` and set `PREFIX` and `KNOWN_EVENTS` constants). In case the counter class has additional logic included in it, a new `instrumentation_class`, inheriting from `RedisMetric`, needs to be created. This new class needs to include the additional logic from the counter class.
+
+Required options:
+
+- `event`: the event name.
+- `prefix`: the value of the `PREFIX` constant used in the counter classes from the `Gitlab::UsageDataCounters` namespace.
+
+Count unique values for `source_code_pushes` event.
+
+```yaml
+time_frame: all
+data_source: redis
+instrumentation_class: RedisMetric
+options:
+  event: pushes
+  prefix: source_code
+```
+
+### Availability-restrained Redis metrics
+
+If the Redis metric should only be available in the report under some conditions, then you must specify these conditions in a new class that is a child of the `RedisMetric` class.
+
+```ruby
+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+        class MergeUsageCountRedisMetric < RedisMetric
+          available? { Feature.enabled?(:merge_usage_data_missing_key_paths) }
+        end
+      end
+    end
+  end
+end
+```
+
+You must also use the class's name in the YAML setup.
+
+```yaml
+time_frame: all
+data_source: redis
+instrumentation_class: MergeUsageCountRedisMetric
+options:
+  event: pushes
+  prefix: source_code
+```
+
+## Redis HyperLogLog metrics
+
+You can use Redis HyperLogLog metrics to track events not kept in the database and incremented for unique values such as unique users,
+for example, a count of how many different users used the search bar.
+
+[Example of a merge request that adds a `RedisHLL` metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61685).
+
+Count unique values for `i_quickactions_approve` event.
+
+```yaml
+time_frame: 28d
+data_source: redis_hll
+instrumentation_class: RedisHLLMetric
+options:
+  events:
+    - i_quickactions_approve
+```
+
+### Availability-restrained Redis HyperLogLog metrics
+
+If the Redis HyperLogLog metric should only be available in the report under some conditions, then you must specify these conditions in a new class that is a child of the `RedisHLLMetric` class.
+
+```ruby
+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+        class MergeUsageCountRedisHLLMetric < RedisHLLMetric
+          available? { Feature.enabled?(:merge_usage_data_missing_key_paths) }
+        end
+      end
+    end
+  end
+end
+```
+
+You must also use the class's name in the YAML setup.
+
+```yaml
+time_frame: 28d
+data_source: redis_hll
+instrumentation_class: MergeUsageCountRedisHLLMetric
+options:
+  events:
+    - i_quickactions_approve
+```
+
+## Aggregated metrics
+
+<div class="video-fallback">
+  See the video from: <a href="https://www.youtube.com/watch?v=22LbYqHwtUQ">Product Intelligence Office Hours Oct 6th</a> for an aggregated metrics walk-through.
+</div>
+<figure class="video-container">
+  <iframe src="https://www.youtube-nocookie.com/embed/22LbYqHwtUQ" frameborder="0" allowfullscreen> </iframe>
+</figure>
+
+The aggregated metrics feature provides insight into the number of data attributes, for example `pseudonymized_user_ids`, that occurred in a collection of events. For example, you can aggregate the number of users who perform multiple actions such as creating a new issue and opening
+a new merge request.
+
+You can use a YAML file to define your aggregated metrics. The following arguments are required:
+
+- `options.events`: List of event names to aggregate into metric data. All events in this list must
+  use the same data source. Additional data source requirements are described in
+  [Database sourced aggregated metrics](implement.md#database-sourced-aggregated-metrics) and
+  [Redis sourced aggregated metrics](implement.md#redis-sourced-aggregated-metrics).
+- `options.aggregate.operator`: Operator that defines how the aggregated metric data is counted. Available operators are:
+  - `OR`: Removes duplicates and counts all entries that triggered any of the listed events.
+  - `AND`: Removes duplicates and counts all elements that were observed triggering all of the following events.
+- `options.aggregate.attribute`: Information pointing to the attribute that is being aggregated across events.
+- `time_frame`: One or more valid time frames. Use these to limit the data included in aggregated metrics to events within a specific date-range. Valid time frames are:
+  - `7d`: The last 7 days of data.
+  - `28d`: The last 28 days of data.
+  - `all`: All historical data, only available for `database` sourced aggregated metrics.
+- `data_source`: Data source used to collect all events data included in the aggregated metrics. Valid data sources are:
+  - [`database`](implement.md#database-sourced-aggregated-metrics)
+  - [`redis_hll`](implement.md#redis-sourced-aggregated-metrics)
+
+Refer to merge request [98206](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98206) for an example of a merge request that adds an `AggregatedMetric` metric.
+
+Count unique `user_ids` that occurred in at least one of the events: `incident_management_alert_status_changed`,
+`incident_management_alert_assigned`, `incident_management_alert_todo`, `incident_management_alert_create_incident`.
+
+```yaml
+time_frame: 28d
+instrumentation_class: AggregatedMetric
+data_source: redis_hll
+options:
+    aggregate:
+        operator: OR
+        attribute: user_id
+    events:
+        - `incident_management_alert_status_changed`
+        - `incident_management_alert_assigned`
+        - `incident_management_alert_todo`
+        - `incident_management_alert_create_incident`
+```
+
+### Availability-restrained Aggregated metrics
+
+If the Aggregated metric should only be available in the report under specific conditions, then you must specify these conditions in a new class that is a child of the `AggregatedMetric` class.
+
+```ruby
+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+        class MergeUsageCountAggregatedMetric < AggregatedMetric
+          available? { Feature.enabled?(:merge_usage_data_missing_key_paths) }
+        end
+      end
+    end
+  end
+end
+```
+
+You must also use the class's name in the YAML setup.
+
+```yaml
+time_frame: 28d
+instrumentation_class: MergeUsageCountAggregatedMetric
+data_source: redis_hll
+options:
+    aggregate:
+        operator: OR
+        attribute: user_id
+    events:
+        - `incident_management_alert_status_changed`
+        - `incident_management_alert_assigned`
+        - `incident_management_alert_todo`
+        - `incident_management_alert_create_incident`
+```
+
+## Numbers metrics
+
+- `operation`: Operations for the given `data` block. Currently we only support `add` operation.
+- `data`: a `block` which contains an array of numbers.
+- `available?`: Specifies whether the metric should be reported. The default is `true`.
+
+```ruby
+# frozen_string_literal: true
+
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+          class IssuesBoardsCountMetric < NumbersMetric
+            operation :add
+
+            data do |time_frame|
+              [
+                 CountIssuesMetric.new(time_frame: time_frame).value,
+                 CountBoardsMetric.new(time_frame: time_frame).value
+              ]
+            end
+          end
+        end
+      end
+    end
+  end
+end
+```
+
+You must also include the instrumentation class name in the YAML setup.
+
+```yaml
+time_frame: 28d
+instrumentation_class: IssuesBoardsCountMetric
+```
+
+## Generic metrics
+
+You can use generic metrics for other metrics, for example, an instance's database version. Observations type of data will always have a Generic metric counter type.
+
+- `value`: Specifies the value of the metric.
+- `available?`: Specifies whether the metric should be reported. The default is `true`.
+
+[Example of a merge request that adds a generic metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60256).
+
+```ruby
+module Gitlab
+  module Usage
+    module Metrics
+      module Instrumentations
+        class UuidMetric < GenericMetric
+          value do
+            Gitlab::CurrentSettings.uuid
+          end
+        end
+      end
+    end
+  end
+end
+```
+
+## Support for instrumentation classes
+
+There is support for:
+
+- `count`, `distinct_count`, `estimate_batch_distinct_count`, `sum`, and `average` for [database metrics](#database-metrics).
+- [Redis metrics](#redis-metrics).
+- [Redis HLL metrics](#redis-hyperloglog-metrics).
+- `add` for [numbers metrics](#numbers-metrics).
+- [Generic metrics](#generic-metrics), which are metrics based on settings or configurations.
+
+There is no support for:
+
+- `add`, `histogram` for database metrics.
+
+You can [track the progress to support these](https://gitlab.com/groups/gitlab-org/-/epics/6118).
+
+## Create a new metric instrumentation class
+
+To create a stub instrumentation for a Service Ping metric, you can use a dedicated [generator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_generator.rb):
+
+The generator takes the class name as an argument and the following options:
+
+- `--type=TYPE` Required. Indicates the metric type. It must be one of: `database`, `generic`, `redis`, `numbers`.
+- `--operation` Required for `database` & `numbers` type.
+  - For `database` it must be one of: `count`, `distinct_count`, `estimate_batch_distinct_count`, `sum`, `average`.
+  - For `numbers` it must be: `add`.
+- `--ee` Indicates if the metric is for EE.
+
+```shell
+rails generate gitlab:usage_metric CountIssues --type database --operation distinct_count
+        create lib/gitlab/usage/metrics/instrumentations/count_issues_metric.rb
+        create spec/lib/gitlab/usage/metrics/instrumentations/count_issues_metric_spec.rb
+```
+
+## Migrate Service Ping metrics to instrumentation classes
+
+This guide describes how to migrate a Service Ping metric from [`lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb) or [`ee/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/usage_data.rb) to instrumentation classes.
+
+1. Choose the metric type:
+
+- [Database metric](#database-metrics)
+- [Redis HyperLogLog metrics](#redis-hyperloglog-metrics)
+- [Redis metric](#redis-metrics)
+- [Numbers metric](#numbers-metrics)
+- [Generic metric](#generic-metrics)
+
+1. Determine the location of instrumentation class: either under `ee` or outside `ee`.
+
+1. [Generate the instrumentation class file](#create-a-new-metric-instrumentation-class).
+
+1. Fill the instrumentation class body:
+
+   - Add code logic for the metric. This might be similar to the metric implementation in `usage_data.rb`.
+   - Add tests for the individual metric [`spec/lib/gitlab/usage/metrics/instrumentations/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/usage/metrics/instrumentations).
+   - Add tests for Service Ping.
+
+1. [Generate the metric definition file](metrics_dictionary.md#create-a-new-metric-definition).
+
+1. Remove the code from [`lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb) or [`ee/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/usage_data.rb).
+
+1. Remove the tests from [`spec/lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/lib/gitlab/usage_data_spec.rb) or [`ee/spec/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/lib/ee/gitlab/usage_data_spec.rb).
+
+## Troubleshoot metrics
+
+Sometimes metrics fail for reasons that are not immediately clear. The failures can be related to performance issues or other problems.
+The following pairing session video gives you an example of an investigation in to a real-world failing metric.
+
+<div class="video-fallback">
+  See the video from: <a href="https://www.youtube.com/watch?v=y_6m2POx2ug">Product Intelligence Office Hours Oct 27th</a> to learn more about the metrics troubleshooting process.
+</div>
+<figure class="video-container">
+  <iframe src="https://www.youtube-nocookie.com/embed/y_6m2POx2ug" frameborder="0" allowfullscreen> </iframe>
+</figure>
diff --git a/doc/development/internal_analytics/service_ping/metrics_lifecycle.md b/doc/development/internal_analytics/service_ping/metrics_lifecycle.md
new file mode 100644
index 00000000000..cc56863690c
--- /dev/null
+++ b/doc/development/internal_analytics/service_ping/metrics_lifecycle.md
@@ -0,0 +1,106 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Service Ping metric lifecycle
+
+The following guidelines explain the steps to follow at each stage of a metric's lifecycle.
+
+## Add a new metric
+
+Follow the [Implement Service Ping](implement.md) guide.
+
+## Change an existing metric
+
+WARNING:
+We want to **PREVENT** changes to the calculation logic or important attributes on any metric as this invalidates comparisons of the same metric across different versions of GitLab.
+
+If you change a metric, you have to consider that not all instances of GitLab are running on the newest version. Old instances will still report the old version of the metric.
+Additionally, a metric's reported numbers are primarily interesting compared to previously reported numbers.
+As a result, if you need to change one of the following parts of a metric, you need to add a new metric instead. It's your choice whether to keep the old metric alongside the new one or [remove it](#remove-a-metric).
+
+- **calculation logic**: This means any changes that can produce a different value than the previous implementation
+- **YAML attributes**: The following attributes are directly used for analysis or calculation: `key_path`, `time_frame`, `value_type`, `data_source`.
+
+If you change the `performance_indicator_type` attribute of a metric or think your case needs an exception from the outlined rules then please notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) teams by `@` mentioning those groups in a comment on the merge request or issue.
+
+You can change any other attributes without impact to the calculation or analysis. See [this video tutorial](https://youtu.be/bYf3c01KCls) for help updating metric attributes.
+
+Currently, the [Metrics Dictionary](https://metrics.gitlab.com/) is built automatically once a day. You can see the change in the dictionary within 24 hours when you change the metric's YAML file.
+
+## Remove a metric
+
+WARNING:
+If a metric is not used in Sisense or any other system after 6 months, the
+Analytics Instrumentation team marks it as inactive and assigns it to the group owner for review.
+
+We are working on automating this process. See [this epic](https://gitlab.com/groups/gitlab-org/-/epics/8988) for details.
+
+Analytics Instrumentation removes metrics from Service Ping if they are not used in any Sisense dashboard.
+
+For an example of the metric removal process, see this [example issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388236).
+
+To remove a metric:
+
+1. Create an issue for removing the metric if none exists yet. The issue needs to outline why the metric should be deleted. You can use this issue to document the removal process.
+
+1. Verify the metric is not used to calculate the conversational index. The
+   conversational index is a measure that reports back to self-managed instances
+   to inform administrators of the progress of DevOps adoption for the instance.
+
+   You can check
+   [`CalculateConvIndexService`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/app/services/calculate_conv_index_service.rb)
+   to view the metrics that are used. The metrics are represented
+   as the keys that are passed as a field argument into the `get_value` method.
+
+1. Verify that removing the metric from the Service Ping payload does not cause
+   errors in [Version App](https://gitlab.com/gitlab-services/version-gitlab-com)
+   when the updated payload is collected and processed. Version App collects
+   and persists all Service Ping reports. To verify Service Ping processing in your local development environment, follow this [guide](https://www.youtube.com/watch?v=FS5emplabRU).
+   Alternatively, you can modify [fixtures](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/spec/support/usage_data_helpers.rb#L540)
+   used to test the [`UsageDataController#create`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/3760ef28/spec/controllers/usage_data_controller_spec.rb#L75)
+   endpoint, and assure that test suite does not fail when metric that you wish to remove is not included into test payload.
+
+1. Remove data from Redis
+
+   For [Ordinary Redis](implement.md#ordinary-redis-counters) counters remove data stored in Redis.
+
+   - Add a migration to remove the data from Redis for the related Redis keys. For more details, see [this MR example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82604/diffs).
+
+1. Create an issue in the
+   [GitLab Data Team project](https://gitlab.com/gitlab-data/analytics/-/issues).
+   Ask for confirmation that the metric is not referred to in any SiSense dashboards and
+   can be safely removed from Service Ping. Use this
+   [example issue](https://gitlab.com/gitlab-data/analytics/-/issues/15266) for guidance.
+
+1. Notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) by `@` mentioning those groups in a comment in the issue from step 1 regarding the deletion of the metric.
+   Many Service Ping metrics are relied upon for health score and XMAU reporting and unexpected changes to those metrics could break reporting.
+
+1. After you verify the metric can be safely removed,
+   update the attributes of the metric's YAML definition:
+
+   - Set the `status:` to `removed`.
+   - Set `removed_by_url:` to the URL of the MR removing the metric
+   - Set `milestone_removed:` to the number of the
+     milestone in which the metric was removed.
+
+   Do not remove the metric's YAML definition altogether. Some self-managed
+   instances might not immediately update to the latest version of GitLab, and
+   therefore continue to report the removed metric. The Analytics Instrumentation team
+   requires a record of all removed metrics to identify and filter them.
+
+   For example please take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#b01f429a54843feb22265100c0e4fec1b7da1240_10_10).
+
+1. After you verify the metric can be safely removed,
+   remove the metric's instrumentation from
+   [`lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb)
+   or
+   [`ee/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/usage_data.rb).
+
+   For example please take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#6335dc533bd21df26db9de90a02dd66278c2390d_167_167).
+
+1. Remove any other records related to the metric:
+   - The feature flag YAML file at [`config/feature_flags/*/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/feature_flags).
+   - The entry in the known events YAML file at [`lib/gitlab/usage_data_counters/known_events/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/usage_data_counters/known_events).
diff --git a/doc/development/internal_analytics/service_ping/performance_indicator_metrics.md b/doc/development/internal_analytics/service_ping/performance_indicator_metrics.md
new file mode 100644
index 00000000000..d7811c52bb1
--- /dev/null
+++ b/doc/development/internal_analytics/service_ping/performance_indicator_metrics.md
@@ -0,0 +1,16 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Performance Indicator Metrics guide
+
+This guide describes how to use metrics definitions to define [performance indicator](https://about.gitlab.com/handbook/product/analytics-instrumentation-guide/#implementing-product-performance-indicators) metrics.
+
+To use a metric definition to manage a performance indicator:
+
+1. Create a merge request that includes related changes.
+1. Use labels `~"analytics instrumentation"`, `"~Data Warehouse::Impact Check"`.
+1. Update the metric definition `performance_indicator_type` [field](metrics_dictionary.md#metrics-definition-and-validation).
+1. Create an issue in GitLab Product Data Insights project with the [PI Chart Help template](https://gitlab.com/gitlab-data/product-analytics/-/issues/new?issuable_template=PI%20Chart%20Help) to have the new metric visualized.
diff --git a/doc/development/internal_analytics/service_ping/review_guidelines.md b/doc/development/internal_analytics/service_ping/review_guidelines.md
new file mode 100644
index 00000000000..31b6c3f5580
--- /dev/null
+++ b/doc/development/internal_analytics/service_ping/review_guidelines.md
@@ -0,0 +1,80 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Service Ping review guidelines
+
+This page includes introductory material for a
+[Analytics Instrumentation](https://about.gitlab.com/handbook/engineering/development/analytics/analytics-instrumentation/)
+review, and is specific to Service Ping related reviews. For broader advice and
+general best practices for code reviews, refer to our [code review guide](../../code_review.md).
+
+## Resources for reviewers
+
+- [Service Ping Guide](index.md)
+- [Metrics Dictionary](https://metrics.gitlab.com/)
+
+## Review process
+
+We recommend a Analytics Instrumentation review when a merge request (MR) touches
+any of the following Service Ping files:
+
+- `usage_data*` files.
+- The Metrics Dictionary, including files in:
+  - [`config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/metrics).
+  - [`ee/config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/config/metrics).
+  - [`schema.json`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/schema.json).
+- Analytics Instrumentation tooling. For example,
+  [`Gitlab::UsageMetricDefinitionGenerator`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_definition_generator.rb)
+
+### Roles and process
+
+#### The merge request **author** should
+
+- Decide whether a Analytics Instrumentation review is needed. You can skip the Analytics Instrumentation
+review and remove the labels if the changes are not related to the Analytics Instrumentation domain and
+are regular backend changes.
+- If a Analytics Instrumentation review is needed, add the labels
+  `~analytics instrumentation` and `~analytics instrumentation::review pending`.
+- For merge requests authored by Analytics Instrumentation team members:
+  - Assign both the `~backend` and `~analytics instrumentation` reviews to another Analytics Instrumentation team member.
+  - Assign the maintainer review to someone outside of the Analytics Instrumentation group.
+- Assign an
+  [engineer](https://gitlab.com/groups/gitlab-org/analytics-section/analytics-instrumentation/engineers/-/group_members?with_inherited_permissions=exclude) from the Analytics Instrumentation team for a review.
+- Set the correct attributes in the metric's YAML definition:
+  - `product_section`, `product_stage`, `product_group`
+  - Provide a clear description of the metric.
+- Add a changelog [according to guidelines](../../changelog.md).
+
+#### The Analytics Instrumentation **reviewer** should
+
+- Perform a first-pass review on the merge request and suggest improvements to the author.
+- Check the [metrics location](metrics_dictionary.md#metric-key_path) in
+  the Service Ping JSON payload.
+- Add the `~database` label and ask for a [database review](../../database_review.md) for
+  metrics that are based on Database.
+- Add `~Data Warehouse::Impact Check` for any database metric that has a query change. Changes in queries can affect [data operations](https://about.gitlab.com/handbook/business-technology/data-team/how-we-work/triage/#gitlabcom-db-structure-changes).
+- For tracking using Redis HLL (HyperLogLog):
+  - Check if a [feature flag is needed](implement.md#recommendations).
+- For a metric's YAML definition:
+  - Check the metric's `description`.
+  - Check the metric's `key_path`.
+  - Check the `product_section`, `product_stage`, and `product_group` fields.
+    Read the [stages file](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml).
+  - Check the file location. Consider the time frame, and if the file should be under `ee`.
+  - Check the tiers.
+- If a metric was changed or removed: Make sure the MR author notified the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) by `@` mentioning those groups in a comment on the issue for the MR and all of these groups have acknowledged the removal.
+- Metrics instrumentations
+  - Recommend using metrics instrumentation for new metrics, [if possible](metrics_instrumentation.md#support-for-instrumentation-classes).
+- Approve the MR, and relabel the MR with `~"analytics instrumentation::approved"`.
+
+## Review workload distribution
+
+[Danger bot](../../dangerbot.md) adds the list of changed Analytics Instrumentation files
+and pings the
+[`@gitlab-org/analytics-section/analytics-instrumentation/engineers`](https://gitlab.com/groups/gitlab-org/analytics-section/analytics-instrumentation/engineers/-/group_members?with_inherited_permissions=exclude) group for merge requests
+that are not drafts.
+
+Any of the Analytics Instrumentation engineers can be assigned for the Analytics Instrumentation review.
diff --git a/doc/development/internal_analytics/service_ping/troubleshooting.md b/doc/development/internal_analytics/service_ping/troubleshooting.md
new file mode 100644
index 00000000000..2b285b85bd0
--- /dev/null
+++ b/doc/development/internal_analytics/service_ping/troubleshooting.md
@@ -0,0 +1,164 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Troubleshooting Service Ping
+
+## Service Ping Payload drop
+
+### Symptoms
+
+You will be alerted by the [Data team](https://about.gitlab.com/handbook/business-technology/data-team/) and their [Monte Carlo alerting](https://about.gitlab.com/handbook/business-technology/data-team/platform/monte-carlo/).
+
+### Locating the problem
+
+First you need to identify at which stage in Service Ping data pipeline the drop is occurring.
+
+Start at [Service Ping Health Dashboard](https://app.periscopedata.com/app/gitlab/968489) on Sisense.
+
+You can use [this query](https://gitlab.com/gitlab-org/gitlab/-/issues/347298#note_836685350) as an example, to start detecting when the drop started.
+
+### Troubleshoot the GitLab application layer
+
+For results about an investigation conducted into an unexpected drop in Service ping Payload events volume, see [this issue](https://gitlab.com/gitlab-data/analytics/-/issues/11071).
+
+### Troubleshoot VersionApp layer
+
+Check if the [export jobs](https://gitlab.com/gitlab-services/version-gitlab-com#data-export-using-pipeline-schedules) are successful.
+
+Check [Service Ping errors](https://app.periscopedata.com/app/gitlab/968489?widget=14609989&udv=0) in the [Service Ping Health Dashboard](https://app.periscopedata.com/app/gitlab/968489).
+
+### Troubleshoot Google Storage layer
+
+Check if the files are present in [Google Storage](https://console.cloud.google.com/storage/browser/cloudsql-gs-production-efd5e8-cloudsql-exports;tab=objects?project=gs-production-efd5e8&prefix=&forceOnObjectsSortingFiltering=false).
+
+### Troubleshoot the data warehouse layer
+
+Reach out to the [Data team](https://about.gitlab.com/handbook/business-technology/data-team/) to ask about current state of data warehouse. On their handbook page there is a [section with contact details](https://about.gitlab.com/handbook/business-technology/data-team/#how-to-connect-with-us).
+
+### Cannot disable Service Ping with the configuration file
+
+The method to disable Service Ping with the GitLab configuration file does not work in
+GitLab versions 9.3.0 to 13.12.3. To disable it, you must use the Admin Area in
+the GitLab UI instead. For more information, see
+[this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/333269).
+
+GitLab functionality and application settings cannot override or circumvent
+restrictions at the network layer. If Service Ping is blocked by your firewall,
+you are not impacted by this bug.
+
+#### Check if you are affected
+
+You can check if you were affected by this bug by using the Admin Area or by
+checking the configuration file of your GitLab instance:
+
+- Using the Admin Area:
+
+  1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+  1. Select **Admin Area**.
+  1. On the left sidebar, select **Settings > Metrics and profiling**.
+  1. Expand **Usage statistics**.
+  1. Are you able to check or uncheck the checkbox to disable Service Ping?
+
+     - If _yes_, your GitLab instance is not affected by this bug.
+     - If you can't check or uncheck the checkbox, you are affected by this bug.
+       See the steps on [how to fix this](#how-to-fix-the-cannot-disable-service-ping-bug).
+
+- Checking your GitLab instance configuration file:
+
+  To check whether you're impacted by this bug, check your instance configuration
+  settings. The configuration file in which Service Ping can be disabled depends
+  on your installation and deployment method, but is typically one of the following:
+
+  - `/etc/gitlab/gitlab.rb` for Linux package installations and Docker.
+  - `charts.yaml` for GitLab Helm and cloud-native Kubernetes deployments.
+  - `gitlab.yml` for GitLab installations from source.
+
+  To check the relevant configuration file for strings that indicate whether
+  Service Ping is disabled, you can use `grep`:
+
+  ```shell
+  # Linux package
+  grep "usage_ping_enabled'\] = false" /etc/gitlab/gitlab.rb
+
+  # Kubernetes charts
+  grep "enableUsagePing: false" values.yaml
+
+  # From source
+  grep "usage_ping_enabled'\] = false" gitlab/config.yml
+  ```
+
+  If you see any output after running the relevant command, your GitLab instance
+  may be affected by the bug. Otherwise, your instance is not affected.
+
+#### How to fix the "Cannot disable Service Ping" bug
+
+To work around this bug, you have two options:
+
+- [Update](../../../update/index.md) to GitLab 13.12.4 or newer to fix this bug.
+- If you can't update to GitLab 13.12.4 or newer, enable Service Ping in the
+  configuration file, then disable Service Ping in the UI. For example, if you're
+  using the Linux package:
+
+  1. Edit `/etc/gitlab/gitlab.rb`:
+
+     ```ruby
+     gitlab_rails['usage_ping_enabled'] = true
+     ```
+
+  1. Reconfigure GitLab:
+
+     ```shell
+     sudo gitlab-ctl reconfigure
+     ```
+
+  1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+  1. Select **Admin Area**.
+  1. On the left sidebar, select **Settings > Metrics and profiling**.
+  1. Expand **Usage statistics**.
+  1. Clear the **Enable Service Ping** checkbox.
+  1. Select **Save Changes**.
+
+## Generate Service Ping
+
+### Generate or get the cached Service Ping in rails console
+
+Use the following method in the [rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session).
+
+```ruby
+Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values, cached: true)
+```
+
+### Generate a fresh new Service Ping
+
+Use the following method in the [rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session).
+
+This also refreshes the cached Service Ping displayed in the Admin Area.
+
+```ruby
+Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)
+```
+
+### Generate and print
+
+Generates Service Ping data in JSON format.
+
+```shell
+gitlab-rake gitlab:usage_data:generate
+```
+
+Generates Service Ping data in YAML format:
+
+```shell
+gitlab-rake gitlab:usage_data:dump_sql_in_yaml
+```
+
+### Generate and send Service Ping
+
+Prints the metrics saved in `conversational_development_index_metrics`.
+
+```shell
+gitlab-rake gitlab:usage_data:generate_and_send
+```
diff --git a/doc/development/internal_analytics/service_ping/usage_data.md b/doc/development/internal_analytics/service_ping/usage_data.md
new file mode 100644
index 00000000000..b6ec3e00670
--- /dev/null
+++ b/doc/development/internal_analytics/service_ping/usage_data.md
@@ -0,0 +1,69 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Usage Data Metrics guide
+
+This guide describes deprecated usage for metrics in `usage_data.rb`.
+
+NOTE:
+Implementing metrics direct in `usage_data.rb` is deprecated, We recommend you use [instrumentation classes](metrics_instrumentation.md).
+
+## Ordinary batch counters
+
+Simple count of a given `ActiveRecord_Relation`, does a non-distinct batch count, smartly reduces `batch_size`, and handles errors.
+Handles the `ActiveRecord::StatementInvalid` error.
+
+Method:
+
+```ruby
+count(relation, column = nil, batch: true, start: nil, finish: nil)
+```
+
+Arguments:
+
+- `relation` the ActiveRecord_Relation to perform the count
+- `column` the column to perform the count on, by default is the primary key
+- `batch`: default `true` to use batch counting
+- `start`: custom start of the batch counting to avoid complex min calculations
+- `end`: custom end of the batch counting to avoid complex min calculations
+
+Examples:
+
+```ruby
+count(User.active)
+count(::Clusters::Cluster.aws_installed.enabled, :cluster_id)
+count(::Clusters::Cluster.aws_installed.enabled, :cluster_id, start: ::Clusters::Cluster.minimum(:id), finish: ::Clusters::Cluster.maximum(:id))
+```
+
+## Distinct batch counters
+
+Distinct count of a given `ActiveRecord_Relation` on given column, a distinct batch count, smartly reduces `batch_size`, and handles errors.
+Handles the `ActiveRecord::StatementInvalid` error.
+
+Method:
+
+```ruby
+distinct_count(relation, column = nil, batch: true, batch_size: nil, start: nil, finish: nil)
+```
+
+Arguments:
+
+- `relation`: the ActiveRecord_Relation to perform the count
+- `column`: the column to perform the distinct count, by default is the primary key
+- `batch`: default `true` to use batch counting
+- `batch_size`: if none set it uses default value 10000 from `Gitlab::Database::BatchCounter`
+- `start`: custom start of the batch counting to avoid complex min calculations
+- `end`: custom end of the batch counting to avoid complex min calculations
+
+WARNING:
+Counting over non-unique columns can lead to performance issues. For more information, see the [iterating tables in batches](../../database/iterating_tables_in_batches.md) guide.
+
+Examples:
+
+```ruby
+distinct_count(::Project, :creator_id)
+distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id))
+```
diff --git a/doc/development/internal_analytics/snowplow/event_dictionary_guide.md b/doc/development/internal_analytics/snowplow/event_dictionary_guide.md
new file mode 100644
index 00000000000..6e8947e0210
--- /dev/null
+++ b/doc/development/internal_analytics/snowplow/event_dictionary_guide.md
@@ -0,0 +1,91 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Event dictionary guide
+
+NOTE:
+The event dictionary is a work in progress, and this process is subject to change.
+
+This guide describes the event dictionary and how it's implemented.
+
+## Event definition and validation
+
+This process is meant to document all Snowplow events and ensure consistency. Every Snowplow event needs to have such a definition. Event definitions must comply with the [JSON Schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/events/schema.json).
+
+All event definitions are stored in the following directories:
+
+- [`config/events`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/events)
+- [`ee/config/events`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/config/events)
+
+Each event is defined in a separate YAML file consisting of the following fields:
+
+| Field                  | Required | Additional information                                                                                                                                                                      |
+|------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `description`          | yes      | A description of the event.                                                                                                                                                                 |
+| `category`             | yes      | The event category (see [Event schema](index.md#event-schema)).                                                                                                                             |
+| `action`               | yes      | The event action (see [Event schema](index.md#event-schema)).                                                                                                                               |
+| `label_description`    | no       | A description of the event label (see [Event schema](index.md#event-schema)).                                                                                                               |
+| `property_description` | no       | A description of the event property (see [Event schema](index.md#event-schema)).                                                                                                            |
+| `value_description`    | no       | A description of the event value (see [Event schema](index.md#event-schema)).                                                                                                               |
+| `extra_properties`     | no       | The type and description of each extra property sent with the event.                                                                                                                        |
+| `identifiers`          | no       | A list of identifiers sent with the event. Can be set to one or more of `project`, `user`, or `namespace`.                                                                                  |
+| `iglu_schema_url`      | no       | The URL to the custom schema sent with the event, for example, `iglu:com.gitlab/gitlab_experiment/jsonschema/1-0-0`.                                                                        |
+| `product_section`      | yes      | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml).                                                                                                |
+| `product_stage`        | no       | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the event.                                                                                        |
+| `product_group`        | yes      | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the event.                                                                                  |
+| `milestone`            | no       | The milestone when the event is introduced.                                                                                                                                                 |
+| `introduced_by_url`    | no       | The URL to the merge request that introduced the event.                                                                                                                                     |
+| `distributions`        | yes      | The [distributions](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. Can be set to one or more of `ce` or `ee`.  |
+| `tiers`                | yes      | The [tiers](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/) where the tracked feature is available. Can be set to one or more of `free`, `premium`, or `ultimate`. |
+
+### Example event definition
+
+The linked [`uuid`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/events/epics_promote.yml)
+YAML file includes an example event definition.
+
+```yaml
+description: Issue promoted to epic
+category: epics
+action: promote
+property_description: The string "issue_id"
+value_description: ID of the issue
+extra_properties:
+  weight:
+    type: integer
+    description: Weight of the issue
+identifiers:
+- project
+- user
+- namespace
+product_section: dev
+product_stage: plan
+product_group: group::product planning
+milestone: "11.10"
+introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/10537
+distributions:
+- ee
+tiers:
+- premium
+- ultimate
+```
+
+## Create a new event definition
+
+Use the dedicated [event definition generator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/snowplow_event_definition_generator.rb)
+to create new event definitions.
+
+The `category` and `action` of each event are included in the filename to standardize file naming.
+
+The generator takes three options:
+
+- `--ee`: Indicates if the event is for EE.
+- `--category=CATEGORY`: Indicates the `category` of the event.
+- `--action=ACTION`: Indicates the `action` of the event.
+
+```shell
+bundle exec rails generate gitlab:snowplow_event_definition --category Groups::EmailCampaignsController --action click
+create  create  config/events/groups__email_campaigns_controller_click.yml
+```
diff --git a/doc/development/internal_analytics/snowplow/implementation.md b/doc/development/internal_analytics/snowplow/implementation.md
new file mode 100644
index 00000000000..5ad97cf528c
--- /dev/null
+++ b/doc/development/internal_analytics/snowplow/implementation.md
@@ -0,0 +1,523 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Implement Snowplow tracking
+
+This page describes how to:
+
+- Implement Snowplow frontend and backend tracking
+- Test Snowplow events
+
+## Event definitions
+
+Every Snowplow event, regardless of frontend or backend, requires a corresponding event definition. These definitions document the event and its properties to make it easier to maintain and analyze.
+These definitions can be browsed in the [event dictionary](https://metrics.gitlab.com/snowplow/). The [event dictionary guide](event_dictionary_guide.md) provides instructions for setting up an event definition.
+
+## Snowplow JavaScript frontend tracking
+
+GitLab provides a `Tracking` interface that wraps the [Snowplow JavaScript tracker](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/)
+to track custom events.
+
+For the recommended frontend tracking implementation, see [Usage recommendations](#usage-recommendations).
+
+Structured events and page views include the [`gitlab_standard`](schemas.md#gitlab_standard)
+context, using the `window.gl.snowplowStandardContext` object which includes
+[default data](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/views/layouts/_snowplow.html.haml)
+as base:
+
+| Property | Example |
+| -------- | ------- |
+| `context_generated_at` | `"2022-01-01T01:00:00.000Z"` |
+| `environment` | `"production"` |
+| `extra` | `{}` |
+| `namespace_id` | `123` |
+| `plan` | `"gold"` |
+| `project_id` | `456` |
+| `source` | `"gitlab-rails"` |
+| `user_id` | `789`* |
+| `is_gitlab_team_member` | `true`|
+
+_\* Undergoes a pseudonymization process at the collector level._
+
+These properties [are overridden](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/get_standard_context.js)
+with frontend-specific values, like `source` (`gitlab-javascript`), `google_analytics_id`
+and the custom `extra` object. You can modify this object for any subsequent
+structured event that fires, although this is not recommended.
+
+Tracking implementations must have an `action` and a `category`. You can provide additional
+properties from the [event schema](index.md#event-schema), in
+addition to an `extra` object that accepts key-value pairs.
+
+| Property      | Type   | Default value              | Description                                                                                                                                                                                 |
+|:-----------|:-------|:---------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `category` | string | `document.body.dataset.page` | Page or subsection of a page in which events are captured.                                                                                                                                  |
+| `action`   | string | `'generic'`                  | Action the user is taking. Clicks must be `click` and activations must be `activate`. For example, focusing a form field is `activate_form_input`, and clicking a button is `click_button`. |
+| `data`     | object | `{}`                         | Additional data such as `label`, `property`, `value` as described in [Event schema](index.md#event-schema), `context` for custom contexts, and `extra` (key-value pairs object).            |
+
+### Usage recommendations
+
+- Use [data attributes](#implement-data-attribute-tracking) on HTML elements that emit `click`, `show.bs.dropdown`, or `hide.bs.dropdown` events.
+- Use the [Vue mixin](#implement-vue-component-tracking) for tracking custom events, or if the supported events for data attributes are not propagating. For example, clickable components that don't emit `click`.
+- Use the [tracking class](#implement-raw-javascript-tracking) when tracking in vanilla JavaScript files.
+
+### Implement data attribute tracking
+
+To implement tracking for HAML or Vue templates, add a [`data-track` attribute](#data-track-attributes) to the element.
+
+The following example shows `data-track-*` attributes assigned to a button:
+
+```haml
+%button.btn{ data: { track_action: "click_button", track_label: "template_preview", track_property: "my-template" } }
+```
+
+```html
+<button class="btn"
+  data-track-action="click_button"
+  data-track-label="template_preview"
+  data-track-property="my-template"
+  data-track-extra='{ "template_variant": "primary" }'
+/>
+```
+
+#### `data-track` attributes
+
+| Attribute             | Required | Description |
+|:----------------------|:---------|:------------|
+| `data-track-action`    | true     | Action the user is taking. Clicks must be prepended with `click` and activations must be prepended with `activate`. For example, focusing a form field is `activate_form_input` and clicking a button is `click_button`. Replaces `data-track-event`, which was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/290962) in GitLab 13.11. |
+| `data-track-label`    | false    | The specific element or object to act on. This can be: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown list; or the name or title attribute of a record being created. |
+| `data-track-property` | false    | Any additional property of the element, or object being acted on. |
+| `data-track-value`    | false    | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. If omitted, this is the element's `value` property or `undefined`. For checkboxes, the default value is the element's checked attribute or `0` when unchecked. The value is parsed as numeric before sending the event. |
+| `data-track-extra` | false    | A key-value pair object passed as a valid JSON string. This attribute is added to the `extra` property in our [`gitlab_standard`](schemas.md#gitlab_standard) schema. |
+| `data-track-context`  | false    | To append a custom context object, passed as a valid JSON string. |
+
+#### Event listeners
+
+Event listeners bind at the document level to handle click events in elements with data attributes.
+This allows them to be handled when the DOM re-renders or changes. Document-level binding reduces
+the likelihood that click events stop propagating up the DOM tree.
+
+If click events stop propagating, you must implement listeners and [Vue component tracking](#implement-vue-component-tracking) or [raw JavaScript tracking](#implement-raw-javascript-tracking).
+
+#### Helper methods
+
+You can use the following Ruby helpers:
+
+```ruby
+tracking_attrs(label, action, property) # { data: { track_label... } }
+
+tracking_attrs_data(label, action, property) # { track_label... }
+```
+
+You can also use it on HAML templates:
+
+```haml
+%button{ **tracking_attrs('main_navigation', 'click_button', 'navigation') }
+
+// When merging with additional data
+// %button{ data: { platform: "...", **tracking_attrs_data('main_navigation', 'click_button', 'navigation') } }
+```
+
+If you use the GitLab helper method [`nav_link`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/helpers/tab_helper.rb#L76), you must wrap `html_options` under the `html_options` keyword argument. If you
+use the `ActionView` helper method [`link_to`](https://api.rubyonrails.org/classes/ActionView/Helpers/UrlHelper.html#method-i-link_to), you don't need to wrap `html_options`.
+
+```ruby
+# Bad
+= nav_link(controller: ['dashboard/groups', 'explore/groups'], data: { track_label: "explore_groups",
+track_action: "click_button" })
+
+# Good
+= nav_link(controller: ['dashboard/groups', 'explore/groups'], html_options: { data: { track_label:
+"explore_groups", track_action: "click_button" } })
+
+# Good (other helpers)
+= link_to explore_groups_path, title: _("Explore"), data: { track_label: "explore_groups", track_action:
+"click_button" }
+```
+
+### Implement Vue component tracking
+
+For custom event tracking, use the [Vue mixin](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/tracking.js#L207). It exposes `Tracking.event` as the `track` method.
+You can specify tracking options by creating a `tracking` data object or
+computed property, and as a second parameter: `this.track('click_button', opts)`.
+These options override any defaults and allow the values to be dynamic from props or based on state:
+
+| Property | Type | Default | Example |
+| -- | -- | -- | -- |
+| `category` | string | `document.body.dataset.page` | `'code_quality_walkthrough'` |
+| `label` | string | `''` | `'process_start_button'` |
+| `property` | string | `''` | `'asc'` or `'desc'` |
+| `value` | integer | `undefined` | `0`, `1`, `500` |
+| `extra` | object | `{}` | `{ selectedVariant: this.variant }` |
+
+To implement Vue component tracking:
+
+1. Import the `Tracking` library and call the `mixin` method:
+
+   ```javascript
+   import Tracking from '~/tracking';
+
+   const trackingMixin = Tracking.mixin();
+
+   // Optionally provide default properties
+   // const trackingMixin = Tracking.mixin({ label: 'right_sidebar' });
+   ```
+
+1. Use the mixin in the component:
+
+   ```javascript
+   export default {
+     mixins: [trackingMixin],
+     // Or
+     // mixins: [Tracking.mixin()],
+     // mixins: [Tracking.mixin({ label: 'right_sidebar' })],
+
+     data() {
+       return {
+         expanded: false,
+       };
+     },
+   };
+   ```
+
+1. You can specify tracking options in by creating a `tracking` data object
+or computed property:
+
+   ```javascript
+   export default {
+     name: 'RightSidebar',
+
+     mixins: [Tracking.mixin()],
+
+     data() {
+       return {
+         expanded: false,
+         variant: '',
+         tracking: {
+           label: 'right_sidebar',
+           // property: '',
+           // value: '',
+           // experiment: '',
+           // extra: {},
+         },
+       };
+     },
+
+     // Or
+     // computed: {
+     //   tracking() {
+     //     return {
+     //       property: this.variant,
+     //       extra: { expanded: this.expanded },
+     //     };
+     //   },
+     // },
+   };
+   ```
+
+1. Call the `track` method. Tracking options can be passed as the second parameter:
+
+   ```javascript
+   this.track('click_button', {
+     label: 'right_sidebar',
+   });
+   ```
+
+   Or use the `track` method in the template:
+
+   ```html
+   <template>
+     <div>
+       <button data-testid="toggle" @click="toggle">Toggle</button>
+
+       <div v-if="expanded">
+         <p>Hello world!</p>
+         <button @click="track('click_button')">Track another event</button>
+       </div>
+     </div>
+   </template>
+   ```
+
+#### Testing example
+
+```javascript
+export default {
+  name: 'CountDropdown',
+
+  mixins: [Tracking.mixin({ label: 'count_dropdown' })],
+
+  data() {
+    return {
+      variant: 'counter',
+      count: 0,
+    };
+  },
+
+  methods: {
+    handleChange({ target }) {
+      const { variant } = this;
+
+      this.count = Number(target.value);
+
+      this.track('change_value', {
+        value: this.count,
+        extra: { variant }
+      });
+    },
+  },
+};
+```
+
+```javascript
+import { mockTracking } from 'helpers/tracking_helper';
+// mockTracking(category, documentOverride, spyMethod)
+
+describe('CountDropdown.vue', () => {
+  let trackingSpy;
+  let wrapper;
+
+  ...
+
+  beforeEach(() => {
+    trackingSpy = mockTracking(undefined, wrapper.element, jest.spyOn);
+  });
+
+  const findDropdown = () => wrapper.find('[data-testid="dropdown"]');
+
+  it('tracks change event', () => {
+    const dropdown = findDropdown();
+    dropdown.element.value = 30;
+    dropdown.trigger('change');
+
+    expect(trackingSpy).toHaveBeenCalledWith(undefined, 'change_value', {
+      value: 30,
+      label: 'count_dropdown',
+      extra: { variant: 'counter' },
+    });
+  });
+});
+```
+
+### Implement raw JavaScript tracking
+
+To track from a vanilla JavaScript file, use the `Tracking.event` static function
+(calls [`dispatchSnowplowEvent`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/dispatch_snowplow_event.js)).
+
+The following example demonstrates tracking a click on a button by manually calling `Tracking.event`.
+
+```javascript
+import Tracking from '~/tracking';
+
+const button = document.getElementById('create_from_template_button');
+
+button.addEventListener('click', () => {
+  Tracking.event(undefined, 'click_button', {
+    label: 'create_from_template',
+    property: 'template_preview',
+    extra: {
+      templateVariant: 'primary',
+      valid: 1,
+    },
+  });
+});
+```
+
+#### Testing example
+
+```javascript
+import Tracking from '~/tracking';
+
+describe('MyTracking', () => {
+  let wrapper;
+
+  beforeEach(() => {
+    jest.spyOn(Tracking, 'event');
+  });
+
+  const findButton = () => wrapper.find('[data-testid="create_from_template"]');
+
+  it('tracks event', () => {
+    findButton().trigger('click');
+
+    expect(Tracking.event).toHaveBeenCalledWith(undefined, 'click_button', {
+      label: 'create_from_template',
+      property: 'template_preview',
+      extra: {
+        templateVariant: 'primary',
+        valid: true,
+      },
+    });
+  });
+});
+```
+
+### Form tracking
+
+To enable Snowplow automatic [form tracking](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracking-specific-events/#form-tracking):
+
+1. Call `Tracking.enableFormTracking` when the DOM is ready.
+1. Provide a `config` object that includes at least one of the following elements:
+    - `forms` determines the forms to track. Identified by the CSS class name.
+    - `fields` determines the fields inside the tracked forms to track. Identified by the field `name`.
+1. Optional. Provide a list of contexts as the second argument. The [`gitlab_standard`](schemas.md#gitlab_standard) schema is excluded from these events.
+
+```javascript
+Tracking.enableFormTracking({
+  forms: { allow: ['sign-in-form', 'password-recovery-form'] },
+  fields: { allow: ['terms_and_conditions', 'newsletter_agreement'] },
+});
+```
+
+#### Testing example
+
+```javascript
+import Tracking from '~/tracking';
+
+describe('MyFormTracking', () => {
+  let formTrackingSpy;
+
+  beforeEach(() => {
+    formTrackingSpy = jest
+      .spyOn(Tracking, 'enableFormTracking')
+      .mockImplementation(() => null);
+  });
+
+  it('initialized with the correct configuration', () => {
+    expect(formTrackingSpy).toHaveBeenCalledWith({
+      forms: { allow: ['sign-in-form', 'password-recovery-form'] },
+      fields: { allow: ['terms_and_conditions', 'newsletter_agreement'] },
+    });
+  });
+});
+```
+
+## Implement Ruby backend tracking
+
+`Gitlab::Tracking` is an interface that wraps the [Snowplow Ruby Tracker](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/ruby-tracker/) for tracking custom events.
+Backend tracking provides:
+
+- User behavior tracking
+- Instrumentation to monitor and visualize performance over time in a section or aspect of code.
+
+To add custom event tracking and instrumentation, call the `GitLab::Tracking.event` class method.
+For example:
+
+```ruby
+class Projects::CreateService < BaseService
+  def execute
+    project = Project.create(params)
+
+    Gitlab::Tracking.event('Projects::CreateService', 'create_project', label: project.errors.full_messages.to_sentence,
+                           property: project.valid?.to_s, project: project, user: current_user, namespace: namespace)
+  end
+end
+```
+
+Use the following arguments:
+
+| Argument   | Type                      | Default value | Description                                                                                                                       |
+|------------|---------------------------|---------------|-----------------------------------------------------------------------------------------------------------------------------------|
+| `category` | String                    |               | Area or aspect of the application. For example,  `HealthCheckController` or `Lfs::FileTransformer`.                  |
+| `action`   | String                    |               | The action being taken. For example, a controller action such as `create`, or an Active Record callback. |
+| `label`    | String                    | `nil`           | The specific element or object to act on. This can be one of the following: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown list; or the name or title attribute of a record being created.                                                          |
+| `property` | String                    | `nil`           | Any additional property of the element, or object being acted on.                                                          |
+| `value`    | Numeric                   | `nil`           | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility.                                                          |
+| `context`  | Array\[SelfDescribingJSON\] | `nil`           | An array of custom contexts to send with this event. Most events should not have any custom contexts.                             |
+| `project`  | Project                   | `nil`           | The project associated with the event. |
+| `user`     | User                      | `nil`           | The user associated with the event. This value undergoes a pseudonymization process at the collector level. |
+| `namespace` | Namespace                | `nil`           | The namespace associated with the event. |
+| `extra`   | Hash                | `{}`         | Additional keyword arguments are collected into a hash and sent with the event. |
+
+### Unit testing
+
+To test backend Snowplow events, use the `expect_snowplow_event` helper. For more information, see
+[testing best practices](../../testing_guide/best_practices.md#test-snowplow-events).
+
+### Performance
+
+We use the [AsyncEmitter](https://snowplow.github.io/snowplow-ruby-tracker/SnowplowTracker/AsyncEmitter.html) when tracking events, which allows for instrumentation calls to be run in a background thread. This is still an active area of development.
+
+## Develop and test Snowplow
+
+To develop and test a Snowplow event, there are several tools to test frontend and backend events:
+
+| Testing Tool                                 | Frontend Tracking  | Backend Tracking    | Local Development Environment | Production Environment | Production Environment |
+|----------------------------------------------|--------------------|---------------------|-------------------------------|------------------------|------------------------|
+| Snowplow Analytics Debugger Chrome Extension | Yes | No | Yes            | Yes     | Yes     |
+| Snowplow Inspector Chrome Extension          | Yes | No | Yes            | Yes     | Yes     |
+| Snowplow Micro                               | Yes | Yes  | Yes            | No    | No    |
+
+### Test frontend events
+
+Before you test frontend events in development, you must:
+
+1. [Enable Snowplow tracking in the Admin Area](index.md#enable-snowplow-tracking).
+1. Turn off ad blockers that could prevent Snowplow JavaScript from loading in your environment.
+1. Turn off "Do Not Track" (DNT) in your browser.
+
+All URLs are pseudonymized. The entity identifier [replaces](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracker-setup/other-parameters-2/#setting-a-custom-page-url-and-referrer-url) personally identifiable
+information (PII). PII includes usernames, group, and project names.
+Page titles are hardcoded as `GitLab` for the same reason.
+
+#### Snowplow Analytics Debugger Chrome Extension
+
+[Snowplow Analytics Debugger](https://www.iglooanalytics.com/blog/snowplow-analytics-debugger-chrome-extension.html) is a browser extension for testing frontend events. It works in production, staging, and local development environments.
+
+1. Install the [Snowplow Analytics Debugger](https://chrome.google.com/webstore/detail/snowplow-analytics-debugg/jbnlcgeengmijcghameodeaenefieedm) Chrome browser extension.
+1. Open Chrome DevTools to the Snowplow Analytics Debugger tab.
+
+#### Snowplow Inspector Chrome Extension
+
+Snowplow Inspector Chrome Extension is a browser extension for testing frontend events. This works in production, staging, and local development environments.
+
+<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
+For a video tutorial, see the [Snowplow plugin walk through](https://www.youtube.com/watch?v=g4rqnIZ1Mb4).
+
+1. Install [Snowplow Inspector](https://chrome.google.com/webstore/detail/snowplow-inspector/maplkdomeamdlngconidoefjpogkmljm?hl=en).
+1. To open the extension, select the Snowplow Inspector icon beside the address bar.
+1. Click around on a webpage with Snowplow to see JavaScript events firing in the inspector window.
+
+### Test backend events with Snowplow Micro
+
+[Snowplow Micro](https://snowplow.io/blog/introducing-snowplow-micro/) is a
+Docker-based solution for testing backend and frontend in a local development environment. Snowplow Micro
+records the same events as the full Snowplow pipeline. To query events, use the Snowplow Micro API.
+
+It can be set up automatically using [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit).
+See the [how-to docs](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/snowplow_micro.md) for more details.
+
+1. Set the environment variable to tell the GDK to use Snowplow Micro in development. This overrides two `application_settings` options:
+   - `snowplow_enabled` setting will instead return `true` from `Gitlab::Tracking.enabled?`
+   - `snowplow_collector_hostname` setting will instead always return `localhost:9090` (or whatever port is set for `snowplow_micro.port` GDK setting) from `Gitlab::Tracking.collector_hostname`.
+With Snowplow Micro set up you can now manually test backend Snowplow events:
+
+1. Send a test Snowplow event from the Rails console:
+
+   ```ruby
+   Gitlab::Tracking.event('category', 'action')
+   ```
+
+1. Navigate to `localhost:9090/micro/good` to see the event.
+
+#### Useful links
+
+- [Snowplow Micro repository](https://github.com/snowplow-incubator/snowplow-micro)
+- [Installation guide recording](https://www.youtube.com/watch?v=OX46fo_A0Ag)
+
+### Troubleshoot
+
+To control content security policy warnings when using an external host, modify `config/gitlab.yml`
+to allow or prevent them. To allow them, add the relevant host for `connect_src`. For example, for
+`https://snowplow.trx.gitlab.net`:
+
+```yaml
+development:
+  <<: *base
+  gitlab:
+    content_security_policy:
+      enabled: true
+      directives:
+        connect_src: "'self' http://localhost:* http://127.0.0.1:* ws://localhost:* wss://localhost:* ws://127.0.0.1:* https://snowplow.trx.gitlab.net/"
+```
diff --git a/doc/development/internal_analytics/snowplow/index.md b/doc/development/internal_analytics/snowplow/index.md
new file mode 100644
index 00000000000..8265bceaf06
--- /dev/null
+++ b/doc/development/internal_analytics/snowplow/index.md
@@ -0,0 +1,201 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Snowplow development guidelines
+
+Snowplow is an enterprise-grade marketing and Analytics Instrumentation platform that tracks how users engage with our website and application.
+
+[Snowplow](https://snowplow.io/) consists of several loosely-coupled sub-systems:
+
+- **Trackers** fire Snowplow events. Snowplow has twelve trackers that cover web, mobile, desktop, server, and IoT.
+- **Collectors** receive Snowplow events from trackers. We use different event collectors that synchronize events to Amazon S3, Apache Kafka, or Amazon Kinesis.
+- **Enrich** cleans raw Snowplow events, enriches them, and puts them into storage. There is a Hadoop-based enrichment process, and a Kinesis-based or Kafka-based process.
+- **Storage** stores Snowplow events. We store the Snowplow events in a flat file structure on S3, and in the Redshift and PostgreSQL databases.
+- **Data modeling** joins event-level data with other data sets, aggregates them into smaller data sets, and applies business logic. This produces a clean set of tables for data analysis. We use data models for Redshift and Looker.
+- **Analytics** are performed on Snowplow events or on aggregate tables.
+
+![Snowplow flow](../../img/snowplow_flow.png)
+
+## Enable Snowplow tracking
+
+Tracking can be enabled at:
+
+- The instance level, which enables tracking on both the frontend and backend layers.
+- The user level. User tracking can be disabled on a per user basis.
+  GitLab respects the [Do Not Track](https://www.eff.org/issues/do-not-track) standard, so any user who has enabled the Do Not Track option in their browser is not tracked at a user level.
+
+Snowplow tracking is configured to send data for GitLab.com to a collector configured by GitLab. By default, self-managed
+instances do not have a collector configured and do not collect data via Snowplow.
+
+You can configure your self-managed GitLab instance to use a custom Snowplow collector.
+
+1. On the left sidebar, expand the top-most chevron (**{chevron-down}**).
+1. Select **Admin Area**.
+1. On the left sidebar, select **Settings > General**.
+1. Expand **Snowplow**.
+1. Select **Enable Snowplow tracking** and enter your Snowplow configuration information. For example:
+
+   | Name               | Value                         |
+   |--------------------|-------------------------------|
+   | Collector hostname | `your-snowplow-collector.net` |
+   | App ID             | `gitlab`                      |
+   | Cookie domain      | `.your-gitlab-instance.com`   |
+
+1. Select **Save changes**.
+
+## Snowplow request flow
+
+The following example shows a basic request/response flow between the following components:
+
+- Snowplow JS / Ruby Trackers on GitLab.com
+- [GitLab.com Snowplow Collector](https://gitlab.com/gitlab-com/gl-infra/readiness/-/blob/master/library/snowplow/index.md)
+- The GitLab S3 Bucket
+- The GitLab Snowflake Data Warehouse
+- Sisense:
+
+```mermaid
+sequenceDiagram
+    participant Snowplow JS (Frontend)
+    participant Snowplow Ruby (Backend)
+    participant GitLab.com Snowplow Collector
+    participant S3 Bucket
+    participant Snowflake DW
+    participant Sisense Dashboards
+    Snowplow JS (Frontend) ->> GitLab.com Snowplow Collector: FE Tracking event
+    Snowplow Ruby (Backend) ->> GitLab.com Snowplow Collector: BE Tracking event
+    loop Process using Kinesis Stream
+      GitLab.com Snowplow Collector ->> GitLab.com Snowplow Collector: Log raw events
+      GitLab.com Snowplow Collector ->> GitLab.com Snowplow Collector: Enrich events
+      GitLab.com Snowplow Collector ->> GitLab.com Snowplow Collector: Write to disk
+    end
+    GitLab.com Snowplow Collector ->> S3 Bucket: Kinesis Firehose
+    Note over GitLab.com Snowplow Collector, S3 Bucket: Pseudonymization
+    S3 Bucket->>Snowflake DW: Import data
+    Snowflake DW->>Snowflake DW: Transform data using dbt
+    Snowflake DW->>Sisense Dashboards: Data available for querying
+```
+
+For more details about the architecture, see [Snowplow infrastructure](infrastructure.md).
+
+## Event schema
+
+All the events must be consistent. If each feature captures events differently, it can be difficult
+to perform analysis.
+
+Each event provides attributes that describe the event.
+
+| Attribute | Type    | Required | Description |
+| --------- | ------- | -------- | ----------- |
+| category  | text    | true     | The page or backend section of the application. Unless infeasible, use the Rails page attribute by default in the frontend, and namespace + class name on the backend, for example, `Notes::CreateService`. |
+| action    | text    | true     | The action the user takes, or aspect that's being instrumented. The first word must describe the action or aspect. For example, clicks must be `click`, activations must be `activate`, creations must be `create`. Use underscores to describe what was acted on. For example, activating a form field is `activate_form_input`, an interface action like clicking on a dropdown list is `click_dropdown`, a behavior like creating a project record from the backend is `create_project`. |
+| label     | text    | false    | The specific element or object to act on. This can be one of the following: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown list; or the name or title attribute of a record being created. For Service Ping metrics adapted to Snowplow events, this should be the full metric [key path](../service_ping/metrics_dictionary.md#metric-key_path) taken from its definition file. |
+| property  | text    | false    | Any additional property of the element, or object being acted on. For Service Ping metrics adapted to Snowplow events, this should be additional information or context that can help analyze the event. For example, in the case of `usage_activity_by_stage_monthly.create.merge_requests_users`, there are four different possible merge request actions: "create", "merge", "comment", and "close". Each of these would be a possible property value. |
+| value     | decimal | false    | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. |
+| context   | vector  | false    | Additional data in the form of a [self-describing JSON](https://docs.snowplow.io/docs/pipeline-components-and-applications/iglu/common-architecture/self-describing-json-schemas/) to describe the event if the attributes are not sufficient. Each context must have its schema defined to assure data integrity. Refer to the list of GitLab-defined contexts for more details. |
+
+### Examples
+
+| Category* | Label            | Action                | Property** | Value |
+|-------------|------------------|-----------------------|----------|:-----:|
+| `[root:index]` | `main_navigation`            | `click_navigation_link` | `[link_label]`   | - |
+| `[groups:boards:show]` | `toggle_swimlanes` | `click_toggle_button` | - | `[is_active]` |
+| `[projects:registry:index]` | `registry_delete` | `click_button` | - | - |
+| `[projects:registry:index]` | `registry_delete` | `confirm_deletion` | - | - |
+| `[projects:blob:show]` | `congratulate_first_pipeline` | `click_button` | `[human_access]` | - |
+| `[projects:clusters:new]` | `chart_options` | `generate_link` | `[chart_link]` | - |
+| `[projects:clusters:new]` | `chart_options` | `click_add_label_button` | `[label_id]` | - |
+| `API::NpmPackages` | `counts.package_events_i_package_push_package_by_deploy_token` | `push_package` | `npm` | - |
+
+_* If you choose to omit the category you can use the default._<br>
+_** Use property for variable strings._
+
+### Reference SQL
+
+#### Last 20 `reply_comment_button` events
+
+```sql
+SELECT
+  session_id,
+  event_id,
+  event_label,
+  event_action,
+  event_property,
+  event_value,
+  event_category,
+  contexts
+FROM legacy.snowplow_structured_events_all
+WHERE
+  event_label = 'reply_comment_button'
+  AND event_action = 'click_button'
+  -- AND event_category = 'projects:issues:show'
+  -- AND event_value = 1
+ORDER BY collector_tstamp DESC
+LIMIT 20
+```
+
+#### Last 100 page view events
+
+```sql
+SELECT
+  -- page_url,
+  -- page_title,
+  -- referer_url,
+  -- marketing_medium,
+  -- marketing_source,
+  -- marketing_campaign,
+  -- browser_window_width,
+  -- device_is_mobile
+  *
+FROM legacy.snowplow_page_views_30
+ORDER BY page_view_start DESC
+LIMIT 100
+```
+
+#### Top 20 users who fired `reply_comment_button` in the last 30 days
+
+```sql
+SELECT
+  count(*) as hits,
+  se_action,
+  se_category,
+  gsc_pseudonymized_user_id
+FROM legacy.snowplow_gitlab_events_30
+WHERE
+  se_label = 'reply_comment_button'
+  AND gsc_pseudonymized_user_id IS NOT NULL
+GROUP BY gsc_pseudonymized_user_id, se_category, se_action
+ORDER BY count(*) DESC
+LIMIT 20
+```
+
+#### Query JSON formatted data
+
+```sql
+SELECT
+  derived_tstamp,
+  contexts:data[0]:data:extra:old_format as CURRENT_FORMAT,
+  contexts:data[0]:data:extra:value as UPDATED_FORMAT
+FROM legacy.snowplow_structured_events_all
+WHERE event_action in ('wiki_format_updated')
+ORDER BY derived_tstamp DESC
+LIMIT 100
+```
+
+### Web-specific parameters
+
+Snowplow JavaScript adds [web-specific parameters](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/snowplow-tracker-protocol/#Web-specific_parameters) to all web events by default.
+
+## Related topics
+
+- [Snowplow data structure](https://docs.snowplow.io/docs/understanding-your-pipeline/canonical-event/)
+- [Our Iglu schema registry](https://gitlab.com/gitlab-org/iglu)
+- [List of events used in our codebase (Event Dictionary)](https://metrics.gitlab.com/snowplow/)
+- [Analytics Instrumentation Guide](https://about.gitlab.com/handbook/product/analytics-instrumentation-guide/)
+- [Service Ping Guide](../service_ping/index.md)
+- [Analytics Instrumentation Direction](https://about.gitlab.com/direction/analytics/analytics-instrumentation/)
+- [Data Analysis Process](https://about.gitlab.com/handbook/business-technology/data-team/#data-analysis-process/)
+- [Data for Product Managers](https://about.gitlab.com/handbook/business-technology/data-team/programs/data-for-product-managers/)
+- [Data Infrastructure](https://about.gitlab.com/handbook/business-technology/data-team/platform/infrastructure/)
diff --git a/doc/development/internal_analytics/snowplow/infrastructure.md b/doc/development/internal_analytics/snowplow/infrastructure.md
new file mode 100644
index 00000000000..9679abac6b7
--- /dev/null
+++ b/doc/development/internal_analytics/snowplow/infrastructure.md
@@ -0,0 +1,101 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Snowplow infrastructure
+
+Snowplow events on GitLab SaaS fired by a [tracker](implementation.md) go through an AWS pipeline, managed by GitLab.
+
+## Event flow in the AWS pipeline
+
+Every event goes through a collector, enricher, and pseudonymization lambda. The event is then dumped to S3 storage where it can be picked up by the Snowflake data warehouse.
+
+Deploying and managing the infrastructure is automated using Terraform in the current [Terraform repository](https://gitlab.com/gitlab-com/gl-infra/config-mgmt/-/tree/master/environments/aws-snowplow).
+
+```mermaid
+graph LR
+    GL[GitLab.com]-->COL
+
+    subgraph aws-cloud[AWS]
+    COL[Collector]-->|snowplow-raw-good|ENR
+    COL[Collector]-->|snowplow-raw-bad|FRBE
+        subgraph firehoserbe[Firehose]
+        FRBE[AWS Lambda]
+      end
+    FRBE-->S3RBE
+
+    ENR[Enricher]-->|snowplow-enriched-bad|FEBE
+      subgraph firehoseebe[Firehose]
+        FEBE[AWS Lambda]
+      end
+    FEBE-->S3EBE
+
+    ENR[Enricher]-->|snowplow-enriched-good|FRGE
+      subgraph firehosege[Firehose]
+        FRGE[AWS Lambda]
+      end
+    FRGE-->S3GE
+  end
+
+    subgraph snowflake[Data warehouse]
+    S3RBE[S3 raw-bad]-->BE[gitlab_bad_events]
+    S3EBE[S3 enriched-bad]-->BE[gitlab_bad_events]
+    S3GE[S3 output]-->GE[gitlab_events]
+  end
+```
+
+See [Snowplow technology 101](https://github.com/snowplow/snowplow/#snowplow-technology-101) for Snowplow's own documentation and an overview how collectors and enrichers work.
+
+### Pseudonymization
+
+In contrast to a typical Snowplow pipeline, after enrichment, GitLab Snowplow events go through a [pseudonymization service](https://gitlab.com/gitlab-org/analytics-section/analytics-instrumentation/snowplow-pseudonymization) in the form of an AWS Lambda service before they are stored in S3 storage.
+
+#### Why events need to be pseudonymized
+
+GitLab is bound by its [obligations to community](https://about.gitlab.com/handbook/product/analytics-instrumentation-guide/service-usage-data-commitment/)
+and by [legal regulations](https://about.gitlab.com/handbook/legal/privacy/services-usage-data/) to protect the privacy of its users.
+
+GitLab must provide valuable insights for business decisions, and there is a need
+for a better understanding of different users' behavior patterns. The
+pseudonymization process helps you find a compromise between these two requirements.
+
+Pseudonymization processes personally identifiable information inside a Snowplow event in an irreversible fashion
+maintaining deterministic output for given input, while masking any relation to that input.
+
+#### How events are pseudonymized
+
+Pseudonymization uses an allowlist that provides privacy by default. Therefore, each
+attribute received as part of a Snowplow event is pseudonymized unless the attribute
+is an allowed exception.
+
+Pseudonymization is done using the HMAC-SHA256 keyed hash algorithm.
+Attributes are combined with a secret salt to replace each identifiable information with a pseudonym.
+
+### S3 bucket data lake to Snowflake
+
+See [Data team's Snowplow Overview](https://about.gitlab.com/handbook/business-technology/data-team/platform/snowplow/) for further details how data is ingested into our Snowflake data warehouse.
+
+## Monitoring
+
+There are several tools that monitor Snowplow events tracking in different stages of the processing pipeline:
+
+- [Analytics Instrumentation Grafana dashboard](https://dashboards.gitlab.net/d/product-intelligence-main/product-intelligence-product-intelligence?orgId=1) monitors backend events sent from a GitLab.com instance to a collectors fleet. This dashboard provides information about:
+  - The number of events that successfully reach Snowplow collectors.
+  - The number of events that failed to reach Snowplow collectors.
+  - The number of backend events that were sent.
+- [AWS CloudWatch dashboard](https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#dashboards:name=SnowPlow;start=P3D) monitors the state of the events in a processing pipeline. The pipeline starts from Snowplow collectors, goes through to enrichers and pseudonymization, and then up to persistence in an S3 bucket. From S3, the events are imported into the Snowflake Data Warehouse. You must have AWS access rights to view this dashboard. For more information, see [monitoring](https://gitlab.com/gitlab-org/analytics-section/analytics-instrumentation/snowplow-pseudonymization#monitoring) in the Snowplow Events pseudonymization service documentation.
+- [Sisense dashboard](https://app.periscopedata.com/app/gitlab/417669/Snowplow-Summary-Dashboard) provides information about the number of good and bad events imported into the Data Warehouse, in addition to the total number of imported Snowplow events.
+
+For more information, see this [video walk-through](https://www.youtube.com/watch?v=NxPS0aKa_oU).
+
+## Related topics
+
+- [Snowplow technology 101](https://github.com/snowplow/snowplow/#snowplow-technology-101)
+- [Snowplow pseudonymization AWS Lambda project](https://gitlab.com/gitlab-org/analytics-section/analytics-instrumentation/snowplow-pseudonymization)
+- [Analytics Instrumentation Guide](https://about.gitlab.com/handbook/product/analytics-instrumentation-guide/)
+- [Data Infrastructure](https://about.gitlab.com/handbook/business-technology/data-team/platform/infrastructure/)
+- [Snowplow architecture overview (internal)](https://www.youtube.com/watch?v=eVYJjzspsLU)
+- [Snowplow architecture overview slide deck (internal)](https://docs.google.com/presentation/d/16gQEO5CAg8Tx4NBtfnZj-GF4juFI6HfEPWcZgH4Rn14/edit?usp=sharing)
+- [AWS Lambda implementation (internal)](https://youtu.be/cQd0mdMhkQA)
diff --git a/doc/development/internal_analytics/snowplow/review_guidelines.md b/doc/development/internal_analytics/snowplow/review_guidelines.md
new file mode 100644
index 00000000000..03d1812cbfc
--- /dev/null
+++ b/doc/development/internal_analytics/snowplow/review_guidelines.md
@@ -0,0 +1,44 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Snowplow review guidelines
+
+This page includes introductory material for an
+[Analytics Instrumentation](https://about.gitlab.com/handbook/engineering/development/analytics/analytics-instrumentation/)
+review, and is specific to Snowplow related reviews. For broader advice and
+general best practices for code reviews, refer to our [code review guide](../../code_review.md).
+
+## Resources for reviewers
+
+- [Snowplow Guide](index.md)
+- [Event Dictionary](https://metrics.gitlab.com/snowplow/)
+
+## Review process
+
+We recommend an Analytics Instrumentation review when a merge request (MR) involves changes in
+events or touches Snowplow related files.
+
+### Roles and process
+
+#### The merge request **author** should
+
+- For frontend events, when relevant, add a screenshot of the event in
+  the [testing tool](implementation.md#develop-and-test-snowplow) used.
+- For backend events, when relevant, add the output of the
+  [Snowplow Micro](implementation.md#test-backend-events-with-snowplow-micro) good events
+  `GET http://localhost:9090/micro/good` (it might be a good idea
+  to reset with `GET http://localhost:9090/micro/reset` first).
+- Add or update the event definition file according to the [Event Dictionary Guide](event_dictionary_guide.md).
+
+#### The Analytics Instrumentation **reviewer** should
+
+- Check that the [event schema](index.md#event-schema) is correct.
+- Check the [usage recommendations](implementation.md#usage-recommendations).
+- Check that an event definition file was created or updated in accordance with the [Event Dictionary Guide](event_dictionary_guide.md).
+- If needed, check that the events are firing locally using one of the
+[testing tools](implementation.md#develop-and-test-snowplow) available.
+- Approve the MR, and relabel the MR with `~"analytics instrumentation::approved"`.
+- If the snowplow event mirrors a RedisHLL event, then tag @mdrussell to review if the payload is usable for this purpose.
diff --git a/doc/development/internal_analytics/snowplow/schemas.md b/doc/development/internal_analytics/snowplow/schemas.md
new file mode 100644
index 00000000000..21142f68d39
--- /dev/null
+++ b/doc/development/internal_analytics/snowplow/schemas.md
@@ -0,0 +1,190 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Snowplow schemas
+
+This page provides Snowplow schema reference for GitLab events.
+
+## `gitlab_standard`
+
+We are including the [`gitlab_standard` schema](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_standard/jsonschema/) for structured events and page views.
+
+The [`StandardContext`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/tracking/standard_context.rb)
+class represents this schema in the application. Some properties are
+[automatically populated for frontend events](implementation.md#snowplow-javascript-frontend-tracking),
+and can be [provided manually for backend events](implementation.md#implement-ruby-backend-tracking).
+
+| Field Name              |      Required       | Default value                | Type                      | Description                                                                                                             |
+|-------------------------|:-------------------:|------------------------------|---------------------------|-------------------------------------------------------------------------------------------------------------------------|
+| `project_id`            | **{dotted-circle}** | Current project ID *         | integer                   |                                                                                                                         |
+| `namespace_id`          | **{dotted-circle}** | Current group/namespace ID * | integer                   |                                                                                                                         |
+| `user_id`               | **{dotted-circle}** | Current user ID *            | integer                   | User database record ID attribute. This value undergoes a pseudonymization process at the collector level.              |
+| `context_generated_at`  | **{dotted-circle}** | Current timestamp            | string (date time format) | Timestamp indicating when context was generated.                                                                        |
+| `environment`           | **{check-circle}**  | Current environment          | string (max 32 chars)     | Name of the source environment, such as `production` or `staging`                                                       |
+| `source`                | **{check-circle}**  | Event source                 | string (max 32 chars)     | Name of the source application, such as  `gitlab-rails` or `gitlab-javascript`                                          |
+| `plan`                  | **{dotted-circle}** | Current namespace plan *     | string (max 32 chars)     | Name of the plan for the namespace, such as `free`, `premium`, or `ultimate`. Automatically picked from the `namespace`. |
+| `google_analytics_id`   | **{dotted-circle}** | GA ID value *                | string (max 32 chars)     | Google Analytics ID, present when set from our marketing sites.                                                         |
+| `is_gitlab_team_member` | **{dotted-circle}** |                              | boolean                   | Indicates if the events is triggered by a GitLab team member |
+| `extra`                 | **{dotted-circle}** |                              | JSON                      | Any additional data associated with the event, in the form of key-value pairs                                           |
+
+_\* Default value present for frontend events only_
+
+## Default Schema
+
+Frontend events include a [web-specific schema](https://docs.snowplow.io/docs/understanding-your-pipeline/canonical-event/#web-specific-fields) provided by Snowplow.
+All URLs are pseudonymized. The entity identifier [replaces](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracker-setup/other-parameters-2/#setting-a-custom-page-url-and-referrer-url) personally identifiable
+information (PII). PII includes usernames, group, and project names.
+Page titles are hardcoded as `GitLab` for the same reason.
+
+| Field Name               | Required            | Type      | Description                                                                                                                      |
+|--------------------------|---------------------|-----------|----------------------------------------------------------------------------------------------------------------------------------|
+| `app_id`                   | **{check-circle}**  | string    | Unique identifier for website / application                                                                                      |
+| `base_currency`            | **{dotted-circle}** | string    | Reporting currency                                                                                                               |
+| `br_colordepth`            | **{dotted-circle}** | integer   | Browser color depth                                                                                                              |
+| `br_cookies`               | **{dotted-circle}** | boolean   | Does the browser permit cookies?                                                                                                 |
+| `br_family`                | **{dotted-circle}** | string    | Browser family                                                                                                                   |
+| `br_features_director`     | **{dotted-circle}** | boolean   | Director plugin installed?                                                                                                       |
+| `br_features_flash`        | **{dotted-circle}** | boolean   | Flash plugin installed?                                                                                                          |
+| `br_features_gears`        | **{dotted-circle}** | boolean   | Google gears installed?                                                                                                          |
+| `br_features_java`         | **{dotted-circle}** | boolean   | Java plugin installed?                                                                                                           |
+| `br_features_pdf`          | **{dotted-circle}** | boolean   | Adobe PDF plugin installed?                                                                                                      |
+| `br_features_quicktime`    | **{dotted-circle}** | boolean   | Quicktime plugin installed?                                                                                                      |
+| `br_features_realplayer`   | **{dotted-circle}** | boolean   | RealPlayer plugin installed?                                                                                                     |
+| `br_features_silverlight`  | **{dotted-circle}** | boolean   | Silverlight plugin installed?                                                                                                    |
+| `br_features_windowsmedia` | **{dotted-circle}** | boolean   | Windows media plugin installed?                                                                                                  |
+| `br_lang`                  | **{dotted-circle}** | string    | Language the browser is set to                                                                                                   |
+| `br_name`                  | **{dotted-circle}** | string    | Browser name                                                                                                                     |
+| `br_renderengine`          | **{dotted-circle}** | string    | Browser rendering engine                                                                                                         |
+| `br_type`                  | **{dotted-circle}** | string    | Browser type                                                                                                                     |
+| `br_version`               | **{dotted-circle}** | string    | Browser version                                                                                                                  |
+| `br_viewheight`            | **{dotted-circle}** | string    | Browser viewport height                                                                                                          |
+| `br_viewwidth`             | **{dotted-circle}** | string    | Browser viewport width                                                                                                           |
+| `collector_tstamp`         | **{dotted-circle}** | timestamp | Time stamp for the event recorded by the collector                                                                               |
+| `contexts`                 | **{dotted-circle}** |           |                                                                                                                                  |
+| `derived_contexts`         | **{dotted-circle}** |           | Contexts derived in the Enrich process                                                                                           |
+| `derived_tstamp`           | **{dotted-circle}** | timestamp | Timestamp making allowance for inaccurate device clock                                                                          |
+| `doc_charset`              | **{dotted-circle}** | string    | Web page's character encoding                                                                                                    |
+| `doc_height`               | **{dotted-circle}** | string    | Web page height                                                                                                                  |
+| `doc_width`                | **{dotted-circle}** | string    | Web page width                                                                                                                   |
+| `domain_sessionid`         | **{dotted-circle}** | string    | Unique identifier (UUID) for this visit of this `user_id` to this domain                                                         |
+| `domain_sessionidx`        | **{dotted-circle}** | integer   | Index of number of visits that this `user_id` has made to this domain (The first visit is `1`)                                   |
+| `domain_userid`            | **{dotted-circle}** | string    | Unique identifier for a user, based on a first party cookie (so domain specific)                                                 |
+| `dvce_created_tstamp`      | **{dotted-circle}** | timestamp | Timestamp when event occurred, as recorded by client device                                                                      |
+| `dvce_ismobile`            | **{dotted-circle}** | boolean   | Indicates whether device is mobile                                                                                               |
+| `dvce_screenheight`        | **{dotted-circle}** | string    | Screen / monitor resolution                                                                                                      |
+| `dvce_screenwidth`         | **{dotted-circle}** | string    | Screen / monitor resolution                                                                                                      |
+| `dvce_sent_tstamp`         | **{dotted-circle}** | timestamp | Timestamp when event was sent by client device to collector                                                                      |
+| `dvce_type`                | **{dotted-circle}** | string    | Type of device                                                                                                                   |
+| `etl_tags`                 | **{dotted-circle}** | string    | JSON of tags for this ETL run                                                                                                    |
+| `etl_tstamp`               | **{dotted-circle}** | timestamp | Timestamp event began ETL                                                                                                        |
+| `event`                    | **{dotted-circle}** | string    | Event type                                                                                                                       |
+| `event_fingerprint`        | **{dotted-circle}** | string    | Hash client-set event fields                                                                                                     |
+| `event_format`             | **{dotted-circle}** | string    | Format for event                                                                                                                 |
+| `event_id`                 | **{dotted-circle}** | string    | Event UUID                                                                                                                       |
+| `event_name`               | **{dotted-circle}** | string    | Event name                                                                                                                       |
+| `event_vendor`             | **{dotted-circle}** | string    | The company who developed the event model                                                                                        |
+| `event_version`            | **{dotted-circle}** | string    | Version of event schema                                                                                                          |
+| `geo_city`                 | **{dotted-circle}** | string    | City of IP origin                                                                                                                |
+| `geo_country`              | **{dotted-circle}** | string    | Country of IP origin                                                                                                             |
+| `geo_latitude`             | **{dotted-circle}** | string    | An approximate latitude                                                                                                          |
+| `geo_longitude`            | **{dotted-circle}** | string    | An approximate longitude                                                                                                         |
+| `geo_region`               | **{dotted-circle}** | string    | Region of IP origin                                                                                                              |
+| `geo_region_name`          | **{dotted-circle}** | string    | Region of IP origin                                                                                                              |
+| `geo_timezone`             | **{dotted-circle}** | string    | Time zone of IP origin                                                                                                            |
+| `geo_zipcode`              | **{dotted-circle}** | string    | Zip (postal) code of IP origin                                                                                                   |
+| `ip_domain`                | **{dotted-circle}** | string    | Second level domain name associated with the visitor's IP address                                                                |
+| `ip_isp`                   | **{dotted-circle}** | string    | Visitor's ISP                                                                                                                    |
+| `ip_netspeed`              | **{dotted-circle}** | string    | Visitor's connection type                                                                                                        |
+| `ip_organization`          | **{dotted-circle}** | string    | Organization associated with the visitor's IP address – defaults to ISP name if none is found                                    |
+| `mkt_campaign`             | **{dotted-circle}** | string    | The campaign ID                                                                                                                  |
+| `mkt_clickid`              | **{dotted-circle}** | string    | The click ID                                                                                                                     |
+| `mkt_content`              | **{dotted-circle}** | string    | The content or ID of the ad.                                                                   |
+| `mkt_medium`               | **{dotted-circle}** | string    | Type of traffic source                                                                                                           |
+| `mkt_network`              | **{dotted-circle}** | string    | The ad network to which the click ID belongs                                                                                     |
+| `mkt_source`               | **{dotted-circle}** | string    | The company / website where the traffic came from                                                                                |
+| `mkt_term`                 | **{dotted-circle}** | string    | Keywords associated with the referrer                                                                                        |
+| `name_tracker`             | **{dotted-circle}** | string    | The tracker namespace                                                                                                            |
+| `network_userid`           | **{dotted-circle}** | string    | Unique identifier for a user, based on a cookie from the collector (so set at a network level and shouldn't be set by a tracker) |
+| `os_family`                | **{dotted-circle}** | string    | Operating system family                                                                                                          |
+| `os_manufacturer`          | **{dotted-circle}** | string    | Manufacturers of operating system                                                                                                |
+| `os_name`                  | **{dotted-circle}** | string    | Name of operating system                                                                                                         |
+| `os_timezone`              | **{dotted-circle}** | string    | Client operating system time zone                                                                                                 |
+| `page_referrer`            | **{dotted-circle}** | string    | Referrer URL                                                                                                                     |
+| `page_title`               | **{dotted-circle}** | string    | To not expose personal identifying information, the page title is hardcoded as `GitLab`                                          |
+| `page_url`                 | **{dotted-circle}** | string    | Page URL                                                                                                                         |
+| `page_urlfragment`         | **{dotted-circle}** | string    | Fragment aka anchor                                                                                                              |
+| `page_urlhost`             | **{dotted-circle}** | string    | Host aka domain                                                                                                                  |
+| `page_urlpath`             | **{dotted-circle}** | string    | Path to page                                                                                                                     |
+| `page_urlport`             | **{dotted-circle}** | integer   | Port if specified, 80 if not                                                                                                     |
+| `page_urlquery`            | **{dotted-circle}** | string    | Query string                                                                                                                      |
+| `page_urlscheme`           | **{dotted-circle}** | string    | Scheme (protocol name)                                                                                                              |
+| `platform`                 | **{dotted-circle}** | string    | The platform the app runs on                                                                                                     |
+| `pp_xoffset_max`           | **{dotted-circle}** | integer   | Maximum page x offset seen in the last ping period                                                                               |
+| `pp_xoffset_min`           | **{dotted-circle}** | integer   | Minimum page x offset seen in the last ping period                                                                               |
+| `pp_yoffset_max`           | **{dotted-circle}** | integer   | Maximum page y offset seen in the last ping period                                                                               |
+| `pp_yoffset_min`           | **{dotted-circle}** | integer   | Minimum page y offset seen in the last ping period                                                                               |
+| `refr_domain_userid`       | **{dotted-circle}** | string    | The Snowplow `domain_userid` of the referring website                                                                              |
+| `refr_dvce_tstamp`         | **{dotted-circle}** | timestamp | The time of attaching the `domain_userid` to the inbound link                                                                      |
+| `refr_medium`              | **{dotted-circle}** | string    | Type of referer                                                                                                                  |
+| `refr_source`              | **{dotted-circle}** | string    | Name of referer if recognised                                                                                                    |
+| `refr_term`                | **{dotted-circle}** | string    | Keywords if source is a search engine                                                                                            |
+| `refr_urlfragment`         | **{dotted-circle}** | string    | Referer URL fragment                                                                                                             |
+| `refr_urlhost`             | **{dotted-circle}** | string    | Referer host                                                                                                                     |
+| `refr_urlpath`             | **{dotted-circle}** | string    | Referer page path                                                                                                                |
+| `refr_urlport`             | **{dotted-circle}** | integer   | Referer port                                                                                                                     |
+| `refr_urlquery`            | **{dotted-circle}** | string    | Referer URL query string                                                                                                          |
+| `refr_urlscheme`           | **{dotted-circle}** | string    | Referer scheme                                                                                                                   |
+| `se_action`                | **{dotted-circle}** | string    | The action / event itself                                                                                                        |
+| `se_category`              | **{dotted-circle}** | string    | The category of event                                                                                                            |
+| `se_label`                 | **{dotted-circle}** | string    | A label often used to refer to the 'object' the action is performed on                                                           |
+| `se_property`              | **{dotted-circle}** | string    | A property associated with either the action or the object                                                                       |
+| `se_value`                 | **{dotted-circle}** | decimal   | A value associated with the user action                                                                                          |
+| `ti_category`              | **{dotted-circle}** | string    | Item category                                                                                                                    |
+| `ti_currency`              | **{dotted-circle}** | string    | Currency                                                                                                                         |
+| `ti_name`                  | **{dotted-circle}** | string    | Item name                                                                                                                        |
+| `ti_orderid`               | **{dotted-circle}** | string    | Order ID                                                                                                                         |
+| `ti_price`                 | **{dotted-circle}** | decimal   | Item price                                                                                                                       |
+| `ti_price_base`            | **{dotted-circle}** | decimal   | Item price in base currency                                                                                                      |
+| `ti_quantity`              | **{dotted-circle}** | integer   | Item quantity                                                                                                                    |
+| `ti_sku`                   | **{dotted-circle}** | string    | Item SKU                                                                                                                         |
+| `tr_affiliation`           | **{dotted-circle}** | string    | Transaction affiliation (such as channel)                                                                                           |
+| `tr_city`                  | **{dotted-circle}** | string    | Delivery address: city                                                                                                           |
+| `tr_country`               | **{dotted-circle}** | string    | Delivery address: country                                                                                                        |
+| `tr_currency`              | **{dotted-circle}** | string    | Transaction Currency                                                                                                             |
+| `tr_orderid`               | **{dotted-circle}** | string    | Order ID                                                                                                                         |
+| `tr_shipping`              | **{dotted-circle}** | decimal   | Delivery cost charged                                                                                                            |
+| `tr_shipping_base`         | **{dotted-circle}** | decimal   | Shipping cost in base currency                                                                                                   |
+| `tr_state`                 | **{dotted-circle}** | string    | Delivery address: state                                                                                                          |
+| `tr_tax`                   | **{dotted-circle}** | decimal   | Transaction tax value (such as amount of VAT included)                                                                              |
+| `tr_tax_base`              | **{dotted-circle}** | decimal   | Tax applied in base currency                                                                                                     |
+| `tr_total`                 | **{dotted-circle}** | decimal   | Transaction total value                                                                                                          |
+| `tr_total_base`            | **{dotted-circle}** | decimal   | Total amount of transaction in base currency                                                                                     |
+| `true_tstamp`              | **{dotted-circle}** | timestamp | User-set exact timestamp                                                                                                         |
+| `txn_id`                   | **{dotted-circle}** | string    | Transaction ID                                                                                                                   |
+| `unstruct_event`           | **{dotted-circle}** | JSON      | The properties of the event                                                                                                      |
+| `uploaded_at`              | **{dotted-circle}** |           |                                                                                                                                  |
+| `user_fingerprint`         | **{dotted-circle}** | integer   | User identifier based on (hopefully unique) browser features                                                                     |
+| `user_id`                  | **{dotted-circle}** | string    | Unique identifier for user, set by the business using setUserId                                                                  |
+| `user_ipaddress`           | **{dotted-circle}** | string    | IP address                                                                                                                       |
+| `useragent`                | **{dotted-circle}** | string    | User agent (expressed as a browser string)                                                                                                |
+| `v_collector`              | **{dotted-circle}** | string    | Collector version                                                                                                                |
+| `v_etl`                    | **{dotted-circle}** | string    | ETL version                                                                                                                      |
+| `v_tracker`                | **{dotted-circle}** | string    | Identifier for Snowplow tracker                                                                                                  |
+
+## `gitlab_service_ping`
+
+Backend events converted from ServicePing (`redis` and `redis_hll`) must include [ServicePing context](https://gitlab.com/gitlab-org/iglu/-/tree/master/public/schemas/com.gitlab/gitlab_service_ping/jsonschema)
+using the [helper class](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/tracking/service_ping_context.rb).
+
+An example of converted `redis_hll` [event with context](https://gitlab.com/gitlab-org/gitlab/-/edit/master/app/controllers/concerns/product_analytics_tracking.rb#L58).
+
+| Field Name    |      Required       | Type                   | Description                                                                                                                                                        |
+|---------------|:-------------------:|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `data_source` | **{check-circle}**  | string (max 64 chars)  | The `data_source` attribute from the metrics YAML definition. |
+| `event_name`* | **{dotted-circle}** | string (max 128 chars) | When there is a many-to-many relationship between events and metrics, this field contains the name of a Redis event that can be used for aggregations in downstream systems |
+| `key_path`*   | **{dotted-circle}** | string (max 256 chars) | The `key_path` attribute from the metrics YAML definition |
+
+_\* Either `event_name` or `key_path` is required_
diff --git a/doc/development/internal_analytics/snowplow/troubleshooting.md b/doc/development/internal_analytics/snowplow/troubleshooting.md
new file mode 100644
index 00000000000..885f4e0c16f
--- /dev/null
+++ b/doc/development/internal_analytics/snowplow/troubleshooting.md
@@ -0,0 +1,80 @@
+---
+stage: Analytics
+group: Analytics Instrumentation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Troubleshooting Snowplow
+
+## Monitoring
+
+This page covers dashboards and alerts coming from a number of internal tools.
+
+For a brief video overview of the tools used to monitor Snowplow usage, please check out [this internal video](https://www.youtube.com/watch?v=NxPS0aKa_oU) (you must be logged into GitLab Unfiltered to view).
+
+## Good events drop
+
+### Symptoms
+
+You will be alarmed via a [Sisense alert](https://app.periscopedata.com/app/gitlab/alert/Volume-of-Snowplow-Good-events/5a5f80ef34fe450da5ebb84eaa84067f/edit) that is sent to `#g_product_intelligence` Slack channel
+
+### Locating the problem
+
+First you need to identify at which stage in Snowplow the data pipeline the drop is occurring.
+Start at [Snowplow dashboard](https://console.aws.amazon.com/systems-manager/resource-groups/cloudwatch?dashboard=SnowPlow&region=us-east-1#) on CloudWatch,
+if you do not have access to CloudWatch you need to create an [access request issue](https://gitlab.com/gitlab-com/team-member-epics/access-requests/-/issues/9730) first.
+While on CloudWatch dashboard set time range to last 4 weeks, to get better picture of system characteristics over time. Than visit following charts:
+
+1. `ELB New Flow Count` and `Collector Auto Scaling Group Network In/Out` - they show in order: number of connections to collectors via load balancers and data volume (in bytes) processed by collectors. If there is drop visible there, it means less events were fired from the GitLab application. Proceed to [application layer guide](#troubleshooting-gitlab-application-layer) for more details
+1. `Firehose Records to S3` - it shows how many event records were saved to S3 bucket, if there was drop on this chart but not on the charts from 1. it means that problem is located at AWS infrastructure layer, please refer to [AWS layer guide](#troubleshooting-aws-layer)
+1. If drop wasn't visible on any of previous charts it means that problem is at data warehouse layer, please refer to [data warehouse layer guide](#troubleshooting-data-warehouse-layer)
+
+### Troubleshooting GitLab application layer
+
+Drop occurring at application layer can be symptom of some issue, but it might be also a result of normal application lifecycle, intended changes done to analytics instrumentation or experiments tracking
+or even a result of a public holiday in some regions of the world with a larger user-base. To verify if there is an underlying problem to solve, you can check following things:
+
+1. Check `about.gitlab.com` website traffic on [Google Analytics](https://analytics.google.com/analytics/web/) to verify if some public holiday might impact overall use of GitLab system
+   1. You may require to open an access request for Google Analytics access first, for example: [access request internal issue](https://gitlab.com/gitlab-com/team-member-epics/access-requests/-/issues/1772)
+1. Plot `select date(dvce_created_tstamp) , event , count(*) from legacy.snowplow_unnested_events_90 where dvce_created_tstamp > '2021-06-15' and dvce_created_tstamp < '2021-07-10' group by 1 , 2 order by 1 , 2` in SiSense to see what type of events was responsible for drop
+1. Plot `select date(dvce_created_tstamp) ,se_category , count(*) from legacy.snowplow_unnested_events_90 where dvce_created_tstamp > '2021-06-15' and dvce_created_tstamp < '2021-07-31' and event = 'struct' group by  1 , 2 order by  1, 2` what events recorded the biggest drops in suspected category
+1. Check if there was any MR merged that might cause reduction in reported events, pay an attention to ~"analytics instrumentation" and ~"growth experiment" labeled MRs
+1. Check (via [Grafana explore tab](https://dashboards.gitlab.net/explore) ) following Prometheus counters `gitlab_snowplow_events_total`, `gitlab_snowplow_failed_events_total` and `gitlab_snowplow_successful_events_total` to see how many events were fired correctly from GitLab.com. Example query to use `sum(rate(gitlab_snowplow_successful_events_total{env="gprd"}[5m])) / sum(rate(gitlab_snowplow_events_total{env="gprd"}[5m]))` would chart rate at which number of good events rose in comparison to events sent in total. If number drops from 1 it means that problem might be in communication between GitLab and AWS collectors fleet.
+1. Check [logs in Kibana](https://log.gprd.gitlab.net/app/discover#) and filter with `{ "query": { "match_phrase": { "json.message": "failed to be reported to collector at" } } }` if there are some failed events logged
+
+For results about an investigation conducted into an unexpected drop in snowplow events volume, see [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/335206).
+
+### Troubleshooting AWS layer
+
+Already conducted investigations:
+
+- [Steep decrease of Snowplow page views](https://gitlab.com/gitlab-org/gitlab/-/issues/268009)
+- [`snowplow.trx.gitlab.net` unreachable](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5073)
+
+### Troubleshooting data warehouse layer
+
+Reach out to [Data team](https://about.gitlab.com/handbook/business-technology/data-team/) to ask about current state of data warehouse. On their handbook page there is a [section with contact details](https://about.gitlab.com/handbook/business-technology/data-team/#how-to-connect-with-us)
+
+## Delay in Snowplow Enrichers
+
+If there is an alert for **Snowplow Raw Good Stream Backing Up**, we receive an email notification. This sometimes happens because Snowplow Enrichers don't scale well enough for the amount of Snowplow events.
+
+If the delay goes over 48 hours, we lose data.
+
+### Contact SRE on-call
+
+Send a message in the [#infrastructure_lounge](https://gitlab.slack.com/archives/CB3LSMEJV) Slack channel using the following template:
+
+```markdown
+Hello team!
+
+We received an alert for [Snowplow Raw Good Stream Backing Up](https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#alarmsV2:alarm/SnowPlow+Raw+Good+Stream+Backing+Up?).
+
+Enrichers are not scalling well for the amount of events we receive.
+
+See the [dashboard](https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#dashboards:name=SnowPlow).
+
+Could we get assistance to fix the delay?
+
+Thank you!
+```
diff --git a/doc/development/internal_api/index.md b/doc/development/internal_api/index.md
index c1c0177609b..5fceb9013da 100644
--- a/doc/development/internal_api/index.md
+++ b/doc/development/internal_api/index.md
@@ -14,7 +14,7 @@ working on the GitLab codebase.
 This documentation does not yet include the internal API used by
 GitLab Pages.
 
-## Adding new endpoints
+## Add new endpoints
 
 API endpoints should be externally accessible by default, with proper authentication and authorization.
 Before adding a new internal endpoint, consider if the API would potentially be
@@ -492,13 +492,14 @@ curl --request GET --header "Gitlab-Kas-Api-Request: <JWT token>" \
 Called from GitLab agent server (`kas`) to increase the usage
 metric counters.
 
-| Attribute                                                                  | Type          | Required | Description                                                                                                      |
-|:---------------------------------------------------------------------------|:--------------|:---------|:-----------------------------------------------------------------------------------------------------------------|
-| `counters`                                                                 | hash          | no | The number to increase the `k8s_api_proxy_request` counter by                                                    |
-| `counters["k8s_api_proxy_request"]`                                        | integer       | no | The number to increase the `k8s_api_proxy_request` counter by                                                    |
-| `counters["gitops_sync"]`                                                  | integer       | no | The number to increase the `gitops_sync` counter by                                                              |
-| `unique_counters`                                                          | hash          | no | The number to increase the `k8s_api_proxy_request` counter by                                                    |
-| `unique_counters["agent_users_using_ci_tunnel"]`                              | integer array | no | The set of unique user ids that have interacted a CI Tunnel to track the `agent_users_using_ci_tunnel` metric event |
+| Attribute                                        | Type          | Required | Description                                                                                                          |
+|:-------------------------------------------------|:--------------|:---------|:---------------------------------------------------------------------------------------------------------------------|
+| `counters`                                       | hash          | no       | Hash of counters                                                                                                     |
+| `counters["k8s_api_proxy_request"]`              | integer       | no       | The number to increase the `k8s_api_proxy_request` counter by                                                        |
+| `counters["gitops_sync"]`                        | integer       | no       | The number to increase the `gitops_sync` counter by                                                                  |
+| `counters["flux_git_push_notifications_total"]`  | integer       | no       | The number to increase the `flux_git_push_notifications_total` counter by                                            |
+| `unique_counters`                                | hash          | no       | Array of unique numbers                                                                                              |
+| `unique_counters["agent_users_using_ci_tunnel"]` | integer array | no       | The set of unique user ids that have interacted a CI Tunnel to track the `agent_users_using_ci_tunnel` metric event  |
 
 ```plaintext
 POST /internal/kubernetes/usage_metrics
@@ -663,9 +664,9 @@ Example response:
 
 The subscriptions endpoint is used by [CustomersDot](https://gitlab.com/gitlab-org/customers-gitlab-com) (`customers.gitlab.com`) to apply subscriptions including trials, and add-on purchases, for personal namespaces or top-level groups within GitLab.com.
 
-### Creating a subscription
+### Create a subscription
 
-Use a POST to create a subscription.
+Use a POST command to create a subscription.
 
 ```plaintext
 POST /namespaces/:id/gitlab_subscription
@@ -714,7 +715,7 @@ Example response:
 }
 ```
 
-### Updating a subscription
+### Update a subscription
 
 Use a PUT command to update an existing subscription.
 
@@ -765,7 +766,7 @@ Example response:
 }
 ```
 
-### Retrieving a subscription
+### Retrieve a subscription
 
 Use a GET command to view an existing subscription.
 
@@ -809,6 +810,107 @@ Example response:
 
 - CustomersDot
 
+## Subscription add-on purchases (excluding storage and compute packs)
+
+The subscription add-on purchase endpoint is used by [CustomersDot](https://gitlab.com/gitlab-org/customers-gitlab-com) (`customers.gitlab.com`) to apply subscription add-on purchases like code suggestions for personal namespaces, or top-level groups within GitLab.com. It is not used to apply storage and compute pack purchases.
+
+### Create a subscription add-on purchase
+
+Use a POST command to create a subscription add-on purchase.
+
+```plaintext
+POST /namespaces/:id/subscription_add_on_purchase/:add_on_name
+```
+
+| Attribute   | Type    | Required | Description |
+|:------------|:--------|:---------|:------------|
+| `quantity` | integer | yes | Amount of units in the subscription add-on purchase (Example: Number of seats for a code suggestions add-on) |
+| `expires_on` | date | yes | Expiration date of the subscription add-on purchase |
+| `purchase_xid` | string | yes | Identifier for the subscription add-on purchase (Example: Subscription name for a code suggestions add-on) |
+
+Example request:
+
+```shell
+curl --request POST --header "PRIVATE-TOKEN: <admin_access_token>" "https://gitlab.com/api/v4/namespaces/1234/subscription_add_on_purchase/code_suggestions?&quantity=10&expires_on="2024-07-15"&purchase_xid="A-S12345678""
+```
+
+Example response:
+
+```json
+{
+  "namespace_id":1234,
+  "namespace_name":"A Namespace Name",
+  "add_on":"Code Suggestions",
+  "quantity":10,
+  "expires_on":"2024-07-15",
+  "purchase_xid":"A-S12345678"
+}
+```
+
+### Update a subscription add-on purchases
+
+Use a PUT command to update an existing subscription add-on purchase.
+
+```plaintext
+PUT /namespaces/:id/subscription_add_on_purchase/:add_on_name
+```
+
+| Attribute   | Type    | Required | Description |
+|:------------|:--------|:---------|:------------|
+| `quantity` | integer | yes | Amount of units in the subscription add-on purchase (Example: Number of seats for a code suggestions add-on) |
+| `expires_on` | date | yes | Expiration date of the subscription add-on purchase |
+| `purchase_xid` | string | yes | Identifier for the subscription add-on purchase (Example: Subscription name for a code suggestions add-on) |
+
+Example request:
+
+```shell
+curl --request PUT --header "PRIVATE-TOKEN: <admin_access_token>" "https://gitlab.com/api/v4/namespaces/1234/subscription_add_on_purchase/code_suggestions?&quantity=15&expires_on="2024-07-15"&purchase_xid="A-S12345678""
+```
+
+Example response:
+
+```json
+{
+  "namespace_id":1234,
+  "namespace_name":"A Namespace Name",
+  "add_on":"Code Suggestions",
+  "quantity":15,
+  "expires_on":"2024-07-15",
+  "purchase_xid":"A-S12345678"
+}
+```
+
+### Retrieve a subscription add-on purchases
+
+Use a GET command to view an existing subscription add-on purchase.
+
+```plaintext
+GET /namespaces/:id/subscription_add_on_purchase/:add_on_name
+```
+
+Example request:
+
+```shell
+curl --header "PRIVATE-TOKEN: <admin_access_token>" "https://gitlab.com/api/v4/namespaces/1234/subscription_add_on_purchase/code_suggestions"
+```
+
+Example response:
+
+```json
+{
+  "namespace_id":1234,
+  "namespace_name":"A Namespace Name",
+  "add_on":"Code Suggestions",
+  "quantity":15,
+  "expires_on":"2024-07-15",
+  "purchase_xid":"A-S12345678"
+}
+```
+
+### Known consumers
+
+- CustomersDot
+
 ## Storage limit exclusions
 
 The namespace storage limit exclusion endpoints manage storage limit exclusions on top-level namespaces on GitLab.com.
@@ -827,7 +929,7 @@ Example request:
 ```shell
 curl --request GET \
   --url "https://gitlab.com/v4/namespaces/storage/limit_exclusions" \
-  --header 'PRIVATE-TOKEN: <admin access token>' 
+  --header 'PRIVATE-TOKEN: <admin access token>'
 ```
 
 Example response:
@@ -910,14 +1012,16 @@ Example response:
 
 - GitLab.com Admin Area
 
-## CI/CD minutes provisioning
+## Compute quota provisioning
 
-The CI/CD Minutes endpoints are used by [CustomersDot](https://gitlab.com/gitlab-org/customers-gitlab-com) (`customers.gitlab.com`)
-to apply additional packs of CI/CD minutes, for personal namespaces or top-level groups within GitLab.com.
+> [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" and "units of compute" in GitLab 16.1.
 
-### Creating an additional pack
+The compute quota endpoints are used by [CustomersDot](https://gitlab.com/gitlab-org/customers-gitlab-com) (`customers.gitlab.com`)
+to apply additional packs of units of compute, for personal namespaces or top-level groups in GitLab.com.
 
-Use a POST to create additional packs.
+### Create an additional pack
+
+Use a POST command to create additional packs.
 
 ```plaintext
 POST /namespaces/:id/minutes
@@ -925,9 +1029,9 @@ POST /namespaces/:id/minutes
 
 | Attribute   | Type    | Required | Description |
 |:------------|:--------|:---------|:------------|
-| `packs`     | array   | yes      | An array of purchased minutes packs |
+| `packs`     | array   | yes      | An array of purchased compute packs |
 | `packs[expires_at]` | date   | yes      | Expiry date of the purchased pack|
-| `packs[number_of_minutes]`  | integer    | yes       | Number of additional minutes |
+| `packs[number_of_minutes]`  | integer    | yes       | Number of additional units of compute |
 | `packs[purchase_xid]` | string  | yes       | The unique ID of the purchase |
 
 Example request:
@@ -961,9 +1065,9 @@ Example response:
 ]
 ```
 
-### Moving additional packs
+### Move additional packs
 
-Use a `PATCH` to move additional packs from one namespace to another.
+Use a `PATCH` command to move additional packs from one namespace to another.
 
 ```plaintext
 PATCH /namespaces/:id/minutes/move/:target_id
@@ -999,7 +1103,7 @@ Example response:
 The `upcoming_reconciliations` endpoint is used by [CustomersDot](https://gitlab.com/gitlab-org/customers-gitlab-com) (`customers.gitlab.com`)
 to update upcoming reconciliations for namespaces.
 
-### Updating `upcoming_reconciliations`
+### Update `upcoming_reconciliations`
 
 Use a PUT command to update `upcoming_reconciliations`.
 
@@ -1033,7 +1137,7 @@ Example response:
 200
 ```
 
-### Deleting an `upcoming_reconciliation`
+### Delete an `upcoming_reconciliation`
 
 Use a DELETE command to delete an `upcoming_reconciliation`.
 
@@ -1254,7 +1358,7 @@ Example request:
 
 ```shell
 curl --verbose --request PATCH "https://gitlab.example.com/api/scim/v2/groups/test_group/Users/f0b1d561c-21ff-4092-beab-8154b17f82f2" \
-     --data '{ "Operations": [{"op":"Add","path":"name.formatted","value":"New Name"}] }' \
+     --data '{ "Operations": [{"op":"Update","path":"name.formatted","value":"New Name"}] }' \
      --header "Authorization: Bearer <your_scim_token>" --header "Content-Type: application/scim+json"
 ```
 
@@ -1451,7 +1555,7 @@ Fields that can be updated are:
 | SCIM/IdP field                   | GitLab field                                                                 |
 |:---------------------------------|:-----------------------------------------------------------------------------|
 | `id/externalId`                  | `extern_uid`                                                                 |
-| `active`                         | Identity removal if `active` = `false`                                       |
+| `active`                         | If `false`, the user is blocked, but the SCIM identity remains linked.       |
 
 ```plaintext
 PATCH /api/scim/v2/application/Users/:id
@@ -1468,7 +1572,7 @@ Example request:
 
 ```shell
 curl --verbose --request PATCH "https://gitlab.example.com/api/scim/v2/application/Users/f0b1d561c-21ff-4092-beab-8154b17f82f2" \
-     --data '{ "Operations": [{"op":"Add","path":"name.formatted","value":"New Name"}] }' \
+     --data '{ "Operations": [{"op":"Update","path":"active","value":"false"}] }' \
      --header "Authorization: Bearer <your_scim_token>" --header "Content-Type: application/scim+json"
 ```
 
diff --git a/doc/development/jh_features_review.md b/doc/development/jh_features_review.md
index 7b81ecfe8f5..7e90d79c067 100644
--- a/doc/development/jh_features_review.md
+++ b/doc/development/jh_features_review.md
@@ -20,6 +20,10 @@ We have two kinds of changes related to JH:
     codes under `jh/`.
   - We will generalize this so both EE and JH can share the same mechanism,
     then we wouldn't have to treat them differently.
+  - Database migrations and database schema changes which are required to
+    support running JH edition. See
+    [JiHu guidelines for database changes](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/jihu-database-change-process.html)
+    for details.
 
 If needed, review the corresponding JH merge request located at [JH repository](https://jihulab.com/gitlab-cn/gitlab).
 
@@ -33,9 +37,12 @@ the reference from being misidentified as a missing partial and subsequently del
 
 ## Process overview
 
-See the [merge request process](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/#merge-request-process)
-on the JiHu Support handbook.
-This page is the single source of truth for JiHu-related processes.
+Read the following process guides:
+
+- [Contribution review process](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/jihu-contribution-review-process.html)
+- [Database change process](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/jihu-database-change-process.html)
+- [Security review process](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/jihu-security-review-process.html)
+- [Merge request process](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/#merge-request-process)
 
 ## Act as EE when `jh/` does not exist or when `EE_ONLY=1`
 
diff --git a/doc/development/labels/index.md b/doc/development/labels/index.md
index 50b6ec74575..dfd2a9e4fd3 100644
--- a/doc/development/labels/index.md
+++ b/doc/development/labels/index.md
@@ -194,6 +194,8 @@ The current department labels are:
 
 - `~"UX"`
 - `~"Quality"`
+- `~"infrastructure"`
+- `~"security"`
 
 ## Team labels
 
@@ -206,8 +208,10 @@ people.
 
 The current team labels are:
 
-- `~"Delivery"~`
+- `~"Delivery"`
 - `~"Technical Writing"`
+- `~"Engineering Productivity"`
+- `~"Contributor Success"`
 
 ### Naming and color convention
 
diff --git a/doc/development/licensing.md b/doc/development/licensing.md
index cb703e98264..02b6af6ee49 100644
--- a/doc/development/licensing.md
+++ b/doc/development/licensing.md
@@ -48,6 +48,27 @@ For all of the above, please include `--why "Reason"` and `--who "My Name"` so t
 
 More detailed information on how the gem and its commands work is available in the [License Finder README](https://github.com/pivotal/LicenseFinder).
 
+## Getting an unknown or Lead licensed software approved
+
+We sometimes need to use third-party softwares whose license is not part of the Blue Oak Council
+license list, or is marked as Lead-rated in the list. In this case, the use-case needs to be
+legal-approved before the software can be installed. More on this can be [found in the Handbook](https://about.gitlab.com/handbook/legal/product/#using-open-source-software).
+
+To get legal approval, follow these steps:
+
+1. Create a new [legal issue](https://gitlab.com/gitlab-com/legal-and-compliance/-/issues/new?issuable_template=general-legal-template). Make sure to include as many details as possible:
+   - What license is the software using?
+   - How and where will it be used?
+   - Is it being vendored or forked, or will we be using the upstream project?
+   - Any relevant links.
+1. After the usage has been legal-approved, allowlist the software in the GitLab project.
+   See [License Finder commands](#license-finder-commands) above.
+1. Make sure the software is also recognized by Omnibus. Create a new MR against the [`omnibus-gitlab`](https://gitlab.com/gitlab-org/omnibus-gitlab)
+   project. Refer to [this MR](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/6870)
+   for an example of what the changes should look like. You'll need to edit the following files:
+   - `lib/gitlab/license/analyzer.rb`
+   - `support/dependency_decisions.yml`
+
 ## Encryption keys
 
 If your license was created in your local development or staging environment for Customers Portal or License App, an environment variable called `GITLAB_LICENSE_MODE` with the value `test` needs to be set to use the correct decryption key.
diff --git a/doc/development/merge_request_concepts/diffs/development.md b/doc/development/merge_request_concepts/diffs/development.md
index 1c22eff34db..37d87d4e4eb 100644
--- a/doc/development/merge_request_concepts/diffs/development.md
+++ b/doc/development/merge_request_concepts/diffs/development.md
@@ -72,6 +72,14 @@ contents, individual commits, and the files containing changes.
 Diff content is usually accessed through this class. Logic is often applied
 to diff, file, and commit content before it is returned to a user.
 
+#### `MergeRequestDiff#commits_count`
+
+When `MergeRequestDiff` is saved, associated `MergeRequestDiffCommit` records are
+counted and cached into the `commits_count` column. This number displays on the
+merge request page as the counter for the **Commits** tab.
+
+If `MergeRequestDiffCommit` records are deleted, the counter doesn't update.
+
 ### `MergeRequestDiffCommit`
 
 `MergeRequestDiffCommit` is defined in `app/models/merge_request_diff_commit.rb`.
@@ -119,6 +127,23 @@ relationship to the change, such as:
 - Its ordering in the diff.
 - The raw diff output itself.
 
+#### External diff storage
+
+By default, diff data of a `MergeRequestDiffFile` is stored in `diff` column in
+the `merge_request_diff_files` table. On some installations, the table can grow
+too large, so they're configured to store diffs on external storage to save space.
+To configure it, see [Merge request diffs storage](../../../administration/merge_request_diffs.md).
+
+When configured to use external storage:
+
+- The `diff` column in the database is left `NULL`.
+- The associated `MergeRequestDiff` record sets the `stored_externally` attribute
+  to `true` on creation of `MergeRequestDiff`.
+
+A cron job named `ScheduleMigrateExternalDiffsWorker` is also scheduled at
+minute 15 of every hour. This migrates `diff` that are still stored in the
+database to external storage.
+
 ### `MergeRequestDiffDetail`
 
 `MergeRequestDiffDetail` is defined in `app/models/merge_request_diff_detail.rb`.
diff --git a/doc/development/merge_request_concepts/rate_limits.md b/doc/development/merge_request_concepts/rate_limits.md
index 97d20b57eb4..62c50e3d044 100644
--- a/doc/development/merge_request_concepts/rate_limits.md
+++ b/doc/development/merge_request_concepts/rate_limits.md
@@ -14,7 +14,7 @@ Every new feature should have safe usage limits included in its implementation.
 Limits are applicable for:
 
 - System-level resource pools such as API requests, SSHD connections, database connections, storage, and so on.
-- Domain-level objects such as CI/CD minutes, groups, sign-in attempts, and so on.
+- Domain-level objects such as compute quota, groups, sign-in attempts, and so on.
 
 ## When limits are required
 
diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md
index 513659d0f68..33f51c20446 100644
--- a/doc/development/migration_style_guide.md
+++ b/doc/development/migration_style_guide.md
@@ -973,6 +973,8 @@ Under the hood, it works like this:
 
 ```ruby
 class SwapPrimaryKey < Gitlab::Database::Migration[2.1]
+  disable_ddl_transaction!
+
   TABLE_NAME = :table_name
   PRIMARY_KEY = :table_name_pkey
   OLD_INDEX_NAME = :old_index_name
diff --git a/doc/development/navigation_sidebar.md b/doc/development/navigation_sidebar.md
index cd543012ddd..f7ab75b62f4 100644
--- a/doc/development/navigation_sidebar.md
+++ b/doc/development/navigation_sidebar.md
@@ -15,10 +15,7 @@ the sidebar is a work in progress, and so is this documentation.
 
 ## Enable the new navigation sidebar
 
-To enable the new navigation sidebar:
-
-- Enable the `super_sidebar_nav` feature flag.
-- Select your avatar, then turn on the **New navigation** toggle.
+To enable the new navigation sidebar, select your avatar, then turn on the **New navigation** toggle.
 
 ## Adding items to the sidebar
 
diff --git a/doc/development/packages/debian_repository.md b/doc/development/packages/debian_repository.md
index 1523d777e7b..b3e9bedfdd4 100644
--- a/doc/development/packages/debian_repository.md
+++ b/doc/development/packages/debian_repository.md
@@ -102,18 +102,33 @@ Next, if the file is a `.changes` format:
       1. A [Release file](https://wiki.debian.org/DebianRepository/Format#A.22Release.22_files) is written, signed by the GPG key, and then stored.
    1. Old component files are destroyed.
 
-This diagram shows the path taken after a file is uploaded to the Debian API:
+The three following diagrams show the path taken after a file is uploaded to the Debian API:
 
 ```mermaid
 sequenceDiagram
+    autonumber
+    actor Client
     Client->>+DebianProjectPackages: PUT projects/:id/packages/debian/:file_name
+    Note over DebianProjectPackages: If `.changes` file or distribution param present
+    DebianProjectPackages->>+CreateTemporaryPackageService: Create temporary package
+    Note over DebianProjectPackages: Else
     DebianProjectPackages->>+FindOrCreateIncomingService: Create "incoming" package
+    Note over DebianProjectPackages: Finally
     DebianProjectPackages->>+CreatePackageFileService: Create "unknown" file
-    Note over DebianProjectPackages: If `.changes` file
-    DebianProjectPackages->>+ProcessChangesWorker: Schedule worker to process the file
+    Note over CreatePackageFileService: If `.changes` file or distribution param present
+    CreatePackageFileService->>+ProcessPackageFileWorker: Schedule worker to process the file
     DebianProjectPackages->>+Client: 202 Created
-    ProcessChangesWorker->>+ProcessChangesService: Start service
-    ProcessChangesService->>+ExtractChangesMetadataService: Extract changesmetadata
+
+    ProcessPackageFileWorker->>+ProcessPackageFileService: Start service
+```
+
+`ProcessPackageFileWorker` background job:
+
+```mermaid
+sequenceDiagram
+    autonumber
+    ProcessPackageFileWorker->>+ProcessPackageFileService: Start service
+    ProcessPackageFileService->>+ExtractChangesMetadataService: Extract changes metadata
     ExtractChangesMetadataService->>+ExtractMetadataService: Extract file metadata
     ExtractMetadataService->>+ParseDebian822Service: run `dpkg --field` to get control file
     ExtractMetadataService->>+ExtractDebMetadataService: If .deb, .udeb or ddeb
@@ -123,9 +138,9 @@ sequenceDiagram
     ExtractMetadataService->>+ParseDebian822Service: if .dsc, .changes, or buildinfo
     ParseDebian822Service-->>-ExtractMetadataService:  Parse String as Debian RFC822 control data format
     ExtractMetadataService-->>-ExtractChangesMetadataService: Parse Metadata file
-    ExtractChangesMetadataService-->>-ProcessChangesService: Return list of files and hashes from the .changes file
+    ExtractChangesMetadataService-->>-ProcessPackageFileService: Return list of files and hashes from the .changes file
     loop process files listed in .changes
-        ProcessChangesService->>+ExtractMetadataService: Process file
+        ProcessPackageFileService->>+ExtractMetadataService: Process file
         ExtractMetadataService->>+ParseDebian822Service: run `dpkg --field` to get control file
         ExtractMetadataService->>+ExtractDebMetadataService: If .deb, .udeb or ddeb
         ExtractDebMetadataService->>+ParseDebian822Service: run `dpkg --field` to get control file
@@ -133,9 +148,18 @@ sequenceDiagram
         ExtractDebMetadataService-->>-ExtractMetadataService: Return the parsed control file
         ExtractMetadataService->>+ParseDebian822Service: if .dsc, .changes, or buildinfo
         ParseDebian822Service-->>-ExtractMetadataService:  Parse String as Debian RFC822 control data format
-        ExtractMetadataService-->>-ProcessChangesService: Use parsed metadata to update "unknown" (or known) file
+        ExtractMetadataService-->>-ProcessPackageFileService: Use parsed metadata to update "unknown" (or known) file
     end
-    ProcessChangesService->>+GenerateDistributionWorker: Find distribution and start service
+    ProcessPackageFileService->>+GenerateDistributionWorker: Find distribution and start service
+
+    GenerateDistributionWorker->>+GenerateDistributionService: Generate distribution
+```
+
+`GenerateDistributionWorker` background job:
+
+```mermaid
+sequenceDiagram
+    autonumber
     GenerateDistributionWorker->>+GenerateDistributionService: Generate distribution
     GenerateDistributionService->>+GenerateDistributionService: generate component files based on new archs and updates from .changes
     GenerateDistributionService->>+GenerateDistributionKeyService: generate GPG key for distribution
@@ -143,7 +167,7 @@ sequenceDiagram
     GenerateDistributionService-->>-GenerateDistributionService: Generate distribution file
     GenerateDistributionService->>+SignDistributionService: Sign release file with GPG key
     SignDistributionService-->>-GenerateDistributionService: Save the signed release file
-    GenerateDistributionWorker->>+GenerateDistributionService: destroy no longer used component files
+    GenerateDistributionService->>+GenerateDistributionService: destroy no longer used component files
 ```
 
 ### Distributions
diff --git a/doc/development/pages/dnsmasq.md b/doc/development/pages/dnsmasq.md
new file mode 100644
index 00000000000..a14e215ccd7
--- /dev/null
+++ b/doc/development/pages/dnsmasq.md
@@ -0,0 +1,65 @@
+---
+type: reference, dev
+stage: Plan
+group: Knowledge
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+description: "dnsmasq configuration guidelines for GitLab Pages"
+---
+
+# Using `dnsmasq` to dynamically handle GitLab Pages subdomains
+
+You can use [`dnsmasq`](https://wiki.debian.org/dnsmasq) to test
+GitLab Pages sites locally without having to configure each site on `/etc/hosts`.
+
+## Use `dnsmasq` on macOS
+
+To use `dnsmasq` on macOS:
+
+1. Install `dnsmasq`:
+
+```console
+brew install dnsmasq
+```
+
+1. Set up the `*.test` domain lookup:
+
+```console
+# Ensure the configuration directory exists
+mkdir -p $(brew --prefix)/etc/
+
+# Add `*.test` to the `127.0.0.1` lookup
+echo 'address=/.test/127.0.0.1' >> $(brew --prefix)/etc/dnsmasq.conf
+
+# Start `dnsmasq`
+sudo brew services start dnsmasq
+```
+
+1. Create a DNS resolver:
+
+```console
+# Ensure the resolver directory exists
+sudo mkdir -p /etc/resolver
+
+# Add the localhost address as a resolver for `.test` domains
+echo "nameserver 127.0.0.1" | sudo tee /etc/resolver/test
+```
+
+You can now create a GitLab Pages site locally with a dynamic domain.
+If you [configure GitLab Pages](index.md#configuring-gitlab-pages-with-gdk) and
+create a `root/html` project, that project is accessible through `http://root.gdk.pages.test:3010/html`.
+
+## Troubleshooting
+
+For GitLab Runner, you must define `gdk.test` in `/etc/hosts`.
+If you're using GitLab Runner locally, you must also configure `/etc/hosts`:
+
+```console
+# Append GDK configuration in `/etc/hosts`
+cat <<-EOF | sudo tee -a /etc/hosts
+
+## GDK
+127.0.0.1  gdk.test
+::1        gdk.test
+# ----------------------------
+EOF
+```
diff --git a/doc/development/pages/index.md b/doc/development/pages/index.md
index 4769dbf427d..25c1ea7a864 100644
--- a/doc/development/pages/index.md
+++ b/doc/development/pages/index.md
@@ -37,7 +37,7 @@ for GitLab Pages, and then one entry for each page site:
 If instead of editing your `/etc/hosts` you'd prefer to use a DNS wildcard, you can use:
 
 - [`nip.io`](https://nip.io)
-- [`dnsmasq`](https://wiki.debian.org/dnsmasq)
+- [`dnsmasq`](dnsmasq.md)
 
 ## Configuring GitLab Pages without GDK
 
diff --git a/doc/development/permissions.md b/doc/development/permissions.md
index 3abadc98501..aa58447b818 100644
--- a/doc/development/permissions.md
+++ b/doc/development/permissions.md
@@ -7,315 +7,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 # Permission development guidelines
 
 There are multiple types of permissions across GitLab, and when implementing
-anything that deals with permissions, all of them should be considered.
+anything that deals with permissions, all of them should be considered. For more information, see:
 
-## Instance
-
-### User types
-
-Each user can be one of the following types:
-
-- Regular.
-- External - access to groups and projects only if direct member.
-- [Internal users](internal_users.md) - system created.
-- [Auditor](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/policies/ee/base_policy.rb#L9):
-  - No access to projects or groups settings menu.
-  - No access to Admin Area.
-  - Read-only access to everything else.
-- [Administrator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/policies/base_policy.rb#L6) - read-write access.
-
-See the [permissions page](../user/permissions.md) for details on how each user type is used.
-
-## Groups and Projects
-
-### General permissions
-
-Groups and projects can have the following visibility levels:
-
-- public (`20`) - an entity is visible to everyone
-- internal (`10`) - an entity is visible to authenticated users
-- private (`0`) - an entity is visible only to the approved members of the entity
-
-By default, subgroups can **not** have higher visibility levels.
-For example, if you create a new private group, it cannot include a public subgroup.
-
-The visibility level of a group can be changed only if all subgroups and
-sub-projects have the same or lower visibility level. For example, a group can be set
-to internal only if all subgroups and projects are internal or private.
-
-WARNING:
-If you migrate an existing group to a lower visibility level, that action does not migrate subgroups
-in the same way. This is a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/22406).
-
-Visibility levels can be found in the `Gitlab::VisibilityLevel` module.
-
-### Feature specific permissions
-
-Additionally, the following project features can have different visibility levels:
-
-- Issues
-- Repository
-  - Merge request
-  - Forks
-  - Pipelines
-- Analytics
-- Requirements
-- Security and Compliance
-- Wiki
-- Snippets
-- Pages
-- Operations
-- Metrics Dashboard
-
-These features can be set to "Everyone with Access" or "Only Project Members".
-They make sense only for public or internal projects because private projects
-can be accessed only by project members by default.
-
-### Members
-
-Users can be members of multiple groups and projects. The following access
-levels are available (defined in the
-[`Gitlab::Access`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/access.rb)
-module):
-
-- No access (`0`)
-- [Minimal access](../user/permissions.md#users-with-minimal-access) (`5`)
-- Guest (`10`)
-- Reporter (`20`)
-- Developer (`30`)
-- Maintainer (`40`)
-- Owner (`50`)
-
-If a user is the member of both a project and the project parent groups, the
-highest permission is the applied access level for the project.
-
-If a user is the member of a project, but not the parent groups, they
-can still view the groups and their entities (like epics).
-
-Project membership (where the group membership is already taken into account)
-is stored in the `project_authorizations` table.
-
-NOTE:
-In [GitLab 14.9](https://gitlab.com/gitlab-org/gitlab/-/issues/351211) and later, projects in personal namespaces have a maximum role of Owner.
-Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/219299) in GitLab 14.8 and earlier, projects in personal namespaces have a maximum role of Maintainer.
-
-### Confidential issues
-
-[Confidential issues](../user/project/issues/confidential_issues.md) can be accessed
-only by project members who are at least
-reporters (they can't be accessed by guests). Additionally they can be accessed
-by their authors and assignees.
-
-### Licensed features
-
-Some features can be accessed only if the user has the correct license plan.
-
-## Permission dependencies
-
-Feature policies can be quite complex and consist of multiple rules.
-Quite often, one permission can be based on another.
-
-Designing good permissions means reusing existing permissions as much as possible
-and making access to features granular.
-
-In the case of a complex resource, it should be broken into smaller pieces of information
-and each piece should be granted a different permission.
-
-A good example in this case is the _Merge Request widget_ and the _Security reports_.
-Depending on the visibility level of the _Pipelines_, the _Security reports_ are either visible
-in the widget or not. So, the _Merge Request widget_, the _Pipelines_, and the _Security reports_,
-have separate permissions. Moreover, the permissions for the _Merge Request widget_
-and the _Pipelines_ are dependencies of the _Security reports_.
-
-### Permission dependencies of Secure features
-
-Secure features have complex permissions since these features are integrated
-into different features like Merge Requests and CI flow.
-
- Here is a list of some permission dependencies.
-
-| Activity level | Resource | Locations |Permission dependency|
-|----------------|----------|-----------|-----|
-| View | License information | Dependency list, License Compliance | Can view repository |
-| View | Dependency information | Dependency list, License Compliance | Can view repository |
-| View | Vulnerabilities information | Dependency list | Can view security findings |
-| View | Black/Whitelisted licenses for the project | License Compliance, merge request  | Can view repository |
-| View | Security findings | merge request, CI job page, Pipeline security tab | Can read the project and CI jobs |
-| View | Vulnerability feedback | merge request | Can read security findings |
-| View | Dependency List page | Project | Can access Dependency information |
-| View | License Compliance page | Project | Can access License information|
-
-## Where should permissions be checked?
-
-We should typically apply defense-in-depth (implementing multiple checks at
-various layers) starting with low-level layers, such as finders and services,
-followed by high-level layers, such as GraphQL, public REST API, and controllers.
-
-See [Guidelines for reusing abstractions](reusing_abstractions.md).
-
-Protecting the same resources at many points means that if one layer of defense is compromised
-or missing, customer data is still protected by the additional layers.
-
-See the permissions section in the [Secure Coding Guidelines](secure_coding_guidelines.md#permissions).
-
-### Considerations
-
-Services or finders are appropriate locations because:
-
-- Multiple endpoints share services or finders so downstream logic is more likely to be re-used.
-- Sometimes authorization logic must be incorporated in DB queries to filter records.
-- Permission checks at the display layer should be avoided except to provide better UX
-  and not as a security check. For example, showing and hiding non-data elements like buttons.
-
-The downsides to defense-in-depth are:
-
-- `DeclarativePolicy` rules are relatively performant, but conditions may perform database calls.
-- Higher maintenance costs.
-
-### Exceptions
-
-Developers can choose to do authorization in only a single area after weighing
-the risks and drawbacks for their specific case.
-
-Prefer domain logic (services or finders) as the source of truth when making exceptions.
-
-Logic, like backend worker logic, might not need authorization based on the current user.
-If the service or finder's constructor does not expect `current_user`, then it typically won't
-check permissions.
-
-### Tips
-
-If a class accepts `current_user`, then it may be responsible for authorization.
-
-### Example: Adding a new API endpoint
-
-By default, we authorize at the endpoint. Checking an existing ability may make sense; if not, then we probably need to add one.
-
-As an aside, most endpoints can be cleanly categorized as a CRUD (create, read, update, destroy) action on a resource. The services and abilities follow suit, which is why many are named like `Projects::CreateService` or `:read_project`.
-
-Say, for example, we extract the whole endpoint into a service. The `can?` check will now be in the service. Say the service reuses an existing finder, which we are modifying for our purposes. Should we make the finder check an ability?
-
-- If the finder doesn't accept `current_user`, and therefore doesn't check permissions, then probably no.
-- If the finder accepts `current_user`, and doesn't check permissions, then it would be a good idea to double check other usages of the finder, and we might consider adding authorization.
-- If the finder accepts `current_user`, and already checks permissions, then either we need to add our case, or the existing checks are appropriate.
-
-### Refactoring permissions
-
-#### Finding existing permissions checks
-
-As mentioned [above](#where-should-permissions-be-checked), permissions are
-often checked in multiple locations for a single endpoint or web request. As a
-result, finding the list of authorization checks that are run for a given endpoint
-can be challenging.
-
-To assist with this, you can locally set `GITLAB_DEBUG_POLICIES=true`.
-
-This outputs information about which abilities are checked in the requests
-made in any specs that you run. The output also includes the line of code where the
-authorization check was made. Caller information is especially helpful in cases
-where there is metaprogramming used because those cases are difficult to find by
-grepping for ability name strings.
-
-Example:
-
-```shell
-# example spec run
-
-GITLAB_DEBUG_POLICIES=true bundle exec rspec spec/controllers/groups_controller_spec.rb:162
-
-# permissions debug output when spec is run; if multiple policy checks are run they will all be in the debug output.
-
-POLICY CHECK DEBUG -> policy: GlobalPolicy, ability: create_group, called_from: ["/gitlab/app/controllers/application_controller.rb:245:in `can?'", "/gitlab/app/controllers/groups_controller.rb:255:in `authorize_create_group!'"]
-```
-
-This flag is meant to help learn more about authorization checks while
-refactoring and should not remain enabled for any specs on the default branch.
-
-#### Understanding logic for individual abilities
-
-References to an ability may appear in a `DeclarativePolicy` class many times
-and depend on conditions and rules which reference other abilities. As a result,
-it can be challenging to know exactly which conditions apply to a particular
-ability.
-
-`DeclarativePolicy` provides a `ability_map` for each Policy class, which
-pulls all Rules for an ability into an array.
-
-Example:
-
-```ruby
-> GroupPolicy.ability_map.map.select { |k,v| k == :read_group_member }
-=> {:read_group_member=>[[:enable, #<Rule can?(:read_group)>], [:prevent, #<Rule ~can_read_group_member>]]}
-
-> GroupPolicy.ability_map.map.select { |k,v| k == :read_group }
-=> {:read_group=>
-  [[:enable, #<Rule public_group>],
-   [:enable, #<Rule logged_in_viewable>],
-   [:enable, #<Rule guest>],
-   [:enable, #<Rule admin>],
-   [:enable, #<Rule has_projects>],
-   [:enable, #<Rule read_package_registry_deploy_token>],
-   [:enable, #<Rule write_package_registry_deploy_token>],
-   [:prevent, #<Rule all?(~public_group, ~admin, user_banned_from_group)>],
-   [:enable, #<Rule auditor>],
-   [:prevent, #<Rule needs_new_sso_session>],
-   [:prevent, #<Rule all?(ip_enforcement_prevents_access, ~owner, ~auditor)>]]}
-```
-
-`DeclarativePolicy` also provides a `debug` method that can be used to
-understand the logic tree for a specific object and actor. The output is similar
-to the list of rules from `ability_map`. But, `DeclarativePolicy` stops
-evaluating rules once one `prevent`s an ability, so it is possible that
-not all conditions are called.
-
-Example:
-
-```ruby
-policy = GroupPolicy.new(User.last,  Group.last)
-policy.debug(:read_group)
-
-- [0] enable when public_group ((@custom_guest_user1 : Group/139))
-- [0] enable when logged_in_viewable ((@custom_guest_user1 : Group/139))
-- [0] enable when admin ((@custom_guest_user1 : Group/139))
-- [0] enable when auditor ((@custom_guest_user1 : Group/139))
-- [14] prevent when all?(~public_group, ~admin, user_banned_from_group) ((@custom_guest_user1 : Group/139))
-- [14] prevent when needs_new_sso_session ((@custom_guest_user1 : Group/139))
-- [16] enable when guest ((@custom_guest_user1 : Group/139))
-- [16] enable when has_projects ((@custom_guest_user1 : Group/139))
-- [16] enable when read_package_registry_deploy_token ((@custom_guest_user1 : Group/139))
-- [16] enable when write_package_registry_deploy_token ((@custom_guest_user1 : Group/139))
-  [21] prevent when all?(ip_enforcement_prevents_access, ~owner, ~auditor) ((@custom_guest_user1 : Group/139))
-
-=> #<DeclarativePolicy::Runner::State:0x000000015c665050
- @called_conditions=
-  #<Set: {
-   "/dp/condition/GroupPolicy/public_group/Group:139",
-   "/dp/condition/GroupPolicy/logged_in_viewable/User:83,Group:139",
-   "/dp/condition/BasePolicy/admin/User:83",
-   "/dp/condition/BasePolicy/auditor/User:83",
-   "/dp/condition/GroupPolicy/user_banned_from_group/User:83,Group:139",
-   "/dp/condition/GroupPolicy/needs_new_sso_session/User:83,Group:139",
-   "/dp/condition/GroupPolicy/guest/User:83,Group:139",
-   "/dp/condition/GroupPolicy/has_projects/User:83,Group:139",
-   "/dp/condition/GroupPolicy/read_package_registry_deploy_token/User:83,Group:139",
-   "/dp/condition/GroupPolicy/write_package_registry_deploy_token/User:83,Group:139"}>,
- @enabled=false,
- @prevented=true>
-```
-
-#### Testing that individual policies are equivalent
-
-You can use the `'equivalent project policy abilities'` shared example to ensure
-that 2 project policy abilities are equivalent for all project visibility levels
-and access levels.
-
-Example:
-
-```ruby
-  context 'when refactoring read_pipeline_schedule and read_pipeline' do
-    let(:old_policy) { :read_pipeline_schedule }
-    let(:new_policy) { :read_pipeline }
-
-    it_behaves_like 'equivalent policies'
-  end
-```
+- [Predefined roles system](permissions/predefined_roles.md): a general overview about predefined roles, user types, feature specific permissions or permissions dependencies.
+- [Authorizations](permissions/authorizations.md): guidance on where to check permissions.
+- [Custom roles](permissions/custom_roles.md): guidance on how to work on custom role, how to introduce a new ability for custom roles, how to refactor permissions.
diff --git a/doc/development/permissions/authorizations.md b/doc/development/permissions/authorizations.md
new file mode 100644
index 00000000000..8d8944562a8
--- /dev/null
+++ b/doc/development/permissions/authorizations.md
@@ -0,0 +1,61 @@
+---
+stage: Manage
+group: Authentication and Authorization
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Authorization
+
+## Where should permissions be checked?
+
+When deciding where to check permissions, apply defense-in-depth by implementing multiple checks at
+different layers. Starting with low-level layers, such as finders and services,
+followed by high-level layers, such as GraphQL, public REST API, and controllers.
+
+For more information, see [guidelines for reusing abstractions](../reusing_abstractions.md).
+
+Protecting the same resources at many points means that if one layer of defense is compromised
+or missing, customer data is still protected by the additional layers.
+
+For more information on permissions, see the permissions section in the [secure coding guidelines](../secure_coding_guidelines.md#permissions).
+
+### Considerations
+
+Services or finders are appropriate locations because:
+
+- Multiple endpoints share services or finders so downstream logic is more likely to be re-used.
+- Sometimes authorization logic must be incorporated in DB queries to filter records.
+- You should avoid permission checks at the display layer except to provide better UX,
+  and not as a security check. For example, showing and hiding non-data elements like buttons.
+
+The downsides to defense-in-depth are:
+
+- `DeclarativePolicy` rules are relatively performant, but conditions may perform database calls.
+- Higher maintenance costs.
+
+### Exceptions
+
+Developers can choose to do authorization in only a single area after weighing
+the risks and drawbacks for their specific case.
+
+Prefer domain logic (services or finders) as the source of truth when making exceptions.
+
+Logic, like backend worker logic, might not need authorization based on the current user.
+If the service or finder's constructor does not expect `current_user`, then it typically does not
+check permissions.
+
+### Tips
+
+If a class accepts `current_user`, then it may be responsible for authorization.
+
+### Example: Adding a new API endpoint
+
+By default, we authorize at the endpoint. Checking an existing ability may make sense; if not, then we probably need to add one.
+
+As an aside, most endpoints can be cleanly categorized as a CRUD (create, read, update, destroy) action on a resource. The services and abilities follow suit, which is why many are named like `Projects::CreateService` or `:read_project`.
+
+Say, for example, we extract the whole endpoint into a service. The `can?` check will now be in the service. Say the service reuses an existing finder, which we are modifying for our purposes. Should we make the finder check an ability?
+
+- If the finder does not accept `current_user`, and therefore does not check permissions, then probably no.
+- If the finder accepts `current_user`, and does not check permissions, then you should double-check other usages of the finder, and you might consider adding authorization.
+- If the finder accepts `current_user`, and already checks permissions, then either we need to add our case, or the existing checks are appropriate.
diff --git a/doc/development/permissions/custom_roles.md b/doc/development/permissions/custom_roles.md
new file mode 100644
index 00000000000..73a8c920894
--- /dev/null
+++ b/doc/development/permissions/custom_roles.md
@@ -0,0 +1,166 @@
+---
+stage: Manage
+group: Authentication and Authorization
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Custom Roles
+
+Users can create custom roles and define those roles by assigning specific abilities. For example, a user could create an "Engineer" role with `read code` and `admin merge requests` abilities, but without abilities like `admin issues`.
+
+## Technical overview
+
+Individual custom roles are stored in the `member_roles` table (`MemberRole` model) and can be defined only for top-level groups. This table includes individual abilities and a `base_access_level` value. This value defines the minimum access level of:
+
+- Users who can be assigned to the custom role.
+- Every ability.
+
+For example, the `read_vulnerability` ability has a minimum access level of `Reporter`. That means only member role records with `base_access_level = REPORTER` (20) or higher can have the `read_vulnerability` value set to `true`. Also, only users who have at least the Reporter role can be assigned that ability.
+
+For now, custom role abilities are supported only at project level.
+
+## How to implement a new ability for custom roles
+
+Usually 2-3 merge requests should be created for a new ability. The rough guidance is following:
+
+1. Pick a feature you want to add abilities to custom roles.
+1. Refactor & consolidate abilities for the feature (1-2 merge requests depending on the feature complexity)
+1. Implement a new ability (1 merge request)
+
+### Refactoring abilities
+
+#### Finding existing abilities checks
+
+Abilities are often [checked in multiple locations](../permissions/authorizations.md#where-should-permissions-be-checked) for a single endpoint or web request. Therefore, it can be difficult to find the list of authorization checks that are run for a given endpoint.
+
+To assist with this, you can locally set `GITLAB_DEBUG_POLICIES=true`.
+
+This outputs information about which abilities are checked in the requests
+made in any specs that you run. The output also includes the line of code where the
+authorization check was made. Caller information is especially helpful in cases
+where there is metaprogramming used because those cases are difficult to find by
+grepping for ability name strings.
+
+For example:
+
+```shell
+# example spec run
+
+GITLAB_DEBUG_POLICIES=true bundle exec rspec spec/controllers/groups_controller_spec.rb:162
+
+# permissions debug output when spec is run; if multiple policy checks are run they will all be in the debug output.
+
+POLICY CHECK DEBUG -> policy: GlobalPolicy, ability: create_group, called_from: ["/gitlab/app/controllers/application_controller.rb:245:in `can?'", "/gitlab/app/controllers/groups_controller.rb:255:in `authorize_create_group!'"]
+```
+
+Use this setting to learn more about authorization checks while
+refactoring. You should not keep this setting enabled for any specs on the default branch.
+
+#### Understanding logic for individual abilities
+
+References to an ability may appear in a `DeclarativePolicy` class many times
+and depend on conditions and rules which reference other abilities. As a result,
+it can be challenging to know exactly which conditions apply to a particular
+ability.
+
+`DeclarativePolicy` provides a `ability_map` for each policy class, which
+pulls all rules for an ability into an array.
+
+For example:
+
+```ruby
+> GroupPolicy.ability_map.map.select { |k,v| k == :read_group_member }
+=> {:read_group_member=>[[:enable, #<Rule can?(:read_group)>], [:prevent, #<Rule ~can_read_group_member>]]}
+
+> GroupPolicy.ability_map.map.select { |k,v| k == :read_group }
+=> {:read_group=>
+  [[:enable, #<Rule public_group>],
+   [:enable, #<Rule logged_in_viewable>],
+   [:enable, #<Rule guest>],
+   [:enable, #<Rule admin>],
+   [:enable, #<Rule has_projects>],
+   [:enable, #<Rule read_package_registry_deploy_token>],
+   [:enable, #<Rule write_package_registry_deploy_token>],
+   [:prevent, #<Rule all?(~public_group, ~admin, user_banned_from_group)>],
+   [:enable, #<Rule auditor>],
+   [:prevent, #<Rule needs_new_sso_session>],
+   [:prevent, #<Rule all?(ip_enforcement_prevents_access, ~owner, ~auditor)>]]}
+```
+
+`DeclarativePolicy` also provides a `debug` method that can be used to
+understand the logic tree for a specific object and actor. The output is similar
+to the list of rules from `ability_map`. But, `DeclarativePolicy` stops
+evaluating rules after you `prevent` an ability, so it is possible that
+not all conditions are called.
+
+Example:
+
+```ruby
+policy = GroupPolicy.new(User.last,  Group.last)
+policy.debug(:read_group)
+
+- [0] enable when public_group ((@custom_guest_user1 : Group/139))
+- [0] enable when logged_in_viewable ((@custom_guest_user1 : Group/139))
+- [0] enable when admin ((@custom_guest_user1 : Group/139))
+- [0] enable when auditor ((@custom_guest_user1 : Group/139))
+- [14] prevent when all?(~public_group, ~admin, user_banned_from_group) ((@custom_guest_user1 : Group/139))
+- [14] prevent when needs_new_sso_session ((@custom_guest_user1 : Group/139))
+- [16] enable when guest ((@custom_guest_user1 : Group/139))
+- [16] enable when has_projects ((@custom_guest_user1 : Group/139))
+- [16] enable when read_package_registry_deploy_token ((@custom_guest_user1 : Group/139))
+- [16] enable when write_package_registry_deploy_token ((@custom_guest_user1 : Group/139))
+  [21] prevent when all?(ip_enforcement_prevents_access, ~owner, ~auditor) ((@custom_guest_user1 : Group/139))
+
+=> #<DeclarativePolicy::Runner::State:0x000000015c665050
+ @called_conditions=
+  #<Set: {
+   "/dp/condition/GroupPolicy/public_group/Group:139",
+   "/dp/condition/GroupPolicy/logged_in_viewable/User:83,Group:139",
+   "/dp/condition/BasePolicy/admin/User:83",
+   "/dp/condition/BasePolicy/auditor/User:83",
+   "/dp/condition/GroupPolicy/user_banned_from_group/User:83,Group:139",
+   "/dp/condition/GroupPolicy/needs_new_sso_session/User:83,Group:139",
+   "/dp/condition/GroupPolicy/guest/User:83,Group:139",
+   "/dp/condition/GroupPolicy/has_projects/User:83,Group:139",
+   "/dp/condition/GroupPolicy/read_package_registry_deploy_token/User:83,Group:139",
+   "/dp/condition/GroupPolicy/write_package_registry_deploy_token/User:83,Group:139"}>,
+ @enabled=false,
+ @prevented=true>
+```
+
+#### Abilities consolidation
+
+Every feature added to custom roles should have minimal abilities. For most features, having `read_*` and `admin_*` should be enough. You should consolidate all:
+
+- View-related abilities under `read_*`. For example, viewing a list or detail.
+- Object updates under `admin_*`. For example, updating an object, adding assignees or closing it that object. Usually, a role that enables `admin_` has to have also `read_` abilities enabled. This is defined in `requirement` option in the `ALL_CUSTOMIZABLE_PERMISSIONS` hash on `MemberRole` model.
+
+There might be features that require additional abilities but try to minimalize those. You can always ask members of the Authentication and Authorization group for their opinion or help.
+
+This is also where your work should begin. Take all the abilities for the feature you work on, and consolidate those abilities into `read_`, `admin_`, or additional abilities if necessary.
+
+### Implement a new ability
+
+To add a new ability to a custom role:
+
+- Add a new column to `member_roles` table, for example in [this change in merge request 114734](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114734/diffs#diff-content-5c53d6f1c29a272a87eecea3f62d017ab6635275).
+- Add the ability to the  `MemberRole` model, `ALL_CUSTOMIZABLE_PERMISSIONS` hash, for example in [this change in merge request 121534](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121534/diffs#ce5ec769500a53ce2b603467d9984fc2b33ca71d_8_8). There are following possible keys in the `ALL_CUSTOMIZABLE_PERMISSIONS` hash:
+
+  - `description` - description of the ability.
+  - `minimal_level` - minimal level a user has to have in order to be able to be assigned to the ability.
+  - `requirement` - required ability for the ability defined in the hash, in case the requirement is `false`, the ability can not be `true`.
+
+- Add the ability to the respective Policy for example in [this change in merge request 114734](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114734/diffs#diff-content-edcbe28bdecbd848d4d9efdc5b5e9bddd2a7299e).
+- Update the specs.
+
+Examples of merge requests adding new abilities to custom roles:
+
+- [Read code](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106256)
+- [Read vulnerability](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114734)
+- [Admin vulnerability](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121534) - this is the newest MR implementing a new custom role ability. Some changes from the previous MRs are not neccessary anymore (eg. change of the Preloader query or adding a method to `User` model).
+
+You should make sure a new custom roles ability is under a feature flag.
+
+### Consuming seats
+
+If a new user with a role `Guest` is added to a member role that includes enablement of an ability that is **not** in the `CUSTOMIZABLE_PERMISSIONS_EXEMPT_FROM_CONSUMING_SEAT` array, a seat is consumed. We simply want to make sure we are charging Ultimate customers for guest users, who have "elevated" abilities. This only applies to billable users on SaaS (billable users that are counted towards namespace subscription). More details about this topic can be found in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390269).
diff --git a/doc/development/permissions/predefined_roles.md b/doc/development/permissions/predefined_roles.md
new file mode 100644
index 00000000000..9afc5966e93
--- /dev/null
+++ b/doc/development/permissions/predefined_roles.md
@@ -0,0 +1,143 @@
+---
+stage: Manage
+group: Authentication and Authorization
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Predefined system of user roles
+
+## Instance
+
+### User types
+
+Each user can be one of the following types:
+
+- Regular.
+- External - access to groups and projects only if direct member.
+- [Internal users](../internal_users.md) - system created.
+- [Auditor](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/policies/ee/base_policy.rb#L9):
+  - No access to projects or groups settings menu.
+  - No access to Admin Area.
+  - Read-only access to everything else.
+- [Administrator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/policies/base_policy.rb#L6) - read-write access.
+
+See the [permissions page](../../user/permissions.md) for details on how each user type is used.
+
+## Groups and Projects
+
+### General permissions
+
+Groups and projects can have the following visibility levels:
+
+- public (`20`) - an entity is visible to everyone
+- internal (`10`) - an entity is visible to authenticated users
+- private (`0`) - an entity is visible only to the approved members of the entity
+
+By default, subgroups can **not** have higher visibility levels.
+For example, if you create a new private group, it cannot include a public subgroup.
+
+The visibility level of a group can be changed only if all subgroups and
+sub-projects have the same or lower visibility level. For example, a group can be set
+to internal only if all subgroups and projects are internal or private.
+
+WARNING:
+If you migrate an existing group to a lower visibility level, that action does not migrate subgroups
+in the same way. This is a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/22406).
+
+Visibility levels can be found in the `Gitlab::VisibilityLevel` module.
+
+### Feature specific permissions
+
+Additionally, the following project features can have different visibility levels:
+
+- Issues
+- Repository
+  - Merge request
+  - Forks
+  - Pipelines
+- Analytics
+- Requirements
+- Security and Compliance
+- Wiki
+- Snippets
+- Pages
+- Operations
+- Metrics Dashboard
+
+These features can be set to "Everyone with Access" or "Only Project Members".
+They make sense only for public or internal projects because private projects
+can be accessed only by project members by default.
+
+### Members
+
+Users can be members of multiple groups and projects. The following access
+levels are available (defined in the
+[`Gitlab::Access`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/access.rb)
+module):
+
+- No access (`0`)
+- [Minimal access](../../user/permissions.md#users-with-minimal-access) (`5`)
+- Guest (`10`)
+- Reporter (`20`)
+- Developer (`30`)
+- Maintainer (`40`)
+- Owner (`50`)
+
+If a user is the member of both a project and the project parent groups, the
+highest permission is the applied access level for the project.
+
+If a user is the member of a project, but not the parent groups, they
+can still view the groups and their entities (like epics).
+
+Project membership (where the group membership is already taken into account)
+is stored in the `project_authorizations` table.
+
+NOTE:
+In [GitLab 14.9](https://gitlab.com/gitlab-org/gitlab/-/issues/351211) and later, projects in personal namespaces have a maximum role of Owner.
+Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/219299) in GitLab 14.8 and earlier, projects in personal namespaces have a maximum role of Maintainer.
+
+### Confidential issues
+
+[Confidential issues](../../user/project/issues/confidential_issues.md) can be accessed
+only by project members who are at least
+reporters (they can't be accessed by guests). Additionally they can be accessed
+by their authors and assignees.
+
+### Licensed features
+
+Some features can be accessed only if the user has the correct license plan.
+
+## Permission dependencies
+
+Feature policies can be quite complex and consist of multiple rules.
+Quite often, one permission can be based on another.
+
+Designing good permissions means reusing existing permissions as much as possible
+and making access to features granular.
+
+In the case of a complex resource, it should be broken into smaller pieces of information
+and each piece should be granted a different permission.
+
+A good example in this case is the _Merge Request widget_ and the _Security reports_.
+Depending on the visibility level of the _Pipelines_, the _Security reports_ are either visible
+in the widget or not. So, the _Merge Request widget_, the _Pipelines_, and the _Security reports_,
+have separate permissions. Moreover, the permissions for the _Merge Request widget_
+and the _Pipelines_ are dependencies of the _Security reports_.
+
+### Permission dependencies of Secure features
+
+Secure features have complex permissions since these features are integrated
+into different features like Merge Requests and CI flow.
+
+ Here is a list of some permission dependencies.
+
+| Activity level | Resource | Locations |Permission dependency|
+|----------------|----------|-----------|-----|
+| View | License information | Dependency list, License Compliance | Can view repository |
+| View | Dependency information | Dependency list, License Compliance | Can view repository |
+| View | Vulnerabilities information | Dependency list | Can view security findings |
+| View | Black/Whitelisted licenses for the project | License Compliance, merge request  | Can view repository |
+| View | Security findings | merge request, CI job page, Pipeline security tab | Can read the project and CI jobs |
+| View | Vulnerability feedback | merge request | Can read security findings |
+| View | Dependency List page | Project | Can access Dependency information |
+| View | License Compliance page | Project | Can access License information|
diff --git a/doc/development/pipelines/index.md b/doc/development/pipelines/index.md
index 651fa2d2ce9..cadba17f03e 100644
--- a/doc/development/pipelines/index.md
+++ b/doc/development/pipelines/index.md
@@ -135,7 +135,7 @@ If so, please have a look at [the Engineering Productivity RUNBOOK on predictive
 ### Fork pipelines
 
 We run only the predictive RSpec & Jest jobs for fork pipelines, unless the `pipeline:run-all-rspec`
-label is set on the MR. The goal is to reduce the CI/CD minutes consumed by fork pipelines.
+label is set on the MR. The goal is to reduce the compute quota consumed by fork pipelines.
 
 See the [experiment issue](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/1170).
 
@@ -186,8 +186,8 @@ This number can be overridden by setting a CI/CD variable named `RSPEC_FAIL_FAST
 
 ## Re-run previously failed tests in merge request pipelines
 
-In order to reduce the feedback time after resolving failed tests for a merge request, the `rspec rspec-pg13-rerun-previous-failed-tests`
-and `rspec rspec-ee-pg13-rerun-previous-failed-tests` jobs run the failed tests from the previous MR pipeline.
+In order to reduce the feedback time after resolving failed tests for a merge request, the `rspec rspec-pg14-rerun-previous-failed-tests`
+and `rspec rspec-ee-pg14-rerun-previous-failed-tests` jobs run the failed tests from the previous MR pipeline.
 
 This was introduced on August 25th 2021, with <https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69053>.
 
@@ -195,7 +195,7 @@ This was introduced on August 25th 2021, with <https://gitlab.com/gitlab-org/git
 
 1. The `detect-previous-failed-tests` job (`prepare` stage) detects the test files associated with failed RSpec
    jobs from the previous MR pipeline.
-1. The `rspec rspec-pg13-rerun-previous-failed-tests` and `rspec rspec-ee-pg13-rerun-previous-failed-tests` jobs
+1. The `rspec rspec-pg14-rerun-previous-failed-tests` and `rspec rspec-ee-pg14-rerun-previous-failed-tests` jobs
    will run the test files gathered by the `detect-previous-failed-tests` job.
 
 ```mermaid
@@ -205,8 +205,8 @@ graph LR
     end
 
     subgraph "test stage";
-        B["rspec rspec-pg13-rerun-previous-failed-tests"];
-        C["rspec rspec-ee-pg13-rerun-previous-failed-tests"];
+        B["rspec rspec-pg14-rerun-previous-failed-tests"];
+        C["rspec rspec-ee-pg14-rerun-previous-failed-tests"];
     end
 
     A --"artifact: list of test files"--> B & C
@@ -442,7 +442,7 @@ This [GitLab JH validation](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-vali
 project variables set.
 
 It's a pull mirror pulling from [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab),
-mirroring only protected branches, `master` and `main-jh`, overriding
+mirroring specific branches: `(master|main-jh)`, overriding
 divergent refs, triggering no pipelines when mirror is updated.
 
 The pulling user is [`@gitlab-jh-validation-bot`](https://gitlab.com/gitlab-jh-validation-bot), who is a maintainer in the project, and also a
@@ -481,6 +481,50 @@ for how it works.
     committed. More context can be found at:
     [Setting it to `false` to skip it](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118938#note_1374688877)
 
+##### Why do we have both the mirror project and validation project?
+
+We have separate projects for a several reasons.
+
+- **Security**: Previously, we had the mirror project only. However, to fully
+  mitigate a [security issue](https://gitlab.com/gitlab-org/gitlab/-/issues/369898),
+  we had to make the mirror project private.
+- **Isolation**: We want to run JH code in a completely isolated and standalone project.
+  We should not run it under the `gitlab-org` group, which is where the mirror
+  project is. The validation project is completely isolated.
+- **Cost**: We don't want to connect to JiHuLab.com from each merge request.
+  It is more cost effective to mirror the code from JiHuLab.com to
+  somewhere at GitLab.com, and have our merge requests fetch code from there.
+  This means that the validation project can fetch code from the mirror, rather
+  than from JiHuLab.com. The mirror project will periodically fetch from
+  JiHuLab.com.
+- **Branch separation/security/efficiency**: We want to mirror all branches,
+  so that we can fetch the corresponding JH branch from JiHuLab.com. However,
+  we don't want to overwrite the `as-if-jh-code-sync` branch in the validation project,
+  because we use it to control the validation pipeline and it has access to
+  `AS_IF_JH_TOKEN`. However, we cannot mirror all branches except a single
+  one. See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/413032) for details.
+
+  Given this issue, the validation project is set to only mirror `master` and
+  `main-jh`. Technically, we don't even need those branches, but we do want to
+  keep the repository up-to-date with all the default branches so that when
+  we push changes from the merge request, we only need to push changes from
+  the merge request, which can be more efficient.
+
+- Separation of concerns:
+  - Validation project only has the following branches:
+    - `master` and `main-jh` to keep changes up-to-date.
+    - `as-if-jh-code-sync` for dependency synchronization.
+      We should never mirror this.
+    - `as-if-jh/*` branches from the merge requests.
+      We should never mirror these.
+  - All branches from the mirror project are all coming from JiHuLab.com.
+    We never push anything to the mirror project, nor does it run any
+    pipelines. CI/CD is disabled in the mirror project.
+
+We can consider merging the two projects to simplify the
+setup and process, but we need to make sure that all of these reasons
+are no longer concerns.
+
 ### `rspec:undercoverage` job
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74859) in GitLab 14.6.
@@ -581,22 +625,22 @@ This should let us:
 
 ### PostgreSQL versions testing
 
-Our test suite runs against PG13 as GitLab.com runs on PG13 and
-[Omnibus defaults to PG13 for new installs and upgrades](../../administration/package_information/postgresql_versions.md).
+Our test suite runs against PostgreSQL 14 as GitLab.com runs on PostgreSQL 14 and
+[Omnibus defaults to PG14 for new installs and upgrades](../../administration/package_information/postgresql_versions.md).
 
-We do run our test suite against PG13 on nightly scheduled pipelines.
+We do run our test suite against PostgreSQL 14 on nightly scheduled pipelines.
 
-We also run our test suite against PG13 upon specific database library changes in MRs and `main` pipelines (with the `rspec db-library-code pg13` job).
+We also run our test suite against PostgreSQL 12 and PostgreSQL 13 upon specific database library changes in merge requests and `main` pipelines (with the `rspec db-library-code pg12` and `rspec db-library-code pg13` jobs).
 
 #### Current versions testing
 
 | Where?                                                                                         | PostgreSQL version                              | Ruby version          |
 |------------------------------------------------------------------------------------------------|-------------------------------------------------|-----------------------|
-| Merge requests                                                                                 | 13 (default version), 12 for DB library changes | 3.0 (default version) |
-| `master` branch commits                                                                        | 13 (default version), 12 for DB library changes | 3.0 (default version) |
-| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour)           | 13 (default version), 12 for DB library changes | 3.0 (default version) |
-| `maintenance` scheduled pipelines for the `ruby2` branch (every odd-numbered hour), see below. | 13 (default version), 12 for DB library changes | 2.7                   |
-| `nightly` scheduled pipelines for the `master` branch                                          | 13 (default version), 12, 14 | 3.0 (default version) |
+| Merge requests                                                                                 | 14 (default version), 13 for DB library changes | 3.0 (default version) |
+| `master` branch commits                                                                        | 14 (default version), 13 for DB library changes | 3.0 (default version) |
+| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour)           | 14 (default version), 13 for DB library changes | 3.0 (default version) |
+| `maintenance` scheduled pipelines for the `ruby2` branch (every odd-numbered hour), see below. | 14 (default version), 13 for DB library changes | 2.7                   |
+| `nightly` scheduled pipelines for the `master` branch                                          | 14 (default version), 12, 13, 15                | 3.0 (default version) |
 
 There are 2 pipeline schedules used for testing Ruby 2.7. One is triggering a
 pipeline in `ruby2-sync` branch, which updates the `ruby2` branch with latest
diff --git a/doc/development/pipelines/internals.md b/doc/development/pipelines/internals.md
index 678297eb3e5..35881db8c6d 100644
--- a/doc/development/pipelines/internals.md
+++ b/doc/development/pipelines/internals.md
@@ -130,16 +130,18 @@ that are scoped to a single [configuration keyword](../../ci/yaml/index.md#job-k
 | `.default-retry` | Allows a job to [retry](../../ci/yaml/index.md#retry) upon `unknown_failure`, `api_failure`, `runner_system_failure`, `job_execution_timeout`, or `stuck_or_timeout_failure`. |
 | `.default-before_script` | Allows a job to use a default `before_script` definition suitable for Ruby/Rails tasks that may need a database running (for example, tests). |
 | `.setup-test-env-cache` | Allows a job to use a default `cache` definition suitable for setting up test environment for subsequent Ruby/Rails tasks. |
-| `.rails-cache` | Allows a job to use a default `cache` definition suitable for Ruby/Rails tasks. |
+| `.ruby-cache` | Allows a job to use a default `cache` definition suitable for Ruby tasks. |
 | `.static-analysis-cache` | Allows a job to use a default `cache` definition suitable for static analysis tasks. |
 | `.coverage-cache` | Allows a job to use a default `cache` definition suitable for coverage tasks. |
 | `.qa-cache` | Allows a job to use a default `cache` definition suitable for QA tasks. |
 | `.yarn-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that do a `yarn install`. |
 | `.assets-compile-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that compile assets. |
-| `.use-pg13` | Allows a job to use the `postgres` 13 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
+| `.use-pg13` | Allows a job to use the `postgres` 13, `redis`, and `rediscluster` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
 | `.use-pg13-ee` | Same as `.use-pg13` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). |
-| `.use-pg14` | Allows a job to use the `postgres` 14 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
+| `.use-pg14` | Allows a job to use the `postgres` 14, `redis`, and `rediscluster` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
 | `.use-pg14-ee` | Same as `.use-pg14` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). |
+| `.use-pg15` | Allows a job to use the `postgres` 15, `redis`, and `rediscluster` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
+| `.use-pg15-ee` | Same as `.use-pg15` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). |
 | `.use-kaniko` | Allows a job to use the `kaniko` tool to build Docker images. |
 | `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` CI/CD variable. |
 | `.use-docker-in-docker` | Allows a job to use Docker in Docker. |
diff --git a/doc/development/pipelines/performance.md b/doc/development/pipelines/performance.md
index d0eef0c86bd..2dbed640fbb 100644
--- a/doc/development/pipelines/performance.md
+++ b/doc/development/pipelines/performance.md
@@ -38,7 +38,6 @@ This works well for the following reasons:
    with fixed keys:
    - `.setup-test-env-cache`
    - `.ruby-cache`
-   - `.rails-cache`
    - `.static-analysis-cache`
    - `.rubocop-cache`
    - `.coverage-cache`
@@ -139,12 +138,3 @@ and `compile-production-assets` jobs to:
    This task is responsible for deciding if assets need to be compiled or not.
    It [compares the `HEAD` `SHA256` hexdigest from `$GITLAB_ASSETS_HASH` with the `master` hexdigest from `cached-assets-hash.txt`](https://gitlab.com/gitlab-org/gitlab/-/blob/c023191ef412e868ae957f3341208a41ca678403/lib/tasks/gitlab/assets.rake#L86).
 1. If the hashes are the same, we don't compile anything. If they're different, we compile the assets.
-
-## Pre-clone step
-
-NOTE:
-We no longer use this optimization for `gitlab-org/gitlab` because the [pack-objects cache](../../administration/gitaly/configure_gitaly.md#pack-objects-cache)
-allows Gitaly to serve the full CI/CD fetch traffic now. See [Git fetch caching](#git-fetch-caching).
-
-The pre-clone step works by using the `CI_PRE_CLONE_SCRIPT` variable
-[defined by GitLab.com shared runners](../../ci/runners/saas/linux_saas_runner.md#pre-clone-script-deprecated).
diff --git a/doc/development/product_qualified_lead_guide/index.md b/doc/development/product_qualified_lead_guide/index.md
index fb8ec478840..9f5a1a1110f 100644
--- a/doc/development/product_qualified_lead_guide/index.md
+++ b/doc/development/product_qualified_lead_guide/index.md
@@ -87,7 +87,7 @@ The hand-raise lead form accepts the following parameters via provide or inject.
     },
 ```
 
-The `ctaTracking` parameters follow [the `data-track` attributes](../snowplow/implementation.md#data-track-attributes) for implementing Snowplow tracking. The provided tracking attributes are attached to the button inside the `HandRaiseLeadButton` component, which triggers the hand-raise lead modal when selected.
+The `ctaTracking` parameters follow [the `data-track` attributes](../internal_analytics/snowplow/implementation.md#data-track-attributes) for implementing Snowplow tracking. The provided tracking attributes are attached to the button inside the `HandRaiseLeadButton` component, which triggers the hand-raise lead modal when selected.
 
 ### Monitor the lead location
 
diff --git a/doc/development/rails_endpoints/index.md b/doc/development/rails_endpoints/index.md
new file mode 100644
index 00000000000..c5a166dd4be
--- /dev/null
+++ b/doc/development/rails_endpoints/index.md
@@ -0,0 +1,79 @@
+---
+stage: Create
+group: Source Code
+info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments"
+type: reference, api
+---
+
+# Rails Endpoints
+
+Rails Endpoints are used by different GitLab components, they cannot be
+used by other consumers. This documentation is intended for people
+working on the GitLab codebase.
+
+These Rails Endpoints:
+
+- May not have extensive documentation or follow the same conventions as our public or private APIs.
+- May not adhere to standardized rules or guidelines.
+- Are designed to serve specific internal purposes in the codebase.
+- Are subject to change at any time.
+
+## Proof of concept period: Feedback Request
+
+We are currently evaluating a new approach for documenting Rails endpoints. Please [check out the Feedback Issue](https://gitlab.com/gitlab-org/gitlab/-/issues/411605) and feel free to share your thoughts, suggestions, or concerns. We appreciate your participation in helping us improve the documentation!
+
+## SAST Scanners
+
+Static Application Security Testing (SAST) checks your source code for known vulnerabilities. When SAST is enabled
+on a Project these endpoints are available.
+
+### List existing merge request code quality findings sorted by files
+
+Get a list of existing code quality Findings, if any, sorted by files.
+
+```plaintext
+GET /projects/:id/merge_requests/:merge_request_iid/codequality_mr_diff_reports.json
+```
+
+Response:
+
+```json
+{
+  "files": {
+    "index.js": [
+      {
+        "line": 1,
+        "description": "Unexpected 'debugger' statement.",
+        "severity": "major"
+      }
+    ]
+  }
+}
+```
+
+### List new, resolved and existing merge request code quality findings
+
+Get a list of new, resolved, and existing code quality Findings, if any.
+
+```plaintext
+GET /projects/:id/merge_requests/:merge_request_iid/codequality_reports.json
+```
+
+```json
+{
+  "status": "failed",
+  "new_errors": [
+    {
+      "description": "Unexpected 'debugger' statement.",
+      "severity": "major",
+      "file_path": "index.js",
+      "line": 1,
+      "web_url": "https://gitlab.com/jannik_lehmann/code-quality-test/-/blob/ed1c1b3052fe6963beda0e416d5e2ba3378eb715/noise.rb#L12",
+      "engine_name": "eslint"
+    }
+  ],
+  "resolved_errors": [],
+  "existing_errors": [],
+  "summary": { "total": 1, "resolved": 0, "errored": 1 }
+}
+```
diff --git a/doc/development/rails_initializers.md b/doc/development/rails_initializers.md
index 24647429a57..93a7b568974 100644
--- a/doc/development/rails_initializers.md
+++ b/doc/development/rails_initializers.md
@@ -6,6 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # Rails initializers
 
+Initializers are executed when the Rails process is started. That means that initializers are also executed during every deploy.
+
 By default, Rails loads Zeitwerk after the initializers in `config/initializers` are loaded.
 Autoloading before Zeitwerk is loaded is now deprecated but because we use a lot of autoloaded
 constants in our initializers, we had to move the loading of Zeitwerk earlier than these
diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md
index 8261330223d..0305cbf25b4 100644
--- a/doc/development/rake_tasks.md
+++ b/doc/development/rake_tasks.md
@@ -310,7 +310,7 @@ One way to generate the initial list is to run the Rake task `rubocop:todo:gener
 bundle exec rake rubocop:todo:generate
 ```
 
-To generate TODO list for specific RuboCop rules, pass them comma-seperated as
+To generate TODO list for specific RuboCop rules, pass them comma-separated as
 argument to the Rake task:
 
 ```shell
diff --git a/doc/development/reusing_abstractions.md b/doc/development/reusing_abstractions.md
index 3bacea859ef..bfb568d2fd2 100644
--- a/doc/development/reusing_abstractions.md
+++ b/doc/development/reusing_abstractions.md
@@ -302,7 +302,7 @@ Some examples:
 - [`DesignManagement::DesignAtVersion`](https://gitlab.com/gitlab-org/gitlab/-/blob/b62ce98cff8e0530210670f9cb0314221181b77f/app/models/design_management/design_at_version.rb)
   is a model that leverages validations to combine designs and versions.
 - [`Ci::Minutes::Usage`](https://gitlab.com/gitlab-org/gitlab/-/blob/ec52f19f7325410177c00fef06379f55ab7cab67/ee/app/models/ci/minutes/usage.rb)
-  is a Value Object that provides [CI/CD minutes usage](../ci/pipelines/cicd_minutes.md)
+  is a Value Object that provides [compute usage](../ci/pipelines/cicd_minutes.md)
   for a given namespace.
 
 #### Model class methods
diff --git a/doc/development/search/advanced_search_migration_styleguide.md b/doc/development/search/advanced_search_migration_styleguide.md
index 2f8cd036dcf..d7c0dddee7b 100644
--- a/doc/development/search/advanced_search_migration_styleguide.md
+++ b/doc/development/search/advanced_search_migration_styleguide.md
@@ -54,7 +54,7 @@ The following migration helpers are available in `ee/app/workers/concerns/elasti
 
 Backfills a specific field in an index. In most cases, the mapping for the field should already be added.
 
-Requires the `index_name` and `field_name` methods.
+Requires the `index_name` and `field_name` methods to backfill a single field.
 
 ```ruby
 class MigrationName < Elastic::Migration
@@ -72,6 +72,24 @@ class MigrationName < Elastic::Migration
 end
 ```
 
+Requires the `index_name` and `field_names` methods to backfill multiple fields if any field is null.
+
+```ruby
+class MigrationName < Elastic::Migration
+  include Elastic::MigrationBackfillHelper
+
+  private
+
+  def index_name
+    Issue.__elasticsearch__.index_name
+  end
+
+  def field_names
+    %w[schema_version visibility_level]
+  end
+end
+```
+
 #### `Elastic::MigrationUpdateMappingsHelper`
 
 Updates a mapping in an index by calling `put_mapping` with the mapping specified.
@@ -152,6 +170,34 @@ class MigrationName < Elastic::Migration
 end
 ```
 
+#### `Elastic::MigrationCreateIndex`
+
+Creates a new index.
+
+Requires:
+
+- The `target_class` and `document_type` methods
+- Mappings and index settings for the class in `ee/lib/elastic/latest/` and `ee/lib/elastic/v12p1/`
+
+WARNING:
+You must perform a follow-up migration to populate the index in the same milestone.
+
+```ruby
+class MigrationName < Elastic::Migration
+  include Elastic::MigrationCreateIndex
+
+  retry_on_failure
+
+  def document_type
+    :epic
+  end
+
+  def target_class
+    Epic
+  end
+end
+```
+
 #### `Elastic::MigrationHelper`
 
 Contains methods you can use when a migration doesn't fit the previous examples.
@@ -215,9 +261,22 @@ class BatchedMigrationName < Elastic::Migration
 end
 ```
 
+## Avoiding downtime in migrations
+
+### Reverting a migration
+
+If a migration fails or is halted on GitLab.com, we prefer to revert the change that introduced the migration. This
+prevents self-managed customers from receiving a broken migration and reduces the need for backports.
+
+### When to merge
+
+We prefer not to merge migrations within 1 week of the release. This allows time for a revert if a migration fails or
+doesn't work as expected. Migrations still in development or review during the final week of the release should be pushed
+to the next milestone.
+
 ### Multi-version compatibility
 
-These advanced search migrations, like any other GitLab changes, need to support the case where
+Advanced search migrations, like any other GitLab changes, need to support the case where
 [multiple versions of the application are running at the same time](../multi_version_compatibility.md).
 
 Depending on the order of deployment, it's possible that the migration
@@ -225,7 +284,7 @@ has started or finished and there's still a server running the application code
 migration. We need to take this into consideration until we can
 [ensure all advanced search migrations start after the deployment has finished](https://gitlab.com/gitlab-org/gitlab/-/issues/321619).
 
-### Reverting a migration
+### High risk migrations
 
 Because Elasticsearch does not support transactions, we always need to design our
 migrations to accommodate a situation where the application
@@ -236,7 +295,26 @@ some data is moved) to a later merge request after the migrations have
 completed successfully. To be safe, for self-managed customers we should also
 defer it to another release if there is risk of important data loss.
 
-### Best practices for advanced search migrations
+## Calculating migration runtime
+
+It's important to understand how long a migration might take to run on GitLab.com. Derive the number of documents that
+will be processed by the migration. This number may come from querying the database or an existing Elasticsearch index.
+Use the following formula to calculate the runtime:
+
+```ruby
+> batch_size = 9_000
+=> 9000
+> throttle_delay = 1.minute
+=> 1 minute
+> number_of_documents = 15_536_906
+=> 15536906
+> (number_of_documents / batch_size) * throttle_delay
+=> 1726 minutes
+> (number_of_documents / batch_size) * throttle_delay / 1.hour
+=> 28
+```
+
+## Best practices for advanced search migrations
 
 Follow these best practices for best results:
 
diff --git a/doc/development/sec/analyzer_development_guide.md b/doc/development/sec/analyzer_development_guide.md
index 31058427b55..af8d1758713 100644
--- a/doc/development/sec/analyzer_development_guide.md
+++ b/doc/development/sec/analyzer_development_guide.md
@@ -88,7 +88,7 @@ The following independent criteria determine which analyzer needs to be run on a
 1. Each analyzer runs a customizable [match interface](https://gitlab.com/gitlab-org/security-products/analyzers/common/-/blob/master/search/search.go) before it performs the actual analysis. For example: [Flawfinder checks for C/C++ files](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder/-/blob/f972ac786268fb649553056a94cda05cdc1248b2/plugin/plugin.go#L14).
 1. For some analyzers that run on generic file extensions, there is a check based on a CI/CD variable. For example: Kubernetes manifests are written in YAML, so [Kubesec](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) runs only when [`SCAN_KUBERNETES_MANIFESTS` is set to true](../../user/application_security/sast/index.md#enabling-kubesec-analyzer).
 
-Step 1 helps prevent wastage of CI/CD minutes that would be spent running analyzers not suitable for the project. However, due to [technical limitations](https://gitlab.com/gitlab-org/gitlab/-/issues/227632), it cannot be used for large projects. Therefore, step 2 acts as final check to ensure a mismatched analyzer is able to exit early.
+Step 1 helps prevent wastage of compute quota that would be spent running analyzers not suitable for the project. However, due to [technical limitations](https://gitlab.com/gitlab-org/gitlab/-/issues/227632), it cannot be used for large projects. Therefore, step 2 acts as final check to ensure a mismatched analyzer is able to exit early.
 
 ## How to test the analyzers
 
@@ -361,3 +361,9 @@ This issue will guide you through the whole release process. In general, you hav
     - Max role: `Developer`
     - Scope of the associated `GITLAB_TOKEN`:
     - Expiry Date of the associated `GITLAB_TOKEN`:
+
+#### Dependency updates
+
+All dependencies and upstream scanners (if any) used in the analyzer source are updated on a monthly cadence which primarily includes security fixes and non-breaking changes.
+
+- Static Analysis team uses a custom internal tool ([SastBot](https://gitlab.com/gitlab-org/security-products/analyzers/sast-analyzer-deps-bot#dependency-update-automation)) to automate dependency management of all the SAST analyzers. SastBot generates MRs on the **8th of each month** and distributes their assignment among Static Analysis team members to take them forward for review. For details on the process, see [Dependency Update Automation](https://gitlab.com/gitlab-org/security-products/analyzers/sast-analyzer-deps-bot#dependency-update-automation).
diff --git a/doc/development/sec/CycloneDX_property_taxonomy.md b/doc/development/sec/cyclonedx_property_taxonomy.md
index 6d09529a194..6d09529a194 100644
--- a/doc/development/sec/CycloneDX_property_taxonomy.md
+++ b/doc/development/sec/cyclonedx_property_taxonomy.md
diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md
index 7a3dc1c01fc..c5e7a58af0d 100644
--- a/doc/development/secure_coding_guidelines.md
+++ b/doc/development/secure_coding_guidelines.md
@@ -344,7 +344,7 @@ Much of the impact is contingent upon the function of the application and the ca
 
 For a demonstration of the impact on GitLab with a realistic attack scenario, see [this video on the GitLab Unfiltered channel](https://www.youtube.com/watch?v=t4PzHNycoKo) (internal, it requires being logged in with the GitLab Unfiltered account).
 
-### When to consider?
+### When to consider
 
 When user submitted data is included in responses to end users, which is just about anywhere.
 
@@ -485,7 +485,7 @@ In order to prevent Path Traversal vulnerabilities, user-controlled filenames or
 
 #### GitLab specific validations
 
-The methods `Gitlab::Utils.check_path_traversal!()` and `Gitlab::Utils.check_allowed_absolute_path!()`
+The methods `Gitlab::PathTraversal.check_path_traversal!()` and `Gitlab::PathTraversal.check_allowed_absolute_path!()`
 can be used to validate user-supplied paths and prevent vulnerabilities.
 `check_path_traversal!()` will detect their Path Traversal payloads and accepts URL-encoded paths.
 `check_allowed_absolute_path!()` will check if a path is absolute and whether it is inside the allowed path list. By default, absolute
@@ -495,7 +495,7 @@ parameter when using `check_allowed_absolute_path!()`.
 To use a combination of both checks, follow the example below:
 
 ```ruby
-Gitlab::Utils.check_allowed_absolute_path_and_path_traversal!(path, path_allowlist)
+Gitlab::PathTraversal.check_allowed_absolute_path_and_path_traversal!(path, path_allowlist)
 ```
 
 In the REST API, we have the [`FilePath`](https://gitlab.com/gitlab-org/security/gitlab/-/blob/master/lib/api/validations/validators/file_path.rb)
@@ -1395,3 +1395,20 @@ Additional resources:
 - <https://github.com/EthicalML/fml-security#exploring-the-owasp-top-10-for-ml>
 - <https://learn.microsoft.com/en-us/security/engineering/threat-modeling-aiml>
 - <https://learn.microsoft.com/en-us/security/engineering/failure-modes-in-machine-learning>
+
+## Local Storage
+
+### Description
+
+Local storage uses a built-in browser storage feature that caches data in read-only UTF-16 key-value pairs. Unlike `sessionStorage`, this mechanism has no built-in expiration mechanism, which can lead to large troves of potentially sensitive information being stored for indefinite periods.
+
+### Impact
+
+Local storage is subject to exfiltration during XSS attacks. These type of attacks highlight the inherent insecurity of storing sensitive information locally.
+
+### Mitigations
+
+If circumstances dictate that local storage is the only option, a couple of precautions should be taken.
+
+- Local storage should only be used for the minimal amount of data possible. Consider alternative storage formats.
+- If you have to store sensitive data using local storage, do so for the minimum time possible, calling `localStorage.removeItem` on the item as soon as we're done with it. Another alternative is to call `localStorage.clear()`.
diff --git a/doc/development/service_ping/implement.md b/doc/development/service_ping/implement.md
index 73b74feb239..c1077793fb9 100644
--- a/doc/development/service_ping/implement.md
+++ b/doc/development/service_ping/implement.md
@@ -1,884 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/service_ping/implement.md'
+remove_date: '2023-08-20'
 ---
 
-# Implement Service Ping
+This document was moved to [another location](../internal_analytics/service_ping/implement.md).
 
-Service Ping consists of two kinds of data:
-
-- **Counters**: Track how often a certain event happened over time, such as how many CI/CD pipelines have run.
-  They are monotonic and usually trend up.
-- **Observations**: Facts collected from one or more GitLab instances and can carry arbitrary data.
-  There are no general guidelines for how to collect those, due to the individual nature of that data.
-
-To implement a new metric in Service Ping, follow these steps:
-
-1. [Implement the required counter](#types-of-counters)
-1. [Name and place the metric](metrics_dictionary.md#metric-key_path)
-1. [Test counters manually using your Rails console](#test-counters-manually-using-your-rails-console)
-1. [Generate the SQL query](#generate-the-sql-query)
-1. [Optimize queries with Database Lab](#optimize-queries-with-database-lab)
-1. [Add the metric definition to the Metrics Dictionary](#add-the-metric-definition)
-1. [Add the metric to the Versions Application](#add-the-metric-to-the-versions-application)
-1. [Create a merge request](#create-a-merge-request)
-1. [Verify your metric](#verify-your-metric)
-1. [Set up and test Service Ping locally](#set-up-and-test-service-ping-locally)
-
-## Instrumentation classes
-
-NOTE:
-Implementing metrics directly in `usage_data.rb` is deprecated.
-When you add or change a Service Ping Metric, you must migrate metrics to [instrumentation classes](metrics_instrumentation.md).
-For information about the progress on migrating Service Ping metrics, see this [epic](https://gitlab.com/groups/gitlab-org/-/epics/5547).
-
-For example, we have the following instrumentation class:
-`lib/gitlab/usage/metrics/instrumentations/count_boards_metric.rb`.
-
-You should add it to `usage_data.rb` as follows:
-
-```ruby
-boards: add_metric('CountBoardsMetric', time_frame: 'all'),
-```
-
-## Types of counters
-
-There are several types of counters for metrics:
-
-- **[Batch counters](#batch-counters)**: Used for counts, sums, and averages.
-- **[Redis counters](#redis-counters):** Used for in-memory counts.
-- **[Alternative counters](#alternative-counters):** Used for settings and configurations.
-
-NOTE:
-Only use the provided counter methods. Each counter method contains a built-in fail-safe mechanism that isolates each counter to avoid breaking the entire Service Ping process.
-
-### Batch counters
-
-For large tables, PostgreSQL can take a long time to count rows due to MVCC [(Multi-version Concurrency Control)](https://en.wikipedia.org/wiki/Multiversion_concurrency_control). Batch counting is a counting method where a single large query is broken into multiple smaller queries. For example, instead of a single query querying 1,000,000 records, with batch counting, you can execute 100 queries of 10,000 records each. Batch counting is useful for avoiding database timeouts as each batch query is significantly shorter than one single long running query.
-
-For GitLab.com, there are extremely large tables with 15 second query timeouts, so we use batch counting to avoid encountering timeouts. Here are the sizes of some GitLab.com tables:
-
-| Table                        | Row counts in millions |
-|------------------------------|------------------------|
-| `merge_request_diff_commits` | 2280                   |
-| `ci_build_trace_sections`    | 1764                   |
-| `merge_request_diff_files`   | 1082                   |
-| `events`                     | 514                    |
-
-Batch counting requires indexes on columns to calculate max, min, and range queries. In some cases,
-you must add a specialized index on the columns involved in a counter.
-
-#### Ordinary batch counters
-
-Create a new [database metrics](metrics_instrumentation.md#database-metrics) instrumentation class with `count` operation for a given `ActiveRecord_Relation`
-
-Method:
-
-```ruby
-add_metric('CountIssuesMetric', time_frame: 'all')
-```
-
-Examples:
-
-Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md).
-
-#### Distinct batch counters
-
-Create a new [database metrics](metrics_instrumentation.md#database-metrics) instrumentation class with `distinct_count` operation for a given `ActiveRecord_Relation`.
-
-Method:
-
-```ruby
-add_metric('CountUsersAssociatingMilestonesToReleasesMetric', time_frame: 'all')
-```
-
-WARNING:
-Counting over non-unique columns can lead to performance issues. For more information, see the [iterating tables in batches](../database/iterating_tables_in_batches.md) guide.
-
-Examples:
-
-Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md).
-
-#### Sum batch operation
-
-Sum the values of a given ActiveRecord_Relation on given column and handles errors.
-Handles the `ActiveRecord::StatementInvalid` error
-
-Method:
-
-```ruby
-add_metric('JiraImportsTotalImportedIssuesCountMetric')
-```
-
-#### Average batch operation
-
-Average the values of a given `ActiveRecord_Relation` on given column and handles errors.
-
-Method:
-
-```ruby
-add_metric('CountIssuesWeightAverageMetric')
-```
-
-Examples:
-
-Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md).
-
-#### Grouping and batch operations
-
-The `count`, `distinct_count` and `sum` batch counters can accept an `ActiveRecord::Relation`
-object, which groups by a specified column. With a grouped relation, the methods do batch counting,
-handle errors, and returns a hash table of key-value pairs.
-
-Examples:
-
-```ruby
-count(Namespace.group(:type))
-# returns => {nil=>179, "Group"=>54}
-
-distinct_count(Project.group(:visibility_level), :creator_id)
-# returns => {0=>1, 10=>1, 20=>11}
-
-sum(Issue.group(:state_id), :weight))
-# returns => {1=>3542, 2=>6820}
-```
-
-#### Add operation
-
-Sum the values given as parameters. Handles the `StandardError`.
-Returns `-1` if any of the arguments are `-1`.
-
-Method:
-
-```ruby
-add(*args)
-```
-
-Examples:
-
-```ruby
-project_imports = distinct_count(::Project.where.not(import_type: nil), :creator_id)
-bulk_imports = distinct_count(::BulkImport, :user_id)
-
- add(project_imports, bulk_imports)
-```
-
-#### Estimated batch counters
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/48233) in GitLab 13.7.
-
-Estimated batch counter functionality handles `ActiveRecord::StatementInvalid` errors
-when used through the provided `estimate_batch_distinct_count` method.
-Errors return a value of `-1`.
-
-WARNING:
-This functionality estimates a distinct count of a specific ActiveRecord_Relation in a given column,
-which uses the [HyperLogLog](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40671.pdf) algorithm.
-As the HyperLogLog algorithm is probabilistic, the **results always include error**.
-The highest encountered error rate is 4.9%.
-
-When correctly used, the `estimate_batch_distinct_count` method enables efficient counting over
-columns that contain non-unique values, which cannot be assured by other counters.
-
-##### `estimate_batch_distinct_count` method
-
-Method:
-
-```ruby
-estimate_batch_distinct_count(relation, column = nil, batch_size: nil, start: nil, finish: nil)
-```
-
-The [method](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/utils/usage_data.rb#L63)
-includes the following arguments:
-
-- `relation`: The ActiveRecord_Relation to perform the count.
-- `column`: The column to perform the distinct count. The default is the primary key.
-- `batch_size`: From `Gitlab::Database::PostgresHll::BatchDistinctCounter::DEFAULT_BATCH_SIZE`. Default value: 10,000.
-- `start`: The custom start of the batch count, to avoid complex minimum calculations.
-- `finish`: The custom end of the batch count to avoid complex maximum calculations.
-
-The method includes the following prerequisites:
-
-- The supplied `relation` must include the primary key defined as the numeric column.
-  For example: `id bigint NOT NULL`.
-- The `estimate_batch_distinct_count` can handle a joined relation. To use its ability to
-  count non-unique columns, the joined relation **must not** have a one-to-many relationship,
-  such as `has_many :boards`.
-- Both `start` and `finish` arguments should always represent primary key relationship values,
-  even if the estimated count refers to another column, for example:
-
-  ```ruby
-    estimate_batch_distinct_count(::Note, :author_id, start: ::Note.minimum(:id), finish: ::Note.maximum(:id))
-  ```
-
-Examples:
-
-1. Simple execution of estimated batch counter, with only relation provided,
-   returned value represents estimated number of unique values in `id` column
-   (which is the primary key) of `Project` relation:
-
-   ```ruby
-     estimate_batch_distinct_count(::Project)
-   ```
-
-1. Execution of estimated batch counter, where provided relation has applied
-   additional filter (`.where(time_period)`), number of unique values estimated
-   in custom column (`:author_id`), and parameters: `start` and `finish` together
-   apply boundaries that defines range of provided relation to analyze:
-
-   ```ruby
-     estimate_batch_distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::Note.minimum(:id), finish: ::Note.maximum(:id))
-   ```
-
-When instrumenting metric with usage of estimated batch counter please add
-`_estimated` suffix to its name, for example:
-
-```ruby
-  "counts": {
-    "ci_builds_estimated": estimate_batch_distinct_count(Ci::Build),
-    ...
-```
-
-### Redis counters
-
-Handles `::Redis::CommandError` and `Gitlab::UsageDataCounters::BaseCounter::UnknownEvent`.
-Returns -1 when a block is sent or hash with all values and -1 when a `counter(Gitlab::UsageDataCounters)` is sent.
-The different behavior is due to 2 different implementations of the Redis counter.
-
-Method:
-
-```ruby
-redis_usage_data(counter, &block)
-```
-
-Arguments:
-
-- `counter`: a counter from `Gitlab::UsageDataCounters`, that has `fallback_totals` method implemented
-- or a `block`: which is evaluated
-
-#### Ordinary Redis counters
-
-Example of implementation: [`Gitlab::UsageDataCounters::WikiPageCounter`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/wiki_page_counter.rb), using Redis methods [`INCR`](https://redis.io/commands/incr/) and [`GET`](https://redis.io/commands/get/).
-
-Events are handled by counter classes in the `Gitlab::UsageDataCounters` namespace, inheriting from `BaseCounter`, that are either:
-
-1. Listed in [`Gitlab::UsageDataCounters::COUNTERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters.rb#L5) to be then included in `Gitlab::UsageData`.
-
-1. Specified in the metric definition using the `RedisMetric` instrumentation class by their `prefix` option to be picked up using the [metric instrumentation](metrics_instrumentation.md) framework. Refer to the [Redis metrics](metrics_instrumentation.md#redis-metrics) documentation for an example implementation.
-
-Inheriting classes are expected to override `KNOWN_EVENTS` and `PREFIX` constants to build event names and associated metrics. For example, for prefix `issues` and events array `%w[create, update, delete]`, three metrics will be added to the Service Ping payload: `counts.issues_create`, `counts.issues_update` and `counts.issues_delete`.
-
-##### `UsageData` API
-
-You can use the `UsageData` API to track events.
-To track events, the `usage_data_api` feature flag must
-be enabled (set to `default_enabled: true`).
-Enabled by default in GitLab 13.7 and later.
-
-##### UsageData API tracking
-
-1. Track events using the [`UsageData` API](#usagedata-api).
-
-   Increment event count using an ordinary Redis counter, for a given event name.
-
-   API requests are protected by checking for a valid CSRF token.
-
-   ```plaintext
-   POST /usage_data/increment_counter
-   ```
-
-   | Attribute | Type | Required | Description |
-   | :-------- | :--- | :------- | :---------- |
-   | `event` | string | yes | The event name to track. |
-
-   Response:
-
-   - `200` if the event was tracked.
-   - `400 Bad request` if the event parameter is missing.
-   - `401 Unauthorized` if the user is not authenticated.
-   - `403 Forbidden` if an invalid CSRF token is provided.
-
-1. Track events using the JavaScript/Vue API helper which calls the [`UsageData` API](#usagedata-api).
-
-   To track events, `usage_data_api` and `usage_data_#{event_name}` must be enabled.
-
-   ```javascript
-   import api from '~/api';
-
-   api.trackRedisCounterEvent('my_already_defined_event_name'),
-   ```
-
-#### Redis HLL counters
-
-WARNING:
-HyperLogLog (HLL) is a probabilistic algorithm and its **results always includes some small error**. According to [Redis documentation](https://redis.io/commands/pfcount/), data from
-used HLL implementation is "approximated with a standard error of 0.81%".
-
-NOTE:
- A user's consent for `usage_stats` (`User.single_user&.requires_usage_stats_consent?`) is not checked during the data tracking stage due to performance reasons. Keys corresponding to those counters are present in Redis even if `usage_stats_consent` is still required. However, no metric is collected from Redis and reported back to GitLab as long as `usage_stats_consent` is required.
-
-With `Gitlab::UsageDataCounters::HLLRedisCounter` we have available data structures used to count unique values.
-
-Implemented using Redis methods [PFADD](https://redis.io/commands/pfadd/) and [PFCOUNT](https://redis.io/commands/pfcount/).
-
-##### Add new events
-
-1. Define events in [`known_events`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/).
-
-   Example event:
-
-   ```yaml
-   - name: users_creating_epics
-     aggregation: weekly
-   ```
-
-   Keys:
-
-   - `name`: unique event name.
-
-     Name format for Redis HLL events `{hll_counters}_<name>`
-
-     [See Metric name](metrics_dictionary.md#metric-name) for a complete guide on metric naming suggestion.
-
-     Example names: `users_creating_epics`, `users_triggering_security_scans`.
-
-   - `aggregation`: may be set to a `:daily` or `:weekly` key. Defines how counting data is stored in Redis.
-     Aggregation on a `daily` basis does not pull more fine grained data.
-
-1. Use one of the following methods to track the event:
-
-   - In the controller using the `ProductAnalyticsTracking` module and the following format:
-
-     ```ruby
-     track_event(*controller_actions, name:, action:, label:, conditions: nil, destinations: [:redis_hll], &block)
-     ```
-
-     Arguments:
-
-     - `controller_actions`: the controller actions to track.
-     - `name`: the event name.
-     - `action`: required if destination is `:snowplow. Action name for the triggered event. See [event schema](../snowplow/index.md#event-schema) for more details.
-     - `label`: required if destination is `:snowplow. Label for the triggered event. See [event schema](../snowplow/index.md#event-schema) for more details.
-     - `conditions`: optional custom conditions. Uses the same format as Rails callbacks.
-     - `destinations`: optional list of destinations. Currently supports `:redis_hll` and `:snowplow`. Default: `:redis_hll`.
-     - `&block`: optional block that computes and returns the `custom_id` that we want to track. This overrides the `visitor_id`.
-
-     Example:
-
-     ```ruby
-     # controller
-     class ProjectsController < Projects::ApplicationController
-       include ProductAnalyticsTracking
-
-       skip_before_action :authenticate_user!, only: :show
-       track_event :index, :show,
-         name: 'users_visiting_projects',
-         action: 'user_perform_visit',
-         label: 'redis_hll_counters.users_visiting_project_monthly',
-         destinations: %i[redis_hll snowplow]
-
-       def index
-         render html: 'index'
-       end
-
-      def new
-        render html: 'new'
-      end
-
-      def show
-        render html: 'show'
-      end
-     end
-     ```
-
-   - In the API using the `increment_unique_values(event_name, values)` helper method.
-
-     Arguments:
-
-     - `event_name`: the event name.
-     - `values`: the values counted. Can be one value or an array of values.
-
-     Example:
-
-     ```ruby
-     get ':id/registry/repositories' do
-       repositories = ContainerRepositoriesFinder.new(
-         user: current_user, subject: user_group
-       ).execute
-
-       increment_unique_values('users_listing_repositories', current_user.id)
-
-       present paginate(repositories), with: Entities::ContainerRegistry::Repository, tags: params[:tags], tags_count: params[:tags_count]
-     end
-     ```
-
-   - Using `track_usage_event(event_name, values)` in services and GraphQL.
-
-     Increment unique values count using Redis HLL, for a given event name.
-
-     Examples:
-
-     - [Track usage event for an incident in a service](https://gitlab.com/gitlab-org/gitlab/-/blob/v13.8.3-ee/app/services/issues/update_service.rb#L66)
-     - [Track usage event for an incident in GraphQL](https://gitlab.com/gitlab-org/gitlab/-/blob/v13.8.3-ee/app/graphql/mutations/alert_management/update_alert_status.rb#L16)
-
-     ```ruby
-       track_usage_event(:incident_management_incident_created, current_user.id)
-     ```
-
-   - Using the [`UsageData` API](#usagedata-api).
-
-     Increment unique users count using Redis HLL, for a given event name.
-
-     API requests are protected by checking for a valid CSRF token.
-
-     ```plaintext
-     POST /usage_data/increment_unique_users
-     ```
-
-     | Attribute | Type | Required | Description |
-     | :-------- | :--- | :------- | :---------- |
-     | `event` | string | yes | The event name to track |
-
-     Response:
-
-     - `200` if the event was tracked, or if tracking failed for any reason.
-     - `400 Bad request` if an event parameter is missing.
-     - `401 Unauthorized` if the user is not authenticated.
-     - `403 Forbidden` if an invalid CSRF token is provided.
-
-   - Using the JavaScript/Vue API helper, which calls the [`UsageData` API](#usagedata-api).
-
-     Example for an existing event already defined in [known events](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/):
-
-     ```javascript
-     import api from '~/api';
-
-     api.trackRedisHllUserEvent('my_already_defined_event_name'),
-     ```
-
-1. Get event data using `Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names:, start_date:, end_date:, context: '')`.
-
-   Arguments:
-
-   - `event_names`: the list of event names.
-   - `start_date`: start date of the period for which we want to get event data.
-   - `end_date`: end date of the period for which we want to get event data.
-   - `context`: context of the event. Allowed values are `default`, `free`, `bronze`, `silver`, `gold`, `starter`, `premium`, `ultimate`.
-
-1. Testing tracking and getting unique events
-
-Trigger events in rails console by using `track_event` method
-
-   ```ruby
-   Gitlab::UsageDataCounters::HLLRedisCounter.track_event('users_viewing_compliance_audit_events', values: 1)
-   Gitlab::UsageDataCounters::HLLRedisCounter.track_event('users_viewing_compliance_audit_events', values: [2, 3])
-   ```
-
-Next, get the unique events for the current week.
-
-   ```ruby
-   # Get unique events for metric for current_week
-   Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'users_viewing_compliance_audit_events',
-   start_date: Date.current.beginning_of_week, end_date: Date.current.next_week)
-   ```
-
-##### Recommendations
-
-We have the following recommendations for [adding new events](#add-new-events):
-
-- Event aggregation: weekly.
-- When adding new metrics, use a [feature flag](../../operations/feature_flags.md) to control the impact.
-It's recommended to disable the new feature flag by default (set `default_enabled: false`).
-- Events can be triggered using the `UsageData` API, which helps when there are > 10 events per change
-
-##### Enable or disable Redis HLL tracking
-
-We can disable tracking completely by using the global flag:
-
-```shell
-/chatops run feature set redis_hll_tracking true
-/chatops run feature set redis_hll_tracking false
-```
-
-##### Known events are added automatically in Service Data payload
-
-Service Ping adds all events [`known_events/*.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events) to Service Data generation under the `redis_hll_counters` key. This column is stored in [version-app as a JSON](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/db/schema.rb#L209).
-For each event we add metrics for the weekly and monthly time frames, and totals for each where applicable:
-
-- `#{event_name}_weekly`: Data for 7 days for daily [aggregation](#add-new-events) events and data for the last complete week for weekly [aggregation](#add-new-events) events.
-- `#{event_name}_monthly`: Data for 28 days for daily [aggregation](#add-new-events) events and data for the last 4 complete weeks for weekly [aggregation](#add-new-events) events.
-
-Example of `redis_hll_counters` data:
-
-```ruby
-{:redis_hll_counters=>
-  {"compliance"=>
-    {"users_viewing_compliance_dashboard_weekly"=>0,
-     "users_viewing_compliance_dashboard_monthly"=>0,
-     "users_viewing_compliance_audit_events_weekly"=>0,
-     "users_viewing_audit_events_monthly"=>0,
-     "compliance_total_unique_counts_weekly"=>0,
-     "compliance_total_unique_counts_monthly"=>0},
- "analytics"=>
-    {"users_viewing_analytics_group_devops_adoption_weekly"=>0,
-     "users_viewing_analytics_group_devops_adoption_monthly"=>0,
-     "analytics_total_unique_counts_weekly"=>0,
-     "analytics_total_unique_counts_monthly"=>0},
-   "ide_edit"=>
-    {"users_editing_by_web_ide_weekly"=>0,
-     "users_editing_by_web_ide_monthly"=>0,
-     "users_editing_by_sfe_weekly"=>0,
-     "users_editing_by_sfe_monthly"=>0,
-     "ide_edit_total_unique_counts_weekly"=>0,
-     "ide_edit_total_unique_counts_monthly"=>0}
- }
-}
-```
-
-Example:
-
-```ruby
-# Redis Counters
-redis_usage_data(Gitlab::UsageDataCounters::WikiPageCounter)
-
-# Define events in common.yml https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/common.yml
-
-# Tracking events
-Gitlab::UsageDataCounters::HLLRedisCounter.track_event('users_expanding_vulnerabilities', values: visitor_id)
-
-# Get unique events for metric
-redis_usage_data { Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'users_expanding_vulnerabilities', start_date: 28.days.ago, end_date: Date.current) }
-```
-
-### Alternative counters
-
-Handles `StandardError` and fallbacks into -1 this way not all measures fail if we encounter one exception.
-Mainly used for settings and configurations.
-
-Method:
-
-```ruby
-alt_usage_data(value = nil, fallback: -1, &block)
-```
-
-Arguments:
-
-- `value`: a static value in which case the value is returned.
-- or a `block`: which is evaluated
-- `fallback: -1`: the common value used for any metrics that are failing.
-
-Example:
-
-```ruby
-alt_usage_data { Gitlab::VERSION }
-alt_usage_data { Gitlab::CurrentSettings.uuid }
-alt_usage_data(999)
-```
-
-### Add counters to build new metrics
-
-When adding the results of two counters, use the `add` Service Data method that
-handles fallback values and exceptions. It also generates a valid [SQL export](index.md#export-service-ping-data).
-
-Example:
-
-```ruby
-add(User.active, User.bot)
-```
-
-### Prometheus queries
-
-In those cases where operational metrics should be part of Service Ping, a database or Redis query is unlikely
-to provide useful data. Instead, Prometheus might be more appropriate, because most GitLab architectural
-components publish metrics to it that can be queried back, aggregated, and included as Service Data.
-
-NOTE:
-Prometheus as a data source for Service Ping is only available for single-node Omnibus installations
-that are running the [bundled Prometheus](../../administration/monitoring/prometheus/index.md) instance.
-
-To query Prometheus for metrics, a helper method is available to `yield` a fully configured
-`PrometheusClient`, given it is available as per the note above:
-
-```ruby
-with_prometheus_client do |client|
-  response = client.query('<your query>')
-  ...
-end
-```
-
-Refer to [the `PrometheusClient` definition](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/prometheus_client.rb)
-for how to use its API to query for data.
-
-### Fallback values for Service Ping
-
-We return fallback values in these cases:
-
-| Case                        | Value |
-|-----------------------------|-------|
-| Deprecated Metric ([Removed with version 14.3](https://gitlab.com/gitlab-org/gitlab/-/issues/335894)) | -1000 |
-| Timeouts, general failures  | -1    |
-| Standard errors in counters | -2    |
-| Histogram metrics failure   | { '-1' => -1 } |
-
-## Test counters manually using your Rails console
-
-```ruby
-# count
-Gitlab::UsageData.count(User.active)
-Gitlab::UsageData.count(::Clusters::Cluster.aws_installed.enabled, :cluster_id)
-
-# count distinct
-Gitlab::UsageData.distinct_count(::Project, :creator_id)
-Gitlab::UsageData.distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id))
-```
-
-## Generate the SQL query
-
-Your Rails console returns the generated SQL queries. For example:
-
-```ruby
-pry(main)> Gitlab::UsageData.count(User.active)
-   (2.6ms)  SELECT "features"."key" FROM "features"
-   (15.3ms)  SELECT MIN("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4))
-   (2.4ms)  SELECT MAX("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4))
-   (1.9ms)  SELECT COUNT("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4)) AND "users"."id" BETWEEN 1 AND 100000
-```
-
-## Optimize queries with Database Lab
-
-[Database Lab](../database/database_lab.md) is a service that uses a production clone to test queries.
-
-- GitLab.com's production database has a 15 second timeout.
-- Any single query must stay below the [1 second execution time](../database/query_performance.md#timing-guidelines-for-queries) with cold caches.
-- Add a specialized index on columns involved to reduce the execution time.
-
-To understand the query's execution, we add the following information
-to a merge request description:
-
-- For counters that have a `time_period` test, we add information for both:
-  - `time_period = {}` for all time periods.
-  - `time_period = { created_at: 28.days.ago..Time.current }` for the last 28 days.
-- Execution plan and query time before and after optimization.
-- Query generated for the index and time.
-- Migration output for up and down execution.
-
-For more details, see the [database review guide](../database_review.md#preparation-when-adding-or-modifying-queries).
-
-### Optimization recommendations and examples
-
-- Use specialized indexes. For examples, see these merge requests:
-  - [Example 1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26871)
-  - [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26445)
-- Use defined `start` and `finish`. These values can be memoized and reused, as in this
-  [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37155).
-- Avoid joins and unnecessary complexity in your queries. See this
-  [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36316) as an example.
-- Set a custom `batch_size` for `distinct_count`, as in this [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38000).
-
-## Add the metric definition
-
-See the [Metrics Dictionary guide](metrics_dictionary.md) for more information.
-
-## Add the metric to the Versions Application
-
-Check if the new metric must be added to the Versions Application. See the `usage_data` [schema](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/db/schema.rb#L147) and Service Data [parameters accepted](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/app/services/usage_ping.rb). Any metrics added under the `counts` key are saved in the `stats` column.
-
-## Create a merge request
-
-Create a merge request for the new Service Ping metric, and do the following:
-
-- Add the `feature` label to the merge request. A metric is a user-facing change and is part of expanding the Service Ping feature.
-- Add a changelog entry that complies with the [changelog entries guide](../changelog.md).
-- Ask for an Analytics Instrumentation review.
-  On GitLab.com, we have DangerBot set up to monitor Analytics Instrumentation related files and recommend a [Analytics Instrumentation review](review_guidelines.md).
-
-## Verify your metric
-
-On GitLab.com, the Product Intelligence team regularly [monitors Service Ping](https://gitlab.com/groups/gitlab-org/-/epics/6000).
-They may alert you that your metrics need further optimization to run quicker and with greater success.
-
-The Service Ping JSON payload for GitLab.com is shared in the
-[#g_product_intelligence](https://gitlab.slack.com/archives/CL3A7GFPF) Slack channel every week.
-
-You may also use the [Service Ping QA dashboard](https://app.periscopedata.com/app/gitlab/632033/Usage-Ping-QA) to check how well your metric performs.
-The dashboard allows filtering by GitLab version, by "Self-managed" and "SaaS", and shows you how many failures have occurred for each metric. Whenever you notice a high failure rate, you can re-optimize your metric.
-
-Use [Metrics Dictionary](https://metrics.gitlab.com/) [copy query to clipboard feature](https://www.youtube.com/watch?v=n4o65ivta48&list=PL05JrBw4t0Krg3mbR6chU7pXtMt_es6Pb) to get a query ready to run in Sisense for a specific metric.
-
-## Set up and test Service Ping locally
-
-To set up Service Ping locally, you must:
-
-1. [Set up local repositories](#set-up-local-repositories).
-1. [Test local setup](#test-local-setup).
-1. Optional. [Test Prometheus-based Service Ping](#test-prometheus-based-service-ping).
-
-### Set up local repositories
-
-1. Clone and start [GitLab](https://gitlab.com/gitlab-org/gitlab-development-kit).
-1. Clone and start [Versions Application](https://gitlab.com/gitlab-services/version-gitlab-com).
-   Make sure you run `docker-compose up` to start a PostgreSQL and Redis instance.
-1. Point GitLab to the Versions Application endpoint instead of the default endpoint:
-   1. Open [service_ping/submit_service.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb#L5) locally and modify `STAGING_BASE_URL`.
-   1. Set it to the local Versions Application URL: `http://localhost:3000`.
-
-### Test local setup
-
-1. Using the `gitlab` Rails console, manually trigger Service Ping:
-
-   ```ruby
-   GitlabServicePingWorker.new.perform('triggered_from_cron' => false)
-   ```
-
-1. Use the `versions` Rails console to check the Service Ping was successfully received,
-   parsed, and stored in the Versions database:
-
-   ```ruby
-   UsageData.last
-   ```
-
-## Test Prometheus-based Service Ping
-
-If the data submitted includes metrics [queried from Prometheus](#prometheus-queries)
-you want to inspect and verify, you must:
-
-- Ensure that a Prometheus server is running locally.
-- Ensure the respective GitLab components are exporting metrics to the Prometheus server.
-
-If you do not need to test data coming from Prometheus, no further action
-is necessary. Service Ping should degrade gracefully in the absence of a running Prometheus server.
-
-Three kinds of components may export data to Prometheus, and are included in Service Ping:
-
-- [`node_exporter`](https://github.com/prometheus/node_exporter): Exports node metrics
-  from the host machine.
-- [`gitlab-exporter`](https://gitlab.com/gitlab-org/gitlab-exporter): Exports process metrics
-  from various GitLab components.
-- Other various GitLab services, such as Sidekiq and the Rails server, which export their own metrics.
-
-### Test with an Omnibus container
-
-This is the recommended approach to test Prometheus-based Service Ping.
-
-To verify your change, build a new Omnibus image from your code branch using CI/CD, download the image,
-and run a local container instance:
-
-1. From your merge request, select the `qa` stage, then trigger the `e2e:package-and-test` job. This job triggers an Omnibus
-   build in a [downstream pipeline of the `omnibus-gitlab-mirror` project](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/-/pipelines).
-1. In the downstream pipeline, wait for the `gitlab-docker` job to finish.
-1. Open the job logs and locate the full container name including the version. It takes the following form: `registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>`.
-1. On your local machine, make sure you are signed in to the GitLab Docker registry. You can find the instructions for this in
-   [Authenticate to the GitLab Container Registry](../../user/packages/container_registry/authenticate_with_container_registry.md).
-1. Once signed in, download the new image by using `docker pull registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>`
-1. For more information about working with and running Omnibus GitLab containers in Docker, refer to [GitLab Docker images](../../install/docker.md) documentation.
-
-### Test with GitLab development toolkits
-
-This is the less recommended approach, because it comes with a number of difficulties when emulating a real GitLab deployment.
-
-The [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) is not set up to run a Prometheus server or `node_exporter` alongside other GitLab components. If you would
-like to do so, [Monitoring the GDK with Prometheus](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/prometheus/index.md#monitoring-the-gdk-with-prometheus) is a good start.
-
-The [GCK](https://gitlab.com/gitlab-org/gitlab-compose-kit) has limited support for testing Prometheus based Service Ping.
-By default, it comes with a fully configured Prometheus service that is set up to scrape a number of components.
-However, it has the following limitations:
-
-- It does not run a `gitlab-exporter` instance, so several `process_*` metrics from services such as Gitaly may be missing.
-- While it runs a `node_exporter`, `docker-compose` services emulate hosts, meaning that it normally reports itself as not associated
-  with any of the other running services. That is not how node metrics are reported in a production setup, where `node_exporter`
-  always runs as a process alongside other GitLab components on any given node. For Service Ping, none of the node data would therefore
-  appear to be associated to any of the services running, because they all appear to be running on different hosts. To alleviate this problem, the `node_exporter` in GCK was arbitrarily "assigned" to the `web` service, meaning only for this service `node_*` metrics appears in Service Ping.
-
-## Aggregated metrics
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/45979) in GitLab 13.6.
-
-WARNING:
-This feature is intended solely for internal GitLab use.
-
-The aggregated metrics feature provides insight into the data attributes in a collection of Service Ping metrics.
-This aggregation allows you to count data attributes in events without counting each occurrence of the same data attribute in multiple events.
-For example, you can aggregate the number of users who perform several actions, such as creating a new issue and opening a new merge request.
-You can then count each user that performed any combination of these actions.
-
-### Defining aggregated metric via metric YAML definition
-
-To add data for aggregated metrics to the Service Ping payload,
-create metric YAML definition file following [Aggregated metric instrumentation guide](metrics_instrumentation.md#aggregated-metrics).
-
-### Redis sourced aggregated metrics
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/45979) in GitLab 13.6.
-
-To declare the aggregate of events collected with [Redis HLL Counters](#redis-hll-counters),
-you must fulfill the following requirements:
-
-1. All events listed at `events` attribute must come from
-   [`known_events/*.yml`](#known-events-are-added-automatically-in-service-data-payload) files.
-1. All events listed at `events` attribute must have the same `aggregation` attribute.
-1. `time_frame` does not include `all` value, which is unavailable for Redis sourced aggregated metrics.
-
-While it is possible to aggregate EE-only events together with events that occur in all GitLab editions, it's important to remember that doing so may produce high variance between data collected from EE and CE GitLab instances.
-
-### Database sourced aggregated metrics
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52784) in GitLab 13.9.
-
-To declare an aggregate of metrics based on events collected from database, follow
-these steps:
-
-1. [Persist the metrics for aggregation](#persist-metrics-for-aggregation).
-1. [Add new aggregated metric definition](#add-new-aggregated-metric-definition).
-
-#### Persist metrics for aggregation
-
-Only metrics calculated with [Estimated Batch Counters](#estimated-batch-counters)
-can be persisted for database sourced aggregated metrics. To persist a metric,
-inject a Ruby block into the
-[`estimate_batch_distinct_count`](#estimate_batch_distinct_count-method) method.
-This block should invoke the
-`Gitlab::Usage::Metrics::Aggregates::Sources::PostgresHll.save_aggregated_metrics`
-[method](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage/metrics/aggregates/sources/postgres_hll.rb#L21),
-which stores `estimate_batch_distinct_count` results for future use in aggregated metrics.
-
-The `Gitlab::Usage::Metrics::Aggregates::Sources::PostgresHll.save_aggregated_metrics`
-method accepts the following arguments:
-
-- `metric_name`: The name of metric to use for aggregations. Should be the same
-  as the key under which the metric is added into Service Ping.
-- `recorded_at_timestamp`: The timestamp representing the moment when a given
-  Service Ping payload was collected. You should use the convenience method `recorded_at`
-  to fill `recorded_at_timestamp` argument, like this: `recorded_at_timestamp: recorded_at`
-- `time_period`: The time period used to build the `relation` argument passed into
-  `estimate_batch_distinct_count`. To collect the metric with all available historical
-  data, set a `nil` value as time period: `time_period: nil`.
-- `data`: HyperLogLog buckets structure representing unique entries in `relation`.
-  The `estimate_batch_distinct_count` method always passes the correct argument
-  into the block, so `data` argument must always have a value equal to block argument,
-  like this: `data: result`
-
-Example metrics persistence:
-
-```ruby
-class UsageData
-  def count_secure_pipelines(time_period)
-    ...
-    relation = ::Security::Scan.by_scan_types(scan_type).where(time_period)
-
-    pipelines_with_secure_jobs['dependency_scanning_pipeline'] = estimate_batch_distinct_count(relation, :pipeline_id, batch_size: 1000, start: start_id, finish: finish_id) do |result|
-      ::Gitlab::Usage::Metrics::Aggregates::Sources::PostgresHll
-        .save_aggregated_metrics(metric_name: 'dependency_scanning_pipeline', recorded_at_timestamp: recorded_at, time_period: time_period, data: result)
-    end
-  end
-end
-```
-
-#### Add new aggregated metric definition
-
-After all metrics are persisted, you can add an aggregated metric definition following [Aggregated metric instrumentation guide](metrics_instrumentation.md#aggregated-metrics).
-To declare the aggregate of metrics collected with [Estimated Batch Counters](#estimated-batch-counters),
-you must fulfill the following requirements:
-
-- Metrics names listed in the `events:` attribute, have to use the same names you passed in the `metric_name` argument while persisting metrics in previous step.
-- Every metric listed in the `events:` attribute, has to be persisted for **every** selected `time_frame:` value.
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/service_ping/index.md b/doc/development/service_ping/index.md
index 3504a730eed..d0806ed375b 100644
--- a/doc/development/service_ping/index.md
+++ b/doc/development/service_ping/index.md
@@ -1,509 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/service_ping/index.md'
+remove_date: '2023-08-20'
 ---
 
-# Service Ping development guidelines
+This document was moved to [another location](../internal_analytics/service_ping/index.md).
 
-> - Introduced in GitLab Ultimate 11.2, more statistics.
-> - In GitLab 14.1, [renamed from Usage Ping to Service Ping](https://gitlab.com/groups/gitlab-org/-/epics/5990). In 14.0 and earlier, use the Usage Ping documentation for the Rails commands appropriate to your version.
-
-Service Ping is a GitLab process that collects and sends a weekly payload to GitLab.
-The payload provides important high-level data that helps our product, support,
-and sales teams understand how GitLab is used. The data helps to:
-
-- Compare counts month over month (or week over week) to get a rough sense for how an instance uses
-  different product features.
-- Collect other facts that help us classify and understand GitLab installations.
-- Calculate our stage monthly active users (SMAU), which helps to measure the success of our stages
-  and features.
-
-Service Ping information is not anonymous. It's linked to the instance's hostname, but does
-not contain project names, usernames, or any other specific data.
-
-Service Ping is enabled by default. However, you can [disable](../../user/admin_area/settings/usage_statistics.md#enable-or-disable-usage-statistics) it on any self-managed instance. When Service Ping is enabled, GitLab gathers data from the other instances and can show your instance's usage statistics to your users.
-
-## Service Ping terminology
-
-We use the following terminology to describe the Service Ping components:
-
-- **Service Ping**: the process that collects and generates a JSON payload.
-- **Service Data**: the contents of the Service Ping JSON payload. This includes metrics.
-- **Metrics**: primarily made up of row counts for different tables in an instance's database. Each
-  metric has a corresponding [metric definition](metrics_dictionary.md#metrics-definition-and-validation)
-  in a YAML file.
-- **MAU**: monthly active users.
-- **WAU**: weekly active users.
-
-### Limitations
-
-- Service Ping does not track frontend events things like page views, link clicks, or user sessions.
-- Service Ping focuses only on aggregated backend events.
-
-Because of these limitations we recommend you:
-
-- Instrument your products with Snowplow for more detailed analytics on GitLab.com.
-- Use Service Ping to track aggregated backend events on self-managed instances.
-
-## Service Ping request flow
-
-The following example shows a basic request/response flow between a GitLab instance, the Versions Application, the License Application, Salesforce, the GitLab S3 Bucket, the GitLab Snowflake Data Warehouse, and Sisense:
-
-```mermaid
-sequenceDiagram
-    participant GitLab Instance
-    participant Versions Application
-    participant Licenses Application
-    participant Salesforce
-    participant S3 Bucket
-    participant Snowflake DW
-    participant Sisense Dashboards
-    GitLab Instance->>Versions Application: Send Service Ping
-    loop Process usage data
-        Versions Application->>Versions Application: Parse usage data
-        Versions Application->>Versions Application: Write to database
-        Versions Application->>Versions Application: Update license ping time
-    end
-    loop Process data for Salesforce
-        Versions Application-xLicenses Application: Request Zuora subscription id
-        Licenses Application-xVersions Application: Zuora subscription id
-        Versions Application-xSalesforce: Request Zuora account id  by Zuora subscription id
-        Salesforce-xVersions Application: Zuora account id
-        Versions Application-xSalesforce: Usage data for the Zuora account
-    end
-    Versions Application->>S3 Bucket: Export Versions database
-    S3 Bucket->>Snowflake DW: Import data
-    Snowflake DW->>Snowflake DW: Transform data using dbt
-    Snowflake DW->>Sisense Dashboards: Data available for querying
-    Versions Application->>GitLab Instance: DevOps Score (Conversational Development Index)
-```
-
-## How Service Ping works
-
-1. The Service Ping [cron job](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/workers/gitlab_service_ping_worker.rb#L24) is set in Sidekiq to run weekly.
-1. When the cron job runs, it calls [`Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb).
-1. `Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)` [cascades down](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb) to ~400+ other counter method calls.
-1. The response of all methods calls are [merged together](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb#L68) into a single JSON payload.
-1. The JSON payload is then [posted to the Versions application](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb#L20)
-   If a firewall exception is needed, the required URL depends on several things. If
-   the hostname is `version.gitlab.com`, the protocol is `TCP`, and the port number is `443`,
-   the required URL is <https://version.gitlab.com/>.
-1. In case of an error, it will be reported to the Version application along with following pieces of information:
-
-   - `uuid` - GitLab instance unique identifier
-   - `hostname` - GitLab instance hostname
-   - `version` - GitLab instance current versions
-   - `elapsed` - Amount of time which passed since Service Ping report process started and moment of error occurrence
-   - `message` - Error message
-
-   <pre>
-   <code>
-   {
-     "uuid"=>"02333324-1cd7-4c3b-a45b-a4993f05fb1d",
-     "hostname"=>"127.0.0.1",
-     "version"=>"14.7.0-pre",
-     "elapsed"=>0.006946,
-     "message"=>'PG::UndefinedColumn: ERROR:  column \"non_existent_attribute\" does not exist\nLINE 1: SELECT COUNT(non_existent_attribute) FROM \"issues\" /*applica...'
-   }
-   </code>
-   </pre>
-
-1. Finally, the timing metadata information that is used for diagnostic purposes is submitted to the Versions application. It consists of a list of metric identifiers and the time it took to calculate the metrics:
-
-   > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37911) in GitLab 15.0 [with a flag](../../user/feature_flags.md), enabled by default.
-   > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/295289) in GitLab 15.2. [Feature flag `measure_service_ping_metric_collection`](https://gitlab.com/gitlab-org/gitlab/-/issues/358128) removed.
-
-```ruby
-    {
-      "metadata"=>
-      {
-        "uuid"=>"0000000-0000-0000-0000-000000000000",
-        "metrics"=>
-        [{"name"=>"version", "time_elapsed"=>1.1811964213848114e-05},
-         {"name"=>"installation_type", "time_elapsed"=>0.00017242692410945892},
-         {"name"=>"license_billable_users", "time_elapsed"=>0.009520471096038818},
-         ....
-         {"name"=>"counts.clusters_platforms_eks",
-          "time_elapsed"=>0.05638605775311589},
-         {"name"=>"counts.clusters_platforms_gke",
-          "time_elapsed"=>0.40995341585949063},
-         {"name"=>"counts.clusters_platforms_user",
-          "time_elapsed"=>0.06410990096628666},
-         {"name"=>"counts.clusters_management_project",
-          "time_elapsed"=>0.24020783510059118}
-        ]
-      }
-    }
-```
-
-### On a Geo secondary site
-
-We also collect metrics specific to [Geo](../../administration/geo/index.md) secondary sites to send with Service Ping.
-
-1. The [Geo secondary service ping cron job](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/workers/geo/secondary_usage_data_cron_worker.rb) is set in Sidekiq to run weekly.
-1. When the cron job runs, it calls [`SecondaryUsageData.update_metrics!`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/geo/secondary_usage_data.rb#L33). This collects the relevant metrics from Prometheus and stores the data in the Geo secondary tracking database for transmission to the primary site during a [Geo node status update](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/geo_node_status.rb#L105).
-1. Geo node status data is sent with the JSON payload in the process described above. The following is an example of the payload where each object in the array represents a Geo node:
-
-   ```json
-   [
-     {
-       "repository_verification_enabled"=>true,
-       "repositories_replication_enabled"=>true,
-       "repositories_synced_count"=>24,
-       "repositories_failed_count"=>0,
-       "git_fetch_event_count_weekly"=>nil,
-       "git_push_event_count_weekly"=>nil,
-       ... other geo node status fields
-     }
-   ]
-   ```
-
-## Implementing Service Ping
-
-See the [implement Service Ping](implement.md) guide.
-
-## Example Service Ping payload
-
-The following is example content of the Service Ping payload.
-
-```json
-{
-  "uuid": "0000000-0000-0000-0000-000000000000",
-  "hostname": "example.com",
-  "version": "12.10.0-pre",
-  "installation_type": "omnibus-gitlab",
-  "active_user_count": 999,
-  "recorded_at": "2020-04-17T07:43:54.162+00:00",
-  "edition": "EEU",
-  "license_md5": "00000000000000000000000000000000",
-  "license_sha256": "0000000000000000000000000000000000000000000000000000000000000000",
-  "license_id": null,
-  "historical_max_users": 999,
-  "licensee": {
-    "Name": "ABC, Inc.",
-    "Email": "email@example.com",
-    "Company": "ABC, Inc."
-  },
-  "license_user_count": 999,
-  "license_starts_at": "2020-01-01",
-  "license_expires_at": "2021-01-01",
-  "license_plan": "ultimate",
-  "license_add_ons": {
-  },
-  "license_trial": false,
-  "counts": {
-    "assignee_lists": 999,
-    "boards": 999,
-    "ci_builds": 999,
-    ...
-  },
-  "container_registry_enabled": true,
-  "dependency_proxy_enabled": false,
-  "gitlab_shared_runners_enabled": true,
-  "gravatar_enabled": true,
-  "influxdb_metrics_enabled": true,
-  "ldap_enabled": false,
-  "mattermost_enabled": false,
-  "omniauth_enabled": true,
-  "prometheus_enabled": false,
-  "prometheus_metrics_enabled": false,
-  "reply_by_email_enabled": "incoming+%{key}@incoming.gitlab.com",
-  "signup_enabled": true,
-  "projects_with_expiration_policy_disabled": 999,
-  "projects_with_expiration_policy_enabled": 999,
-  ...
-  "elasticsearch_enabled": true,
-  "license_trial_ends_on": null,
-  "geo_enabled": false,
-  "git": {
-    "version": {
-      "major": 2,
-      "minor": 26,
-      "patch": 1
-    }
-  },
-  "gitaly": {
-    "version": "12.10.0-rc1-93-g40980d40",
-    "servers": 56,
-    "clusters": 14,
-    "filesystems": [
-      "EXT_2_3_4"
-    ]
-  },
-  "gitlab_pages": {
-    "enabled": true,
-    "version": "1.17.0"
-  },
-  "container_registry_server": {
-    "vendor": "gitlab",
-    "version": "2.9.1-gitlab"
-  },
-  "database": {
-    "adapter": "postgresql",
-    "version": "9.6.15",
-    "pg_system_id": 6842684531675334351,
-    "flavor": "Cloud SQL for PostgreSQL"
-  },
-  "analytics_unique_visits": {
-    "g_analytics_contribution": 999,
-    ...
-  },
-  "usage_activity_by_stage": {
-    "configure": {
-      "project_clusters_enabled": 999,
-      ...
-    },
-    "create": {
-      "merge_requests": 999,
-      ...
-    },
-    "manage": {
-      "events": 999,
-      ...
-    },
-    "monitor": {
-      "clusters": 999,
-      ...
-    },
-    "package": {
-      "projects_with_packages": 999
-    },
-    "plan": {
-      "issues": 999,
-      ...
-    },
-    "release": {
-      "deployments": 999,
-      ...
-    },
-    "secure": {
-      "user_container_scanning_jobs": 999,
-      ...
-    },
-    "verify": {
-      "ci_builds": 999,
-      ...
-    }
-  },
-  "usage_activity_by_stage_monthly": {
-    "configure": {
-      "project_clusters_enabled": 999,
-      ...
-    },
-    "create": {
-      "merge_requests": 999,
-      ...
-    },
-    "manage": {
-      "events": 999,
-      ...
-    },
-    "monitor": {
-      "clusters": 999,
-      ...
-    },
-    "package": {
-      "projects_with_packages": 999
-    },
-    "plan": {
-      "issues": 999,
-      ...
-    },
-    "release": {
-      "deployments": 999,
-      ...
-    },
-    "secure": {
-      "user_container_scanning_jobs": 999,
-      ...
-    },
-    "verify": {
-      "ci_builds": 999,
-      ...
-    }
-  },
-  "topology": {
-    "duration_s": 0.013836685999194742,
-    "application_requests_per_hour": 4224,
-    "query_apdex_weekly_average": 0.996,
-    "failures": [],
-    "nodes": [
-      {
-        "node_memory_total_bytes": 33269903360,
-        "node_memory_utilization": 0.35,
-        "node_cpus": 16,
-        "node_cpu_utilization": 0.2,
-        "node_uname_info": {
-          "machine": "x86_64",
-          "sysname": "Linux",
-          "release": "4.19.76-linuxkit"
-        },
-        "node_services": [
-          {
-            "name": "web",
-            "process_count": 16,
-            "process_memory_pss": 233349888,
-            "process_memory_rss": 788220927,
-            "process_memory_uss": 195295487,
-            "server": "puma"
-          },
-          {
-            "name": "sidekiq",
-            "process_count": 1,
-            "process_memory_pss": 734080000,
-            "process_memory_rss": 750051328,
-            "process_memory_uss": 731533312
-          },
-          ...
-        ],
-        ...
-      },
-      ...
-    ]
-  }
-}
-```
-
-## Notable changes
-
-In GitLab 14.6, [`flavor`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75587) was added to try to detect the underlying managed database variant.
-Possible values are "Amazon Aurora PostgreSQL", "PostgreSQL on Amazon RDS", "Cloud SQL for PostgreSQL",
-"Azure Database for PostgreSQL - Flexible Server", or "null".
-
-In GitLab 13.5, `pg_system_id` was added to send the [PostgreSQL system identifier](https://www.2ndquadrant.com/en/blog/support-for-postgresqls-system-identifier-in-barman/).
-
-## Export Service Ping data
-
-Rake tasks exist to export Service Ping data in different formats.
-
-- The Rake tasks export the raw SQL queries for `count`, `distinct_count`, `sum`.
-- The Rake tasks export the Redis counter class or the line of the Redis block for `redis_usage_data`.
-- The Rake tasks calculate the `alt_usage_data` metrics.
-
-In the home directory of your local GitLab installation run the following Rake tasks for the YAML and JSON versions respectively:
-
-```shell
-# for YAML export of SQL queries
-bin/rake gitlab:usage_data:dump_sql_in_yaml
-
-# for JSON export of SQL queries
-bin/rake gitlab:usage_data:dump_sql_in_json
-
-# for JSON export of Non SQL data
-bin/rake gitlab:usage_data:dump_non_sql_in_json
-
-# You may pipe the output into a file
-bin/rake gitlab:usage_data:dump_sql_in_yaml > ~/Desktop/usage-metrics-2020-09-02.yaml
-```
-
-## Generate Service Ping
-
-To generate Service Ping, use [Teleport](https://goteleport.com/docs/) or a detached screen session on a remote server.
-
-### Triggering
-
-#### Trigger Service Ping with Teleport
-
-1. Request temporary [access](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to the required environment.
-1. After your approval is issued, [access the Rails console](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/teleport/Connect_to_Rails_Console_via_Teleport.md#access-approval).
-1. Run `GitlabServicePingWorker.new.perform('triggered_from_cron' => false)`.
-
-#### Trigger Service Ping with a detached screen session
-
-1. Connect to bastion with agent forwarding:
-
-   ```shell
-   ssh -A lb-bastion.gprd.gitlab.com
-   ```
-
-1. Create named screen:
-
-   ```shell
-   screen -S <username>_usage_ping_<date>
-   ```
-
-1. Connect to console host:
-
-   ```shell
-   ssh $USER-rails@console-01-sv-gprd.c.gitlab-production.internal
-   ```
-
-1. Run:
-
-   ```shell
-   GitlabServicePingWorker.new.perform('triggered_from_cron' => false)
-   ```
-
-1. To detach from screen, press `ctrl + A`, `ctrl + D`.
-1. Exit from bastion:
-
-   ```shell
-   exit
-   ```
-
-1. Get the metrics duration from logs:
-
-Search in Google Console logs for `time_elapsed`. [Query example](https://cloudlogging.app.goo.gl/nWheZvD8D3nWazNe6).
-
-### Verification (After approx 30 hours)
-
-#### Verify with Teleport
-
-1. Follow [the steps](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to request a new access to the required environment and connect to the Rails console
-1. Check the last payload in `raw_usage_data` table: `RawUsageData.last.payload`
-1. Check the when the payload was sent: `RawUsageData.last.sent_at`
-
-#### Verify using detached screen session
-
-1. Reconnect to bastion:
-
-   ```shell
-   ssh -A lb-bastion.gprd.gitlab.com
-   ```
-
-1. Find your screen session:
-
-   ```shell
-   screen -ls
-   ```
-
-1. Attach to your screen session:
-
-   ```shell
-   screen -x 14226.mwawrzyniak_usage_ping_2021_01_22
-   ```
-
-1. Check the last payload in `raw_usage_data` table:
-
-   ```shell
-   RawUsageData.last.payload
-   ```
-
-1. Check the when the payload was sent:
-
-   ```shell
-   RawUsageData.last.sent_at
-   ```
-
-### Skip database write operations
-
-To skip database write operations, DevOps report creation, and storage of usage data payload, pass an optional argument:
-
-```shell
-skip_db_write:
-GitlabServicePingWorker.new.perform('triggered_from_cron' => false, 'skip_db_write' => true)
-```
-
-## Monitoring
-
-Service Ping reporting process state is monitored with [internal SiSense dashboard](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health).
-
-## Related topics
-
-- [Product Intelligence Guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)
-- [Snowplow Guide](../snowplow/index.md)
-- [Product Intelligence Direction](https://about.gitlab.com/direction/analytics/product-intelligence/)
-- [Data Analysis Process](https://about.gitlab.com/handbook/business-technology/data-team/#data-analysis-process/)
-- [Data for Product Managers](https://about.gitlab.com/handbook/business-technology/data-team/programs/data-for-product-managers/)
-- [Data Infrastructure](https://about.gitlab.com/handbook/business-technology/data-team/platform/infrastructure/)
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/service_ping/metrics_dictionary.md b/doc/development/service_ping/metrics_dictionary.md
index f36a97bcf6b..fecab4916f5 100644
--- a/doc/development/service_ping/metrics_dictionary.md
+++ b/doc/development/service_ping/metrics_dictionary.md
@@ -1,322 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/service_ping/metrics_dictionary.md'
+remove_date: '2023-08-20'
 ---
 
-# Metrics Dictionary Guide
+This document was moved to [another location](../internal_analytics/service_ping/metrics_dictionary.md).
 
-[Service Ping](index.md) metrics are defined in individual YAML files definitions from which the
-[Metrics Dictionary](https://metrics.gitlab.com/) is built. Currently, the metrics dictionary is built automatically once a day. When a change to a metric is made in a YAML file, you can see the change in the dictionary within 24 hours.
-This guide describes the dictionary and how it's implemented.
-
-## Metrics Definition and validation
-
-We are using [JSON Schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/schema.json) to validate the metrics definition.
-
-This process is meant to ensure consistent and valid metrics defined for Service Ping. All metrics *must*:
-
-- Comply with the defined [JSON schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/schema.json).
-- Have a unique `key_path` .
-- Have an owner.
-
-All metrics are stored in YAML files:
-
-- [`config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/metrics)
-
-WARNING:
-Only metrics with a metric definition YAML and whose status is not `removed` are added to the Service Ping JSON payload.
-
-Each metric is defined in a separate YAML file consisting of a number of fields:
-
-| Field               | Required | Additional information                                         |
-|---------------------|----------|----------------------------------------------------------------|
-| `key_path`          | yes      | JSON key path for the metric, location in Service Ping payload.  |
-| `name`              | no       | Metric name suggestion. Can replace the last part of `key_path`. |
-| `description`       | yes      |                                                                |
-| `product_section`   | yes      | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml). |
-| `product_stage`     | yes       | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the metric. |
-| `product_group`     | yes      | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the metric. |
-| `value_type`        | yes      | `string`; one of [`string`, `number`, `boolean`, `object`](https://json-schema.org/understanding-json-schema/reference/type.html).                                                     |
-| `status`            | yes      | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `removed`, `broken`. |
-| `time_frame`        | yes      | `string`; may be set to a value like `7d`, `28d`, `all`, `none`. |
-| `data_source`       | yes      | `string`; may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `system`, `license`. |
-| `data_category`     | yes      | `string`; [categories](#data-category) of the metric, may be set to `operational`, `optional`, `subscription`, `standard`. The default value is `optional`.|
-| `instrumentation_class` | yes   | `string`; [the class that implements the metric](metrics_instrumentation.md).  |
-| `distribution`      | yes      | `array`; may be set to one of `ce, ee` or `ee`. The [distribution](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available.  |
-| `performance_indicator_type`  | no      | `array`; may be set to one of [`gmau`, `smau`, `paid_gmau`, `umau` or `customer_health_score`](https://about.gitlab.com/handbook/business-technology/data-team/data-catalog/xmau-analysis/). |
-| `tier`              | yes      | `array`; may contain one or a combination of `free`, `premium` or `ultimate`. The [tier](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. This should be verbose and contain all tiers where a metric is available. |
-| `milestone`         | yes       | The milestone when the metric is introduced and when it's available to self-managed instances with the official GitLab release. |
-| `milestone_removed` | no       | The milestone when the metric is removed. |
-| `introduced_by_url` | no       | The URL to the merge request that introduced the metric to be available for self-managed instances. |
-| `removed_by_url`    | no       | The URL to the merge request that removed the metric. |
-| `repair_issue_url`  | no       | The URL of the issue that was created to repair a metric with a `broken` status. |
-| `options`           | no       | `object`: options information needed to calculate the metric value. |
-| `skip_validation`   | no       | This should **not** be set. [Used for imported metrics until we review, update and make them valid](https://gitlab.com/groups/gitlab-org/-/epics/5425). |
-
-### Metric `key_path`
-
-The `key_path` of the metric is the location in the JSON Service Ping payload.
-
-The `key_path` could be composed from multiple parts separated by `.` and it must be unique.
-
-We recommend to add the metric in one of the top-level keys:
-
-- `settings`: for settings related metrics.
-- `counts_weekly`: for counters that have data for the most recent 7 days.
-- `counts_monthly`: for counters that have data for the most recent 28 days.
-- `counts`: for counters that have data for all time.
-
-NOTE:
-We can't control what the metric's `key_path` is, because some of them are generated dynamically in `usage_data.rb`.
-For example, see [Redis HLL metrics](implement.md#redis-hll-counters).
-
-### Metric name
-
-To improve metric discoverability by a wider audience, each metric with
-instrumentation added at an appointed `key_path` receives a `name` attribute
-filled with the name suggestion, corresponding to the metric `data_source` and instrumentation.
-Metric name suggestions can contain two types of elements:
-
-1. **User input prompts**: enclosed by angle brackets (`< >`), these pieces should be replaced or
-   removed when you create a metrics YAML file.
-1. **Fixed suggestion**: plaintext parts generated according to well-defined algorithms.
-   They are based on underlying instrumentation, and must not be changed.
-
-For a metric name to be valid, it must not include any prompt, and fixed suggestions
-must not be changed.
-
-#### Generate a metric name suggestion
-
-The metric YAML generator can suggest a metric name for you.
-To generate a metric name suggestion, first instrument the metric at the provided `key_path`.
-Then, generate the metric's YAML definition and
-return to the instrumentation and update it.
-
-1. Add the metric instrumentation class to `lib/gitlab/usage/metrics/instrumentations/`.
-1. Add the metric logic in the instrumentation class.
-1. Run the [metrics YAML generator](metrics_dictionary.md#create-a-new-metric-definition).
-1. Use the metric name suggestion to select a suitable metric name.
-1. Update the metric's YAML definition with the correct `key_path`.
-
-### Metric statuses
-
-Metric definitions can have one of the following statuses:
-
-- `active`: Metric is used and reports data.
-- `broken`: Metric reports broken data (for example, -1 fallback), or does not report data at all. A metric marked as `broken` must also have the `repair_issue_url` attribute.
-- `removed`: Metric was removed, but it may appear in Service Ping payloads sent from instances running on older versions of GitLab.
-
-### Metric `value_type`
-
-Metric definitions can have one of the following values for `value_type`:
-
-- `boolean`
-- `number`
-- `string`
-- `object`: A metric with `value_type: object` must have `value_json_schema` with a link to the JSON schema for the object.
-In general, we avoid complex objects and prefer one of the `boolean`, `number`, or `string` value types.
-An example of a metric that uses `value_type: object` is `topology` (`/config/metrics/settings/20210323120839_topology.yml`),
-which has a related schema in `/config/metrics/objects_schemas/topology_schema.json`.
-
-### Metric `time_frame`
-
-A metric's time frame is calculated based on the `time_frame` field and the `data_source` of the metric.
-For `redis_hll` metrics, the type of aggregation is also taken into consideration. In this context, the term "aggregation" refers to [chosen events data storage interval](implement.md#add-new-events), and is **NOT** related to the Aggregated Metrics feature.
-For more information about the aggregation type of each feature, see the [`common.yml` file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/common.yml). Weeks run from Monday to Sunday.
-
-| data_source            | time_frame | aggregation    | Description                                     |
-|------------------------|------------|----------------|-------------------------------------------------|
-| any                    | `none`     | not applicable | A type of data that’s not tracked over time, such as settings and configuration information |
-| `database`             | `all`      | not applicable | The whole time the metric has been active (all-time interval) |
-| `database`             | `7d`       | not applicable | 9 days ago to 2 days ago |
-| `database`             | `28d`      | not applicable | 30 days ago to 2 days ago |
-| `redis`                | `all`      | not applicable | The whole time the metric has been active (all-time interval) |
-| `redis_hll`            | `7d`       | `daily`        | Most recent 7 complete days |
-| `redis_hll`            | `7d`       | `weekly`       | Most recent complete week |
-| `redis_hll`            | `28d`      | `daily`        | Most recent 28 complete days |
-| `redis_hll`            | `28d`      | `weekly`       | Most recent 4 complete weeks |
-
-### Data category
-
-We use the following categories to classify a metric:
-
-- `operational`: Required data for operational purposes.
-- `optional`: Default value for a metric. Data that is optional to collect. This can be [enabled or disabled](../../user/admin_area/settings/usage_statistics.md#enable-or-disable-usage-statistics) in the Admin Area.
-- `subscription`: Data related to licensing.
-- `standard`: Standard set of identifiers that are included when collecting data.
-
-An aggregate metric is a metric that is the sum of two or more child metrics. Service Ping uses the data category of
-the aggregate metric to determine whether or not the data is included in the reported Service Ping payload.
-
-### Metric name suggestion examples
-
-#### Metric with `data_source: database`
-
-For a metric instrumented with SQL:
-
-```sql
-SELECT COUNT(DISTINCT user_id) FROM clusters WHERE clusters.management_project_id IS NOT NULL
-```
-
-- **Suggested name**: `count_distinct_user_id_from_<adjective describing: '(clusters.management_project_id IS NOT NULL)'>_clusters`
-- **Prompt**: `<adjective describing: '(clusters.management_project_id IS NOT NULL)'>`
-  should be replaced with an adjective that best represents filter conditions, such as `project_management`
-- **Final metric name**: For example, `count_distinct_user_id_from_project_management_clusters`
-
-For metric instrumented with SQL:
-
-```sql
-SELECT COUNT(DISTINCT clusters.user_id)
-FROM clusters_applications_helm
-INNER JOIN clusters ON clusters.id = clusters_applications_helm.cluster_id
-WHERE clusters_applications_helm.status IN (3, 5)
-```
-
-- **Suggested name**: `count_distinct_user_id_from_<adjective describing: '(clusters_applications_helm.status IN (3, 5))'>_clusters_<with>_<adjective describing: '(clusters_applications_helm.status IN (3, 5))'>_clusters_applications_helm`
-- **Prompt**: `<adjective describing: '(clusters_applications_helm.status IN (3, 5))'>`
-  should be replaced with an adjective that best represents filter conditions
-- **Final metric name**: `count_distinct_user_id_from_clusters_with_available_clusters_applications_helm`
-
-In the previous example, the prompt is irrelevant, and user can remove it. The second
-occurrence corresponds with the `available` scope defined in `Clusters::Concerns::ApplicationStatus`.
-It can be used as the right adjective to replace prompt.
-
-The `<with>` represents a suggested conjunction for the suggested name of the joined relation.
-The person documenting the metric can use it by either:
-
-- Removing the surrounding `<>`.
-- Using a different conjunction, such as `having` or `including`.
-
-#### Metric with `data_source: redis` or `redis_hll`
-
-For metrics instrumented with a Redis-based counter, the suggested name includes
-only the single prompt to be replaced by the person working with metrics YAML.
-
-- **Prompt**: `<please fill metric name, suggested format is: {subject}_{verb}{ing|ed}_{object} eg: users_creating_epics or merge_requests_viewed_in_single_file_mode>`
-- **Final metric name**: We suggest the metric name should follow the format of
-  `{subject}_{verb}{ing|ed}_{object}`, such as `user_creating_epics`, `users_triggering_security_scans`,
-  or `merge_requests_viewed_in_single_file_mode`
-
-#### Metric with `data_source: prometheus` or `system`
-
-For metrics instrumented with Prometheus or coming from the operating system,
-the suggested name includes only the single prompt by person working with metrics YAML.
-
-- **Prompt**: `<please fill metric name>`
-- **Final metric name**: Due to the variety of cases that can apply to this kind of metric,
-  no naming convention exists. Each person instrumenting a metric should use their
-  best judgment to come up with a descriptive name.
-
-### Example YAML metric definition
-
-The linked [`uuid`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/license/uuid.yml)
-YAML file includes an example metric definition, where the `uuid` metric is the GitLab
-instance unique identifier.
-
-```yaml
-key_path: uuid
-description: GitLab instance unique identifier
-product_section: analytics
-product_stage: analytics
-product_group: product_intelligence
-value_type: string
-status: active
-milestone: 9.1
-instrumentation_class: UuidMetric
-introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1521
-time_frame: none
-data_source: database
-distribution:
-- ce
-- ee
-tier:
-- free
-- premium
-- ultimate
-```
-
-### Create a new metric definition
-
-The GitLab codebase provides a dedicated [generator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_definition_generator.rb) to create new metric definitions.
-
-For uniqueness, the generated files include a timestamp prefix in ISO 8601 format.
-
-The generator takes a list of key paths and 3 options as arguments. It creates metric YAML definitions in the corresponding location:
-
-- `--ee`, `--no-ee` Indicates if metric is for EE.
-- `--dir=DIR` Indicates the metric directory. It must be one of: `counts_7d`, `7d`, `counts_28d`, `28d`, `counts_all`, `all`, `settings`, `license`.
-- `--class_name=CLASS_NAME` Indicates the instrumentation class. For example `UsersCreatingIssuesMetric`, `UuidMetric`
-
-**Single metric example**
-
-```shell
-bundle exec rails generate gitlab:usage_metric_definition counts.issues --dir=7d --class_name=CountIssues
-// Creates 1 file
-// create  config/metrics/counts_7d/issues.yml
-```
-
-**Multiple metrics example**
-
-```shell
-bundle exec rails generate gitlab:usage_metric_definition counts.issues counts.users --dir=7d --class_name=CountUsersCreatingIssues
-// Creates 2 files
-// create  config/metrics/counts_7d/issues.yml
-// create  config/metrics/counts_7d/users.yml
-```
-
-NOTE:
-To create a metric definition used in EE, add the `--ee` flag.
-
-```shell
-bundle exec rails generate gitlab:usage_metric_definition counts.issues --ee --dir=7d --class_name=CountUsersCreatingIssues
-// Creates 1 file
-// create  ee/config/metrics/counts_7d/issues.yml
-```
-
-### Metrics added dynamic to Service Ping payload
-
-The [Redis HLL metrics](implement.md#known-events-are-added-automatically-in-service-data-payload) are added automatically to Service Ping payload.
-
-A YAML metric definition is required for each metric. A dedicated generator is provided to create metric definitions for Redis HLL events.
-
-The generator takes `category` and `events` arguments, as the root key is `redis_hll_counters`, and creates two metric definitions for each of the events (for weekly and monthly time frames):
-
-**Single metric example**
-
-```shell
-bundle exec rails generate gitlab:usage_metric_definition:redis_hll issues count_users_closing_issues
-// Creates 2 files
-// create  config/metrics/counts_7d/count_users_closing_issues_weekly.yml
-// create  config/metrics/counts_28d/count_users_closing_issues_monthly.yml
-```
-
-**Multiple metrics example**
-
-```shell
-bundle exec rails generate gitlab:usage_metric_definition:redis_hll issues count_users_closing_issues count_users_reopening_issues
-// Creates 4 files
-// create  config/metrics/counts_7d/count_users_closing_issues_weekly.yml
-// create  config/metrics/counts_28d/count_users_closing_issues_monthly.yml
-// create  config/metrics/counts_7d/count_users_reopening_issues_weekly.yml
-// create  config/metrics/counts_28d/count_users_reopening_issues_monthly.yml
-```
-
-To create a metric definition used in EE, add the `--ee` flag.
-
-```shell
-bundle exec rails generate gitlab:usage_metric_definition:redis_hll issues users_closing_issues --ee
-// Creates 2 files
-// create  config/metrics/counts_7d/i_closed_weekly.yml
-// create  config/metrics/counts_28d/i_closed_monthly.yml
-```
-
-## Metrics Dictionary
-
-[Metrics Dictionary is a separate application](https://gitlab.com/gitlab-org/analytics-section/product-intelligence/metric-dictionary).
-
-All metrics available in Service Ping are in the [Metrics Dictionary](https://metrics.gitlab.com/).
-
-### Copy query to clipboard
-
-To check if a metric has data in Sisense, use the copy query to clipboard feature. This copies a query that's ready to use in Sisense. The query gets the last five service ping data for GitLab.com for a given metric. For information about how to check if a Service Ping metric has data in Sisense, see this [demo](https://www.youtube.com/watch?v=n4o65ivta48).
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/service_ping/metrics_instrumentation.md b/doc/development/service_ping/metrics_instrumentation.md
index 7441a2d1bd4..5a4dfc325e2 100644
--- a/doc/development/service_ping/metrics_instrumentation.md
+++ b/doc/development/service_ping/metrics_instrumentation.md
@@ -1,478 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/service_ping/metrics_instrumentation.md'
+remove_date: '2023-08-20'
 ---
 
-# Metrics instrumentation guide
+This document was moved to [another location](../internal_analytics/service_ping/metrics_instrumentation.md).
 
-This guide describes how to develop Service Ping metrics using metrics instrumentation.
-
-<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
-For a video tutorial, see the [Adding Service Ping metric via instrumentation class](https://youtu.be/p2ivXhNxUoY).
-
-## Nomenclature
-
-- **Instrumentation class**:
-  - Inherits one of the metric classes: `DatabaseMetric`, `RedisMetric`, `RedisHLLMetric`, `NumbersMetric` or `GenericMetric`.
-  - Implements the logic that calculates the value for a Service Ping metric.
-
-- **Metric definition**
-  The Service Data metric YAML definition.
-
-- **Hardening**:
-  Hardening a method is the process that ensures the method fails safe, returning a fallback value like -1.
-
-## How it works
-
-A metric definition has the [`instrumentation_class`](metrics_dictionary.md) field, which can be set to a class.
-
-The defined instrumentation class should inherit one of the existing metric classes: `DatabaseMetric`, `RedisMetric`, `RedisHLLMetric`, `NumbersMetric` or `GenericMetric`.
-
-The current convention is that a single instrumentation class corresponds to a single metric. On rare occasions, there are exceptions to that convention like [Redis metrics](#redis-metrics). To use a single instrumentation class for more than one metric, please reach out to one of the `@gitlab-org/analytics-section/product-intelligence/engineers` members to consult about your case.
-
-Using the instrumentation classes ensures that metrics can fail safe individually, without breaking the entire
- process of Service Ping generation.
-
-We have built a domain-specific language (DSL) to define the metrics instrumentation.
-
-## Database metrics
-
-You can use database metrics to track data kept in the database, for example, a count of issues that exist on a given instance.
-
-- `operation`: Operations for the given `relation`, one of `count`, `distinct_count`, `sum`, and `average`.
-- `relation`: Assigns lambda that returns the `ActiveRecord::Relation` for the objects we want to perform the `operation`. The assigned lambda can accept up to one parameter. The parameter is hashed and stored under the `options` key in the metric definition.
-- `start`: Specifies the start value of the batch counting, by default is `relation.minimum(:id)`.
-- `finish`: Specifies the end value of the batch counting, by default is `relation.maximum(:id)`.
-- `cache_start_and_finish_as`: Specifies the cache key for `start` and `finish` values and sets up caching them. Use this call when `start` and `finish` are expensive queries that should be reused between different metric calculations.
-- `available?`: Specifies whether the metric should be reported. The default is `true`.
-- `timestamp_column`: Optionally specifies timestamp column for metric used to filter records for time constrained metrics. The default is `created_at`.
-
-[Example of a merge request that adds a database metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60022).
-
-```ruby
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-        class CountIssuesMetric < DatabaseMetric
-          operation :count
-
-          relation ->(options) { Issue.where(confidential: options[:confidential]) }
-        end
-      end
-    end
-  end
-end
-```
-
-### Ordinary batch counters Example
-
-```ruby
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-        class CountIssuesMetric < DatabaseMetric
-          operation :count
-
-          start { Issue.minimum(:id) }
-          finish { Issue.maximum(:id) }
-
-          relation { Issue }
-        end
-      end
-    end
-  end
-end
-```
-
-### Distinct batch counters Example
-
-```ruby
-# frozen_string_literal: true
-
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-        class CountUsersAssociatingMilestonesToReleasesMetric < DatabaseMetric
-          operation :distinct_count, column: :author_id
-
-          relation { Release.with_milestones }
-
-          start { Release.minimum(:author_id) }
-          finish { Release.maximum(:author_id) }
-        end
-      end
-    end
-  end
-end
-```
-
-### Sum Example
-
-```ruby
-# frozen_string_literal: true
-
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-        class JiraImportsTotalImportedIssuesCountMetric < DatabaseMetric
-          operation :sum, column: :imported_issues_count
-
-          relation { JiraImportState.finished }
-        end
-      end
-    end
-  end
-end
-```
-
-### Average Example
-
-```ruby
-# frozen_string_literal: true
-
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-        class CountIssuesWeightAverageMetric < DatabaseMetric
-          operation :average, column: :weight
-
-          relation { Issue }
-        end
-      end
-    end
-  end
-end
-```
-
-## Redis metrics
-
-You can use Redis metrics to track events not kept in the database, for example, a count of how many times the search bar has been used.
-
-[Example of a merge request that adds `Redis` metrics](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103455).
-
-The `RedisMetric` class can only be used as the `instrumentation_class` for Redis metrics with simple counters classes (classes that only inherit `BaseCounter` and set `PREFIX` and `KNOWN_EVENTS` constants). In case the counter class has additional logic included in it, a new `instrumentation_class`, inheriting from `RedisMetric`, needs to be created. This new class needs to include the additional logic from the counter class.
-
-Required options:
-
-- `event`: the event name.
-- `prefix`: the value of the `PREFIX` constant used in the counter classes from the `Gitlab::UsageDataCounters` namespace.
-
-Count unique values for `source_code_pushes` event.
-
-```yaml
-time_frame: all
-data_source: redis
-instrumentation_class: RedisMetric
-options:
-  event: pushes
-  prefix: source_code
-```
-
-### Availability-restrained Redis metrics
-
-If the Redis metric should only be available in the report under some conditions, then you must specify these conditions in a new class that is a child of the `RedisMetric` class.
-
-```ruby
-# frozen_string_literal: true
-
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-        class MergeUsageCountRedisMetric < RedisMetric
-          available? { Feature.enabled?(:merge_usage_data_missing_key_paths) }
-        end
-      end
-    end
-  end
-end
-```
-
-You must also use the class's name in the YAML setup.
-
-```yaml
-time_frame: all
-data_source: redis
-instrumentation_class: MergeUsageCountRedisMetric
-options:
-  event: pushes
-  prefix: source_code
-```
-
-## Redis HyperLogLog metrics
-
-You can use Redis HyperLogLog metrics to track events not kept in the database and incremented for unique values such as unique users,
-for example, a count of how many different users used the search bar.
-
-[Example of a merge request that adds a `RedisHLL` metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61685).
-
-Count unique values for `i_quickactions_approve` event.
-
-```yaml
-time_frame: 28d
-data_source: redis_hll
-instrumentation_class: RedisHLLMetric
-options:
-  events:
-    - i_quickactions_approve
-```
-
-### Availability-restrained Redis HyperLogLog metrics
-
-If the Redis HyperLogLog metric should only be available in the report under some conditions, then you must specify these conditions in a new class that is a child of the `RedisHLLMetric` class.
-
-```ruby
-# frozen_string_literal: true
-
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-        class MergeUsageCountRedisHLLMetric < RedisHLLMetric
-          available? { Feature.enabled?(:merge_usage_data_missing_key_paths) }
-        end
-      end
-    end
-  end
-end
-```
-
-You must also use the class's name in the YAML setup.
-
-```yaml
-time_frame: 28d
-data_source: redis_hll
-instrumentation_class: MergeUsageCountRedisHLLMetric
-options:
-  events:
-    - i_quickactions_approve
-```
-
-## Aggregated metrics
-
-<div class="video-fallback">
-  See the video from: <a href="https://www.youtube.com/watch?v=22LbYqHwtUQ">Product Intelligence Office Hours Oct 6th</a> for an aggregated metrics walk-through.
-</div>
-<figure class="video-container">
-  <iframe src="https://www.youtube-nocookie.com/embed/22LbYqHwtUQ" frameborder="0" allowfullscreen> </iframe>
-</figure>
-
-The aggregated metrics feature provides insight into the number of data attributes, for example `pseudonymized_user_ids`, that occurred in a collection of events. For example, you can aggregate the number of users who perform multiple actions such as creating a new issue and opening
-a new merge request.
-
-You can use a YAML file to define your aggregated metrics. The following arguments are required:
-
-- `options.events`: List of event names to aggregate into metric data. All events in this list must
-  use the same data source. Additional data source requirements are described in
-  [Database sourced aggregated metrics](implement.md#database-sourced-aggregated-metrics) and
-  [Redis sourced aggregated metrics](implement.md#redis-sourced-aggregated-metrics).
-- `options.aggregate.operator`: Operator that defines how the aggregated metric data is counted. Available operators are:
-  - `OR`: Removes duplicates and counts all entries that triggered any of the listed events.
-  - `AND`: Removes duplicates and counts all elements that were observed triggering all of the following events.
-- `options.aggregate.attribute`: Information pointing to the attribute that is being aggregated across events.
-- `time_frame`: One or more valid time frames. Use these to limit the data included in aggregated metrics to events within a specific date-range. Valid time frames are:
-  - `7d`: The last 7 days of data.
-  - `28d`: The last 28 days of data.
-  - `all`: All historical data, only available for `database` sourced aggregated metrics.
-- `data_source`: Data source used to collect all events data included in the aggregated metrics. Valid data sources are:
-  - [`database`](implement.md#database-sourced-aggregated-metrics)
-  - [`redis_hll`](implement.md#redis-sourced-aggregated-metrics)
-
-Refer to merge request [98206](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98206) for an example of a merge request that adds an `AggregatedMetric` metric.
-
-Count unique `user_ids` that occurred in at least one of the events: `incident_management_alert_status_changed`,
-`incident_management_alert_assigned`, `incident_management_alert_todo`, `incident_management_alert_create_incident`.
-
-```yaml
-time_frame: 28d
-instrumentation_class: AggregatedMetric
-data_source: redis_hll
-options:
-    aggregate:
-        operator: OR
-        attribute: user_id
-    events:
-        - `incident_management_alert_status_changed`
-        - `incident_management_alert_assigned`
-        - `incident_management_alert_todo`
-        - `incident_management_alert_create_incident`
-```
-
-### Availability-restrained Aggregated metrics
-
-If the Aggregated metric should only be available in the report under specific conditions, then you must specify these conditions in a new class that is a child of the `AggregatedMetric` class.
-
-```ruby
-# frozen_string_literal: true
-
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-        class MergeUsageCountAggregatedMetric < AggregatedMetric
-          available? { Feature.enabled?(:merge_usage_data_missing_key_paths) }
-        end
-      end
-    end
-  end
-end
-```
-
-You must also use the class's name in the YAML setup.
-
-```yaml
-time_frame: 28d
-instrumentation_class: MergeUsageCountAggregatedMetric
-data_source: redis_hll
-options:
-    aggregate:
-        operator: OR
-        attribute: user_id
-    events:
-        - `incident_management_alert_status_changed`
-        - `incident_management_alert_assigned`
-        - `incident_management_alert_todo`
-        - `incident_management_alert_create_incident`
-```
-
-## Numbers metrics
-
-- `operation`: Operations for the given `data` block. Currently we only support `add` operation.
-- `data`: a `block` which contains an array of numbers.
-- `available?`: Specifies whether the metric should be reported. The default is `true`.
-
-```ruby
-# frozen_string_literal: true
-
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-          class IssuesBoardsCountMetric < NumbersMetric
-            operation :add
-
-            data do |time_frame|
-              [
-                 CountIssuesMetric.new(time_frame: time_frame).value,
-                 CountBoardsMetric.new(time_frame: time_frame).value
-              ]
-            end
-          end
-        end
-      end
-    end
-  end
-end
-```
-
-You must also include the instrumentation class name in the YAML setup.
-
-```yaml
-time_frame: 28d
-instrumentation_class: IssuesBoardsCountMetric
-```
-
-## Generic metrics
-
-You can use generic metrics for other metrics, for example, an instance's database version. Observations type of data will always have a Generic metric counter type.
-
-- `value`: Specifies the value of the metric.
-- `available?`: Specifies whether the metric should be reported. The default is `true`.
-
-[Example of a merge request that adds a generic metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60256).
-
-```ruby
-module Gitlab
-  module Usage
-    module Metrics
-      module Instrumentations
-        class UuidMetric < GenericMetric
-          value do
-            Gitlab::CurrentSettings.uuid
-          end
-        end
-      end
-    end
-  end
-end
-```
-
-## Support for instrumentation classes
-
-There is support for:
-
-- `count`, `distinct_count`, `estimate_batch_distinct_count`, `sum`, and `average` for [database metrics](#database-metrics).
-- [Redis metrics](#redis-metrics).
-- [Redis HLL metrics](#redis-hyperloglog-metrics).
-- `add` for [numbers metrics](#numbers-metrics).
-- [Generic metrics](#generic-metrics), which are metrics based on settings or configurations.
-
-There is no support for:
-
-- `add`, `histogram` for database metrics.
-
-You can [track the progress to support these](https://gitlab.com/groups/gitlab-org/-/epics/6118).
-
-## Create a new metric instrumentation class
-
-To create a stub instrumentation for a Service Ping metric, you can use a dedicated [generator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_generator.rb):
-
-The generator takes the class name as an argument and the following options:
-
-- `--type=TYPE` Required. Indicates the metric type. It must be one of: `database`, `generic`, `redis`, `numbers`.
-- `--operation` Required for `database` & `numbers` type.
-  - For `database` it must be one of: `count`, `distinct_count`, `estimate_batch_distinct_count`, `sum`, `average`.
-  - For `numbers` it must be: `add`.
-- `--ee` Indicates if the metric is for EE.
-
-```shell
-rails generate gitlab:usage_metric CountIssues --type database --operation distinct_count
-        create lib/gitlab/usage/metrics/instrumentations/count_issues_metric.rb
-        create spec/lib/gitlab/usage/metrics/instrumentations/count_issues_metric_spec.rb
-```
-
-## Migrate Service Ping metrics to instrumentation classes
-
-This guide describes how to migrate a Service Ping metric from [`lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb) or [`ee/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/usage_data.rb) to instrumentation classes.
-
-1. Choose the metric type:
-
-- [Database metric](#database-metrics)
-- [Redis HyperLogLog metrics](#redis-hyperloglog-metrics)
-- [Redis metric](#redis-metrics)
-- [Numbers metric](#numbers-metrics)
-- [Generic metric](#generic-metrics)
-
-1. Determine the location of instrumentation class: either under `ee` or outside `ee`.
-
-1. [Generate the instrumentation class file](#create-a-new-metric-instrumentation-class).
-
-1. Fill the instrumentation class body:
-
-   - Add code logic for the metric. This might be similar to the metric implementation in `usage_data.rb`.
-   - Add tests for the individual metric [`spec/lib/gitlab/usage/metrics/instrumentations/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/usage/metrics/instrumentations).
-   - Add tests for Service Ping.
-
-1. [Generate the metric definition file](metrics_dictionary.md#create-a-new-metric-definition).
-
-1. Remove the code from [`lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb) or [`ee/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/usage_data.rb).
-
-1. Remove the tests from [`spec/lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/lib/gitlab/usage_data_spec.rb) or [`ee/spec/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/spec/lib/ee/gitlab/usage_data_spec.rb).
-
-## Troubleshoot metrics
-
-Sometimes metrics fail for reasons that are not immediately clear. The failures can be related to performance issues or other problems.
-The following pairing session video gives you an example of an investigation in to a real-world failing metric.
-
-<div class="video-fallback">
-  See the video from: <a href="https://www.youtube.com/watch?v=y_6m2POx2ug">Product Intelligence Office Hours Oct 27th</a> to learn more about the metrics troubleshooting process.
-</div>
-<figure class="video-container">
-  <iframe src="https://www.youtube-nocookie.com/embed/y_6m2POx2ug" frameborder="0" allowfullscreen> </iframe>
-</figure>
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/service_ping/metrics_lifecycle.md b/doc/development/service_ping/metrics_lifecycle.md
index 318db6895fb..520b18139ff 100644
--- a/doc/development/service_ping/metrics_lifecycle.md
+++ b/doc/development/service_ping/metrics_lifecycle.md
@@ -1,106 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/service_ping/metrics_lifecycle.md'
+remove_date: '2023-08-20'
 ---
 
-# Service Ping metric lifecycle
+This document was moved to [another location](../internal_analytics/service_ping/metrics_lifecycle.md).
 
-The following guidelines explain the steps to follow at each stage of a metric's lifecycle.
-
-## Add a new metric
-
-Follow the [Implement Service Ping](implement.md) guide.
-
-## Change an existing metric
-
-WARNING:
-We want to **PREVENT** changes to the calculation logic or important attributes on any metric as this invalidates comparisons of the same metric across different versions of GitLab.
-
-If you change a metric, you have to consider that not all instances of GitLab are running on the newest version. Old instances will still report the old version of the metric.
-Additionally, a metric's reported numbers are primarily interesting compared to previously reported numbers.
-As a result, if you need to change one of the following parts of a metric, you need to add a new metric instead. It's your choice whether to keep the old metric alongside the new one or [remove it](#remove-a-metric).
-
-- **calculation logic**: This means any changes that can produce a different value than the previous implementation
-- **YAML attributes**: The following attributes are directly used for analysis or calculation: `key_path`, `time_frame`, `value_type`, `data_source`.
-
-If you change the `performance_indicator_type` attribute of a metric or think your case needs an exception from the outlined rules then please notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) teams by `@` mentioning those groups in a comment on the merge request or issue.
-
-You can change any other attributes without impact to the calculation or analysis. See [this video tutorial](https://youtu.be/bYf3c01KCls) for help updating metric attributes.
-
-Currently, the [Metrics Dictionary](https://metrics.gitlab.com/) is built automatically once a day. You can see the change in the dictionary within 24 hours when you change the metric's YAML file.
-
-## Remove a metric
-
-WARNING:
-If a metric is not used in Sisense or any other system after 6 months, the
-Product Intelligence team marks it as inactive and assigns it to the group owner for review.
-
-We are working on automating this process. See [this epic](https://gitlab.com/groups/gitlab-org/-/epics/8988) for details.
-
-Product Intelligence removes metrics from Service Ping if they are not used in any Sisense dashboard.
-
-For an example of the metric removal process, see this [example issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388236).
-
-To remove a metric:
-
-1. Create an issue for removing the metric if none exists yet. The issue needs to outline why the metric should be deleted. You can use this issue to document the removal process.
-
-1. Verify the metric is not used to calculate the conversational index. The
-   conversational index is a measure that reports back to self-managed instances
-   to inform administrators of the progress of DevOps adoption for the instance.
-
-   You can check
-   [`CalculateConvIndexService`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/app/services/calculate_conv_index_service.rb)
-   to view the metrics that are used. The metrics are represented
-   as the keys that are passed as a field argument into the `get_value` method.
-
-1. Verify that removing the metric from the Service Ping payload does not cause
-   errors in [Version App](https://gitlab.com/gitlab-services/version-gitlab-com)
-   when the updated payload is collected and processed. Version App collects
-   and persists all Service Ping reports. To verify Service Ping processing in your local development environment, follow this [guide](https://www.youtube.com/watch?v=FS5emplabRU).
-   Alternatively, you can modify [fixtures](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/spec/support/usage_data_helpers.rb#L540)
-   used to test the [`UsageDataController#create`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/3760ef28/spec/controllers/usage_data_controller_spec.rb#L75)
-   endpoint, and assure that test suite does not fail when metric that you wish to remove is not included into test payload.
-
-1. Remove data from Redis
-
-   For [Ordinary Redis](implement.md#ordinary-redis-counters) counters remove data stored in Redis.
-
-   - Add a migration to remove the data from Redis for the related Redis keys. For more details, see [this MR example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82604/diffs).
-
-1. Create an issue in the
-   [GitLab Data Team project](https://gitlab.com/gitlab-data/analytics/-/issues).
-   Ask for confirmation that the metric is not referred to in any SiSense dashboards and
-   can be safely removed from Service Ping. Use this
-   [example issue](https://gitlab.com/gitlab-data/analytics/-/issues/15266) for guidance.
-
-1. Notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) by `@` mentioning those groups in a comment in the issue from step 1 regarding the deletion of the metric.
-   Many Service Ping metrics are relied upon for health score and XMAU reporting and unexpected changes to those metrics could break reporting.
-
-1. After you verify the metric can be safely removed,
-   update the attributes of the metric's YAML definition:
-
-   - Set the `status:` to `removed`.
-   - Set `removed_by_url:` to the URL of the MR removing the metric
-   - Set `milestone_removed:` to the number of the
-     milestone in which the metric was removed.
-
-   Do not remove the metric's YAML definition altogether. Some self-managed
-   instances might not immediately update to the latest version of GitLab, and
-   therefore continue to report the removed metric. The Product Intelligence team
-   requires a record of all removed metrics to identify and filter them.
-
-   For example please take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#b01f429a54843feb22265100c0e4fec1b7da1240_10_10).
-
-1. After you verify the metric can be safely removed,
-   remove the metric's instrumentation from
-   [`lib/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data.rb)
-   or
-   [`ee/lib/ee/gitlab/usage_data.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/ee/gitlab/usage_data.rb).
-
-   For example please take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#6335dc533bd21df26db9de90a02dd66278c2390d_167_167).
-
-1. Remove any other records related to the metric:
-   - The feature flag YAML file at [`config/feature_flags/*/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/feature_flags).
-   - The entry in the known events YAML file at [`lib/gitlab/usage_data_counters/known_events/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/usage_data_counters/known_events).
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/service_ping/performance_indicator_metrics.md b/doc/development/service_ping/performance_indicator_metrics.md
index 0ca663ce09a..eda7224732d 100644
--- a/doc/development/service_ping/performance_indicator_metrics.md
+++ b/doc/development/service_ping/performance_indicator_metrics.md
@@ -1,16 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/service_ping/performance_indicator_metrics.md'
+remove_date: '2023-08-20'
 ---
 
-# Performance Indicator Metrics guide
+This document was moved to [another location](../internal_analytics/service_ping/performance_indicator_metrics.md).
 
-This guide describes how to use metrics definitions to define [performance indicator](https://about.gitlab.com/handbook/product/product-intelligence-guide/#implementing-product-performance-indicators) metrics.
-
-To use a metric definition to manage a performance indicator:
-
-1. Create a merge request that includes related changes.
-1. Use labels `~"product intelligence"`, `"~Data Warehouse::Impact Check"`.
-1. Update the metric definition `performance_indicator_type` [field](metrics_dictionary.md#metrics-definition-and-validation).
-1. Create an issue in GitLab Product Data Insights project with the [PI Chart Help template](https://gitlab.com/gitlab-data/product-analytics/-/issues/new?issuable_template=PI%20Chart%20Help) to have the new metric visualized.
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/service_ping/review_guidelines.md b/doc/development/service_ping/review_guidelines.md
index 8128f1e371b..d5805f615e2 100644
--- a/doc/development/service_ping/review_guidelines.md
+++ b/doc/development/service_ping/review_guidelines.md
@@ -1,82 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/service_ping/review_guidelines.md'
+remove_date: '2023-08-20'
 ---
 
-# Service Ping review guidelines
+This document was moved to [another location](../internal_analytics/service_ping/review_guidelines.md).
 
-This page includes introductory material for a
-[Analytics Instrumentation](https://about.gitlab.com/handbook/engineering/development/analytics/analytics-instrumentation/)
-review, and is specific to Service Ping related reviews. For broader advice and
-general best practices for code reviews, refer to our [code review guide](../code_review.md).
-
-## Resources for reviewers
-
-- [Service Ping Guide](index.md)
-- [Metrics Dictionary](https://metrics.gitlab.com/)
-
-## Review process
-
-We recommend a Analytics Instrumentation review when a merge request (MR) touches
-any of the following Service Ping files:
-
-- `usage_data*` files.
-- The Metrics Dictionary, including files in:
-  - [`config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/metrics).
-  - [`ee/config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/config/metrics).
-  - [`schema.json`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/schema.json).
-- Analytics Instrumentation tooling. For example,
-  [`Gitlab::UsageMetricDefinitionGenerator`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_definition_generator.rb)
-
-### Roles and process
-
-#### The merge request **author** should
-
-- Decide whether a Analytics Instrumentation review is needed. You can skip the Analytics Instrumentation
-review and remove the labels if the changes are not related to the Analytics Instrumentation domain and
-are regular backend changes.
-- If a Analytics Instrumentation review is needed, add the labels
-  `~analytics instrumentation` and `~analytics instrumentation::review pending`.
-- For merge requests authored by Analytics Instrumentation team members:
-  - Assign both the `~backend` and `~analytics instrumentation` reviews to another Analytics Instrumentation team member.
-  - Assign the maintainer review to someone outside of the Analytics Instrumentation group.
-- Assign an
-  [engineer](https://gitlab.com/groups/gitlab-org/analytics-section/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) from the Analytics Instrumentation team for a review.
-- Set the correct attributes in the metric's YAML definition:
-  - `product_section`, `product_stage`, `product_group`
-  - Provide a clear description of the metric.
-- Add a changelog [according to guidelines](../changelog.md).
-
-#### The Analytics Instrumentation **reviewer** should
-
-- Perform a first-pass review on the merge request and suggest improvements to the author.
-- Check the [metrics location](metrics_dictionary.md#metric-key_path) in
-  the Service Ping JSON payload.
-- Suggest that the author checks the [naming suggestion](metrics_dictionary.md#generate-a-metric-name-suggestion) while
-  generating the metric's YAML definition.
-- Add the `~database` label and ask for a [database review](../database_review.md) for
-  metrics that are based on Database.
-- Add `~Data Warehouse::Impact Check` for any database metric that has a query change. Changes in queries can affect [data operations](https://about.gitlab.com/handbook/business-technology/data-team/how-we-work/triage/#gitlabcom-db-structure-changes).
-- For tracking using Redis HLL (HyperLogLog):
-  - Check if a [feature flag is needed](implement.md#recommendations).
-- For a metric's YAML definition:
-  - Check the metric's `description`.
-  - Check the metric's `key_path`.
-  - Check the `product_section`, `product_stage`, and `product_group` fields.
-    Read the [stages file](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml).
-  - Check the file location. Consider the time frame, and if the file should be under `ee`.
-  - Check the tiers.
-- If a metric was changed or removed: Make sure the MR author notified the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) by `@` mentioning those groups in a comment on the issue for the MR and all of these groups have acknowledged the removal.
-- Metrics instrumentations
-  - Recommend using metrics instrumentation for new metrics, [if possible](metrics_instrumentation.md#support-for-instrumentation-classes).
-- Approve the MR, and relabel the MR with `~"analytics instrumentation::approved"`.
-
-## Review workload distribution
-
-[Danger bot](../dangerbot.md) adds the list of changed Analytics Instrumentation files
-and pings the
-[`@gitlab-org/analytics-section/product-intelligence/engineers`](https://gitlab.com/groups/gitlab-org/analytics-section/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) group for merge requests
-that are not drafts.
-
-Any of the Analytics Instrumentation engineers can be assigned for the Analytics Instrumentation review.
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/service_ping/troubleshooting.md b/doc/development/service_ping/troubleshooting.md
index 1b896efb726..31b04c1a6bc 100644
--- a/doc/development/service_ping/troubleshooting.md
+++ b/doc/development/service_ping/troubleshooting.md
@@ -1,164 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/service_ping/troubleshooting.md'
+remove_date: '2023-08-20'
 ---
 
-# Troubleshooting Service Ping
+This document was moved to [another location](../internal_analytics/service_ping/troubleshooting.md).
 
-## Service Ping Payload drop
-
-### Symptoms
-
-You will be alerted by a [Sisense alert](https://app.periscopedata.com/app/gitlab/alert/Service-ping-payload-drop-Alert/5a5b8766b8af43b4a75d6e9c684a365f/edit) that is sent to `#g_product_intelligence` Slack channel
-
-### Locating the problem
-
-First you need to identify at which stage in Service Ping data pipeline the drop is occurring.
-
-Start at [Service Ping Health Dashboard](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health) on Sisense.
-
-The alert compares the current daily value with the daily value from previous week, excluding the last 48 hours as this is the interval where we export the data in GCP and get it into DWH.
-
-You can use [this query](https://gitlab.com/gitlab-org/gitlab/-/issues/347298#note_836685350) as an example, to start detecting when the drop started.
-
-### Troubleshoot the GitLab application layer
-
-For results about an investigation conducted into an unexpected drop in Service ping Payload events volume, see [this issue](https://gitlab.com/gitlab-data/analytics/-/issues/11071).
-
-### Troubleshoot VersionApp layer
-
-Check if the [export jobs](https://gitlab.com/gitlab-services/version-gitlab-com#data-export-using-pipeline-schedules) are successful.
-
-Check [Service Ping errors](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health?widget=14609989&udv=0) in the [Service Ping Health Dashboard](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health).
-
-### Troubleshoot Google Storage layer
-
-Check if the files are present in [Google Storage](https://console.cloud.google.com/storage/browser/cloudsql-gs-production-efd5e8-cloudsql-exports;tab=objects?project=gs-production-efd5e8&prefix=&forceOnObjectsSortingFiltering=false).
-
-### Troubleshoot the data warehouse layer
-
-Reach out to the [Data team](https://about.gitlab.com/handbook/business-technology/data-team/) to ask about current state of data warehouse. On their handbook page there is a [section with contact details](https://about.gitlab.com/handbook/business-technology/data-team/#how-to-connect-with-us).
-
-### Cannot disable Service Ping with the configuration file
-
-The method to disable Service Ping with the GitLab configuration file does not work in
-GitLab versions 9.3.0 to 13.12.3. To disable it, you must use the Admin Area in
-the GitLab UI instead. For more information, see
-[this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/333269).
-
-GitLab functionality and application settings cannot override or circumvent
-restrictions at the network layer. If Service Ping is blocked by your firewall,
-you are not impacted by this bug.
-
-#### Check if you are affected
-
-You can check if you were affected by this bug by using the Admin Area or by
-checking the configuration file of your GitLab instance:
-
-- Using the Admin Area:
-
-  1. On the top bar, select **Main menu > Admin**.
-  1. On the left sidebar, select **Settings > Metrics and profiling**.
-  1. Expand **Usage Statistics**.
-  1. Are you able to check or uncheck the checkbox to disable Service Ping?
-
-     - If _yes_, your GitLab instance is not affected by this bug.
-     - If you can't check or uncheck the checkbox, you are affected by this bug.
-       See the steps on [how to fix this](#how-to-fix-the-cannot-disable-service-ping-bug).
-
-- Checking your GitLab instance configuration file:
-
-  To check whether you're impacted by this bug, check your instance configuration
-  settings. The configuration file in which Service Ping can be disabled depends
-  on your installation and deployment method, but is typically one of the following:
-
-  - `/etc/gitlab/gitlab.rb` for Omnibus GitLab Linux Package and Docker.
-  - `charts.yaml` for GitLab Helm and cloud-native Kubernetes deployments.
-  - `gitlab.yml` for GitLab installations from source.
-
-  To check the relevant configuration file for strings that indicate whether
-  Service Ping is disabled, you can use `grep`:
-
-  ```shell
-  # Linux package
-  grep "usage_ping_enabled'\] = false" /etc/gitlab/gitlab.rb
-
-  # Kubernetes charts
-  grep "enableUsagePing: false" values.yaml
-
-  # From source
-  grep "usage_ping_enabled'\] = false" gitlab/config.yml
-  ```
-
-  If you see any output after running the relevant command, your GitLab instance
-  may be affected by the bug. Otherwise, your instance is not affected.
-
-#### How to fix the "Cannot disable Service Ping" bug
-
-To work around this bug, you have two options:
-
-- [Update](../../update/index.md) to GitLab 13.12.4 or newer to fix this bug.
-- If you can't update to GitLab 13.12.4 or newer, enable Service Ping in the
-  configuration file, then disable Service Ping in the UI. For example, if you're
-  using the Linux package:
-
-  1. Edit `/etc/gitlab/gitlab.rb`:
-
-     ```ruby
-     gitlab_rails['usage_ping_enabled'] = true
-     ```
-
-  1. Reconfigure GitLab:
-
-     ```shell
-     sudo gitlab-ctl reconfigure
-     ```
-
-  1. In GitLab, on the top bar, select **Main menu > Admin**.
-  1. On the left sidebar, select **Settings > Metrics and profiling**.
-  1. Expand **Usage Statistics**.
-  1. Clear the **Enable Service Ping** checkbox.
-  1. Select **Save Changes**.
-
-## Generate Service Ping
-
-### Generate or get the cached Service Ping in rails console
-
-Use the following method in the [rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session).
-
-```ruby
-Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values, cached: true)
-```
-
-### Generate a fresh new Service Ping
-
-Use the following method in the [rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session).
-
-This also refreshes the cached Service Ping displayed in the Admin Area.
-
-```ruby
-Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)
-```
-
-### Generate and print
-
-Generates Service Ping data in JSON format.
-
-```shell
-gitlab-rake gitlab:usage_data:generate
-```
-
-Generates Service Ping data in YAML format:
-
-```shell
-gitlab-rake gitlab:usage_data:dump_sql_in_yaml
-```
-
-### Generate and send Service Ping
-
-Prints the metrics saved in `conversational_development_index_metrics`.
-
-```shell
-gitlab-rake gitlab:usage_data:generate_and_send
-```
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/service_ping/usage_data.md b/doc/development/service_ping/usage_data.md
index b3bdaedd60a..94ae90273d0 100644
--- a/doc/development/service_ping/usage_data.md
+++ b/doc/development/service_ping/usage_data.md
@@ -1,69 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/service_ping/usage_data.md'
+remove_date: '2023-08-20'
 ---
 
-# Usage Data Metrics guide
+This document was moved to [another location](../internal_analytics/service_ping/usage_data.md).
 
-This guide describes deprecated usage for metrics in `usage_data.rb`.
-
-NOTE:
-Implementing metrics direct in `usage_data.rb` is deprecated, We recommend you use [instrumentation classes](metrics_instrumentation.md).
-
-## Ordinary batch counters
-
-Simple count of a given `ActiveRecord_Relation`, does a non-distinct batch count, smartly reduces `batch_size`, and handles errors.
-Handles the `ActiveRecord::StatementInvalid` error.
-
-Method:
-
-```ruby
-count(relation, column = nil, batch: true, start: nil, finish: nil)
-```
-
-Arguments:
-
-- `relation` the ActiveRecord_Relation to perform the count
-- `column` the column to perform the count on, by default is the primary key
-- `batch`: default `true` to use batch counting
-- `start`: custom start of the batch counting to avoid complex min calculations
-- `end`: custom end of the batch counting to avoid complex min calculations
-
-Examples:
-
-```ruby
-count(User.active)
-count(::Clusters::Cluster.aws_installed.enabled, :cluster_id)
-count(::Clusters::Cluster.aws_installed.enabled, :cluster_id, start: ::Clusters::Cluster.minimum(:id), finish: ::Clusters::Cluster.maximum(:id))
-```
-
-## Distinct batch counters
-
-Distinct count of a given `ActiveRecord_Relation` on given column, a distinct batch count, smartly reduces `batch_size`, and handles errors.
-Handles the `ActiveRecord::StatementInvalid` error.
-
-Method:
-
-```ruby
-distinct_count(relation, column = nil, batch: true, batch_size: nil, start: nil, finish: nil)
-```
-
-Arguments:
-
-- `relation`: the ActiveRecord_Relation to perform the count
-- `column`: the column to perform the distinct count, by default is the primary key
-- `batch`: default `true` to use batch counting
-- `batch_size`: if none set it uses default value 10000 from `Gitlab::Database::BatchCounter`
-- `start`: custom start of the batch counting to avoid complex min calculations
-- `end`: custom end of the batch counting to avoid complex min calculations
-
-WARNING:
-Counting over non-unique columns can lead to performance issues. For more information, see the [iterating tables in batches](../database/iterating_tables_in_batches.md) guide.
-
-Examples:
-
-```ruby
-distinct_count(::Project, :creator_id)
-distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id))
-```
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/sidekiq/index.md b/doc/development/sidekiq/index.md
index 2010a21130d..d510414baa3 100644
--- a/doc/development/sidekiq/index.md
+++ b/doc/development/sidekiq/index.md
@@ -114,6 +114,41 @@ def perform(project_id)
 end
 ```
 
+## Deferring Sidekiq workers
+
+Sidekiq workers are deferred by two ways,
+
+1. Manual: Feature flags can be used to explicitly defer a particular worker, more details can be found [here](../feature_flags/index.md#deferring-sidekiq-jobs).
+1. Automatic: Similar to the [throttling mechanism](../database/batched_background_migrations.md#throttling-batched-migrations) in batched migrations, database health indicators are used to defer a Sidekiq worker.
+
+   To use the automatic deferring mechanism, worker has to opt-in by calling `defer_on_database_health_signal` with gitlab_schema, delay_by (time to delay) and tables (which is used by autovacuum db indicator) as it's parameters.
+
+   **Example:**
+
+   ```ruby
+    module Chaos
+      class SleepWorker # rubocop:disable Scalability/IdempotentWorker
+        include ApplicationWorker
+
+        data_consistency :always
+
+        sidekiq_options retry: 3
+        include ChaosQueue
+
+        defer_on_database_health_signal :gitlab_main, 1.minute, [:users]
+
+        def perform(duration_s)
+          Gitlab::Chaos.sleep(duration_s)
+        end
+      end
+    end
+   ```
+
+For deferred jobs, logs contain the following to indicate the source:
+
+- `job_status`: `deferred`
+- `job_deferred_by`: 'feature_flag' or 'database_health_check'
+
 ## Sidekiq Queues
 
 Previously, each worker had its own queue, which was automatically set based on the
diff --git a/doc/development/snowplow/event_dictionary_guide.md b/doc/development/snowplow/event_dictionary_guide.md
index 6e8947e0210..2bea681bf59 100644
--- a/doc/development/snowplow/event_dictionary_guide.md
+++ b/doc/development/snowplow/event_dictionary_guide.md
@@ -1,91 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/snowplow/event_dictionary_guide.md'
+remove_date: '2023-08-20'
 ---
 
-# Event dictionary guide
+This document was moved to [another location](../internal_analytics/snowplow/event_dictionary_guide.md).
 
-NOTE:
-The event dictionary is a work in progress, and this process is subject to change.
-
-This guide describes the event dictionary and how it's implemented.
-
-## Event definition and validation
-
-This process is meant to document all Snowplow events and ensure consistency. Every Snowplow event needs to have such a definition. Event definitions must comply with the [JSON Schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/events/schema.json).
-
-All event definitions are stored in the following directories:
-
-- [`config/events`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/events)
-- [`ee/config/events`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/config/events)
-
-Each event is defined in a separate YAML file consisting of the following fields:
-
-| Field                  | Required | Additional information                                                                                                                                                                      |
-|------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `description`          | yes      | A description of the event.                                                                                                                                                                 |
-| `category`             | yes      | The event category (see [Event schema](index.md#event-schema)).                                                                                                                             |
-| `action`               | yes      | The event action (see [Event schema](index.md#event-schema)).                                                                                                                               |
-| `label_description`    | no       | A description of the event label (see [Event schema](index.md#event-schema)).                                                                                                               |
-| `property_description` | no       | A description of the event property (see [Event schema](index.md#event-schema)).                                                                                                            |
-| `value_description`    | no       | A description of the event value (see [Event schema](index.md#event-schema)).                                                                                                               |
-| `extra_properties`     | no       | The type and description of each extra property sent with the event.                                                                                                                        |
-| `identifiers`          | no       | A list of identifiers sent with the event. Can be set to one or more of `project`, `user`, or `namespace`.                                                                                  |
-| `iglu_schema_url`      | no       | The URL to the custom schema sent with the event, for example, `iglu:com.gitlab/gitlab_experiment/jsonschema/1-0-0`.                                                                        |
-| `product_section`      | yes      | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml).                                                                                                |
-| `product_stage`        | no       | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the event.                                                                                        |
-| `product_group`        | yes      | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the event.                                                                                  |
-| `milestone`            | no       | The milestone when the event is introduced.                                                                                                                                                 |
-| `introduced_by_url`    | no       | The URL to the merge request that introduced the event.                                                                                                                                     |
-| `distributions`        | yes      | The [distributions](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. Can be set to one or more of `ce` or `ee`.  |
-| `tiers`                | yes      | The [tiers](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/) where the tracked feature is available. Can be set to one or more of `free`, `premium`, or `ultimate`. |
-
-### Example event definition
-
-The linked [`uuid`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/events/epics_promote.yml)
-YAML file includes an example event definition.
-
-```yaml
-description: Issue promoted to epic
-category: epics
-action: promote
-property_description: The string "issue_id"
-value_description: ID of the issue
-extra_properties:
-  weight:
-    type: integer
-    description: Weight of the issue
-identifiers:
-- project
-- user
-- namespace
-product_section: dev
-product_stage: plan
-product_group: group::product planning
-milestone: "11.10"
-introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/10537
-distributions:
-- ee
-tiers:
-- premium
-- ultimate
-```
-
-## Create a new event definition
-
-Use the dedicated [event definition generator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/snowplow_event_definition_generator.rb)
-to create new event definitions.
-
-The `category` and `action` of each event are included in the filename to standardize file naming.
-
-The generator takes three options:
-
-- `--ee`: Indicates if the event is for EE.
-- `--category=CATEGORY`: Indicates the `category` of the event.
-- `--action=ACTION`: Indicates the `action` of the event.
-
-```shell
-bundle exec rails generate gitlab:snowplow_event_definition --category Groups::EmailCampaignsController --action click
-create  create  config/events/groups__email_campaigns_controller_click.yml
-```
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/snowplow/implementation.md b/doc/development/snowplow/implementation.md
index 37948f2a3e0..a9e4e252a53 100644
--- a/doc/development/snowplow/implementation.md
+++ b/doc/development/snowplow/implementation.md
@@ -1,522 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/snowplow/implementation.md'
+remove_date: '2023-08-20'
 ---
 
-# Implement Snowplow tracking
+This document was moved to [another location](../internal_analytics/snowplow/implementation.md).
 
-This page describes how to:
-
-- Implement Snowplow frontend and backend tracking
-- Test Snowplow events
-
-## Event definitions
-
-Every Snowplow event, regardless of frontend or backend, requires a corresponding event definition. These definitions document the event and its properties to make it easier to maintain and analyze.
-These defintions can be browsed in the [event dictionary](https://metrics.gitlab.com/snowplow). The [event dictionary guide](event_dictionary_guide.md) provides instructions for setting up an event definition.
-
-## Snowplow JavaScript frontend tracking
-
-GitLab provides a `Tracking` interface that wraps the [Snowplow JavaScript tracker](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/)
-to track custom events.
-
-For the recommended frontend tracking implementation, see [Usage recommendations](#usage-recommendations).
-
-Structured events and page views include the [`gitlab_standard`](schemas.md#gitlab_standard)
-context, using the `window.gl.snowplowStandardContext` object which includes
-[default data](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/views/layouts/_snowplow.html.haml)
-as base:
-
-| Property | Example |
-| -------- | ------- |
-| `context_generated_at` | `"2022-01-01T01:00:00.000Z"` |
-| `environment` | `"production"` |
-| `extra` | `{}` |
-| `namespace_id` | `123` |
-| `plan` | `"gold"` |
-| `project_id` | `456` |
-| `source` | `"gitlab-rails"` |
-| `user_id` | `789`* |
-
-_\* Undergoes a pseudonymization process at the collector level._
-
-These properties [are overridden](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/get_standard_context.js)
-with frontend-specific values, like `source` (`gitlab-javascript`), `google_analytics_id`
-and the custom `extra` object. You can modify this object for any subsequent
-structured event that fires, although this is not recommended.
-
-Tracking implementations must have an `action` and a `category`. You can provide additional
-properties from the [event schema](index.md#event-schema), in
-addition to an `extra` object that accepts key-value pairs.
-
-| Property      | Type   | Default value              | Description                                                                                                                                                                                 |
-|:-----------|:-------|:---------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `category` | string | `document.body.dataset.page` | Page or subsection of a page in which events are captured.                                                                                                                                  |
-| `action`   | string | `'generic'`                  | Action the user is taking. Clicks must be `click` and activations must be `activate`. For example, focusing a form field is `activate_form_input`, and clicking a button is `click_button`. |
-| `data`     | object | `{}`                         | Additional data such as `label`, `property`, `value` as described in [Event schema](index.md#event-schema), `context` for custom contexts, and `extra` (key-value pairs object).            |
-
-### Usage recommendations
-
-- Use [data attributes](#implement-data-attribute-tracking) on HTML elements that emit `click`, `show.bs.dropdown`, or `hide.bs.dropdown` events.
-- Use the [Vue mixin](#implement-vue-component-tracking) for tracking custom events, or if the supported events for data attributes are not propagating. For example, clickable components that don't emit `click`.
-- Use the [tracking class](#implement-raw-javascript-tracking) when tracking in vanilla JavaScript files.
-
-### Implement data attribute tracking
-
-To implement tracking for HAML or Vue templates, add a [`data-track` attribute](#data-track-attributes) to the element.
-
-The following example shows `data-track-*` attributes assigned to a button:
-
-```haml
-%button.btn{ data: { track_action: "click_button", track_label: "template_preview", track_property: "my-template" } }
-```
-
-```html
-<button class="btn"
-  data-track-action="click_button"
-  data-track-label="template_preview"
-  data-track-property="my-template"
-  data-track-extra='{ "template_variant": "primary" }'
-/>
-```
-
-#### `data-track` attributes
-
-| Attribute             | Required | Description |
-|:----------------------|:---------|:------------|
-| `data-track-action`    | true     | Action the user is taking. Clicks must be prepended with `click` and activations must be prepended with `activate`. For example, focusing a form field is `activate_form_input` and clicking a button is `click_button`. Replaces `data-track-event`, which was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/290962) in GitLab 13.11. |
-| `data-track-label`    | false    | The specific element or object to act on. This can be: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown list in the top bar; or the name or title attribute of a record being created. |
-| `data-track-property` | false    | Any additional property of the element, or object being acted on. |
-| `data-track-value`    | false    | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. If omitted, this is the element's `value` property or `undefined`. For checkboxes, the default value is the element's checked attribute or `0` when unchecked. The value is parsed as numeric before sending the event. |
-| `data-track-extra` | false    | A key-value pair object passed as a valid JSON string. This attribute is added to the `extra` property in our [`gitlab_standard`](schemas.md#gitlab_standard) schema. |
-| `data-track-context`  | false    | To append a custom context object, passed as a valid JSON string. |
-
-#### Event listeners
-
-Event listeners bind at the document level to handle click events in elements with data attributes.
-This allows them to be handled when the DOM re-renders or changes. Document-level binding reduces
-the likelihood that click events stop propagating up the DOM tree.
-
-If click events stop propagating, you must implement listeners and [Vue component tracking](#implement-vue-component-tracking) or [raw JavaScript tracking](#implement-raw-javascript-tracking).
-
-#### Helper methods
-
-You can use the following Ruby helpers:
-
-```ruby
-tracking_attrs(label, action, property) # { data: { track_label... } }
-
-tracking_attrs_data(label, action, property) # { track_label... }
-```
-
-You can also use it on HAML templates:
-
-```haml
-%button{ **tracking_attrs('main_navigation', 'click_button', 'navigation') }
-
-// When merging with additional data
-// %button{ data: { platform: "...", **tracking_attrs_data('main_navigation', 'click_button', 'navigation') } }
-```
-
-If you use the GitLab helper method [`nav_link`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/helpers/tab_helper.rb#L76), you must wrap `html_options` under the `html_options` keyword argument. If you
-use the `ActionView` helper method [`link_to`](https://api.rubyonrails.org/classes/ActionView/Helpers/UrlHelper.html#method-i-link_to), you don't need to wrap `html_options`.
-
-```ruby
-# Bad
-= nav_link(controller: ['dashboard/groups', 'explore/groups'], data: { track_label: "explore_groups",
-track_action: "click_button" })
-
-# Good
-= nav_link(controller: ['dashboard/groups', 'explore/groups'], html_options: { data: { track_label:
-"explore_groups", track_action: "click_button" } })
-
-# Good (other helpers)
-= link_to explore_groups_path, title: _("Explore"), data: { track_label: "explore_groups", track_action:
-"click_button" }
-```
-
-### Implement Vue component tracking
-
-For custom event tracking, use the [Vue mixin](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/tracking.js#L207). It exposes `Tracking.event` as the `track` method.
-You can specify tracking options by creating a `tracking` data object or
-computed property, and as a second parameter: `this.track('click_button', opts)`.
-These options override any defaults and allow the values to be dynamic from props or based on state:
-
-| Property | Type | Default | Example |
-| -- | -- | -- | -- |
-| `category` | string | `document.body.dataset.page` | `'code_quality_walkthrough'` |
-| `label` | string | `''` | `'process_start_button'` |
-| `property` | string | `''` | `'asc'` or `'desc'` |
-| `value` | integer | `undefined` | `0`, `1`, `500` |
-| `extra` | object | `{}` | `{ selectedVariant: this.variant }` |
-
-To implement Vue component tracking:
-
-1. Import the `Tracking` library and call the `mixin` method:
-
-   ```javascript
-   import Tracking from '~/tracking';
-
-   const trackingMixin = Tracking.mixin();
-
-   // Optionally provide default properties
-   // const trackingMixin = Tracking.mixin({ label: 'right_sidebar' });
-   ```
-
-1. Use the mixin in the component:
-
-   ```javascript
-   export default {
-     mixins: [trackingMixin],
-     // Or
-     // mixins: [Tracking.mixin()],
-     // mixins: [Tracking.mixin({ label: 'right_sidebar' })],
-
-     data() {
-       return {
-         expanded: false,
-       };
-     },
-   };
-   ```
-
-1. You can specify tracking options in by creating a `tracking` data object
-or computed property:
-
-   ```javascript
-   export default {
-     name: 'RightSidebar',
-
-     mixins: [Tracking.mixin()],
-
-     data() {
-       return {
-         expanded: false,
-         variant: '',
-         tracking: {
-           label: 'right_sidebar',
-           // property: '',
-           // value: '',
-           // experiment: '',
-           // extra: {},
-         },
-       };
-     },
-
-     // Or
-     // computed: {
-     //   tracking() {
-     //     return {
-     //       property: this.variant,
-     //       extra: { expanded: this.expanded },
-     //     };
-     //   },
-     // },
-   };
-   ```
-
-1. Call the `track` method. Tracking options can be passed as the second parameter:
-
-   ```javascript
-   this.track('click_button', {
-     label: 'right_sidebar',
-   });
-   ```
-
-   Or use the `track` method in the template:
-
-   ```html
-   <template>
-     <div>
-       <button data-testid="toggle" @click="toggle">Toggle</button>
-
-       <div v-if="expanded">
-         <p>Hello world!</p>
-         <button @click="track('click_button')">Track another event</button>
-       </div>
-     </div>
-   </template>
-   ```
-
-#### Testing example
-
-```javascript
-export default {
-  name: 'CountDropdown',
-
-  mixins: [Tracking.mixin({ label: 'count_dropdown' })],
-
-  data() {
-    return {
-      variant: 'counter',
-      count: 0,
-    };
-  },
-
-  methods: {
-    handleChange({ target }) {
-      const { variant } = this;
-
-      this.count = Number(target.value);
-
-      this.track('change_value', {
-        value: this.count,
-        extra: { variant }
-      });
-    },
-  },
-};
-```
-
-```javascript
-import { mockTracking } from 'helpers/tracking_helper';
-// mockTracking(category, documentOverride, spyMethod)
-
-describe('CountDropdown.vue', () => {
-  let trackingSpy;
-  let wrapper;
-
-  ...
-
-  beforeEach(() => {
-    trackingSpy = mockTracking(undefined, wrapper.element, jest.spyOn);
-  });
-
-  const findDropdown = () => wrapper.find('[data-testid="dropdown"]');
-
-  it('tracks change event', () => {
-    const dropdown = findDropdown();
-    dropdown.element.value = 30;
-    dropdown.trigger('change');
-
-    expect(trackingSpy).toHaveBeenCalledWith(undefined, 'change_value', {
-      value: 30,
-      label: 'count_dropdown',
-      extra: { variant: 'counter' },
-    });
-  });
-});
-```
-
-### Implement raw JavaScript tracking
-
-To track from a vanilla JavaScript file, use the `Tracking.event` static function
-(calls [`dispatchSnowplowEvent`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/dispatch_snowplow_event.js)).
-
-The following example demonstrates tracking a click on a button by manually calling `Tracking.event`.
-
-```javascript
-import Tracking from '~/tracking';
-
-const button = document.getElementById('create_from_template_button');
-
-button.addEventListener('click', () => {
-  Tracking.event(undefined, 'click_button', {
-    label: 'create_from_template',
-    property: 'template_preview',
-    extra: {
-      templateVariant: 'primary',
-      valid: 1,
-    },
-  });
-});
-```
-
-#### Testing example
-
-```javascript
-import Tracking from '~/tracking';
-
-describe('MyTracking', () => {
-  let wrapper;
-
-  beforeEach(() => {
-    jest.spyOn(Tracking, 'event');
-  });
-
-  const findButton = () => wrapper.find('[data-testid="create_from_template"]');
-
-  it('tracks event', () => {
-    findButton().trigger('click');
-
-    expect(Tracking.event).toHaveBeenCalledWith(undefined, 'click_button', {
-      label: 'create_from_template',
-      property: 'template_preview',
-      extra: {
-        templateVariant: 'primary',
-        valid: true,
-      },
-    });
-  });
-});
-```
-
-### Form tracking
-
-To enable Snowplow automatic [form tracking](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracking-specific-events/#form-tracking):
-
-1. Call `Tracking.enableFormTracking` when the DOM is ready.
-1. Provide a `config` object that includes at least one of the following elements:
-    - `forms` determines the forms to track. Identified by the CSS class name.
-    - `fields` determines the fields inside the tracked forms to track. Identified by the field `name`.
-1. Optional. Provide a list of contexts as the second argument. The [`gitlab_standard`](schemas.md#gitlab_standard) schema is excluded from these events.
-
-```javascript
-Tracking.enableFormTracking({
-  forms: { allow: ['sign-in-form', 'password-recovery-form'] },
-  fields: { allow: ['terms_and_conditions', 'newsletter_agreement'] },
-});
-```
-
-#### Testing example
-
-```javascript
-import Tracking from '~/tracking';
-
-describe('MyFormTracking', () => {
-  let formTrackingSpy;
-
-  beforeEach(() => {
-    formTrackingSpy = jest
-      .spyOn(Tracking, 'enableFormTracking')
-      .mockImplementation(() => null);
-  });
-
-  it('initialized with the correct configuration', () => {
-    expect(formTrackingSpy).toHaveBeenCalledWith({
-      forms: { allow: ['sign-in-form', 'password-recovery-form'] },
-      fields: { allow: ['terms_and_conditions', 'newsletter_agreement'] },
-    });
-  });
-});
-```
-
-## Implement Ruby backend tracking
-
-`Gitlab::Tracking` is an interface that wraps the [Snowplow Ruby Tracker](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/ruby-tracker/) for tracking custom events.
-Backend tracking provides:
-
-- User behavior tracking
-- Instrumentation to monitor and visualize performance over time in a section or aspect of code.
-
-To add custom event tracking and instrumentation, call the `GitLab::Tracking.event` class method.
-For example:
-
-```ruby
-class Projects::CreateService < BaseService
-  def execute
-    project = Project.create(params)
-
-    Gitlab::Tracking.event('Projects::CreateService', 'create_project', label: project.errors.full_messages.to_sentence,
-                           property: project.valid?.to_s, project: project, user: current_user, namespace: namespace)
-  end
-end
-```
-
-Use the following arguments:
-
-| Argument   | Type                      | Default value | Description                                                                                                                       |
-|------------|---------------------------|---------------|-----------------------------------------------------------------------------------------------------------------------------------|
-| `category` | String                    |               | Area or aspect of the application. For example,  `HealthCheckController` or `Lfs::FileTransformer`.                  |
-| `action`   | String                    |               | The action being taken. For example, a controller action such as `create`, or an Active Record callback. |
-| `label`    | String                    | `nil`           | The specific element or object to act on. This can be one of the following: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown list in the top bar; or the name or title attribute of a record being created.                                                          |
-| `property` | String                    | `nil`           | Any additional property of the element, or object being acted on.                                                          |
-| `value`    | Numeric                   | `nil`           | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility.                                                          |
-| `context`  | Array\[SelfDescribingJSON\] | `nil`           | An array of custom contexts to send with this event. Most events should not have any custom contexts.                             |
-| `project`  | Project                   | `nil`           | The project associated with the event. |
-| `user`     | User                      | `nil`           | The user associated with the event. This value undergoes a pseudonymization process at the collector level. |
-| `namespace` | Namespace                | `nil`           | The namespace associated with the event. |
-| `extra`   | Hash                | `{}`         | Additional keyword arguments are collected into a hash and sent with the event. |
-
-### Unit testing
-
-To test backend Snowplow events, use the `expect_snowplow_event` helper. For more information, see
-[testing best practices](../testing_guide/best_practices.md#test-snowplow-events).
-
-### Performance
-
-We use the [AsyncEmitter](https://snowplow.github.io/snowplow-ruby-tracker/SnowplowTracker/AsyncEmitter.html) when tracking events, which allows for instrumentation calls to be run in a background thread. This is still an active area of development.
-
-## Develop and test Snowplow
-
-To develop and test a Snowplow event, there are several tools to test frontend and backend events:
-
-| Testing Tool                                 | Frontend Tracking  | Backend Tracking    | Local Development Environment | Production Environment | Production Environment |
-|----------------------------------------------|--------------------|---------------------|-------------------------------|------------------------|------------------------|
-| Snowplow Analytics Debugger Chrome Extension | Yes | No | Yes            | Yes     | Yes     |
-| Snowplow Inspector Chrome Extension          | Yes | No | Yes            | Yes     | Yes     |
-| Snowplow Micro                               | Yes | Yes  | Yes            | No    | No    |
-
-### Test frontend events
-
-Before you test frontend events in development, you must:
-
-1. [Enable Snowplow tracking in the Admin Area](index.md#enable-snowplow-tracking).
-1. Turn off ad blockers that could prevent Snowplow JavaScript from loading in your environment.
-1. Turn off "Do Not Track" (DNT) in your browser.
-
-All URLs are pseudonymized. The entity identifier [replaces](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracker-setup/other-parameters-2/#setting-a-custom-page-url-and-referrer-url) personally identifiable
-information (PII). PII includes usernames, group, and project names.
-Page titles are hardcoded as `GitLab` for the same reason.
-
-#### Snowplow Analytics Debugger Chrome Extension
-
-[Snowplow Analytics Debugger](https://www.iglooanalytics.com/blog/snowplow-analytics-debugger-chrome-extension.html) is a browser extension for testing frontend events. It works in production, staging, and local development environments.
-
-1. Install the [Snowplow Analytics Debugger](https://chrome.google.com/webstore/detail/snowplow-analytics-debugg/jbnlcgeengmijcghameodeaenefieedm) Chrome browser extension.
-1. Open Chrome DevTools to the Snowplow Analytics Debugger tab.
-
-#### Snowplow Inspector Chrome Extension
-
-Snowplow Inspector Chrome Extension is a browser extension for testing frontend events. This works in production, staging, and local development environments.
-
-<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
-For a video tutorial, see the [Snowplow plugin walk through](https://www.youtube.com/watch?v=g4rqnIZ1Mb4).
-
-1. Install [Snowplow Inspector](https://chrome.google.com/webstore/detail/snowplow-inspector/maplkdomeamdlngconidoefjpogkmljm?hl=en).
-1. To open the extension, select the Snowplow Inspector icon beside the address bar.
-1. Click around on a webpage with Snowplow to see JavaScript events firing in the inspector window.
-
-### Test backend events with Snowplow Micro
-
-[Snowplow Micro](https://snowplow.io/blog/introducing-snowplow-micro/) is a
-Docker-based solution for testing backend and frontend in a local development environment. Snowplow Micro
-records the same events as the full Snowplow pipeline. To query events, use the Snowplow Micro API.
-
-It can be set up automatically using [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit).
-See the [how-to docs](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/snowplow_micro.md) for more details.
-
-1. Set the environment variable to tell the GDK to use Snowplow Micro in development. This overrides two `application_settings` options:
-   - `snowplow_enabled` setting will instead return `true` from `Gitlab::Tracking.enabled?`
-   - `snowplow_collector_hostname` setting will instead always return `localhost:9090` (or whatever port is set for `snowplow_micro.port` GDK setting) from `Gitlab::Tracking.collector_hostname`.
-With Snowplow Micro set up you can now manually test backend Snowplow events:
-
-1. Send a test Snowplow event from the Rails console:
-
-   ```ruby
-   Gitlab::Tracking.event('category', 'action')
-   ```
-
-1. Navigate to `localhost:9090/micro/good` to see the event.
-
-#### Useful links
-
-- [Snowplow Micro repository](https://github.com/snowplow-incubator/snowplow-micro)
-- [Installation guide recording](https://www.youtube.com/watch?v=OX46fo_A0Ag)
-
-### Troubleshoot
-
-To control content security policy warnings when using an external host, modify `config/gitlab.yml`
-to allow or prevent them. To allow them, add the relevant host for `connect_src`. For example, for
-`https://snowplow.trx.gitlab.net`:
-
-```yaml
-development:
-  <<: *base
-  gitlab:
-    content_security_policy:
-      enabled: true
-      directives:
-        connect_src: "'self' http://localhost:* http://127.0.0.1:* ws://localhost:* wss://localhost:* ws://127.0.0.1:* https://snowplow.trx.gitlab.net/"
-```
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/snowplow/index.md b/doc/development/snowplow/index.md
index 4ccb90c22a6..c0e53fe3b1b 100644
--- a/doc/development/snowplow/index.md
+++ b/doc/development/snowplow/index.md
@@ -1,202 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/snowplow/index.md'
+remove_date: '2023-08-20'
 ---
 
-# Snowplow development guidelines
+This document was moved to [another location](../internal_analytics/snowplow/index.md).
 
-Snowplow is an enterprise-grade marketing and Product Intelligence platform that tracks how users engage with our website and application.
-
-[Snowplow](https://snowplow.io/) consists of several loosely-coupled sub-systems:
-
-- **Trackers** fire Snowplow events. Snowplow has twelve trackers that cover web, mobile, desktop, server, and IoT.
-- **Collectors** receive Snowplow events from trackers. We use different event collectors that synchronize events to Amazon S3, Apache Kafka, or Amazon Kinesis.
-- **Enrich** cleans raw Snowplow events, enriches them, and puts them into storage. There is a Hadoop-based enrichment process, and a Kinesis-based or Kafka-based process.
-- **Storage** stores Snowplow events. We store the Snowplow events in a flat file structure on S3, and in the Redshift and PostgreSQL databases.
-- **Data modeling** joins event-level data with other data sets, aggregates them into smaller data sets, and applies business logic. This produces a clean set of tables for data analysis. We use data models for Redshift and Looker.
-- **Analytics** are performed on Snowplow events or on aggregate tables.
-
-![Snowplow flow](../img/snowplow_flow.png)
-
-## Enable Snowplow tracking
-
-Tracking can be enabled at:
-
-- The instance level, which enables tracking on both the frontend and backend layers.
-- The user level. User tracking can be disabled on a per user basis.
-  GitLab respects the [Do Not Track](https://www.eff.org/issues/do-not-track) standard, so any user who has enabled the Do Not Track option in their browser is not tracked at a user level.
-
-Snowplow tracking is configured to send data for GitLab.com to a collector configured by GitLab. By default, self-managed
-instances do not have a collector configured and do not collect data via Snowplow.
-
-You can configure your self-managed GitLab instance to use a custom Snowplow collector.
-
-1. On the top bar, select **Main menu > Admin**, then select **Settings > General**.
-   Alternatively, go to `admin/application_settings/general` in your browser.
-
-1. Expand **Snowplow**.
-
-1. Select **Enable Snowplow tracking** and enter your Snowplow configuration information. For example:
-
-   | Name               | Value                         |
-   |--------------------|-------------------------------|
-   | Collector hostname | `your-snowplow-collector.net` |
-   | App ID             | `gitlab`                      |
-   | Cookie domain      | `.your-gitlab-instance.com`   |
-
-1. Select **Save changes**.
-
-## Snowplow request flow
-
-The following example shows a basic request/response flow between the following components:
-
-- Snowplow JS / Ruby Trackers on GitLab.com
-- [GitLab.com Snowplow Collector](https://gitlab.com/gitlab-com/gl-infra/readiness/-/blob/master/library/snowplow/index.md)
-- The GitLab S3 Bucket
-- The GitLab Snowflake Data Warehouse
-- Sisense:
-
-```mermaid
-sequenceDiagram
-    participant Snowplow JS (Frontend)
-    participant Snowplow Ruby (Backend)
-    participant GitLab.com Snowplow Collector
-    participant S3 Bucket
-    participant Snowflake DW
-    participant Sisense Dashboards
-    Snowplow JS (Frontend) ->> GitLab.com Snowplow Collector: FE Tracking event
-    Snowplow Ruby (Backend) ->> GitLab.com Snowplow Collector: BE Tracking event
-    loop Process using Kinesis Stream
-      GitLab.com Snowplow Collector ->> GitLab.com Snowplow Collector: Log raw events
-      GitLab.com Snowplow Collector ->> GitLab.com Snowplow Collector: Enrich events
-      GitLab.com Snowplow Collector ->> GitLab.com Snowplow Collector: Write to disk
-    end
-    GitLab.com Snowplow Collector ->> S3 Bucket: Kinesis Firehose
-    Note over GitLab.com Snowplow Collector, S3 Bucket: Pseudonymization
-    S3 Bucket->>Snowflake DW: Import data
-    Snowflake DW->>Snowflake DW: Transform data using dbt
-    Snowflake DW->>Sisense Dashboards: Data available for querying
-```
-
-For more details about the architecture, see [Snowplow infrastructure](infrastructure.md).
-
-## Event schema
-
-All the events must be consistent. If each feature captures events differently, it can be difficult
-to perform analysis.
-
-Each event provides attributes that describe the event.
-
-| Attribute | Type    | Required | Description |
-| --------- | ------- | -------- | ----------- |
-| category  | text    | true     | The page or backend section of the application. Unless infeasible, use the Rails page attribute by default in the frontend, and namespace + class name on the backend, for example, `Notes::CreateService`. |
-| action    | text    | true     | The action the user takes, or aspect that's being instrumented. The first word must describe the action or aspect. For example, clicks must be `click`, activations must be `activate`, creations must be `create`. Use underscores to describe what was acted on. For example, activating a form field is `activate_form_input`, an interface action like clicking on a dropdown list is `click_dropdown`, a behavior like creating a project record from the backend is `create_project`. |
-| label     | text    | false    | The specific element or object to act on. This can be one of the following: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown list in the top bar; or the name or title attribute of a record being created. For Service Ping metrics adapted to Snowplow events, this should be the full metric [key path](../service_ping/metrics_dictionary.md#metric-key_path) taken from its definition file. |
-| property  | text    | false    | Any additional property of the element, or object being acted on. For Service Ping metrics adapted to Snowplow events, this should be additional information or context that can help analyze the event. For example, in the case of `usage_activity_by_stage_monthly.create.merge_requests_users`, there are four different possible merge request actions: "create", "merge", "comment", and "close". Each of these would be a possible property value. |
-| value     | decimal | false    | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. |
-| context   | vector  | false    | Additional data in the form of a [self-describing JSON](https://docs.snowplow.io/docs/pipeline-components-and-applications/iglu/common-architecture/self-describing-json-schemas/) to describe the event if the attributes are not sufficient. Each context must have its schema defined to assure data integrity. Refer to the list of GitLab-defined contexts for more details. |
-
-### Examples
-
-| Category* | Label            | Action                | Property** | Value |
-|-------------|------------------|-----------------------|----------|:-----:|
-| `[root:index]` | `main_navigation`            | `click_navigation_link` | `[link_label]`   | - |
-| `[groups:boards:show]` | `toggle_swimlanes` | `click_toggle_button` | - | `[is_active]` |
-| `[projects:registry:index]` | `registry_delete` | `click_button` | - | - |
-| `[projects:registry:index]` | `registry_delete` | `confirm_deletion` | - | - |
-| `[projects:blob:show]` | `congratulate_first_pipeline` | `click_button` | `[human_access]` | - |
-| `[projects:clusters:new]` | `chart_options` | `generate_link` | `[chart_link]` | - |
-| `[projects:clusters:new]` | `chart_options` | `click_add_label_button` | `[label_id]` | - |
-| `API::NpmPackages` | `counts.package_events_i_package_push_package_by_deploy_token` | `push_package` | `npm` | - |
-
-_* If you choose to omit the category you can use the default._<br>
-_** Use property for variable strings._
-
-### Reference SQL
-
-#### Last 20 `reply_comment_button` events
-
-```sql
-SELECT
-  session_id,
-  event_id,
-  event_label,
-  event_action,
-  event_property,
-  event_value,
-  event_category,
-  contexts
-FROM legacy.snowplow_structured_events_all
-WHERE
-  event_label = 'reply_comment_button'
-  AND event_action = 'click_button'
-  -- AND event_category = 'projects:issues:show'
-  -- AND event_value = 1
-ORDER BY collector_tstamp DESC
-LIMIT 20
-```
-
-#### Last 100 page view events
-
-```sql
-SELECT
-  -- page_url,
-  -- page_title,
-  -- referer_url,
-  -- marketing_medium,
-  -- marketing_source,
-  -- marketing_campaign,
-  -- browser_window_width,
-  -- device_is_mobile
-  *
-FROM legacy.snowplow_page_views_30
-ORDER BY page_view_start DESC
-LIMIT 100
-```
-
-#### Top 20 users who fired `reply_comment_button` in the last 30 days
-
-```sql
-SELECT
-  count(*) as hits,
-  se_action,
-  se_category,
-  gsc_pseudonymized_user_id
-FROM legacy.snowplow_gitlab_events_30
-WHERE
-  se_label = 'reply_comment_button'
-  AND gsc_pseudonymized_user_id IS NOT NULL
-GROUP BY gsc_pseudonymized_user_id, se_category, se_action
-ORDER BY count(*) DESC
-LIMIT 20
-```
-
-#### Query JSON formatted data
-
-```sql
-SELECT
-  derived_tstamp,
-  contexts:data[0]:data:extra:old_format as CURRENT_FORMAT,
-  contexts:data[0]:data:extra:value as UPDATED_FORMAT
-FROM legacy.snowplow_structured_events_all
-WHERE event_action in ('wiki_format_updated')
-ORDER BY derived_tstamp DESC
-LIMIT 100
-```
-
-### Web-specific parameters
-
-Snowplow JavaScript adds [web-specific parameters](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/snowplow-tracker-protocol/#Web-specific_parameters) to all web events by default.
-
-## Related topics
-
-- [Snowplow data structure](https://docs.snowplow.io/docs/understanding-your-pipeline/canonical-event/)
-- [Our Iglu schema registry](https://gitlab.com/gitlab-org/iglu)
-- [List of events used in our codebase (Event Dictionary)](https://metrics.gitlab.com/snowplow/)
-- [Product Intelligence Guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)
-- [Service Ping Guide](../service_ping/index.md)
-- [Product Intelligence Direction](https://about.gitlab.com/direction/analytics/product-intelligence/)
-- [Data Analysis Process](https://about.gitlab.com/handbook/business-technology/data-team/#data-analysis-process/)
-- [Data for Product Managers](https://about.gitlab.com/handbook/business-technology/data-team/programs/data-for-product-managers/)
-- [Data Infrastructure](https://about.gitlab.com/handbook/business-technology/data-team/platform/infrastructure/)
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/snowplow/infrastructure.md b/doc/development/snowplow/infrastructure.md
index ac146542630..6374af40ffe 100644
--- a/doc/development/snowplow/infrastructure.md
+++ b/doc/development/snowplow/infrastructure.md
@@ -1,101 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/snowplow/infrastructure.md'
+remove_date: '2023-08-20'
 ---
 
-# Snowplow infrastructure
+This document was moved to [another location](../internal_analytics/snowplow/infrastructure.md).
 
-Snowplow events on GitLab SaaS fired by a [tracker](implementation.md) go through an AWS pipeline, managed by GitLab.
-
-## Event flow in the AWS pipeline
-
-Every event goes through a collector, enricher, and pseudonymization lambda. The event is then dumped to S3 storage where it can be picked up by the Snowflake data warehouse.
-
-Deploying and managing the infrastructure is automated using Terraform in the current [Terraform repository](https://gitlab.com/gitlab-com/gl-infra/config-mgmt/-/tree/master/environments/aws-snowplow).
-
-```mermaid
-graph LR
-    GL[GitLab.com]-->COL
-
-    subgraph aws-cloud[AWS]
-    COL[Collector]-->|snowplow-raw-good|ENR
-    COL[Collector]-->|snowplow-raw-bad|FRBE
-        subgraph firehoserbe[Firehose]
-        FRBE[AWS Lambda]
-      end
-    FRBE-->S3RBE
-
-    ENR[Enricher]-->|snowplow-enriched-bad|FEBE
-      subgraph firehoseebe[Firehose]
-        FEBE[AWS Lambda]
-      end
-    FEBE-->S3EBE
-
-    ENR[Enricher]-->|snowplow-enriched-good|FRGE
-      subgraph firehosege[Firehose]
-        FRGE[AWS Lambda]
-      end
-    FRGE-->S3GE
-  end
-
-    subgraph snowflake[Data warehouse]
-    S3RBE[S3 raw-bad]-->BE[gitlab_bad_events]
-    S3EBE[S3 enriched-bad]-->BE[gitlab_bad_events]
-    S3GE[S3 output]-->GE[gitlab_events]
-  end
-```
-
-See [Snowplow technology 101](https://github.com/snowplow/snowplow/#snowplow-technology-101) for Snowplow's own documentation and an overview how collectors and enrichers work.
-
-### Pseudonymization
-
-In contrast to a typical Snowplow pipeline, after enrichment, GitLab Snowplow events go through a [pseudonymization service](https://gitlab.com/gitlab-org/analytics-section/product-intelligence/snowplow-pseudonymization) in the form of an AWS Lambda service before they are stored in S3 storage.
-
-#### Why events need to be pseudonymized
-
-GitLab is bound by its [obligations to community](https://about.gitlab.com/handbook/product/product-intelligence-guide/service-usage-data-commitment/)
-and by [legal regulations](https://about.gitlab.com/handbook/legal/privacy/services-usage-data/) to protect the privacy of its users.
-
-GitLab must provide valuable insights for business decisions, and there is a need
-for a better understanding of different users' behavior patterns. The
-pseudonymization process helps you find a compromise between these two requirements.
-
-Pseudonymization processes personally identifiable information inside a Snowplow event in an irreversible fashion
-maintaining deterministic output for given input, while masking any relation to that input.
-
-#### How events are pseudonymized
-
-Pseudonymization uses an allowlist that provides privacy by default. Therefore, each
-attribute received as part of a Snowplow event is pseudonymized unless the attribute
-is an allowed exception.
-
-Pseudonymization is done using the HMAC-SHA256 keyed hash algorithm.
-Attributes are combined with a secret salt to replace each identifiable information with a pseudonym.
-
-### S3 bucket data lake to Snowflake
-
-See [Data team's Snowplow Overview](https://about.gitlab.com/handbook/business-technology/data-team/platform/snowplow/) for further details how data is ingested into our Snowflake data warehouse.
-
-## Monitoring
-
-There are several tools that monitor Snowplow events tracking in different stages of the processing pipeline:
-
-- [Product Intelligence Grafana dashboard](https://dashboards.gitlab.net/d/product-intelligence-main/product-intelligence-product-intelligence?orgId=1) monitors backend events sent from a GitLab.com instance to a collectors fleet. This dashboard provides information about:
-  - The number of events that successfully reach Snowplow collectors.
-  - The number of events that failed to reach Snowplow collectors.
-  - The number of backend events that were sent.
-- [AWS CloudWatch dashboard](https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#dashboards:name=SnowPlow;start=P3D) monitors the state of the events in a processing pipeline. The pipeline starts from Snowplow collectors, goes through to enrichers and pseudonymization, and then up to persistence in an S3 bucket. From S3, the events are imported into the Snowflake Data Warehouse. You must have AWS access rights to view this dashboard. For more information, see [monitoring](https://gitlab.com/gitlab-org/analytics-section/product-intelligence/snowplow-pseudonymization#monitoring) in the Snowplow Events pseudonymization service documentation.
-- [Sisense dashboard](https://app.periscopedata.com/app/gitlab/417669/Snowplow-Summary-Dashboard) provides information about the number of good and bad events imported into the Data Warehouse, in addition to the total number of imported Snowplow events.
-
-For more information, see this [video walk-through](https://www.youtube.com/watch?v=NxPS0aKa_oU).
-
-## Related topics
-
-- [Snowplow technology 101](https://github.com/snowplow/snowplow/#snowplow-technology-101)
-- [Snowplow pseudonymization AWS Lambda project](https://gitlab.com/gitlab-org/analytics-section/product-intelligence/snowplow-pseudonymization)
-- [Product Intelligence Guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)
-- [Data Infrastructure](https://about.gitlab.com/handbook/business-technology/data-team/platform/infrastructure/)
-- [Snowplow architecture overview (internal)](https://www.youtube.com/watch?v=eVYJjzspsLU)
-- [Snowplow architecture overview slide deck (internal)](https://docs.google.com/presentation/d/16gQEO5CAg8Tx4NBtfnZj-GF4juFI6HfEPWcZgH4Rn14/edit?usp=sharing)
-- [AWS Lambda implementation (internal)](https://youtu.be/cQd0mdMhkQA)
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/snowplow/review_guidelines.md b/doc/development/snowplow/review_guidelines.md
index d5432cc9075..f4752e08dde 100644
--- a/doc/development/snowplow/review_guidelines.md
+++ b/doc/development/snowplow/review_guidelines.md
@@ -1,44 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/snowplow/review_guidelines.md'
+remove_date: '2023-08-20'
 ---
 
-# Snowplow review guidelines
+This document was moved to [another location](../internal_analytics/snowplow/review_guidelines.md).
 
-This page includes introductory material for an
-[Analytics Instrumentation](https://about.gitlab.com/handbook/engineering/development/analytics/analytics-instrumentation/)
-review, and is specific to Snowplow related reviews. For broader advice and
-general best practices for code reviews, refer to our [code review guide](../code_review.md).
-
-## Resources for reviewers
-
-- [Snowplow Guide](index.md)
-- [Event Dictionary](https://metrics.gitlab.com/snowplow/)
-
-## Review process
-
-We recommend an Analytics Instrumentation review when a merge request (MR) involves changes in
-events or touches Snowplow related files.
-
-### Roles and process
-
-#### The merge request **author** should
-
-- For frontend events, when relevant, add a screenshot of the event in
-  the [testing tool](implementation.md#develop-and-test-snowplow) used.
-- For backend events, when relevant, add the output of the
-  [Snowplow Micro](implementation.md#test-backend-events-with-snowplow-micro) good events
-  `GET http://localhost:9090/micro/good` (it might be a good idea
-  to reset with `GET http://localhost:9090/micro/reset` first).
-- Add or update the event definition file according to the [Event Dictionary Guide](event_dictionary_guide.md).
-
-#### The Analytics Instrumentation **reviewer** should
-
-- Check that the [event schema](index.md#event-schema) is correct.
-- Check the [usage recommendations](implementation.md#usage-recommendations).
-- Check that an event definition file was created or updated in accordance with the [Event Dictionary Guide](event_dictionary_guide.md).
-- If needed, check that the events are firing locally using one of the
-[testing tools](implementation.md#develop-and-test-snowplow) available.
-- Approve the MR, and relabel the MR with `~"product intelligence::approved"`.
-- If the snowplow event mirrors a RedisHLL event, then tag @mdrussell to review if the payload is usable for this purpose.
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/snowplow/schemas.md b/doc/development/snowplow/schemas.md
index 0ef6ed04aa3..7e00ddd976d 100644
--- a/doc/development/snowplow/schemas.md
+++ b/doc/development/snowplow/schemas.md
@@ -1,189 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/snowplow/schemas.md'
+remove_date: '2023-08-20'
 ---
 
-# Snowplow schemas
+This document was moved to [another location](../internal_analytics/snowplow/schemas.md).
 
-This page provides Snowplow schema reference for GitLab events.
-
-## `gitlab_standard`
-
-We are including the [`gitlab_standard` schema](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_standard/jsonschema/) for structured events and page views.
-
-The [`StandardContext`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/tracking/standard_context.rb)
-class represents this schema in the application. Some properties are
-[automatically populated for frontend events](implementation.md#snowplow-javascript-frontend-tracking),
-and can be [provided manually for backend events](implementation.md#implement-ruby-backend-tracking).
-
-| Field Name     | Required            | Default value | Type                  | Description                                                                                 |
-|----------------|:-------------------:|-----------------------|--|---------------------------------------------------------------------------------------------|
-| `project_id`   | **{dotted-circle}** | Current project ID * | integer               |                                                                 |
-| `namespace_id` | **{dotted-circle}** | Current group/namespace ID * | integer               |                                                               |
-| `user_id`      | **{dotted-circle}** | Current user ID * | integer               | User database record ID attribute. This value undergoes a pseudonymization process at the collector level. |
-| `context_generated_at` | **{dotted-circle}** | Current timestamp | string (date time format) | Timestamp indicating when context was generated.                                                                |
-| `environment`  | **{check-circle}**  | Current environment | string (max 32 chars) | Name of the source environment, such as `production` or `staging`             |
-| `source`       | **{check-circle}**  | Event source | string (max 32 chars) | Name of the source application, such as  `gitlab-rails` or `gitlab-javascript` |
-| `plan`         | **{dotted-circle}**  | Current namespace plan * | string (max 32 chars) | Name of the plan for the namespace, such as `free`, `premium`, or `ultimate`. Automatically picked from the `namespace`. |
-| `google_analytics_id` | **{dotted-circle}**  | GA ID value * | string (max 32 chars) | Google Analytics ID, present when set from our marketing sites. |
-| `extra`        | **{dotted-circle}**  |  | JSON                  | Any additional data associated with the event, in the form of key-value pairs |
-
-_\* Default value present for frontend events only_
-
-## Default Schema
-
-Frontend events include a [web-specific schema](https://docs.snowplow.io/docs/understanding-your-pipeline/canonical-event/#web-specific-fields) provided by Snowplow.
-All URLs are pseudonymized. The entity identifier [replaces](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracker-setup/other-parameters-2/#setting-a-custom-page-url-and-referrer-url) personally identifiable
-information (PII). PII includes usernames, group, and project names.
-Page titles are hardcoded as `GitLab` for the same reason.
-
-| Field Name               | Required            | Type      | Description                                                                                                                      |
-|--------------------------|---------------------|-----------|----------------------------------------------------------------------------------------------------------------------------------|
-| `app_id`                   | **{check-circle}**  | string    | Unique identifier for website / application                                                                                      |
-| `base_currency`            | **{dotted-circle}** | string    | Reporting currency                                                                                                               |
-| `br_colordepth`            | **{dotted-circle}** | integer   | Browser color depth                                                                                                              |
-| `br_cookies`               | **{dotted-circle}** | boolean   | Does the browser permit cookies?                                                                                                 |
-| `br_family`                | **{dotted-circle}** | string    | Browser family                                                                                                                   |
-| `br_features_director`     | **{dotted-circle}** | boolean   | Director plugin installed?                                                                                                       |
-| `br_features_flash`        | **{dotted-circle}** | boolean   | Flash plugin installed?                                                                                                          |
-| `br_features_gears`        | **{dotted-circle}** | boolean   | Google gears installed?                                                                                                          |
-| `br_features_java`         | **{dotted-circle}** | boolean   | Java plugin installed?                                                                                                           |
-| `br_features_pdf`          | **{dotted-circle}** | boolean   | Adobe PDF plugin installed?                                                                                                      |
-| `br_features_quicktime`    | **{dotted-circle}** | boolean   | Quicktime plugin installed?                                                                                                      |
-| `br_features_realplayer`   | **{dotted-circle}** | boolean   | RealPlayer plugin installed?                                                                                                     |
-| `br_features_silverlight`  | **{dotted-circle}** | boolean   | Silverlight plugin installed?                                                                                                    |
-| `br_features_windowsmedia` | **{dotted-circle}** | boolean   | Windows media plugin installed?                                                                                                  |
-| `br_lang`                  | **{dotted-circle}** | string    | Language the browser is set to                                                                                                   |
-| `br_name`                  | **{dotted-circle}** | string    | Browser name                                                                                                                     |
-| `br_renderengine`          | **{dotted-circle}** | string    | Browser rendering engine                                                                                                         |
-| `br_type`                  | **{dotted-circle}** | string    | Browser type                                                                                                                     |
-| `br_version`               | **{dotted-circle}** | string    | Browser version                                                                                                                  |
-| `br_viewheight`            | **{dotted-circle}** | string    | Browser viewport height                                                                                                          |
-| `br_viewwidth`             | **{dotted-circle}** | string    | Browser viewport width                                                                                                           |
-| `collector_tstamp`         | **{dotted-circle}** | timestamp | Time stamp for the event recorded by the collector                                                                               |
-| `contexts`                 | **{dotted-circle}** |           |                                                                                                                                  |
-| `derived_contexts`         | **{dotted-circle}** |           | Contexts derived in the Enrich process                                                                                           |
-| `derived_tstamp`           | **{dotted-circle}** | timestamp | Timestamp making allowance for inaccurate device clock                                                                          |
-| `doc_charset`              | **{dotted-circle}** | string    | Web page's character encoding                                                                                                    |
-| `doc_height`               | **{dotted-circle}** | string    | Web page height                                                                                                                  |
-| `doc_width`                | **{dotted-circle}** | string    | Web page width                                                                                                                   |
-| `domain_sessionid`         | **{dotted-circle}** | string    | Unique identifier (UUID) for this visit of this `user_id` to this domain                                                         |
-| `domain_sessionidx`        | **{dotted-circle}** | integer   | Index of number of visits that this `user_id` has made to this domain (The first visit is `1`)                                   |
-| `domain_userid`            | **{dotted-circle}** | string    | Unique identifier for a user, based on a first party cookie (so domain specific)                                                 |
-| `dvce_created_tstamp`      | **{dotted-circle}** | timestamp | Timestamp when event occurred, as recorded by client device                                                                      |
-| `dvce_ismobile`            | **{dotted-circle}** | boolean   | Indicates whether device is mobile                                                                                               |
-| `dvce_screenheight`        | **{dotted-circle}** | string    | Screen / monitor resolution                                                                                                      |
-| `dvce_screenwidth`         | **{dotted-circle}** | string    | Screen / monitor resolution                                                                                                      |
-| `dvce_sent_tstamp`         | **{dotted-circle}** | timestamp | Timestamp when event was sent by client device to collector                                                                      |
-| `dvce_type`                | **{dotted-circle}** | string    | Type of device                                                                                                                   |
-| `etl_tags`                 | **{dotted-circle}** | string    | JSON of tags for this ETL run                                                                                                    |
-| `etl_tstamp`               | **{dotted-circle}** | timestamp | Timestamp event began ETL                                                                                                        |
-| `event`                    | **{dotted-circle}** | string    | Event type                                                                                                                       |
-| `event_fingerprint`        | **{dotted-circle}** | string    | Hash client-set event fields                                                                                                     |
-| `event_format`             | **{dotted-circle}** | string    | Format for event                                                                                                                 |
-| `event_id`                 | **{dotted-circle}** | string    | Event UUID                                                                                                                       |
-| `event_name`               | **{dotted-circle}** | string    | Event name                                                                                                                       |
-| `event_vendor`             | **{dotted-circle}** | string    | The company who developed the event model                                                                                        |
-| `event_version`            | **{dotted-circle}** | string    | Version of event schema                                                                                                          |
-| `geo_city`                 | **{dotted-circle}** | string    | City of IP origin                                                                                                                |
-| `geo_country`              | **{dotted-circle}** | string    | Country of IP origin                                                                                                             |
-| `geo_latitude`             | **{dotted-circle}** | string    | An approximate latitude                                                                                                          |
-| `geo_longitude`            | **{dotted-circle}** | string    | An approximate longitude                                                                                                         |
-| `geo_region`               | **{dotted-circle}** | string    | Region of IP origin                                                                                                              |
-| `geo_region_name`          | **{dotted-circle}** | string    | Region of IP origin                                                                                                              |
-| `geo_timezone`             | **{dotted-circle}** | string    | Time zone of IP origin                                                                                                            |
-| `geo_zipcode`              | **{dotted-circle}** | string    | Zip (postal) code of IP origin                                                                                                   |
-| `ip_domain`                | **{dotted-circle}** | string    | Second level domain name associated with the visitor's IP address                                                                |
-| `ip_isp`                   | **{dotted-circle}** | string    | Visitor's ISP                                                                                                                    |
-| `ip_netspeed`              | **{dotted-circle}** | string    | Visitor's connection type                                                                                                        |
-| `ip_organization`          | **{dotted-circle}** | string    | Organization associated with the visitor's IP address – defaults to ISP name if none is found                                    |
-| `mkt_campaign`             | **{dotted-circle}** | string    | The campaign ID                                                                                                                  |
-| `mkt_clickid`              | **{dotted-circle}** | string    | The click ID                                                                                                                     |
-| `mkt_content`              | **{dotted-circle}** | string    | The content or ID of the ad.                                                                   |
-| `mkt_medium`               | **{dotted-circle}** | string    | Type of traffic source                                                                                                           |
-| `mkt_network`              | **{dotted-circle}** | string    | The ad network to which the click ID belongs                                                                                     |
-| `mkt_source`               | **{dotted-circle}** | string    | The company / website where the traffic came from                                                                                |
-| `mkt_term`                 | **{dotted-circle}** | string    | Keywords associated with the referrer                                                                                        |
-| `name_tracker`             | **{dotted-circle}** | string    | The tracker namespace                                                                                                            |
-| `network_userid`           | **{dotted-circle}** | string    | Unique identifier for a user, based on a cookie from the collector (so set at a network level and shouldn't be set by a tracker) |
-| `os_family`                | **{dotted-circle}** | string    | Operating system family                                                                                                          |
-| `os_manufacturer`          | **{dotted-circle}** | string    | Manufacturers of operating system                                                                                                |
-| `os_name`                  | **{dotted-circle}** | string    | Name of operating system                                                                                                         |
-| `os_timezone`              | **{dotted-circle}** | string    | Client operating system time zone                                                                                                 |
-| `page_referrer`            | **{dotted-circle}** | string    | Referrer URL                                                                                                                     |
-| `page_title`               | **{dotted-circle}** | string    | To not expose personal identifying information, the page title is hardcoded as `GitLab`                                          |
-| `page_url`                 | **{dotted-circle}** | string    | Page URL                                                                                                                         |
-| `page_urlfragment`         | **{dotted-circle}** | string    | Fragment aka anchor                                                                                                              |
-| `page_urlhost`             | **{dotted-circle}** | string    | Host aka domain                                                                                                                  |
-| `page_urlpath`             | **{dotted-circle}** | string    | Path to page                                                                                                                     |
-| `page_urlport`             | **{dotted-circle}** | integer   | Port if specified, 80 if not                                                                                                     |
-| `page_urlquery`            | **{dotted-circle}** | string    | Query string                                                                                                                      |
-| `page_urlscheme`           | **{dotted-circle}** | string    | Scheme (protocol name)                                                                                                              |
-| `platform`                 | **{dotted-circle}** | string    | The platform the app runs on                                                                                                     |
-| `pp_xoffset_max`           | **{dotted-circle}** | integer   | Maximum page x offset seen in the last ping period                                                                               |
-| `pp_xoffset_min`           | **{dotted-circle}** | integer   | Minimum page x offset seen in the last ping period                                                                               |
-| `pp_yoffset_max`           | **{dotted-circle}** | integer   | Maximum page y offset seen in the last ping period                                                                               |
-| `pp_yoffset_min`           | **{dotted-circle}** | integer   | Minimum page y offset seen in the last ping period                                                                               |
-| `refr_domain_userid`       | **{dotted-circle}** | string    | The Snowplow `domain_userid` of the referring website                                                                              |
-| `refr_dvce_tstamp`         | **{dotted-circle}** | timestamp | The time of attaching the `domain_userid` to the inbound link                                                                      |
-| `refr_medium`              | **{dotted-circle}** | string    | Type of referer                                                                                                                  |
-| `refr_source`              | **{dotted-circle}** | string    | Name of referer if recognised                                                                                                    |
-| `refr_term`                | **{dotted-circle}** | string    | Keywords if source is a search engine                                                                                            |
-| `refr_urlfragment`         | **{dotted-circle}** | string    | Referer URL fragment                                                                                                             |
-| `refr_urlhost`             | **{dotted-circle}** | string    | Referer host                                                                                                                     |
-| `refr_urlpath`             | **{dotted-circle}** | string    | Referer page path                                                                                                                |
-| `refr_urlport`             | **{dotted-circle}** | integer   | Referer port                                                                                                                     |
-| `refr_urlquery`            | **{dotted-circle}** | string    | Referer URL query string                                                                                                          |
-| `refr_urlscheme`           | **{dotted-circle}** | string    | Referer scheme                                                                                                                   |
-| `se_action`                | **{dotted-circle}** | string    | The action / event itself                                                                                                        |
-| `se_category`              | **{dotted-circle}** | string    | The category of event                                                                                                            |
-| `se_label`                 | **{dotted-circle}** | string    | A label often used to refer to the 'object' the action is performed on                                                           |
-| `se_property`              | **{dotted-circle}** | string    | A property associated with either the action or the object                                                                       |
-| `se_value`                 | **{dotted-circle}** | decimal   | A value associated with the user action                                                                                          |
-| `ti_category`              | **{dotted-circle}** | string    | Item category                                                                                                                    |
-| `ti_currency`              | **{dotted-circle}** | string    | Currency                                                                                                                         |
-| `ti_name`                  | **{dotted-circle}** | string    | Item name                                                                                                                        |
-| `ti_orderid`               | **{dotted-circle}** | string    | Order ID                                                                                                                         |
-| `ti_price`                 | **{dotted-circle}** | decimal   | Item price                                                                                                                       |
-| `ti_price_base`            | **{dotted-circle}** | decimal   | Item price in base currency                                                                                                      |
-| `ti_quantity`              | **{dotted-circle}** | integer   | Item quantity                                                                                                                    |
-| `ti_sku`                   | **{dotted-circle}** | string    | Item SKU                                                                                                                         |
-| `tr_affiliation`           | **{dotted-circle}** | string    | Transaction affiliation (such as channel)                                                                                           |
-| `tr_city`                  | **{dotted-circle}** | string    | Delivery address: city                                                                                                           |
-| `tr_country`               | **{dotted-circle}** | string    | Delivery address: country                                                                                                        |
-| `tr_currency`              | **{dotted-circle}** | string    | Transaction Currency                                                                                                             |
-| `tr_orderid`               | **{dotted-circle}** | string    | Order ID                                                                                                                         |
-| `tr_shipping`              | **{dotted-circle}** | decimal   | Delivery cost charged                                                                                                            |
-| `tr_shipping_base`         | **{dotted-circle}** | decimal   | Shipping cost in base currency                                                                                                   |
-| `tr_state`                 | **{dotted-circle}** | string    | Delivery address: state                                                                                                          |
-| `tr_tax`                   | **{dotted-circle}** | decimal   | Transaction tax value (such as amount of VAT included)                                                                              |
-| `tr_tax_base`              | **{dotted-circle}** | decimal   | Tax applied in base currency                                                                                                     |
-| `tr_total`                 | **{dotted-circle}** | decimal   | Transaction total value                                                                                                          |
-| `tr_total_base`            | **{dotted-circle}** | decimal   | Total amount of transaction in base currency                                                                                     |
-| `true_tstamp`              | **{dotted-circle}** | timestamp | User-set exact timestamp                                                                                                         |
-| `txn_id`                   | **{dotted-circle}** | string    | Transaction ID                                                                                                                   |
-| `unstruct_event`           | **{dotted-circle}** | JSON      | The properties of the event                                                                                                      |
-| `uploaded_at`              | **{dotted-circle}** |           |                                                                                                                                  |
-| `user_fingerprint`         | **{dotted-circle}** | integer   | User identifier based on (hopefully unique) browser features                                                                     |
-| `user_id`                  | **{dotted-circle}** | string    | Unique identifier for user, set by the business using setUserId                                                                  |
-| `user_ipaddress`           | **{dotted-circle}** | string    | IP address                                                                                                                       |
-| `useragent`                | **{dotted-circle}** | string    | User agent (expressed as a browser string)                                                                                                |
-| `v_collector`              | **{dotted-circle}** | string    | Collector version                                                                                                                |
-| `v_etl`                    | **{dotted-circle}** | string    | ETL version                                                                                                                      |
-| `v_tracker`                | **{dotted-circle}** | string    | Identifier for Snowplow tracker                                                                                                  |
-
-## `gitlab_service_ping`
-
-Backend events converted from ServicePing (`redis` and `redis_hll`) must include [ServicePing context](https://gitlab.com/gitlab-org/iglu/-/tree/master/public/schemas/com.gitlab/gitlab_service_ping/jsonschema)
-using the [helper class](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/tracking/service_ping_context.rb).
-
-An example of converted `redis_hll` [event with context](https://gitlab.com/gitlab-org/gitlab/-/edit/master/app/controllers/concerns/product_analytics_tracking.rb#L58).
-
-| Field Name    |      Required       | Type                   | Description                                                                                                                                                        |
-|---------------|:-------------------:|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `data_source` | **{check-circle}**  | string (max 64 chars)  | The `data_source` attribute from the metrics YAML definition. |
-| `event_name`* | **{dotted-circle}** | string (max 128 chars) | When there is a many-to-many relationship between events and metrics, this field contains the name of a Redis event that can be used for aggregations in downstream systems |
-| `key_path`*   | **{dotted-circle}** | string (max 256 chars) | The `key_path` attribute from the metrics YAML definition |
-
-_\* Either `event_name` or `key_path` is required_
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/snowplow/troubleshooting.md b/doc/development/snowplow/troubleshooting.md
index 92267dfcb0c..ed1f5033239 100644
--- a/doc/development/snowplow/troubleshooting.md
+++ b/doc/development/snowplow/troubleshooting.md
@@ -1,80 +1,11 @@
 ---
-stage: Analytics
-group: Analytics Instrumentation
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../internal_analytics/snowplow/troubleshooting.md'
+remove_date: '2023-08-20'
 ---
 
-# Troubleshooting Snowplow
+This document was moved to [another location](../internal_analytics/snowplow/troubleshooting.md).
 
-## Monitoring
-
-This page covers dashboards and alerts coming from a number of internal tools.
-
-For a brief video overview of the tools used to monitor Snowplow usage, please check out [this internal video](https://www.youtube.com/watch?v=NxPS0aKa_oU) (you must be logged into GitLab Unfiltered to view).
-
-## Good events drop
-
-### Symptoms
-
-You will be alarmed via a [Sisense alert](https://app.periscopedata.com/app/gitlab/alert/Volume-of-Snowplow-Good-events/5a5f80ef34fe450da5ebb84eaa84067f/edit) that is sent to `#g_product_intelligence` Slack channel
-
-### Locating the problem
-
-First you need to identify at which stage in Snowplow the data pipeline the drop is occurring.
-Start at [Snowplow dashboard](https://console.aws.amazon.com/systems-manager/resource-groups/cloudwatch?dashboard=SnowPlow&region=us-east-1#) on CloudWatch,
-if you do not have access to CloudWatch you need to create an [access request issue](https://gitlab.com/gitlab-com/team-member-epics/access-requests/-/issues/9730) first.
-While on CloudWatch dashboard set time range to last 4 weeks, to get better picture of system characteristics over time. Than visit following charts:
-
-1. `ELB New Flow Count` and `Collector Auto Scaling Group Network In/Out` - they show in order: number of connections to collectors via load balancers and data volume (in bytes) processed by collectors. If there is drop visible there, it means less events were fired from the GitLab application. Proceed to [application layer guide](#troubleshooting-gitlab-application-layer) for more details
-1. `Firehose Records to S3` - it shows how many event records were saved to S3 bucket, if there was drop on this chart but not on the charts from 1. it means that problem is located at AWS infrastructure layer, please refer to [AWS layer guide](#troubleshooting-aws-layer)
-1. If drop wasn't visible on any of previous charts it means that problem is at data warehouse layer, please refer to [data warehouse layer guide](#troubleshooting-data-warehouse-layer)
-
-### Troubleshooting GitLab application layer
-
-Drop occurring at application layer can be symptom of some issue, but it might be also a result of normal application lifecycle, intended changes done to product intelligence or experiments tracking
-or even a result of a public holiday in some regions of the world with a larger user-base. To verify if there is an underlying problem to solve, you can check following things:
-
-1. Check `about.gitlab.com` website traffic on [Google Analytics](https://analytics.google.com/analytics/web/) to verify if some public holiday might impact overall use of GitLab system
-   1. You may require to open an access request for Google Analytics access first, for example: [access request internal issue](https://gitlab.com/gitlab-com/team-member-epics/access-requests/-/issues/1772)
-1. Plot `select date(dvce_created_tstamp) , event , count(*) from legacy.snowplow_unnested_events_90 where dvce_created_tstamp > '2021-06-15' and dvce_created_tstamp < '2021-07-10' group by 1 , 2 order by 1 , 2` in SiSense to see what type of events was responsible for drop
-1. Plot `select date(dvce_created_tstamp) ,se_category , count(*) from legacy.snowplow_unnested_events_90 where dvce_created_tstamp > '2021-06-15' and dvce_created_tstamp < '2021-07-31' and event = 'struct' group by  1 , 2 order by  1, 2` what events recorded the biggest drops in suspected category
-1. Check if there was any MR merged that might cause reduction in reported events, pay an attention to ~"product intelligence" and ~"growth experiment" labeled MRs
-1. Check (via [Grafana explore tab](https://dashboards.gitlab.net/explore) ) following Prometheus counters `gitlab_snowplow_events_total`, `gitlab_snowplow_failed_events_total` and `gitlab_snowplow_successful_events_total` to see how many events were fired correctly from GitLab.com. Example query to use `sum(rate(gitlab_snowplow_successful_events_total{env="gprd"}[5m])) / sum(rate(gitlab_snowplow_events_total{env="gprd"}[5m]))` would chart rate at which number of good events rose in comparison to events sent in total. If number drops from 1 it means that problem might be in communication between GitLab and AWS collectors fleet.
-1. Check [logs in Kibana](https://log.gprd.gitlab.net/app/discover#) and filter with `{ "query": { "match_phrase": { "json.message": "failed to be reported to collector at" } } }` if there are some failed events logged
-
-For results about an investigation conducted into an unexpected drop in snowplow events volume, see [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/335206).
-
-### Troubleshooting AWS layer
-
-Already conducted investigations:
-
-- [Steep decrease of Snowplow page views](https://gitlab.com/gitlab-org/gitlab/-/issues/268009)
-- [`snowplow.trx.gitlab.net` unreachable](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/5073)
-
-### Troubleshooting data warehouse layer
-
-Reach out to [Data team](https://about.gitlab.com/handbook/business-technology/data-team/) to ask about current state of data warehouse. On their handbook page there is a [section with contact details](https://about.gitlab.com/handbook/business-technology/data-team/#how-to-connect-with-us)
-
-## Delay in Snowplow Enrichers
-
-If there is an alert for **Snowplow Raw Good Stream Backing Up**, we receive an email notification. This sometimes happens because Snowplow Enrichers don't scale well enough for the amount of Snowplow events.
-
-If the delay goes over 48 hours, we lose data.
-
-### Contact SRE on-call
-
-Send a message in the [#infrastructure_lounge](https://gitlab.slack.com/archives/CB3LSMEJV) Slack channel using the following template:
-
-```markdown
-Hello team!
-
-We received an alert for [Snowplow Raw Good Stream Backing Up](https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#alarmsV2:alarm/SnowPlow+Raw+Good+Stream+Backing+Up?).
-
-Enrichers are not scalling well for the amount of events we receive.
-
-See the [dashboard](https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#dashboards:name=SnowPlow).
-
-Could we get assistance to fix the delay?
-
-Thank you!
-```
+<!-- This redirect file can be deleted after <2023-08-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html
diff --git a/doc/development/spam_protection_and_captcha/graphql_api.md b/doc/development/spam_protection_and_captcha/graphql_api.md
index 383b52df1fc..5723433203b 100644
--- a/doc/development/spam_protection_and_captcha/graphql_api.md
+++ b/doc/development/spam_protection_and_captcha/graphql_api.md
@@ -16,9 +16,8 @@ related to changing a model's confidential/public flag.
 The main steps are:
 
 1. Use `include Mutations::SpamProtection` in your mutation.
-1. Create a `spam_params` instance based on the request. Obtain the request from the context
-   via `context[:request]` when creating the `SpamParams` instance.
-1. Pass `spam_params` to the relevant Service class constructor.
+1. Pass `perform_spam_check: true` to the Update Service class constructor.
+   It is set to `true` by default in the Create Service.
 1. After you create or update the `Spammable` model instance, call `#check_spam_action_response!`
    and pass it the model instance. This call:
    1. Performs the necessary spam checks on the model.
@@ -44,13 +43,10 @@ module Mutations
       include Mutations::SpamProtection
 
       def resolve(args)
-        spam_params = ::Spam::SpamParams.new_from_request(request: context[:request])
-
         service_response = ::Widgets::CreateService.new(
           project: project,
           current_user: current_user,
-          params: args,
-          spam_params: spam_params
+          params: args
         ).execute
 
         widget = service_response.payload[:widget]
diff --git a/doc/development/spam_protection_and_captcha/model_and_services.md b/doc/development/spam_protection_and_captcha/model_and_services.md
index 2c7ec8c3f50..5f91bf47af4 100644
--- a/doc/development/spam_protection_and_captcha/model_and_services.md
+++ b/doc/development/spam_protection_and_captcha/model_and_services.md
@@ -20,7 +20,7 @@ To do this:
 
 1. [Add `Spammable` support to the ActiveRecord model](#add-spammable-support-to-the-activerecord-model).
 1. [Add support for the `mark_as_spam` action to the controller](#add-support-for-the-mark_as_spam-action-to-the-controller).
-1. [Add a call to SpamActionService to the execute method of services](#add-a-call-to-spamactionservice-to-the-execute-method-of-services).
+1. [Add a call to `check_for_spam` to the execute method of services](#add-a-call-to-check_for_spam-to-the-execute-method-of-services).
 
 ## Add `Spammable` support to the ActiveRecord model
 
@@ -80,13 +80,11 @@ NOTE:
 There may be other changes needed to controllers, depending on how the feature is
 implemented. See [Web UI](web_ui.md) for more details.
 
-## Add a call to SpamActionService to the execute method of services
+## Add a call to `check_for_spam` to the execute method of services
 
 This approach applies to any service which can persist spammable attributes:
 
-1. In the relevant Create or Update service under `app/services`, pass in a populated
-   `Spam::SpamParams` instance. (Refer to instructions later on in this page.)
-1. Use it and the `Spammable` model instance to execute a `Spam::SpamActionService` instance.
+1. In the relevant Create or Update service under `app/services`, call the `check_for_spam` method on the model.
 1. If the spam check fails:
    - An error is added to the model, which causes it to be invalid and prevents it from being saved.
    - The `needs_recaptcha` property is set to `true`.
@@ -95,38 +93,24 @@ This approach applies to any service which can persist spammable attributes:
 
 Make these changes to each relevant service:
 
-1. Change the constructor to take a `spam_params:` argument as a required named argument.
-
-   Using named arguments for the constructor helps you identify all the calls to
-   the constructor that need changing. It's less risky because the interpreter raises
-   type errors unless the caller is changed to pass the `spam_params` argument.
-   If you use an IDE (such as RubyMine) which supports this, your
-   IDE flags it as an error in the editor.
-
-1. In the constructor, set the `@spam_params` instance variable from the `spam_params` constructor
-   argument. Add an `attr_reader: :spam_params` in the `private` section of the class.
-
-1. In the `execute` method, add a call to execute the `Spam::SpamActionService`.
+1. In the `execute` method, call the `check_for_spam` method on the model.
    (You can also use `before_create` or `before_update`, if the service
    uses that pattern.) This method uses named arguments, so its usage is clear if
    you refer to existing examples. However, two important considerations exist:
-   1. The `SpamActionService` must be executed _after_ all necessary changes are made to
+   1. The `check_for_spam` must be executed _after_ all necessary changes are made to
       the unsaved (and dirty) `Spammable` model instance. This ordering ensures
       spammable attributes exist to be spam-checked.
-   1. The `SpamActionService` must be executed _before_ the model is checked for errors and
+   1. The `check_for_spam` must be executed _before_ the model is checked for errors and
       attempting a `save`. If potential spam is detected in the model's changed attributes, we must prevent a save.
 
 ```ruby
 module Widget
   class CreateService < ::Widget::BaseService
-    # NOTE: We require the spam_params and do not default it to nil, because
-    # spam_checking is likely to be necessary.  However, if there is not a request available in scope
-    # in the caller (for example, a note created via email) and the required arguments to the
-    # SpamParams constructor are not otherwise available, spam_params: must be explicitly passed as nil.
-    def initialize(project:, current_user: nil, params: {}, spam_params:)
+    # NOTE: We add a default value of `true` for `perform_spam_check`, because spam checking is likely to be necessary.
+    def initialize(project:, current_user: nil, params: {}, perform_spam_check: true)
       super(project: project, current_user: current_user, params: params)
 
-      @spam_params = spam_params
+      @perform_spam_check = perform_spam_check
     end
 
     def execute
@@ -136,13 +120,7 @@ module Widget
 
       # NOTE: do this AFTER the spammable model is instantiated, but BEFORE
       # it is validated or saved.
-      Spam::SpamActionService.new(
-        spammable: widget,
-        spam_params: spam_params,
-        user: current_user,
-        # Or `action: :update` for a UpdateService or service for an existing model.
-        action: :create
-      ).execute
+      widget.check_for_spam(user: current_user, action: :create) if perform_spam_check
 
       # Possibly more code related to saving model, but should not change any attributes.
 
@@ -151,5 +129,5 @@ module Widget
 
     private
 
-    attr_reader :spam_params
+    attr_reader :perform_spam_check
 ```
diff --git a/doc/development/spam_protection_and_captcha/rest_api.md b/doc/development/spam_protection_and_captcha/rest_api.md
index d7012ffb418..76ffbc2f157 100644
--- a/doc/development/spam_protection_and_captcha/rest_api.md
+++ b/doc/development/spam_protection_and_captcha/rest_api.md
@@ -16,8 +16,8 @@ related to changing a model's confidential/public flag.
 The main steps are:
 
 1. Add `helpers SpammableActions::CaptchaCheck::RestApiActionsSupport` in your `resource`.
-1. Create a `spam_params` instance based on the request.
-1. Pass `spam_params` to the relevant Service class constructor.
+1. Pass `perform_spam_check: true` to the Update Service class constructor.
+   It is set to `true` by default in the Create Service.
 1. After you create or update the `Spammable` model instance, call `#check_spam_action_response!`,
    save the created or updated instance in a variable.
 1. Identify the error handling logic for the `failure` case of the request,
@@ -53,8 +53,7 @@ module API
 
       post do
         #...
-        spam_params = ::Spam::SpamParams.new_from_request(request: request)
-        service_response = ::Snippets::CreateService.new(project: nil, current_user: current_user, params: attrs, spam_params: spam_params).execute
+        service_response = ::Snippets::CreateService.new(project: nil, current_user: current_user, params: attrs).execute
         snippet = service_response.payload[:snippet]
 
         if service_response.success?
@@ -71,8 +70,7 @@ module API
 
       put ':id' do
         #...
-        spam_params = ::Spam::SpamParams.new_from_request(request: request)
-        service_response = ::Snippets::UpdateService.new(project: nil, current_user: current_user, params: attrs, spam_params: spam_params).execute(snippet)
+        service_response = ::Snippets::UpdateService.new(project: nil, current_user: current_user, params: attrs, perform_spam_check: true).execute(snippet)
 
         snippet = service_response.payload[:snippet]
 
diff --git a/doc/development/spam_protection_and_captcha/web_ui.md b/doc/development/spam_protection_and_captcha/web_ui.md
index 0ae5e98f399..0cc17854010 100644
--- a/doc/development/spam_protection_and_captcha/web_ui.md
+++ b/doc/development/spam_protection_and_captcha/web_ui.md
@@ -75,11 +75,9 @@ sequenceDiagram
 The backend is also cleanly abstracted via mixin modules and helper methods. The three main
 changes required to the relevant backend controller actions (normally just `create`/`update`) are:
 
-1. Create a `SpamParams` parameter object instance based on the request, using the static
-   `#new_from_request` factory method. This method takes a request, and returns a `SpamParams` instance.
-1. Pass the created `SpamParams` instance as the `spam_params` named argument to the
-   Service class constructor, which you should have already added. If the spam check indicates
-   the changes to the model are possibly spam, then:
+1. Pass `perform_spam_check: true` to the Update Service class constructor.
+   It is set to `true` by default in the Create Service.
+1. If the spam check indicates the changes to the model are possibly spam, then:
    - An error is added to the model.
    - The `needs_recaptcha` property on the model is set to true.
 1. Wrap the existing controller action return value (rendering or redirecting) in a block passed to
@@ -116,12 +114,10 @@ module WidgetsActions
   include SpammableActions::CaptchaCheck::JsonFormatActionsSupport
 
   def create
-    spam_params = ::Spam::SpamParams.new_from_request(request: request)
     widget = ::Widgets::CreateService.new(
       project: project,
       current_user: current_user,
-      params: params,
-      spam_params: spam_params
+      params: params
     ).execute
 
     respond_to do |format|
@@ -166,13 +162,11 @@ class WidgetsController < ApplicationController
 
   def update
     # Existing logic to find the `widget` model instance...
-
-    spam_params = ::Spam::SpamParams.new_from_request(request: request)
     ::Widgets::UpdateService.new(
       project: project,
       current_user: current_user,
       params: params,
-      spam_params: spam_params
+      perform_spam_check: true
     ).execute(widget)
 
     respond_to do |format|
diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md
index e257bddb900..90267a517b8 100644
--- a/doc/development/testing_guide/best_practices.md
+++ b/doc/development/testing_guide/best_practices.md
@@ -388,9 +388,10 @@ With [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/375983) we defined th
 
 For tests that are not meeting the thresholds it is recommended to create issues and improve the tests duration.
 
-| Date | Feature tests | Controllers and Requests tests | Other |
-| --- | --- | --- | --- |
-| 2023-02-15 | 67.42 seconds | 44.66 seconds | 76.86 seconds |
+| Date | Feature tests | Controllers and Requests tests | Unit | Other | Method |
+| :-: | :-: | :-: | :-: | :-: | :-: |
+| 2023-02-15 | 67.42 seconds | 44.66 seconds | - | 76.86 seconds | Top slow test eliminating the maximum |
+| 2023-06-15 | 50.13 seconds | 19.20 seconds | 27.12 | 45.40 seconds | Avg for top 100 slow tests|
 
 #### Avoid repeating expensive actions
 
@@ -1140,8 +1141,17 @@ variables example can be used, but avoid this if at all possible.
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61171) in GitLab 14.0.
 
 Specs that require Elasticsearch must be marked with the `:elastic` trait. This
-creates and deletes indices between examples to ensure a clean index, so that there is no room
-for polluting the tests with nonessential data.
+creates and deletes indices before and after all examples.
+
+The `:elastic_delete_by_query` trait was added to reduce runtime for pipelines by creating and deleting indices at the
+start and end of each context only. The [Elasticsearch delete by query API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html)
+is used to delete data in all indices between examples to ensure a clean index.
+
+The `:elastic_clean` trait creates and deletes indices between examples to ensure a clean index. This way, tests are not
+polluted with non-essential data. If using the `:elastic` or `:elastic_delete_by_query` trait
+is causing issues, use `:elastic_clean` instead. `:elastic_clean` is significantly slower than the other traits
+and should be used sparingly.
+
 Most tests for Elasticsearch logic relate to:
 
 - Creating data in PostgreSQL and waiting for it to be indexed in Elasticsearch.
@@ -1150,11 +1160,8 @@ Most tests for Elasticsearch logic relate to:
 
 There are some exceptions, such as checking for structural changes rather than individual records in an index.
 
-The `:elastic_delete_by_query` trait was added to reduce run time for pipelines by creating and deleting indices
-at the start and end of each context only. The [Elasticsearch DeleteByQuery API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html)
-is used to delete data in all indices in between examples to ensure a clean index.
-
-Note that Elasticsearch indexing uses [`Gitlab::Redis::SharedState`](../../../ee/development/redis.md#gitlabrediscachesharedstatequeues).
+NOTE:
+Elasticsearch indexing uses [`Gitlab::Redis::SharedState`](../../../ee/development/redis.md#gitlabrediscachesharedstatequeues).
 Therefore, the Elasticsearch traits dynamically use the `:clean_gitlab_redis_shared_state` trait.
 You do not need to add `:clean_gitlab_redis_shared_state` manually.
 
diff --git a/doc/development/testing_guide/end_to_end/capybara_to_chemlab_migration_guide.md b/doc/development/testing_guide/end_to_end/capybara_to_chemlab_migration_guide.md
index f3911176263..7bac76c88e8 100644
--- a/doc/development/testing_guide/end_to_end/capybara_to_chemlab_migration_guide.md
+++ b/doc/development/testing_guide/end_to_end/capybara_to_chemlab_migration_guide.md
@@ -13,21 +13,21 @@ Given the view:
 ```html
 <form id="my-form">
     <label for="first-name">First name</label>
-    <input type="text" name="first-name" data-qa-selector="first_name" />
+    <input type="text" name="first-name" data-testid="first_name" />
 
     <label for="last-name">Last name</label>
-    <input type="text" name="last-name" data-qa-selector="last_name" />
+    <input type="text" name="last-name" data-testid="last_name" />
 
     <label for="company-name">Company name</label>
-    <input type="text" name="company-name" data-qa-selector="company_name" />
+    <input type="text" name="company-name" data-testid="company_name" />
 
     <label for="user-name">User name</label>
-    <input type="text" name="user-name" data-qa-selector="user_name" />
+    <input type="text" name="user-name" data-testid="user_name" />
 
     <label for="password">Password</label>
-    <input type="password" name="password" data-qa-selector="password" />
+    <input type="password" name="password" data-testid="password" />
 
-    <input type="submit" value="Continue" data-qa-selector="continue"/>
+    <input type="submit" value="Continue" data-testid="continue"/>
 </form>
 ```
 
@@ -137,12 +137,12 @@ Since the element type is preserved within the Page Library, there is no need to
 
 ```html
 <!-- Before -->
-<input type="text" name="first-name" data-qa-selector="first_name_field" />
-<input type="submit" name="continue" value="Continue" data-qa-selector="continue_button" />
+<input type="text" name="first-name" data-testid="first_name_field" />
+<input type="submit" name="continue" value="Continue" data-testid="continue_button" />
 
 <!-- After -->
-<input type="text" name="first-name" data-qa-selector="first_name" />
-<input type="submit" name="continue" value="Continue" data-qa-selector="continue" />
+<input type="text" name="first-name" data-testid="first_name" />
+<input type="submit" name="continue" value="Continue" data-testid="continue" />
 ```
 
 This makes it much easier for Developers to write tests and contributes to testability since we can write the Page Library while we look at the UI.
diff --git a/doc/development/testing_guide/end_to_end/execution_context_selection.md b/doc/development/testing_guide/end_to_end/execution_context_selection.md
index 4d83884f4d0..0082bb6ee23 100644
--- a/doc/development/testing_guide/end_to_end/execution_context_selection.md
+++ b/doc/development/testing_guide/end_to_end/execution_context_selection.md
@@ -44,8 +44,8 @@ Matches use:
 | `gitlab.com and staging.gitlab.com`      | `only: { subdomain: /(staging.)?/, domain: 'gitlab' }` | `(staging.)?gitlab.com` |
 | `dev.gitlab.org`                         | `only: { tld: '.org', domain: 'gitlab', subdomain: 'dev' }` | `(dev).gitlab.org` |
 | `staging.gitlab.com and domain.gitlab.com` | `only: { subdomain: %i[staging domain] }` | `(staging\|domain).+.com`             |
-| The `nightly` pipeline                     | `only: { pipeline: :nightly }` | "nightly" |
-| The `nightly` and `canary` pipelines | `only: { pipeline: [:nightly, :canary] }` | ["nightly"](https://gitlab.com/gitlab-org/quality/nightly) and ["canary"](https://gitlab.com/gitlab-org/quality/canary) |
+| The `nightly` pipeline                     | `only: { pipeline: :nightly }` | ["nightly scheduled pipeline"](https://gitlab.com/gitlab-org/gitlab/-/pipeline_schedules) |
+| The `nightly` and `canary` pipelines | `only: { pipeline: [:nightly, :canary] }` | ["nightly scheduled pipeline"](https://gitlab.com/gitlab-org/gitlab/-/pipeline_schedules) and ["canary"](https://gitlab.com/gitlab-org/quality/canary) |
 | The `ee:instance` job | `only: { job: 'ee:instance' }` | The `ee:instance` job in any pipeline |
 | Any `quarantine` job | `only: { job: '.*quarantine' }` | Any job ending in `quarantine` in any pipeline |
 | Any run where condition evaluates to a truthy value | `only: { condition: -> { ENV['TEST_ENV'] == 'true' } }` | Any run where `TEST_ENV` is set to true
@@ -87,8 +87,8 @@ Matches use:
 | `gitlab.com and staging.gitlab.com`      | `except: { subdomain: /(staging.)?/, domain: 'gitlab' }` | `(staging.)?gitlab.com` |
 | `dev.gitlab.org`                         | `except: { tld: '.org', domain: 'gitlab', subdomain: 'dev' }` | `(dev).gitlab.org` |
 | `staging.gitlab.com and domain.gitlab.com` | `except: { subdomain: %i[staging domain] }` | `(staging\|domain).+.com`             |
-| The `nightly` pipeline                     | `except: { pipeline: :nightly }` | ["nightly"](https://gitlab.com/gitlab-org/quality/nightly) |
-| The `nightly` and `canary` pipelines | `except: { pipeline: [:nightly, :canary] }` | ["nightly"](https://gitlab.com/gitlab-org/quality/nightly) and ["canary"](https://gitlab.com/gitlab-org/quality/canary) |
+| The `nightly` pipeline                     | `only: { pipeline: :nightly }` | ["nightly scheduled pipeline"](https://gitlab.com/gitlab-org/gitlab/-/pipeline_schedules) |
+| The `nightly` and `canary` pipelines | `only: { pipeline: [:nightly, :canary] }` | ["nightly scheduled pipeline"](https://gitlab.com/gitlab-org/gitlab/-/pipeline_schedules) and ["canary"](https://gitlab.com/gitlab-org/quality/canary) |
 | The `ee:instance` job | `except: { job: 'ee:instance' }` | The `ee:instance` job in any pipeline |
 | Any `quarantine` job | `except: { job: '.*quarantine' }` | Any job ending in `quarantine` in any pipeline |
 | Any run except where condition evaluates to a truthy value | `except: { condition: -> { ENV['TEST_ENV'] == 'true' } }` | Any run where `TEST_ENV` is not set to true
diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md
index 77ceb9fa546..adf679c44a2 100644
--- a/doc/development/testing_guide/end_to_end/index.md
+++ b/doc/development/testing_guide/end_to_end/index.md
@@ -21,8 +21,8 @@ using the [GitLab QA orchestrator](https://gitlab.com/gitlab-org/gitlab-qa) tool
 ### Testing nightly builds
 
 We run scheduled pipelines each night to test nightly builds created by Omnibus.
-You can find these pipelines at <https://gitlab.com/gitlab-org/quality/nightly/pipelines>
-(requires the Developer role). Results are reported in the `#qa-nightly` Slack channel.
+You can find these pipelines at <https://gitlab.com/gitlab-org/gitlab/-/pipeline_schedules>
+(requires the Developer role). Results are reported in the `#qa-master` Slack channel.
 
 ### Testing staging
 
@@ -97,8 +97,8 @@ This problem was discovered in <https://gitlab.com/gitlab-org/gitlab-qa/-/issues
 work-around was suggested in <https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4717>.
 A feature proposal to segregate access control regarding running pipelines from ability to push/merge was also created at <https://gitlab.com/gitlab-org/gitlab/-/issues/24585>.
 
-For more technical details on CI/CD setup and documentation on adding new test jobs to `package-and-test` pipeline, see
-[`package_and_test` setup documentation](package_and_test_pipeline.md).
+For more technical details on CI/CD setup and documentation on adding new test jobs to `e2e:package-and-test` pipeline, see
+[`e2e:package_and_test` setup documentation](package_and_test_pipeline.md).
 
 #### With merged results pipelines
 
@@ -165,8 +165,8 @@ In order to limit amount of tests executed in a merge request, dynamic selection
 on changed files and merge request labels. Following criteria determine which tests will run:
 
 1. Changes in `qa` framework code would execute the full suite
-1. Changes in particular `_spec.rb` file in `qa` folder would execute only that particular test
-1. Merge request with backend changes and label `devops::manage` would execute all e2e tests related to `manage` stage
+1. Changes in particular `_spec.rb` file in `qa` folder would execute only that particular test. In this case knapsack will not be used to run jobs in parallel.
+1. Merge request with backend changes and label `devops::manage` would execute all e2e tests related to `manage` stage. Jobs will be run in parallel in this case using knapsack.
 
 #### Overriding selective test execution
 
@@ -198,7 +198,7 @@ Use these environment variables to configure metrics export:
 | -------- | -------- | ----------- |
 | `QA_INFLUXDB_URL` | `true` | Should be set to `https://influxdb.quality.gitlab.net`. No default value. |
 | `QA_INFLUXDB_TOKEN` | `true` | InfluxDB write token that can be found under `Influxdb auth tokens` document in `Gitlab-QA` `1Password` vault. No default value. |
-| `QA_RUN_TYPE` | `false` | Arbitrary name for test execution, like `package-and-test`. Automatically inferred from the project name for live environment test executions. No default value. |
+| `QA_RUN_TYPE` | `false` | Arbitrary name for test execution, like `e2e:package-and-test`. Automatically inferred from the project name for live environment test executions. No default value. |
 | `QA_EXPORT_TEST_METRICS` | `false` | Flag to enable or disable metrics export to InfluxDB. Defaults to `false`. |
 | `QA_SAVE_TEST_METRICS` | `false` | Flag to enable or disable saving metrics as JSON file. Defaults to `false`. |
 
diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md
index 6a599ce9a50..3e13aaa4f02 100644
--- a/doc/development/testing_guide/end_to_end/page_objects.md
+++ b/doc/development/testing_guide/end_to_end/page_objects.md
@@ -98,7 +98,7 @@ end
 The `view` DSL method corresponds to the Rails view, partial, or Vue component that renders the elements.
 
 The `element` DSL method in turn declares an element for which a corresponding
-`data-qa-selector=element_name_snaked` data attribute must be added to the view file.
+`testid=element_name` data attribute must be added, if not already, to the view file.
 
 You can also define a value (String or Regexp) to match to the actual view
 code but **this is deprecated** in favor of the above method for two reasons:
@@ -112,7 +112,7 @@ view 'app/views/my/view.html.haml' do
 
   ### Good ###
 
-  # Implicitly require the CSS selector `[data-qa-selector="logout_button"]` to be present in the view
+  # Implicitly require the CSS selector `[data-testid="logout_button"]` to be present in the view
   element :logout_button
 
   ### Bad ###
@@ -140,38 +140,38 @@ view 'app/views/my/view.html.haml' do
 end
 ```
 
-To add these elements to the view, you must change the Rails view, partial, or Vue component by adding a `data-qa-selector` attribute
+To add these elements to the view, you must change the Rails view, partial, or Vue component by adding a `data-testid` attribute
 for each element defined.
 
-In our case, `data-qa-selector="login_field"`, `data-qa-selector="password_field"` and `data-qa-selector="sign_in_button"`
+In our case, `data-testid="login_field"`, `data-testid="password_field"` and `data-testid="sign_in_button"`
 
 `app/views/my/view.html.haml`
 
 ```haml
-= f.text_field :login, class: "form-control top", autofocus: "autofocus", autocapitalize: "off", autocorrect: "off", required: true, title: "This field is required.", data: { qa_selector: 'login_field' }
-= f.password_field :password, class: "form-control bottom", required: true, title: "This field is required.", data: { qa_selector: 'password_field' }
-= f.submit "Sign in", class: "btn btn-confirm", data: { qa_selector: 'sign_in_button' }
+= f.text_field :login, class: "form-control top", autofocus: "autofocus", autocapitalize: "off", autocorrect: "off", required: true, title: "This field is required.", data: { testid: 'login_field' }
+= f.password_field :password, class: "form-control bottom", required: true, title: "This field is required.", data: { testid: 'password_field' }
+= f.submit "Sign in", class: "btn btn-confirm", data: { testid: 'sign_in_button' }
 ```
 
 Things to note:
 
-- The name of the element and the `qa_selector` must match and be snake cased
+- The name of the element and the `data-testid` must match and be either snake cased or kebab cased
 - If the element appears on the page unconditionally, add `required: true` to the element. See
   [Dynamic element validation](dynamic_element_validation.md)
-- You may see `.qa-selector` classes in existing Page Objects. We should prefer the [`data-qa-selector`](#data-qa-selector-vs-qa-selector)
-  method of definition over the `.qa-selector` CSS class
+- You may see `data-qa-selector` classes in existing Page Objects. We should prefer the [`data-testid`](#data-testid-vs-data-qa-selector)
+  method of definition over the `data-qa-selector` CSS class
 
-### `data-qa-selector` vs `.qa-selector`
+### `data-testid` vs `data-qa-selector`
 
-> Introduced in GitLab 12.1
+> Introduced in GitLab 16.1
 
 There are two supported methods of defining elements within a view.
 
+1. `data-testid`
 1. `data-qa-selector` attribute
-1. `.qa-selector` class
 
-Any existing `.qa-selector` class should be considered deprecated
-and we should prefer the `data-qa-selector` method of definition.
+Any existing `data-qa-selector` class should be considered deprecated
+and we should prefer the `data-testid` method of definition.
 
 ### Dynamic element selection
 
@@ -193,7 +193,7 @@ Given the following Rails view (using GitLab Issues as an example):
 ```haml
 %ul.issues-list
  - @issues.each do |issue|
-   %li.issue{data: { qa_selector: 'issue', qa_issue_title: issue.title } }= link_to issue
+   %li.issue{data: { testid: 'issue', qa_issue_title: issue.title } }= link_to issue
 ```
 
 We can select on that specific issue by matching on the Rails model.
@@ -225,7 +225,7 @@ end
 ```haml
 %ol
   - @some_model.each_with_index do |model, idx|
-    %li.model{ data: { qa_selector: 'model', qa_index: idx } }
+    %li.model{ data: { testid: 'model', qa_index: idx } }
 ```
 
 ```ruby
diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
index a42d5e3df1d..b698eb53f75 100644
--- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
+++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
@@ -35,8 +35,7 @@ This is a partial list of the [RSpec metadata](https://rspec.info/features/3-12/
 | `:mixed_env`                | The test should only be executed in environments that have a paired canary version available through traffic routing based on the existence of the `gitlab_canary=true` cookie. Tests in this category are switching the cookie mid-test to validate mixed deployment environments.                                                                                                                                                                                                                          |
 | `:object_storage`           | The test requires a GitLab instance to be configured to use multiple [object storage types](../../../administration/object_storage.md). Uses MinIO as the object storage server.                                                                                                                                                                                                                                                                                                                             |
 | `:only`                     | The test is only to be run in specific execution contexts. See [test execution context selection](execution_context_selection.md) for more information.                                                                                                                                                                                                                                                                                                                                                      |
-| `:orchestrated`             | The GitLab instance under test may be [configured by `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#orchestrated-tests) to be different to the default GitLab configuration, or `gitlab-qa` may launch additional services in separate Docker containers, or both. Tests tagged with `:orchestrated` are excluded when testing environments where we can't dynamically modify the GitLab configuration (for example, Staging).                             |
-| `:packages`                 | The test requires a GitLab instance that has the [Package Registry](../../../administration/packages/index.md#gitlab-package-registry-administration) enabled.                                                                                                                                                                                                                                                                                                                                                       |
+| `:orchestrated`             | The GitLab instance under test may be [configured by `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#orchestrated-tests) to be different to the default GitLab configuration, or `gitlab-qa` may launch additional services in separate Docker containers, or both. Tests tagged with `:orchestrated` are excluded when testing environments where we can't dynamically modify the GitLab configuration (for example, Staging).                             |                                                                       |
 | `:product_group`            | Specifies what product group the test belongs to. See [Product sections, stages, groups, and categories](https://about.gitlab.com/handbook/product/categories/) for the comprehensive groups list.                                                                                                                                                                                                                                                                                                            |
 | `:quarantine`               | The test has been [quarantined](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/debugging-qa-test-failures/#quarantining-tests), runs in a separate job that only includes quarantined tests, and is allowed to fail. The test is skipped in its regular job so that if it fails it doesn't hold up the pipeline. Note that you can also [quarantine a test only when it runs in a specific context](execution_context_selection.md#quarantine-a-test-for-a-specific-environment). |
 | `:relative_url`             | The test requires a GitLab instance to be installed under a [relative URL](../../../install/relative_url.md).                                                                                                                                                                                                                                                                                                                                                                                                |
diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
index adf7bccb7fb..0544b331b1a 100644
--- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
+++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md
@@ -428,8 +428,6 @@ For instructions on how to run these tests using the `gitlab-qa` gem, please ref
 
 Tests that are tagged with `:mobile` can be run against specified mobile devices using cloud emulator/simulator services.
 
-These tests run in the [nightly pipeline](https://gitlab.com/gitlab-org/quality/nightly/-/pipelines) in the `ce:remote_mobile_safari` job.
-
 ### How to run mobile tests with Sauce Labs
 
 Running directly against an environment like staging is not recommended because Sauce Labs test logs expose credentials. Therefore, it is best practice and the default to use a tunnel.
@@ -587,18 +585,18 @@ You can verify whether GitLab is appropriately redirecting your session to the `
 
 The above spec is verbose, written specifically this way to ensure the idea behind the implementation is clear. We recommend following the practices detailed within our [Beginner's guide to writing end-to-end tests](beginners_guide.md).
 
-## OpenID Connect (OIDC) tests
+## Tests for GitLab as OpenID Connect (OIDC) and OAuth provider
 
-To run the [`login_via_oidc_with_gitlab_as_idp_spec`](https://gitlab.com/gitlab-org/gitlab/-/blob/188e2c876a17a097448d7f3ed35bdf264fed0d3b/qa/qa/specs/features/browser_ui/1_manage/login/login_via_oidc_with_gitlab_as_idp_spec.rb) on your local machine:
+To run the [`login_via_oauth_and_oidc_with_gitlab_as_idp_spec`](https://gitlab.com/gitlab-org/gitlab/-/blob/2e2c8bcfa4f68cd39041806af531038ce4d2ab04/qa/qa/specs/features/browser_ui/1_manage/login/login_via_oauth_and_oidc_with_gitlab_as_idp_spec.rb) on your local machine:
 
 1. Make sure your GDK is set to run on a non-localhost address such as `gdk.test:3000`.
 1. Configure a [loopback interface to 172.16.123.1](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/6fe7b46403229f12ab6d903f99b024e0b82cb94a/doc/howto/local_network.md#create-loopback-interface).
 1. Make sure Docker Desktop or Rancher Desktop is running.
-1. Add an entry to your `/etc/hosts` file for `gitlab-oidc-consumer.bridge` pointing to `127.0.0.1`.
+1. Add an entry to your `/etc/hosts` file for `gitlab-oidc-consumer.bridge` and `gitlab-oauth-consumer.bridge` pointing to `127.0.0.1`.
 1. From the `qa` directory, run the following command. To set the GitLab image you want to use, update the `RELEASE` variable. For example, to use the latest EE image, set `RELEASE` to `gitlab/gitlab-ee:latest`:
 
    ```shell
    bundle install
 
-   RELEASE_REGISTRY_URL='registry.gitlab.com' RELEASE_REGISTRY_USERNAME='<your_gitlab_username>' RELEASE_REGISTRY_PASSWORD='<your_gitlab_personal_access_token>' RELEASE='registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:1d5a644145dfe901ea7648d825f8f9f3006d0acf' GITLAB_QA_ADMIN_ACCESS_TOKEN="<your_gdk_admin_personal_access_token>" QA_DEBUG=true CHROME_HEADLESS=false bundle exec bin/qa Test::Instance::All http://gdk.test:3000 qa/specs/features/browser_ui/1_manage/login/login_via_oidc_with_gitlab_as_idp_spec.rb
+   RELEASE_REGISTRY_URL='registry.gitlab.com' RELEASE_REGISTRY_USERNAME='<your_gitlab_username>' RELEASE_REGISTRY_PASSWORD='<your_gitlab_personal_access_token>' RELEASE='registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:c0ae46db6b31ea231b2de88961cd687acf634179' GITLAB_QA_ADMIN_ACCESS_TOKEN="<your_gdk_admin_personal_access_token>" QA_DEBUG=true CHROME_HEADLESS=false bundle exec bin/qa Test::Instance::All http://gdk.test:3000 qa/specs/features/browser_ui/1_manage/login/login_via_oauth_and_oidc_with_gitlab_as_idp_spec.rb
    ```
diff --git a/doc/development/testing_guide/end_to_end/test_infrastructure.md b/doc/development/testing_guide/end_to_end/test_infrastructure.md
new file mode 100644
index 00000000000..00d4507d35e
--- /dev/null
+++ b/doc/development/testing_guide/end_to_end/test_infrastructure.md
@@ -0,0 +1,32 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# End-to-end Testing Infrastructure for Cloud Integrations
+
+This content is about infrastructure we integrate with GitLab QA test scenarios, at the end-to-end level.
+
+## What infrastructure do we have in place?
+
+We currently use GCP and AWS platforms to test a few end-to-end scenarios. These are separated from other
+sandbox projects to prevent accidental deletion of resources that run beyond automated test runs. If you do not have
+access to these accounts already, you can create an access request. In GCP, we use `group-qa-tests-566cc6`
+and in AWS, the cloud account `eng-quality-ops-ci-cd-shared-infra-498dbd5a`.
+
+If you have test scenarios that require these platforms, we encourage you to use the existing infrastructure and
+accounts so we can efficiently consolidate and maintain our end-to-end test suite.
+
+## Why do we have this infrastructure?
+
+GitLab has several features that integrate well with known Cloud Providers. To fully test this integration,
+we have infrastructure in place that connects GitLab QA with these providers.
+
+We currently use GCP for its Cloud Storage resources to test Object Storage (GCS) and to create Kubernetes clusters. We also use AWS to test Object Storage (S3).
+
+## How do we maintain this infrastructure?
+
+We have an active [Janitor](https://gitlab.com/gitlab-com/gl-infra/gitlab-gcp-janitor) project that ensures resources in GCP are cleaned up if a test fails to remove it. The Janitor jobs run daily on a scheduled pipeline and target exclusively GCP `group-qa-tests-566cc6`.
+
+AWS uses lifecycle management rules to delete objects after 1 day. We have an established [process](https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbooks/rotating-credentials.md) that enables anyone with access to these environments to rotate credentials.
diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md
index 40be7a7d094..9114ec3d179 100644
--- a/doc/development/testing_guide/flaky_tests.md
+++ b/doc/development/testing_guide/flaky_tests.md
@@ -13,62 +13,48 @@ eventually.
 
 ## What are the potential cause for a test to be flaky?
 
-### Unclean environment
+### State leak
 
-**Label:** `flaky-test::unclean environment`
+**Label:** `flaky-test::state leak`
 
-**Description:** The environment got dirtied by a previous test. The actual cause is probably not the flaky test here.
+**Description:** Data state has leaked from a previous test. The actual cause is probably not the flaky test here.
 
 **Difficulty to reproduce:** Moderate. Usually, running the same spec files until the one that's failing reproduces the problem.
 
-**Resolution:** Fix the previous tests and/or places where the environment is modified, so that
+**Resolution:** Fix the previous tests and/or places where the test data or environment is modified, so that
 it's reset to a pristine test after each test.
 
 **Examples:**
 
-- [Example 1](https://gitlab.com/gitlab-org/gitlab/-/issues/378414#note_1142026988): A migration
+- [Example 1](https://gitlab.com/gitlab-org/gitlab/-/issues/402915): State leakage can result from
+  data records created with `let_it_be` shared between test examples, while some test modifies the model
+  either deliberately or unwillingly causing out-of-sync data in test examples. This can result in `PG::QueryCanceled: ERROR` in the subsequent test examples or retries.
+  For more information about state leakages and resolution options, see [GitLab testing best practices](best_practices.md#lets-talk-about-let).
+- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/issues/378414#note_1142026988): A migration
   test might roll-back the database, perform its testing, and then roll-up the database in an
   inconsistent state, so that following tests might not know about certain columns.
-- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/issues/368500): A test modifies data that is
+- [Example 3](https://gitlab.com/gitlab-org/gitlab/-/issues/368500): A test modifies data that is
   used by a following test.
-- [Example 3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103434#note_1172316521): A test for a database query passes in a fresh database, but in a
+- [Example 4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103434#note_1172316521): A test for a database query passes in a fresh database, but in a
   CI/CD pipeline where the database is used to process previous test sequences, the test fails. This likely
-    means that the query itself needs to be updated to work in a non-clean database.
-
-### Ordering assertion
-
-**Label:** `flaky-test::ordering assertion`
-
-**Description:** The test is expecting a specific order in the data under test yet the data is in
-a non-deterministic order.
-
-**Difficulty to reproduce:** Easy. Usually, running the test locally several times would reproduce
-the problem.
-
-**Resolution:** Depending on the problem, you might want to:
-
-- loosen the assertion if the test shouldn't care about ordering but only on the elements
-- fix the test by specifying a deterministic ordering
-- fix the app code by specifying a deterministic ordering
-
-**Examples:**
-
-- [Example 1](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10148/diffs): Without
-  specifying `ORDER BY`, database will not give deterministic ordering, or data race happening
-  in the tests.
-- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106936/diffs).
+  means that the query itself needs to be updated to work in a non-clean database.
 
 ### Dataset-specific
 
 **Label:** `flaky-test::dataset-specific`
 
-**Description:** The test assumes the dataset is in a particular (usually limited) state, which
+**Description:** The test assumes the dataset is in a particular (usually limited) state or order, which
 might not be true depending on when the test run during the test suite.
 
 **Difficulty to reproduce:** Moderate, as the amount of data needed to reproduce the issue might be
-difficult to achieve locally.
+difficult to achieve locally. Ordering issues are easier to reproduce by repeatedly running the tests several times.
 
-**Resolution:** Fix the test to not assume that the dataset is in a particular state, don't hardcode IDs.
+**Resolution:**
+
+- Fix the test to not assume that the dataset is in a particular state, don't hardcode IDs.
+- Loosen the assertion if the test shouldn't care about ordering but only on the elements.
+- Fix the test by specifying a deterministic ordering.
+- Fix the app code by specifying a deterministic ordering.
 
 **Examples:**
 
@@ -81,11 +67,10 @@ difficult to achieve locally.
   suite, it might pass as not enough records were created before it, but as soon as it would run
   later in the suite, there could be a record that actually has the ID `42`, hence the test would
   start to fail.
-- [Example 3](https://gitlab.com/gitlab-org/gitlab/-/issues/402915): State leakage can result from
-  data records created with `let_it_be` shared between test examples, while some test modifies the model
-  either deliberately or unwillingly causing out-of-sync data in test examples. This can result in `PG::QueryCanceled: ERROR` in the subsequent test examples or retries.
-  For more information about state leakages and resolution options,
-  see [GitLab testing best practices](best_practices.md#lets-talk-about-let).
+- [Example 3](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10148/diffs): Without
+  specifying `ORDER BY`, database is not given deterministic ordering, or data race can happen
+  in the tests.
+- [Example 4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106936/diffs).
 
 ### Random input
 
diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md
index d596c21bd5c..b1fde5ed139 100644
--- a/doc/development/testing_guide/frontend_testing.md
+++ b/doc/development/testing_guide/frontend_testing.md
@@ -218,7 +218,6 @@ it('exists', () => {
   wrapper.find('.js-foo');
   wrapper.find('.btn-primary');
   wrapper.find('.qa-foo-component');
-  wrapper.find('[data-qa-selector="foo"]');
 });
 ```
 
@@ -226,7 +225,7 @@ It is recommended to use `kebab-case` for `data-testid` attribute.
 
 It is not recommended that you add `.js-*` classes just for testing purposes. Only do this if there are no other feasible options available.
 
-Do not use a `.qa-*` class or `data-qa-selector` attribute for any tests other than QA end-to-end testing.
+Do not use `.qa-*` class attributes for any tests other than QA end-to-end testing.
 
 ### Querying for child components
 
diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md
index 5008564de01..38fda1bb742 100644
--- a/doc/development/testing_guide/index.md
+++ b/doc/development/testing_guide/index.md
@@ -79,4 +79,8 @@ Everything you should know about how to test migrations.
 
 Introduction to contract testing, how to run the tests, and how to write them.
 
+## [Test results tracking](test_results_tracking.md)
+
+How we track our test suite run results.
+
 [Return to Development documentation](../index.md)
diff --git a/doc/development/testing_guide/test_results_tracking.md b/doc/development/testing_guide/test_results_tracking.md
new file mode 100644
index 00000000000..28407e806a3
--- /dev/null
+++ b/doc/development/testing_guide/test_results_tracking.md
@@ -0,0 +1,29 @@
+---
+stage: none
+group: unassigned
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Test results tracking
+
+We developed the [`gitlab_quality-test_tooling`](https://gitlab.com/gitlab-org/ruby/gems/gitlab_quality-test_tooling) gem that includes several commands to automate test results tracking.
+
+The goal of this gem is to have a consolidated set of tooling that we use accross our various test suite (e.g. GitLab Rails & E2E test suites).
+
+The initial motivation and development was tracked by [this epic](https://gitlab.com/groups/gitlab-org/-/epics/10536).
+
+## Rails test results tracking
+
+We [plan to use](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/122008) the `relate-failure-issue` command from the gem (see the gem's README for details about the command).
+
+## End-to-end test results tracking
+
+This is described specifically at <https://about.gitlab.com/handbook/engineering/quality/#test-results-tracking>.
+
+For the E2E test suite, we use the following commands from the gem (see the gem's README for details about each command):
+
+- `prepare-stage-reports`
+- `generate-test-session`
+- `report-results`
+- `update-screenshot-paths`
+- `relate-failure-issue`
diff --git a/doc/development/uploads/index.md b/doc/development/uploads/index.md
index a62e8ea2d58..92b52d2b59c 100644
--- a/doc/development/uploads/index.md
+++ b/doc/development/uploads/index.md
@@ -1,6 +1,6 @@
 ---
-stage: none
-group: unassigned
+stage: SaaS Platforms
+group: Scalability
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
 ---
 
diff --git a/doc/development/uploads/working_with_uploads.md b/doc/development/uploads/working_with_uploads.md
index 5575297ad6b..e487f2a19d3 100644
--- a/doc/development/uploads/working_with_uploads.md
+++ b/doc/development/uploads/working_with_uploads.md
@@ -1,6 +1,6 @@
 ---
-stage: none
-group: unassigned
+stage: SaaS Platforms
+group: Scalability
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
 ---
 
diff --git a/doc/development/value_stream_analytics.md b/doc/development/value_stream_analytics.md
index afbd1a76a1b..e056ce9b064 100644
--- a/doc/development/value_stream_analytics.md
+++ b/doc/development/value_stream_analytics.md
@@ -365,7 +365,7 @@ Seed issues and merge requests for value stream analytics:
 
 Seed DORA daily metrics for value stream, insights and CI/CD analytics:
 
-1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project.
 1. On the project's homepage, in the upper-left corner, copy the **Project ID**. You need it in a later step.
 1. [Create an environment for your selected project from the UI](../ci/environments/index.md#create-a-static-environment) named `production`.
 1. Open the rails console:
diff --git a/doc/development/vs_code_debugging.md b/doc/development/vs_code_debugging.md
index 08aa4688bfd..129eddf853b 100644
--- a/doc/development/vs_code_debugging.md
+++ b/doc/development/vs_code_debugging.md
@@ -6,12 +6,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # VS Code debugging
 
-This document describes how to set up Rails debugging in [VS Code](https://code.visualstudio.com/).
+This document describes how to set up Rails debugging in [Visual Studio Code (VSCode)](https://code.visualstudio.com/) using the [GitLab Development Kit (GDK)](contributing/first_contribution.md#step-1-configure-the-gitlab-development-kit).
 
 ## Setup
 
 1. Install the `debug` gem by running `gem install debug` inside your `gitlab` folder.
-1. Add the following configuration to your `.vscode/tasks.json` file:
+1. Install the [VSCode Ruby rdbg Debugger](https://marketplace.visualstudio.com/items?itemName=KoichiSasada.vscode-rdbg) extension to add support for the `rdbg` debugger type to VSCode.
+1. In case you want to automatically stop and start GitLab and its associated Ruby Rails server, you may add the following VSCode task to your configuration under the `.vscode/tasks.json` file:
 
     ```json
     {
@@ -20,7 +21,7 @@ This document describes how to set up Rails debugging in [VS Code](https://code.
           {
             "label": "start rdbg",
             "type": "shell",
-            "command": "gdk stop rails-web && GITLAB_RAILS_RACK_TIMEOUT_ENABLE_LOGGING=false PUMA_SINGLE_MODE=true rdbg --open -c -- bin/rails s",
+            "command": "gdk stop rails-web && GITLAB_RAILS_RACK_TIMEOUT_ENABLE_LOGGING=false PUMA_SINGLE_MODE=true rdbg --open -c bin/rails server",
             "isBackground": true,
             "problemMatcher": {
               "owner": "rails",
@@ -42,26 +43,29 @@ This document describes how to set up Rails debugging in [VS Code](https://code.
 
     ```json
     {
-        // Use IntelliSense to learn about possible attributes.
-        // Hover to view descriptions of existing attributes.
-        // For more information, see https://go.microsoft.com/fwlink/?linkid=830387.
         "version": "0.2.0",
         "configurations": [
           {
             "type": "rdbg",
             "name": "Attach with rdbg",
             "request": "attach",
+
+            // remove the following "preLaunchTask" if you do not wish to stop and start
+            // GitLab via VS Code but manually on a separate terminal.
             "preLaunchTask": "start rdbg"
           }
         ]
     }
     ```
 
+WARNING:
+The VSCode Ruby extension might have issues finding the correct Ruby installation and the appropriate `rdbg` command. In this case, add `"rdbgPath": "/home/user/.asdf/shims/` (in the case of asdf) to the launch configuration above.
+
 ## Debugging
 
-Prerequisite:
+### Prerequisites
 
-- You must have a running GDK instance.
+- You must have a running [GDK](contributing/first_contribution.md#step-1-configure-the-gitlab-development-kit) instance.
 
 To start debugging, do one of the following:
 
diff --git a/doc/development/work_items_widgets.md b/doc/development/work_items_widgets.md
index bafceccdafe..6c453f49db1 100644
--- a/doc/development/work_items_widgets.md
+++ b/doc/development/work_items_widgets.md
@@ -146,18 +146,24 @@ You can update widgets using custom fine-grained mutations (for example, `WorkIt
 ### Widget callbacks
 
 When updating the widget together with the work item's mutation, backend code should be implemented using
-callback classes that inherit from `Issuable::Callbacks::Base`. These classes have callback methods
+callback classes that inherit from `WorkItems::Callbacks::Base`. These classes have callback methods
 that are named similar to ActiveRecord callbacks and behave similarly.
 
-Callback classes with the same name as the widget are automatically used. For example, `Issuable::Callbacks::Milestone`
-is called when the work item has the milestone widget. To use a different class, you can override the `callback_class`
+Callback classes with the same name as the widget are automatically used. For example, `WorkItems::Callbacks::AwardEmoji`
+is called when the work item has the `AwardEmoji` widget. To use a different class, you can override the `callback_class`
 class method.
 
+When a callback class is also used for other issuables like merge requests or epics, define the class under `Issuable::Callbacks`
+and add the class to the list in `IssuableBaseService#available_callbacks`. These are executed for both work item updates and
+legacy issue, merge request, or epic updates.
+
 #### Available callbacks
 
 - `after_initialize` is called after the work item is initialized by the `BuildService` and before
   the work item is saved by the `CreateService` and `UpdateService`. This callback runs outside the
   creation or update database transaction.
+- `before_update` is called before the work item is saved by the `UpdateService`. This callback runs
+  within the update database transaction.
 - `after_update_commit` is called after the DB update transaction is committed by the `UpdateService`.
 - `after_save_commit` is called after the creation or DB update transaction is committed by the
   `CreateService` or `UpdateService`.
diff --git a/doc/development/workspace/index.md b/doc/development/workspace/index.md
deleted file mode 100644
index ca404702d72..00000000000
--- a/doc/development/workspace/index.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../organization/index.md'
-remove_date: '2023-06-13'
----
-
-This document was moved to [another location](../organization/index.md).
-
-<!-- This redirect file can be deleted after <2023-06-13>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->