Add latest changes from gitlab-org/gitlab@15-1-stable-eev15.1.0-rc42

6 files changed, 146 insertions, 17 deletions

diff --git a/.gitlab/issue_templates/Deprecations.md b/.gitlab/issue_templates/Deprecations.md
index 2e48c272316..3dfed1a1fc1 100644
--- a/.gitlab/issue_templates/Deprecations.md
+++ b/.gitlab/issue_templates/Deprecations.md
@@ -47,7 +47,7 @@ Please add links to the relevant merge requests.
 
 - As soon as possible, but no later than the third milestone preceding the major release (for example, given the following release schedule: `14.8, 14.9, 14.10, 15.0` – `14.8` is the third milestone preceding the major release):
     - [ ] A [deprecation entry](https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-a-deprecation-entry) has been created so the deprecation will appear in release posts and on the [general deprecation page](https://docs.gitlab.com/ee/update/deprecations).
-    - [ ] Documentation has been updated to add a note about the [end-of-life](https://docs.gitlab.com/ee/development/documentation/styleguide/#end-of-life-for-features-or-products) and to mark the feature as [deprecated](https://docs.gitlab.com/ee/development/documentation/styleguide/#deprecated-features).
+    - [ ] Documentation has been updated to mark the feature as [deprecated](https://docs.gitlab.com/ee/development/documentation/versions.html#deprecations-and-removals).
 - [ ] On or before the major milestone: A [removal entry](https://about.gitlab.com/handbook/marketing/blog/release-posts/#removals) has been created so the removal will appear on the [removals by milestones](https://docs.gitlab.com/ee/update/removals) page and be announced in the release post.
 - On the major milestone:
     - [ ] The deprecated item has been removed.
diff --git a/.gitlab/issue_templates/Feature Flag Roll Out.md b/.gitlab/issue_templates/Feature Flag Roll Out.md
index 52f189f09f0..1b3d82cf522 100644
--- a/.gitlab/issue_templates/Feature Flag Roll Out.md
+++ b/.gitlab/issue_templates/Feature Flag Roll Out.md
@@ -62,8 +62,7 @@ _Consider adding links to check for Sentry errors, Production logs for 5xx, 302s
 - Ensure that the feature MRs have been deployed to non-production environments.
     - [ ] `/chatops run auto_deploy status <merge-commit-of-your-feature>`
 - [ ] Enable the feature globally on non-production environments.
-    - [ ] `/chatops run feature set <feature-flag-name> true --dev`
-    - [ ] `/chatops run feature set <feature-flag-name> true --staging`
+    - [ ] `/chatops run feature set <feature-flag-name> true --dev --staging`
 - [ ] Verify that the feature works as expected. Posting the QA result in this issue is preferable.
       The best environment to validate the feature in is [staging-canary](https://about.gitlab.com/handbook/engineering/infrastructure/environments/#staging-canary)
       as this is the first environment deployed to. Note you will need to make sure you are configured to use canary as outlined [here](https://about.gitlab.com/handbook/engineering/infrastructure/environments/canary-stage/)
@@ -74,12 +73,9 @@ _Consider adding links to check for Sentry errors, Production logs for 5xx, 302s
 - Ensure that the feature MRs have been deployed to both production and canary.
     - [ ] `/chatops run auto_deploy status <merge-commit-of-your-feature>`
 - If you're using [project-actor](https://docs.gitlab.com/ee/development/feature_flags/#feature-actors), you must enable the feature on these entries:
-    - [ ] `/chatops run feature set --project=gitlab-org/gitlab <feature-flag-name> true`
-    - [ ] `/chatops run feature set --project=gitlab-org/gitlab-foss <feature-flag-name> true`
-    - [ ] `/chatops run feature set --project=gitlab-com/www-gitlab-com <feature-flag-name> true`
+    - [ ] `/chatops run feature set --project=gitlab-org/gitlab,gitlab-org/gitlab-foss,gitlab-com/www-gitlab-com <feature-flag-name> true`
 - If you're using [group-actor](https://docs.gitlab.com/ee/development/feature_flags/#feature-actors), you must enable the feature on these entries:
-    - [ ] `/chatops run feature set --group=gitlab-org <feature-flag-name> true`
-    - [ ] `/chatops run feature set --group=gitlab-com <feature-flag-name> true`
+    - [ ] `/chatops run feature set --group=gitlab-org,gitlab-com <feature-flag-name> true`
 - If you're using [user-actor](https://docs.gitlab.com/ee/development/feature_flags/#feature-actors), you must enable the feature on these entries:
     - [ ] `/chatops run feature set --user=<your-username> <feature-flag-name> true`
 - [ ] Verify that the feature works on the specific entries. Posting the QA result in this issue is preferable.
@@ -124,9 +120,7 @@ To do so, follow these steps:
       the feature can be officially announced in a release blog post.
     - [ ] `/chatops run release check <merge-request-url> <milestone>`
 - [ ] Consider cleaning up the feature flag from all environments by running these chatops command in `#production` channel. Otherwise these settings may override the default enabled.
-    - [ ] `/chatops run feature delete <feature-flag-name> --dev`
-    - [ ] `/chatops run feature delete <feature-flag-name> --staging`
-    - [ ] `/chatops run feature delete <feature-flag-name>`
+    - [ ] `/chatops run feature delete <feature-flag-name> --dev --staging --production`
 - [ ] Close [the feature issue](ISSUE LINK) to indicate the feature will be released in the current milestone.
 - [ ] Set the next milestone to this rollout issue for scheduling [the flag removal](#release-the-feature).
 - [ ] (Optional) You can [create a separate issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20Flag%20Cleanup) for scheduling the steps below to [Release the feature](#release-the-feature).
@@ -162,9 +156,7 @@ You can either [create a follow-up issue for Feature Flag Cleanup](https://gitla
     - [ ] `/chatops run release check <merge-request-url> <milestone>`
 - [ ] Close [the feature issue](ISSUE LINK) to indicate the feature will be released in the current milestone.
 - [ ] If not already done, clean up the feature flag from all environments by running these chatops command in `#production` channel:
-    - [ ] `/chatops run feature delete <feature-flag-name> --dev`
-    - [ ] `/chatops run feature delete <feature-flag-name> --staging`
-    - [ ] `/chatops run feature delete <feature-flag-name>`
+    - [ ] `/chatops run feature delete <feature-flag-name> --dev --staging --production`
 - [ ] Close this rollout issue.
 
 ## Rollback Steps
diff --git a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
index bfcf7aca7b5..34e6e70015b 100644
--- a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
+++ b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
@@ -48,9 +48,13 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
 - [ ] Create the migration file in `ee/db/geo/migrate`:
 
   ```shell
-  bin/rails generate geo_migration CreateCoolWidgetRegistry
+  bin/rails generate migration CreateCoolWidgetRegistry --database geo
   ```
 
+Geo should continue using `Gitlab::Database::Migration[1.0]` until the `gitlab_geo` schema is supported, and is for the time being exempt from being validated by `Gitlab::Database::Migration[2.0]`. This requires a developer to manually amend the migration file to change from `[2.0]` to `[1.0]` due to the migration defaults being 2.0.
+
+For more information, see the [Enable Geo migrations to use Migration[2.0]](https://gitlab.com/gitlab-org/gitlab/-/issues/363491) issue.
+
 - [ ] Replace the contents of the migration file with the following. Note that we cannot add a foreign key constraint on `cool_widget_id` because the `cool_widgets` table is in a different database. The application code must handle logic such as propagating deletions.
 
   ```ruby
diff --git a/.gitlab/issue_templates/Geo Replicate a new blob type.md b/.gitlab/issue_templates/Geo Replicate a new blob type.md
index ff678666191..e6f96c575d2 100644
--- a/.gitlab/issue_templates/Geo Replicate a new blob type.md
+++ b/.gitlab/issue_templates/Geo Replicate a new blob type.md
@@ -50,9 +50,13 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
 - [ ] Create the migration file in `ee/db/geo/migrate`:
 
   ```shell
-  bin/rails generate geo_migration CreateCoolWidgetRegistry
+  bin/rails generate migration CreateCoolWidgetRegistry --database geo
   ```
 
+Geo should continue using `Gitlab::Database::Migration[1.0]` until the `gitlab_geo` schema is supported, and is for the time being exempt from being validated by `Gitlab::Database::Migration[2.0]`. This requires a developer to manually amend the migration file to change from `[2.0]` to `[1.0]` due to the migration defaults being 2.0.
+
+For more information, see the [Enable Geo migrations to use Migration[2.0]](https://gitlab.com/gitlab-org/gitlab/-/issues/363491) issue.
+
 - [ ] Replace the contents of the migration file with the following. Note that we cannot add a foreign key constraint on `cool_widget_id` because the `cool_widgets` table is in a different database. The application code must handle logic such as propagating deletions.
 
   ```ruby
diff --git a/.gitlab/issue_templates/Performance Indicator Metric.md b/.gitlab/issue_templates/Performance Indicator Metric.md
index f4d8885b119..8019be8cad5 100644
--- a/.gitlab/issue_templates/Performance Indicator Metric.md
+++ b/.gitlab/issue_templates/Performance Indicator Metric.md
@@ -3,7 +3,7 @@ Performance Indicator Metric issues are used for adding, updating, or removing p
 
 Please title your issue with the following format: "{action}(Add|Update|Remove) Metric name as performance indicator"
 
-Example of title: "Add static_site_editor_views as gmau"
+Example of title: "Add some_feature_views as gmau"
 
 -->
 
diff --git a/.gitlab/issue_templates/Service Ping reporting and monitoring.md b/.gitlab/issue_templates/Service Ping reporting and monitoring.md
new file mode 100644
index 00000000000..1c0d221318b
--- /dev/null
+++ b/.gitlab/issue_templates/Service Ping reporting and monitoring.md
@@ -0,0 +1,129 @@
+<!-- This issue template is used by https://about.gitlab.com/handbook/engineering/development/growth/product-intelligence/ for tracking effort around Service Ping reporting for GitLab.com -->
+
+The [Product Intelligence group](https://about.gitlab.com/handbook/engineering/development/growth/product-intelligence/) runs manual reporting of ServicePing for GitLab.com on a weekly basis. This issue captures:
+
+- Captures the work required to complete the reporting process,.
+- Captures the follow-up tasks that are focused on metrics performance verification.
+- Identifies any potential issues.
+
+# New metrics to be verified
+
+<!-- Add new metrics that must be verified -->
+
+# Failed metrics
+
+Broken metrics issues are marked with the ~"broken metric" label.
+
+# Use a detached screen session to generate Service Ping for GitLab.com
+
+## Prerequisites
+
+1. Add your SSH key to the local SSH agent: `ssh-add`. Your SSH key is required to connect to a Rails console from the bastion host.
+
+## Triggering
+
+1. Add the SSH key to the local SSH agent: `ssh-add`.
+1. Connect to the bastion with SSH agent forwarding: `ssh -A lb-bastion.gprd.gitlab.com`.
+1. Note which bastion host machine was assigned. For example: `<username>@bastion-01-inf-gprd.c.gitlab-production.internal:~$` shows that you are connected to `bastion-01-inf-gprd.c.gitlab-production.internal`.
+1. Create a named screen: `screen -S $USER-service-ping-$(date +%F)`.
+1. Connect to the console host: `ssh $USER-rails@console-01-sv-gprd.c.gitlab-production.internal`.
+1. Run: `ServicePing::SubmitService.new.execute`.
+1. Press <kbd>Control</kbd>+<kbd>a</kbd> followed by <kbd>Control</kbd>+<kbd>d</kbd> to detach from the screen session.
+1. Exit from the bastion: `exit`.
+
+## Verification (After approximately 30 hours)
+
+1. Reconnect to the bastion: `ssh -A lb-bastion.gprd.gitlab.com`. Make sure that you are connected to the same host machine that ServicePing was started on. For example, to connect directly to the host machine, use `ssh bastion-01-inf-gprd.c.gitlab-production.internal`.
+1. Find your screen session: `screen -ls`.
+1. Attach to your screen session: `screen -x 14226.mwawrzyniak_service_ping_2021_01_22`.
+1. Check the last payload in the `raw_usage_data` table: `RawUsageData.last.payload`.
+1. Check the when the payload was sent: `RawUsageData.last.sent_at`.
+
+## Stop the Service Ping process
+
+Use either of these processes:
+
+1. Reconnect to the bastion host machine. For example, use: `ssh bastion-01-inf-gprd.c.gitlab-production.internal`.
+1. Find your screen session: `$ screen -ls`.
+1. Attach to your screen session: `$ sudo -u <username> screen -r`.
+1. Press <kbd>Control</kbd>+<kbd>c</kbd> to stop the Service Ping process.
+
+OR
+
+1. Reconnect to the bastion host machine. For example, type: `ssh bastion-01-inf-gprd.c.gitlab-production.internal`.
+1. List all process started by your username: `ps faux | grep <username>`.
+1. Locate the username that owns ServicePing reporting.
+1. Send the kill signal for the ServicePing PID: `kill -9 <service_ping_pid>`.
+
+## Service Ping process triggering (through a long-running SSH session)
+
+1. Connect to the `gprd` Rails console.
+1. Run `SubmitUsagePingService.new.execute`. This process requires more than 30 hours to complete.
+1. Find the last payload in the `raw_usage_data` table: `RawUsageData.last.payload`.
+1. Check the when the payload was sent: `RawUsageData.last.sent_at`.
+
+```plaintext
+ServicePing::SubmitService.new.execute
+
+# Get the payload
+RawUsageData.last.payload
+
+# Time when payload was sent to VersionsAppp
+RawUsageData.last.sent_at
+```
+
+# Verify Service Ping in VersionsApp
+
+To verify that the ServicePing was received in the VersionsApp do the following steps:
+
+1. Go to the VersionsApp console and locate: `RawUsageData.find(uuid: '')`.
+1. Check the object. Either:
+   - Go to the Rails console and check the related `RawUsageData` object.
+   - Go to the VersionsApp UI <https://version.gitlab.com/usage_data/usage_data_id>.
+
+```ruby
+/bin/herokuish procfile exec rails console
+
+puts UsageData.select(:recorded_at, :app_server_type).where(hostname: 'gitlab.com', uuid: 'ea8bf810-1d6f-4a6a-b4fd-93e8cbd8b57f').order('id desc').limit(5).to_json
+
+puts UsageData.find(21635202).raw_usage_data.payload.to_json
+```
+
+# Monitoring events tracked using Redis HLL
+
+Trigger some events from the User Interface.
+
+```ruby
+Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'event_name', start_date: 28.days.ago, end_date: Date.current)
+```
+
+# Troubleshooting
+
+## Connecting to a Rails console host fails with `Permission denied (publickey).`.
+
+Make sure you add the SSH key to the local SSH agent with: `ssh-add`. If you don't add your SSH key, your key won't be forwarded
+when you run `ssh -A`, and you will not be able to connect to a Rails console host.
+
+# What to do if you get mentioned
+
+In this issue, we keep the track of new metrics added to the Service Ping, and the metrics that are timing out.
+
+If you get mentioned, check the failing metric and open an optimization issue.
+
+# Service Ping manual generation for GitLab.com schedule
+
+| Generation start date | GitLab developer handle | Link to comment with payload |
+| --------------------- | ----------------------- | ---------------------------- |
+| 2022-04-18            |                         |                              |
+| 2022-04-25            |                         |                              |
+| 2022-05-02            |                         |                              |
+| 2022-05-09            |                         |                              |
+| 2022-05-16            |                         |                              |
+
+<!-- Do not edit below this line -->
+
+/confidential
+/label ~"group::product intelligence" ~"devops::growth" ~backend ~"section::growth" ~"Category:Service Ping"
+/epic https://gitlab.com/groups/gitlab-org/-/epics/6000
+/weight 5
+/title Monitor and Generate GitLab.com Service Ping