Add latest changes from gitlab-org/gitlab@15-0-stable-eev15.0.0-rc42

6 files changed, 80 insertions, 59 deletions

diff --git a/.gitlab/issue_templates/Default.md b/.gitlab/issue_templates/Default.md
index f87b82e341b..ab97a24cce7 100644
--- a/.gitlab/issue_templates/Default.md
+++ b/.gitlab/issue_templates/Default.md
@@ -9,3 +9,5 @@ If you are experiencing an issue when using GitLab.com, your first port of call
 If you feel that your issue can be categorized as a reproducible bug or a feature proposal, please use one of the issue templates provided and include as much information as possible.
 
 Thank you for helping to make GitLab a better product.
+
+<!-- template sourced from https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Default.md -->
diff --git a/.gitlab/issue_templates/Feature Flag Cleanup.md b/.gitlab/issue_templates/Feature Flag Cleanup.md
index eedb35a4b5f..d32b0c874d4 100644
--- a/.gitlab/issue_templates/Feature Flag Cleanup.md
+++ b/.gitlab/issue_templates/Feature Flag Cleanup.md
@@ -41,7 +41,7 @@ Are there any other stages or teams involved that need to be kept in the loop?
       the feature can be officially announced in a release blog post.
     - [ ] `/chatops run auto_deploy status <merge-commit-of-cleanup-mr>`
 - [ ] Close [the feature issue](ISSUE LINK) to indicate the feature will be released in the current milestone.
-- [ ] Clean up the feature flag from all environments by running these chatops command in `#production` channel:
+- [ ] If not already done, clean up the feature flag from all environments by running these chatops command in `#production` channel:
     - [ ] `/chatops run feature delete <feature-flag-name> --dev`
     - [ ] `/chatops run feature delete <feature-flag-name> --staging`
     - [ ] `/chatops run feature delete <feature-flag-name>`
@@ -49,4 +49,3 @@ Are there any other stages or teams involved that need to be kept in the loop?
 
 
 /label ~"feature flag" ~"type::feature" ~"feature::addition"
-/assign DRI
diff --git a/.gitlab/issue_templates/Feature Flag Roll Out.md b/.gitlab/issue_templates/Feature Flag Roll Out.md
index 0462742513c..52f189f09f0 100644
--- a/.gitlab/issue_templates/Feature Flag Roll Out.md
+++ b/.gitlab/issue_templates/Feature Flag Roll Out.md
@@ -123,6 +123,10 @@ To do so, follow these steps:
       If the merge request was deployed before [the monthly release was tagged](https://about.gitlab.com/handbook/engineering/releases/#self-managed-releases-1),
       the feature can be officially announced in a release blog post.
     - [ ] `/chatops run release check <merge-request-url> <milestone>`
+- [ ] Consider cleaning up the feature flag from all environments by running these chatops command in `#production` channel. Otherwise these settings may override the default enabled.
+    - [ ] `/chatops run feature delete <feature-flag-name> --dev`
+    - [ ] `/chatops run feature delete <feature-flag-name> --staging`
+    - [ ] `/chatops run feature delete <feature-flag-name>`
 - [ ] Close [the feature issue](ISSUE LINK) to indicate the feature will be released in the current milestone.
 - [ ] Set the next milestone to this rollout issue for scheduling [the flag removal](#release-the-feature).
 - [ ] (Optional) You can [create a separate issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20Flag%20Cleanup) for scheduling the steps below to [Release the feature](#release-the-feature).
@@ -157,7 +161,7 @@ You can either [create a follow-up issue for Feature Flag Cleanup](https://gitla
       the feature can be officially announced in a release blog post.
     - [ ] `/chatops run release check <merge-request-url> <milestone>`
 - [ ] Close [the feature issue](ISSUE LINK) to indicate the feature will be released in the current milestone.
-- [ ] Clean up the feature flag from all environments by running these chatops command in `#production` channel:
+- [ ] If not already done, clean up the feature flag from all environments by running these chatops command in `#production` channel:
     - [ ] `/chatops run feature delete <feature-flag-name> --dev`
     - [ ] `/chatops run feature delete <feature-flag-name> --staging`
     - [ ] `/chatops run feature delete <feature-flag-name>`
@@ -172,4 +176,3 @@ You can either [create a follow-up issue for Feature Flag Cleanup](https://gitla
 ```
 
 /label ~"feature flag" ~"type::feature" ~"feature::addition"
-/assign DRI
diff --git a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
index e858f80ffaa..bfcf7aca7b5 100644
--- a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
+++ b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
@@ -104,7 +104,7 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
 - [ ] Run Geo tracking database migrations:
 
   ```shell
-  bin/rake geo:db:migrate
+  bin/rake db:migrate:geo
   ```
 
 - [ ] Be sure to commit the relevant changes in `ee/db/geo/structure.sql`
@@ -303,12 +303,6 @@ That's all of the required database changes.
         git_access_class.error_message(:no_repo)
       end
 
-      # The feature flag follows the format `geo_#{replicable_name}_replication`,
-      # so here it would be `geo_cool_widget_replication`
-      def self.replication_enabled_by_default?
-        false
-      end
-
       override :verification_feature_flag_enabled?
       def self.verification_feature_flag_enabled?
         # We are adding verification at the same time as replication, so we
@@ -673,34 +667,48 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
 
 Individual Cool Widget replication and verification data should now be available via the GraphQL API.
 
-### Release Geo support of Cool Widgets
+#### Step 4. Handle batch destroy
 
-- [ ] In the rollout issue you created when creating the feature flag, modify the Roll Out Steps:
-  - [ ] Cross out any steps related to testing on production GitLab.com, because Geo is not running on production GitLab.com at the moment.
-  - [ ] Add a step to `Test replication and verification of Cool Widgets on a non-GDK-deployment. For example, using GitLab Environment Toolkit`.
-  - [ ] Add a step to `Ping the Geo PM and EM to coordinate testing`. For example, you might add steps to generate Cool Widgets, and then a Geo engineer may take it from there.
-- [ ] In `ee/config/feature_flags/development/geo_cool_widget_replication.yml`, set `default_enabled: true`
+If batch destroy logic is implemented for a replicable, then that logic must be "replicated" by Geo secondaries. The easiest way to do this is use `Geo::BatchEventCreateWorker` to bulk insert a delete event for each replicable.
 
-- [ ] In `ee/app/replicators/geo/cool_widget_replicator.rb`, delete the `self.replication_enabled_by_default?` method:
+For example, if `FastDestroyAll` is used, then you may be able to [use `begin_fast_destroy` and `finalize_fast_destroy` hooks, like we did for uploads](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69763).
 
-  ```ruby
-  module Geo
-    class CoolWidgetReplicator < Gitlab::Geo::Replicator
-      ...
-      # REMOVE THIS LINE IF IT IS NO LONGER NEEDED
-      extend ::Gitlab::Utils::Override
+Or if a special service is used to batch delete records and their associated data, then you probably need to [hook into that service, like we did for job artifacts](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79530).
 
-      # REMOVE THIS METHOD
-      def self.replication_enabled_by_default?
-        false
-      end
-      # REMOVE THIS METHOD
+As illustrated by the above two examples, batch destroy logic cannot be handled automatically by Geo secondaries without restricting the way other teams perform batch destroys. It is up to you to produce `Geo::BatchEventCreateWorker` attributes before the records are deleted, and then enqueue `Geo::BatchEventCreateWorker` after the records are deleted.
 
-      ...
+- [ ] Ensure that any batch destroy of this replicable is replicated to secondary sites
+- [ ] Regardless of implementation details, please verify in specs that when the parent object is removed, the new `Geo::Event` records are created:
+
+```ruby
+  describe '#destroy' do
+    subject { create(:cool_widget) }
+
+    context 'when running in a Geo primary node' do
+      let_it_be(:primary) { create(:geo_node, :primary) }
+      let_it_be(:secondary) { create(:geo_node) }
+
+      it 'logs an event to the Geo event log when bulk removal is used', :sidekiq_inline do
+        stub_current_geo_node(primary)
+
+        expect { subject.project.destroy! }.to change(Geo::Event.where(replicable_name: :cool_widget, event_name: :deleted), :count).by(1)
+
+        payload = Geo::Event.where(replicable_name: :cool_widget, event_name: :deleted).last.payload
+
+        expect(payload['model_record_id']).to eq(subject.id)
+        expect(payload['blob_path']).to eq(subject.relative_path)
+        expect(payload['uploader_class']).to eq('CoolWidgetUploader')
+      end
     end
   end
-  ```
+```
+### Release Geo support of Cool Widgets
 
+- [ ] In the rollout issue you created when creating the feature flag, modify the Roll Out Steps:
+  - [ ] Cross out any steps related to testing on production GitLab.com, because Geo is not running on production GitLab.com at the moment.
+  - [ ] Add a step to `Test replication and verification of Cool Widgets on a non-GDK-deployment. For example, using GitLab Environment Toolkit`.
+  - [ ] Add a step to `Ping the Geo PM and EM to coordinate testing`. For example, you might add steps to generate Cool Widgets, and then a Geo engineer may take it from there.
+- [ ] In `ee/config/feature_flags/development/geo_cool_widget_replication.yml`, set `default_enabled: true`
 - [ ] In `ee/app/graphql/types/geo/geo_node_type.rb`, remove the `feature_flag` option for the released type:
 
   ```ruby
diff --git a/.gitlab/issue_templates/Geo Replicate a new blob type.md b/.gitlab/issue_templates/Geo Replicate a new blob type.md
index 0cbfd79c958..ff678666191 100644
--- a/.gitlab/issue_templates/Geo Replicate a new blob type.md
+++ b/.gitlab/issue_templates/Geo Replicate a new blob type.md
@@ -104,7 +104,7 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
 - [ ] Run Geo tracking database migrations:
 
   ```shell
-  bin/rake geo:db:migrate
+  bin/rake db:migrate:geo
   ```
 
 - [ ] Be sure to commit the relevant changes in `ee/db/geo/structure.sql`
@@ -291,12 +291,6 @@ That's all of the required database changes.
         model_record.file
       end
 
-      # The feature flag follows the format `geo_#{replicable_name}_replication`,
-      # so here it would be `geo_cool_widget_replication`
-      def self.replication_enabled_by_default?
-        false
-      end
-
       override :verification_feature_flag_enabled?
       def self.verification_feature_flag_enabled?
         # We are adding verification at the same time as replication, so we
@@ -637,35 +631,49 @@ The GraphQL API is used by `Admin > Geo > Replication Details` views, and is dir
 
 Individual Cool Widget replication and verification data should now be available via the GraphQL API.
 
-### Release Geo support of Cool Widgets
+#### Step 4. Handle batch destroy
 
-- [ ] In the rollout issue you created when creating the feature flag, modify the Roll Out Steps:
-  - [ ] Cross out any steps related to testing on production GitLab.com, because Geo is not running on production GitLab.com at the moment.
-  - [ ] Add a step to `Test replication and verification of Cool Widgets on a non-GDK-deployment. For example, using GitLab Environment Toolkit`.
-  - [ ] Add a step to `Ping the Geo PM and EM to coordinate testing`. For example, you might add steps to generate Cool Widgets, and then a Geo engineer may take it from there.
-- [ ] In `ee/config/feature_flags/development/geo_cool_widget_replication.yml`, set `default_enabled: true`
+If batch destroy logic is implemented for a replicable, then that logic must be "replicated" by Geo secondaries. The easiest way to do this is use `Geo::BatchEventCreateWorker` to bulk insert a delete event for each replicable.
 
-- [ ] In `ee/app/replicators/geo/cool_widget_replicator.rb`, delete the `self.replication_enabled_by_default?` method:
+For example, if `FastDestroyAll` is used, then you may be able to [use `begin_fast_destroy` and `finalize_fast_destroy` hooks, like we did for uploads](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69763).
 
-  ```ruby
-  module Geo
-    class CoolWidgetReplicator < Gitlab::Geo::Replicator
-      ...
-      # REMOVE THIS LINE IF IT IS NO LONGER NEEDED
-      extend ::Gitlab::Utils::Override
+Or if a special service is used to batch delete records and their associated data, then you probably need to [hook into that service, like we did for job artifacts](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79530).
 
-      ...
-      # REMOVE THIS METHOD
-      def self.replication_enabled_by_default?
-        false
-      end
-      # REMOVE THIS METHOD
+As illustrated by the above two examples, batch destroy logic cannot be handled automatically by Geo secondaries without restricting the way other teams perform batch destroys. It is up to you to produce `Geo::BatchEventCreateWorker` attributes before the records are deleted, and then enqueue `Geo::BatchEventCreateWorker` after the records are deleted.
 
-      ...
+- [ ] Ensure that any batch destroy of this replicable is replicated to secondary sites
+- [ ] Regardless of implementation details, please verify in specs that when the parent object is removed, the new `Geo::Event` records are created:
+
+```ruby
+  describe '#destroy' do
+    subject { create(:cool_widget) }
+
+    context 'when running in a Geo primary node' do
+      let_it_be(:primary) { create(:geo_node, :primary) }
+      let_it_be(:secondary) { create(:geo_node) }
+
+      it 'logs an event to the Geo event log when bulk removal is used', :sidekiq_inline do
+        stub_current_geo_node(primary)
+
+        expect { subject.project.destroy! }.to change(Geo::Event.where(replicable_name: :cool_widget, event_name: :deleted), :count).by(1)
+
+        payload = Geo::Event.where(replicable_name: :cool_widget, event_name: :deleted).last.payload
+
+        expect(payload['model_record_id']).to eq(subject.id)
+        expect(payload['blob_path']).to eq(subject.relative_path)
+        expect(payload['uploader_class']).to eq('CoolWidgetUploader')
+      end
     end
   end
-  ```
+```
 
+### Release Geo support of Cool Widgets
+
+- [ ] In the rollout issue you created when creating the feature flag, modify the Roll Out Steps:
+  - [ ] Cross out any steps related to testing on production GitLab.com, because Geo is not running on production GitLab.com at the moment.
+  - [ ] Add a step to `Test replication and verification of Cool Widgets on a non-GDK-deployment. For example, using GitLab Environment Toolkit`.
+  - [ ] Add a step to `Ping the Geo PM and EM to coordinate testing`. For example, you might add steps to generate Cool Widgets, and then a Geo engineer may take it from there.
+- [ ] In `ee/config/feature_flags/development/geo_cool_widget_replication.yml`, set `default_enabled: true`
 - [ ] In `ee/app/graphql/types/geo/geo_node_type.rb`, remove the `feature_flag` option for the released type:
 
   ```ruby
diff --git a/.gitlab/issue_templates/Security developer workflow.md b/.gitlab/issue_templates/Security developer workflow.md
index 5c1b669a88f..4cced5a25fe 100644
--- a/.gitlab/issue_templates/Security developer workflow.md
+++ b/.gitlab/issue_templates/Security developer workflow.md
@@ -44,6 +44,7 @@ After your merge request has been approved according to our [approval guidelines
 - [ ] Fill in any upgrade notes that users may need to take into account in the [details section](#details)
 - [ ] Add Yes/No and further details if needed to the migration and settings columns in the [details section](#details)
 - [ ] Add the nickname of the external user who found the issue (and/or HackerOne profile) to the Thanks row in the [details section](#details)
+- [ ] If this includes a breaking change, make sure it is mentioned for the relevant versions in [`doc/update/index.md`](https://gitlab.com/gitlab-org/security/gitlab/-/blob/master/doc/update/index.md#version-specific-upgrading-instructions)
 
 ## Summary