Add latest changes from gitlab-org/gitlab@16-0-stable-eev16.0.0-rc42

3 files changed, 87 insertions, 86 deletions

diff --git a/doc/development/sidekiq/compatibility_across_updates.md b/doc/development/sidekiq/compatibility_across_updates.md
index b417a099228..d20f4230fc8 100644
--- a/doc/development/sidekiq/compatibility_across_updates.md
+++ b/doc/development/sidekiq/compatibility_across_updates.md
@@ -46,30 +46,30 @@ following example deprecates and then removes `arg2` from the `perform_async` me
 1. Provide a default value (usually `nil`) and use a comment to mark the
    argument as deprecated in the coming minor release. (Release M)
 
-    ```ruby
-    class ExampleWorker
-      # Keep arg2 parameter for backwards compatibility.
-      def perform(object_id, arg1, arg2 = nil)
-        # ...
-      end
-    end
-    ```
+   ```ruby
+   class ExampleWorker
+     # Keep arg2 parameter for backwards compatibility.
+     def perform(object_id, arg1, arg2 = nil)
+       # ...
+     end
+   end
+   ```
 
 1. One minor release later, stop using the argument in `perform_async`. (Release M+1)
 
-    ```ruby
-    ExampleWorker.perform_async(object_id, arg1)
-    ```
+   ```ruby
+   ExampleWorker.perform_async(object_id, arg1)
+   ```
 
 1. At the next major release, remove the value from the worker class. (Next major release)
 
-    ```ruby
-    class ExampleWorker
-      def perform(object_id, arg1)
-        # ...
-      end
-    end
-    ```
+   ```ruby
+   class ExampleWorker
+     def perform(object_id, arg1)
+       # ...
+     end
+   end
+   ```
 
 ### Add an argument
 
@@ -84,29 +84,29 @@ This approach requires multiple releases.
 
 1. Add the argument to the worker with a default value (Release M).
 
-    ```ruby
-    class ExampleWorker
-      def perform(object_id, new_arg = nil)
-        # ...
-      end
-    end
-    ```
+   ```ruby
+   class ExampleWorker
+     def perform(object_id, new_arg = nil)
+       # ...
+     end
+   end
+   ```
 
 1. Add the new argument to all the invocations of the worker (Release M+1).
 
-    ```ruby
-    ExampleWorker.perform_async(object_id, new_arg)
-    ```
+   ```ruby
+   ExampleWorker.perform_async(object_id, new_arg)
+   ```
 
 1. Remove the default value (Release M+2).
 
-    ```ruby
-    class ExampleWorker
-      def perform(object_id, new_arg)
-        # ...
-      end
-    end
-    ```
+   ```ruby
+   class ExampleWorker
+     def perform(object_id, new_arg)
+       # ...
+     end
+   end
+   ```
 
 #### Parameter hash
 
@@ -115,13 +115,13 @@ uses a parameter hash.
 
 1. Use a parameter hash in the worker to allow future flexibility.
 
-    ```ruby
-    class ExampleWorker
-      def perform(object_id, params = {})
-        # ...
-      end
-    end
-    ```
+   ```ruby
+   class ExampleWorker
+     def perform(object_id, params = {})
+       # ...
+     end
+   end
+   ```
 
 ## Removing worker classes
 
@@ -131,54 +131,55 @@ To remove a worker class, follow these steps over two minor releases:
 
 1. Remove any code that enqueues the jobs.
 
-    For example, if there is a UI component or an API endpoint that a user can interact with that results in the worker instance getting enqueued, make sure those surface areas are either removed or updated in a way that the worker instance is no longer enqueued.
+   For example, if there is a UI component or an API endpoint that a user can interact with that results in the worker instance getting enqueued, make sure those surface areas are either removed or updated in a way that the worker instance is no longer enqueued.
 
-    This ensures that instances related to the worker class are no longer being enqueued.
+   This ensures that instances related to the worker class are no longer being enqueued.
 
 1. Ensure both the frontend and backend code no longer relies on any of the work that used to be done by the worker.
 1. In the relevant worker classes, replace the contents of the `perform` method with a no-op, while keeping any arguments in tact.
 
-    For example, if you're working with the following `ExampleWorker`:
+   For example, if you're working with the following `ExampleWorker`:
 
-      ```ruby
-        class ExampleWorker
-          def perform(object_id)
-            SomeService.run!(object_id)
-          end
-        end
-      ```
+   ```ruby
+     class ExampleWorker
+       def perform(object_id)
+         SomeService.run!(object_id)
+       end
+     end
+   ```
 
-    Implementing the no-op might look like this:
+   Implementing the no-op might look like this:
 
-      ```ruby
-        class ExampleWorker
-          def perform(object_id); end
-        end
-      ```
+   ```ruby
+     class ExampleWorker
+       def perform(object_id); end
+     end
+   ```
 
-    By implementing this no-op, you can avoid unnecessary cycles once any deprecated jobs that are still enqueued eventually get processed.
+   By implementing this no-op, you can avoid unnecessary cycles once any deprecated jobs that are still enqueued eventually get processed.
 
 ### In a subsequent, separate minor release
 
 1. Delete the worker class file and follow the guidance in our [Sidekiq queues documentation](../sidekiq/index.md#sidekiq-queues) around running Rake tasks to regenerate/update related files.
 1. Add a migration (not a post-deployment migration) that uses `sidekiq_remove_jobs`:
 
-    ```ruby
-    class RemoveMyDeprecatedWorkersJobInstances < Gitlab::Database::Migration[2.0]
-      DEPRECATED_JOB_CLASSES = %w[
-        MyDeprecatedWorkerOne
-        MyDeprecatedWorkerTwo
-      ]
-
-      def up
-        sidekiq_remove_jobs(job_klasses: DEPRECATED_JOB_CLASSES)
-      end
-
-      def down
-        # This migration removes any instances of deprecated workers and cannot be undone.
-      end
-    end
-    ```
+   ```ruby
+   class RemoveMyDeprecatedWorkersJobInstances < Gitlab::Database::Migration[2.0]
+     DEPRECATED_JOB_CLASSES = %w[
+       MyDeprecatedWorkerOne
+       MyDeprecatedWorkerTwo
+     ]
+     # Always use `disable_ddl_transaction!` while using the `sidekiq_remove_jobs` method, as we had multiple production incidents due to `idle-in-transaction` timeout.
+     disable_ddl_transaction!
+     def up
+       sidekiq_remove_jobs(job_klasses: DEPRECATED_JOB_CLASSES)
+     end
+
+     def down
+       # This migration removes any instances of deprecated workers and cannot be undone.
+     end
+   end
+   ```
 
 ## Renaming queues
 
diff --git a/doc/development/sidekiq/index.md b/doc/development/sidekiq/index.md
index 355f5a3b753..2010a21130d 100644
--- a/doc/development/sidekiq/index.md
+++ b/doc/development/sidekiq/index.md
@@ -4,7 +4,7 @@ group: unassigned
 info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
 ---
 
-# Sidekiq guides
+# Sidekiq development guidelines
 
 We use [Sidekiq](https://github.com/mperham/sidekiq) as our background
 job processor. These guides are for writing jobs that works well on
diff --git a/doc/development/sidekiq/worker_attributes.md b/doc/development/sidekiq/worker_attributes.md
index a3bfe5f27cc..1e3104c5e86 100644
--- a/doc/development/sidekiq/worker_attributes.md
+++ b/doc/development/sidekiq/worker_attributes.md
@@ -242,7 +242,7 @@ can put unsustainable load on the primary database server. We therefore added th
 By configuring a worker's `data_consistency` field, we can then allow the scheduler to target read replicas
 under several strategies outlined below.
 
-## Trading immediacy for reduced primary load
+### Trading immediacy for reduced primary load
 
 We require Sidekiq workers to make an explicit decision around whether they need to use the
 primary database node for all reads and writes, or whether reads can be served from replicas. This is
@@ -259,7 +259,8 @@ that mostly or exclusively perform writes, or workers that read their own writes
 into data consistency issues should a stale record be read back from a replica. **Try to avoid
 these scenarios, since `:always` should be considered the exception, not the rule.**
 
-To allow for reads to be served from replicas, we added two additional consistency modes: `:sticky` and `:delayed`.
+To allow for reads to be served from replicas, we added two additional consistency modes: `:sticky` and `:delayed`. A RuboCop rule
+reminds the developer when `:always` data consistency mode is used. If workers require the primary database, you can disable the rule in-line.
 
 When you declare either `:sticky` or `:delayed` consistency, workers become eligible for database
 load-balancing.
@@ -268,18 +269,17 @@ In both cases, if the replica is not up-to-date and the time from scheduling the
  the jobs sleep up to the minimum delay interval (0.8 seconds). This gives the replication process time to finish.
 The difference is in what happens when there is still replication lag after the delay: `sticky` workers
 switch over to the primary right away, whereas `delayed` workers fail fast and are retried once.
-If they still encounter replication lag, they also switch to the primary instead.
-**If your worker never performs any writes, it is strongly advised to apply one of these consistency settings,
-since it never needs to rely on the primary database node.**
+If the workers still encounter replication lag, they switch to the primary instead. **If your worker never performs any writes,
+it is strongly advised to apply `:sticky` or `:delayed` consistency settings, since the worker never needs to rely on the primary database node.**
 
 The table below shows the `data_consistency` attribute and its values, ordered by the degree to which
 they prefer read replicas and wait for replicas to catch up:
 
-| **Data Consistency**  | **Description**  |
-|--------------|-----------------------------|
-| `:always`    | The job is required to use the primary database (default). It should be used for workers that primarily perform writes, have strict requirements around data consistency when reading their own writes, or are cron jobs. |
-| `:sticky`    | The job prefers replicas, but switches to the primary for writes or when encountering replication lag. It should be used for jobs that require to be executed as fast as possible but can sustain a small initial queuing delay.  |
-| `:delayed`   | The job prefers replicas, but switches to the primary for writes. When encountering replication lag before the job starts, the job is retried once. If the replica is still not up to date on the next retry, it switches to the primary. It should be used for jobs where delaying execution further typically does not matter, such as cache expiration or web hooks execution. |
+| **Data consistency**  | **Description**  | **Guideline** |
+|--------------|-----------------------------|----------|
+| `:always`    | The job is required to use the primary database (default). | It should be used for workers that primarily perform writes, have strict requirements around data consistency when reading their own writes, or are cron jobs. |
+| `:sticky`    | The job prefers replicas, but switches to the primary for writes or when encountering replication lag. | It should be used for jobs that require to be executed as fast as possible but can sustain a small initial queuing delay.  |
+| `:delayed`   | The job prefers replicas, but switches to the primary for writes. When encountering replication lag before the job starts, the job is retried once. If the replica is still not up to date on the next retry, it switches to the primary. | It should be used for jobs where delaying execution further typically does not matter, such as cache expiration or web hooks execution. |
 
 In all cases workers read either from a replica that is fully caught up,
 or from the primary node, so data consistency is always ensured.