Add latest changes from gitlab-org/gitlab@14-2-stable-eev14.2.0-rc42

4 files changed, 133 insertions, 55 deletions

diff --git a/doc/administration/operations/extra_sidekiq_processes.md b/doc/administration/operations/extra_sidekiq_processes.md
index 1f195bcc378..2cc4e3a4551 100644
--- a/doc/administration/operations/extra_sidekiq_processes.md
+++ b/doc/administration/operations/extra_sidekiq_processes.md
@@ -95,14 +95,14 @@ To view the Sidekiq processes in GitLab:
 
 ## Negate settings
 
-To have the additional Sidekiq processes work on every queue **except** the ones
+To have the Sidekiq process work on every queue **except** the ones
 you list. In this example, we exclude all import-related jobs from a Sidekiq node:
 
-1. After you follow the steps for [starting extra processes](#start-multiple-processes),
-   edit `/etc/gitlab/gitlab.rb` and add:
+1. Edit `/etc/gitlab/gitlab.rb` and add:
 
    ```ruby
    sidekiq['negate'] = true
+   sidekiq['queue_selector'] = true
    sidekiq['queue_groups'] = [
       "feature_category=importers"
    ]
@@ -140,50 +140,6 @@ sidekiq['queue_groups'] = [
 ]
 ```
 
-### Disable Sidekiq cluster
-
-WARNING:
-Sidekiq cluster is [scheduled](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/240)
-to be the only way to start Sidekiq in GitLab 14.0.
-
-By default, the Sidekiq service runs `sidekiq-cluster`. To disable this behavior,
-add the following to the Sidekiq configuration:
-
-```ruby
-sidekiq['enable'] = true
-sidekiq['cluster'] = false
-```
-
-All of the aforementioned configuration options for `sidekiq`
-are available. By default, they are configured as follows:
-
-```ruby
-sidekiq['queue_selector'] = false
-sidekiq['interval'] = nil
-sidekiq['max_concurrency'] = 50
-sidekiq['min_concurrency'] = nil
-sidekiq['negate'] = false
-sidekiq['queue_groups'] = ['*']
-sidekiq['shutdown_timeout'] = 25
-```
-
-`sidekiq_cluster` must be disabled if you decide to configure the
-cluster as above.
-
-When disabling `sidekiq_cluster`, you must copy your configuration for
-`sidekiq_cluster`over to `sidekiq`. Anything configured for
-`sidekiq_cluster` is overridden by the options for `sidekiq` when
-setting `sidekiq['cluster'] = true`.
-
-When using this feature, the service called `sidekiq` is now
-running `sidekiq-cluster`.
-
-The [concurrency](#manage-concurrency) and other options configured
-for Sidekiq are respected.
-
-By default, logs for `sidekiq-cluster` go to `/var/log/gitlab/sidekiq`
-like regular Sidekiq logs.
-
 ## Ignore all import queues
 
 When [importing from GitHub](../../user/project/import/github.md) or
@@ -264,7 +220,7 @@ being equal to `max_concurrency`.
 Running a single Sidekiq process is the default in GitLab 12.10 and earlier.
 
 WARNING:
-Running Sidekiq directly is scheduled to be removed in GitLab
+Running Sidekiq directly was removed in GitLab
 [14.0](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/240).
 
 1. Edit `/etc/gitlab/gitlab.rb` and add:
diff --git a/doc/administration/operations/extra_sidekiq_routing.md b/doc/administration/operations/extra_sidekiq_routing.md
index 80540b7ba46..6938f8a7012 100644
--- a/doc/administration/operations/extra_sidekiq_routing.md
+++ b/doc/administration/operations/extra_sidekiq_routing.md
@@ -103,9 +103,11 @@ based on a subset of worker attributes:
 - `worker_name` - the worker name. The other attributes are typically more useful as
   they are more general, but this is available in case a particular worker needs
   to be selected.
-- `name` - the queue name. The other attributes are typically more useful as
-  they are more general, but this is available in case a particular queue needs
-  to be selected.
+- `name` - the queue name generated from the worker name. The other attributes
+  are typically more useful as they are more general, but this is available in
+  case a particular queue needs to be selected. Because this is generated from
+  the worker name, it does not change based on the result of other routing
+  rules.
 - `resource_boundary` - if the queue is bound by `cpu`, `memory`, or
   `unknown`. For example, the `ProjectExportWorker` is memory bound as it has
   to load data in memory before saving it for export.
diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md
index bb0756cf948..5c1271486c0 100644
--- a/doc/administration/operations/fast_ssh_key_lookup.md
+++ b/doc/administration/operations/fast_ssh_key_lookup.md
@@ -48,7 +48,7 @@ lookups via database lookup.
 
 As part of [setting up Geo](../geo/index.md#setup-instructions),
 you are required to follow the steps outlined below for both the primary and
-secondary nodes, but note that the `Write to "authorized keys" file` checkbox
+secondary nodes, but the `Write to "authorized keys" file` checkbox
 only needs to be unchecked on the primary node since it is reflected
 automatically on the secondary if database replication is working.
 
diff --git a/doc/administration/operations/moving_repositories.md b/doc/administration/operations/moving_repositories.md
index fba20da9aea..765cf64e735 100644
--- a/doc/administration/operations/moving_repositories.md
+++ b/doc/administration/operations/moving_repositories.md
@@ -7,8 +7,7 @@ type: reference
 
 # Moving repositories managed by GitLab **(FREE SELF)**
 
-Sometimes you need to move all repositories managed by GitLab to
-another file system or another server.
+You can move all repositories managed by GitLab to another file system or another server.
 
 ## Moving data within a GitLab instance
 
@@ -28,7 +27,128 @@ For more information, see:
   querying and scheduling snippet repository moves.
 - [The API documentation](../../api/group_repository_storage_moves.md) details the endpoints for
   querying and scheduling group repository moves **(PREMIUM SELF)**.
-- [Migrate to Gitaly Cluster](../gitaly/praefect.md#migrate-to-gitaly-cluster).
+- [Migrate to Gitaly Cluster](../gitaly/index.md#migrate-to-gitaly-cluster).
+
+### Move Repositories
+
+GitLab repositories can be associated with projects, groups, and snippets. Each of these types
+have a separate API to schedule the respective repositories to move. To move all repositories
+on a GitLab instance, each of these types must be scheduled to move for each storage.
+
+WARNING:
+To move repositories into a [Gitaly Cluster](../gitaly/index.md#gitaly-cluster) in GitLab versions
+13.12 to 14.1, you must [enable the `gitaly_replicate_repository_direct_fetch` feature flag](../feature_flags.md).
+
+Each repository is made read-only for the duration of the move. The repository is not writable
+until the move has completed.
+
+To move repositories:
+
+1. Ensure all storages are accessible to the GitLab instance. In this example, these are
+   `<original_storage_name>` and `<cluster_storage_name>`.
+1. [Configure repository storage weights](../repository_storage_paths.md#configure-where-new-repositories-are-stored)
+   so that the new storages receives all new projects. This stops new projects from being created
+   on existing storages while the migration is in progress.
+1. Schedule repository moves for:
+   - [Projects](#bulk-schedule-project-moves).
+   - [Snippets](#bulk-schedule-snippet-moves).
+   - [Groups](#bulk-schedule-group-moves). **(PREMIUM SELF)**
+
+### Bulk schedule project moves
+
+Use the API to schedule project moves:
+
+1. [Schedule repository storage moves for all projects on a storage shard](../../api/project_repository_storage_moves.md#schedule-repository-storage-moves-for-all-projects-on-a-storage-shard)
+   using the API. For example:
+
+   ```shell
+   curl --request POST --header "Private-Token: <your_access_token>" \
+        --header "Content-Type: application/json" \
+        --data '{"source_storage_name":"<original_storage_name>","destination_storage_name":"<cluster_storage_name>"}' \
+        "https://gitlab.example.com/api/v4/project_repository_storage_moves"
+   ```
+
+1. [Query the most recent repository moves](../../api/project_repository_storage_moves.md#retrieve-all-project-repository-storage-moves)
+   using the API. The response indicates either:
+   - The moves have completed successfully. The `state` field is `finished`.
+   - The moves are in progress. Re-query the repository move until it completes successfully.
+   - The moves have failed. Most failures are temporary and are solved by rescheduling the move.
+
+1. After the moves are complete, use the API to [query projects](../../api/projects.md#list-all-projects) and confirm that all projects have moved. None of the projects should be returned with the
+   `repository_storage` field set to the old storage. For example:
+
+   ```shell
+   curl --header "Private-Token: <your_access_token>" --header "Content-Type: application/json" \
+   "https://gitlab.example.com/api/v4/projects?repository_storage=<original_storage_name>"
+   ```
+
+   Alternatively use [the rails console](../operations/rails_console.md) to confirm that all
+   projects have moved. Run the following in the rails console:
+
+   ```ruby
+   ProjectRepository.for_repository_storage('<original_storage_name>')
+   ```
+
+1. Repeat for each storage as required.
+
+### Bulk schedule snippet moves
+
+Use the API to schedule snippet moves:
+
+1. [Schedule repository storage moves for all snippets on a storage shard](../../api/snippet_repository_storage_moves.md#schedule-repository-storage-moves-for-all-snippets-on-a-storage-shard). For example:
+
+   ```shell
+   curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
+        --header "Content-Type: application/json" \
+        --data '{"source_storage_name":"<original_storage_name>","destination_storage_name":"<cluster_storage_name>"}' \
+        "https://gitlab.example.com/api/v4/snippet_repository_storage_moves"
+   ```
+
+1. [Query the most recent repository moves](../../api/snippet_repository_storage_moves.md#retrieve-all-snippet-repository-storage-moves)
+The response indicates either:
+   - The moves have completed successfully. The `state` field is `finished`.
+   - The moves are in progress. Re-query the repository move until it completes successfully.
+   - The moves have failed. Most failures are temporary and are solved by rescheduling the move.
+
+1. After the moves are complete, use [the rails console](../operations/rails_console.md) to confirm
+   that all snippets have moved. No snippets should be returned for the original storage. Run the
+   following in the rails console:
+
+   ```ruby
+   SnippetRepository.for_repository_storage('<original_storage_name>')
+   ```
+
+1. Repeat for each storage as required.
+
+### Bulk schedule group moves **(PREMIUM SELF)**
+
+Use the API to schedule group moves:
+
+1. [Schedule repository storage moves for all groups on a storage shard](../../api/group_repository_storage_moves.md#schedule-repository-storage-moves-for-all-groups-on-a-storage-shard)
+. For example:
+
+    ```shell
+    curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
+         --header "Content-Type: application/json" \
+         --data '{"source_storage_name":"<original_storage_name>","destination_storage_name":"<cluster_storage_name>"}' \
+         "https://gitlab.example.com/api/v4/group_repository_storage_moves"
+    ```
+
+1. [Query the most recent repository moves](../../api/group_repository_storage_moves.md#retrieve-all-group-repository-storage-moves)
+. The response indicates either:
+   - The moves have completed successfully. The `state` field is `finished`.
+   - The moves are in progress. Re-query the repository move until it completes successfully.
+   - The moves have failed. Most failures are temporary and are solved by rescheduling the move.
+
+1. After the moves are complete, use [the rails console](../operations/rails_console.md) to confirm
+   that all groups have moved. No groups should be returned for the original storage. Run the
+   following in the rails console:
+
+   ```ruby
+   GroupWikiRepository.for_repository_storage('<original_storage_name>')
+   ```
+
+1. Repeat for each storage as required.
 
 ## Migrating to another GitLab instance