diff options
Diffstat (limited to 'doc/administration/gitaly/praefect.md')
| -rw-r--r-- | doc/administration/gitaly/praefect.md | 133 |
1 files changed, 2 insertions, 131 deletions
diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index 6a794dba4f9..4af7f1a58a5 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -432,7 +432,7 @@ On the **Praefect** node: WARNING: If you have data on an already existing storage called `default`, you should configure the virtual storage with another name and - [migrate the data to the Gitaly Cluster storage](#migrate-to-gitaly-cluster) + [migrate the data to the Gitaly Cluster storage](index.md#migrate-to-gitaly-cluster) afterwards. Replace `PRAEFECT_INTERNAL_TOKEN` with a strong secret, which is used by @@ -896,7 +896,7 @@ Particular attention should be shown to: WARNING: If you have existing data stored on the default Gitaly storage, - you should [migrate the data your Gitaly Cluster storage](#migrate-to-gitaly-cluster) + you should [migrate the data your Gitaly Cluster storage](index.md#migrate-to-gitaly-cluster) first. ```ruby @@ -1574,132 +1574,3 @@ sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.t - Replace the placeholder `<virtual-storage>` with the virtual storage containing the Gitaly node storage to be checked. - Replace the placeholder `<up-to-date-storage>` with the Gitaly storage name containing up to date repositories. - Replace the placeholder `<outdated-storage>` with the Gitaly storage name containing outdated repositories. - -## Migrate to Gitaly Cluster - -Whether migrating to Gitaly Cluster because of [NFS support deprecation](index.md#nfs-deprecation-notice) -or to move from single Gitaly nodes, the basic process involves: - -1. Create the required storage. -1. Create and configure Gitaly Cluster. -1. [Move the repositories](#move-repositories). - -When creating the storage, see some -[repository storage recommendations](faq.md#what-are-some-repository-storage-recommendations). - -### Move Repositories - -WARNING: -To move repositories into a Gitaly Cluster in GitLab versions 13.12 to 14.1, you must -[enable the `gitaly_replicate_repository_direct_fetch` feature flag](../feature_flags.md). - -To migrate to Gitaly Cluster, existing repositories stored outside Gitaly Cluster must be -moved. There is no automatic migration but the moves can be scheduled with the GitLab API. - -GitLab repositories can be associated with projects, groups, and snippets. Each of these types -have a separate API to schedule the respective repositories to move. To move all repositories -on a GitLab instance, each of these types must be scheduled to move for each storage. - -Each repository is made read-only for the duration of the move. The repository is not writable -until the move has completed. - -After creating and configuring Gitaly Cluster: - -1. Ensure all storages are accessible to the GitLab instance. In this example, these are - `<original_storage_name>` and `<cluster_storage_name>`. -1. [Configure repository storage weights](../repository_storage_paths.md#configure-where-new-repositories-are-stored) - so that the Gitaly Cluster receives all new projects. This stops new projects from being created - on existing Gitaly nodes while the migration is in progress. -1. Schedule repository moves for: - - [Projects](#bulk-schedule-project-moves). - - [Snippets](#bulk-schedule-snippet-moves). - - [Groups](#bulk-schedule-group-moves). **(PREMIUM SELF)** - -#### Bulk schedule project moves - -1. [Schedule repository storage moves for all projects on a storage shard](../../api/project_repository_storage_moves.md#schedule-repository-storage-moves-for-all-projects-on-a-storage-shard) using the API. For example: - - ```shell - curl --request POST --header "Private-Token: <your_access_token>" \ - --header "Content-Type: application/json" \ - --data '{"source_storage_name":"<original_storage_name>","destination_storage_name":"<cluster_storage_name>"}' \ - "https://gitlab.example.com/api/v4/project_repository_storage_moves" - ``` - -1. [Query the most recent repository moves](../../api/project_repository_storage_moves.md#retrieve-all-project-repository-storage-moves) - using the API. The query indicates either: - - The moves have completed successfully. The `state` field is `finished`. - - The moves are in progress. Re-query the repository move until it completes successfully. - - The moves have failed. Most failures are temporary and are solved by rescheduling the move. - -1. After the moves are complete, [query projects](../../api/projects.md#list-all-projects) - using the API to confirm that all projects have moved. No projects should be returned - with `repository_storage` field set to the old storage. - - ```shell - curl --header "Private-Token: <your_access_token>" --header "Content-Type: application/json" \ - "https://gitlab.example.com/api/v4/projects?repository_storage=<original_storage_name>" - ``` - - Alternatively use [the rails console](../operations/rails_console.md) to - confirm that all projects have moved. Run the following in the rails console: - - ```ruby - ProjectRepository.for_repository_storage('<original_storage_name>') - ``` - -1. Repeat for each storage as required. - -#### Bulk schedule snippet moves - -1. [Schedule repository storage moves for all snippets on a storage shard](../../api/snippet_repository_storage_moves.md#schedule-repository-storage-moves-for-all-snippets-on-a-storage-shard) using the API. For example: - - ```shell - curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ - --header "Content-Type: application/json" \ - --data '{"source_storage_name":"<original_storage_name>","destination_storage_name":"<cluster_storage_name>"}' \ - "https://gitlab.example.com/api/v4/snippet_repository_storage_moves" - ``` - -1. [Query the most recent repository moves](../../api/snippet_repository_storage_moves.md#retrieve-all-snippet-repository-storage-moves) - using the API. The query indicates either: - - The moves have completed successfully. The `state` field is `finished`. - - The moves are in progress. Re-query the repository move until it completes successfully. - - The moves have failed. Most failures are temporary and are solved by rescheduling the move. - -1. After the moves are complete, use [the rails console](../operations/rails_console.md) to - confirm that all snippets have moved. No snippets should be returned for the original - storage. Run the following in the rails console: - - ```ruby - SnippetRepository.for_repository_storage('<original_storage_name>') - ``` - -1. Repeat for each storage as required. - -#### Bulk schedule group moves **(PREMIUM SELF)** - -1. [Schedule repository storage moves for all groups on a storage shard](../../api/group_repository_storage_moves.md#schedule-repository-storage-moves-for-all-groups-on-a-storage-shard) using the API. - - ```shell - curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ - --header "Content-Type: application/json" \ - --data '{"source_storage_name":"<original_storage_name>","destination_storage_name":"<cluster_storage_name>"}' \ - "https://gitlab.example.com/api/v4/group_repository_storage_moves" - ``` - -1. [Query the most recent repository moves](../../api/group_repository_storage_moves.md#retrieve-all-group-repository-storage-moves) - using the API. The query indicates either: - - The moves have completed successfully. The `state` field is `finished`. - - The moves are in progress. Re-query the repository move until it completes successfully. - - The moves have failed. Most failures are temporary and are solved by rescheduling the move. - -1. After the moves are complete, use [the rails console](../operations/rails_console.md) to - confirm that all groups have moved. No groups should be returned for the original - storage. Run the following in the rails console: - - ```ruby - GroupWikiRepository.for_repository_storage('<original_storage_name>') - ``` - -1. Repeat for each storage as required. |