diff options
Diffstat (limited to 'doc/administration/gitaly')
| -rw-r--r-- | doc/administration/gitaly/configure_gitaly.md | 7 | ||||
| -rw-r--r-- | doc/administration/gitaly/index.md | 23 | ||||
| -rw-r--r-- | doc/administration/gitaly/praefect.md | 17 | ||||
| -rw-r--r-- | doc/administration/gitaly/recovery.md | 38 | ||||
| -rw-r--r-- | doc/administration/gitaly/troubleshooting.md | 2 |
5 files changed, 68 insertions, 19 deletions
diff --git a/doc/administration/gitaly/configure_gitaly.md b/doc/administration/gitaly/configure_gitaly.md index 5b868c274cd..ac03c3ffc02 100644 --- a/doc/administration/gitaly/configure_gitaly.md +++ b/doc/administration/gitaly/configure_gitaly.md @@ -1089,9 +1089,10 @@ lots of CI fetch traffic. The pack-objects cache wraps `git pack-objects`, an internal part of Git that gets invoked indirectly via the PostUploadPack and -SSHUploadPack Gitaly RPCs. These are the RPCs that Gitaly runs when a -user does a Git fetch via HTTP or SSH, respectively. When the cache is -enabled, anything that uses PostUploadPack or SSHUploadPack can +SSHUploadPack Gitaly RPCs. Gitaly runs PostUploadPack when a +user does a Git fetch via HTTP, or SSHUploadPack when a +user does a Git fetch via SSH. +When the cache is enabled, anything that uses PostUploadPack or SSHUploadPack can benefit from it. It is orthogonal to: - The transport (HTTP or SSH). diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index b053da7ac9b..c7f7c4c58a5 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -72,7 +72,7 @@ the current status of these issues, please refer to the referenced issues and ep |:--------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------| | Gitaly Cluster + Geo - Issues retrying failed syncs | If Gitaly Cluster is used on a Geo secondary site, repositories that have failed to sync could continue to fail when Geo tries to resync them. Recovering from this state requires assistance from support to run manual steps. | No known solution prior to GitLab 15.0. In GitLab 15.0 to 15.2, enable the [`gitaly_praefect_generated_replica_paths` feature flag](#praefect-generated-replica-paths-gitlab-150-and-later). In GitLab 15.3, the feature flag is enabled by default. | | Praefect unable to insert data into the database due to migrations not being applied after an upgrade | If the database is not kept up to date with completed migrations, then the Praefect node is unable to perform normal operation. | Make sure the Praefect database is up and running with all migrations completed (For example: `/opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-migrate-status` should show a list of all applied migrations). Consider [requesting upgrade assistance](https://about.gitlab.com/support/scheduling-upgrade-assistance/) so your upgrade plan can be reviewed by support. | -| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup, it's best to snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. | +| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup, it's best to [shut down GitLab](../read_only_gitlab.md#shut-down-the-gitlab-ui), snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. | ### Snapshot backup and recovery limitations @@ -598,6 +598,7 @@ To migrate to Gitaly Cluster: 1. Create the required storage. Refer to [repository storage recommendations](praefect.md#repository-storage-recommendations). 1. Create and configure [Gitaly Cluster](praefect.md). +1. Configure the existing Gitaly instance [to use TPC](praefect.md#use-tcp-for-existing-gitlab-instances), if not already configured that way. 1. [Move the repositories](../operations/moving_repositories.md#move-repositories). To migrate to Gitaly Cluster, existing repositories stored outside Gitaly Cluster must be moved. There is no automatic migration but the moves can be scheduled with the GitLab API. @@ -660,7 +661,14 @@ The code removed from GitLab during the Gitaly migration project affected these performance workaround for these NFS-based deployments, we re-introduced some of the old Rugged code. This re-introduced code is informally referred to as the "Rugged patches". -### How it works +### Automatic detection + +> Automatic detection for Rugged [disabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95445) in GitLab 15.3. + +FLAG: +On self-managed GitLab, by default automatic detection of whether Rugged should be used (per storage) is not available. +To make it available, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named +`skip_rugged_auto_detect`. The Ruby methods that perform direct Git access are behind [feature flags](../../development/gitaly.md#legacy-rugged-code), disabled by default. It wasn't @@ -685,12 +693,13 @@ To see if GitLab can access the repository file system directly, we use the foll - GitLab Rails tries to read the metadata file directly. If it exists, and if the UUID's match, assume we have direct access. -Versions of GitLab 15.3 and later disable direct Git access by default. +Direct Git access is: -For versions of GitLab prior to 15.3, direct Git access is enabled by -default in Omnibus GitLab because it fills in the correct repository -paths in the GitLab configuration file `config/gitlab.yml`. This -satisfies the UUID check. +- [Disabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95445) by default in GitLab 15.3 and later for + compatibility with [Praefect-generated replica paths](#praefect-generated-replica-paths-gitlab-150-and-later). It + can be enabled if Rugged [feature flags](../../development/gitaly.md#legacy-rugged-code) are enabled. +- Enabled by default in GitLab 15.2 and earlier because it fills in the correct repository paths in the GitLab + configuration file `config/gitlab.yml`. This satisfies the UUID check. ### Transition to Gitaly Cluster diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index 488e6a0df5f..bd03aa1bdbc 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -97,7 +97,7 @@ If you [installed](https://about.gitlab.com/install/) GitLab using the Omnibus G ### Preparation -Before beginning, you should already have a working GitLab instance. +Before beginning, you should already have a working GitLab instance. [Learn how to install GitLab](https://about.gitlab.com/install/). Provision a PostgreSQL server. We recommend using the PostgreSQL that is shipped @@ -164,7 +164,8 @@ The replication state is internal to each instance of GitLab and should not be replicated. These instructions help set up a single PostgreSQL database, which creates a single point of -failure. Alternatively, [you can use PostgreSQL replication and failover](../postgresql/replication_and_failover.md). +failure. To avoid this, you can configure your own clustered PostgreSQL. Support for PostgreSQL replication and failover using Omnibus GitLab is being tracked in +[a relevant epic](https://gitlab.com/groups/gitlab-org/-/epics/7814). The following options are available: @@ -221,7 +222,7 @@ For using Omnibus-provided PgBouncer you need to take the following additional s recommend using the PostgreSQL that is shipped with Omnibus as the backend. The following instructions only work on Omnibus-provided PostgreSQL: -1. For Omnibus-provided PgBouncer, you need to use the hash of `praefect` user instead the of the +1. For Omnibus-provided PgBouncer, you need to use the hash of `praefect` password instead the of the actual password: ```sql @@ -331,7 +332,7 @@ To configure the additional connection, you must either: #### Configure a new PgBouncer database with `pool_mode = session` We recommend using PgBouncer with `session` pool mode. You can use the -[bundled PgBouncer](../postgresql/pgbouncer.md) or use an external PgBouncer and +[bundled PgBouncer](../postgresql/pgbouncer.md) or use an external PgBouncer and [configure it manually](https://www.pgbouncer.org/config.html). The following example uses the bundled PgBouncer and sets up two separate connection pools on PostgreSQL host, @@ -620,7 +621,7 @@ Updates to example must be made at: gitlab-ctl reconfigure ``` -1. To ensure that Praefect +1. To ensure that Praefect [has updated its Prometheus listen address](https://gitlab.com/gitlab-org/gitaly/-/issues/2734), [restart Praefect](../restart_gitlab.md#omnibus-gitlab-restart): @@ -884,7 +885,7 @@ For more information on Gitaly server configuration, see our gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN' ``` -1. Configure and `internal_api_url`, which is also needed for `git push` operations: +1. Configure an `internal_api_url`, which is also needed for `git push` operations: ```ruby # Configure the gitlab-shell API callback URL. Without this, `git push` will @@ -928,7 +929,7 @@ For more information on Gitaly server configuration, see our gitlab-ctl reconfigure ``` -1. To ensure that Gitaly +1. To ensure that Gitaly [has updated its Prometheus listen address](https://gitlab.com/gitlab-org/gitaly/-/issues/2734), [restart Gitaly](../restart_gitlab.md#omnibus-gitlab-restart): @@ -1114,7 +1115,7 @@ Particular attention should be shown to: 1. Check that the Praefect storage is configured to store new repositories: - 1. On the top bar, select **Menu > Admin**. + 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > Repository**. 1. Expand the **Repository storage** section. diff --git a/doc/administration/gitaly/recovery.md b/doc/administration/gitaly/recovery.md index bd4846a986d..4bbf25d7cdd 100644 --- a/doc/administration/gitaly/recovery.md +++ b/doc/administration/gitaly/recovery.md @@ -430,6 +430,44 @@ This command fails if: - The repository is already being tracked by the Praefect database. - The repository does not exist on disk. +### Manually track large numbers of repositories + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/6319) in GitLab 15.4. + +The `track-repositories` Praefect sub-command adds large batches of on-disk repositories to the Praefect database for tracking. This can +be useful when migrating an existing instance to new infrastructure and ingesting all existing repositories into a fresh Gitaly Cluster. + +```shell +# Omnibus GitLab install +sudo gitlab-ctl praefect track-repositories --input-path /path/to/input.json + +# Source install +sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml track-repositories -input-path /path/to/input.json +``` + +The command validates that all entries: + +- Are formatted correctly and contain required fields. +- Correspond to a valid Git repository on disk. +- Are not currently tracked in the Praefect database. + +If any entry fails these checks, the command aborts prior to attempting to track a repository. + +- `input-path` is the path to a file containing a list of repositories formatted as newline-delimited JSON objects. Objects must contain the following keys: + - `relative_path`: corresponds with `repository` in [track-repositories](#manually-track-repositories). + - `authoritative-storage`: the storage Praefect is to treat as the primary. + - `virtual-storage`: the virtual storage the repository is located in. + + For example: + + ```json + {"relative_path":"@hashed/f5/ca/f5ca38f748a1d6eaf726b8a42fb575c3c71f1864a8143301782de13da2d9202b.git","authoritative_storage":"gitaly-1","virtual_storage":"default"} + {"relative_path":"@hashed/f8/9f/f89f8d0e735a91c5269ab08d72fa27670d000e7561698d6e664e7b603f5c4e40.git","authoritative_storage":"gitaly-2","virtual_storage":"default"} + ``` + +- `-replicate-immediately`, causes the command to replicate the repository to its secondaries immediately. + Otherwise, replication jobs are scheduled for execution in the database and are picked up by a Praefect background process. + ### List virtual storage details > [Introduced](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4609) in GitLab 15.1. diff --git a/doc/administration/gitaly/troubleshooting.md b/doc/administration/gitaly/troubleshooting.md index e308fa9da43..285ec3d2631 100644 --- a/doc/administration/gitaly/troubleshooting.md +++ b/doc/administration/gitaly/troubleshooting.md @@ -20,7 +20,7 @@ and our advice on [parsing the `gitaly/current` file](../logs/log_parsing.md#par When using standalone Gitaly servers, you must make sure they are the same version as GitLab to ensure full compatibility: -1. On the top bar, select **Menu > Admin** on your GitLab instance. +1. On the top bar, select **Main menu > Admin** on your GitLab instance. 1. On the left sidebar, select **Overview > Gitaly Servers**. 1. Confirm all Gitaly servers indicate that they are up to date. |