diff options
Diffstat (limited to 'doc/administration/gitaly/praefect.md')
| -rw-r--r-- | doc/administration/gitaly/praefect.md | 68 |
1 files changed, 51 insertions, 17 deletions
diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index fb9f3e9c361..f6166d713b1 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -19,9 +19,6 @@ Configure Gitaly Cluster using either: Smaller GitLab installations may need only [Gitaly itself](index.md). -To upgrade a Gitaly Cluster, follow the documentation for -[zero downtime upgrades](../../update/zero_downtime.md#gitaly-cluster). - ## Requirements The minimum recommended configuration for a Gitaly Cluster requires: @@ -54,7 +51,7 @@ Achieving acceptable latency between Gitaly nodes: are designed for this type of synchronization. Latency of less than 2ms should be sufficient for Gitaly Cluster. If you can't provide low network latencies for replication (for example, between distant locations), consider Geo. For -more information, see [How does Gitaly Cluster compare to Geo](faq.md#how-does-gitaly-cluster-compare-to-geo). +more information, see [Comparison to Geo](index.md#comparison-to-geo). Gitaly Cluster [components](index.md#components) communicate with each other over many routes. Your firewall rules must allow the following for Gitaly Cluster to function properly: @@ -64,6 +61,7 @@ allow the following for Gitaly Cluster to function properly: | GitLab | Praefect load balancer | `2305` | `3305` | | Praefect load balancer | Praefect | `2305` | `3305` | | Praefect | Gitaly | `8075` | `9999` | +| Praefect | GitLab (internal API) | `80` | `443` | | Gitaly | GitLab (internal API) | `80` | `443` | | Gitaly | Praefect load balancer | `2305` | `3305` | | Gitaly | Praefect | `2305` | `3305` | @@ -74,6 +72,16 @@ Gitaly does not directly connect to Praefect. However, requests from Gitaly to t load balancer may still be blocked unless firewalls on the Praefect nodes allow traffic from the Gitaly nodes. +### Praefect database storage + +The requirements are relatively low because the database contains only metadata of: + +- Where repositories are located. +- Some queued work. + +It depends on the number of repositories, but a useful minimum is 5-10 GB, similar to the main +GitLab application database. + ## Setup Instructions If you [installed](https://about.gitlab.com/install/) GitLab using the Omnibus GitLab package @@ -168,6 +176,18 @@ The following options are available: - Use a cloud-managed PostgreSQL service. AWS [Relational Database Service](https://aws.amazon.com/rds/) is recommended. +Setting up PostgreSQL creates empty Praefect tables. For more information, see the +[relevant troubleshooting section](troubleshooting.md#relation-does-not-exist-errors). + +#### Running GitLab and Praefect databases on the same server + +The GitLab application database and the Praefect database can be run on the same server. However, Praefect should have +its own database server when using Omnibus GitLab PostgreSQL. If there is a failover, Praefect isn't aware and starts to +fail as the database it's trying to use would either: + +- Be unavailable. +- In read-only mode. + #### Manual database setup To complete this section you need: @@ -278,8 +298,12 @@ reads distribution caching is enabled by configuration #### Use PgBouncer To reduce PostgreSQL resource consumption, we recommend setting up and configuring -[PgBouncer](https://www.pgbouncer.org/) in front of the PostgreSQL instance. To do -this, you must point Praefect to PgBouncer by setting database parameters on Praefect configuration: +[PgBouncer](https://www.pgbouncer.org/) in front of the PostgreSQL instance. However, PgBouncer isn't required because +Praefect makes a low number of connections. If you choose to use PgBouncer, you can use the same PgBouncer instance for +both the GitLab application database and the Praefect database. + +To configure PgBouncer in front of the PostgreSQL instance, you must point Praefect to PgBouncer by setting database +parameters on Praefect configuration: ```ruby praefect['database_host'] = PGBOUNCER_HOST @@ -475,7 +499,7 @@ Updates to example must be made at: # Some metrics run queries against the database. Enabling separate database metrics allows # these metrics to be collected when the metrics are - # scraped on a separate /db_metrics endpoint. + # scraped on a separate /db_metrics endpoint. praefect['separate_database_metrics'] = true ``` @@ -519,7 +543,7 @@ Updates to example must be made at: WARNING: If you have data on an already existing storage called `default`, you should configure the virtual storage with another name and - [migrate the data to the Gitaly Cluster storage](index.md#migrating-to-gitaly-cluster) + [migrate the data to the Gitaly Cluster storage](index.md#migrate-to-gitaly-cluster) afterwards. Replace `PRAEFECT_INTERNAL_TOKEN` with a strong secret, which is used by @@ -632,12 +656,8 @@ and on all Praefect clients that communicate with it following the procedure des Note the following: -- The certificate must specify the address you use to access the Praefect server. If - addressing the Praefect server by: - - - Hostname, you can either use the Common Name field for this, or add it as a Subject - Alternative Name. - - IP address, you must add it as a Subject Alternative Name to the certificate. +- The certificate must specify the address you use to access the Praefect server. You must add the hostname or IP + address as a Subject Alternative Name to the certificate. - When running Praefect sub-commands such as `dial-nodes` and `list-untracked-repositories` from the command line with [Gitaly TLS enabled](configure_gitaly.md#enable-tls-support), you must set the `SSL_CERT_DIR` or `SSL_CERT_FILE` environment variable so that the Gitaly certificate is trusted. For example: @@ -651,6 +671,8 @@ Note the following: This allows you to do a gradual transition from unencrypted to encrypted traffic, if necessary. + To disable the unencrypted listener, set `praefect['listen_addr'] = nil`. + To configure Praefect with TLS: **For Omnibus GitLab** @@ -992,7 +1014,7 @@ Particular attention should be shown to: WARNING: If you have existing data stored on the default Gitaly storage, - you should [migrate the data your Gitaly Cluster storage](index.md#migrating-to-gitaly-cluster) + you should [migrate the data your Gitaly Cluster storage](index.md#migrate-to-gitaly-cluster) first. ```ruby @@ -1107,7 +1129,7 @@ Particular attention should be shown to: #### Use TCP for existing GitLab instances When adding Gitaly Cluster to an existing Gitaly instance, the existing Gitaly storage -must use a TCP address. If `gitaly_address` is not specified, then a Unix socket is used, +must be listening on TCP/TLS. If `gitaly_address` is not specified, then a Unix socket is used, which prevents the communication with the cluster. For example: @@ -1116,7 +1138,7 @@ For example: git_data_dirs({ 'default' => { 'gitaly_address' => 'tcp://old-gitaly.internal:8075' }, 'cluster' => { - 'gitaly_address' => 'tcp://<PRAEFECT_LOADBALANCER_HOST>:2305', + 'gitaly_address' => 'tls://<PRAEFECT_LOADBALANCER_HOST>:3305', 'gitaly_token' => '<praefect_external_token>' } }) @@ -1223,6 +1245,18 @@ You can configure: If `default_replication_factor` is unset, the repositories are always replicated on every node defined in `virtual_storages`. If a new node is introduced to the virtual storage, both new and existing repositories are replicated to the node automatically. +### Repository storage recommendations + +The size of the required storage can vary between instances and depends on the set +[replication factor](index.md#replication-factor). You might want to include implementing +repository storage redundancy. + +For a replication factor: + +- Of `1`: NFS, Gitaly, and Gitaly Cluster have roughly the same storage requirements. +- More than `1`: The amount of required storage is `used space * replication factor`. `used space` + should include any planned future growth. + ## Repository verification > [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/4080) in GitLab 15.0. |