diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-03-03 21:08:16 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-03-03 21:08:16 +0300 |
commit | e9c2bf267862e22c0770cc7b3a1ed97a8b87a7fd (patch) | |
tree | 7b778e44f210132af1233ceb8801b388ac3519f5 /doc | |
parent | 946771d0b016ae92b15a60bc3290a33b94191ffe (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r-- | doc/administration/audit_events.md | 2 | ||||
-rw-r--r-- | doc/administration/gitaly/img/praefect_architecture_v12_9.png | bin | 0 -> 158447 bytes | |||
-rw-r--r-- | doc/administration/gitaly/praefect.md | 756 | ||||
-rw-r--r-- | doc/api/commits.md | 18 | ||||
-rw-r--r-- | doc/api/groups.md | 2 |
5 files changed, 477 insertions, 301 deletions
diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index 2f4079396e6..5b3052789e4 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -79,6 +79,8 @@ From there, you can see the following actions: - Release was added to a project - Release was updated - Release milestone associations changed +- Permission to approve merge requests by committers was updated ([introduced](https://gitlab.com/gitlab-org/gitlab/issues/7531) in GitLab 12.9) +- Permission to approve merge requests by authors was updated ([introduced](https://gitlab.com/gitlab-org/gitlab/issues/7531) in GitLab 12.9) ### Instance events **(PREMIUM ONLY)** diff --git a/doc/administration/gitaly/img/praefect_architecture_v12_9.png b/doc/administration/gitaly/img/praefect_architecture_v12_9.png Binary files differnew file mode 100644 index 00000000000..3937789094c --- /dev/null +++ b/doc/administration/gitaly/img/praefect_architecture_v12_9.png diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index 7f314bc5e31..20c57683708 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -1,359 +1,525 @@ -# Praefect +# Praefect: High Availability -NOTE: **Note:** Praefect is an experimental service, and for testing purposes only at -this time. +NOTE: **Note:** Praefect is an experimental service, and data loss is likely. Praefect is an optional reverse-proxy for [Gitaly](../index.md) to manage a -cluster of Gitaly nodes for high availability through replication. -If a Gitaly node becomes unavailable, it will be possible to fail over to a -warm Gitaly replica. +cluster of Gitaly nodes for high availability. Initially, high availability +be implemented through asynchronous replication. If a Gitaly node becomes +unavailable, it will be possible to fail over to a warm Gitaly replica. The first minimal version will support: - Eventual consistency of the secondary replicas. -- Manual fail over from the primary to the secondary. +- Automatic fail over from the primary to the secondary. +- Reporting of possible data loss if replication queue is non empty. Follow the [HA Gitaly epic](https://gitlab.com/groups/gitlab-org/-/epics/1489) for updates and roadmap. -## Omnibus +## Requirements for configuring Gitaly for High Availability -### Architecture +NOTE: **Note:** this reference architecture is not highly available because +Praefect is a single point of failure. -The most common architecture for Praefect is simplified in the diagram below: +The minimal [alpha](https://about.gitlab.com/handbook/product/#alpha-beta-ga) +reference architecture additionally requires: -```mermaid -graph TB - GitLab --> Praefect; - Praefect --- PostgreSQL; - Praefect --> Gitaly1; - Praefect --> Gitaly2; - Praefect --> Gitaly3; -``` +- 1 Praefect node +- 1 PostgreSQL server (PostgreSQL 9.6 or newer) +- 3 Gitaly nodes (1 primary, 2 secondary) + +![Alpha architecture diagram](img/praefect_architecture_v12_9.png) + +See the [design +document](https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/design_ha.md) +for implementation details. + +## Setup Instructions -Where `GitLab` is the collection of clients that can request Git operations. -The Praefect node has three storage nodes attached. Praefect itself doesn't -store data, but connects to three Gitaly nodes, `Gitaly-1`, `Gitaly-2`, and `Gitaly-3`. +If you [installed](https://about.gitlab.com/install/) GitLab using the Omnibus +package (highly recommended), follow the steps below: -In order to keep track of replication state, Praefect relies on a -PostgreSQL database. This database is a single point of failure so you -should use a highly available PostgreSQL server for this. GitLab -itself needs a HA PostgreSQL server too, so you could optionally co-locate the Praefect -SQL database on the PostgreSQL server you use for the rest of GitLab. +1. [Preparation](#preparation) +1. [Configuring the Praefect database](#postgresql) +1. [Configuring the Praefect proxy/router](#praefect) +1. [Configuring each Gitaly node](#gitaly) (once for each Gitaly node) +1. [Updating the GitLab server configuration](#gitlab) -Praefect may be enabled on its own node or can be run on the GitLab server. -In the example below we will use a separate server, but the optimal configuration -for Praefect is still being determined. +### Preparation -Praefect will handle all Gitaly RPC requests to its child nodes. However, the child nodes -will still need to communicate with the GitLab server via its internal API for authentication -purposes. +Before beginning, you should already have a working GitLab instance. [Learn how +to install GitLab](https://about.gitlab.com/install/). -### Setup +Provision a PostgreSQL server (PostgreSQL 9.6 or newer). Configuration through +the GitLab Omnibus distribution is not yet supported. Follow this +[issue](https://gitlab.com/gitlab-org/gitaly/issues/2476) for updates. -In this setup guide we will start by configuring Praefect, then its child -Gitaly nodes, and lastly the GitLab server configuration. +Prepare all your new nodes by [installing +GitLab](https://about.gitlab.com/install/). + +- 1 Praefect node (minimal storage required) +- 3 Gitaly nodes (high CPU, high memory, fast storage) + +You will need the IP/host address for each node. + +1. `POSTGRESQL_SERVER_ADDRESS`: the IP/host address of the PostgreSQL server +1. `PRAEFECT_SERVER_ADDRESS`: the IP/host address of the Praefect server +1. `GITALY_SERVER_ADDRESS`: the IP/host address of each Gitaly node #### Secrets -We need to manage the following secrets and make them match across hosts: - -1. `GITLAB_SHELL_SECRET_TOKEN`: this is used by Git hooks to make - callback HTTP API requests to GitLab when accepting a Git push. This - secret is shared with GitLab Shell for legacy reasons. -1. `PRAEFECT_EXTERNAL_TOKEN`: repositories hosted on your Praefect - cluster can only be accessed by Gitaly clients that carry this - token. -1. `PRAEFECT_INTERNAL_TOKEN`: this token is used for replication - traffic inside your Praefect cluster. This is distinct from - `PRAEFECT_EXTERNAL_TOKEN` because Gitaly clients must not be able to - access internal nodes of the Praefect cluster directly; that could - lead to data loss. +The communication between components is secured with different secrets, which +are described below. Before you begin, generate a unique secret for each, and +make note of it. This will make it easy to replace these placeholder tokens +with secure tokens as you complete the setup process. + +1. `GITLAB_SHELL_SECRET_TOKEN`: this is used by Git hooks to make callback HTTP + API requests to GitLab when accepting a Git push. This secret is shared with + GitLab Shell for legacy reasons. +1. `PRAEFECT_EXTERNAL_TOKEN`: repositories hosted on your Praefect cluster can + only be accessed by Gitaly clients that carry this token. +1. `PRAEFECT_INTERNAL_TOKEN`: this token is used for replication traffic inside + your Praefect cluster. This is distinct from `PRAEFECT_EXTERNAL_TOKEN` + because Gitaly clients must not be able to access internal nodes of the + Praefect cluster directly; that could lead to data loss. 1. `PRAEFECT_SQL_PASSWORD`: this password is used by Praefect to connect to - PostgreSQL. + PostgreSQL. We will note in the instructions below where these secrets are required. -#### Network addresses +### PostgreSQL -1. `POSTGRESQL_SERVER_ADDRESS`: the host name or IP address of your PostgreSQL server +NOTE: **Note:** don't reuse the GitLab application database for the Praefect +database. -#### PostgreSQL +To complete this section you will need: -To set up a Praefect cluster you need a highly available PostgreSQL -server. You need PostgreSQL 9.6 or newer. Praefect needs to have a SQL -user with the right to create databases. +- 1 Praefect node +- 1 PostgreSQL server (PostgreSQL 9.6 or newer) + - An SQL user with permissions to create databases -In the instructions below we assume you have administrative access to -your PostgreSQL server via `psql`. Depending on your environment, you -may also be able to do this via the web interface of your cloud -platform, or via your configuration management system, etc. +During this section, we will configure the PostgreSQL server, from the Praefect +node, using `psql` which is installed by GitLab Omnibus. -Below we assume that you have administrative access as the `postgres` -user. First open a `psql` session as the `postgres` user: +1. SSH into the **Praefect** node and login as root: -```shell -/opt/gitlab/embedded/bin/psql -h POSTGRESQL_SERVER_ADDRESS -U postgres -d template1 -``` + ```shell + sudo -i + ``` -Once you are connected, run the following command. Replace -`PRAEFECT_SQL_PASSWORD` with the actual (random) password you -generated for the `praefect` SQL user: +1. Connect to the PostgreSQL server with administrative access. This is likely + the `postgres` user. The database `template1` is used because it is created + by default on all PostgreSQL servers. -```sql -CREATE ROLE praefect WITH LOGIN CREATEDB PASSWORD 'PRAEFECT_SQL_PASSWORD'; -\q -``` + ```shell + /opt/gitlab/embedded/bin/psql -U postgres -d template1 -h POSTGRESQL_SERVER_ADDRESS + ``` -Now connect as the `praefect` user to create the database. This has -the side effect of verifying that you have access: + Create a new user `praefect` which will be used by Praefect. Replace + `PRAEFECT_SQL_PASSWORD` with the strong password you generated in the + preparation step. -```shell -/opt/gitlab/embedded/bin/psql -h POSTGRESQL_SERVER_ADDRESS -U praefect -d template1 -``` + ```sql + CREATE ROLE praefect WITH LOGIN CREATEDB PASSWORD 'PRAEFECT_SQL_PASSWORD'; + ``` -Once you have connected as the `praefect` user, run: +1. Reconnect to the PostgreSQL server, this time as the `praefect` user: -```sql -CREATE DATABASE praefect_production WITH ENCODING=UTF8; -\q -``` + ```shell + /opt/gitlab/embedded/bin/psql -U praefect -d template1 -h POSTGRESQL_SERVER_ADDRESS + ``` -#### Praefect - -On the Praefect node we disable all other services, including Gitaly. We list each -Gitaly node that will be connected to Praefect as members of the `praefect` hash in `praefect['virtual_storages']`. - -In the example below, the Gitaly nodes are named `gitaly-N`. Note that one -node is designated as primary by setting the primary to `true`. - -If you are using an uncrypted connection to Postgres, set `praefect['database_sslmode']` to false. - -If you are using an encrypted connection with a client certificate, -`praefect['database_sslcert']` and `praefect['database_sslkey']` will need to be set. -If you are using a custom CA, also set `praefect['database_sslrootcert']`: - -```ruby -# /etc/gitlab/gitlab.rb on praefect server - -# Avoid running unnecessary services on the Gitaly server -postgresql['enable'] = false -redis['enable'] = false -nginx['enable'] = false -prometheus['enable'] = false -grafana['enable'] = false -unicorn['enable'] = false -sidekiq['enable'] = false -gitlab_workhorse['enable'] = false -gitaly['enable'] = false - -# Prevent database connections during 'gitlab-ctl reconfigure' -gitlab_rails['rake_cache_clear'] = false -gitlab_rails['auto_migrate'] = false - -praefect['enable'] = true - -# Make Praefect accept connections on all network interfaces. You must use -# firewalls to restrict access to this address/port. -praefect['listen_addr'] = '0.0.0.0:2305' - -# Replace PRAEFECT_EXTERNAL_TOKEN with a real secret -praefect['auth_token'] = 'PRAEFECT_EXTERNAL_TOKEN' - -# Replace each instance of PRAEFECT_INTERNAL_TOKEN below with a real -# secret, distinct from PRAEFECT_EXTERNAL_TOKEN. -# Name of storage hash must match storage name in git_data_dirs on GitLab server. -praefect['virtual_storages'] = { - 'praefect' => { - 'gitaly-1' => { - # Replace GITALY_URL_OR_IP below with the real address to connect to. - 'address' => 'tcp://GITALY_URL_OR_IP:8075', - 'token' => 'PRAEFECT_INTERNAL_TOKEN', - 'primary' => true - }, - 'gitaly-2' => { - # Replace GITALY_URL_OR_IP below with the real address to connect to. - 'address' => 'tcp://GITALY_URL_OR_IP:8075', - 'token' => 'PRAEFECT_INTERNAL_TOKEN' - }, - 'gitaly-3' => { - # Replace GITALY_URL_OR_IP below with the real address to connect to. - 'address' => 'tcp://GITALY_URL_OR_IP:8075', - 'token' => 'PRAEFECT_INTERNAL_TOKEN' - } - } -} - -# Replace POSTGRESQL_SERVER below with a real IP/host address of the database. -praefect['database_host'] = 'POSTGRESQL_SERVER_ADDRESS' -praefect['database_port'] = 5432 -praefect['database_user'] = 'praefect' -# Replace PRAEFECT_SQL_PASSWORD below with a real password of the database. -praefect['database_password'] = 'PRAEFECT_SQL_PASSWORD' -praefect['database_dbname'] = 'praefect_production' - -# Uncomment the line below if you do not want to use an encrypted -# connection to PostgreSQL -# praefect['database_sslmode'] = 'disable' - -# Uncomment and modify these lines if you are using a TLS client -# certificate to connect to PostgreSQL -# praefect['database_sslcert'] = '/path/to/client-cert' -# praefect['database_sslkey'] = '/path/to/client-key' - -# Uncomment and modify this line if your PostgreSQL server uses a custom -# CA -# praefect['database_sslrootcert'] = '/path/to/rootcert' -``` + Create a new database `praefect_production`. By creating the database while + connected as the `praefect` user, we are confident they have access. -Replace `POSTGRESQL_SERVER_ADDRESS`, `PRAEFECT_EXTERNAL_TOKEN`, `PRAEFECT_INTERNAL_TOKEN`, -and `PRAEFECT_SQL_PASSWORD` with their respective values. + ```sql + CREATE DATABASE praefect_production WITH ENCODING=UTF8; + ``` -Save the file and reconfigure Praefect: +The database used by Praefect is now configured. -```shell -sudo gitlab-ctl reconfigure -``` +### Praefect -After you reconfigure, verify that Praefect can reach PostgreSQL: +To complete this section you will need: -```shell -sudo -u git /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-ping -``` +- [Configured PostgreSQL server](#postgresql), including: + - IP/host address (`POSTGRESQL_SERVER_ADDRESS`) + - password (`PRAEFECT_SQL_PASSWORD`) -If the check fails, make sure you have followed the steps correctly. If you edit `/etc/gitlab/gitlab.rb`, -remember to run `sudo gitlab-ctl reconfigure` again before trying the -`sql-ping` command. - -#### Gitaly - -Next we will configure each Gitaly server assigned to Praefect. Configuration for these -is the same as a normal standalone Gitaly server, except that we use storage names and -auth tokens from Praefect instead of GitLab. - -Below is an example configuration for `gitaly-1`, the only difference for the -other Gitaly nodes is the storage name under `git_data_dirs`. - -Note that `gitaly['auth_token']` matches the `token` value listed under `praefect['virtual_storages']` -on the Praefect node. - -```ruby -# /etc/gitlab/gitlab.rb on gitaly node inside praefect cluster - -# Avoid running unnecessary services on the Gitaly server -postgresql['enable'] = false -redis['enable'] = false -nginx['enable'] = false -prometheus['enable'] = false -grafana['enable'] = false -unicorn['enable'] = false -sidekiq['enable'] = false -gitlab_workhorse['enable'] = false -prometheus_monitoring['enable'] = false - -# Prevent database connections during 'gitlab-ctl reconfigure' -gitlab_rails['rake_cache_clear'] = false -gitlab_rails['auto_migrate'] = false - -# Replace GITLAB_SHELL_SECRET_TOKEN below with real secret -gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN' - -# Configure the gitlab-shell API callback URL. Without this, `git push` will -# fail. This can be your 'front door' GitLab URL or an internal load -# balancer. -# Possible values could be: 'http://10.23.101.53', 'https://gitlab.example.com', -# etc. Please replace GITLAB_SERVER_ADDRESS with proper value and change schema -# to 'https' in case you use encrypted connection. -gitlab_rails['internal_api_url'] = 'http://GITLAB_SERVER_ADDRESS' - -# Replace PRAEFECT_INTERNAL_TOKEN below with a real secret. -gitaly['auth_token'] = 'PRAEFECT_INTERNAL_TOKEN' - -# Make Gitaly accept connections on all network interfaces. You must use -# firewalls to restrict access to this address/port. -# Comment out following line if you only want to support TLS connections -gitaly['listen_addr'] = "0.0.0.0:8075" - -git_data_dirs({ - # Update this to the name of this Gitaly server which will be later - # exposed in the UI under "Admin area > Gitaly" - "gitaly-1" => { - "path" => "/var/opt/gitlab/git-data" - } -}) -``` +Praefect should be run on a dedicated node. Do not run Praefect on the +application server, or a Gitaly node. -Replace `GITLAB_SHELL_SECRET_TOKEN` and `PRAEFECT_INTERNAL_TOKEN` -with their respective values. +1. SSH into the **Praefect** node and login as root: -For more information on Gitaly server configuration, see our [Gitaly documentation](index.md#3-gitaly-server-configuration). + ```shell + sudo -i + ``` -When finished editing the configuration file for each Gitaly server, run the -reconfigure command to put changes into effect: +1. Disable all other services by editing `/etc/gitlab/gitlab.rb`: -```shell -sudo gitlab-ctl reconfigure -``` + ```ruby + # Disable all other services on the Praefect node + postgresql['enable'] = false + redis['enable'] = false + nginx['enable'] = false + prometheus['enable'] = false + grafana['enable'] = false + unicorn['enable'] = false + sidekiq['enable'] = false + gitlab_workhorse['enable'] = false + gitaly['enable'] = false + + # Enable only the Praefect service + praefect['enable'] = true + + # Prevent database connections during 'gitlab-ctl reconfigure' + gitlab_rails['rake_cache_clear'] = false + gitlab_rails['auto_migrate'] = false + ``` + +1. Configure **Praefect** to listen on network interfaces by editing + `/etc/gitlab/gitlab.rb`: + + ```ruby + # Make Praefect accept connections on all network interfaces. + # Use firewalls to restrict access to this address/port. + praefect['listen_addr'] = '0.0.0.0:2305' + ``` + +1. Configure a strong `auth_token` for **Praefect** by editing + `/etc/gitlab/gitlab.rb`. This will be needed by clients outside the cluster + (like GitLab Shell) to communicate with the Praefect cluster : + + ```ruby + praefect['auth_token'] = 'PRAEFECT_EXTERNAL_TOKEN' + ``` + +1. Configure **Praefect** to connect to the PostgreSQL database by editing + `/etc/gitlab/gitlab.rb`. + + You will need to replace `POSTGRESQL_SERVER_ADDRESS` with the IP/host address + of the database, and `PRAEFECT_SQL_PASSWORD` with the strong password set + above. + + ```ruby + praefect['database_host'] = 'POSTGRESQL_SERVER_ADDRESS' + praefect['database_port'] = 5432 + praefect['database_user'] = 'praefect' + praefect['database_password'] = 'PRAEFECT_SQL_PASSWORD' + praefect['database_dbname'] = 'praefect_production' + ``` + + If you want to use a TLS client certificate, the options below can be used: + + ```ruby + # Connect to PostreSQL using a TLS client certificate + # praefect['database_sslcert'] = '/path/to/client-cert' + # praefect['database_sslkey'] = '/path/to/client-key' -When all Gitaly servers are configured, you can run the Praefect connection + # Trust a custom certificate authority + # praefect['database_sslrootcert'] = '/path/to/rootcert' + ``` + + By default Praefect will refuse to make an unencrypted connection to + PostgreSQL. You can override this by uncommenting the following line: + + ```ruby + # praefect['database_sslmode'] = 'disable' + ``` + +1. Configure the **Praefect** cluster to connect to each Gitaly node in the + cluster by editing `/etc/gitlab/gitlab.rb`. + + In the example below we have configured one cluster named `praefect`. This + cluster has three Gitaly nodes `gitaly-1`, `gitaly-2`, and `gitaly-3`, which + will be replicas of each other. + + Replace `PRAEFECT_INTERNAL_TOKEN` with a strong secret, which will be used by + Praefect when communicating with Gitaly nodes in the cluster. This token is + distinct from the `PRAEFECT_EXTERNAL_TOKEN`. + + Replace `GITALY_HOST` with the IP/host address of the each Gitaly node. + + More Gitaly nodes can be added to the cluster to increase the number of + replicas. More clusters can also be added for very large GitLab instances. + + NOTE: **Note:** The `gitaly-1` node is currently denoted the primary. This + can be used to manually fail from one node to another. This will be removed + in the future to allow for automatic failover. + + ```ruby + # Name of storage hash must match storage name in git_data_dirs on GitLab + # server ('praefect') and in git_data_dirs on Gitaly nodes ('gitaly-1') + praefect['virtual_storages'] = { + 'praefect' => { + 'gitaly-1' => { + 'address' => 'tcp://GITALY_HOST:8075', + 'token' => 'PRAEFECT_INTERNAL_TOKEN', + 'primary' => true + }, + 'gitaly-2' => { + 'address' => 'tcp://GITALY_HOST:8075', + 'token' => 'PRAEFECT_INTERNAL_TOKEN' + }, + 'gitaly-3' => { + 'address' => 'tcp://GITALY_HOST:8075', + 'token' => 'PRAEFECT_INTERNAL_TOKEN' + } + } + } + ``` + +1. Save the changes to `/etc/gitlab/gitlab.rb` and [reconfigure Praefect](../restart_gitlab.md#omnibus-gitlab-reconfigure): + + ```shell + sudo gitlab-ctl reconfigure + ``` + +1. Verify that Praefect can reach PostgreSQL: + + ```shell + sudo -u git /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-ping + ``` + + If the check fails, make sure you have followed the steps correctly. If you + edit `/etc/gitlab/gitlab.rb`, remember to run `sudo gitlab-ctl reconfigure` + again before trying the `sql-ping` command. + +### Gitaly + +NOTE: **Note:** Complete these steps for **each** Gitaly node. + +To complete this section you will need: + +- [Configured Praefect node](#praefect) +- 3 (or more) servers, with GitLab installed, to be configured as Gitaly nodes. + These should be dedicated nodes, do not run other services on these nodes. + +Every Gitaly server assigned to the Praefect cluster needs to be configured. The +configuration is the same as a normal [standalone Gitaly server](../index.md), +except: + +- the storage names are exposed to Praefect, not GitLab +- the secret token is shared with Praefect, not GitLab + +The configuration of all Gitaly nodes in the Praefect cluster can be identical, +because we rely on Praefect to route operations correctly. + +Particular attention should be shown to: + +- the `gitaly['auth_token']` configured in this section must match the `token` + value under `praefect['virtual_storages']` on the Praefect node. This was set + in the [previous section](#praefect). This document uses the placeholder + `PRAEFECT_INTERNAL_TOKEN` throughout. +- the storage names in `git_data_dirs` configured in this section must match the + storage names under `praefect['virtual_storages']` on the Praefect node. This + was set in the [previous section](#praefect). This document uses `gitaly-1`, + `gitaly-2`, and `gitaly-3` as Gitaly storage names. + +For more information on Gitaly server configuration, see our [Gitaly +documentation](index.md#3-gitaly-server-configuration). + +1. SSH into the **Gitaly** node and login as root: + + ```shell + sudo -i + ``` + +1. Disable all other services by editing `/etc/gitlab/gitlab.rb`: + + ```ruby + # Disable all other services on the Praefect node + postgresql['enable'] = false + redis['enable'] = false + nginx['enable'] = false + prometheus['enable'] = false + grafana['enable'] = false + unicorn['enable'] = false + sidekiq['enable'] = false + gitlab_workhorse['enable'] = false + prometheus_monitoring['enable'] = false + + # Enable only the Praefect service + gitaly['enable'] = true + + # Prevent database connections during 'gitlab-ctl reconfigure' + gitlab_rails['rake_cache_clear'] = false + gitlab_rails['auto_migrate'] = false + ``` + +1. Configure **Gitaly** to listen on network interfaces by editing + `/etc/gitlab/gitlab.rb`: + + ```ruby + # Make Gitaly accept connections on all network interfaces. + # Use firewalls to restrict access to this address/port. + gitaly['listen_addr'] = '0.0.0.0:8075' + ``` + +1. Configure a strong `auth_token` for **Gitaly** by editing + `/etc/gitlab/gitlab.rb`. This will be needed by clients to communicate with + this Gitaly nodes. Typically, this token will be the same for all Gitaly + nodes. + + ```ruby + gitaly['auth_token'] = 'PRAEFECT_INTERNAL_TOKEN' + ``` + +1. Configure the GitLab Shell `secret_token`, and `internal_api_url` which are + needed for `git push` operations. + + If you have already configured [Gitaly on its own server](../index.md) + + ```ruby + gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN' + + # Configure the gitlab-shell API callback URL. Without this, `git push` will + # fail. This can be your front door GitLab URL or an internal load balancer. + # Examples: 'https://example.gitlab.com', 'http://1.2.3.4' + gitlab_rails['internal_api_url'] = 'GITLAB_SERVER_URL' + ``` + +1. Configure the storage location for Git data by setting `git_data_dirs` in + `/etc/gitlab/gitlab.rb`. Each Gitaly node should have a unique storage name + (eg `gitaly-1`). + + Instead of configuring `git_data_dirs` uniquely for each Gitaly node, it is + often easier to have include the configuration for all Gitaly nodes on every + Gitaly node. This is supported because the Praefect `virtual_storages` + configuration maps each storage name (eg `gitaly-1`) to a specific node, and + requests are routed accordingly. This means every Gitaly node in your fleet + can share the same configuration. + + ```ruby + # You can include the data dirs for all nodes in the same config, because + # Praefect will only route requests according to the addresses provided in the + # prior step. + git_data_dirs({ + "gitaly-1" => { + "path" => "/var/opt/gitlab/git-data" + }, + "gitaly-2" => { + "path" => "/var/opt/gitlab/git-data" + }, + "gitaly-3" => { + "path" => "/var/opt/gitlab/git-data" + } + }) + ``` + +1. Save the changes to `/etc/gitlab/gitlab.rb` and [reconfigure Gitaly](../restart_gitlab.md#omnibus-gitlab-reconfigure): + + ```shell + sudo gitlab-ctl reconfigure + ``` + +**Complete these steps for each Gitaly node!** + +After all Gitaly nodes are configured, you can run the Praefect connection checker to verify Praefect can connect to all Gitaly servers in the Praefect -config. This can be done by running the following command on the Praefect -server: +config. -```shell -sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml dial-nodes -``` +1. SSH into the **Praefect** node and run the Praefect connection checker: -#### GitLab - -When Praefect is running, it should be exposed as a storage to GitLab. This -is done through setting the `git_data_dirs`. Assuming the default storage -is present, there should be two storages available to GitLab: - -```ruby -# /etc/gitlab/gitlab.rb on gitlab server - -# Replace PRAEFECT_URL_OR_IP below with real address Praefect can be accessed at. -# Replace PRAEFECT_EXTERNAL_TOKEN below with real secret. -git_data_dirs({ - "default" => { - "path" => "/var/opt/gitlab/git-data" - }, - "praefect" => { - "gitaly_address" => "tcp://PRAEFECT_URL_OR_IP:2305", - "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN' - } -}) - -# Replace GITLAB_SHELL_SECRET_TOKEN below with real secret -gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN' - -# Possible values could be: 'http://10.23.101.53', 'https://gitlab.example.com', -# etc. Please replace GITLAB_SERVER_ADDRESS with proper value and change schema -# to 'https' in case you use encrypted connection. For more info please refer -# to https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-the-external-url-for-gitlab -external_url "http://<GITLAB_SERVER_ADDRESS>" -``` + ```shell + sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml dial-nodes + ``` -Replace `GITLAB_SHELL_SECRET_TOKEN` and `PRAEFECT_EXTERNAL_TOKEN` -with their respective values. +### GitLab -Note that the storage name used is the same as the `praefect['virtual_storage_name']` set -on the Praefect node. +To complete this section you will need: -Save your changes and reconfigure GitLab: +- [Configured Praefect node](#praefect) +- [Configured Gitaly nodes](#gitaly) -```shell -sudo gitlab-ctl reconfigure -``` +The Praefect cluster needs to be exposed as a storage location to the GitLab +application. This is done by updating the `git_data_dirs`. + +Particular attention should be shown to: + +- the storage name added to `git_data_dirs` in this section must match the + storage name under `praefect['virtual_storages']` on the Praefect node. This + was set in the [Praefect](#praefect) section of this guide. This document uses + `praefect` as the Praefect storage name. + +1. SSH into the **GitLab** node and login as root: + + ```shell + sudo -i + ``` + +1. Add the Praefect cluster as a storage location by editing + `/etc/gitlab/gitlab.rb`. + + You will need to replace: + + - `PRAEFECT_URL_OR_IP` with the IP/host address of the Praefect node + - `PRAEFECT_EXTERNAL_TOKEN` with the real secret -Run `sudo gitlab-rake gitlab:gitaly:check` to confirm that GitLab can reach Praefect. + ```ruby + git_data_dirs({ + "default" => { + "path" => "/var/opt/gitlab/git-data" + }, + "praefect" => { + "gitaly_address" => "tcp://PRAEFECT_URL_OR_IP:2305", + "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN' + } + }) + ``` -### Testing Praefect +1. Configure the `gitlab_shell['secret_token']` so that callbacks from Gitaly + nodes during a `git push` are properly authenticated by editing + `/etc/gitlab/gitlab.rb`: + + You will need to replace `GITLAB_SHELL_SECRET_TOKEN` with the real secret. + + ```ruby + gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN' + ``` + +1. Save the changes to `/etc/gitlab/gitlab.rb` and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure): + + ```shell + sudo gitlab-ctl reconfigure + ``` + +1. Verify that GitLab can reach Praefect: + + ```shell + sudo gitlab-rake gitlab:gitaly:check + ``` + +1. Update the **Repository storage** settings from **Admin Area > Settings > + Repository > Repository storage** to make the newly configured Praefect + cluster the storage location for new Git repositories. + + - Deselect the **default** storage location + - Select the **praefect** storage location + +1. Verify everything is still working by creating a new project. Check the + "Initialize repository with a README" box so that there is content in the + repository that viewed. If the project is created, and you can see the + README file, it works! + +Congratulations! You have configured a highly available Praefect cluster, and + +## Migrating existing repositories to Praefect + +If your GitLab instance already has repositories, these won't be migrated +automatically. + +Repositories may be moved from one storage location using the [Repository +API](../../api/projects.html#edit-project): + +```shell +curl --request PUT \ + --header "PRIVATE-TOKEN: <your_access_token>" \ + --data "repository_storage=praefect" \ + https://example.gitlab.com/api/v4/projects/123 +``` -To test Praefect, first set it as the default storage node for new projects -using **Admin Area > Settings > Repository > Repository storage**. Next, -create a new project and check the "Initialize repository with a README" box. +## Debugging Praefect If you receive an error, check `/var/log/gitlab/gitlab-rails/production.log`. diff --git a/doc/api/commits.md b/doc/api/commits.md index ca82eb4c442..90632a2faa0 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -42,7 +42,8 @@ Example response: "message": "Replace sanitize with escape once", "parent_ids": [ "6104942438c14ec7bd21c6cd5bd995272b3faff6" - ] + ], + "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746" }, { "id": "6104942438c14ec7bd21c6cd5bd995272b3faff6", @@ -56,7 +57,8 @@ Example response: "message": "Sanitize for network graph", "parent_ids": [ "ae1d9fb46aa2b07ee9836d49862ec4e2c46fbbba" - ] + ], + "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746" } ] ``` @@ -156,7 +158,8 @@ Example response: "deletions": 2, "total": 4 }, - "status": null + "status": null, + "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746" } ``` @@ -235,7 +238,8 @@ Example response: "deletions": 10, "total": 25 }, - "status": "running" + "status": "running", + "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/6104942438c14ec7bd21c6cd5bd995272b3faff6" } ``` @@ -314,7 +318,8 @@ Example response: "message": "Feature added\n\nSigned-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>\n", "parent_ids": [ "a738f717824ff53aebad8b090c1b79a14f2bd9e8" - ] + ], + "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/8b090c1b79a14f2bd9e8a738f717824ff53aebad" } ``` @@ -370,7 +375,8 @@ Example response: "authored_date":"2018-11-08T15:55:26.000Z", "committer_name":"Administrator", "committer_email":"admin@example.com", - "committed_date":"2018-11-08T15:55:26.000Z" + "committed_date":"2018-11-08T15:55:26.000Z", + "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/8b090c1b79a14f2bd9e8a738f717824ff53aebad" } ``` diff --git a/doc/api/groups.md b/doc/api/groups.md index e48cb78b2cc..a4fc3f95c5c 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -492,6 +492,7 @@ Parameters: | `auto_devops_enabled` | boolean | no | Default to Auto DevOps pipeline for all projects within this group. | | `subgroup_creation_level` | string | no | Allowed to create subgroups. Can be `owner` (Owners), or `maintainer` (Maintainers). | | `emails_disabled` | boolean | no | Disable email notifications | +| `avatar` | mixed | no | Image file for avatar of the group. [Introduced in GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/issues/36681) | | `mentions_disabled` | boolean | no | Disable the capability of a group from getting mentioned | | `lfs_enabled` | boolean | no | Enable/disable Large File Storage (LFS) for the projects in this group. | | `request_access_enabled` | boolean | no | Allow users to request member access. | @@ -553,6 +554,7 @@ PUT /groups/:id | `auto_devops_enabled` | boolean | no | Default to Auto DevOps pipeline for all projects within this group. | | `subgroup_creation_level` | string | no | Allowed to create subgroups. Can be `owner` (Owners), or `maintainer` (Maintainers). | | `emails_disabled` | boolean | no | Disable email notifications | +| `avatar` | mixed | no | Image file for avatar of the group. [Introduced in GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/issues/36681) | | `mentions_disabled` | boolean | no | Disable the capability of a group from getting mentioned | | `lfs_enabled` (optional) | boolean | no | Enable/disable Large File Storage (LFS) for the projects in this group. | | `request_access_enabled` | boolean | no | Allow users to request member access. | |