diff options
Diffstat (limited to 'doc/administration/geo/replication/configuration.md')
| -rw-r--r-- | doc/administration/geo/replication/configuration.md | 163 |
1 files changed, 83 insertions, 80 deletions
diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index e8ffa1ae91a..5b22741f578 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -7,35 +7,35 @@ type: howto # Geo configuration **(PREMIUM SELF)** -## Configuring a new **secondary** node +## Configuring a new **secondary** site NOTE: -This is the final step in setting up a **secondary** Geo node. Stages of the +This is the final step in setting up a **secondary** Geo site. Stages of the setup process must be completed in the documented order. Before attempting the steps in this stage, [complete all prior stages](../setup/index.md#using-omnibus-gitlab). -The basic steps of configuring a **secondary** node are to: +The basic steps of configuring a **secondary** site are to: -- Replicate required configurations between the **primary** node and the **secondary** nodes. -- Configure a tracking database on each **secondary** node. -- Start GitLab on each **secondary** node. +- Replicate required configurations between the **primary** site and the **secondary** sites. +- Configure a tracking database on each **secondary** site. +- Start GitLab on each **secondary** site. You are encouraged to first read through all the steps before executing them in your testing/production environment. NOTE: -**Do not** set up any custom authentication for the **secondary** nodes. This is handled by the **primary** node. +**Do not** set up any custom authentication for the **secondary** sites. This is handled by the **primary** site. Any change that requires access to the **Admin Area** needs to be done in the -**primary** node because the **secondary** node is a read-only replica. +**primary** site because the **secondary** site is a read-only replica. ### Step 1. Manually replicate secret GitLab values GitLab stores a number of secret values in the `/etc/gitlab/gitlab-secrets.json` -file which *must* be the same on all nodes. Until there is -a means of automatically replicating these between nodes (see [issue #3789](https://gitlab.com/gitlab-org/gitlab/-/issues/3789)), -they must be manually replicated to the **secondary** node. +file which *must* be the same on all of a site's nodes. Until there is +a means of automatically replicating these between sites (see [issue #3789](https://gitlab.com/gitlab-org/gitlab/-/issues/3789)), +they must be manually replicated to **all nodes of the secondary site**. -1. SSH into the **primary** node, and execute the command below: +1. SSH into a **Rails node on your primary** site, and execute the command below: ```shell sudo cat /etc/gitlab/gitlab-secrets.json @@ -43,7 +43,7 @@ they must be manually replicated to the **secondary** node. This displays the secrets that need to be replicated, in JSON format. -1. SSH into the **secondary** node and login as the `root` user: +1. SSH **into each node on your secondary Geo site** and login as the `root` user: ```shell sudo -i @@ -55,7 +55,7 @@ they must be manually replicated to the **secondary** node. mv /etc/gitlab/gitlab-secrets.json /etc/gitlab/gitlab-secrets.json.`date +%F` ``` -1. Copy `/etc/gitlab/gitlab-secrets.json` from the **primary** node to the **secondary** node, or +1. Copy `/etc/gitlab/gitlab-secrets.json` from the **Rails node on your primary** site to **each node on your secondary** site, or copy-and-paste the file contents between nodes: ```shell @@ -72,28 +72,28 @@ they must be manually replicated to the **secondary** node. chmod 0600 /etc/gitlab/gitlab-secrets.json ``` -1. Reconfigure the **secondary** node for the change to take effect: +1. Reconfigure **each Rails, Sidekiq and Gitaly nodes on your secondary** site for the change to take effect: ```shell gitlab-ctl reconfigure gitlab-ctl restart ``` -### Step 2. Manually replicate the **primary** node's SSH host keys +### Step 2. Manually replicate the **primary** site's SSH host keys GitLab integrates with the system-installed SSH daemon, designating a user (typically named `git`) through which all access requests are handled. In a [Disaster Recovery](../disaster_recovery/index.md) situation, GitLab system -administrators promote a **secondary** node to the **primary** node. DNS records for the -**primary** domain should also be updated to point to the new **primary** node -(previously a **secondary** node). Doing so avoids the need to update Git remotes and API URLs. +administrators promote a **secondary** site to the **primary** site. DNS records for the +**primary** domain should also be updated to point to the new **primary** site +(previously a **secondary** site). Doing so avoids the need to update Git remotes and API URLs. -This causes all SSH requests to the newly promoted **primary** node to +This causes all SSH requests to the newly promoted **primary** site to fail due to SSH host key mismatch. To prevent this, the primary SSH host -keys must be manually replicated to the **secondary** node. +keys must be manually replicated to the **secondary** site. -1. SSH into the **secondary** node and login as the `root` user: +1. SSH into **each node on your secondary** site and login as the `root` user: ```shell sudo -i @@ -105,34 +105,34 @@ keys must be manually replicated to the **secondary** node. find /etc/ssh -iname ssh_host_* -exec cp {} {}.backup.`date +%F` \; ``` -1. Copy OpenSSH host keys from the **primary** node: +1. Copy OpenSSH host keys from the **primary** site: - If you can access your **primary** node using the **root** user: + If you can access one of the **nodes on your primary** site serving SSH traffic (usually, the main GitLab Rails application nodes) using the **root** user: ```shell - # Run this from the secondary node, change `<primary_node_fqdn>` for the IP or FQDN of the server + # Run this from the secondary site, change `<primary_site_fqdn>` for the IP or FQDN of the server scp root@<primary_node_fqdn>:/etc/ssh/ssh_host_*_key* /etc/ssh ``` If you only have access through a user with `sudo` privileges: ```shell - # Run this from your primary node: + # Run this from the node on your primary site: sudo tar --transform 's/.*\///g' -zcvf ~/geo-host-key.tar.gz /etc/ssh/ssh_host_*_key* - # Run this from your secondary node: - scp <user_with_sudo>@<primary_node_fqdn>:geo-host-key.tar.gz . + # Run this on each node on your secondary site: + scp <user_with_sudo>@<primary_site_fqdn>:geo-host-key.tar.gz . tar zxvf ~/geo-host-key.tar.gz -C /etc/ssh ``` -1. On your **secondary** node, ensure the file permissions are correct: +1. On **each node on your secondary** site, ensure the file permissions are correct: ```shell chown root:root /etc/ssh/ssh_host_*_key* chmod 0600 /etc/ssh/ssh_host_*_key* ``` -1. To verify key fingerprint matches, execute the following command on both nodes: +1. To verify key fingerprint matches, execute the following command on both primary and secondary nodes on each site: ```shell for file in /etc/ssh/ssh_host_*_key; do ssh-keygen -lf $file; done @@ -160,7 +160,7 @@ keys must be manually replicated to the **secondary** node. NOTE: The output for private keys and public keys command should generate the same fingerprint. -1. Restart `sshd` on your **secondary** node: +1. Restart `sshd` on **each node on your secondary** site: ```shell # Debian or Ubuntu installations @@ -175,31 +175,34 @@ keys must be manually replicated to the **secondary** node. SSH into your GitLab **secondary** server in a new terminal. If you are unable to connect, verify the permissions are correct according to the previous steps. -### Step 3. Add the **secondary** node +### Step 3. Add the **secondary** site -1. SSH into your GitLab **secondary** server and login as root: +1. SSH into **each Rails and Sidekiq node on your secondary** site and login as root: ```shell sudo -i ``` -1. Edit `/etc/gitlab/gitlab.rb` and add a **unique** name for your node. You need this in the next steps: +1. Edit `/etc/gitlab/gitlab.rb` and add a **unique** name for your site. You need this in the next steps: ```ruby - # The unique identifier for the Geo node. - gitlab_rails['geo_node_name'] = '<node_name_here>' + ## + ## The unique identifier for the Geo site. See + ## https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html#common-settings + ## + gitlab_rails['geo_node_name'] = '<site_name_here>' ``` -1. Reconfigure the **secondary** node for the change to take effect: +1. Reconfigure **each Rails and Sidekiq node on your secondary** site for the change to take effect: ```shell gitlab-ctl reconfigure ``` -1. On the top bar of the primary node, select **Menu >** **{admin}** **Admin**. -1. On the left sidebar, select **Geo > Nodes**. -1. Select **Add site**. -  +1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the left sidebar, select **Geo > Sites**. +1. Select **New site**. +  1. Fill in **Name** with the `gitlab_rails['geo_node_name']` in `/etc/gitlab/gitlab.rb`. These values must always match *exactly*, character for character. @@ -207,10 +210,10 @@ keys must be manually replicated to the **secondary** node. values must always match, but it doesn't matter if one ends with a `/` and the other doesn't. 1. Optionally, choose which groups or storage shards should be replicated by the - **secondary** node. Leave blank to replicate all. Read more in + **secondary** site. Leave blank to replicate all. Read more in [selective synchronization](#selective-synchronization). -1. Select **Add node** to add the **secondary** node. -1. SSH into your GitLab **secondary** server and restart the services: +1. Select **Add site** to add the **secondary** site. +1. SSH into **each Rails, and Sidekiq node on your secondary** site and restart the services: ```shell gitlab-ctl restart @@ -222,58 +225,58 @@ keys must be manually replicated to the **secondary** node. gitlab-rake gitlab:geo:check ``` -1. SSH into your **primary** server and login as root to verify the - **secondary** node is reachable or there are any common issue with your Geo setup: +1. SSH into a **Rails or Sidekiq server on your primary** site and login as root to verify the + **secondary** site is reachable or there are any common issue with your Geo setup: ```shell gitlab-rake gitlab:geo:check ``` -Once added to the Geo administration page and restarted, the **secondary** node automatically starts -replicating missing data from the **primary** node in a process known as **backfill**. -Meanwhile, the **primary** node starts to notify each **secondary** node of any changes, so -that the **secondary** node can act on those notifications immediately. +Once added to the Geo administration page and restarted, the **secondary** site automatically starts +replicating missing data from the **primary** site in a process known as **backfill**. +Meanwhile, the **primary** site starts to notify each **secondary** site of any changes, so +that the **secondary** site can act on those notifications immediately. -Be sure the _secondary_ node is running and accessible. You can sign in to the -_secondary_ node with the same credentials as were used with the _primary_ node. +Be sure the _secondary_ site is running and accessible. You can sign in to the +_secondary_ site with the same credentials as were used with the _primary_ site. -### Step 4. (Optional) Configuring the **secondary** node to trust the **primary** node +### Step 4. (Optional) Configuring the **secondary** site to trust the **primary** site -You can safely skip this step if your **primary** node uses a CA-issued HTTPS certificate. +You can safely skip this step if your **primary** site uses a CA-issued HTTPS certificate. -If your **primary** node is using a self-signed certificate for *HTTPS* support, you -need to add that certificate to the **secondary** node's trust store. Retrieve the -certificate from the **primary** node and follow +If your **primary** site is using a self-signed certificate for *HTTPS* support, you +need to add that certificate to the **secondary** site's trust store. Retrieve the +certificate from the **primary** site and follow [these instructions](https://docs.gitlab.com/omnibus/settings/ssl.html) -on the **secondary** node. +on the **secondary** site. ### Step 5. Enable Git access over HTTP/HTTPS Geo synchronizes repositories over HTTP/HTTPS, and therefore requires this clone -method to be enabled. This is enabled by default, but if converting an existing node to Geo it should be checked: +method to be enabled. This is enabled by default, but if converting an existing site to Geo it should be checked: -On the **primary** node: +On the **primary** site: 1. On the top bar, select **Menu >** **{admin}** **Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility and access controls**. 1. Ensure "Enabled Git access protocols" is set to either "Both SSH and HTTP(S)" or "Only HTTP(S)". -### Step 6. Verify proper functioning of the **secondary** node +### Step 6. Verify proper functioning of the **secondary** site -You can sign in to the **secondary** node with the same credentials you used with -the **primary** node. After you sign in: +You can sign in to the **secondary** site with the same credentials you used with +the **primary** site. After you sign in: 1. On the top bar, select **Menu >** **{admin}** **Admin**. -1. On the left sidebar, select **Geo > Nodes**. -1. Verify that it's correctly identified as a **secondary** Geo node, and that +1. On the left sidebar, select **Geo > Sites**. +1. Verify that it's correctly identified as a **secondary** Geo site, and that Geo is enabled. The initial replication, or 'backfill', is probably still in progress. You -can monitor the synchronization process on each Geo node from the **primary** -node's **Geo Nodes** dashboard in your browser. +can monitor the synchronization process on each Geo site from the **primary** +site's **Geo Sites** dashboard in your browser. - + If your installation isn't working properly, check the [troubleshooting document](troubleshooting.md). @@ -286,10 +289,10 @@ The two most obvious issues that can become apparent in the dashboard are: - You are using a custom certificate or custom CA (see the [troubleshooting document](troubleshooting.md)). - The instance is firewalled (check your firewall rules). -Please note that disabling a **secondary** node stops the synchronization process. +Disabling a **secondary** site stops the synchronization process. -Please note that if `git_data_dirs` is customized on the **primary** node for multiple -repository shards you must duplicate the same configuration on each **secondary** node. +If `git_data_dirs` is customized on the **primary** site for multiple +repository shards you must duplicate the same configuration on each **secondary** site. Point your users to the [Using a Geo Site guide](usage.md). @@ -304,7 +307,7 @@ Currently, this is what is synced: ## Selective synchronization Geo supports selective synchronization, which allows administrators to choose -which projects should be synchronized by **secondary** nodes. +which projects should be synchronized by **secondary** sites. A subset of projects can be chosen, either by group or by storage shard. The former is ideal for replicating data belonging to a subset of users, while the latter is more suited to progressively rolling out Geo to a large GitLab @@ -312,22 +315,22 @@ instance. It is important to note that selective synchronization: -1. Does not restrict permissions from **secondary** nodes. -1. Does not hide project metadata from **secondary** nodes. +1. Does not restrict permissions from **secondary** sites. +1. Does not hide project metadata from **secondary** sites. - Since Geo currently relies on PostgreSQL replication, all project metadata - gets replicated to **secondary** nodes, but repositories that have not been + gets replicated to **secondary** sites, but repositories that have not been selected are empty. 1. Does not reduce the number of events generated for the Geo event log. - - The **primary** node generates events as long as any **secondary** nodes are present. - Selective synchronization restrictions are implemented on the **secondary** nodes, - not the **primary** node. + - The **primary** site generates events as long as any **secondary** sites are present. + Selective synchronization restrictions are implemented on the **secondary** sites, + not the **primary** site. ### Git operations on unreplicated repositories > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2562) in GitLab 12.10 for HTTP(S) and in GitLab 13.0 for SSH. Git clone, pull, and push operations over HTTP(S) and SSH are supported for repositories that -exist on the **primary** node but not on **secondary** nodes. This situation can occur +exist on the **primary** site but not on **secondary** sites. This situation can occur when: - Selective synchronization does not include the project attached to the repository. @@ -335,7 +338,7 @@ when: ## Upgrading Geo -See the [updating the Geo nodes document](updating_the_geo_nodes.md). +See the [updating the Geo sites document](updating_the_geo_nodes.md). ## Troubleshooting |