Add latest changes from gitlab-org/gitlab@master

1 files changed, 23 insertions, 0 deletions

diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md
index 8c766774868..5276e3a7816 100644
--- a/doc/administration/gitaly/praefect.md
+++ b/doc/administration/gitaly/praefect.md
@@ -682,6 +682,29 @@ for example behind a load balancer, `failover_enabled` should be disabled. The r
 is no coordination that currently happens across different Praefect instances, so there could be a situation where
 two Praefect instances think two different Gitaly nodes are the primary.
 
+## Backend Node Recovery
+
+When a Praefect backend node fails and is no longer able to
+replicate changes, the backend node will start to drift from the primary. If
+that node eventually recovers, it will need to be reconciled with the current
+primary. The primary node is considered the single source of truth for the
+state of a shard. The Praefect `reconcile` subcommand allows for the manual
+reconciliation between a backend node and the current primary.
+
+Run the following command on the Praefect server after all placeholders
+(`<virtual-storage>` and `<target-storage>`) have been replaced:
+
+```shell
+sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml reconcile -virtual <virtual-storage> -target <target-storage>
+```
+
+- Replace the placeholder `<virtual-storage>` with the virtual storage containing the backend node storage to be checked.
+- Replace the placeholder `<target-storage>` with the backend storage name.
+
+The command will return a list of repositories that were found to be
+inconsistent against the current primary. Each of these inconsistencies will
+also be logged with an accompanying replication job ID.
+
 ## Grafana
 
 Grafana is included with GitLab, and can be used to monitor your Praefect