Add latest changes from gitlab-org/gitlab@master

3 files changed, 104 insertions, 0 deletions

diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md
index ef73b765745..b534e84191a 100644
--- a/doc/administration/gitaly/index.md
+++ b/doc/administration/gitaly/index.md
@@ -850,3 +850,8 @@ To remove the proxy setting, run the following commands (depending on which vari
 unset http_proxy
 unset https_proxy
 ```
+
+### Praefect
+
+Praefect is an experimental daemon that allows for replication of the Git data.
+It can be setup with omnibus, [as explained here](./praefect.md).
diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md
new file mode 100644
index 00000000000..24a5a6df768
--- /dev/null
+++ b/doc/administration/gitaly/praefect.md
@@ -0,0 +1,93 @@
+# Praefect
+
+NOTE: **Note:** Praefect is an experimental service, and for testing purposes only at
+this time.
+
+## Omnibus
+
+### Architecture
+
+For this document, the following network topology is assumed:
+
+```mermaid
+graph TB
+  GitLab --> Gitaly;
+  GitLab --> Praefect;
+  Praefect --> Preafect-Git-1;
+  Praefect --> Preafect-Git-2;
+  Praefect --> Preafect-Git-3;
+```
+
+Where `GitLab` is the collection of clients that can request Git operations.
+`Gitaly` is a Gitaly server before using Praefect. The Praefect node has two
+storage nodes attached. Praefect itself doesn't storage data, but connects to
+three Gitaly nodes, `Praefect-Git-1`,  `Praefect-Git-2`, and `Praefect-Git-3`.
+There should be no knowledge other than with Praefect about the existence of
+the `Praefect-Git-X` nodes.
+
+### Enable the daemon
+
+Praefect is expected to run on their own host, this means that no other service
+other than the support services run on this machine.
+
+Praefect is disabled by default, to enable praefect uncomment the following line
+and set it to `true`: `# praefect['enable'] = false'`
+
+```ruby
+praefect['enable'] = true
+```
+
+By default praefect will listen on port `:2305`. It's recommended to enable
+prometheus to expose metrics. Uncomment the line so it looks like:
+
+```ruby
+praefect['prometheus_listen_addr'] = "localhost:9652"
+```
+
+Preafect needs at least one storage to store the Git data on. This node should
+run Gitaly and should not be listed as storage for GitLab itself, that is, the
+only way it receives traffic is through Praefect and it's not listed in the
+`git_data_dirs` on any `gitlab.rb` in your GitLab cluster.
+
+To set the nodes as depicted in the diagram above, the configuration should look
+like:
+
+```ruby
+praefect['storage_nodes'] = [
+  {
+    'storage' => 'praefect-git-1',
+    'address' => 'tcp://praefect-git-1.internal',
+    'primary' => true
+}
+  {
+    'storage' => 'praefect-git-2',
+    'address' => 'tcp://praefect-git-2.internal'
+  },
+  {
+    'storage' => 'praefect-git-3',
+    'address' => 'tcp://praefect-git-3.internal'
+  }
+]
+```
+
+Save the file, and run `gitlab-ctl reconfigure`. To test if Praefect is running,
+you could run `gitlab-ctl status` which should list praefect as being up.
+
+### Enable Preafect as storage backend in GitLab
+
+When Praefect is running, it should be exposed as a storage to GitLab. This
+is done through setting the `git_data_dirs`. Assuming the default storage
+configuration is used, there would be two storages available to GitLab:
+
+```ruby
+git_data_dirs({
+  "default" => {
+    "gitaly_address" => "tcp://gitaly.internal"
+  },
+  "praefect" => {
+    "gitaly_address" => "tcp://praefect.internal:2305"
+  }
+})
+```
+
+Restart GitLab using `gitlab-ctl restart` on the GitLab node.
diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md
index f01b83f4bd2..8710774bbb1 100644
--- a/doc/development/pipelines.md
+++ b/doc/development/pipelines.md
@@ -27,6 +27,7 @@ The current stages are:
 - `review`: This stage includes jobs that deploy the GitLab and Docs Review Apps.
 - `qa`: This stage includes jobs that perform QA tasks against the Review App
   that is deployed in the previous stage.
+- `notification`: This stage includes jobs that sends notifications about pipeline status.
 - `post-test`: This stage includes jobs that build reports or gather data from
   the previous stages' jobs (e.g. coverage, Knapsack metadata etc.).
 - `pages`: This stage includes a job that deploys the various reports as
@@ -191,6 +192,11 @@ subgraph "`qa` stage"
     dast -.-> |depends on| G;
     end
 
+subgraph "`notification` stage"
+    NOTIFICATION1["schedule:package-and-qa:notify-success<br>(on_success)"] -.-> |needs| P;
+    NOTIFICATION2["schedule:package-and-qa:notify-failure<br>(on_failure)"] -.-> |needs| P;
+    end
+
 subgraph "`post-test` stage"
     M
     end