Merge branch 'doc/sentinel' of gitlab.com:gitlab-org/gitlab-ce into doc/sentinel

3 files changed, 1107 insertions, 330 deletions

diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md
index d74a786ac24..d5a5aef7ec0 100644
--- a/doc/administration/high_availability/README.md
+++ b/doc/administration/high_availability/README.md
@@ -7,19 +7,10 @@ highly available.
 
 ## Architecture
 
-### Active/Passive
-
-For pure high-availability/failover with no scaling you can use an
-active/passive configuration. This utilizes DRBD (Distributed Replicated
-Block Device) to keep all data in sync. DRBD requires a low latency link to
-remain in sync. It is not advisable to attempt to run DRBD between data centers
-or in different cloud availability zones.
+There are two kinds of setups:
 
-Components/Servers Required:
-
-- 2 servers/virtual machines (one active/one passive)
-
-![Active/Passive HA Diagram](../img/high_availability/active-passive-diagram.png)
+- active/active
+- active/passive
 
 ### Active/Active
 
@@ -28,12 +19,24 @@ user requests simultaneously. The database, Redis, and GitLab application are
 all deployed on separate servers. The configuration is **only** highly-available
 if the database, Redis and storage are also configured as such.
 
-![Active/Active HA Diagram](../img/high_availability/active-active-diagram.png)
-
-**Steps to configure active/active:**
+Follow the steps below to configure an active/active setup:
 
 1. [Configure the database](database.md)
 1. [Configure Redis](redis.md)
 1. [Configure NFS](nfs.md)
 1. [Configure the GitLab application servers](gitlab.md)
 1. [Configure the load balancers](load_balancer.md)
+
+![Active/Active HA Diagram](../img/high_availability/active-active-diagram.png)
+
+### Active/Passive
+
+For pure high-availability/failover with no scaling you can use an
+active/passive configuration. This utilizes DRBD (Distributed Replicated
+Block Device) to keep all data in sync. DRBD requires a low latency link to
+remain in sync. It is not advisable to attempt to run DRBD between data centers
+or in different cloud availability zones.
+
+Components/Servers Required: 2 servers/virtual machines (one active/one passive)
+
+![Active/Passive HA Diagram](../img/high_availability/active-passive-diagram.png)
diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md
index e416514bedf..a87c1dde55d 100644
--- a/doc/administration/high_availability/redis.md
+++ b/doc/administration/high_availability/redis.md
@@ -1,417 +1,809 @@
 # Configuring Redis for GitLab HA
 
-You can choose to install and manage Redis yourself, or you can use the one
-that comes bundled with GitLab Omnibus packages.
+High Availability with [Redis] is possible using a **Master** x **Slave**
+topology with a [Redis Sentinel][sentinel] service to watch and automatically
+start the failover procedure.
 
-> **Note:** Redis does not require authentication by default. See
+You can choose to install and manage Redis and Sentinel yourself, use
+a hosted cloud solution or you can use the one that comes bundled with
+Omnibus GitLab packages.
+
+> **Notes:**
+- Redis requires authentication for High Availability. See
   [Redis Security](http://redis.io/topics/security) documentation for more
   information. We recommend using a combination of a Redis password and tight
   firewall rules to secure your Redis service.
+- You are highly encouraged to read the [Redis Sentinel][sentinel] documentation
+  before configuring Redis HA with GitLab to fully understand the topology and
+  architecture.
+- This is the documentation for the Omnibus GitLab packages. For installations
+  from source, follow the [Redis HA source installation](redis_source.md) guide.
+- Redis Sentinel daemon is bundled with Omnibus GitLab Enterprise Edition only.
+  For configuring Sentinel with the Omnibus GitLab Community Edition and
+  installations from source, read the
+  [Available configuration setups](#available-configuration-setups) section
+  below.
 
 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 **Table of Contents**
 
-- [Configure your own Redis server](#configure-your-own-redis-server)
-- [Configure Redis using Omnibus](#configure-redis-using-omnibus)
-- [Experimental Redis Sentinel support](#experimental-redis-sentinel-support)
-  - [Redis setup](#redis-setup)
-    - [Source install](#source-install)
-    - [Omnibus Install](#omnibus-install)
-    - [Troubleshooting Replication](#troubleshooting-replication)
-  - [Sentinel](#sentinel)
-    - [Sentinel setup (Community Edition)](#sentinel-setup-community-edition)
-    - [Sentinel setup (EE Only)](#sentinel-setup-ee-only)
-  - [GitLab setup](#gitlab-setup)
-  - [Sentinel troubleshooting](#sentinel-troubleshooting)
-    - [Omnibus install](#omnibus-install)
-    - [Source install](#source-install-1)
+- [Overview](#overview)
+  - [High Availability with Sentinel](#high-availability-with-sentinel)
+  - [Recommended setup](#recommended-setup)
+  - [Redis setup overview](#redis-setup-overview)
+  - [Sentinel setup overview](#sentinel-setup-overview)
+  - [Available configuration setups](#available-configuration-setups)
+- [Configuring Redis HA](#configuring-redis-ha)
+  - [Prerequisites](#prerequisites)
+  - [Step 1. Configuring the master Redis instance](#step-1-configuring-the-master-redis-instance)
+  - [Step 2. Configuring the slave Redis instances](#step-2-configuring-the-slave-redis-instances)
+  - [Step 3. Configuring the Redis Sentinel instances](#step-3-configuring-the-redis-sentinel-instances)
+  - [Step 4. Configuring the GitLab application](#step-4-configuring-the-gitlab-application)
+- [Switching from an existing single-machine installation to Redis HA](#switching-from-an-existing-single-machine-installation-to-redis-ha)
+- [Example of a minimal configuration with 1 master, 2 slaves and 3 Sentinels](#example-of-a-minimal-configuration-with-1-master-2-slaves-and-3-sentinels)
+  - [Example configuration for Redis master and Sentinel 1](#example-configuration-for-redis-master-and-sentinel-1)
+  - [Example configuration for Redis slave 1 and Sentinel 2](#example-configuration-for-redis-slave-1-and-sentinel-2)
+  - [Example configuration for Redis slave 2 and Sentinel 3](#example-configuration-for-redis-slave-2-and-sentinel-3)
+  - [Example configuration for the GitLab application](#example-configuration-for-the-gitlab-application)
+- [Advanced configuration](#advanced-configuration)
+  - [Control running services](#control-running-services)
+- [Troubleshooting](#troubleshooting)
+  - [Troubleshooting Redis replication](#troubleshooting-redis-replication)
+  - [Troubleshooting Sentinel](#troubleshooting-sentinel)
+- [Changelog](#changelog)
+  - [Experimental Redis Sentinel support](#experimental-redis-sentinel-support)
+- [Further reading](#further-reading)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
-## Configure your own Redis server
+## Overview
 
-If you're hosting GitLab on a cloud provider, you can optionally use a
-managed service for Redis. For example, AWS offers a managed ElastiCache service
-that runs Redis.
+Before diving into the details of setting up Redis and Redis Sentinel for HA,
+make sure you read this Overview section to better understand how the components
+are tied together.
 
-## Configure Redis using Omnibus
+You need at least `3` independent machines: physical, or VMs running into
+distinct physical machines. It is essential that all master and slaves Redis
+instances run in different machines. If you fail to provision the machines in
+that specific way, any issue with the shared environment can bring your entire
+setup down.
 
-If you don't want to bother setting up your own Redis server, you can use the
-one bundled with Omnibus. In this case, you should disable all services except
-Redis.
+It is OK to run a Sentinel along with a master or slave Redis instance.
+No more than one Sentinel in the same machine though.
 
-1. Download/install GitLab Omnibus using **steps 1 and 2** from
-   [GitLab downloads](https://about.gitlab.com/downloads). Do not complete other
-   steps on the download page.
-1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration.
-   Be sure to change the `external_url` to match your eventual GitLab front-end
-   URL:
+You also need to take in consideration the underlying network topology,
+making sure you have redundant connectivity between Redis / Sentinel and
+GitLab instances, otherwise the networks will become a single point of
+failure.
 
-    ```ruby
-    external_url 'https://gitlab.example.com'
-
-    # Disable all services except Redis
-    redis['enable'] = true
-    bootstrap['enable'] = false
-    nginx['enable'] = false
-    unicorn['enable'] = false
-    sidekiq['enable'] = false
-    postgresql['enable'] = false
-    gitlab_rails['enable'] = false
-    gitlab_workhorse['enable'] = false
-    mailroom['enable'] = false
-
-    # Redis configuration
-    redis['port'] = 6379
-    redis['bind'] = '0.0.0.0'
+Make sure that you read this document once as a whole before configuring the
+components below.
 
-    # If you wish to use Redis authentication (recommended)
-    redis['password'] = 'redis-password-goes-here'
-    ```
+### High Availability with Sentinel
 
-1. Run `sudo touch /etc/gitlab/skip-auto-migrations` to prevent database migrations
-   from running on upgrade. Only the primary GitLab application server should
-   handle migrations.
+>**Notes:**
+- Starting with GitLab `8.11`, you can configure a list of Redis Sentinel
+  servers that will monitor a group of Redis servers to provide failover support.
+- Starting with GitLab `8.14`, the Omnibus GitLab Enterprise Edition package
+  comes with Redis Sentinel daemon built-in.
 
-1. Run `sudo gitlab-ctl reconfigure` to install and configure Redis.
+High Availability with Redis requires a few things:
 
-    > **Note**: This `reconfigure` step will result in some errors.
-      That's OK - don't be alarmed.
+- Multiple Redis instances
+- Run Redis in a **Master** x **Slave** topology
+- Multiple Sentinel instances
+- Application support and visibility to all Sentinel and Redis instances
 
-## Experimental Redis Sentinel support
+Redis Sentinel can handle the most important tasks in an HA environment and that's
+to help keep servers online with minimal to no downtime. Redis Sentinel:
 
-> [Introduced][ce-1877] in GitLab 8.11, improved in 8.13.
+- Monitors **Master** and **Slaves** instances to see if they are available
+- Promotes a **Slave** to **Master** when the **Master** fails
+- Demotes a **Master** to **Slave** when the failed **Master** comes back online
+  (to prevent data-partitioning)
+- Can be queried by clients to always connect to the current **Master** server
 
-Since GitLab 8.11, you can configure a list of Redis Sentinel servers that
-will monitor a group of Redis servers to provide you with a standard failover
-support.
+When a **Master** fails to respond, it's the client's responsibility to handle
+timeout and reconnect (querying a **Sentinel** for a new **Master**).
 
 To get a better understanding on how to correctly setup Sentinel, please read
 the [Redis Sentinel documentation](http://redis.io/topics/sentinel) first, as
-failing to configure it correctly can lead to data loss.
+failing to configure it correctly can lead to data loss or can bring your
+whole cluster down, invalidating the failover effort.
+
+### Recommended setup
+
+For a minimal setup, you will install the Omnibus GitLab package in `3`
+**independent** machines, both with **Redis** and **Sentinel**:
 
-Redis Sentinel can handle the most important tasks in a HA environment to help
-keep servers online with minimal to no downtime:
+- Redis Master + Sentinel
+- Redis Slave + Sentinel
+- Redis Slave + Sentinel
 
-- Monitors master and slave instances to see if they are available
-- Promote a slave to master when the master fails.
-- Demote a master to slave when failed master comes back online (to prevent
-  data-partitioning).
-- Can be queried by clients to always connect to the correct master server.
+If you are not sure or don't understand why and where the amount of nodes come
+from, read [Redis setup overview](#redis-setup-overview) and
+[Sentinel setup overview](#sentinel-setup-overview).
 
-There is currently one exception to the Sentinel support: `mail_room`, the
-component that processes incoming emails. It doesn't support Sentinel yet, but
-we hope to integrate a future release that does support it soon.
+For a recommended setup that can resist more failures, you will install
+the Omnibus GitLab package in `5` **independent** machines, both with
+**Redis** and **Sentinel**:
 
-The configuration consists of three parts:
+- Redis Master + Sentinel
+- Redis Slave + Sentinel
+- Redis Slave + Sentinel
+- Redis Slave + Sentinel
+- Redis Slave + Sentinel
 
-- Setup Redis Master and Slave nodes
-- Setup Sentinel nodes
-- Setup GitLab
+### Redis setup overview
 
-> **IMPORTANT**: You need at least 3 independent machines: physical, or VMs
-running into distinct physical machines. If you fail to provision the
-machines in that specific way, any issue with the shared environment can
-bring your entire setup down.
+You must have at least `3` Redis servers: `1` Master, `2` Slaves, and they
+need to be each in a independent machine (see explanation above).
 
-Read carefully how to configure those components below.
+You can have additional Redis nodes, that will help survive a situation
+where more nodes goes down. Whenever there is only `2` nodes online, a failover
+will not be initiated.
 
-### Redis setup
+As an example, if you have `6` Redis nodes, a maximum of `3` can be
+simultaneously down.
 
-You must have at least `3` Redis servers: `1` Master, `2` Slaves, and they need to
-be each in a independent machine (see explanation above).
+Please note that there are different requirements for Sentinel nodes.
+If you host them in the same Redis machines, you may need to take
+that restrictions into consideration when calculating the amount of
+nodes to be provisioned. See [Sentinel setup overview](#sentinel-setup-overview)
+documentation for more information.
 
-They should be configured the same way and with similar server specs, as
-in a failover situation, any `Slave` can be elected as the new `Master` by
+All Redis nodes should be configured the same way and with similar server specs, as
+in a failover situation, any **Slave** can be promoted as the new **Master** by
 the Sentinel servers.
 
-With Sentinel, you must define a password to protect the access as both
-Sentinel instances and other redis instances should be able to talk to
+The replication requires authentication, so you need to define a password to
+protect all Redis nodes and the Sentinels. They will all share the same
+password, and all instances must be able to talk to
 each other over the network.
 
-You'll need to define both `requirepass` and `masterauth` in all
-nodes because they can be re-configured at any time by the Sentinels
-during a failover, and change it's status as `Master` or `Slave`.
+### Sentinel setup overview
 
-Initial `Slave` nodes will have in `redis.conf` an additional `slaveof` line
-pointing to the initial `Master`.
+Sentinels watch both other Sentinels and Redis nodes. Whenever a Sentinel
+detects that a Redis node is not responding, it will announce that to the
+other Sentinels. They have to reach the **quorum**, that is the minimum amount
+of Sentinels that agrees a node is down, in order to be able to start a failover.
 
-#### Source install
+Whenever the **quorum** is met, the **majority** of all known Sentinel nodes
+need to be available and reachable, so that they can elect the Sentinel **leader**
+who will take all the decisions to restore the service availability by:
 
-**Master Redis instance**
+- Promoting a new **Master**
+- Reconfiguring the other **Slaves** and make them point to the new **Master**
+- Announce the new **Master** to every other Sentinel peer
+- Reconfigure the old **Master** and demote to **Slave** when it comes back online
 
-You need to make the following changes in `redis.conf`:
+You must have at least `3` Redis Sentinel servers, and they need to
+be each in a independent machine (that are believed to fail independently),
+ideally in different geographical areas.
 
-1. Define a `bind` address pointing to a local IP that your other machines
-   can reach you. If you really need to bind to an external acessible IP, make
-   sure you add extra firewall rules to prevent unauthorized access:
+You can configure them in the same machines where you've configured the other
+Redis servers, but understand that if a whole node goes down, you loose both
+a Sentinel and a Redis instance.
 
-   ```conf
-   # By default, if no "bind" configuration directive is specified, Redis listens
-   # for connections from all the network interfaces available on the server.
-   # It is possible to listen to just one or multiple selected interfaces using
-   # the "bind" configuration directive, followed by one or more IP addresses.
-   #
-   # Examples:
-   #
-   # bind 192.168.1.100 10.0.0.1
-   # bind 127.0.0.1 ::1
-   bind 0.0.0.0 # This will bind to all interfaces
-   ```
+The number of sentinels should ideally always be an **odd** number, for the
+consensus algorithm to be effective in the case of a failure.
 
-1. Define a `port` to force redis to listin on TCP so other machines can
-   connect to it:
+In a `3` nodes topology, you can only afford `1` Sentinel node going down.
+Whenever the **majority** of the Sentinels goes down, the network partition
+protection prevents destructive actions and a failover **will not be started**.
 
-   ```conf
-   # Accept connections on the specified port, default is 6379 (IANA #815344).
-   # If port 0 is specified Redis will not listen on a TCP socket.
-   port 6379
-   ```
+Here are some examples:
 
-1. Set up password authentication (use the same password in all nodes)
+- With `5` or `6` sentinels, a maximum of `2` can go down for a failover begin.
+- With `7` sentinels, a maximum of `3` nodes can go down.
 
-    ```conf
-    requirepass "redis-password-goes-here"
-    masterauth "redis-password-goes-here"
-    ```
+The **Leader** election can sometimes fail the voting round when **consensus**
+is not achieved (see the odd number of nodes requirement above). In that case,
+a new attempt will be made after the amount of time defined in
+`sentinel['failover_timeout']` (in milliseconds).
 
-1. Restart the Redis services for the changes to take effect.
+>**Note:**
+We will see where `sentinel['failover_timeout']` is defined later.
+
+The `failover_timeout` variable has a lot of different use cases. According to
+the official documentation:
+
+- The time needed to re-start a failover after a previous failover was
+  already tried against the same master by a given Sentinel, is two
+  times the failover timeout.
+
+- The time needed for a slave replicating to a wrong master according
+  to a Sentinel current configuration, to be forced to replicate
+  with the right master, is exactly the failover timeout (counting since
+  the moment a Sentinel detected the misconfiguration).
+
+- The time needed to cancel a failover that is already in progress but
+  did not produced any configuration change (SLAVEOF NO ONE yet not
+  acknowledged by the promoted slave).
+
+- The maximum time a failover in progress waits for all the slaves to be
+  reconfigured as slaves of the new master. However even after this time
+  the slaves will be reconfigured by the Sentinels anyway, but not with
+  the exact parallel-syncs progression as specified.
+
+### Available configuration setups
+
+Based on your infrastructure setup and how you have installed GitLab, there are
+multiple ways to configure Redis HA. Omnibus GitLab packages have Redis and/or
+Redis Sentinel bundled with them so you only need to focus on configuration.
+Pick the one that suits your needs.
+
+- [Installations from source][source]: You need to install Redis and Sentinel
+  yourself. Use the [Redis HA installation from source](redis_source.md)
+  documentation.
+- [Omnibus GitLab **Community Edition** (CE) package][ce]: Redis is bundled, so you
+  can use the package with only the Redis service enabled as described in steps
+  1 and 2 of this document (works for both master and slave setups). To install
+  and configure Sentinel, jump directly to the Sentinel section in the
+  [Redis HA installation from source](redis_source.md#step-3-configuring-the-redis-sentinel-instances) documentation.
+- [Omnibus GitLab **Enterprise Edition** (EE) package][ee]: Both Redis and Sentinel
+  are bundled in the package, so you can use the EE package to setup the whole
+  Redis HA infrastructure (master, slave and Sentinel) which is described in
+  this document.
+- If you have installed GitLab using the Omnibus GitLab packages (CE or EE),
+  but you want to use your own external Redis server, follow steps 1-3 in the
+  [Redis HA installation from source](redis_source.md) documentation, then go
+  straight to step 4 in this guide to
+  [set up the GitLab application](#step-4-configuring-the-gitlab-application).
+
+## Configuring Redis HA
+
+This is the section where we install and setup the new Redis instances.
+
+>**Notes:**
+- We assume that you install GitLab and all HA components from scratch. If you
+  already have it installed and running, read how to
+  [switch from a single-machine installation to Redis HA](#switching-from-an-existing-single-machine-installation-to-redis-ha).
+- Redis nodes (both master and slaves) will need the same password defined in
+  `redis['password']`. At any time during a failover the Sentinels can
+  reconfigure a node and change its status from master to slave and vice versa.
+
+### Prerequisites
+
+The prerequisites for a HA Redis setup are the following:
+
+1. Provision the minimum required number of instances as specified in the
+   [recommended setup](#recommended-setup) section.
+1. **Do NOT** install Redis or Redis Sentinel in the same machines your
+   GitLab application is running on. You can however opt in to install Redis
+   and Sentinel in the same machine (each in independent ones is recommended
+   though).
+1. All Redis nodes must be able to talk to each other and accept incoming
+   connections over Redis (`6379`) and Sentinel (`26379`) ports (unless you
+   change the default ones).
+1. The server that hosts the GitLab application must be able to access the
+   Redis nodes.
+1. Protect the nodes from access from external networks ([Internet][it]), using
+   firewall.
+
+### Step 1. Configuring the master Redis instance
+
+1. SSH into the **master** Redis server and login as root:
 
-**Slave Redis instance**
+    ```
+    sudo -i
+    ```
 
-1. Follow same instructions from master with the extra change in `redis.conf`:
+1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab
+   package you want using **steps 1 and 2** from the GitLab downloads page.
+     - Make sure you select the correct Omnibus package, with the same version
+       and type (Community, Enterprise editions) of your current install.
+     - Do not complete any other steps on the download page.
 
-   ```conf
-   # IP and port of the master Redis server
-   slaveof 10.10.10.10 6379
-   ```
+1. Edit `/etc/gitlab/gitlab.rb` and add the contents:
 
-1. Restart the Redis services for the changes to take effect.
+    ```ruby
+    # Enable the master role and disable all other services in the machine
+    # (you can still enable Sentinel).
+    redis_master_role['enable'] = true
+
+    # IP address pointing to a local IP that the other machines can reach to.
+    # You can also set bind to '0.0.0.0' which listen in all interfaces.
+    # If you really need to bind to an external accessible IP, make
+    # sure you add extra firewall rules to prevent unauthorized access.
+    redis['bind'] = '10.0.0.1'
+
+    # Define a port so Redis can listen for TCP requests which will allow other
+    # machines to connect to it.
+    redis['port'] = 6379
 
-#### Omnibus Install
+    # Set up password authentication for Redis (use the same password in all nodes).
+    redis['password'] = 'redis-password-goes-here'
+    ```
 
-You need to install the omnibus package in 3 different and independent machines.
-We will elect one as the initial `Master` and the other 2 as `Slaves`.
+1. To prevent database migrations from running on upgrade, run:
 
-If you are migrating from a single machine install, you may want to setup the
-machines as Slaves, pointing to the original machine as `Master`, to migrate
-the data first, and than switch to this setup.
+    ```
+    sudo touch /etc/gitlab/skip-auto-migrations
+    ```
 
-To disable redis in the single install, edit `/etc/gitlab/gitlab.rb`:
+    Only the primary GitLab application server should handle migrations.
 
-```ruby
-redis['enable'] = false
-```
+1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
 
-**Master Redis instances**
+### Step 2. Configuring the slave Redis instances
 
-You need to make the following changes in `/etc/gitlab/gitlab.rb`:
+1. SSH into the **slave** Redis server and login as root:
 
-1. Define a `redis['bind']` address pointing to a local IP that your other machines
-   can reach you. If you really need to bind to an external acessible IP, make
-   sure you add extra firewall rules to prevent unauthorized access.
-1. Define a `redis['port']` to force redis to listin on TCP so other machines can
-   connect to it.
-1. Set up password authentication with `redis['master_password']` (use the same
-   password in all nodes).
+    ```
+    sudo -i
+    ```
 
-```ruby
-## Redis TCP support (will disable UNIX socket transport)
-redis['bind'] = '0.0.0.0' # or specify an IP to bind to a single one
-redis['port'] = 6379
-redis['requirepass'] = 'redis-password-goes-here'
-redis['master_password'] = 'redis-password-goes-here'
-```
+1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab
+   package you want using **steps 1 and 2** from the GitLab downloads page.
+     - Make sure you select the correct Omnibus package, with the same version
+       and type (Community, Enterprise editions) of your current install.
+     - Do not complete any other steps on the download page.
 
-Reconfigure GitLab Omnibus for the changes to take effect: `sudo gitlab-ctl reconfigure`
+1. Edit `/etc/gitlab/gitlab.rb` and add the contents:
 
-**Slave Redis instances**
+    ```ruby
+    # Enable the slave role and disable all other services in the machine
+    # (you can still enable Sentinel). This will also set automatically
+    # `redis['master'] = false`.
+    redis_slave_role['enable'] = true
+
+    # IP address pointing to a local IP that the other machines can reach to.
+    # You can also set bind to '0.0.0.0' which listen in all interfaces.
+    # If you really need to bind to an external accessible IP, make
+    # sure you add extra firewall rules to prevent unauthorized access.
+    redis['bind'] = '10.0.0.2'
+
+    # Define a port so Redis can listen for TCP requests which will allow other
+    # machines to connect to it.
+    redis['port'] = 6379
 
-You need to make the same changes listed for the `Master` instance,
-with an additional `Slave` section as in the example below:
+    # The same password for Redeis authentication you set up for the master node.
+    redis['password'] = 'redis-password-goes-here'
 
-```ruby
-## Redis TCP support (will disable UNIX socket transport)
-redis['bind'] = '0.0.0.0' # or specify an IP to bind to a single one
-redis['port'] = 6379
-redis['requirepass'] = 'redis-password-goes-here'
-redis['master_password'] = 'redis-password-goes-here'
+    # The IP of the master Redis node.
+    redis['master_ip'] = '10.0.0.1'
 
-## Slave redis instance
-redis['master'] = false
-redis['master_ip'] = '10.10.10.10' # IP of master Redis server
-redis['master_port'] = 6379 # Port of master Redis server
-```
+    # Port of master Redis server, uncomment to change to non default. Defaults
+    # to `6379`.
+    #redis['master_port'] = 6379
+    ```
 
-Reconfigure GitLab Omnibus for the changes to take effect: `sudo gitlab-ctl reconfigure`
+1. To prevent database migrations from running on upgrade, run:
 
-#### Troubleshooting Replication
+    ```
+    sudo touch /etc/gitlab/skip-auto-migrations
+    ```
 
-You can check if everything is correct by connecting to each server using
-`redis-cli` application, and sending the `INFO` command.
+    Only the primary GitLab application server should handle migrations.
 
-If authentication was correctly defined, it should fail with:
-`NOAUTH Authentication required` error. Try to authenticate with the
-previous defined password with `AUTH redis-password-goes-here` and
-try the `INFO` command again.
+1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
+1. Go through the steps again for all the other slave nodes.
 
-Look for the `# Replication` section where you should see some important
-information like the `role` of the server.
+---
 
-When connected to a `master` redis, you will see the number of connected
-`slaves`, and a list of each with connection details.
+These values don't have to be changed again in `/etc/gitlab/gitlab.rb` after
+a failover, as the nodes will be managed by the Sentinels, and even after a
+`gitlab-ctl reconfigure`, they will get their configuration restored by
+the same Sentinels.
 
-When it's a `slave`, you will see details of the master connection and if
-its `up` or `down`.
+### Step 3. Configuring the Redis Sentinel instances
 
----
+>**Note:**
+Redis Sentinel is bundled with Omnibus GitLab Enterprise Edition only. The
+following section assumes you are using Omnibus GitLab Enterprise Edition.
+For the Omnibus Community Edition and installations from source, follow the
+[Redis HA source install](redis_source.md) guide.
 
 Now that the Redis servers are all set up, let's configure the Sentinel
 servers.
 
 If you are not sure if your Redis servers are working and replicating
-correctly, please read the [Troubleshooting  Replication](#troubleshooting-replication)
+correctly, please read the [Troubleshooting Replication](#troubleshooting-replication)
 and fix it before proceeding with Sentinel setup.
 
-### Sentinel
-
 You must have at least `3` Redis Sentinel servers, and they need to
-be each in a independent machine. You can install them in the same
-machines you installed the other `3` Redis servers.
+be each in an independent machine. You can configure them in the same
+machines where you've configured the other Redis servers.
+
+With GitLab Enterprise Edition, you can use the Omnibus package to setup
+multiple machines with the Sentinel daemon.
+
+---
+
+1. SSH into the server that will host Redis Sentinel and login as root:
+
+    ```
+    sudo -i
+    ```
+
+1. **You can omit this step if the Sentinels will be hosted in the same node as
+   the other Redis instances.**
+
+     [Download/install](https://about.gitlab.com/downloads-ee) the
+     Omnibus GitLab Enterprise Edition package using **steps 1 and 2** from the
+     GitLab downloads page.
+       - Make sure you select the correct Omnibus package, with the same version
+         the GitLab application is running.
+       - Do not complete any other steps on the download page.
+
+1. Edit `/etc/gitlab/gitlab.rb` and add the contents (if you are installing the
+   Sentinels in the same node as the other Redis instances, some values might
+   be duplicate below):
+
+
+    ```ruby
+    redis_sentinel_role['enable'] = true
+
+    # Must be the same in every sentinel node
+    redis['master_name'] = 'gitlab-redis'
+
+    # The same password for Redis authentication you set up for the master node.
+    redis['password'] = 'redis-password-goes-here'
+
+    # The IP of the master Redis node.
+    redis['master_ip'] = '10.0.0.1'
+
+    # Define a port so Redis can listen for TCP requests which will allow other
+    # machines to connect to it.
+    redis['port'] = 6379
+
+    # Port of master Redis server, uncomment to change to non default. Defaults
+    # to `6379`.
+    #redis['master_port'] = 6379
+
+    ## Configure Sentinel
+    sentinel['bind'] = '10.0.0.1'
+
+    # Port that Sentinel listens on, uncomment to change to non default. Defaults
+    # to `26379`.
+    # sentinel['port'] = 26379
+
+    ## Quorum must reflect the amount of voting sentinels it take to start a failover.
+    ## Value must NOT be greater then the amount of sentinels.
+    ##
+    ## The quorum can be used to tune Sentinel in two ways:
+    ## 1. If a the quorum is set to a value smaller than the majority of Sentinels
+    ##    we deploy, we are basically making Sentinel more sensible to master failures,
+    ##    triggering a failover as soon as even just a minority of Sentinels is no longer
+    ##    able to talk with the master.
+    ## 1. If a quorum is set to a value greater than the majority of Sentinels, we are
+    ##    making Sentinel able to failover only when there are a very large number (larger
+    ##    than majority) of well connected Sentinels which agree about the master being down.s
+    sentinel['quorum'] = 2
+
+    ## Consider unresponsive server down after x amount of ms.
+    # sentinel['down_after_milliseconds'] = 10000
+
+    ## Specifies the failover timeout in milliseconds. It is used in many ways:
+    ##
+    ## - The time needed to re-start a failover after a previous failover was
+    ##   already tried against the same master by a given Sentinel, is two
+    ##   times the failover timeout.
+    ##
+    ## - The time needed for a slave replicating to a wrong master according
+    ##   to a Sentinel current configuration, to be forced to replicate
+    ##   with the right master, is exactly the failover timeout (counting since
+    ##   the moment a Sentinel detected the misconfiguration).
+    ##
+    ## - The time needed to cancel a failover that is already in progress but
+    ##   did not produced any configuration change (SLAVEOF NO ONE yet not
+    ##   acknowledged by the promoted slave).
+    ##
+    ## - The maximum time a failover in progress waits for all the slaves to be
+    ##   reconfigured as slaves of the new master. However even after this time
+    ##   the slaves will be reconfigured by the Sentinels anyway, but not with
+    ##   the exact parallel-syncs progression as specified.
+    # sentinel['failover_timeout'] = 60000
+    ```
+
+1. To prevent database migrations from running on upgrade, run:
+
+    ```
+    sudo touch /etc/gitlab/skip-auto-migrations
+    ```
 
-This number is required for the consensus algorithm to be effective
-in the case of a failure. You should always have and `odd` number
-of Sentinel nodes provisioned.
+    Only the primary GitLab application server should handle migrations.
 
-Here is a simple explanation on how Sentinel handles a failover:
+1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
+1. Go through the steps again for all the other Sentinel nodes.
 
-When a number of Sentinels (`quorum` value) agree the fact the `master` is
-not reachable, the **majority** of the sentinels must elect a temporary
-Sentinel `leader`, that will be responsible to start the failover proceedings.
+### Step 4. Configuring the GitLab application
 
-As an example, for a cluster of `3` Sentinels, at least `2` must agree on a
-`leader`. If you have total of `5` at least `3` must agree on the leader.
+The final part is to inform the main GitLab application server of the Redis
+Sentinels servers and authentication credentials.
 
-The `quorum` is only used to detect failure, not to elect the `leader`.
+You can enable or disable Sentinel support at any time in new or existing
+installations. From the GitLab application perspective, all it requires is
+the correct credentials for the Sentinel nodes.
 
-Official [Sentinel documentation](http://redis.io/topics/sentinel#example-sentinel-deployments)
-also lists different network topologies and warns againts situations like
-network partition and how it can affect the state of the HA solution. Make
-sure you read it carefully and understand the implications in your current
-setup.
+While it doesn't require a list of all Sentinel nodes, in case of a failure,
+it needs to access at least one of the listed.
 
-To make Sentinel setup easier, ee provide an [automated way to setup and run](#sentinel-setup-ee-only)
-the Sentinel daemon with GitLab EE.
+>**Note:**
+The following steps should be performed in the [GitLab application server](gitlab.md)
+which ideally should not have Redis or Sentinels on it for a HA setup.
 
-#### Sentinel setup (Community Edition)
+1. Edit `/etc/gitlab/gitlab.rb` and add/change the following lines:
 
-For GitLab CE, you need to install, configure, execute and monitor Sentinel
-by yourself.
+    ```
+    ## Must be the same in every sentinel node
+    redis['master_name'] = 'gitlab-redis'
+
+    ## The same password for Redis authentication you set up for the master node.
+    redis['password'] = 'redis-password-goes-here'
+
+    ## A list of sentinels with `host` and `port`
+    gitlab_rails['redis_sentinels'] = [
+      {'host' => '10.0.0.1', 'port' => 26379},
+      {'host' => '10.0.0.2', 'port' => 26379},
+      {'host' => '10.0.0.3', 'port' => 26379}
+    ]
+    ```
 
-Here is an example configuration file (`sentinel.conf`) for a minimal Sentinel
-node:
+1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
 
-```conf
-bind 0.0.0.0 # bind to all interfaces or change to a specific IP
-port 26379 # default sentinel port
-sentinel auth-pass gitlab-redis redis-password-goes-here
-sentinel monitor gitlab-redis 10.0.0.1 6379 2
-sentinel down-after-milliseconds gitlab-redis 10000
-sentinel config-epoch gitlab-redis 0
-sentinel leader-epoch gitlab-redis 0
+## Switching from an existing single-machine installation to Redis HA
+
+If you already have a single-machine GitLab install running, you will need to
+replicate from this machine first, before de-activating the Redis instance
+inside it.
+
+Your single-machine install will be the initial **Master**, and the `3` others
+should be configured as **Slave** pointing to this machine.
+
+After replication catches up, you will need to stop services in the
+single-machine install, to rotate the **Master** to one of the new nodes.
+
+Make the required changes in configuration and restart the new nodes again.
+
+To disable redis in the single install, edit `/etc/gitlab/gitlab.rb`:
+
+```ruby
+redis['enable'] = false
 ```
 
-#### Sentinel setup (EE Only)
+If you fail to replicate first, you may loose data (unprocessed background jobs).
+
+## Example of a minimal configuration with 1 master, 2 slaves and 3 Sentinels
+
+>**Note:**
+Redis Sentinel is bundled with Omnibus GitLab Enterprise Edition only. For
+different setups, read the
+[available configuration setups](#available-configuration-setups) section.
+
+In this example we consider that all servers have an internal network
+interface with IPs in the `10.0.0.x` range, and that they can connect
+to each other using these IPs.
+
+In a real world usage, you would also setup firewall rules to prevent
+unauthorized access from other machines and block traffic from the
+outside (Internet).
+
+We will use the same `3` nodes with **Redis** + **Sentinel** topology
+discussed in [Redis setup overview](#redis-setup-overview) and
+[Sentinel setup overview](#sentinel-setup-overview) documentation.
+
+Here is a list and description of each **machine** and the assigned **IP**:
+
+* `10.0.0.1`: Redis Master + Sentinel 1
+* `10.0.0.2`: Redis Slave 1 + Sentinel 2
+* `10.0.0.3`: Redis Slave 2 + Sentinel 3
+* `10.0.0.4`: GitLab application
+
+Please note that after the initial configuration, if a failover is initiated
+by the Sentinel nodes, the Redis nodes will be reconfigured and the **Master**
+will change permanently (including in `redis.conf`) from one node to the other,
+until a new failover is initiated again.
+
+The same thing will happen with `sentinel.conf` that will be overridden after the
+initial execution, after any new sentinel node starts watching the **Master**,
+or a failover promotes a different **Master** node.
 
-To setup sentinel, you must edit `/etc/gitlab/gitlab.rb` file.
-This is a minimal configuration required to run the daemon:
+### Example configuration for Redis master and Sentinel 1
+
+In `/etc/gitlab/gitlab.rb`:
 
 ```ruby
+redis_master_role['enable'] = true
+redis_sentinel_role['enable'] = true
+redis['bind'] = '10.0.0.1'
+redis['port'] = 6379
+redis['password'] = 'redis-password-goes-here'
 redis['master_name'] = 'gitlab-redis' # must be the same in every sentinel node
+redis['master_password'] = 'redis-password-goes-here' # the same value defined in redis['password'] in the master instance
 redis['master_ip'] = '10.0.0.1' # ip of the initial master redis instance
-redis['master_port'] = 6379 # port of the initial master redis instance
-redis['master_password'] = 'your-secure-password-here' # the same value defined in redis['password'] in the master instance
+#redis['master_port'] = 6379 # port of the initial master redis instance, uncomment to change to non default
+sentinel['bind'] = '10.0.0.1'
+# sentinel['port'] = 26379 # uncomment to change default port
+sentinel['quorum'] = 2
+# sentinel['down_after_milliseconds'] = 10000
+# sentinel['failover_timeout'] = 60000
+```
 
-sentinel['enable'] = true
-# sentinel['port'] = 26379
+[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
 
-## Quorum must reflect the amount of voting sentinels it take to start a failover.
-sentinel['quorum'] = 2
+### Example configuration for Redis slave 1 and Sentinel 2
 
-## Consider unresponsive server down after x amount of ms.
+In `/etc/gitlab/gitlab.rb`:
+
+```ruby
+redis_slave_role['enable'] = true
+redis_sentinel_role['enable'] = true
+redis['bind'] = '10.0.0.2'
+redis['port'] = 6379
+redis['password'] = 'redis-password-goes-here'
+redis['master_password'] = 'redis-password-goes-here'
+redis['master_ip'] = '10.0.0.1' # IP of master Redis server
+#redis['master_port'] = 6379 # Port of master Redis server, uncomment to change to non default
+redis['master_name'] = 'gitlab-redis' # must be the same in every sentinel node
+sentinel['bind'] = '10.0.0.2'
+# sentinel['port'] = 26379 # uncomment to change default port
+sentinel['quorum'] = 2
 # sentinel['down_after_milliseconds'] = 10000
+# sentinel['failover_timeout'] = 60000
+```
+
+[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
+
+### Example configuration for Redis slave 2 and Sentinel 3
+
+In `/etc/gitlab/gitlab.rb`:
 
+```ruby
+redis_slave_role['enable'] = true
+redis_sentinel_role['enable'] = true
+redis['bind'] = '10.0.0.3'
+redis['port'] = 6379
+redis['password'] = 'redis-password-goes-here'
+redis['master_password'] = 'redis-password-goes-here'
+redis['master_ip'] = '10.0.0.1' # IP of master Redis server
+#redis['master_port'] = 6379 # Port of master Redis server, uncomment to change to non default
+redis['master_name'] = 'gitlab-redis' # must be the same in every sentinel node
+sentinel['bind'] = '10.0.0.3'
+# sentinel['port'] = 26379 # uncomment to change default port
+sentinel['quorum'] = 2
+# sentinel['down_after_milliseconds'] = 10000
 # sentinel['failover_timeout'] = 60000
 ```
 
-When you install Sentinel in a separate machine, you need to control which
-other services will be running in it. Take a look at the following variables
-and enable or disable whenever it fits your strategy:
+[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
+
+### Example configuration for the GitLab application
+
+In `/etc/gitlab/gitlab.rb`:
 
 ```ruby
-# Enabled Redis and Sentinel services
-redis['enable'] = true
+redis['master_name'] = 'gitlab-redis'
+redis['password'] = 'redis-password-goes-here'
+gitlab_rails['redis_sentinels'] = [
+  {'host' => '10.0.0.1', 'port' => 26379},
+  {'host' => '10.0.0.2', 'port' => 26379},
+  {'host' => '10.0.0.3', 'port' => 26379}
+]
+```
+
+[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
+
+## Advanced configuration
+
+Omnibus GitLab configures some things behind the curtains to make the sysadmins'
+lives easier. If you want to know what happens underneath keep reading.
+
+### Control running services
+
+In the previous example, we've used `redis_sentinel_role` and
+`redis_master_role` which simplifies the amount of configuration changes.
+
+If you want more control, here is what each one sets for you automatically
+when enabled:
+
+```ruby
+## Redis Sentinel Role
+redis_sentinel_role['enable'] = true
+
+# When Sentinel Role is enabled, the following services are also enabled
 sentinel['enable'] = true
 
-# Disabled all other services
+# The following services are disabled
 redis['enable'] = false
 bootstrap['enable'] = false
 nginx['enable'] = false
-unicorn['enable'] = false
-sidekiq['enable'] = false
 postgresql['enable'] = false
-gitlab_workhorse['enable'] = false
 gitlab_rails['enable'] = false
 mailroom['enable'] = false
-```
 
-Remember that enabling a new service may also require additional configuration
-params (like `redis` for example).
+-------
 
----
+## Redis master/slave Role
+redis_master_role['enable'] = true # enable only one of them
+redis_slave_role['enable'] = true # enable only one of them
 
-The final part is to inform the main GitLab application server of the Redis
-master and the new sentinels servers.
+# When Redis Master or Slave role are enabled, the following services are
+# enabled/disabled. Note that if Redis and Sentinel roles are combined, both
+# services will be enabled.
 
-### GitLab setup
+# The following services are disabled
+sentinel['enable'] = false
+bootstrap['enable'] = false
+nginx['enable'] = false
+postgresql['enable'] = false
+gitlab_rails['enable'] = false
+mailroom['enable'] = false
 
-You can enable or disable sentinel support at any time in new or existing
-installations. From the GitLab application perspective, all it requires is
-the correct credentials for the master Redis and for a few Sentinel nodes.
+# For Redis Slave role, also change this setting from default 'true' to 'false':
+redis['master'] = false
+```
 
-It doesn't require a list of all Sentinel nodes, as in case of a failure,
-the application will need to query only one of them.
+You can find the relevant attributes defined in [gitlab_rails.rb][omnifile].
 
->**Note:**
-The following steps should be performed in the [GitLab application server](gitlab.md).
+## Troubleshooting
 
-**For source based installations**
+There are a lot of moving parts that needs to be taken care carefully
+in order for the HA setup to work as expected.
 
-1. Edit `/home/git/gitlab/config/resque.yml` following the example in
-   `/home/git/gitlab/config/resque.yml.example`, and uncomment the sentinels
-   line, changing to the correct server credentials.
-1. Restart GitLab for the changes to take effect.
+Before proceeding with the troubleshooting below, check your firewall rules:
 
-**For Omnibus installations**
+- Redis machines
+   - Accept TCP connection in `6379`
+   - Connect to the other Redis machines via TCP in `6379`
+- Sentinel machines
+   - Accept TCP connection in `26379`
+   - Connect to other Sentinel machines via TCP in `26379`
+   - Connect to the Redis machines via TCP in `6379`
 
-1. Edit `/etc/gitlab/gitlab.rb` and add/change the following lines:
+### Troubleshooting Redis replication
 
-    ```ruby
-    redis['master_name'] = "gitlab-redis"
-    redis['master_password'] = 'redis-password-goes-here'
-    gitlab_rails['redis_sentinels'] = [
-      {'host' => '10.10.10.1', 'port' => 26379},
-      {'host' => '10.10.10.2', 'port' => 26379},
-      {'host' => '10.10.10.3', 'port' => 26379}
-    ]
-    ```
+You can check if everything is correct by connecting to each server using
+`redis-cli` application, and sending the `INFO` command.
+
+If authentication was correctly defined, it should fail with:
+`NOAUTH Authentication required` error. Try to authenticate with the
+previous defined password with `AUTH redis-password-goes-here` and
+try the `INFO` command again.
+
+Look for the `# Replication` section where you should see some important
+information like the `role` of the server.
+
+When connected to a `master` redis, you will see the number of connected
+`slaves`, and a list of each with connection details:
+
+```
+# Replication
+role:master
+connected_slaves:1
+slave0:ip=10.133.5.21,port=6379,state=online,offset=208037514,lag=1
+master_repl_offset:208037658
+repl_backlog_active:1
+repl_backlog_size:1048576
+repl_backlog_first_byte_offset:206989083
+repl_backlog_histlen:1048576
+```
 
-1. [Reconfigure] the GitLab for the changes to take effect.
+When it's a `slave`, you will see details of the master connection and if
+its `up` or `down`:
 
-### Sentinel troubleshooting
+```
+# Replication
+role:slave
+master_host:10.133.1.58
+master_port:6379
+master_link_status:up
+master_last_io_seconds_ago:1
+master_sync_in_progress:0
+slave_repl_offset:208096498
+slave_priority:100
+slave_read_only:1
+connected_slaves:0
+master_repl_offset:0
+repl_backlog_active:0
+repl_backlog_size:1048576
+repl_backlog_first_byte_offset:0
+repl_backlog_histlen:0
+```
 
-#### Omnibus install
+### Troubleshooting Sentinel
 
 If you get an error like: `Redis::CannotConnectError: No sentinels available.`,
 there may be something wrong with your configuration files or it can be related
@@ -424,42 +816,6 @@ The way the redis connector `redis-rb` works with sentinel is a bit
 non-intuitive. We try to hide the complexity in omnibus, but it still requires
 a few extra configs.
 
-#### Source install
-
-If you get an error like: `Redis::CannotConnectError: No sentinels available.`,
-there may be something wrong with your configuration files or it can be related
-to [this issue][gh-531].
-
-It's a bit non-intuitive the way you have to config `resque.yml` and
-`sentinel.conf`, otherwise `redis-rb` will not work properly.
-
-The `master-group-name` ('gitlab-redis') defined in (`sentinel.conf`)
-**must** be used as the hostname in GitLab (`resque.yml` for source installations
-or `gitlab-rails['redis_*']` in Omnibus):
-
-```conf
-# sentinel.conf:
-sentinel monitor gitlab-redis 10.10.10.10 6379 2
-sentinel down-after-milliseconds gitlab-redis 10000
-sentinel config-epoch gitlab-redis 0
-sentinel leader-epoch gitlab-redis 0
-```
-
-```yaml
-# resque.yaml
-production:
-  url: redis://:myredispassword@gitlab-redis/
-  sentinels:
-    -
-      host: slave1.example.com # or use ip
-      port: 26380 # point to sentinel, not to redis port
-    -
-      host: slave2.exampl.com # or use ip
-      port: 26381 # point to sentinel, not to redis port
-```
-
-When in doubt, please read [Redis Sentinel documentation](http://redis.io/topics/sentinel)
-
 ---
 
 To make sure your configuration is correct:
@@ -487,8 +843,8 @@ To make sure your configuration is correct:
 1. To simulate a failover on master Redis, SSH into the Redis server and run:
 
     ```bash
-    # port must match your master redis port
-     redis-cli -h localhost -p 6379 DEBUG sleep 60
+    # port must match your master redis port, and the sleep time must be a few seconds bigger than defined one
+     redis-cli -h localhost -p 6379 DEBUG sleep 20
     ```
 
 1. Then back in the Rails console from the first step, run:
@@ -500,10 +856,34 @@ To make sure your configuration is correct:
     You should see a different port after a few seconds delay
     (the failover/reconnect time).
 
----
 
-Read more on high-availability configuration:
+## Changelog
+
+Changes to Redis HA over time.
+
+**8.14**
+
+- Redis Sentinel support is production-ready and bundled in the Omnibus GitLab
+  Enterprise Edition package
+- Documentation restructure for better readability
+
+**8.11**
+
+- Experimental Redis Sentinel support was added
+
+### Experimental Redis Sentinel support
+
+>
+Experimental Redis Sentinel support was [Introduced][ce-1877] in GitLab 8.11.
+Starting with 8.14, Redis Sentinel is no longer experimental.
+If you used with versions `< 8.14` before, please check the updated
+documentation here.
+
+## Further reading
+
+Read more on High Availability:
 
+1. [High Availability Overview](README.md)
 1. [Configure the database](database.md)
 1. [Configure NFS](nfs.md)
 1. [Configure the GitLab application servers](gitlab.md)
@@ -514,3 +894,10 @@ Read more on high-availability configuration:
 [reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure
 [gh-531]: https://github.com/redis/redis-rb/issues/531
 [gh-534]: https://github.com/redis/redis-rb/issues/534
+[redis]: http://redis.io/
+[sentinel]: http://redis.io/topics/sentinel
+[omnifile]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/libraries/gitlab_rails.rb
+[source]: ../../install/installation.md
+[ce]: https://about.gitlab.com/downloads
+[ee]: https://about.gitlab.com/downloads-ee
+[it]: https://gitlab.com/gitlab-org/gitlab-ce/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png
diff --git a/doc/administration/high_availability/redis_source.md b/doc/administration/high_availability/redis_source.md
new file mode 100644
index 00000000000..e7a8f47355d
--- /dev/null
+++ b/doc/administration/high_availability/redis_source.md
@@ -0,0 +1,387 @@
+# Configuring non-Omnibus Redis for GitLab HA
+
+This is the documentation for configuring a Highly Available Redis setup when
+you have installed Redis all by yourself and not using the bundled one that
+comes with the Omnibus packages.
+
+We cannot stress enough the importance of reading the
+[Overview section](redis.md#overview) of the Omnibus Redis HA as it provides
+some invaluable information to the configuration of Redis. Please proceed to
+read it before going forward with this guide.
+
+We also highly recommend that you use the Omnibus GitLab packages, as we
+optimize them specifically for GitLab, and we will take care of upgrading Redis
+to the latest supported version.
+
+If you're not sure whether this guide is for you, please refer to
+[Available configuration setups](redis.md#available-configuration-setups) in
+the Omnibus Redis HA documentation.
+
+---
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+**Table of Contents**
+
+- [Configuring your own Redis server](#configuring-your-own-redis-server)
+  - [Prerequisites](#prerequisites)
+  - [Step 1. Configuring the master Redis instance](#step-1-configuring-the-master-redis-instance)
+  - [Step 2. Configuring the slave Redis instances](#step-2-configuring-the-slave-redis-instances)
+  - [Step 3. Configuring the Redis Sentinel instances](#step-3-configuring-the-redis-sentinel-instances)
+  - [Step 4. Configuring the GitLab application](#step-4-configuring-the-gitlab-application)
+- [Example of minimal configuration with 1 master, 2 slaves and 3 Sentinels](#example-of-minimal-configuration-with-1-master-2-slaves-and-3-sentinels)
+  - [Example configuration for Redis master and Sentinel 1](#example-configuration-for-redis-master-and-sentinel-1)
+  - [Example configuration for Redis slave 1 and Sentinel 2](#example-configuration-for-redis-slave-1-and-sentinel-2)
+  - [Example configuration for Redis slave 2 and Sentinel 3](#example-configuration-for-redis-slave-2-and-sentinel-3)
+  - [Example configuration of the GitLab application](#example-configuration-of-the-gitlab-application)
+- [Troubleshooting](#troubleshooting)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## Configuring your own Redis server
+
+This is the section where we install and setup the new Redis instances.
+
+### Prerequisites
+
+- All Redis servers in this guide must be configured to use a TCP connection
+  instead of a socket. To configure Redis to use TCP connections you need to
+  define both `bind` and `port` in the Redis config file. You can bind to all
+  interfaces (`0.0.0.0`) or specify the IP of the desired interface
+  (e.g., one from an internal network).
+- Since Redis 3.2, you must define a password to receive external connections
+  (`requirepass`).
+- If you are using Redis with Sentinel, you will also need to define the same
+  password for the slave password definition (`masterauth`) in the same instance.
+
+In addition, read the prerequisites as described in the
+[Omnibus Redis HA document](redis.md#prerequisites) since they provide some
+valuable information for the general setup.
+
+### Step 1. Configuring the master Redis instance
+
+Assuming that the Redis master instance IP is `10.0.0.1`:
+
+1. [Install Redis](../../install/installation.md#6-redis)
+1. Edit `/etc/redis/redis.conf`:
+
+    ```conf
+    ## Define a `bind` address pointing to a local IP that your other machines
+    ## can reach you. If you really need to bind to an external accessible IP, make
+    ## sure you add extra firewall rules to prevent unauthorized access:
+    bind 10.0.0.1
+
+    ## Define a `port` to force redis to listen on TCP so other machines can
+    ## connect to it (default port is `6379`).
+    port 6379
+
+    ## Set up password authentication (use the same password in all nodes).
+    ## The password should be defined equal for both `requirepass` and `masterauth`
+    ## when setting up Redis to use with Sentinel.
+    requirepass redis-password-goes-here
+    masterauth redis-password-goes-here
+    ```
+
+1. Restart the Redis service for the changes to take effect.
+
+### Step 2. Configuring the slave Redis instances
+
+Assuming that the Redis slave instance IP is `10.0.0.2`:
+
+1. [Install Redis](../../install/installation.md#6-redis)
+1. Edit `/etc/redis/redis.conf`:
+
+    ```conf
+    ## Define a `bind` address pointing to a local IP that your other machines
+    ## can reach you. If you really need to bind to an external accessible IP, make
+    ## sure you add extra firewall rules to prevent unauthorized access:
+    bind 10.0.0.2
+
+    ## Define a `port` to force redis to listen on TCP so other machines can
+    ## connect to it (default port is `6379`).
+    port 6379
+
+    ## Set up password authentication (use the same password in all nodes).
+    ## The password should be defined equal for both `requirepass` and `masterauth`
+    ## when setting up Redis to use with Sentinel.
+    requirepass redis-password-goes-here
+    masterauth redis-password-goes-here
+
+    ## Define `slaveof` pointing to the Redis master instance with IP and port.
+    slaveof 10.0.0.1 6379
+    ```
+
+1. Restart the Redis service for the changes to take effect.
+1. Go through the steps again for all the other slave nodes.
+
+### Step 3. Configuring the Redis Sentinel instances
+
+Sentinel is a special type of Redis server. It inherits most of the basic
+configuration options you can define in `redis.conf`, with specific ones
+starting with `sentinel` prefix.
+
+Assuming that the Redis Sentinel is installed on the same instance as Redis
+master with IP `10.0.0.1` (some settings might overlap with the master):
+
+1. [Install Redis Sentinel](http://redis.io/topics/sentinel)
+1. Edit `/etc/redis/sentinel.conf`:
+
+    ```conf
+    ## Define a `bind` address pointing to a local IP that your other machines
+    ## can reach you. If you really need to bind to an external accessible IP, make
+    ## sure you add extra firewall rules to prevent unauthorized access:
+    bind 10.0.0.1
+
+    ## Define a `port` to force Sentinel to listen on TCP so other machines can
+    ## connect to it (default port is `6379`).
+    port 26379
+
+    ## Set up password authentication (use the same password in all nodes).
+    ## The password should be defined equal for both `requirepass` and `masterauth`
+    ## when setting up Redis to use with Sentinel.
+    requirepass redis-password-goes-here
+    masterauth redis-password-goes-here
+
+    ## Define with `sentinel auth-pass` the same shared password you have
+    ## defined for both Redis master and slaves instances.
+    sentinel auth-pass gitlab-redis redis-password-goes-here
+
+    ## Define with `sentinel monitor` the IP and port of the Redis
+    ## master node, and the quorum required to start a failover.
+    sentinel monitor gitlab-redis 10.0.0.1 6379 2
+
+    ## Define with `sentinel down-after-milliseconds` the time in `ms`
+    ## that an unresponsive server will be considered down.
+    sentinel down-after-milliseconds gitlab-redis 10000
+
+    ## Define a value for `sentinel failover_timeout` in `ms`. This has multiple
+    ## meanings:
+    ##
+    ## * The time needed to re-start a failover after a previous failover was
+    ##   already tried against the same master by a given Sentinel, is two
+    ##   times the failover timeout.
+    ##
+    ## * The time needed for a slave replicating to a wrong master according
+    ##   to a Sentinel current configuration, to be forced to replicate
+    ##   with the right master, is exactly the failover timeout (counting since
+    ##   the moment a Sentinel detected the misconfiguration).
+    ##
+    ## * The time needed to cancel a failover that is already in progress but
+    ##   did not produced any configuration change (SLAVEOF NO ONE yet not
+    ##   acknowledged by the promoted slave).
+    ##
+    ## * The maximum time a failover in progress waits for all the slaves to be
+    ##   reconfigured as slaves of the new master. However even after this time
+    ##   the slaves will be reconfigured by the Sentinels anyway, but not with
+    ##   the exact parallel-syncs progression as specified.
+    sentinel failover_timeout 30000
+    ```
+1. Restart the Redis service for the changes to take effect.
+1. Go through the steps again for all the other Sentinel nodes.
+
+### Step 4. Configuring the GitLab application
+
+You can enable or disable Sentinel support at any time in new or existing
+installations. From the GitLab application perspective, all it requires is
+the correct credentials for the Sentinel nodes.
+
+While it doesn't require a list of all Sentinel nodes, in case of a failure,
+it needs to access at least one of listed ones.
+
+The following steps should be performed in the [GitLab application server](gitlab.md)
+which ideally should not have Redis or Sentinels in the same machine for a HA
+setup:
+
+1. Edit `/home/git/gitlab/config/resque.yml` following the example in
+   [resque.yml.example][resque], and uncomment the Sentinel lines, pointing to
+   the correct server credentials:
+
+    ```yaml
+    # resque.yaml
+    production:
+      url: redis://:redi-password-goes-here@gitlab-redis/
+      sentinels:
+        -
+          host: 10.0.0.1
+          port: 26379 # point to sentinel, not to redis port
+        -
+          host: 10.0.0.2
+          port: 26379 # point to sentinel, not to redis port
+        -
+          host: 10.0.0.3
+          port: 26379 # point to sentinel, not to redis port
+    ```
+
+1. [Restart GitLab][restart] for the changes to take effect.
+
+## Example of minimal configuration with 1 master, 2 slaves and 3 Sentinels
+
+In this example we consider that all servers have an internal network
+interface with IPs in the `10.0.0.x` range, and that they can connect
+to each other using these IPs.
+
+In a real world usage, you would also setup firewall rules to prevent
+unauthorized access from other machines, and block traffic from the
+outside ([Internet][it]).
+
+For this example, **Sentinel 1** will be configured in the same machine as the
+**Redis Master**, **Sentinel 2** and **Sentinel 3** in the same machines as the
+**Slave 1** and **Slave 2** respectively.
+
+Here is a list and description of each **machine** and the assigned **IP**:
+
+* `10.0.0.1`: Redis Master + Sentinel 1
+* `10.0.0.2`: Redis Slave 1 + Sentinel 2
+* `10.0.0.3`: Redis Slave 2 + Sentinel 3
+* `10.0.0.4`: GitLab application
+
+Please note that after the initial configuration, if a failover is initiated
+by the Sentinel nodes, the Redis nodes will be reconfigured and the **Master**
+will change permanently (including in `redis.conf`) from one node to the other,
+until a new failover is initiated again.
+
+The same thing will happen with `sentinel.conf` that will be overridden after the
+initial execution, after any new sentinel node starts watching the **Master**,
+or a failover promotes a different **Master** node.
+
+### Example configuration for Redis master and Sentinel 1
+
+1. In `/etc/redis/redis.conf`:
+
+    ```conf
+    bind 10.0.0.1
+    port 6379
+    requirepass redis-password-goes-here
+    masterauth redis-password-goes-here
+    ```
+
+1. In `/etc/redis/sentinel.conf`:
+
+    ```conf
+    bind 10.0.0.1
+    port 26379
+    sentinel auth-pass gitlab-redis redis-password-goes-here
+    sentinel monitor gitlab-redis 10.0.0.1 6379 2
+    sentinel down-after-milliseconds gitlab-redis 10000
+    sentinel failover_timeout 30000
+    ```
+
+1. Restart the Redis service for the changes to take effect.
+
+### Example configuration for Redis slave 1 and Sentinel 2
+
+1. In `/etc/redis/redis.conf`:
+
+    ```conf
+    bind 10.0.0.2
+    port 6379
+    requirepass redis-password-goes-here
+    masterauth redis-password-goes-here
+    slaveof 10.0.0.1 6379
+    ```
+
+1. In `/etc/redis/sentinel.conf`:
+
+    ```conf
+    bind 10.0.0.2
+    port 26379
+    sentinel auth-pass gitlab-redis redis-password-goes-here
+    sentinel monitor gitlab-redis 10.0.0.1 6379 2
+    sentinel down-after-milliseconds gitlab-redis 10000
+    sentinel failover_timeout 30000
+    ```
+
+1. Restart the Redis service for the changes to take effect.
+
+### Example configuration for Redis slave 2 and Sentinel 3
+
+1. In `/etc/redis/redis.conf`:
+
+    ```conf
+    bind 10.0.0.3
+    port 6379
+    requirepass redis-password-goes-here
+    masterauth redis-password-goes-here
+    slaveof 10.0.0.1 6379
+    ```
+
+1. In `/etc/redis/sentinel.conf`:
+
+    ```conf
+    bind 10.0.0.3
+    port 26379
+    sentinel auth-pass gitlab-redis redis-password-goes-here
+    sentinel monitor gitlab-redis 10.0.0.1 6379 2
+    sentinel down-after-milliseconds gitlab-redis 10000
+    sentinel failover_timeout 30000
+    ```
+
+1. Restart the Redis service for the changes to take effect.
+
+### Example configuration of the GitLab application
+
+1. Edit `/home/git/gitlab/config/resque.yml`:
+
+    ```yaml
+    production:
+      url: redis://:redi-password-goes-here@gitlab-redis/
+      sentinels:
+        -
+          host: 10.0.0.1
+          port: 26379 # point to sentinel, not to redis port
+        -
+          host: 10.0.0.2
+          port: 26379 # point to sentinel, not to redis port
+        -
+          host: 10.0.0.3
+          port: 26379 # point to sentinel, not to redis port
+    ```
+
+1. [Restart GitLab][restart] for the changes to take effect.
+
+## Troubleshooting
+
+We have a more detailed [Troubleshooting](redis.md#troubleshooting) explained
+in the documentation for Omnibus GitLab installations. Here we will list only
+the things that are specific to a source installation.
+
+If you get an error in GitLab like: `Redis::CannotConnectError: No sentinels available.`,
+there may be something wrong with your configuration files or it can be related
+to [this upstream issue][gh-531].
+
+It's a bit non-intuitive the way you have to config `resque.yml` and
+`sentinel.conf`, otherwise `redis-rb` will not work properly.
+
+The `master-group-name` ('gitlab-redis') defined in (`sentinel.conf`)
+**must** be used as the hostname in GitLab (`resque.yml`):
+
+```conf
+# sentinel.conf:
+sentinel monitor gitlab-redis 10.0.0.1 6379 2
+sentinel down-after-milliseconds gitlab-redis 10000
+sentinel config-epoch gitlab-redis 0
+sentinel leader-epoch gitlab-redis 0
+```
+
+```yaml
+# resque.yaml
+production:
+  url: redis://:myredispassword@gitlab-redis/
+  sentinels:
+    -
+      host: 10.0.0.1
+      port: 26379 # point to sentinel, not to redis port
+    -
+      host: 10.0.0.2
+      port: 26379 # point to sentinel, not to redis port
+    -
+      host: 10.0.0.3
+      port: 26379 # point to sentinel, not to redis port
+```
+
+When in doubt, please read [Redis Sentinel documentation](http://redis.io/topics/sentinel)
+
+[gh-531]: https://github.com/redis/redis-rb/issues/531
+[downloads]: https://about.gitlab.com/downloads
+[restart]: ../restart_gitlab.md#installations-from-source
+[it]: https://gitlab.com/gitlab-org/gitlab-ce/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png