Merge branch 'decouple-secret-keys' into 'master'

Store OTP secret key in secrets.yml

## What does this MR do?

Migrate the value of `.secret` to `config/secrets.yml` if present, so that `.secret` can be rotated without preventing all users with 2FA from logging in. (On a clean setup, generate different keys for each.)

## Are there points in the code the reviewer needs to double check?

I'm not sure we actually need `.secret` at all after this, but it seems safer not to touch it.

## Why was this MR needed?

We have some DB encryption keys in `config/secrets.yml`, and one in `.secret`. They should all be in the same place.

## What are the relevant issue numbers?

#3963, which isn't closed until I make the relevant changes in Omnibus too.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- ~~API support added~~
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

See merge request !5274

2 files changed, 19 insertions, 17 deletions

diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index 5fa96736d59..835af5443a3 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -11,12 +11,13 @@ You can only restore a backup to exactly the same version of GitLab that you cre
 on, for example 7.2.1. The best way to migrate your repositories from one server to
 another is through backup restore.
 
-You need to keep a separate copy of `/etc/gitlab/gitlab-secrets.json`
-(for omnibus packages) or `/home/git/gitlab/.secret` (for installations
-from source). This file contains the database encryption key used
-for two-factor authentication. If you restore a GitLab backup without
-restoring the database encryption key, users who have two-factor
-authentication enabled will lose access to your GitLab server.
+You need to keep separate copies of `/etc/gitlab/gitlab-secrets.json` and
+`/etc/gitlab/gitlab.rb` (for omnibus packages) or
+`/home/git/gitlab/config/secrets.yml` (for installations from source). This file
+contains the database encryption keys used for two-factor authentication and CI
+secret variables, among other things. If you restore a GitLab backup without
+restoring the database encryption key, users who have two-factor authentication
+enabled will lose access to your GitLab server.
 
 ```
 # use this command if you've installed GitLab with the Omnibus package
@@ -221,11 +222,12 @@ of using encryption in the first place!
 
 If you use an Omnibus package please see the [instructions in the readme to backup your configuration](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#backup-and-restore-omnibus-gitlab-configuration).
 If you have a cookbook installation there should be a copy of your configuration in Chef.
-If you have an installation from source, please consider backing up your `.secret` file, `gitlab.yml` file, any SSL keys and certificates, and your [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079).
+If you have an installation from source, please consider backing up your `config/secrets.yml` file, `gitlab.yml` file, any SSL keys and certificates, and your [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079).
 
-At the very **minimum** you should backup `/etc/gitlab/gitlab-secrets.json`
-(Omnibus) or `/home/git/gitlab/.secret` (source) to preserve your
-database encryption key.
+At the very **minimum** you should backup `/etc/gitlab/gitlab.rb` and
+`/etc/gitlab/gitlab-secrets.json` (Omnibus), or
+`/home/git/gitlab/config/secrets.yml` (source) to preserve your database
+encryption key.
 
 ## Restore a previously created backup
 
@@ -240,11 +242,11 @@ the SQL database it needs to import data into ('gitlabhq_production').
 All existing data will be either erased (SQL) or moved to a separate
 directory (repositories, uploads).
 
-If some or all of your GitLab users are using two-factor authentication
-(2FA) then you must also make sure to restore
-`/etc/gitlab/gitlab-secrets.json` (Omnibus) or `/home/git/gitlab/.secret`
-(installations from source). Note that you need to run `gitlab-ctl
-reconfigure` after changing `gitlab-secrets.json`.
+If some or all of your GitLab users are using two-factor authentication (2FA)
+then you must also make sure to restore `/etc/gitlab/gitlab.rb` and
+`/etc/gitlab/gitlab-secrets.json` (Omnibus), or
+`/home/git/gitlab/config/secrets.yml` (installations from source). Note that you
+need to run `gitlab-ctl reconfigure` after changing `gitlab-secrets.json`.
 
 ### Installation from source
 
diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md
index 629d38efc53..8a5e2d6e16b 100644
--- a/doc/raketasks/user_management.md
+++ b/doc/raketasks/user_management.md
@@ -60,8 +60,8 @@ block_auto_created_users: false
 ## Disable Two-factor Authentication (2FA) for all users
 
 This task will disable 2FA for all users that have it enabled. This can be
-useful if GitLab's `.secret` file has been lost and users are unable to login,
-for example.
+useful if GitLab's `config/secrets.yml` file has been lost and users are unable
+to login, for example.
 
 ```bash
 # omnibus-gitlab