Add latest changes from gitlab-org/gitlab@13-2-stable-ee

3 files changed, 80 insertions, 33 deletions

diff --git a/doc/raketasks/README.md b/doc/raketasks/README.md
index eaaf1ebed99..b7cfc18534b 100644
--- a/doc/raketasks/README.md
+++ b/doc/raketasks/README.md
@@ -29,9 +29,10 @@ The following are available Rake tasks:
 | [Import repositories](import.md)                                                                    | Import bare repositories into your GitLab instance.                                       |
 | [Import large project exports](../development/import_project.md#importing-via-a-rake-task)          | Import large GitLab [project exports](../user/project/settings/import_export.md).         |
 | [Integrity checks](../administration/raketasks/check.md)                                            | Check the integrity of repositories, files, and LDAP.                                     |
-| [LDAP maintenance](../administration/raketasks/ldap.md)                                    | [LDAP](../administration/auth/ldap/index.md)-related tasks.                                     |
+| [LDAP maintenance](../administration/raketasks/ldap.md)                                             | [LDAP](../administration/auth/ldap/index.md)-related tasks.                               |
 | [List repositories](list_repos.md)                                                                  | List of all GitLab-managed Git repositories on disk.                                      |
 | [Migrate Snippets to Git](migrate_snippets.md)                                                      | Migrate GitLab Snippets to Git repositories and show migration status                     |
+| [Praefect Rake tasks](../administration/raketasks/praefect.md)                                      | [Praefect](../administration/gitaly/praefect.md)-related tasks.                           |
 | [Project import/export](../administration/raketasks/project_import_export.md)                       | Prepare for [project exports and imports](../user/project/settings/import_export.md).     |
 | [Sample Prometheus data](generate_sample_prometheus_data.md)                                        | Generate sample Prometheus data.                                                          |
 | [Repository storage](../administration/raketasks/storage.md)                                        | List and migrate existing projects and attachments from legacy storage to hashed storage. |
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index 18c1cba54f7..d8522cc19a0 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -1,4 +1,4 @@
-# Backing up and restoring GitLab **(CORE ONLY)**
+# Back up and restore GitLab **(CORE ONLY)**
 
 GitLab provides Rake tasks for backing up and restoring GitLab instances.
 
@@ -26,14 +26,6 @@ installed on your system.
     sudo yum install rsync
     ```
 
-- **Tar**: Backup and restore tasks use `tar` under the hood to create and extract
-  archives. Ensure you have version 1.30 or above of `tar` available in your
-  system. To check the version, run:
-
-  ```shell
-  tar --version
-  ```
-
 ## Backup timestamp
 
 NOTE: **Note:**
@@ -75,7 +67,7 @@ Use this command if you've installed GitLab with the Omnibus package:
 sudo gitlab-backup create
 ```
 
-NOTE: **Note**
+NOTE: **Note:**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
 Use this if you've installed GitLab from source:
@@ -90,7 +82,7 @@ If you are running GitLab within a Docker container, you can run the backup from
 docker exec -t <container name> gitlab-backup create
 ```
 
-NOTE: **Note**
+NOTE: **Note:**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
 If you are using the [GitLab Helm chart](https://gitlab.com/gitlab-org/charts/gitlab) on a
@@ -206,7 +198,7 @@ To use the `copy` strategy instead of the default streaming strategy, specify
 sudo gitlab-backup create STRATEGY=copy
 ```
 
-NOTE: **Note**
+NOTE: **Note:**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
 #### Backup filename
@@ -221,7 +213,7 @@ By default a backup file is created according to the specification in [the Backu
 sudo gitlab-backup create BACKUP=dump
 ```
 
-NOTE: **Note**
+NOTE: **Note:**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
 The resulting file will then be `dump_gitlab_backup.tar`. This is useful for systems that make use of rsync and incremental backups, and will result in considerably faster transfer speeds.
@@ -236,7 +228,7 @@ Note that the `--rsyncable` option in `gzip` is not guaranteed to be available o
 sudo gitlab-backup create BACKUP=dump GZIP_RSYNCABLE=yes
 ```
 
-NOTE: **Note**
+NOTE: **Note:**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
 #### Excluding specific directories from the backup
@@ -264,7 +256,7 @@ For Omnibus GitLab packages:
 sudo gitlab-backup create SKIP=db,uploads
 ```
 
-NOTE: **Note**
+NOTE: **Note:**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
 For installations from source:
@@ -466,7 +458,13 @@ For Omnibus GitLab packages:
    gitlab_rails['backup_upload_connection'] = {
      'provider' => 'Google',
      'google_storage_access_key_id' => 'Access Key',
-     'google_storage_secret_access_key' => 'Secret'
+     'google_storage_secret_access_key' => 'Secret',
+
+     ## If you have CNAME buckets (foo.example.com), you might run into SSL issues
+     ## when uploading backups ("hostname foo.example.com.storage.googleapis.com
+     ## does not match the server certificate"). In that case, uncomnent the following
+     ## setting. See: https://github.com/fog/fog/issues/2834
+     #'path_style' => true
    }
    gitlab_rails['backup_upload_remote_directory'] = 'my.google.bucket'
    ```
@@ -499,7 +497,7 @@ sudo gitlab-backup create DIRECTORY=daily
 sudo gitlab-backup create DIRECTORY=weekly
 ```
 
-NOTE: **Note**
+NOTE: **Note:**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
 #### Uploading to locally mounted shares
@@ -517,7 +515,8 @@ backups will be copied to, and will be created if it does not exist. If the
 directory that you want to copy the tarballs to is the root of your mounted
 directory, just use `.` instead.
 
-NOTE: **Note:** Since file system performance may affect GitLab's overall performance, we do not recommend using EFS for storage. See the [relevant documentation](../administration/high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) for more details.
+NOTE: **Note:**
+Since file system performance may affect GitLab's overall performance, we do not recommend using EFS for storage. See the [relevant documentation](../administration/high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) for more details.
 
 For Omnibus GitLab packages:
 
@@ -605,7 +604,7 @@ For Omnibus GitLab packages:
    0 2 * * * /opt/gitlab/bin/gitlab-backup create CRON=1
    ```
 
-   NOTE: **Note**
+   NOTE: **Note:**
    For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`.
 
 For installations from source:
@@ -679,7 +678,7 @@ You can only restore a backup to **exactly the same version and type (CE/EE)** o
 GitLab that you created it on, for example CE 9.1.0.
 
 If your backup is a different version than the current installation, you will
-need to [downgrade your GitLab installation](https://docs.gitlab.com/omnibus/update/README.html#downgrading)
+need to [downgrade your GitLab installation](https://docs.gitlab.com/omnibus/update/README.html#downgrade)
 before restoring the backup.
 
 ### Restore prerequisites
@@ -806,7 +805,7 @@ restore:
 sudo gitlab-backup restore BACKUP=11493107454_2018_04_25_10.6.4-ce
 ```
 
-NOTE: **Note**
+NOTE: **Note:**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:restore`.
 
 CAUTION: **Warning:**
@@ -828,6 +827,12 @@ If there is a GitLab version mismatch between your backup tar file and the insta
 version of GitLab, the restore command will abort with an error. Install the
 [correct GitLab version](https://packages.gitlab.com/gitlab/) and try again.
 
+NOTE: **Note:**
+There is currently a [known issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/3470) for restore not working
+with `pgbouncer`. In order to workaround the issue, the Rails node will need to bypass `pgbouncer` and connect
+directly to the primary database node. This can be done by setting `gitlab_rails['db_host']` and `gitlab_rails['port']`
+to connect to the primary database node and [reconfiguring GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure).
+
 ### Restore for Docker image and GitLab Helm chart installations
 
 For GitLab installations using the Docker image or the GitLab Helm chart on
@@ -845,10 +850,25 @@ backup location (default location is `/var/opt/gitlab/backups`).
 For Docker installations, the restore task can be run from host:
 
 ```shell
-docker exec -it <name of container> gitlab-backup restore
+# Stop the processes that are connected to the database
+docker exec -it <name of container> gitlab-ctl stop unicorn
+docker exec -it <name of container> gitlab-ctl stop puma
+docker exec -it <name of container> gitlab-ctl stop sidekiq
+
+# Verify that the processes are all down before continuing
+docker exec -it <name of container> gitlab-ctl status
+
+# Run the restore
+docker exec -it <name of container> gitlab-backup restore BACKUP=11493107454_2018_04_25_10.6.4-ce
+
+# Restart the GitLab container
+docker restart <name of container>
+
+# Check GitLab
+docker exec -it <name of container> gitlab-rake gitlab:check SANITIZE=true
 ```
 
-NOTE: **Note**
+NOTE: **Note:**
 For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:restore`.
 
 CAUTION: **Warning:**
@@ -859,6 +879,28 @@ use `gitlab-backup restore` to avoid this issue.
 The GitLab Helm chart uses a different process, documented in
 [restoring a GitLab Helm chart installation](https://gitlab.com/gitlab-org/charts/gitlab/blob/master/doc/backup-restore/restore.md).
 
+### Restoring only one or a few project(s) or group(s) from a backup
+
+While the Rake task used to restore a GitLab instance doesn't support
+restoring a single project or group, you can use a workaround by
+restoring your backup to a separate, temporary GitLab instance, and
+export your project or group from there:
+
+1. [Install a new GitLab](../install/README.md) instance at the same version as
+   the backed-up instance from which you want to restore.
+1. [Restore the backup](#restore-gitlab) into this new instance and
+   export your [project](../user/project/settings/import_export.md)
+   or [group](../user/group/settings/import_export.md). Make sure to
+   read the **Important Notes** on either export feature's documentation
+   to understand what will be exported and what not.
+1. Once the export is complete, go to the old instance and import it.
+1. After importing only the project(s) or group(s) that you wanted is complete,
+   you may delete the new, temporary GitLab instance.
+
+NOTE: **Note:**
+A feature request to provide direct restore of individual projects or groups
+is being discussed in [issue #17517](https://gitlab.com/gitlab-org/gitlab/-/issues/17517).
+
 ## Alternative backup strategies
 
 If your GitLab server contains a lot of Git repository data you may find the GitLab backup script to be too slow.
@@ -914,7 +956,7 @@ Be advised that, backup is successfully restored in spite of these warnings.
 The Rake task runs this as the `gitlab` user which does not have the superuser access to the database. When restore is initiated it will also run as `gitlab` user but it will also try to alter the objects it does not have access to.
 Those objects have no influence on the database backup/restore but they give this annoying warning.
 
-For more information see similar questions on PostgreSQL issue tracker[here](https://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com) and [here](https://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us) as well as [stack overflow](https://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql).
+For more information see similar questions on PostgreSQL issue tracker [here](https://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com) and [here](https://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us) as well as [stack overflow](https://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql).
 
 ### When the secrets file is lost
 
diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md
index 5bdae998ec9..cf4edea383b 100644
--- a/doc/raketasks/cleanup.md
+++ b/doc/raketasks/cleanup.md
@@ -2,7 +2,7 @@
 
 GitLab provides Rake tasks for cleaning up GitLab instances.
 
-## Remove unreferenced LFS files from filesystem
+## Remove unreferenced LFS files
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36628) in GitLab 12.10.
 
@@ -42,7 +42,7 @@ Note that this Rake task only removes the references to LFS files. Unreferenced
 later (once a day). If you need to garbage collect them immediately, run
 `rake gitlab:cleanup:orphan_lfs_files` described below.
 
-## Remove unreferenced LFS files
+### Remove unreferenced LFS files immediately
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36628) in GitLab 12.10.
 
@@ -64,7 +64,11 @@ $ sudo gitlab-rake gitlab:cleanup:orphan_lfs_files
 I, [2020-01-08T20:51:17.148765 #43765]  INFO -- : Removed unreferenced LFS files: 12
 ```
 
-## Remove garbage from filesystem
+## Clean up project upload files
+
+Clean up project upload files if they don't exist in GitLab database.
+
+### Clean up project upload files from filesystem
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20863) in GitLab 11.2.
 
@@ -100,11 +104,11 @@ I, [2018-07-27T12:08:33.755624 #89817]  INFO -- : Did fix /opt/gitlab/embedded/s
 I, [2018-07-27T12:08:33.760257 #89817]  INFO -- : Did move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png
 ```
 
-## Remove garbage from object storage
+### Clean up project upload files from object storage
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20918) in GitLab 11.2.
 
-Remove object store upload files if they don't exist in GitLab database.
+Move object store upload files to a lost and found directory if they don't exist in GitLab database.
 
 ```shell
 # omnibus-gitlab
@@ -142,7 +146,7 @@ When you notice there are more job artifacts files on disk than there
 should be, you can run:
 
 ```shell
-gitlab-rake gitlab:cleanup:orphan_job_artifact_files
+sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files
 ```
 
 This command:
@@ -156,13 +160,13 @@ delete. Run the command with `DRY_RUN=false` if you actually want to
 delete the files:
 
 ```shell
-gitlab-rake gitlab:cleanup:orphan_job_artifact_files DRY_RUN=false
+sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files DRY_RUN=false
 ```
 
 You can also limit the number of files to delete with `LIMIT`:
 
 ```shell
-gitlab-rake gitlab:cleanup:orphan_job_artifact_files LIMIT=100
+sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files LIMIT=100
 ```
 
 This will only delete up to 100 files from disk. You can use this to