Add latest changes from gitlab-org/gitlab@13-10-stable-eev13.10.0-rc40

3 files changed, 139 insertions, 104 deletions

diff --git a/doc/update/index.md b/doc/update/index.md
index a07a6f4b0b7..5c26ddb7ec6 100644
--- a/doc/update/index.md
+++ b/doc/update/index.md
@@ -138,16 +138,46 @@ pending_job_classes = scheduled_queue.select { |job| job["class"] == "Background
 pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) }
 ```
 
+## Checking for pending Advanced Search migrations
+
+This section is only applicable if you have enabled the [Elasticsearch
+integration](../integration/elasticsearch.md).
+
+Major releases require all [Advanced Search
+migrations](../integration/elasticsearch.md#advanced-search-migrations) to
+be finished from the most recent minor release in your current version
+before the major version upgrade. You can find pending migrations by
+running the following command:
+
+**For Omnibus installations**
+
+```shell
+sudo gitlab-rake gitlab:elastic:list_pending_migrations
+```
+
+**For installations from source**
+
+```shell
+cd /home/git/gitlab
+sudo -u git -H bundle exec rake gitlab:elastic:list_pending_migrations
+```
+
+### What do I do if my Advanced Search migrations are stuck?
+
+See [how to retry a halted
+migration](../integration/elasticsearch.md#retry-a-halted-migration).
+
 ## Upgrade paths
 
-Although you can generally upgrade through multiple GitLab versions in one go,
+You can generally upgrade through multiple GitLab versions in one go,
+although this is discouraged for [zero downtime upgrades](#upgrading-without-downtime) and
 sometimes this can cause issues.
 
 Find where your version sits in the upgrade path below, and upgrade GitLab
 accordingly, while also consulting the
 [version-specific upgrade instructions](#version-specific-upgrading-instructions):
 
-`8.11.x` -> `8.12.0` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> `11.11.8` -> `12.0.12` -> `12.1.17` -> `12.10.14` -> `13.0.14` -> `13.1.11` - > `13.x (latest)`
+`8.11.Z` -> `8.12.0` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> `11.11.8` -> `12.0.12` -> `12.1.17` -> `12.10.14` -> `13.0.14` -> `13.1.11` - > [latest `13.Y.Z`](https://about.gitlab.com/releases/categories/releases/)
 
 The following table, while not exhaustive, shows some examples of the supported
 upgrade paths.
@@ -174,13 +204,20 @@ Identify a [supported upgrade path](#upgrade-paths).
 
 More significant migrations may occur during major release upgrades. To ensure these are successful:
 
-1. Increment to the first minor version (`x.0.x`) during the major version jump.
+1. Increment to the first minor version (`X.0.Z`) during the major version jump.
 1. Proceed with upgrading to a newer release.
 
 It's also important to ensure that any background migrations have been fully completed
 before upgrading to a new major version. To see the current size of the `background_migration` queue,
 [Check for background migrations before upgrading](#checking-for-background-migrations-before-upgrading).
 
+If you have enabled the [Elasticsearch
+integration](../integration/elasticsearch.md), then you will also need to ensure
+all Advanced Search migrations are completed in the last minor version within
+your current version. Be sure to [check for pending Advanced Search
+migrations](#checking-for-pending-advanced-search-migrations) before proceeding
+with the major version upgrade.
+
 If your GitLab instance has any runners associated with it, it is very
 important to upgrade GitLab Runner to match the GitLab minor version that was
 upgraded to. This is to ensure [compatibility with GitLab versions](https://docs.gitlab.com/runner/#compatibility-with-gitlab-versions).
@@ -192,7 +229,8 @@ patch version of GitLab without having to take your GitLab instance offline.
 However, for this to work there are the following requirements:
 
 - You can only upgrade 1 minor release at a time. So from 9.1 to 9.2, not to
-   9.3.
+   9.3. If you skip releases, database modifications may be run in the wrong
+   sequence [and leave the database schema in a broken state](https://gitlab.com/gitlab-org/gitlab/-/issues/321542).
 - You have to use [post-deployment
    migrations](../development/post_deployment_migrations.md) (included in
    [zero downtime update steps below](#steps)).
@@ -235,7 +273,7 @@ which is the latest patch release of 9.4. When GitLab 9.5.0 is released this
 installation can be safely upgraded to 9.5.0 without requiring downtime if the
 requirements mentioned above are met. You can also skip 9.5.0 and upgrade to
 9.5.1 after it's released, but you **can not** upgrade straight to 9.6.0; you
-_have_ to first upgrade to a 9.5.x release.
+_have_ to first upgrade to a 9.5.Z release.
 
 **Example 2:** You are running a large GitLab installation using version 9.4.2,
 which is the latest patch release of 9.4. GitLab 9.5 includes some background
@@ -243,7 +281,7 @@ migrations, and 10.0 will require these to be completed (processing any
 remaining jobs for you). Skipping 9.5 is not possible without downtime, and due
 to the background migrations would require potentially hours of downtime
 depending on how long it takes for the background migrations to complete. To
-work around this you will have to upgrade to 9.5.x first, then wait at least a
+work around this you will have to upgrade to 9.5.Z first, then wait at least a
 week before upgrading to 10.0.
 
 **Example 3:** You use MySQL as the database for GitLab. Any upgrade to a new
@@ -361,9 +399,9 @@ with the older Rails version - which could cause non-GET requests to
 fail for [multi-node GitLab installations](https://docs.gitlab.com/omnibus/update/#multi-node--ha-deployment).
 
 So, if you are using multiple Rails servers and specifically upgrading from 13.0,
-all servers must first be upgraded to 13.1.X before upgrading to 13.2.0 or later:
+all servers must first be upgraded to 13.1.Z before upgrading to 13.2.0 or later:
 
-1. Ensure all GitLab web nodes are on GitLab 13.1.X.
+1. Ensure all GitLab web nodes are on GitLab 13.1.Z.
 1. Optionally, enable the `global_csrf_token` feature flag to enable new
    method of CSRF token generation:
 
@@ -383,33 +421,33 @@ any downgrades would result to all sessions being invalidated and users are logg
 
 ### 12.1.0
 
-If you are planning to upgrade from `12.0.x` to `12.10.x`, it is necessary to
-perform an intermediary upgrade to `12.1.x` before upgrading to `12.10.x` to
+If you are planning to upgrade from `12.0.Z` to `12.10.Z`, it is necessary to
+perform an intermediary upgrade to `12.1.Z` before upgrading to `12.10.Z` to
 avoid issues like [#215141](https://gitlab.com/gitlab-org/gitlab/-/issues/215141).
 
 ### 12.0.0
 
 In 12.0.0 we made various database related changes. These changes require that
-users first upgrade to the latest 11.11 patch release. After upgraded to 11.11.x,
-users can upgrade to 12.0.x. Failure to do so may result in database migrations
+users first upgrade to the latest 11.11 patch release. After upgraded to 11.11.Z,
+users can upgrade to 12.0.Z. Failure to do so may result in database migrations
 not being applied, which could lead to application errors.
 
-It is also required that you upgrade to 12.0.x before moving to a later version
-of 12.x.
+It is also required that you upgrade to 12.0.Z before moving to a later version
+of 12.Y.
 
 Example 1: you are currently using GitLab 11.11.8, which is the latest patch
-release for 11.11.x. You can upgrade as usual to 12.0.x.
+release for 11.11.Z. You can upgrade as usual to 12.0.Z.
 
-Example 2: you are currently using a version of GitLab 10.x. To upgrade, first
-upgrade to the last 10.x release (10.8.7) then the last 11.x release (11.11.8).
-After upgraded to 11.11.8 you can safely upgrade to 12.0.x.
+Example 2: you are currently using a version of GitLab 10.Y. To upgrade, first
+upgrade to the last 10.Y release (10.8.7) then the last 11.Y release (11.11.8).
+After upgraded to 11.11.8 you can safely upgrade to 12.0.Z.
 
 See our [documentation on upgrade paths](../policy/maintenance.md#upgrade-recommendations)
 for more information.
 
 ### Upgrades from versions earlier than 8.12
 
-- `8.11.x` and earlier: you might have to upgrade to `8.12.0` specifically before you can upgrade to `8.17.7`. This was [reported in an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/207259).
+- `8.11.Z` and earlier: you might have to upgrade to `8.12.0` specifically before you can upgrade to `8.17.7`. This was [reported in an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/207259).
 - [CI changes prior to version 8.0](https://docs.gitlab.com/omnibus/update/README.html#updating-gitlab-ci-from-prior-540-to-version-714-via-omnibus-gitlab)
   when it was merged into GitLab.
 
diff --git a/doc/update/upgrading_from_ce_to_ee.md b/doc/update/upgrading_from_ce_to_ee.md
index 36f0078a4f7..71fd4efb16d 100644
--- a/doc/update/upgrading_from_ce_to_ee.md
+++ b/doc/update/upgrading_from_ce_to_ee.md
@@ -63,7 +63,9 @@ sudo -u git -H git checkout EE_BRANCH
 ```shell
 cd /home/git/gitlab
 
-sudo -u git -H bundle install --deployment --without development test mysql aws kerberos
+sudo -u git -H bundle config set deployment 'true'
+sudo -u git -H bundle config set without 'development test mysql aws kerberos'
+sudo -u git -H bundle install
 
 # Optional: clean up old gems
 sudo -u git -H bundle clean
diff --git a/doc/update/upgrading_from_source.md b/doc/update/upgrading_from_source.md
index a76e9fb4504..42acb7552e4 100644
--- a/doc/update/upgrading_from_source.md
+++ b/doc/update/upgrading_from_source.md
@@ -127,49 +127,41 @@ Git v2.28 is recommended.
 To check you are running the minimum required Git version, see
 [Git versions](../install/requirements.md#git-versions).
 
-In Debian or Ubuntu:
-
-```shell
-# Make sure Git is version 2.29.0 or higher
-git --version
+From GitLab 13.6, we recommend you use the [Git version provided by
+Gitaly](https://gitlab.com/gitlab-org/gitaly/-/issues/2729)
+that:
 
-# Remove packaged Git
-sudo apt-get remove git-core
+- Is always at the version required by GitLab.
+- May contain custom patches required for proper operation.
 
+```shell
 # Install dependencies
-sudo apt-get install -y libcurl4-openssl-dev libexpat1-dev gettext libz-dev libssl-dev build-essential
-
-# Download and compile pcre2 from source
-curl --silent --show-error --location "https://ftp.pcre.org/pub/pcre/pcre2-10.33.tar.gz" --output pcre2.tar.gz
-tar -xzf pcre2.tar.gz
-cd pcre2-10.33
-chmod +x configure
-./configure --prefix=/usr --enable-jit
-make
-make install
-
-# Download and compile from source
-cd /tmp
-curl --remote-name --location --progress "https://www.kernel.org/pub/software/scm/git/git-2.29.0.tar.gz"
-echo 'fa08dc8424ef80c0f9bf307877f9e2e49f1a6049e873530d6747c2be770742ff  git-2.29.0.tar.gz' | shasum -a256 -c - && tar -xzf git-2.29.0.tar.gz
-cd git-2.29.0/
-./configure --with-libpcre
-make prefix=/usr/local all
+sudo apt-get install -y libcurl4-openssl-dev libexpat1-dev gettext libz-dev libssl-dev libpcre2-dev build-essential
 
-# Install into /usr/local/bin
-sudo make prefix=/usr/local install
+# Clone the Gitaly repository
+git clone https://gitlab.com/gitlab-org/gitaly.git -b <X-Y-stable> /tmp/gitaly
 
-# You should edit config/gitlab.yml, change the git -> bin_path to /usr/local/bin/git
+# Compile and install Git
+cd /tmp/gitaly
+sudo make git GIT_PREFIX=/usr/local
 ```
 
+Replace `<X-Y-stable>` with the stable branch that matches the GitLab version you want to
+install. For example, if you want to install GitLab 13.6, use the branch name `13-6-stable`.
+
+Remember to set `git -> bin_path` to `/usr/local/bin/git` in `config/gitlab.yml`.
+
 ### 7. Update PostgreSQL
 
 WARNING:
 From GitLab 13.0, you must use at least PostgreSQL 11.
 
-The latest version of GitLab might depend on a more recent PostgreSQL version than what you are currently running (see the [PostgreSQL requirements](../install/requirements.md#postgresql-requirements)).
+The latest version of GitLab might depend on a more recent PostgreSQL version
+than what you're currently running. You may also need to enable some
+extensions. For more information, see the
+[PostgreSQL requirements](../install/requirements.md#postgresql-requirements)
 
-In order to upgrade PostgreSQL, please refer to its [documentation](https://www.postgresql.org/docs/11/upgrading.html).
+To upgrade PostgreSQL, refer to its [documentation](https://www.postgresql.org/docs/11/upgrading.html).
 
 ### 8. Get latest code
 
@@ -177,8 +169,7 @@ In order to upgrade PostgreSQL, please refer to its [documentation](https://www.
 cd /home/git/gitlab
 
 sudo -u git -H git fetch --all --prune
-sudo -u git -H git checkout -- db/structure.sql # local changes will be restored automatically
-sudo -u git -H git checkout -- locale
+sudo -u git -H git checkout -- Gemfile.lock db/structure.sql locale
 ```
 
 For GitLab Community Edition:
@@ -199,55 +190,7 @@ cd /home/git/gitlab
 sudo -u git -H git checkout BRANCH-ee
 ```
 
-### 9. Update GitLab Shell
-
-```shell
-cd /home/git/gitlab-shell
-
-sudo -u git -H git fetch --all --tags --prune
-sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION)
-sudo -u git -H make build
-```
-
-### 10. Update GitLab Workhorse
-
-Install and compile GitLab Workhorse.
-
-```shell
-cd /home/git/gitlab
-
-sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse]" RAILS_ENV=production
-```
-
-### 11. Update Gitaly
-
-#### Compile Gitaly
-
-```shell
-cd /home/git/gitaly
-sudo -u git -H git fetch --all --tags --prune
-sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION)
-sudo -u git -H make
-```
-
-### 12. Update GitLab Pages
-
-#### Only needed if you use GitLab Pages
-
-Install and compile GitLab Pages. GitLab Pages uses
-[GNU Make](https://www.gnu.org/software/make/).
-If you are not using Linux you may have to run `gmake` instead of
-`make` below.
-
-```shell
-cd /home/git/gitlab-pages
-
-sudo -u git -H git fetch --all --tags --prune
-sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION)
-sudo -u git -H make
-```
-
-### 13. Update configuration files
+### 9. Update configuration files
 
 #### New configuration options for `gitlab.yml`
 
@@ -320,12 +263,17 @@ For Ubuntu 16.04.1 LTS:
 sudo systemctl daemon-reload
 ```
 
-### 14. Install libraries, migrations, etc
+### 10. Install libraries, migrations, etc
 
 ```shell
 cd /home/git/gitlab
 
-sudo -u git -H bundle install --deployment --without development test mysql aws kerberos
+# If you haven't done so during installation or a previous upgrade already
+sudo -u git -H bundle config set deployment 'true'
+sudo -u git -H bundle config set without 'development test mysql aws kerberos'
+
+# Update gems
+sudo -u git -H bundle install
 
 # Optional: clean up old gems
 sudo -u git -H bundle clean
@@ -334,7 +282,6 @@ sudo -u git -H bundle clean
 sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
 
 # Compile GetText PO files
-
 sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production
 
 # Update node dependencies and recompile assets
@@ -344,6 +291,54 @@ sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:c
 sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production
 ```
 
+### 11. Update GitLab Shell
+
+```shell
+cd /home/git/gitlab-shell
+
+sudo -u git -H git fetch --all --tags --prune
+sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION)
+sudo -u git -H make build
+```
+
+### 12. Update GitLab Workhorse
+
+Install and compile GitLab Workhorse.
+
+```shell
+cd /home/git/gitlab
+
+sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse]" RAILS_ENV=production
+```
+
+### 13. Update Gitaly
+
+#### Compile Gitaly
+
+```shell
+cd /home/git/gitaly
+sudo -u git -H git fetch --all --tags --prune
+sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION)
+sudo -u git -H make
+```
+
+### 14. Update GitLab Pages
+
+#### Only needed if you use GitLab Pages
+
+Install and compile GitLab Pages. GitLab Pages uses
+[GNU Make](https://www.gnu.org/software/make/).
+If you are not using Linux you may have to run `gmake` instead of
+`make` below.
+
+```shell
+cd /home/git/gitlab-pages
+
+sudo -u git -H git fetch --all --tags --prune
+sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION)
+sudo -u git -H make
+```
+
 ### 15. Start application
 
 ```shell