Add latest changes from gitlab-org/gitlab@master

3 files changed, 62 insertions, 65 deletions

diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md
index ae6df691edc..ddb8f0aaa61 100644
--- a/doc/ci/review_apps/index.md
+++ b/doc/ci/review_apps/index.md
@@ -16,7 +16,7 @@ Review apps:
 
 - Provide an automatic live preview of changes made in a feature branch by spinning up a dynamic environment for your merge requests.
 - Allow designers and product managers to see your changes without needing to check out your branch and run your changes in a sandbox environment.
-- Are fully integrated with the [GitLab DevOps LifeCycle](../../index.md#the-entire-devops-lifecycle).
+- Are fully integrated with the [GitLab DevOps LifeCycle](https://about.gitlab.com/stages-devops-lifecycle/).
 - Allow you to deploy your changes wherever you want.
 
 ![review apps workflow](img/continuous-delivery-review-apps.svg)
diff --git a/doc/development/database/batched_background_migrations.md b/doc/development/database/batched_background_migrations.md
index abc96664481..3e54a78757a 100644
--- a/doc/development/database/batched_background_migrations.md
+++ b/doc/development/database/batched_background_migrations.md
@@ -56,44 +56,66 @@ in the directory `lib/gitlab/background_migration/`.
 
 ### Execution mechanism
 
-After being enqueued, batched background migrations are picked in sequential order by a Sidekiq worker.
-At this point, batched jobs are created - or retried and iterate over a range of records that are fetched based
-on the configuration defined inside the migration class. Each job instance receives a set of
-arguments based on this migration configuration. After each job execution, the optimization mechanism takes
-place to decide if the migration can be optimized.
+Batched background migrations are picked from the queue in the order they are enqueued. Multiple migrations are fetched
+and executed in parallel, as long they are in active state and do not target the same database table.
+The default number of migrations processed in parallel is 2, for GitLab.com this limit is configured to 4.
+Once migration is picked for execution, a job is created for the specific batch. After each job execution, migration's
+batch size may be increased or decreased, based on the performance of the last 20 jobs.
 
 ```plantuml
 @startuml
 hide empty description
 skinparam ConditionEndStyle hline
-start
-rectangle Queue {
-  :Migration Three;
-  note right: Waiting for Migration Two
-  :Migration Two;
-  note right: Waiting for Migration One
+left to right direction
+rectangle "Batched Background Migration Queue" as migrations {
+  rectangle "Migration N (active)" as migrationn
+  rectangle "Migration 1 (completed)" as migration1
+  rectangle "Migration 2 (active)" as migration2
+  rectangle "Migration 3 (on hold)" as migration3
+  rectangle "Migration 4 (active)" as migration4
+  migration1 -[hidden]> migration2
+  migration2 -[hidden]> migration3
+  migration3 -[hidden]> migration4
+  migration4 -[hidden]> migrationn
+}
+rectangle "Execution Workers" as workers {
+ rectangle "Execution Worker 1 (busy)" as worker1
+ rectangle "Execution Worker 2 (available)" as worker2
+ worker1 -[hidden]> worker2
 }
+migration2 --> [Scheduling Worker]
+migration4 --> [Scheduling Worker]
+[Scheduling Worker] --> worker2
+@enduml
+```
+
+Soon as a worker is available, the BBM is processed by the runner.
+
+```plantuml
+@startuml
+hide empty description
+start
 rectangle Runner {
-  :Migration One;
-  note right: Active migration
-  if (Still have records to migrate?) then (Yes)
-    if (Have a pending batched job?) then (Yes)
-      :Retry batched job;
-    else (No)
-      :Create a batched job;
-    endif
-    :Evaluate DB health;
-    note right: Table autovacuum, Patroni Apdex, Write-ahead logging
-    if (Evaluation signs to stop?) then (Yes)
-      :Put migration on hold;
+  :Migration;
+  if (Have reached batching bounds?) then (Yes)
+    if (Have jobs to retry?) then (Yes)
+      :Fetch the batched job;
     else (No)
-      :Optimize migration;
+      :Finish active migration;
+      stop
     endif
   else (No)
-    :Finish active migration;
+    :Create a batched job;
+  endif
+  :Execute batched job;
+  :Evaluate DB health;
+  note right: Checks for table autovacuum, Patroni Apdex, Write-ahead logging
+  if (Evaluation signs to stop?) then (Yes)
+    :Put migration on hold;
+  else (No)
+    :Optimize migration;
   endif
 }
-stop
 @enduml
 ```
 
@@ -161,13 +183,16 @@ the migration as `failed`) if any of the following is true:
 
 Because batched migrations are update heavy and there were few incidents in the past because of the heavy load from migrations while the database was underperforming, a throttling mechanism exists to mitigate them.
 
-These database indicators are checked to throttle a migration. On getting a stop signal, the migration is paused for a set time (10 minutes):
+These database indicators are checked to throttle a migration. On getting a
+stop signal, the migration is paused for a set time (10 minutes):
 
 - WAL queue pending archival crossing a threshold.
 - Active autovacuum on the tables on which the migration works on.
 - Patroni apdex SLI dropping below the SLO.
 
-It's an ongoing effort to add more indicators to further enhance the database health check framework, more details can be found in this [epic](https://gitlab.com/groups/gitlab-org/-/epics/7594).
+It's an ongoing effort to add more indicators to further enhance the
+database health check framework. For more details, see
+[epic 7594](https://gitlab.com/groups/gitlab-org/-/epics/7594).
 
 ### Isolation
 
@@ -348,21 +373,6 @@ class CopyColumnUsingBackgroundMigrationJob < BatchedMigrationJob
 end
 ```
 
-## Throttling batched migrations
-
-Because batched migrations are update heavy and there were few incidents in the past because of the heavy load from migrations while the database was underperforming, a throttling mechanism exists to mitigate them.
-
-These database indicators are checked to throttle a migration. On getting a
-stop signal, the migration is paused for a set time (10 minutes):
-
-- WAL queue pending archival crossing a threshold.
-- Active autovacuum on the tables on which the migration works on.
-- Patroni apdex SLI dropping below the SLO.
-
-It's an ongoing effort to add more indicators to further enhance the
-database health check framework. For more details, see
-[epic 7594](https://gitlab.com/groups/gitlab-org/-/epics/7594).
-
 ## Additional filters
 
 By default, when creating background jobs to perform the migration, batched background migrations
diff --git a/doc/index.md b/doc/index.md
index 654204048af..367f18ec159 100644
--- a/doc/index.md
+++ b/doc/index.md
@@ -19,16 +19,15 @@ description: 'Learn how to use and administer GitLab, the most scalable Git-base
 
 # GitLab Docs
 
-Welcome to the GitLab documentation!
+Explore the different areas of the documentation:
 
 |                         |                         |
 |:------------------------|:------------------------|
 | [**Use GitLab**](user/index.md)<br>Get started with GitLab features and functionality. | [**Administer GitLab**](administration/index.md)<br/>Administer a self-managed GitLab instance. |
-| [**New to Git and GitLab?**](tutorials/index.md)<br/>Start learning about Git and GitLab. | [**Contribute to GitLab development**](#contributing-to-gitlab)<br/>Create new GitLab functionality and documentation. |
+| [**New to Git and GitLab?**](tutorials/index.md)<br/>Start learning about Git and GitLab. | [**Contribute to GitLab development**](#contribute-to-gitlab)<br/>Create new GitLab functionality and documentation. |
 | [**Coming to GitLab from another platform?**](#coming-to-gitlab-from-another-platform)<br/>Learn how to move to GitLab. | [**Build an integration with GitLab**](#build-an-integration-with-gitlab)<br/>Integrate with Jira and other common applications. |
-| [**Choose a subscription**](subscriptions/index.md)<br/>Determine which subscription tier makes sense for you. | [**Install GitLab**](https://about.gitlab.com/install/)<br/>Install GitLab on different platforms. |
+| [**Choose a subscription**](subscriptions/index.md)<br/>Determine which subscription tier makes sense for you. | [**Install GitLab**](install/index.md)<br/>Install GitLab on different platforms. |
 | [**Reference architectures**](administration/reference_architectures/index.md)<br/>View recommended deployments at scale. | [**Upgrade GitLab**](update/index.md)<br/>Upgrade your GitLab self-managed instance to the latest version. |
-| [**GitLab releases**](https://about.gitlab.com/releases/)<br/>See what's new in GitLab. |  |
 
 ## Popular topics
 
@@ -48,15 +47,7 @@ Have a look at some of our most popular topics:
 | [SSL configuration for Linux package installations](https://docs.gitlab.com/omnibus/settings/ssl/index.html)      | SSL settings for self-managed instances installed using Linux packages. |
 | [GitLab.com settings](user/gitlab_com/index.md)                                            | Settings used for GitLab.com. |
 
-## The entire DevOps lifecycle
-
-GitLab is the first single application for software development, security,
-and operations that enables [Concurrent DevOps](https://about.gitlab.com/topics/concurrent-devops/).
-GitLab makes the software lifecycle faster and radically improves the speed of business.
-
-GitLab provides solutions for [each of the stages of the DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/).
-
-### User account
+## User account
 
 For more information about GitLab account management, see:
 
@@ -78,7 +69,7 @@ If you are coming to GitLab from another platform, the following information is
 
 ## Build an integration with GitLab
 
-There are many ways to integrate with GitLab, including:
+You can build integrations with GitLab:
 
 | Topic                                      | Description |
 |:-------------------------------------------|:------------|
@@ -86,15 +77,11 @@ There are many ways to integrate with GitLab, including:
 | [GitLab GraphQL API](api/graphql/index.md) | Integrate with GitLab using our GraphQL API. |
 | [Integrations](integration/index.md)      | Integrations with third-party products. |
 
-## Contributing to GitLab
-
-GitLab Community Edition is [open source](https://gitlab.com/gitlab-org/gitlab-foss/)
-and GitLab Enterprise Edition is [open-core](https://gitlab.com/gitlab-org/gitlab/).
+## Contribute to GitLab
 
 Learn how to contribute to GitLab with the following resources:
 
 | Topic                                                       | Description |
 |:------------------------------------------------------------|:------------|
-| [Development](development/index.md)                        | How to contribute to GitLab development. |
-| [Legal](legal/index.md)                                    | Contributor license agreements. |
-| [Writing documentation](development/documentation/index.md) | How to contribute to GitLab Docs. |
+| [Contribute to GitLab development](development/index.md).  | How to contribute to GitLab development. |
+| [Contribute to the documentation](development/documentation/index.md)             | How to contribute to GitLab documentation. |