Add latest changes from gitlab-org/gitlab@master

4 files changed, 77 insertions, 12 deletions

diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md
index 88d86f1213b..4d2ef141622 100644
--- a/doc/administration/pages/index.md
+++ b/doc/administration/pages/index.md
@@ -938,6 +938,67 @@ In installations from source:
 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source)
    for the changes to take effect.
 
+## ZIP storage
+
+In GitLab 14.0 the underlaying storage format of GitLab Pages is changing from
+files stored directly in disk to a single ZIP archive per project.
+
+These ZIP archives can be stored either locally on disk storage or on the [object storage](#using-object-storage) if it is configured.
+
+[Starting from GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/245308) ZIP archives are stored every time pages site is updated.
+
+### Migrate legacy storage to ZIP storage
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/59003) in GitLab 13.11.
+
+GitLab will [try to automatically migrate](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/54578) the old storage format to the new ZIP-based one when you upgrade to GitLab 13.11 or further.
+However, some projects may fail to be migrated for different reasons.
+To verify that all projects have been migrated successfully, you can manually run the migration:
+
+```shell
+gitlab-rake gitlab:pages:migrate_legacy_storage
+```
+
+It's safe to interrupt this task and run it multiple times.
+
+There are two most common problems this task can report:
+
+- `Missing public directory` error:
+
+  ```txt
+  E, [2021-04-09T13:11:52.534768 #911919] ERROR -- : project_id: 1 /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test failed to be migrated in 0.07 seconds: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test
+  ```
+
+  In this case, you should verify that these projects don't have pages deployed, and re-run the migration with an additional flag to mark those projects as not deployed with GitLab Pages:
+
+  ```shell
+  sudo PAGES_MIGRATION_MARK_PROJECTS_AS_NOT_DEPLOYED=true gitlab-rake gitlab:pages:migrate_legacy_storage
+  ```
+
+- File `is invalid` error:
+
+  ```txt
+  E, [2021-04-09T14:43:05.821767 #923322] ERROR -- : project_id: 1 /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test failed to be migrated: /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test/public/link is invalid, input_dir: /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test
+  ```
+
+  This error indicates invalid files on disk storage, most commonly symlinks leading outside of the `public` directory.
+  You can manually remove these files, or just ignore them during migration:
+
+  ```shell
+  sudo PAGES_MIGRATION_IGNORE_INVALID_ENTRIES=true gitlab-rake gitlab:pages:migrate_legacy_storage
+  ```
+
+### Rolling back ZIP migration
+
+If you find that migrated data is invalid, you can remove all migrated data by running:
+
+```shell
+sudo gitlab-rake gitlab:pages:clean_migrated_zip_storage
+```
+
+This will not remove any data from the legacy disk storage and the GitLab Pages daemon will automatically fallback
+to using that.
+
 ## Backup
 
 GitLab Pages are part of the [regular backup](../../raketasks/backup_restore.md), so there is no separate backup to configure.
diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md
index 1cd1c146b1e..97af1fe8d3c 100644
--- a/doc/administration/reference_architectures/10k_users.md
+++ b/doc/administration/reference_architectures/10k_users.md
@@ -22,10 +22,10 @@ full list of reference architectures, see
 | PostgreSQL*                                | 3           | 8 vCPU, 30 GB memory    | `n1-standard-8`  | `m5.2xlarge` | `D8s v3`  |
 | PgBouncer*                                 | 3           | 2 vCPU, 1.8 GB memory   | `n1-highcpu-2`   | `c5.large`   | `F2s v2`  |
 | Internal load balancing node               | 1           | 2 vCPU, 1.8 GB memory   | `n1-highcpu-2`   | `c5.large`   | `F2s v2`  |
-| Redis - Cache*                             | 3           | 4 vCPU, 15 GB memory    | `n1-standard-4`  | `m5.xlarge`  | `D4s v3`  |
-| Redis - Queues / Shared State*             | 3           | 4 vCPU, 15 GB memory    | `n1-standard-4`  | `m5.xlarge`  | `D4s v3`  |
-| Redis Sentinel - Cache*                    | 3           | 1 vCPU, 1.7 GB memory   | `g1-small`       | `t3.small`   | `B1MS`    |
-| Redis Sentinel - Queues / Shared State*    | 3           | 1 vCPU, 1.7 GB memory   | `g1-small`       | `t3.small`   | `B1MS`    |
+| Redis - Cache**                            | 3           | 4 vCPU, 15 GB memory    | `n1-standard-4`  | `m5.xlarge`  | `D4s v3`  |
+| Redis - Queues / Shared State**            | 3           | 4 vCPU, 15 GB memory    | `n1-standard-4`  | `m5.xlarge`  | `D4s v3`  |
+| Redis Sentinel - Cache**                   | 3           | 1 vCPU, 1.7 GB memory   | `g1-small`       | `t3.small`   | `B1MS`    |
+| Redis Sentinel - Queues / Shared State**   | 3           | 1 vCPU, 1.7 GB memory   | `g1-small`       | `t3.small`   | `B1MS`    |
 | Gitaly                                     | 3           | 16 vCPU, 60 GB memory   | `n1-standard-16` | `m5.4xlarge` | `D16s v3` |
 | Praefect                                   | 3           | 2 vCPU, 1.8 GB memory   | `n1-highcpu-2`   | `c5.large`   | `F2s v2`  |
 | Praefect PostgreSQL*                       | 1+          | 2 vCPU, 1.8 GB memory   | `n1-highcpu-2`   | `c5.large`   | `F2s v2`  |
@@ -37,7 +37,9 @@ full list of reference architectures, see
 
 NOTE:
 Components marked with * can be optionally run on reputable
-third party external PaaS solutions such as Google Cloud SQL or Memorystore.
+third party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work.
+Components marked with ** can be optionally run on reputable
+third party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work.
 
 ```plantuml
 @startuml 10k
@@ -2401,10 +2403,10 @@ services where applicable):
 | PostgreSQL*                                | 3     | 8 vCPU, 30 GB memory    | `n1-standard-8`  |
 | PgBouncer*                                 | 3     | 2 vCPU, 1.8 GB memory   | `n1-highcpu-2`   |
 | Internal load balancing node               | 1     | 2 vCPU, 1.8 GB memory   | `n1-highcpu-2`   |
-| Redis - Cache*                             | 3     | 4 vCPU, 15 GB memory    | `n1-standard-4`  |
-| Redis - Queues / Shared State*             | 3     | 4 vCPU, 15 GB memory    | `n1-standard-4`  |
-| Redis Sentinel - Cache*                    | 3     | 1 vCPU, 1.7 GB memory   | `g1-small`       |
-| Redis Sentinel - Queues / Shared State*    | 3     | 1 vCPU, 1.7 GB memory   | `g1-small`       |
+| Redis - Cache**                            | 3     | 4 vCPU, 15 GB memory    | `n1-standard-4`  |
+| Redis - Queues / Shared State**            | 3     | 4 vCPU, 15 GB memory    | `n1-standard-4`  |
+| Redis Sentinel - Cache**                   | 3     | 1 vCPU, 1.7 GB memory   | `g1-small`       |
+| Redis Sentinel - Queues / Shared State**   | 3     | 1 vCPU, 1.7 GB memory   | `g1-small`       |
 | Gitaly                                     | 3     | 16 vCPU, 60 GB memory   | `n1-standard-16` |
 | Praefect                                   | 3     | 2 vCPU, 1.8 GB memory   | `n1-highcpu-2`   |
 | Praefect PostgreSQL*                       | 1+    | 2 vCPU, 1.8 GB memory   | `n1-highcpu-2`   |
@@ -2412,7 +2414,9 @@ services where applicable):
 
 NOTE:
 Components marked with * can be optionally run on reputable
-third party external PaaS solutions such as Google Cloud SQL or Memorystore.
+third party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work.
+Components marked with ** can be optionally run on reputable
+third party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work.
 
 ```plantuml
 @startuml 10k
diff --git a/doc/api/README.md b/doc/api/README.md
index b310ca0fd7d..1a914fb1dbe 100644
--- a/doc/api/README.md
+++ b/doc/api/README.md
@@ -760,7 +760,7 @@ The correct encoding for the query parameter would be:
 ## Clients
 
 There are many unofficial GitLab API Clients for most of the popular programming
-languages. For a complete list, visit the [GitLab website](https://about.gitlab.com/partners/#api-clients).
+languages. For a complete list, visit the [GitLab website](https://about.gitlab.com/partners/technology-partners/#api-clients).
 
 ## Rate limits
 
diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md
index 1e536b0cbec..6e38534b044 100644
--- a/doc/user/gitlab_com/index.md
+++ b/doc/user/gitlab_com/index.md
@@ -113,7 +113,7 @@ or over the repository size limit, you can [reduce your repository size with Git
 
 | Setting                       | GitLab.com  | Default       |
 | -----------                   | ----------- | ------------- |
-| [Repository size including LFS](../admin_area/settings/account_and_limit_settings.md) | 10 GB       | Unlimited     |
+| [Repository size including LFS](../admin_area/settings/account_and_limit_settings.md#repository-size-limit) | 10 GB       | Unlimited     |
 | Maximum import size           | 5 GB        | Unlimited ([Modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50MB to unlimited in GitLab 13.8.    |
 
 NOTE: