diff options
Diffstat (limited to 'doc/administration/reference_architectures/50k_users.md')
| -rw-r--r-- | doc/administration/reference_architectures/50k_users.md | 70 |
1 files changed, 21 insertions, 49 deletions
diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index ca159d62f1f..87d1408b568 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -35,13 +35,13 @@ full list of reference architectures, see | GitLab Rails | 12 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | | Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | | Object storage<sup>4</sup> | - | - | - | - | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information. - [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. - - [Google AlloyDB](https://cloud.google.com/alloydb) and [Amazon RDS Multi-AZ DB clusters](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) have not been tested and are not recommended. Both solutions are specifically not expected to work with GitLab Geo. + - [Google AlloyDB](https://cloud.google.com/alloydb) and [Amazon RDS Multi-AZ DB cluster](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) have not been tested and are not recommended. Both solutions are specifically not expected to work with GitLab Geo. + - Note that [Amazon RDS Multi-AZ DB instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZSingleStandby.html) is a separate product and is supported. - [Amazon Aurora](https://aws.amazon.com/rds/aurora/) is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440). - Consul is primarily used for Omnibus PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information. @@ -189,7 +189,7 @@ CI pipelines alike. As such, large repositories come with notable cost and typically will require more resources to handle, significantly so in some cases. It's therefore **strongly** recommended then to review large repositories -to ensure they maintain good repo health and reduce their size wherever possible. +to ensure they maintain good health and reduce their size wherever possible. NOTE: If best practices aren't followed and large repositories are present on the environment, @@ -227,9 +227,6 @@ To set up GitLab and its components to accommodate up to 50,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) - to have shared disk storage service as an alternative to Gitaly or object - storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. @@ -296,8 +293,8 @@ could also be used, those load balancers have not been validated. ### Balancing algorithm -We recommend that a least-connection load balancing algorithm or equivalent -is used wherever possible to ensure equal spread of calls to the nodes and good performance. +You should use a least-connection load balancing algorithm or equivalent +wherever possible to ensure equal spread of calls to the nodes and good performance. We don't recommend the use of round-robin algorithms as they are known to not spread connections equally in practice. @@ -1288,7 +1285,7 @@ in the second step, do not supply the `EXTERNAL_URL` value. # PostgreSQL configuration postgresql['listen_address'] = '0.0.0.0' - postgresql['max_connections'] = 200 + postgresql['max_connections'] = 500 # Prevent database migrations from running on upgrade automatically gitlab_rails['auto_migrate'] = false @@ -1772,9 +1769,8 @@ To configure Praefect with TLS: Sidekiq requires connection to the [Redis](#configure-redis), [PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. - `10.6.0.101`: Sidekiq 1 - `10.6.0.102`: Sidekiq 2 @@ -1872,8 +1868,8 @@ Updates to example must be made at: ## Set number of Sidekiq queue processes to the same number as available CPUs sidekiq['queue_groups'] = ['*'] * 4 - ## Set number of Sidekiq threads per queue process to the recommend number of 10 - sidekiq['max_concurrency'] = 10 + ## Set number of Sidekiq threads per queue process to the recommend number of 20 + sidekiq['max_concurrency'] = 20 # Monitoring consul['enable'] = true @@ -1931,7 +1927,7 @@ Updates to example must be made at: NOTE: If you find that the environment's Sidekiq job processing is slow with long queues, more nodes can be added as required. You can also tune your Sidekiq nodes to -run [multiple Sidekiq processes](../operations/extra_sidekiq_processes.md). +run [multiple Sidekiq processes](../sidekiq/extra_sidekiq_processes.md). <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> @@ -1942,9 +1938,8 @@ run [multiple Sidekiq processes](../operations/extra_sidekiq_processes.md). ## Configure GitLab Rails This section describes how to configure the GitLab application (Rails) component. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. The following IPs will be used as an example: @@ -2092,7 +2087,6 @@ On each node perform the following: 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from the first Omnibus node you configured and add or replace the file of the same name on this server. If this is the first Omnibus node you are configuring then you can skip this step. - 1. To ensure database migrations are only run during reconfigure and not automatically on upgrade, run: ```shell @@ -2103,9 +2097,7 @@ On each node perform the following: [GitLab Rails post-configuration](#gitlab-rails-post-configuration) section. 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -1. [Enable incremental logging](#enable-incremental-logging), unless you are using [NFS](#configure-nfs-optional). - +1. [Enable incremental logging](#enable-incremental-logging). 1. Confirm the node can connect to Gitaly: ```shell @@ -2230,9 +2222,6 @@ To configure the Monitoring node: ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -It's recommended over [NFS](#configure-nfs-optional) and in general it's better -in larger setups as object storage is typically much more performant, reliable, -and scalable. GitLab has been tested on a number of object storage providers: @@ -2256,7 +2245,7 @@ NOTE: When using the [storage-specific form](../object_storage.md#storage-specific-configuration) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, -which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). +which was deprecated in 14.9, requires shared storage such as NFS. Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2275,22 +2264,6 @@ GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. -## Configure NFS (optional) - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) -after the release of GitLab 15.6. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) @@ -2340,12 +2313,10 @@ Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more The following tables and diagram detail the hybrid environment using the same formats as the normal environment above. -First are the components that run in Kubernetes. The recommendation at this time is to -use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EKS) and associated machine types, but the memory -and CPU requirements should translate to most other providers. We hope to update this in the -future with further specific cloud provider details. +First are the components that run in Kubernetes. These run across several node groups, although you can change +the overall makeup as desired as long as the minimum CPU and Memory requirements are observed. -| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | +| Service Node Group | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory | |---------------------|-------|-------------------------|-----------------|--------------|---------------------------------| | Webservice | 16 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `m5.8xlarge` | 510 vCPU, 472 GB memory | | Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory | @@ -2354,7 +2325,7 @@ future with further specific cloud provider details. - For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. - Nodes configuration is shown as it is forced to ensure pod vCPU / memory ratios and avoid scaling during **performance testing**. - - In production deployments, there is no need to assign pods to nodes. A minimum of three nodes in three different availability zones is strongly recommended to align with resilient cloud architecture practices. + - In production deployments, there is no need to assign pods to specific nodes. A minimum of three nodes per node group in three different availability zones is strongly recommended to align with resilient cloud architecture practices. Next are the backend components that run on static compute VMs via Omnibus (or External PaaS services where applicable): @@ -2376,7 +2347,8 @@ services where applicable): <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information. - [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. - - [Google AlloyDB](https://cloud.google.com/alloydb) and [Amazon RDS Multi-AZ DB clusters](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) have not been tested and are not recommended. Both solutions are specifically not expected to work with GitLab Geo. + - [Google AlloyDB](https://cloud.google.com/alloydb) and [Amazon RDS Multi-AZ DB cluster](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) have not been tested and are not recommended. Both solutions are specifically not expected to work with GitLab Geo. + - Note that [Amazon RDS Multi-AZ DB instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZSingleStandby.html) is a separate product and is supported. - [Amazon Aurora](https://aws.amazon.com/rds/aurora/) is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440). - Consul is primarily used for Omnibus PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information. |