diff options
Diffstat (limited to 'doc/administration/reference_architectures/10k_users.md')
| -rw-r--r-- | doc/administration/reference_architectures/10k_users.md | 81 |
1 files changed, 35 insertions, 46 deletions
diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index d4d522ab1b8..d6a38e1b713 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -17,23 +17,23 @@ full list of reference architectures, see | Service | Nodes | Configuration | GCP | AWS | Azure | |--------------------------------------------|-------------|-------------------------|-----------------|-------------|----------| -| External load balancing node | 1 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Consul | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| PostgreSQL | 3 | 8 vCPU, 30 GB memory | n1-standard-8 | m5.2xlarge | D8s v3 | -| PgBouncer | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Internal load balancing node | 1 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Redis - Cache | 3 | 4 vCPU, 15 GB memory | n1-standard-4 | m5.xlarge | D4s v3 | -| Redis - Queues / Shared State | 3 | 4 vCPU, 15 GB memory | n1-standard-4 | m5.xlarge | D4s v3 | -| Redis Sentinel - Cache | 3 | 1 vCPU, 1.7 GB memory | g1-small | t3.small | B1MS | -| Redis Sentinel - Queues / Shared State | 3 | 1 vCPU, 1.7 GB memory | g1-small | t3.small | B1MS | -| Gitaly Cluster | 3 | 16 vCPU, 60 GB memory | n1-standard-16 | m5.4xlarge | D16s v3 | -| Praefect | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Praefect PostgreSQL | 1+* | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | c5.large | F2s v2 | -| Sidekiq | 4 | 4 vCPU, 15 GB memory | n1-standard-4 | m5.xlarge | D4s v3 | -| GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | n1-highcpu-32 | c5.9xlarge | F32s v2 | -| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | n1-highcpu-4 | c5.xlarge | F4s v2 | +| External load balancing node | 1 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Consul | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| PostgreSQL | 3 | 8 vCPU, 30 GB memory | n1-standard-8 | `m5.2xlarge` | D8s v3 | +| PgBouncer | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Internal load balancing node | 1 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Redis - Cache | 3 | 4 vCPU, 15 GB memory | n1-standard-4 | `m5.xlarge` | D4s v3 | +| Redis - Queues / Shared State | 3 | 4 vCPU, 15 GB memory | n1-standard-4 | `m5.xlarge` | D4s v3 | +| Redis Sentinel - Cache | 3 | 1 vCPU, 1.7 GB memory | g1-small | `t3.small` | B1MS | +| Redis Sentinel - Queues / Shared State | 3 | 1 vCPU, 1.7 GB memory | g1-small | `t3.small` | B1MS | +| Gitaly Cluster | 3 | 16 vCPU, 60 GB memory | n1-standard-16 | `m5.4xlarge` | D16s v3 | +| Praefect | 3 | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Praefect PostgreSQL | 1+* | 2 vCPU, 1.8 GB memory | n1-highcpu-2 | `c5.large` | F2s v2 | +| Sidekiq | 4 | 4 vCPU, 15 GB memory | n1-standard-4 | `m5.xlarge` | D4s v3 | +| GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | n1-highcpu-32 | `c5.9xlarge` | F32s v2 | +| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | n1-highcpu-4 | `c5.xlarge` | F4s v2 | | Object storage | n/a | n/a | n/a | n/a | n/a | -| NFS server | 1 | 4 vCPU, 3.6 GB memory | n1-highcpu-4 | c5.xlarge | F4s v2 | +| NFS server | 1 | 4 vCPU, 3.6 GB memory | n1-highcpu-4 | `c5.xlarge` | F4s v2 | ```plantuml @startuml 10k @@ -1508,7 +1508,7 @@ Praefect requires several secret tokens to secure communications across the Clus Gitaly Cluster nodes are configured in Praefect via a `virtual storage`. Each storage contains the details of each Gitaly node that makes up the cluster. Each storage is also given a name -and this name is used in several areas of the config. In this guide, the name of the storage will be +and this name is used in several areas of the configuration. In this guide, the name of the storage will be `default`. Also, this guide is geared towards new installs, if upgrading an existing environment to use Gitaly Cluster, you may need to use a different name. Refer to the [Praefect documentation](../gitaly/praefect.md#praefect) for more info. @@ -1756,7 +1756,7 @@ Note the following: necessary. - The Internal Load Balancer will also access to the certificates and need to be configured - to allow for TLS passthrough. + to allow for TLS pass-through. Refer to the load balancers documentation on how to configure this. To configure Praefect with TLS: @@ -1940,13 +1940,13 @@ To configure the Sidekiq nodes, on each one: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace @@ -2069,13 +2069,13 @@ On each node perform the following: 'google_project' => '<gcp-project-name>', 'google_json_key_location' => '<path-to-gcp-service-account-key>' } - gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>" - gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>" + gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-artifacts-bucket-name>" + gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-external-diffs-bucket-name>" + gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-lfs-bucket-name>" + gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-uploads-bucket-name>" + gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-packages-bucket-name>" + gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-dependency-proxy-bucket-name>" + gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-terraform-state-bucket-name>" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). @@ -2303,20 +2303,9 @@ on what features you intend to use: | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | Using separate buckets for each data type is the recommended approach for GitLab. - -A limitation of our configuration is that each use of object storage is separately configured. -[We have an issue for improving this](https://gitlab.com/gitlab-org/gitlab/-/issues/23345) -and easily using one bucket with separate folders is one improvement that this might bring. - -There is at least one specific issue with using the same bucket: -when GitLab is deployed with the Helm chart restore from backup -[will not properly function](https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer) -unless separate buckets are used. - -One risk of using a single bucket would be if your organization decided to -migrate GitLab to the Helm deployment in the future. GitLab would run, but the situation with -backups might not be realized until the organization had a critical requirement for the backups to -work. +This ensures there are no collisions across the various types of data GitLab stores. +There are plans to [enable the use of a single bucket](https://gitlab.com/gitlab-org/gitlab/-/issues/292958) +in the future. <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> |