Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2022-09-20 02:18:09 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2022-09-20 02:18:09 +0300
commit6ed4ec3e0b1340f96b7c043ef51d1b33bbe85fde (patch)
treedc4d20fe6064752c0bd323187252c77e0a89144b /doc/administration
parent9868dae7fc0655bd7ce4a6887d4e6d487690eeed (diff)
Add latest changes from gitlab-org/gitlab@15-4-stable-eev15.4.0-rc42
Diffstat (limited to 'doc/administration')
-rw-r--r--doc/administration/application_settings_cache.md8
-rw-r--r--doc/administration/audit_event_streaming.md42
-rw-r--r--doc/administration/audit_events.md8
-rw-r--r--doc/administration/audit_reports.md2
-rw-r--r--doc/administration/auditor_users.md2
-rw-r--r--doc/administration/auth/atlassian.md5
-rw-r--r--doc/administration/auth/index.md28
-rw-r--r--doc/administration/auth/jwt.md5
-rw-r--r--doc/administration/auth/ldap/index.md2
-rw-r--r--doc/administration/auth/ldap/ldap-troubleshooting.md131
-rw-r--r--doc/administration/auth/ldap/ldap_synchronization.md10
-rw-r--r--doc/administration/auth/oidc.md2
-rw-r--r--doc/administration/compliance.md2
-rw-r--r--doc/administration/environment_variables.md1
-rw-r--r--doc/administration/external_pipeline_validation.md12
-rw-r--r--doc/administration/feature_flags.md5
-rw-r--r--doc/administration/geo/disaster_recovery/background_verification.md8
-rw-r--r--doc/administration/geo/disaster_recovery/bring_primary_back.md2
-rw-r--r--doc/administration/geo/disaster_recovery/index.md4
-rw-r--r--doc/administration/geo/disaster_recovery/planned_failover.md10
-rw-r--r--doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md10
-rw-r--r--doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md8
-rw-r--r--doc/administration/geo/index.md6
-rw-r--r--doc/administration/geo/replication/configuration.md14
-rw-r--r--doc/administration/geo/replication/container_registry.md8
-rw-r--r--doc/administration/geo/replication/disable_geo.md2
-rw-r--r--doc/administration/geo/replication/object_storage.md2
-rw-r--r--doc/administration/geo/replication/remove_geo_site.md2
-rw-r--r--doc/administration/geo/replication/troubleshooting.md190
-rw-r--r--doc/administration/geo/replication/tuning.md2
-rw-r--r--doc/administration/geo/replication/version_specific_upgrades.md15
-rw-r--r--doc/administration/geo/secondary_proxy/index.md14
-rw-r--r--doc/administration/gitaly/configure_gitaly.md7
-rw-r--r--doc/administration/gitaly/index.md23
-rw-r--r--doc/administration/gitaly/praefect.md17
-rw-r--r--doc/administration/gitaly/recovery.md38
-rw-r--r--doc/administration/gitaly/troubleshooting.md2
-rw-r--r--doc/administration/housekeeping.md2
-rw-r--r--doc/administration/inactive_project_deletion.md14
-rw-r--r--doc/administration/index.md2
-rw-r--r--doc/administration/instance_limits.md24
-rw-r--r--doc/administration/integration/kroki.md2
-rw-r--r--doc/administration/integration/mailgun.md2
-rw-r--r--doc/administration/integration/plantuml.md2
-rw-r--r--doc/administration/integration/terminal.md6
-rw-r--r--doc/administration/job_artifacts.md15
-rw-r--r--doc/administration/job_logs.md4
-rw-r--r--doc/administration/load_balancer.md30
-rw-r--r--doc/administration/maintenance_mode/index.md6
-rw-r--r--doc/administration/monitoring/gitlab_self_monitoring_project/index.md6
-rw-r--r--doc/administration/monitoring/performance/gitlab_configuration.md2
-rw-r--r--doc/administration/monitoring/performance/grafana_configuration.md6
-rw-r--r--doc/administration/monitoring/performance/performance_bar.md2
-rw-r--r--doc/administration/monitoring/prometheus/gitlab_metrics.md6
-rw-r--r--doc/administration/monitoring/prometheus/index.md5
-rw-r--r--doc/administration/nfs.md30
-rw-r--r--doc/administration/object_storage.md10
-rw-r--r--doc/administration/operations/fast_ssh_key_lookup.md4
-rw-r--r--doc/administration/operations/index.md2
-rw-r--r--doc/administration/operations/puma.md6
-rw-r--r--doc/administration/operations/rails_console.md53
-rw-r--r--doc/administration/operations/ssh_certificates.md2
-rw-r--r--doc/administration/package_information/supported_os.md1
-rw-r--r--doc/administration/packages/container_registry.md12
-rw-r--r--doc/administration/pages/index.md52
-rw-r--r--doc/administration/pages/source.md7
-rw-r--r--doc/administration/polling.md2
-rw-r--r--doc/administration/raketasks/maintenance.md18
-rw-r--r--doc/administration/raketasks/project_import_export.md2
-rw-r--r--doc/administration/raketasks/storage.md2
-rw-r--r--doc/administration/raketasks/uploads/migrate.md3
-rw-r--r--doc/administration/redis/replication_and_failover.md2
-rw-r--r--doc/administration/reference_architectures/10k_users.md189
-rw-r--r--doc/administration/reference_architectures/1k_users.md10
-rw-r--r--doc/administration/reference_architectures/25k_users.md198
-rw-r--r--doc/administration/reference_architectures/2k_users.md173
-rw-r--r--doc/administration/reference_architectures/3k_users.md186
-rw-r--r--doc/administration/reference_architectures/50k_users.md193
-rw-r--r--doc/administration/reference_architectures/5k_users.md179
-rw-r--r--doc/administration/repository_checks.md16
-rw-r--r--doc/administration/repository_storage_paths.md2
-rw-r--r--doc/administration/repository_storage_types.md2
-rw-r--r--doc/administration/restart_gitlab.md2
-rw-r--r--doc/administration/server_hooks.md37
-rw-r--r--doc/administration/sidekiq/extra_sidekiq_processes.md2
-rw-r--r--doc/administration/static_objects_external_storage.md2
-rw-r--r--doc/administration/system_hooks.md2
-rw-r--r--doc/administration/terraform_state.md4
-rw-r--r--doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md166
-rw-r--r--doc/administration/troubleshooting/linux_cheat_sheet.md2
-rw-r--r--doc/administration/troubleshooting/ssl.md12
-rw-r--r--doc/administration/troubleshooting/tracing_correlation_id.md2
-rw-r--r--doc/administration/user_settings.md13
-rw-r--r--doc/administration/whats-new.md46
94 files changed, 1358 insertions, 1073 deletions
diff --git a/doc/administration/application_settings_cache.md b/doc/administration/application_settings_cache.md
index 30fd9ab85a8..88e39a50b6c 100644
--- a/doc/administration/application_settings_cache.md
+++ b/doc/administration/application_settings_cache.md
@@ -18,7 +18,9 @@ extra load for Redis and PostgreSQL.
To change the expiry value:
-**For Omnibus installations**
+::Tabs
+
+:::TabTitle Omnibus package
1. Edit `/etc/gitlab/gitlab.rb`:
@@ -34,7 +36,7 @@ To change the expiry value:
gitlab-ctl restart
```
-**For installations from source**
+:::TabTitle Source
1. Edit `config/gitlab.yml`:
@@ -45,3 +47,5 @@ To change the expiry value:
1. Save the file, and then [restart](restart_gitlab.md#installations-from-source)
GitLab for the changes to take effect.
+
+::EndTabs
diff --git a/doc/administration/audit_event_streaming.md b/doc/administration/audit_event_streaming.md
index 3bb0ce41861..9ec7b81bfd0 100644
--- a/doc/administration/audit_event_streaming.md
+++ b/doc/administration/audit_event_streaming.md
@@ -1,5 +1,5 @@
---
-stage: Manage
+stage: Govern
group: Compliance
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
@@ -16,7 +16,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
> - Custom HTTP headers API [made generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/366524) in GitLab 15.3. [Feature flag `streaming_audit_event_headers`](https://gitlab.com/gitlab-org/gitlab/-/issues/362941) removed.
> - Custom HTTP headers UI [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361630) in GitLab 15.2 [with a flag](feature_flags.md) named `custom_headers_streaming_audit_events_ui`. Disabled by default.
> - Custom HTTP headers UI [made generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/365259) in GitLab 15.3. [Feature flag `custom_headers_streaming_audit_events_ui`](https://gitlab.com/gitlab-org/gitlab/-/issues/365259) removed.
-> - [Improved user experience](https://gitlab.com/gitlab-org/gitlab/-/issues/367963) in GitLab 15.4.
+> - [Improved user experience](https://gitlab.com/gitlab-org/gitlab/-/issues/367963) in GitLab 15.3.
+> - User-specified verification token API support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360813) in GitLab 15.4.
Users can set a streaming destination for a top-level group to receive all audit events about the group, its subgroups, and
projects as structured JSON.
@@ -38,7 +39,7 @@ Streaming destinations receive **all** audit event data, which could include sen
Users with the Owner role for a group can add streaming destinations for it:
-1. On the top bar, select **Menu > Groups** and find your group.
+1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Security & Compliance > Audit events**.
1. On the main area, select **Streams** tab.
1. Select **Add streaming destination** to show the section for adding destinations.
@@ -71,6 +72,25 @@ mutation {
}
```
+Group owners can also optionally specify their own verification token (instead of the default GitLab-generated one) using the GraphQL `auditEventsStreamingHeadersCreate`
+mutation. Verification token length must be within 16 to 24 characters and trailing whitespace are not trimmed. GitLab recommends setting a cryptographically random and unique value. For example:
+
+```graphql
+mutation {
+ externalAuditEventDestinationCreate(input: { destinationUrl: "https://mydomain.io/endpoint/ingest", groupPath: "my-group", verificationToken: "unique-random-verification-token-here" } ) {
+ errors
+ externalAuditEventDestination {
+ id
+ destinationUrl
+ verificationToken
+ group {
+ name
+ }
+ }
+ }
+}
+```
+
Event streaming is enabled if:
- The returned `errors` object is empty.
@@ -97,7 +117,7 @@ Users with the Owner role for a group can list streaming destinations.
To list the streaming destinations:
-1. On the top bar, select **Menu > Groups** and find your group.
+1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Security & Compliance > Audit events**.
1. On the main area, select **Streams** tab.
1. To the right of the item, select **Edit** (**{pencil}**) to see all the custom HTTP headers.
@@ -141,7 +161,7 @@ Users with the Owner role for a group can update streaming destinations.
To update a streaming destinations custom HTTP headers:
-1. On the top bar, select **Menu > Groups** and find your group.
+1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Security & Compliance > Audit events**.
1. On the main area, select **Streams** tab.
1. To the right of the item, select **Edit** (**{pencil}**).
@@ -195,14 +215,14 @@ When the last destination is successfully deleted, streaming is disabled for the
To delete a streaming destination:
-1. On the top bar, select **Menu > Groups** and find your group.
+1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Security & Compliance > Audit events**.
1. On the main area, select the **Streams** tab.
1. To the right of the item, select **Delete** (**{remove}**).
To delete only the custom HTTP headers for a streaming destination:
-1. On the top bar, select **Menu > Groups** and find your group.
+1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Security & Compliance > Audit events**.
1. On the main area, select the **Streams** tab.
1. To the right of the item, **Edit** (**{pencil}**).
@@ -248,9 +268,9 @@ The header is deleted if the returned `errors` object is empty.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345424) in GitLab 14.8.
Each streaming destination has a unique verification token (`verificationToken`) that can be used to verify the authenticity of the event. This
-token is generated when the event destination is created and cannot be changed.
+token is either specified by the Owner or generated automatically when the event destination is created and cannot be changed.
-Each streamed event contains a random alphanumeric identifier for the `X-Gitlab-Event-Streaming-Token` HTTP header that can be verified against
+Each streamed event contains the verification token in the `X-Gitlab-Event-Streaming-Token` HTTP header that can be verified against
the destination's value when [listing streaming destinations](#list-streaming-destinations).
### Use the GitLab UI
@@ -259,7 +279,7 @@ the destination's value when [listing streaming destinations](#list-streaming-de
Users with the Owner role for a group can list streaming destinations and see the verification tokens:
-1. On the top bar, select **Menu > Groups** and find your group.
+1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Security & Compliance > Audit events**.
1. On the main area, select the **Streams**.
1. View the verification token on the right side of each item.
@@ -297,7 +317,7 @@ FLAG:
On self-managed GitLab, by default this feature is available. To hide the
feature, ask an administrator to [disable the feature flag](feature_flags.md) named `audit_event_streaming_git_operations`.
-Streaming audit events can be sent when signed-in users push or pull a project's remote Git repositories:
+Streaming audit events can be sent when signed-in users push, pull, or clone a project's remote Git repositories:
- [Using SSH](../user/ssh.md).
- Using HTTP or HTTPS.
diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md
index 92504c226fb..b6c267bfd0c 100644
--- a/doc/administration/audit_events.md
+++ b/doc/administration/audit_events.md
@@ -1,5 +1,5 @@
---
-stage: Manage
+stage: Govern
group: Compliance
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
@@ -152,7 +152,7 @@ From there, you can see the following actions:
- Added, removed, or updated protected branches
- Release was added to a project
- Release was updated
-- Release was deleted ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/94793/) in GitLab 13.5)
+- Release was deleted ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/94793/) in GitLab 15.3)
- Release milestone associations changed
- Permission to approve merge requests by committers was updated ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7531) in GitLab 12.9)
- Permission to approve merge requests by committers was updated.
@@ -209,7 +209,7 @@ Instance events do not include group or project audit events.
To view the server-wide audit events:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Audit Events**.
The following user actions are recorded:
@@ -309,7 +309,7 @@ audit events.
To export the audit events to CSV:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Audit Events**.
1. Select the available search [filters](#search).
1. Select **Export as CSV**.
diff --git a/doc/administration/audit_reports.md b/doc/administration/audit_reports.md
index e33b5153c5b..e363e7862ea 100644
--- a/doc/administration/audit_reports.md
+++ b/doc/administration/audit_reports.md
@@ -1,5 +1,5 @@
---
-stage: Manage
+stage: Govern
group: Compliance
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
description: 'Learn how to create evidence artifacts typically requested by a 3rd party auditor.'
diff --git a/doc/administration/auditor_users.md b/doc/administration/auditor_users.md
index 561aa5d7b2e..bd48cfdbc4f 100644
--- a/doc/administration/auditor_users.md
+++ b/doc/administration/auditor_users.md
@@ -31,7 +31,7 @@ To create a new user account with auditor access (or change an existing user):
To create a user account with auditor access:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Users**.
1. Create a new user or edit an existing one. Set **Access Level** to **Auditor**.
1. If you created a user, select **Create user**. For an existing user, select **Save changes**.
diff --git a/doc/administration/auth/atlassian.md b/doc/administration/auth/atlassian.md
index 6be53922a5a..1d20d87bdf4 100644
--- a/doc/administration/auth/atlassian.md
+++ b/doc/administration/auth/atlassian.md
@@ -71,7 +71,10 @@ To enable the Atlassian OmniAuth provider for passwordless authentication you mu
1. Change `YOUR_CLIENT_ID` and `YOUR_CLIENT_SECRET` to the Client credentials you received in [application registration](#atlassian-application-registration) steps.
1. Save the configuration file.
-1. [Reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect if you installed GitLab via Omnibus or from source respectively.
+
+1. For the changes to take effect:
+ - If you installed via Omnibus, [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
+ - If you installed from source, [restart GitLab](../restart_gitlab.md#installations-from-source).
On the sign-in page there should now be an Atlassian icon below the regular sign in form. Select the icon to begin the authentication process.
diff --git a/doc/administration/auth/index.md b/doc/administration/auth/index.md
index a412875501e..2644e3a1f50 100644
--- a/doc/administration/auth/index.md
+++ b/doc/administration/auth/index.md
@@ -34,31 +34,3 @@ For more information, see the links shown on this page for each external provide
| **User Removal** | SCIM (remove user from top-level group) | LDAP (remove user from groups and block from the instance) |
1. Using Just-In-Time (JIT) provisioning, user accounts are created when the user first signs in.
-
-## Change apps or configuration
-
-When GitLab doesn't support having multiple providers (such as OAuth), GitLab configuration and user identification must be
-updated at the same time if the provider or app is changed.
-
-These instructions apply to all methods of authentication where GitLab stores an `extern_uid` and it is the only data used
-for user authentication.
-
-When changing apps within a provider, if the user `extern_uid` does not change, only the GitLab configuration must be
-updated.
-
-To swap configurations:
-
-1. Change provider configuration in your `gitlab.rb` file.
-1. Update `extern_uid` for all users that have an identity in GitLab for the previous provider.
-
-To find the `extern_uid`, look at an existing user's current `extern_uid` for an ID that matches the appropriate field in
-your current provider for the same user.
-
-There are two methods to update the `extern_uid`:
-
-- Using the [Users API](../../api/users.md#user-modification). Pass the provider name and the new `extern_uid`.
-- Using the [Rails console](../operations/rails_console.md):
-
- ```ruby
- Identity.where(extern_uid: 'old-id').update!(extern_uid: 'new-id')`
- ```
diff --git a/doc/administration/auth/jwt.md b/doc/administration/auth/jwt.md
index 99cba3f220d..71ab084065a 100644
--- a/doc/administration/auth/jwt.md
+++ b/doc/administration/auth/jwt.md
@@ -70,8 +70,9 @@ JWT provides you with a secret key for you to use.
1. Change `YOUR_APP_SECRET` to the client secret and set `auth_url` to your redirect URL.
1. Save the configuration file.
-1. [Reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect if you
- installed GitLab via Omnibus or from source respectively.
+1. For the changes to take effect:
+ - If you installed via Omnibus, [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
+ - If you installed from source, [restart GitLab](../restart_gitlab.md#installations-from-source).
On the sign in page there should now be a JWT icon below the regular sign in form.
Select the icon to begin the authentication process. JWT asks the user to
diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md
index 2f0a0db9d6f..19f656d2f14 100644
--- a/doc/administration/auth/ldap/index.md
+++ b/doc/administration/auth/ldap/index.md
@@ -344,7 +344,7 @@ The `user_filter` DN can contain special characters. For example:
OU=GitLab\2C Inc,DC=gitlab,DC=com
```
-- Escape open and close brackets with `\28` and `\29`, respectively. For example:
+- Escape open brackets with `\28` and close brackets with `\29`. For example:
```plaintext
OU=Gitlab \28Inc\29,DC=gitlab,DC=com
diff --git a/doc/administration/auth/ldap/ldap-troubleshooting.md b/doc/administration/auth/ldap/ldap-troubleshooting.md
index 64ef27cbf51..a68e6ae2649 100644
--- a/doc/administration/auth/ldap/ldap-troubleshooting.md
+++ b/doc/administration/auth/ldap/ldap-troubleshooting.md
@@ -5,7 +5,9 @@ group: Authentication and Authorization
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
-# LDAP Troubleshooting for Administrators **(FREE SELF)**
+# Troubleshooting LDAP **(FREE SELF)**
+
+If you are an administrator, use the following information to troubleshoot LDAP.
## Common Problems & Workflows
@@ -165,7 +167,7 @@ may see the following message: `Access denied for your LDAP account`.
We have a workaround, based on toggling the access level of affected users:
-1. As an administrator, on the top bar, select **Menu > Admin**.
+1. As an administrator, on the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Users**.
1. Select the name of the affected user.
1. In the user's administrative page, press **Edit** on the top right of the page.
@@ -210,7 +212,7 @@ This shows you which user has this email address. One of two steps must be taken
remove this email as a secondary email and make it a primary one so GitLab
associates this profile to the LDAP identity.
-The user can do either of these steps
+The user can do either of these steps
[in their profile](../../../user/profile/index.md#access-your-user-profile) or an administrator can do it.
#### Projects limit errors
@@ -223,7 +225,7 @@ field contains no data:
To resolve this:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, go to **Settings > General**.
1. Expand both of the following:
- **Account and limit**.
@@ -366,7 +368,7 @@ things to debug the situation.
- Ensure the correct [LDAP group link is added to the GitLab group](ldap_synchronization.md#add-group-links).
- Check that the user has an LDAP identity:
1. Sign in to GitLab as an administrator user.
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Users**.
1. Search for the user.
1. Open the user by selecting their name. Do not select **Edit**.
@@ -411,6 +413,63 @@ If all the above are true and the users are still not getting access,
[look through the output](#example-console-output-after-a-group-sync) to see what happens when
GitLab syncs the `admin_group`.
+#### Sync now button stuck in the UI
+
+The **Sync now** button on the **Group > Members** page of a group can become stuck. The button becomes stuck after it is pressed and the page is reloaded. The button then
+cannot be selected again.
+
+The **Sync now** button can become stuck for many reasons and requires debugging for specific cases. The following are two possible causes and possible solutions to the problem.
+
+##### Invalid memberships
+
+The **Sync now** button becomes stuck if some of the group's members or requesting members are invalid. You can track progress on improving the visibility of this problem in
+a [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/348226). You can use a [Rails console](#rails-console) to confirm if this problem is causing the **Sync now**
+button to be stuck:
+
+```ruby
+# Find the group in question
+group = Group.find_by(name: 'my_gitlab_group')
+
+# Look for errors on the Group itself
+group.valid?
+group.errors.map(&:full_messages)
+
+# Look for errors among the group's members and requesters
+group.requesters.map(&:valid?)
+group.requesters.map(&:errors).map(&:full_messages)
+group.members.map(&:valid?)
+group.members.map(&:errors).map(&:full_messages)
+```
+
+A displayed error can identify the problem and point to a solution. For example, the Support Team has seen the following error:
+
+```ruby
+irb(main):018:0> group.members.map(&:errors).map(&:full_messages)
+=> [["The member's email address is not allowed for this group. Go to the group’s &#39;Settings &gt; General&#39; page, and check &#39;Restrict membership by email domain&#39;."]]
+```
+
+This error showed that an Administrator chose to [restrict group membership by email domain](../../../user/group/access_and_permissions.md#restrict-group-access-by-domain),
+but there was a typo in the domain. After the domain setting was fixed, the **Sync now** button functioned again.
+
+##### Missing LDAP configuration on Sidekiq nodes
+
+The **Sync now** button becomes stuck when GitLab is scaled over multiple nodes and the LDAP configuration is missing from
+[the `/etc/gitlab/gitlab.rb` on the nodes running Sidekiq](../../sidekiq/index.md#configure-ldap-and-user-or-group-synchronization).
+In this case, the Sidekiq jobs seem to disappear.
+
+LDAP is required on the Sidekiq nodes because LDAP has multiple jobs that are
+run asynchronously that require a local LDAP configuration:
+
+- [User sync](ldap_synchronization.md#user-sync).
+- [Group sync](ldap_synchronization.md#group-sync).
+
+You can test whether missing LDAP configuration is the problem by running [the Rake task to check LDAP](#ldap-check)
+on each node that is running Sidekiq. If LDAP is set up correctly on this node, it connects to the LDAP server and returns users.
+
+To solve this issue, [configure LDAP](../../sidekiq/index.md#configure-ldap-and-user-or-group-synchronization) on the Sidekiq nodes.
+When configured, run [the Rake task to check LDAP](#ldap-check) to confirm
+that the GitLab node can connect to LDAP.
+
#### Sync all groups
NOTE:
@@ -430,7 +489,7 @@ Next, [learn how to read the output](#example-console-output-after-a-group-sync)
##### Example console output after a group sync
-Like the output from the user sync, the output from the
+Like the output from the user sync, the output from the
[manual group sync](#sync-all-groups) is also very verbose. However, it contains lots
of helpful information.
@@ -600,6 +659,66 @@ end
You can then [run a UserSync](#sync-all-users) **(PREMIUM SELF)** to sync the latest DN
for each of these users.
+## Could not authenticate you from ldapmain because "Unknown provider"
+
+You can receive the following error when authenticating with an LDAP server:
+
+```plaintext
+Could not authenticate you from Ldapmain because "Unknown provider (ldapsecondary). available providers: ["ldapmain"]".
+```
+
+This error is caused when using an account that previously authenticated with an LDAP server that is renamed or removed from your GitLab configuration. For example:
+
+- Initially, `main` and `secondary` are set in `ldap_servers` in GitLab configuration.
+- The `secondary` setting is removed or renamed to `main`.
+- A user attempting to sign in has an `identify` record for `secondary`, but that is no longer configured.
+
+Use the [Rails console](../../operations/rails_console.md) to list affected users and check what LDAP servers they have identities for:
+
+```ruby
+ldap_identities = Identity.where(provider: "ldapsecondary")
+ldap_identities.each do |identity|
+ u=User.find_by_id(identity.user_id)
+ ui=Identity.where(user_id: identity.user_id)
+ puts "user: #{u.username}\n #{u.email}\n last activity: #{u.last_activity_on}\n #{identity.provider} ID: #{identity.id} external: #{identity.extern_uid}"
+ puts " all identities:"
+ ui.each do |alli|
+ puts " - #{alli.provider} ID: #{alli.id} external: #{alli.extern_uid}"
+ end
+end;nil
+```
+
+You can solve this error in two ways.
+
+### Rename references to the LDAP server
+
+This solution is suitable when the LDAP servers are replicas of each other, and the affected users should be able to sign in using a configured LDAP server. For example, if a
+load balancer is now used to manage LDAP high availability and a separate secondary sign-in option is no longer needed.
+
+NOTE:
+If the LDAP servers aren't replicas of each other, this solution stops affected users from being able to sign in.
+
+To [rename references to the LDAP server](../../raketasks/ldap.md#other-options) that is no longer configured, run:
+
+```shell
+sudo gitlab-rake gitlab:ldap:rename_provider[ldapsecondary,ldapmain]
+```
+
+### Remove the `identity` records that relate to the removed LDAP server
+
+With this solution, affected users can sign in with the configured LDAP servers and a new `identity` record is created by GitLab. In a
+[Rails console](../../operations/rails_console.md), delete the `ldapsecondary` identities:
+
+```ruby
+ldap_identities = Identity.where(provider: "ldapsecondary")
+ldap_identities.each do |identity|
+ puts "Destroying identity: #{identity.id} #{identity.provider}: #{identity.extern_uid}"
+ identity.destroy!
+rescue => e
+ puts 'Error generated when destroying identity:\n ' + e.to_s
+end; nil
+```
+
## Expired license causes errors with multiple LDAP servers
Using [multiple LDAP servers](index.md#use-multiple-ldap-servers) requires a valid license. An expired license can
diff --git a/doc/administration/auth/ldap/ldap_synchronization.md b/doc/administration/auth/ldap/ldap_synchronization.md
index 62706a9e3b9..37a27fc058e 100644
--- a/doc/administration/auth/ldap/ldap_synchronization.md
+++ b/doc/administration/auth/ldap/ldap_synchronization.md
@@ -38,9 +38,11 @@ fail. This means the user cannot sign in or push or pull code.
The process also updates the following user information:
-- Email address
-- SSH public keys (if `sync_ssh_keys` is set)
-- Kerberos identity (if Kerberos is enabled)
+- Name. Because of a [sync issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342598), `name` is not synchronized if
+ [**Prevent users from changing their profile name**](../../../user/admin_area/settings/account_and_limit_settings.md#disable-user-profile-name-changes) is enabled.
+- Email address.
+- SSH public keys if `sync_ssh_keys` is set.
+- Kerberos identity if Kerberos is enabled.
### Adjust LDAP user sync schedule
@@ -192,7 +194,7 @@ When enabled, the following applies:
To enable it, you must:
1. [Configure LDAP](index.md#configure-ldap).
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > General**.
1. Expand the **Visibility and access controls** section.
1. Ensure the **Lock memberships to LDAP synchronization** checkbox is selected.
diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md
index 8c5bf96e99e..9f3c96902f8 100644
--- a/doc/administration/auth/oidc.md
+++ b/doc/administration/auth/oidc.md
@@ -250,7 +250,7 @@ but `LocalAccounts` works for authenticating against local, Active Directory acc
<OutputClaim ClaimTypeReferenceId="signInNames.emailAddress" PartnerClaimType="email" />
```
-1. For OIDC discovery to work with B2C, the policy must be configured with an issuer compatible with the
+1. For OIDC discovery to work with B2C, the policy must be configured with an issuer compatible with the
[OIDC specification](https://openid.net/specs/openid-connect-discovery-1_0.html#rfc.section.4.3).
See the [token compatibility settings](https://docs.microsoft.com/en-us/azure/active-directory-b2c/configure-tokens?pivots=b2c-custom-policy#token-compatibility-settings).
In `TrustFrameworkBase.xml` under `JwtIssuer`, set `IssuanceClaimPattern` to `AuthorityWithTfp`:
diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md
index 8c7f8bf766d..573ffbf4686 100644
--- a/doc/administration/compliance.md
+++ b/doc/administration/compliance.md
@@ -1,5 +1,5 @@
---
-stage: Manage
+stage: Govern
group: Compliance
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
diff --git a/doc/administration/environment_variables.md b/doc/administration/environment_variables.md
index 3adb057daa4..e70758e2774 100644
--- a/doc/administration/environment_variables.md
+++ b/doc/administration/environment_variables.md
@@ -36,6 +36,7 @@ You can use the following environment variables to override certain values:
| `GITLAB_SHARED_RUNNERS_REGISTRATION_TOKEN` | string | Sets the initial registration token used for runners. |
| `RAILS_ENV` | string | The Rails environment; can be one of `production`, `development`, `staging`, or `test`. |
| `UNSTRUCTURED_RAILS_LOG` | string | Enables the unstructured log in addition to JSON logs (defaults to `true`). |
+| `GITLAB_RAILS_CACHE_DEFAULT_TTL_SECONDS` | integer | The default TTL used for entries stored in the Rails-cache. Default is `28800`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95042) in 15.3. |
## Adding more variables
diff --git a/doc/administration/external_pipeline_validation.md b/doc/administration/external_pipeline_validation.md
index 99876cdf503..b9bf0cfdc50 100644
--- a/doc/administration/external_pipeline_validation.md
+++ b/doc/administration/external_pipeline_validation.md
@@ -44,6 +44,7 @@ required number of seconds.
"required" : [
"project",
"user",
+ "credit_card",
"pipeline",
"builds",
"total_builds_count",
@@ -85,6 +86,17 @@ required number of seconds.
"sign_in_count": { "type": "integer" }
}
},
+ "credit_card": {
+ "type": "object",
+ "required": [
+ "similar_cards_count",
+ "similar_holder_names_count"
+ ],
+ "properties": {
+ "similar_cards_count": { "type": "integer" },
+ "similar_holder_names_count": { "type": "integer" }
+ }
+ },
"pipeline": {
"type": "object",
"required": [
diff --git a/doc/administration/feature_flags.md b/doc/administration/feature_flags.md
index 77f7f621a07..75e65c2a9d6 100644
--- a/doc/administration/feature_flags.md
+++ b/doc/administration/feature_flags.md
@@ -117,7 +117,7 @@ Some feature flags can be enabled or disabled on a per project basis:
Feature.enable(:<feature flag>, Project.find(<project id>))
```
-For example, to enable the [`:product_analytics`](../operations/product_analytics.md#enable-or-disable-product-analytics) feature flag for project `1234`:
+For example, to enable the [`:product_analytics`](../operations/product_analytics.md) feature flag for project `1234`:
```ruby
Feature.enable(:product_analytics, Project.find(1234))
@@ -155,6 +155,9 @@ You can view all GitLab administrator set feature flags:
```ruby
Feature.all
=> [#<Flipper::Feature:198220 name="my_awesome_feature", state=:on, enabled_gate_names=[:boolean], adapter=:memoizable>]
+
+# Nice output
+Feature.all.map {|f| [f.name, f.state]}
```
### Unset feature flag
diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md
index 97c9a6c5576..d3aa2c97833 100644
--- a/doc/administration/geo/disaster_recovery/background_verification.md
+++ b/doc/administration/geo/disaster_recovery/background_verification.md
@@ -55,7 +55,7 @@ Feature.enable('geo_repository_verification')
On the **primary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Sites**.
1. Expand **Verification information** tab for that site to view automatic checksumming
status for repositories and wikis. Successes are shown in green, pending work
@@ -65,7 +65,7 @@ On the **primary** site:
On the **secondary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Sites**.
1. Expand **Verification information** tab for that site to view automatic checksumming
status for repositories and wikis. Successes are shown in green, pending work
@@ -93,7 +93,7 @@ increase load and vice versa.
On the **primary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Sites**.
1. Select **Edit** for the **primary** site to customize the minimum
re-verification interval:
@@ -141,7 +141,7 @@ sudo gitlab-rake geo:verification:wiki:reset
If the **primary** and **secondary** sites have a checksum verification mismatch, the cause may not be apparent. To find the cause of a checksum mismatch:
1. On the **primary** site:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Projects**.
1. Find the project that you want to check the checksum differences and
select its name.
diff --git a/doc/administration/geo/disaster_recovery/bring_primary_back.md b/doc/administration/geo/disaster_recovery/bring_primary_back.md
index a2d4f35a7c3..1991b747af0 100644
--- a/doc/administration/geo/disaster_recovery/bring_primary_back.md
+++ b/doc/administration/geo/disaster_recovery/bring_primary_back.md
@@ -41,7 +41,7 @@ To bring the former **primary** site up to date:
NOTE:
If you [changed the DNS records](index.md#step-4-optional-updating-the-primary-domain-dns-record)
- for this site during disaster recovery procedure you may need to
+ for this site during disaster recovery procedure you may need to
[block all the writes to this site](planned_failover.md#prevent-updates-to-the-primary-site)
during this procedure.
diff --git a/doc/administration/geo/disaster_recovery/index.md b/doc/administration/geo/disaster_recovery/index.md
index 0cae94fcec1..e82c130ba01 100644
--- a/doc/administration/geo/disaster_recovery/index.md
+++ b/doc/administration/geo/disaster_recovery/index.md
@@ -93,7 +93,7 @@ Note the following when promoting a secondary:
the **secondary** to the **primary**.
- If you encounter an `ActiveRecord::RecordInvalid: Validation failed: Name has already been taken`
error message during this process, for more information, see this
- [troubleshooting advice](../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node).
+ [troubleshooting advice](../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-site).
- If you run into errors when using `--force` or `--skip-preflight-checks` before 13.5 during this process,
for more information, see this
[troubleshooting advice](../replication/troubleshooting.md#errors-when-using---skip-preflight-checks-or---force).
@@ -774,4 +774,4 @@ If you are running GitLab 14.4 and earlier:
## Troubleshooting
-This section was moved to [another location](../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node).
+This section was moved to [another location](../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-site).
diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md
index 6c3353e7d7e..d0dbecce43a 100644
--- a/doc/administration/geo/disaster_recovery/planned_failover.md
+++ b/doc/administration/geo/disaster_recovery/planned_failover.md
@@ -149,7 +149,7 @@ ensure these processes are close to 100% as possible during active use.
On the **secondary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Sites**.
Replicated objects (shown in green) should be close to 100%,
and there should be no failures (shown in red). If a large proportion of
@@ -177,7 +177,7 @@ This [content was moved to another location](background_verification.md).
On the **primary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Messages**.
1. Add a message notifying users on the maintenance window.
You can check under **Geo > Sites** to estimate how long it
@@ -190,7 +190,7 @@ To ensure that all data is replicated to a secondary site, updates (write reques
be disabled on the **primary** site:
1. Enable [maintenance mode](../../maintenance_mode/index.md) on the **primary** site.
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. On the Sidekiq dashboard, select **Cron**.
1. Select `Disable All` to disable non-Geo periodic background jobs.
@@ -206,7 +206,7 @@ GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary
1. If you are manually replicating any data not managed by Geo, trigger the
final replication process now.
1. On the **primary** site:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. On the Sidekiq dashboard, select **Queues**, and wait for all queues except
those with `geo` in the name to drop to 0.
@@ -221,7 +221,7 @@ GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary
- The Geo log cursor is up to date (0 events behind).
1. On the **secondary** site:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. On the Sidekiq dashboard, select **Queues**, and wait for all the `geo`
queues to drop to 0 queued and 0 running jobs.
diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md
index 46b2ccddefd..11baf383c67 100644
--- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md
+++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md
@@ -68,7 +68,7 @@ GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary
On the **secondary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Sites** to see its status.
Replicated objects (shown in green) should be close to 100%,
and there should be no failures (shown in red). If a large proportion of
@@ -133,7 +133,7 @@ follow these steps to avoid unnecessary data loss:
connection.
1. On the **primary** site:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. On the Sidekiq dashboard, select **Cron**.
1. Select `Disable All` to disable any non-Geo periodic background jobs.
@@ -151,7 +151,7 @@ follow these steps to avoid unnecessary data loss:
[data not managed by Geo](../../replication/datatypes.md#limitations-on-replicationverification),
trigger the final replication process now.
1. On the **primary** site:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. On the Sidekiq dashboard, select **Queues**, and wait for all queues except
those with `geo` in the name to drop to 0.
@@ -166,7 +166,7 @@ follow these steps to avoid unnecessary data loss:
- The Geo log cursor is up to date (0 events behind).
1. On the **secondary** site:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. On the Sidekiq dashboard, select **Queues**, and wait for all the `geo`
queues to drop to 0 queued and 0 running jobs.
@@ -274,7 +274,7 @@ the **secondary** to the **primary**.
WARNING:
If you encounter an `ActiveRecord::RecordInvalid: Validation failed: Name has already been taken` error during this process, read
-[the troubleshooting advice](../../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node).
+[the troubleshooting advice](../../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-site).
The `gitlab-ctl promote-to-primary-node` command cannot be used yet in
conjunction with multiple servers, as it can only
diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md
index da26023e4f9..2958c119c20 100644
--- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md
+++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md
@@ -118,7 +118,7 @@ follow these steps to avoid unnecessary data loss:
connection.
1. On the **primary** site:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. On the Sidekiq dashboard, select **Cron**.
1. Select `Disable All` to disable any non-Geo periodic background jobs.
@@ -136,7 +136,7 @@ follow these steps to avoid unnecessary data loss:
[data not managed by Geo](../../replication/datatypes.md#limitations-on-replicationverification),
trigger the final replication process now.
1. On the **primary** site:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. On the Sidekiq dashboard, select **Queues**, and wait for all queues except
those with `geo` in the name to drop to 0.
@@ -151,7 +151,7 @@ follow these steps to avoid unnecessary data loss:
- The Geo log cursor is up to date (0 events behind).
1. On the **secondary** site:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. On the Sidekiq dashboard, select **Queues**, and wait for all the `geo`
queues to drop to 0 queued and 0 running jobs.
@@ -220,7 +220,7 @@ Note the following when promoting a secondary:
the **secondary** to the **primary**.
- If you encounter an `ActiveRecord::RecordInvalid: Validation failed: Name has already been taken`
error during this process, read
- [the troubleshooting advice](../../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node).
+ [the troubleshooting advice](../../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-site).
To promote the secondary site running GitLab 14.5 and later:
diff --git a/doc/administration/geo/index.md b/doc/administration/geo/index.md
index e3bf5ccdfe7..db298d4fdfc 100644
--- a/doc/administration/geo/index.md
+++ b/doc/administration/geo/index.md
@@ -47,7 +47,7 @@ In addition, it:
Geo provides:
- Read-only **secondary** sites: Maintain one **primary** GitLab site while still enabling read-only **secondary** sites for each of your distributed teams.
-- Authentication system hooks: **Secondary** sites receives all authentication data (like user accounts and logins) from the **primary** instance.
+- Authentication system hooks: **Secondary** sites receive all authentication data (like user accounts and logins) from the **primary** instance.
- An intuitive UI: **Secondary** sites use the same web interface your team has grown accustomed to. In addition, there are visual notifications that block write operations and make it clear that a user is on a **secondary** sites.
### Gitaly Cluster
@@ -124,6 +124,7 @@ The following are required to run Geo:
- Git 2.9 or later
- Git-lfs 2.4.2 or later on the user side when using LFS
- All sites must run [the same GitLab and PostgreSQL versions](setup/database.md#postgresql-replication).
+- If using different operating system versions between Geo sites, [check OS locale data compatibility](replication/troubleshooting.md#check-os-locale-data-compatibility) across Geo sites.
Additionally, check the GitLab [minimum requirements](../../install/requirements.md),
and we recommend you use the latest version of GitLab for a better experience.
@@ -158,7 +159,7 @@ public URL of the primary site is used.
To update the internal URL of the primary Geo site:
-1. On the top bar, go to **Menu > Admin > Geo > Sites**.
+1. On the top bar, select **Main menu > Admin > Geo > Sites**.
1. Select **Edit** on the primary site.
1. Change the **Internal URL**, then select **Save changes**.
@@ -204,6 +205,7 @@ This list of limitations only reflects the latest version of GitLab. If you are
- GitLab Runners cannot register with a **secondary** site. Support for this is [planned for the future](https://gitlab.com/gitlab-org/gitlab/-/issues/3294).
- [Selective synchronization](replication/configuration.md#selective-synchronization) only limits what repositories and files are replicated. The entire PostgreSQL data is still replicated. Selective synchronization is not built to accommodate compliance / export control use cases.
- [Pages access control](../../user/project/pages/pages_access_control.md) doesn't work on secondaries. See [GitLab issue #9336](https://gitlab.com/gitlab-org/gitlab/-/issues/9336) for details.
+- [GitLab chart with Geo](https://docs.gitlab.com/charts/advanced/geo/) does not support [Unified URLs](secondary_proxy/index.md#set-up-a-unified-url-for-geo-sites). See [GitLab issue #3522](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/3522) for more details.
### Limitations on replication/verification
diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md
index 7666a450648..39d1f5ae602 100644
--- a/doc/administration/geo/replication/configuration.md
+++ b/doc/administration/geo/replication/configuration.md
@@ -202,7 +202,7 @@ keys must be manually replicated to the **secondary** site.
```
1. Navigate to the Primary Node GitLab Instance:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Sites**.
1. Select **Add site**.
![Add secondary site](img/adding_a_secondary_v13_3.png)
@@ -222,19 +222,23 @@ keys must be manually replicated to the **secondary** site.
gitlab-ctl restart
```
- Check if there are any common issue with your Geo setup by running:
+ Check if there are any common issues with your Geo setup by running:
```shell
gitlab-rake gitlab:geo:check
```
+ If any of the checks fail, check the [troubleshooting documentation](troubleshooting.md).
+
1. SSH into a **Rails or Sidekiq server on your primary** site and login as root to verify the
- **secondary** site is reachable or there are any common issue with your Geo setup:
+ **secondary** site is reachable or there are any common issues with your Geo setup:
```shell
gitlab-rake gitlab:geo:check
```
+ If any of the checks fail, check the [troubleshooting documentation](troubleshooting.md).
+
Once added to the Geo administration page and restarted, the **secondary** site automatically starts
replicating missing data from the **primary** site in a process known as **backfill**.
Meanwhile, the **primary** site starts to notify each **secondary** site of any changes, so
@@ -305,7 +309,7 @@ method to be enabled. This is enabled by default, but if converting an existing
On the **primary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > General**.
1. Expand **Visibility and access controls**.
1. Ensure "Enabled Git access protocols" is set to either "Both SSH and HTTP(S)" or "Only HTTP(S)".
@@ -315,7 +319,7 @@ On the **primary** site:
You can sign in to the **secondary** site with the same credentials you used with
the **primary** site. After you sign in:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Sites**.
1. Verify that it's correctly identified as a **secondary** Geo site, and that
Geo is enabled.
diff --git a/doc/administration/geo/replication/container_registry.md b/doc/administration/geo/replication/container_registry.md
index b425e5dcc0d..01ba81b6dbe 100644
--- a/doc/administration/geo/replication/container_registry.md
+++ b/doc/administration/geo/replication/container_registry.md
@@ -12,7 +12,7 @@ You can set up a Container Registry on your **secondary** Geo site that mirrors
## Supported container registries
Geo supports the following types of container registries:
-
+
- [Docker](https://docs.docker.com/registry/)
- [OCI](https://github.com/opencontainers/distribution-spec/blob/main/spec.md)
@@ -26,7 +26,7 @@ The following container image formats are support by Geo:
In addition, Geo also supports [BuildKit cache images](https://github.com/moby/buildkit).
-## Supported storage
+## Supported storage
### Docker
@@ -34,7 +34,7 @@ For more information on supported registry storage drivers see
[Docker registry storage drivers](https://docs.docker.com/registry/storage-drivers/)
Read the [Load balancing considerations](https://docs.docker.com/registry/deploying/#load-balancing-considerations)
-when deploying the Registry, and how to set up the storage driver for the GitLab integrated
+when deploying the Registry, and how to set up the storage driver for the GitLab integrated
[Container Registry](../../packages/container_registry.md#use-object-storage).
### Registries that support OCI artifacts
@@ -160,7 +160,7 @@ For each application and Sidekiq node on the **secondary** site:
To verify Container Registry replication is working, on the **secondary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Nodes**.
The initial replication, or "backfill", is probably still in progress.
diff --git a/doc/administration/geo/replication/disable_geo.md b/doc/administration/geo/replication/disable_geo.md
index f0658ae45a2..84bc2e034b9 100644
--- a/doc/administration/geo/replication/disable_geo.md
+++ b/doc/administration/geo/replication/disable_geo.md
@@ -36,7 +36,7 @@ to do that.
To remove the **primary** site:
1. [Remove all secondary Geo sites](remove_geo_site.md)
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Nodes**.
1. Select **Remove** for the **primary** node.
1. Confirm by selecting **Remove** when the prompt appears.
diff --git a/doc/administration/geo/replication/object_storage.md b/doc/administration/geo/replication/object_storage.md
index d2e10678f8c..0336a1669f9 100644
--- a/doc/administration/geo/replication/object_storage.md
+++ b/doc/administration/geo/replication/object_storage.md
@@ -41,7 +41,7 @@ whether they are stored on the local file system or in object storage.
To enable GitLab replication:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Nodes**.
1. Select **Edit** on the **secondary** site.
1. In the **Synchronization Settings** section, find the **Allow this secondary node to replicate content on Object Storage**
diff --git a/doc/administration/geo/replication/remove_geo_site.md b/doc/administration/geo/replication/remove_geo_site.md
index 0d6715a93b7..b136f6cc8b8 100644
--- a/doc/administration/geo/replication/remove_geo_site.md
+++ b/doc/administration/geo/replication/remove_geo_site.md
@@ -9,7 +9,7 @@ type: howto
**Secondary** sites can be removed from the Geo cluster using the Geo administration page of the **primary** site. To remove a **secondary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Nodes**.
1. Select the **Remove** button for the **secondary** site you want to remove.
1. Confirm by selecting **Remove** when the prompt appears.
diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md
index 26d192f62cd..d64ad2549e8 100644
--- a/doc/administration/geo/replication/troubleshooting.md
+++ b/doc/administration/geo/replication/troubleshooting.md
@@ -19,24 +19,24 @@ Here is a list of steps you should take to attempt to fix problem:
Before attempting more advanced troubleshooting:
-- Check [the health of the **secondary** node](#check-the-health-of-the-secondary-node).
+- Check [the health of the **secondary** site](#check-the-health-of-the-secondary-site).
- Check [if PostgreSQL replication is working](#check-if-postgresql-replication-is-working).
-### Check the health of the **secondary** node
+### Check the health of the **secondary** site
-On the **primary** node:
+On the **primary** site:
-1. On the top bar, select **Menu > Admin**.
-1. On the left sidebar, select **Geo > Nodes**.
+1. On the top bar, select **Main menu > Admin**.
+1. On the left sidebar, select **Geo > Sites**.
-We perform the following health checks on each **secondary** node
+We perform the following health checks on each **secondary** site
to help identify if something is wrong:
-- Is the node running?
-- Is the node's secondary database configured for streaming replication?
-- Is the node's secondary tracking database configured?
-- Is the node's secondary tracking database connected?
-- Is the node's secondary tracking database up-to-date?
+- Is the site running?
+- Is the secondary site's database configured for streaming replication?
+- Is the secondary site's tracking database configured?
+- Is the secondary site's tracking database connected?
+- Is the secondary site's tracking database up-to-date?
![Geo health check](img/geo_site_health_v14_0.png)
@@ -48,8 +48,8 @@ health check manually to get this information and a few more details.
#### Health check Rake task
-This Rake task can be run on an app node in the **primary** or **secondary**
-Geo nodes:
+This Rake task can be run on a **Rails** node in the **primary** or **secondary**
+Geo sites:
```shell
sudo gitlab-rake gitlab:geo:check
@@ -275,11 +275,11 @@ sudo gitlab-rake gitlab:geo:check
Checking Geo ... Finished
```
- Ensure you have added the secondary node in the Admin Area of the **primary** node.
- Also ensure you entered the `external_url` or `gitlab_rails['geo_node_name']`
- when adding the secondary node in the Admin Area of the **primary** node.
- In GitLab 12.3 and earlier, edit the secondary node in the Admin Area of the **primary**
- node and ensure that there is a trailing `/` in the `Name` field.
+ Ensure you have added the secondary site in the **Main menu > Admin > Geo > Sites** on the web interface for the **primary** site.
+ Also ensure you entered the `gitlab_rails['geo_node_name']`
+ when adding the secondary site in the Admin Area of the **primary** site.
+ In GitLab 12.3 and earlier, edit the secondary site in the Admin Area of the **primary**
+ site and ensure that there is a trailing `/` in the `Name` field.
- Check returns `Exception: PG::UndefinedTable: ERROR: relation "geo_nodes" does not exist`.
@@ -321,7 +321,7 @@ error messages (indicated by `Database replication working? ... no` in the
This means that the `max_replication_slots` PostgreSQL variable needs to
be set on the **primary** database. This setting defaults to 1. You may need to
-increase this value if you have more **secondary** nodes.
+increase this value if you have more **secondary** sites.
Be sure to restart PostgreSQL for this to take effect. See the
[PostgreSQL replication setup](../setup/database.md#postgresql-replication) guide for more details.
@@ -329,13 +329,13 @@ Be sure to restart PostgreSQL for this to take effect. See the
### Message: `FATAL: could not start WAL streaming: ERROR: replication slot "geo_secondary_my_domain_com" does not exist`?
This occurs when PostgreSQL does not have a replication slot for the
-**secondary** node by that name.
+**secondary** site by that name.
-You may want to rerun the [replication process](../setup/database.md) on the **secondary** node .
+You may want to rerun the [replication process](../setup/database.md) on the **secondary** site .
### Message: "Command exceeded allowed execution time" when setting up replication?
-This may happen while [initiating the replication process](../setup/database.md#step-3-initiate-the-replication-process) on the **secondary** node,
+This may happen while [initiating the replication process](../setup/database.md#step-3-initiate-the-replication-process) on the **secondary** site,
and indicates your initial dataset is too large to be replicated in the default timeout (30 minutes).
Re-run `gitlab-ctl replicate-geo-database`, but include a larger value for
@@ -374,8 +374,8 @@ log data to build up in `pg_xlog`. Removing the unused slots can reduce the amou
Slots where `active` is `f` are not active.
-- When this slot should be active, because you have a **secondary** node configured using that slot,
- sign in to that **secondary** node and check the [PostgreSQL logs](../../logs/index.md#postgresql-logs)
+- When this slot should be active, because you have a **secondary** site configured using that slot,
+ sign in on the web interface for the **secondary** site and check the [PostgreSQL logs](../../logs/index.md#postgresql-logs)
to view why the replication is not running.
- If you are no longer using the slot (for example, you no longer have Geo enabled), you can remove it with in the
@@ -398,12 +398,12 @@ These long-running queries are
[planned to be removed in the future](https://gitlab.com/gitlab-org/gitlab/-/issues/34269),
but as a workaround, we recommend enabling
[`hot_standby_feedback`](https://www.postgresql.org/docs/10/hot-standby.html#HOT-STANDBY-CONFLICT).
-This increases the likelihood of bloat on the **primary** node as it prevents
+This increases the likelihood of bloat on the **primary** site as it prevents
`VACUUM` from removing recently-dead rows. However, it has been used
successfully in production on GitLab.com.
To enable `hot_standby_feedback`, add the following to `/etc/gitlab/gitlab.rb`
-on the **secondary** node:
+on the **secondary** site:
```ruby
postgresql['hot_standby_feedback'] = 'on'
@@ -463,14 +463,14 @@ This happens if data is detected in the `projects` table. When one or more proje
is aborted to prevent accidental data loss. To bypass this message, pass the `--force` option to the command.
In GitLab 13.4, a seed project is added when GitLab is first installed. This makes it necessary to pass `--force` even
-on a new Geo secondary node. There is an [issue to account for seed projects](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5618)
+on a new Geo secondary site. There is an [issue to account for seed projects](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5618)
when checking the database.
### Message: `Synchronization failed - Error syncing repository`
WARNING:
If large repositories are affected by this problem,
-their resync may take a long time and cause significant load on your Geo nodes,
+their resync may take a long time and cause significant load on your Geo sites,
storage and network systems.
If you see the error message `Synchronization failed - Error syncing repository` along with `fatal: fsck error in packed object`, this indicates
@@ -483,7 +483,7 @@ it's possible to override the consistency checks instead. To do that, follow
[the instructions in the Gitaly docs](../../gitaly/configure_gitaly.md#repository-consistency-checks).
You can also get the error message `Synchronization failed - Error syncing repository` along with the following log messages, this indicates that the expected `geo` remote is not present in the `.git/config` file
-of a repository on the secondary Geo node's file system:
+of a repository on the secondary Geo site's file system:
```json
{
@@ -505,7 +505,7 @@ of a repository on the secondary Geo node's file system:
To solve this:
-1. Sign in to the secondary Geo node.
+1. Sign in on the web interface for the secondary Geo site.
1. Back up [the `.git` folder](../../repository_storage_types.md#translate-hashed-storage-paths).
@@ -538,7 +538,7 @@ To solve this:
end
```
-### Very large repositories never successfully synchronize on the **secondary** node
+### Very large repositories never successfully synchronize on the **secondary** site
GitLab places a timeout on all repository clones, including project imports
and Geo synchronization operations. If a fresh `git clone` of a repository
@@ -546,7 +546,8 @@ on the **primary** takes more than the default three hours, you may be affected
To increase the timeout:
-1. On the **secondary** node, add the following line to `/etc/gitlab/gitlab.rb`:
+1. On the **Sidekiq nodes on your secondary** site,
+add the following line to `/etc/gitlab/gitlab.rb`:
```ruby
gitlab_rails['gitlab_shell_git_timeout'] = 14400
@@ -563,9 +564,9 @@ long enough to accommodate a full clone of your largest repositories.
### New LFS objects are never replicated
-If new LFS objects are never replicated to secondary Geo nodes, check the version of
+If new LFS objects are never replicated to secondary Geo sites, check the version of
GitLab you are running. GitLab versions 11.11.x or 12.0.x are affected by
-[a bug that results in new LFS objects not being replicated to Geo secondary nodes](https://gitlab.com/gitlab-org/gitlab/-/issues/32696).
+[a bug that results in new LFS objects not being replicated to Geo secondary sites](https://gitlab.com/gitlab-org/gitlab/-/issues/32696).
To resolve the issue, upgrade to GitLab 12.1 or later.
@@ -574,9 +575,9 @@ To resolve the issue, upgrade to GitLab 12.1 or later.
During a [backfill](../index.md#backfill), failures are scheduled to be retried at the end
of the backfill queue, therefore these failures only clear up **after** the backfill completes.
-### Resetting Geo **secondary** node replication
+### Resetting Geo **secondary** site replication
-If you get a **secondary** node in a broken state and want to reset the replication state,
+If you get a **secondary** site in a broken state and want to reset the replication state,
to start again from scratch, there are a few steps that can help you:
1. Stop Sidekiq and the Geo LogCursor.
@@ -617,8 +618,8 @@ to start again from scratch, there are a few steps that can help you:
1. Optional. Rename other data folders and create new ones.
WARNING:
- You may still have files on the **secondary** node that have been removed from the **primary** node, but this
- removal has not been reflected. If you skip this step, these files are not removed from the Geo node.
+ You may still have files on the **secondary** site that have been removed from the **primary** site, but this
+ removal has not been reflected. If you skip this step, these files are not removed from the Geo **secondary** site.
Any uploaded content (like file attachments, avatars, or LFS objects) is stored in a
subfolder in one of these paths:
@@ -667,7 +668,7 @@ to start again from scratch, there are a few steps that can help you:
### Design repository failures on mirrored projects and project imports
-On the top bar, under **Menu > Admin > Geo > Nodes**,
+On the top bar, under **Main menu > Admin > Geo > Sites**,
if the Design repositories progress bar shows
`Synced` and `Failed` greater than 100%, and negative `Queued`, the instance
is likely affected by
@@ -714,7 +715,7 @@ Counts:
{"synced"=>3}
```
-#### If you are promoting a Geo secondary site running on a single server
+#### If you are promoting a Geo secondary site running on a single node
`gitlab-ctl promotion-preflight-checks` fails due to the existence of
`failed` rows in the `geo_design_registry` table. Use the
@@ -831,10 +832,10 @@ We recommend transferring each failing repository individually and checking for
after each transfer. Follow the [single target `rsync` instructions](../../operations/moving_repositories.md#single-rsync-to-another-server)
to transfer each affected repository from the primary to the secondary site.
-## Fixing errors during a failover or when promoting a secondary to a primary node
+## Fixing errors during a failover or when promoting a secondary to a primary site
The following are possible error messages that might be encountered during failover or
-when promoting a secondary to a primary node with strategies to resolve them.
+when promoting a secondary to a primary site with strategies to resolve them.
### Message: `ActiveRecord::RecordInvalid: Validation failed: Name has already been taken`
@@ -868,14 +869,14 @@ or `gitlab-ctl promote-to-primary-node`, either:
```
- Upgrade to GitLab 12.6.3 or later if it is safe to do so. For example,
- if the failover was just a test. A
+ if the failover was just a test. A
[caching-related bug](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/22021) was fixed.
### Message: `ActiveRecord::RecordInvalid: Validation failed: Enabled Geo primary node cannot be disabled`
-If you disabled a secondary node, either with the [replication pause task](../index.md#pausing-and-resuming-replication)
+If you disabled a secondary site, either with the [replication pause task](../index.md#pausing-and-resuming-replication)
(GitLab 13.2) or by using the user interface (GitLab 13.1 and earlier), you must first
-re-enable the node before you can continue. This is fixed in GitLab 13.4.
+re-enable the site before you can continue. This is fixed in GitLab 13.4.
This can be fixed in the database.
@@ -894,7 +895,7 @@ This can be fixed in the database.
```
1. Run the following command, replacing `https://<secondary url>/` with the URL
- for your secondary server. You can use either `http` or `https`, but ensure that you
+ for your secondary node. You can use either `http` or `https`, but ensure that you
end the URL with a slash (`/`):
```sql
@@ -987,32 +988,31 @@ sudo gitlab-rake geo:set_secondary_as_primary
## Expired artifacts
If you notice for some reason there are more artifacts on the Geo
-secondary node than on the Geo primary node, you can use the Rake task
+**secondary** site than on the Geo **primary** site, you can use the Rake task
to [cleanup orphan artifact files](../../../raketasks/cleanup.md#remove-orphan-artifact-files).
-On a Geo **secondary** node, this command also cleans up all Geo
+On a Geo **secondary** site, this command also cleans up all Geo
registry record related to the orphan files on disk.
## Fixing sign in errors
### Message: The redirect URI included is not valid
-If you are able to sign in to the **primary** node, but you receive this error message
-when attempting to sign in to a **secondary**, you should verify the Geo
-node's URL matches its external URL.
+If you are able to sign in to the web interface for the **primary** site, but you receive this error message
+when attempting to sign in to a **secondary** web interface, you should verify the Geo
+site's URL matches its external URL.
-On the **primary** node:
+On the **primary** site:
-1. On the top bar, select **Menu > Admin**.
-1. On the left sidebar, select **Geo > Nodes**.
+1. On the top bar, select **Main menu > Admin**.
+1. On the left sidebar, select **Geo > Sites**.
1. Find the affected **secondary** site and select **Edit**.
1. Ensure the **URL** field matches the value found in `/etc/gitlab/gitlab.rb`
- in `external_url "https://gitlab.example.com"` on the frontend servers of
- the **secondary** node.
+ in `external_url "https://gitlab.example.com"` on the **Rails nodes of the secondary** site.
## Fixing common errors
-This section documents common error messages reported in the Admin Area, and how to fix them.
+This section documents common error messages reported in the Admin Area on the web interface, and how to fix them.
### Geo database configuration file is missing
@@ -1029,11 +1029,11 @@ has the correct permissions.
Geo cannot reuse an existing tracking database.
It is safest to use a fresh secondary, or reset the whole secondary by following
-[Resetting Geo secondary node replication](#resetting-geo-secondary-node-replication).
+[Resetting Geo secondary site replication](#resetting-geo-secondary-site-replication).
-### Geo node has a database that is writable which is an indication it is not configured for replication with the primary node
+### Geo site has a database that is writable which is an indication it is not configured for replication with the primary site
-This error message refers to a problem with the database replica on a **secondary** node,
+This error message refers to a problem with the database replica on a **secondary** site,
which Geo expects to have access to. It usually means, either:
- An unsupported replication method was used (for example, logical replication).
@@ -1043,24 +1043,24 @@ which Geo expects to have access to. It usually means, either:
Geo **secondary** sites require two separate PostgreSQL instances:
-- A read-only replica of the **primary** node.
+- A read-only replica of the **primary** site.
- A regular, writable instance that holds replication metadata. That is, the Geo tracking database.
This error message indicates that the replica database in the **secondary** site is misconfigured and replication has stopped.
To restore the database and resume replication, you can do one of the following:
-- [Reset the Geo secondary site replication](#resetting-geo-secondary-node-replication).
+- [Reset the Geo secondary site replication](#resetting-geo-secondary-site-replication).
- [Set up a new secondary Geo Omnibus instance](../setup/index.md#using-omnibus-gitlab).
If you set up a new secondary from scratch, you must also [remove the old site from the Geo cluster](remove_geo_site.md#removing-secondary-geo-sites).
-### Geo node does not appear to be replicating the database from the primary node
+### Geo site does not appear to be replicating the database from the primary site
The most common problems that prevent the database from replicating correctly are:
-- **Secondary** nodes cannot reach the **primary** node. Check credentials, firewall rules, and so on.
-- SSL certificate problems. Make sure you copied `/etc/gitlab/gitlab-secrets.json` from the **primary** node.
+- **Secondary** sites cannot reach the **primary** site. Check credentials, [firewall rules](../index.md#firewall-rules), and so on.
+- SSL certificate problems. Make sure you copied `/etc/gitlab/gitlab-secrets.json` from the **primary** site.
- Database storage disk is full.
- Database replication slot is misconfigured.
- Database is not using a replication slot or another alternative and cannot catch-up because WAL files were purged.
@@ -1072,26 +1072,26 @@ Make sure you follow the [Geo database replication](../setup/database.md) instru
If you are using Omnibus GitLab installation, something might have failed during upgrade. You can:
- Run `sudo gitlab-ctl reconfigure`.
-- Manually trigger the database migration by running: `sudo gitlab-rake db:migrate:geo` as root on the **secondary** node.
+- Manually trigger the database migration by running: `sudo gitlab-rake db:migrate:geo` as root on the **secondary** site.
### GitLab indicates that more than 100% of repositories were synced
This can be caused by orphaned records in the project registry. You can clear them
[using a Rake task](../../../administration/raketasks/geo.md#remove-orphaned-project-registries).
-### Geo Admin Area returns 404 error for a secondary node
+### Geo Admin Area returns 404 error for a secondary site
-Sometimes `sudo gitlab-rake gitlab:geo:check` indicates that the **secondary** node is
-healthy, but a 404 Not Found error message for the **secondary** node is returned in the Geo Admin Area on
-the **primary** node.
+Sometimes `sudo gitlab-rake gitlab:geo:check` indicates that **Rails nodes of the secondary** sites are
+healthy, but a 404 Not Found error message for the **secondary** site is returned in the Geo Admin Area on the web interface for
+the **primary** site.
To resolve this issue:
-- Try restarting the **secondary** using `sudo gitlab-ctl restart`.
-- Check `/var/log/gitlab/gitlab-rails/geo.log` to see if the **secondary** node is
- using IPv6 to send its status to the **primary** node. If it is, add an entry to
- the **primary** node using IPv4 in the `/etc/hosts` file. Alternatively, you should
- [enable IPv6 on the **primary** node](https://docs.gitlab.com/omnibus/settings/nginx.html#setting-the-nginx-listen-address-or-addresses).
+- Try restarting **each Rails, Sidekiq and Gitaly nodes on your secondary site** using `sudo gitlab-ctl restart`.
+- Check `/var/log/gitlab/gitlab-rails/geo.log` on Sidekiq nodes to see if the **secondary** site is
+ using IPv6 to send its status to the **primary** site. If it is, add an entry to
+ the **primary** site using IPv4 in the `/etc/hosts` file. Alternatively, you should
+ [enable IPv6 on the **primary** site](https://docs.gitlab.com/omnibus/settings/nginx.html#setting-the-nginx-listen-address-or-addresses).
### Secondary site returns 502 errors with Geo proxying
@@ -1167,7 +1167,7 @@ To fix this issue, set the primary site's internal URL to a URL that is:
You may have problems if you're running a version of [Git LFS](https://git-lfs.github.com/) before 2.4.2.
As noted in [this authentication issue](https://github.com/git-lfs/git-lfs/issues/3025),
-requests redirected from the secondary to the primary node do not properly send the
+requests redirected from the secondary to the primary site do not properly send the
Authorization header. This may result in either an infinite `Authorization <-> Redirect`
loop, or Authorization error messages.
@@ -1194,13 +1194,13 @@ The partial failover to a secondary Geo *site* may be the result of a temporary/
1. SSH into every Sidekiq, PostgresSQL, Gitaly, and Rails node in the **secondary** site and run one of the following commands:
- - To promote the secondary node to primary:
+ - To promote the secondary site to primary:
```shell
sudo gitlab-ctl geo promote
```
- - To promote the secondary node to primary **without any further confirmation**:
+ - To promote the secondary site to primary **without any further confirmation**:
```shell
sudo gitlab-ctl geo promote --force
@@ -1230,3 +1230,37 @@ If the above steps are **not successful**, proceed through the next steps:
1. Verify you can connect to the newly-promoted **primary** site using the URL used previously for the **secondary** site.
1. If successful, the **secondary** site is now promoted to the **primary** site.
+
+## Additional tools
+
+There are useful snippets for manipulating Geo internals in the [GitLab Rails Cheat Sheet](../../troubleshooting/gitlab_rails_cheat_sheet.md#geo). For example, you can find how to manually sync or verify a replicable in Rails console.
+
+## Check OS locale data compatibility
+
+If different operating systems or different operating system versions are deployed across Geo sites, we recommend that you perform a locale data compatibility check setting up Geo.
+
+Geo uses PostgreSQL and Streaming Replication to replicate data across Geo sites. PostgreSQL uses locale data provided by the operating system’s C library for sorting text. If the locale data in the C library is incompatible across Geo sites, erroneous query results that lead to [incorrect behavior on secondary sites](https://gitlab.com/gitlab-org/gitlab/-/issues/360723). See [here](https://wiki.postgresql.org/wiki/Locale_data_changes) for more details.
+
+On all hosts running PostgreSQL, across all Geo sites, run the following shell command:
+
+```shell
+( echo "1-1"; echo "11" ) | LC_COLLATE=en_US.UTF-8 sort
+```
+
+The output will either look like:
+
+```plaintext
+1-1
+11
+```
+
+or the reverse order:
+
+```plaintext
+11
+1-1
+```
+
+If the output is identical on all hosts, then they running compatible versions of locale data.
+
+If the output differs on some hosts, then PostgreSQL replication will not work properly. We advise that you select operating system versions that are compatible.
diff --git a/doc/administration/geo/replication/tuning.md b/doc/administration/geo/replication/tuning.md
index 755ab45a76c..370c50c93db 100644
--- a/doc/administration/geo/replication/tuning.md
+++ b/doc/administration/geo/replication/tuning.md
@@ -14,7 +14,7 @@ in the background.
On the **primary** site:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Geo > Sites**.
1. Select **Edit** of the secondary site you want to tune.
1. Under **Tuning settings**, there are several variables that can be tuned to
diff --git a/doc/administration/geo/replication/version_specific_upgrades.md b/doc/administration/geo/replication/version_specific_upgrades.md
index 350310c7076..6211b5689b1 100644
--- a/doc/administration/geo/replication/version_specific_upgrades.md
+++ b/doc/administration/geo/replication/version_specific_upgrades.md
@@ -157,6 +157,13 @@ to perform the following additional steps for the zero-downtime upgrade:
sudo gitlab-rake db:migrate
```
+1. Hot reload `puma` and `sidekiq` services:
+
+ ```shell
+ sudo gitlab-ctl hup puma
+ sudo gitlab-ctl restart sidekiq
+ ```
+
If you have already run the final `sudo gitlab-rake db:migrate` command on the deploy node and have
encountered the [column rename issue](https://gitlab.com/gitlab-org/gitlab/-/issues/324160), you might
see the following error:
@@ -183,8 +190,8 @@ GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab
each upgraded reference. Delay any upgrade attempts until this is in the
[13.7.5 patch release.](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/3002).
More details are available [in this issue](https://gitlab.com/gitlab-org/git/-/issues/79).
-- A new secret is generated in `/etc/gitlab/gitlab-secrets.json`.
- In an HA GitLab or GitLab Geo environment, secrets need to be the same on all nodes.
+- A new secret is generated in `/etc/gitlab/gitlab-secrets.json`.
+ In an HA GitLab or GitLab Geo environment, secrets need to be the same on all nodes.
Ensure this new secret is also accounted for if you are manually syncing the file across
nodes, or manually specifying secrets in `/etc/gitlab/gitlab.rb`.
@@ -247,7 +254,7 @@ the recommended procedure, see the
## Upgrading to GitLab 12.9
WARNING:
-GitLab 12.9.0 through GitLab 12.9.3 are affected by
+GitLab 12.9.0 through GitLab 12.9.3 are affected by
[a bug that stops repository verification](https://gitlab.com/gitlab-org/gitlab/-/issues/213523).
The issue is fixed in GitLab 12.9.4. Upgrade to GitLab 12.9.4 or later.
@@ -401,6 +408,6 @@ For the recommended procedure, see the
## Upgrading to GitLab 12.0
WARNING:
-This version is affected by a
+This version is affected by a
[bug that results in new LFS objects not being replicated to Geo secondary sites](https://gitlab.com/gitlab-org/gitlab/-/issues/32696).
The issue is fixed in GitLab 12.1. Be sure to upgrade to GitLab 12.1 or later.
diff --git a/doc/administration/geo/secondary_proxy/index.md b/doc/administration/geo/secondary_proxy/index.md
index 6c1812b2754..731b5012663 100644
--- a/doc/administration/geo/secondary_proxy/index.md
+++ b/doc/administration/geo/secondary_proxy/index.md
@@ -12,11 +12,6 @@ type: howto
> - [Disabled by default for different URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/325732) in GitLab 14.6 [with a flag](../../feature_flags.md) named `geo_secondary_proxy_separate_urls`.
> - [Enabled by default for different URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/346112) in GitLab 15.1.
-FLAG:
-On self-managed GitLab, this feature is only available by default for Geo sites using a unified URL. See below to
-[set up a unified URL for Geo sites](#set-up-a-unified-url-for-geo-sites).
-The feature is not ready for production use with separate URLs.
-
Use Geo proxying to:
- Have secondary sites serve read-write traffic by proxying to the primary site.
@@ -110,10 +105,13 @@ gitlab:
## Geo proxying with Separate URLs
-Since GitLab 15.1, Geo secondary proxying is enabled by default for separate URLs also.
+> Geo secondary proxying for separate URLs is [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/346112) in GitLab 15.1.
+
+NOTE:
+The feature flag described in this section is planned to be deprecated and removed in a future release. Support for read-only Geo secondary sites is proposed in [issue 366810](https://gitlab.com/gitlab-org/gitlab/-/issues/366810), you can upvote and share your use cases in that issue.
-There are minor known issues linked in the
-["Geo secondary proxying with separate URLs" epic](https://gitlab.com/groups/gitlab-org/-/epics/6865).
+There are minor known issues linked in the
+["Geo secondary proxying with separate URLs" epic](https://gitlab.com/groups/gitlab-org/-/epics/6865).
You can also add feedback in the epic about any use-cases that
are not possible anymore with proxying enabled.
diff --git a/doc/administration/gitaly/configure_gitaly.md b/doc/administration/gitaly/configure_gitaly.md
index 5b868c274cd..ac03c3ffc02 100644
--- a/doc/administration/gitaly/configure_gitaly.md
+++ b/doc/administration/gitaly/configure_gitaly.md
@@ -1089,9 +1089,10 @@ lots of CI fetch traffic.
The pack-objects cache wraps `git pack-objects`, an internal part of
Git that gets invoked indirectly via the PostUploadPack and
-SSHUploadPack Gitaly RPCs. These are the RPCs that Gitaly runs when a
-user does a Git fetch via HTTP or SSH, respectively. When the cache is
-enabled, anything that uses PostUploadPack or SSHUploadPack can
+SSHUploadPack Gitaly RPCs. Gitaly runs PostUploadPack when a
+user does a Git fetch via HTTP, or SSHUploadPack when a
+user does a Git fetch via SSH.
+When the cache is enabled, anything that uses PostUploadPack or SSHUploadPack can
benefit from it. It is orthogonal to:
- The transport (HTTP or SSH).
diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md
index b053da7ac9b..c7f7c4c58a5 100644
--- a/doc/administration/gitaly/index.md
+++ b/doc/administration/gitaly/index.md
@@ -72,7 +72,7 @@ the current status of these issues, please refer to the referenced issues and ep
|:--------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------|
| Gitaly Cluster + Geo - Issues retrying failed syncs | If Gitaly Cluster is used on a Geo secondary site, repositories that have failed to sync could continue to fail when Geo tries to resync them. Recovering from this state requires assistance from support to run manual steps. | No known solution prior to GitLab 15.0. In GitLab 15.0 to 15.2, enable the [`gitaly_praefect_generated_replica_paths` feature flag](#praefect-generated-replica-paths-gitlab-150-and-later). In GitLab 15.3, the feature flag is enabled by default. |
| Praefect unable to insert data into the database due to migrations not being applied after an upgrade | If the database is not kept up to date with completed migrations, then the Praefect node is unable to perform normal operation. | Make sure the Praefect database is up and running with all migrations completed (For example: `/opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-migrate-status` should show a list of all applied migrations). Consider [requesting upgrade assistance](https://about.gitlab.com/support/scheduling-upgrade-assistance/) so your upgrade plan can be reviewed by support. |
-| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup, it's best to snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. |
+| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup, it's best to [shut down GitLab](../read_only_gitlab.md#shut-down-the-gitlab-ui), snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. |
### Snapshot backup and recovery limitations
@@ -598,6 +598,7 @@ To migrate to Gitaly Cluster:
1. Create the required storage. Refer to
[repository storage recommendations](praefect.md#repository-storage-recommendations).
1. Create and configure [Gitaly Cluster](praefect.md).
+1. Configure the existing Gitaly instance [to use TPC](praefect.md#use-tcp-for-existing-gitlab-instances), if not already configured that way.
1. [Move the repositories](../operations/moving_repositories.md#move-repositories). To migrate to
Gitaly Cluster, existing repositories stored outside Gitaly Cluster must be moved. There is no
automatic migration but the moves can be scheduled with the GitLab API.
@@ -660,7 +661,14 @@ The code removed from GitLab during the Gitaly migration project affected these
performance workaround for these NFS-based deployments, we re-introduced some of the old Rugged
code. This re-introduced code is informally referred to as the "Rugged patches".
-### How it works
+### Automatic detection
+
+> Automatic detection for Rugged [disabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95445) in GitLab 15.3.
+
+FLAG:
+On self-managed GitLab, by default automatic detection of whether Rugged should be used (per storage) is not available.
+To make it available, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named
+`skip_rugged_auto_detect`.
The Ruby methods that perform direct Git access are behind
[feature flags](../../development/gitaly.md#legacy-rugged-code), disabled by default. It wasn't
@@ -685,12 +693,13 @@ To see if GitLab can access the repository file system directly, we use the foll
- GitLab Rails tries to read the metadata file directly. If it exists, and if the UUID's match,
assume we have direct access.
-Versions of GitLab 15.3 and later disable direct Git access by default.
+Direct Git access is:
-For versions of GitLab prior to 15.3, direct Git access is enabled by
-default in Omnibus GitLab because it fills in the correct repository
-paths in the GitLab configuration file `config/gitlab.yml`. This
-satisfies the UUID check.
+- [Disabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95445) by default in GitLab 15.3 and later for
+ compatibility with [Praefect-generated replica paths](#praefect-generated-replica-paths-gitlab-150-and-later). It
+ can be enabled if Rugged [feature flags](../../development/gitaly.md#legacy-rugged-code) are enabled.
+- Enabled by default in GitLab 15.2 and earlier because it fills in the correct repository paths in the GitLab
+ configuration file `config/gitlab.yml`. This satisfies the UUID check.
### Transition to Gitaly Cluster
diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md
index 488e6a0df5f..bd03aa1bdbc 100644
--- a/doc/administration/gitaly/praefect.md
+++ b/doc/administration/gitaly/praefect.md
@@ -97,7 +97,7 @@ If you [installed](https://about.gitlab.com/install/) GitLab using the Omnibus G
### Preparation
-Before beginning, you should already have a working GitLab instance.
+Before beginning, you should already have a working GitLab instance.
[Learn how to install GitLab](https://about.gitlab.com/install/).
Provision a PostgreSQL server. We recommend using the PostgreSQL that is shipped
@@ -164,7 +164,8 @@ The replication state is internal to each instance of GitLab and should
not be replicated.
These instructions help set up a single PostgreSQL database, which creates a single point of
-failure. Alternatively, [you can use PostgreSQL replication and failover](../postgresql/replication_and_failover.md).
+failure. To avoid this, you can configure your own clustered PostgreSQL. Support for PostgreSQL replication and failover using Omnibus GitLab is being tracked in
+[a relevant epic](https://gitlab.com/groups/gitlab-org/-/epics/7814).
The following options are available:
@@ -221,7 +222,7 @@ For using Omnibus-provided PgBouncer you need to take the following additional s
recommend using the PostgreSQL that is shipped with Omnibus as the backend. The following
instructions only work on Omnibus-provided PostgreSQL:
-1. For Omnibus-provided PgBouncer, you need to use the hash of `praefect` user instead the of the
+1. For Omnibus-provided PgBouncer, you need to use the hash of `praefect` password instead the of the
actual password:
```sql
@@ -331,7 +332,7 @@ To configure the additional connection, you must either:
#### Configure a new PgBouncer database with `pool_mode = session`
We recommend using PgBouncer with `session` pool mode. You can use the
-[bundled PgBouncer](../postgresql/pgbouncer.md) or use an external PgBouncer and
+[bundled PgBouncer](../postgresql/pgbouncer.md) or use an external PgBouncer and
[configure it manually](https://www.pgbouncer.org/config.html).
The following example uses the bundled PgBouncer and sets up two separate connection pools on PostgreSQL host,
@@ -620,7 +621,7 @@ Updates to example must be made at:
gitlab-ctl reconfigure
```
-1. To ensure that Praefect
+1. To ensure that Praefect
[has updated its Prometheus listen address](https://gitlab.com/gitlab-org/gitaly/-/issues/2734),
[restart Praefect](../restart_gitlab.md#omnibus-gitlab-restart):
@@ -884,7 +885,7 @@ For more information on Gitaly server configuration, see our
gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN'
```
-1. Configure and `internal_api_url`, which is also needed for `git push` operations:
+1. Configure an `internal_api_url`, which is also needed for `git push` operations:
```ruby
# Configure the gitlab-shell API callback URL. Without this, `git push` will
@@ -928,7 +929,7 @@ For more information on Gitaly server configuration, see our
gitlab-ctl reconfigure
```
-1. To ensure that Gitaly
+1. To ensure that Gitaly
[has updated its Prometheus listen address](https://gitlab.com/gitlab-org/gitaly/-/issues/2734),
[restart Gitaly](../restart_gitlab.md#omnibus-gitlab-restart):
@@ -1114,7 +1115,7 @@ Particular attention should be shown to:
1. Check that the Praefect storage is configured to store new repositories:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Repository**.
1. Expand the **Repository storage** section.
diff --git a/doc/administration/gitaly/recovery.md b/doc/administration/gitaly/recovery.md
index bd4846a986d..4bbf25d7cdd 100644
--- a/doc/administration/gitaly/recovery.md
+++ b/doc/administration/gitaly/recovery.md
@@ -430,6 +430,44 @@ This command fails if:
- The repository is already being tracked by the Praefect database.
- The repository does not exist on disk.
+### Manually track large numbers of repositories
+
+> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/6319) in GitLab 15.4.
+
+The `track-repositories` Praefect sub-command adds large batches of on-disk repositories to the Praefect database for tracking. This can
+be useful when migrating an existing instance to new infrastructure and ingesting all existing repositories into a fresh Gitaly Cluster.
+
+```shell
+# Omnibus GitLab install
+sudo gitlab-ctl praefect track-repositories --input-path /path/to/input.json
+
+# Source install
+sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml track-repositories -input-path /path/to/input.json
+```
+
+The command validates that all entries:
+
+- Are formatted correctly and contain required fields.
+- Correspond to a valid Git repository on disk.
+- Are not currently tracked in the Praefect database.
+
+If any entry fails these checks, the command aborts prior to attempting to track a repository.
+
+- `input-path` is the path to a file containing a list of repositories formatted as newline-delimited JSON objects. Objects must contain the following keys:
+ - `relative_path`: corresponds with `repository` in [track-repositories](#manually-track-repositories).
+ - `authoritative-storage`: the storage Praefect is to treat as the primary.
+ - `virtual-storage`: the virtual storage the repository is located in.
+
+ For example:
+
+ ```json
+ {"relative_path":"@hashed/f5/ca/f5ca38f748a1d6eaf726b8a42fb575c3c71f1864a8143301782de13da2d9202b.git","authoritative_storage":"gitaly-1","virtual_storage":"default"}
+ {"relative_path":"@hashed/f8/9f/f89f8d0e735a91c5269ab08d72fa27670d000e7561698d6e664e7b603f5c4e40.git","authoritative_storage":"gitaly-2","virtual_storage":"default"}
+ ```
+
+- `-replicate-immediately`, causes the command to replicate the repository to its secondaries immediately.
+ Otherwise, replication jobs are scheduled for execution in the database and are picked up by a Praefect background process.
+
### List virtual storage details
> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4609) in GitLab 15.1.
diff --git a/doc/administration/gitaly/troubleshooting.md b/doc/administration/gitaly/troubleshooting.md
index e308fa9da43..285ec3d2631 100644
--- a/doc/administration/gitaly/troubleshooting.md
+++ b/doc/administration/gitaly/troubleshooting.md
@@ -20,7 +20,7 @@ and our advice on [parsing the `gitaly/current` file](../logs/log_parsing.md#par
When using standalone Gitaly servers, you must make sure they are the same version
as GitLab to ensure full compatibility:
-1. On the top bar, select **Menu > Admin** on your GitLab instance.
+1. On the top bar, select **Main menu > Admin** on your GitLab instance.
1. On the left sidebar, select **Overview > Gitaly Servers**.
1. Confirm all Gitaly servers indicate that they are up to date.
diff --git a/doc/administration/housekeeping.md b/doc/administration/housekeeping.md
index 31b7ac9fd3e..cb0156f8e2d 100644
--- a/doc/administration/housekeeping.md
+++ b/doc/administration/housekeeping.md
@@ -27,7 +27,7 @@ GitLab automatically runs `git gc` and `git repack` on repositories after Git pu
You can change how often this happens or turn it off:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Repository**.
1. Expand **Repository maintenance**.
1. In the **Housekeeping** section, configure the [housekeeping options](#housekeeping-options).
diff --git a/doc/administration/inactive_project_deletion.md b/doc/administration/inactive_project_deletion.md
index 224b52d420e..824a444fdd2 100644
--- a/doc/administration/inactive_project_deletion.md
+++ b/doc/administration/inactive_project_deletion.md
@@ -1,23 +1,21 @@
---
-stage: Manage
+stage: Govern
group: Compliance
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Inactive project deletion **(FREE SELF)**
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85689) in GitLab 15.0 [with a flag](../administration/feature_flags.md) named `inactive_projects_deletion`. Disabled by default.
-
-FLAG:
-On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to
-[enable the feature flag](../administration/feature_flags.md) named `inactive_projects_deletion`.
-On GitLab.com, this feature is not available. This feature is not ready for production use.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85689) in GitLab 15.0 [with a flag](../administration/feature_flags.md) named `inactive_projects_deletion`. Disabled by default.
+> - [Feature flag `inactive_projects_deletion`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96803) removed in GitLab 15.4.
Administrators of large GitLab instances can find that over time, projects become inactive and are no longer used.
These projects take up unnecessary disk space. With inactive project deletion, you can identify these projects, warn
the maintainers ahead of time, and then delete the projects if they remain inactive. When an inactive project is
deleted, the action generates an audit event that it was performed by the first active administrator.
+For the default setting on GitLab.com, see the [GitLab.com settings page](../user/gitlab_com/index.md#inactive-project-deletion).
+
## Configure inactive project deletion
You can configure inactive projects deletion or turn it off using either:
@@ -62,7 +60,7 @@ You can use the [Application settings API](../api/settings.md#change-application
To configure inactive projects with the GitLab UI:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Repository**.
1. Expand **Repository maintenance**.
1. In the **Inactive project deletion** section, configure the necessary options.
diff --git a/doc/administration/index.md b/doc/administration/index.md
index 3f2ae3170ab..58284a74bf7 100644
--- a/doc/administration/index.md
+++ b/doc/administration/index.md
@@ -82,7 +82,7 @@ Learn how to install, configure, update, and maintain your GitLab instance.
#### Customizing GitLab appearance
-- [Header logo](../user/admin_area/appearance.md#navigation-bar): Change the logo on all pages and email headers.
+- [Header logo](../user/admin_area/appearance.md#top-bar): Change the logo on all pages and email headers.
- [Favicon](../user/admin_area/appearance.md#favicon): Change the default favicon to your own logo.
- [Branded login page](../user/admin_area/appearance.md#sign-in--sign-up-pages): Customize the login page with your own logo, title, and description.
- ["New Project" page](../user/admin_area/appearance.md#new-project-pages): Customize the text to be displayed on the page that opens whenever your users create a new project.
diff --git a/doc/administration/instance_limits.md b/doc/administration/instance_limits.md
index 31434a8ee6f..0cf2e3a1131 100644
--- a/doc/administration/instance_limits.md
+++ b/doc/administration/instance_limits.md
@@ -158,13 +158,19 @@ Set the limit to `0` to disable it.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80631) in GitLab 14.9.
-This setting limits global search requests.
+This setting limits global search requests as follows:
| Limit | Default (requests per minute) |
|-------------------------|-------------------------------|
| Authenticated user | 30 |
| Unauthenticated user | 10 |
+Depending on the number of enabled [scopes](../user/search/advanced_search.md#global-search-scopes), a global search request can consume two to seven requests per minute. You may want to disable one or more scopes to use fewer requests. Global search requests that exceed the search rate limit per minute return the following error:
+
+```plaintext
+This endpoint has been requested too many times. Try again later.
+```
+
### Pipeline creation rate limit
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362475) in GitLab 15.0.
@@ -581,7 +587,7 @@ This limit is [enabled on GitLab.com](../user/gitlab_com/index.md#gitlab-cicd).
The total number of instance level CI/CD variables is limited at the instance level.
This limit is checked each time a new instance level variable is created. If a new variable
-would cause the total number of variables to exceed the limit, the new variable is created.
+would cause the total number of variables to exceed the limit, the new variable is not created.
On self-managed instances this limit is defined for the `default` plan. By default,
this limit is set to `25`.
@@ -648,7 +654,7 @@ installation, run the following in the [GitLab Rails console](operations/rails_c
Plan.default.actual_limits.update!(ci_max_artifact_size_junit: 10)
```
-### Number of files per GitLab Pages web-site
+### Number of files per GitLab Pages website
The total number of file entries (including directories and symlinks) is limited to `200,000` per
GitLab Pages website.
@@ -663,6 +669,14 @@ For example, to change the limit to `100`:
Plan.default.actual_limits.update!(pages_file_entries: 100)
```
+### Number of custom domains per GitLab Pages website
+
+The total number of custom domains per GitLab Pages website is limited to `150` for [GitLab SaaS](../subscriptions/gitlab_com/index.md).
+
+The default limit for [GitLab self-managed](../subscriptions/self_managed/index.md) is `0` (unlimited).
+To set a limit on your self-managed instance, use the
+[Admin Area](pages/index.md#set-maximum-number-of-gitlab-pages-custom-domains-for-a-project).
+
### Number of registered runners per scope
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321368) in GitLab 13.12. Disabled by default.
@@ -736,7 +750,7 @@ You can change these limits in the [GitLab Rails console](operations/rails_conso
- To update the maximum YAML size, update `max_yaml_size_bytes` with the new value in megabytes:
```ruby
- ApplicationSetting.update!(max_yaml_size_bytes: 2.megabytes)
+ ApplicationSetting.update(max_yaml_size_bytes: 2.megabytes)
```
The `max_yaml_size_bytes` value is not directly tied to the size of the YAML file,
@@ -745,7 +759,7 @@ You can change these limits in the [GitLab Rails console](operations/rails_conso
- To update the maximum YAML depth, update `max_yaml_depth` with the new value in megabytes:
```ruby
- ApplicationSetting.update!(max_yaml_depth: 125)
+ ApplicationSetting.update(max_yaml_depth: 125)
```
### Limit dotenv variables
diff --git a/doc/administration/integration/kroki.md b/doc/administration/integration/kroki.md
index a6fcfe6c80f..fb4659175b0 100644
--- a/doc/administration/integration/kroki.md
+++ b/doc/administration/integration/kroki.md
@@ -56,7 +56,7 @@ read the [Kroki installation](https://docs.kroki.io/kroki/setup/install/#_images
You need to enable Kroki integration from Settings under Admin Area.
To do that, log in with an administrator account and follow these steps:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. Go to **Settings > General**.
1. Expand the **Kroki** section.
1. Select **Enable Kroki** checkbox.
diff --git a/doc/administration/integration/mailgun.md b/doc/administration/integration/mailgun.md
index c007116d213..37e81f220cf 100644
--- a/doc/administration/integration/mailgun.md
+++ b/doc/administration/integration/mailgun.md
@@ -43,7 +43,7 @@ After configuring your Mailgun domain for the webhook endpoints,
you're ready to enable the Mailgun integration:
1. Sign in to GitLab as an [Administrator](../../user/permissions.md) user.
-1. On the top bar, select **Menu >** **{admin}** **Admin**.
+1. On the top bar, select **Main menu >** **{admin}** **Admin**.
1. On the left sidebar, go to **Settings > General** and expand the **Mailgun** section.
1. Select the **Enable Mailgun** check box.
1. Enter the Mailgun HTTP webhook signing key as described in
diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md
index 58c9e01a151..de790c7ce40 100644
--- a/doc/administration/integration/plantuml.md
+++ b/doc/administration/integration/plantuml.md
@@ -211,7 +211,7 @@ stop;
After configuring your local PlantUML server, you're ready to enable the PlantUML integration:
1. Sign in to GitLab as an [Administrator](../../user/permissions.md) user.
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, go to **Settings > General** and expand the **PlantUML** section.
1. Select the **Enable PlantUML** checkbox.
1. Set the PlantUML instance as `https://gitlab.example.com/-/plantuml/`,
diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md
index 88b320941de..41984bbe7a7 100644
--- a/doc/administration/integration/terminal.md
+++ b/doc/administration/integration/terminal.md
@@ -1,6 +1,6 @@
---
-stage: Ecosystem
-group: Integrations
+stage: Configure
+group: Configure
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
@@ -114,7 +114,7 @@ they receive a `Connection failed` message.
By default, terminal sessions do not expire. To limit the terminal session
lifetime in your GitLab instance:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. Select
[**Settings > Web terminal**](../../user/admin_area/settings/index.md#general).
1. Set a `max session time`.
diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md
index a4ab0e07020..14749a9c7f6 100644
--- a/doc/administration/job_artifacts.md
+++ b/doc/administration/job_artifacts.md
@@ -437,6 +437,16 @@ If you need to manually remove job artifacts associated with multiple jobs while
```ruby
builds_to_clear = builds_with_artifacts.where("finished_at < ?", 1.week.ago)
builds_to_clear.find_each do |build|
+ Ci::JobArtifacts::DeleteService.new(build).execute
+ build.update!(artifacts_expire_at: Time.now)
+ end
+ ```
+
+ In [GitLab 15.3 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/372537), use the following instead:
+
+ ```ruby
+ builds_to_clear = builds_with_artifacts.where("finished_at < ?", 1.week.ago)
+ builds_to_clear.find_each do |build|
build.artifacts_expire_at = Time.now
build.erase_erasable_artifacts!
end
@@ -489,7 +499,7 @@ If you need to manually remove **all** job artifacts associated with multiple jo
print "Ci::Build ID #{build.id}... "
if build.erasable?
- build.erase(erased_by: admin_user)
+ Ci::BuildEraseService.new(build, admin_user).execute
puts "Erased"
else
puts "Skipped (Nothing to erase or not erasable)"
@@ -497,6 +507,9 @@ If you need to manually remove **all** job artifacts associated with multiple jo
end
```
+ In [GitLab 15.3 and earlier](https://gitlab.com/gitlab-org/gitlab/-/issues/369132), replace
+ `Ci::BuildEraseService.new(build, admin_user).execute` with `build.erase(erased_by: admin_user)`.
+
`1.week.ago` is a Rails `ActiveSupport::Duration` method which calculates a new
date or time in the past. Other valid examples are:
diff --git a/doc/administration/job_logs.md b/doc/administration/job_logs.md
index d2837bfa96e..84da4e31d92 100644
--- a/doc/administration/job_logs.md
+++ b/doc/administration/job_logs.md
@@ -134,8 +134,8 @@ For more information, see [delete references to missing artifacts](raketasks/che
## Incremental logging architecture
-> - [Deployed behind a feature flag](../user/feature_flags.md), disabled by default.
-> - Enabled on GitLab.com.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/18169) in GitLab 10.8 [with a flag](feature_flags.md) named `ci_enable_live_trace`. Disabled by default.
+> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/241471) in GitLab 13.6.
> - [Recommended for production use](https://gitlab.com/groups/gitlab-org/-/epics/4275) in GitLab 13.6.
> - [Recommended for production use with AWS S3](https://gitlab.com/gitlab-org/gitlab/-/issues/273498) in GitLab 13.7.
> - To use in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-incremental-logging).
diff --git a/doc/administration/load_balancer.md b/doc/administration/load_balancer.md
index 988bddcfe62..87b63c6272d 100644
--- a/doc/administration/load_balancer.md
+++ b/doc/administration/load_balancer.md
@@ -31,7 +31,7 @@ Configure your load balancers to pass connections on port 443 as 'TCP' rather
than 'HTTP(S)' protocol. This passes the connection to the application nodes
NGINX service untouched. NGINX has the SSL certificate and listen on port 443.
-See [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Load Balancers terminate SSL without backend SSL
@@ -41,8 +41,8 @@ The load balancers is be responsible for managing SSL certificates and
terminating SSL.
Because communication between the load balancers and GitLab isn't secure,
-there is some additional configuration needed. See
-[NGINX Proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl)
+there is some additional configuration needed. See the
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
for details.
### Load Balancers terminate SSL with backend SSL
@@ -55,7 +55,7 @@ Traffic is secure between the load balancers and NGINX in this
scenario. There is no need to add configuration for proxied SSL because the
connection is secure all the way. However, configuration must be
added to GitLab to configure SSL certificates. See
-[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
## Ports
@@ -115,14 +115,18 @@ Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`.
It is strongly recommend that multi-node deployments configure load balancers to use the [readiness check](../user/admin_area/monitoring/health_check.md#readiness) to ensure a node is ready to accept traffic, before routing traffic to it. This is especially important when utilizing Puma, as there is a brief period during a restart where Puma doesn't accept requests.
-<!-- ## Troubleshooting
+## Troubleshooting
-Include any troubleshooting steps that you can foresee. If you know beforehand what issues
-one might have when setting this up, or when something is changed, or on upgrading, it's
-important to describe those, too. Think of things that may go wrong and include them here.
-This is important to minimize requests for support, and to avoid doc comments with
-questions that you know someone might ask.
+### The health check is returning a `408` HTTP code via the load balancer
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
-If you have none to add when creating a doc, leave this section in place
-but commented out to help encourage others to add to it in the future. -->
+If you are using [AWS's Classic Load Balancer](https://docs.aws.amazon.com/en_en/elasticloadbalancing/latest/classic/elb-ssl-security-policy.html#ssl-ciphers)
+in GitLab 15.0 or later, you must to enable the `AES256-GCM-SHA384` cipher in NGINX.
+See [AES256-GCM-SHA384 SSL cipher no longer allowed by default by NGINX](https://docs.gitlab.com/omnibus/update/gitlab_15_changes.html#aes256-gcm-sha384-ssl-cipher-no-longer-allowed-by-default-by-nginx)
+for more information.
+
+The default ciphers for a GitLab version can be
+viewed in the [`files/gitlab-cookbooks/gitlab/attributes/default.rb`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb)
+file and selecting the Git tag that correlates with your target GitLab version
+(for example `15.0.5+ee.0`). If required by your load balancer, you can then define
+[custom SSL ciphers](https://docs.gitlab.com/omnibus/settings/ssl.html#use-custom-ssl-ciphers)
+for NGINX.
diff --git a/doc/administration/maintenance_mode/index.md b/doc/administration/maintenance_mode/index.md
index 00e28e650ea..60de8e2fd3a 100644
--- a/doc/administration/maintenance_mode/index.md
+++ b/doc/administration/maintenance_mode/index.md
@@ -21,7 +21,7 @@ Maintenance Mode allows most external actions that do not change internal state.
There are three ways to enable Maintenance Mode as an administrator:
- **Web UI**:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > General**.
1. Expand **Maintenance Mode**, and toggle **Enable Maintenance Mode**.
You can optionally add a message for the banner as well.
@@ -45,7 +45,7 @@ There are three ways to enable Maintenance Mode as an administrator:
There are three ways to disable Maintenance Mode:
- **Web UI**:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > General**.
1. Expand **Maintenance Mode**, and toggle **Enable Maintenance Mode**.
You can optionally add a message for the banner as well.
@@ -173,7 +173,7 @@ it is recommended that you disable all cron jobs except for those related to Geo
To monitor queues and disable jobs:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
### Incident management
diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md
index 2553638291d..4b62a8ae931 100644
--- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md
+++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md
@@ -38,12 +38,12 @@ This project can be used to:
## Create the self monitoring project
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Metrics and profiling** and expand **Self monitoring**.
1. Toggle **Self monitoring** on.
1. After your GitLab instance creates the project, GitLab displays a link to the
project in the text above the **Self monitoring** toggle. You can also find it
- from the top bar by selecting **Menu > Project**, then selecting **Your projects**.
+ from the top bar by selecting **Main menu > Projects**.
## Delete the self monitoring project
@@ -51,7 +51,7 @@ WARNING:
Deleting the self monitoring project removes any changes made to the project. If
you create the project again, it's created in its default state.
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, go to **Settings > Metrics and profiling** and expand **Self monitoring**.
1. Toggle **Self monitoring** off.
1. In the confirmation dialog that opens, select **Delete self monitoring project**.
diff --git a/doc/administration/monitoring/performance/gitlab_configuration.md b/doc/administration/monitoring/performance/gitlab_configuration.md
index 128ddad6555..74db35eebc2 100644
--- a/doc/administration/monitoring/performance/gitlab_configuration.md
+++ b/doc/administration/monitoring/performance/gitlab_configuration.md
@@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
GitLab Performance Monitoring is disabled by default. To enable it and change any of its
settings:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Metrics and profiling**
(`/admin/application_settings/metrics_and_profiling`).
1. Add the necessary configuration changes.
diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md
index 3a76e2e4578..6e9ea0d8d42 100644
--- a/doc/administration/monitoring/performance/grafana_configuration.md
+++ b/doc/administration/monitoring/performance/grafana_configuration.md
@@ -62,7 +62,7 @@ repository.
After setting up Grafana, you can enable a link to access it easily from the
GitLab sidebar:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Metrics and profiling**
and expand **Metrics - Grafana**.
1. Select the **Add a link to Grafana** checkbox.
@@ -72,14 +72,14 @@ GitLab sidebar:
- *Otherwise,* enter the full URL of the Grafana instance.
1. Select **Save changes**.
-GitLab displays your link in the **Menu > Admin > Monitoring > Metrics Dashboard**.
+GitLab displays your link in the **Main menu > Admin > Monitoring > Metrics Dashboard**.
## Required Scopes
> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5822) in GitLab 13.10.
When setting up Grafana through the process above, no scope shows in the screen at
-**Menu > Admin > Applications > GitLab Grafana**. However, the `read_user` scope is
+**Main menu > Admin > Applications > GitLab Grafana**. However, the `read_user` scope is
required and is provided to the application automatically. Setting any scope other than
`read_user` without also including `read_user` leads to this error when you try to log in using
GitLab as the OAuth provider:
diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md
index 759f485c109..c23046158e1 100644
--- a/doc/administration/monitoring/performance/performance_bar.md
+++ b/doc/administration/monitoring/performance/performance_bar.md
@@ -108,7 +108,7 @@ The performance bar is disabled by default for non-administrators. To enable it
for a given group:
1. Sign in as a user with administrator access.
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Metrics and profiling**
(`admin/application_settings/metrics_and_profiling`), and expand
**Profiling - Performance bar**.
diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md
index a2def8a9f64..00dae8e4dd5 100644
--- a/doc/administration/monitoring/prometheus/gitlab_metrics.md
+++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md
@@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
To enable the GitLab Prometheus metrics:
1. Log in to GitLab as a user with administrator access.
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Metrics and profiling**.
1. Find the **Metrics - Prometheus** section, and select **Add link to Prometheus**.
1. [Restart GitLab](../../restart_gitlab.md#omnibus-gitlab-restart) for the changes to take effect.
@@ -146,6 +146,8 @@ The following metrics are available:
| `gitlab_snowplow_failed_events_total` | Counter | 14.1 | Total number of GitLab Snowplow product intelligence events emission failures | |
| `gitlab_snowplow_successful_events_total` | Counter | 14.1 | Total number of GitLab Snowplow product intelligence events emission successes | |
| `gitlab_ci_build_trace_errors_total` | Counter | 14.4 | Total amount of different error types on a build trace | `error_reason` |
+| `gitlab_presentable_object_cacheless_render_real_duration_seconds` | Histogram | 15.3 | Duration of real time spent caching and representing specific web request objects | `controller`, `action` |
+| `cached_object_operations_total` | Counter | 15.3 | Total number of objects cached for specific web requests | `controller`, `action` |
## Metrics controlled by a feature flag
@@ -390,7 +392,7 @@ Some basic Ruby runtime metrics are available:
## Redis metrics
These client metrics are meant to complement Redis server metrics.
-These metrics are broken down per
+These metrics are broken down per
[Redis instance](https://docs.gitlab.com/omnibus/settings/redis.html#running-with-multiple-redis-instances).
These metrics all have a `storage` label which indicates the Redis
instance (`cache`, `shared_state`, and so on).
diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md
index 6f6ac5c5d4b..c4aa607fa4d 100644
--- a/doc/administration/monitoring/prometheus/index.md
+++ b/doc/administration/monitoring/prometheus/index.md
@@ -277,11 +277,6 @@ To use an external Prometheus server:
static_configs:
- targets:
- 1.1.1.1:9168
- - job_name: gitlab_exporter_process
- metrics_path: "/process"
- static_configs:
- - targets:
- - 1.1.1.1:9168
- job_name: gitaly
static_configs:
- targets:
diff --git a/doc/administration/nfs.md b/doc/administration/nfs.md
index 29f00fec3f7..9967cce5b73 100644
--- a/doc/administration/nfs.md
+++ b/doc/administration/nfs.md
@@ -95,34 +95,22 @@ We are investigating the use of
## Improving NFS performance with GitLab
NFS performance with GitLab can in some cases be improved with
-[direct Git access](gitaly/index.md#direct-access-to-git-in-gitlab) using
-[Rugged](https://github.com/libgit2/rugged).
+[direct Git access](gitaly/index.md#direct-access-to-git-in-gitlab) using [Rugged](https://github.com/libgit2/rugged).
-Versions of GitLab after 12.2 and prior to 15.3 automatically detect if
-Rugged can and should be used per storage.
+Depending on the GitLab version, GitLab [automatically detects](gitaly/index.md#automatic-detection) if Rugged can and should
+be used per storage.
-NOTE:
-GitLab 15.3 and later disables this automatic detection. Auto-detection can be enabled via the
-`skip_rugged_auto_detect` feature flag:
-
-```ruby
-Feature.disable(:skip_rugged_auto_detect)
-```
-
-In addition, if you previously enabled Rugged using the feature flag and
-you want to use automatic detection instead, you must unset the feature
-flag:
+If the Rugged feature flag is explicitly set to either `true` or `false`, GitLab uses the value explicitly set. If you
+previously enabled Rugged using the feature flag and you want to use automatic detection instead, you must unset
+the feature flag:
```shell
sudo gitlab-rake gitlab:features:unset_rugged
```
-If the Rugged feature flag is explicitly set to either `true` or `false`, GitLab uses the value explicitly set.
-
-From GitLab 12.7, Rugged is only automatically enabled for use with Puma
-if the [Puma thread count is set to `1`](../install/requirements.md#puma-settings).
-
-To use Rugged with a Puma thread count of more than `1`, enable Rugged using the [feature flag](../development/gitaly.md#legacy-rugged-code).
+From GitLab 12.7, Rugged is only automatically enabled for use with Puma if the
+[Puma thread count is set to `1`](../install/requirements.md#puma-settings). To use Rugged with a Puma thread count of
+more than `1`, enable Rugged using the [feature flag](../development/gitaly.md#legacy-rugged-code).
## NFS server
diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md
index 0299d5f8b0c..fd9ab9b5972 100644
--- a/doc/administration/object_storage.md
+++ b/doc/administration/object_storage.md
@@ -26,7 +26,7 @@ GitLab has been tested by vendors and customers on a number of object storage pr
### Known compatibility issues
-- Dell EMC ECS: Prior to GitLab 13.3, there is a
+- Dell EMC ECS: Prior to GitLab 13.3, there is a
[known bug in GitLab Workhorse that prevents HTTP Range Requests from working with CI job artifacts](https://gitlab.com/gitlab-org/gitlab/-/issues/223806).
Be sure to upgrade to GitLab 13.3.0 or above if you use S3 storage with this hardware.
@@ -528,7 +528,7 @@ supported by consolidated configuration form, refer to the following guides:
| Object storage type | Supported by consolidated configuration? |
|---------------------|------------------------------------------|
-| [Backups](../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | **{dotted-circle}** No |
+| [Backups](../raketasks/backup_gitlab.md#upload-backups-to-a-remote-cloud-storage) | **{dotted-circle}** No |
| [Job artifacts](job_artifacts.md#using-object-storage) including archived job logs | **{check-circle}** Yes |
| [LFS objects](lfs/index.md#storing-lfs-objects-in-remote-object-storage) | **{check-circle}** Yes |
| [Uploads](uploads.md#using-object-storage) | **{check-circle}** Yes |
@@ -578,7 +578,7 @@ real bucket into multiple virtual buckets. If your object storage
bucket is called `my-gitlab-objects` you can configure uploads to go
into `my-gitlab-objects/uploads`, artifacts into
`my-gitlab-objects/artifacts`, etc. The application will act as if
-these are separate buckets. Note that use of bucket prefixes
+these are separate buckets. Note that use of bucket prefixes
[may not work correctly with Helm backups](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/3376).
Helm-based installs require separate buckets to
@@ -693,7 +693,7 @@ configuration.
When configured either with an instance profile or with the consolidated
object configuration, GitLab Workhorse properly uploads files to S3
buckets that have [SSE-S3 or SSE-KMS encryption enabled by default](https://docs.aws.amazon.com/kms/latest/developerguide/services-s3.html).
-Customer master keys (CMKs) and SSE-C encryption are
+Customer master keys (CMKs) and SSE-C encryption are
[not supported since this requires sending the encryption keys in every request](https://gitlab.com/gitlab-org/gitlab/-/issues/226006).
##### Server-side encryption headers
@@ -701,7 +701,7 @@ Customer master keys (CMKs) and SSE-C encryption are
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38240) in GitLab 13.3.
Setting a default encryption on an S3 bucket is the easiest way to
-enable encryption, but you may want to
+enable encryption, but you may want to
[set a bucket policy to ensure only encrypted objects are uploaded](https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-store-kms-encrypted-objects/).
To do this, you must configure GitLab to send the proper encryption headers
in the `storage_options` configuration section:
diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md
index a3240a6041b..8523b881730 100644
--- a/doc/administration/operations/fast_ssh_key_lookup.md
+++ b/doc/administration/operations/fast_ssh_key_lookup.md
@@ -106,7 +106,7 @@ users as long as a large file exists.
To disable writes to the `authorized_keys` file:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Network**.
1. Expand **Performance optimization**.
1. Clear the **Use authorized_keys file to authenticate SSH keys** checkbox.
@@ -125,7 +125,7 @@ This overview is brief. Refer to the above instructions for more context.
1. [Rebuild the `authorized_keys` file](../raketasks/maintenance.md#rebuild-authorized_keys-file).
1. Enable writes to the `authorized_keys` file.
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Network**.
1. Expand **Performance optimization**.
1. Select the **Use authorized_keys file to authenticate SSH keys** checkbox.
diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md
index a6e66abdbdb..179958c6df1 100644
--- a/doc/administration/operations/index.md
+++ b/doc/administration/operations/index.md
@@ -18,7 +18,7 @@ Keep your GitLab instance up and running smoothly.
- [Multiple Sidekiq processes](extra_sidekiq_processes.md): Configure multiple Sidekiq processes to ensure certain queues always have dedicated workers, no matter the number of jobs that must be processed. **(FREE SELF)**
- [Sidekiq routing rules](extra_sidekiq_routing.md): Configure the routing rules to route a job from a worker to a desirable queue. **(FREE SELF)**
- [Puma](puma.md): Understand Puma and puma-worker-killer.
-- Speed up SSH operations by
+- Speed up SSH operations by
[Authorizing SSH users via a fast, indexed lookup to the GitLab database](fast_ssh_key_lookup.md), and/or
by [doing away with user SSH keys stored on GitLab entirely in favor of SSH certificates](ssh_certificates.md).
- [File System Performance Benchmarking](filesystem_benchmarking.md): File system
diff --git a/doc/administration/operations/puma.md b/doc/administration/operations/puma.md
index 5ce469d3e63..8e7594dfc2d 100644
--- a/doc/administration/operations/puma.md
+++ b/doc/administration/operations/puma.md
@@ -156,10 +156,8 @@ For deployments where NFS is used to store Git repositories, GitLab uses
[direct Git access](../gitaly/index.md#direct-access-to-git-in-gitlab) to improve performance by using
[Rugged](https://github.com/libgit2/rugged).
-Rugged usage is automatically enabled if direct Git access
-[is available](../gitaly/index.md#how-it-works)
-and Puma is running single threaded, unless it is disabled by a
-[feature flag](../../development/gitaly.md#legacy-rugged-code).
+Rugged usage is automatically enabled if direct Git access [is available](../gitaly/index.md#automatic-detection) and
+Puma is running single threaded, unless it is disabled by a [feature flag](../../development/gitaly.md#legacy-rugged-code).
MRI Ruby uses a Global VM Lock (GVL). GVL allows MRI Ruby to be multi-threaded, but running at
most on a single core.
diff --git a/doc/administration/operations/rails_console.md b/doc/administration/operations/rails_console.md
index 430dfbc637c..627dfbeb66c 100644
--- a/doc/administration/operations/rails_console.md
+++ b/doc/administration/operations/rails_console.md
@@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Rails console **(FREE SELF)**
-At the heart of GitLab is a web application
+At the heart of GitLab is a web application
[built using the Ruby on Rails framework](https://about.gitlab.com/blog/2018/10/29/why-we-use-rails-to-build-gitlab/).
The [Rails console](https://guides.rubyonrails.org/command_line.html#rails-console).
provides a way to interact with your GitLab instance from the command line, and also grants access to the amazing tools built right into Rails.
@@ -19,7 +19,7 @@ with no consequences, you are strongly advised to do so in a test environment.
The Rails console is for GitLab system administrators who are troubleshooting
a problem or need to retrieve some data that can only be done through direct
-access of the GitLab application. Basic knowledge of Ruby is needed (try
+access of the GitLab application. Basic knowledge of Ruby is needed (try
[this 30-minute tutorial](https://try.ruby-lang.org/) for a quick introduction).
Rails experience is useful but not required.
@@ -136,7 +136,7 @@ root
1
```
-Some basic knowledge of Ruby will be very useful. Try
+Some basic knowledge of Ruby will be very useful. Try
[this 30-minute tutorial](https://try.ruby-lang.org/) for a quick introduction.
Rails experience is helpful but not essential.
@@ -202,6 +202,44 @@ sudo chmod 700 /scripts
sudo gitlab-rails runner /scripts/helloworld.rb
```
+## Find specific methods for an object
+
+```ruby
+Array.methods.select { |m| m.to_s.include? "sing" }
+Array.methods.grep(/sing/)
+```
+
+## Find method source
+
+```ruby
+instance_of_object.method(:foo).source_location
+
+# Example for when we would call project.private?
+project.method(:private?).source_location
+```
+
+## Limiting output
+
+Adding a semicolon(`;`) and a follow-up statement at the end of a statement prevents the default implicit return output. This can be used if you are already explicitly printing details and potentially have a lot of return output:
+
+```ruby
+puts ActiveRecord::Base.descendants; :ok
+Project.select(&:pages_deployed?).each {|p| puts p.pages_url }; true
+```
+
+## Get or store the result of last operation
+
+Underscore(`_`) represents the implicit return of the previous statement. You can use this to quickly assign a variable from the output of the previous command:
+
+```ruby
+Project.last
+# => #<Project id:2537 root/discard>>
+project = _
+# => #<Project id:2537 root/discard>>
+project.id
+# => 2537
+```
+
## Active Record objects
### Looking up database-persisted objects
@@ -331,6 +369,15 @@ D, [2020-03-05T17:18:30.406047 #910] DEBUG -- : User Load (2.6ms) SELECT "use
For more on different ways to retrieve data from the database using Active
Record, please see the [Active Record Query Interface documentation](https://guides.rubyonrails.org/active_record_querying.html).
+## Query the database using an Active Record model
+
+```ruby
+m = Model.where('attribute like ?', 'ex%')
+
+# for example to query the projects
+projects = Project.where('path like ?', 'Oumua%')
+```
+
### Modifying Active Record objects
In the previous section, we learned about retrieving database records using
diff --git a/doc/administration/operations/ssh_certificates.md b/doc/administration/operations/ssh_certificates.md
index 1e405189342..8069dad4d8d 100644
--- a/doc/administration/operations/ssh_certificates.md
+++ b/doc/administration/operations/ssh_certificates.md
@@ -159,7 +159,7 @@ users (especially if they're renewed) than you have deploy keys.
Users can still bypass SSH certificate authentication by manually
uploading an SSH public key to their profile, relying on the
`~/.ssh/authorized_keys` fallback to authenticate it. There's
-currently no feature to prevent this,
+currently no feature to prevent this,
[but there's an open request for adding it](https://gitlab.com/gitlab-org/gitlab/-/issues/23260).
Such a restriction can currently be hacked in by, for example, providing a
diff --git a/doc/administration/package_information/supported_os.md b/doc/administration/package_information/supported_os.md
index e4d7ea72cdc..5ccabd66ed0 100644
--- a/doc/administration/package_information/supported_os.md
+++ b/doc/administration/package_information/supported_os.md
@@ -24,6 +24,7 @@ The following lists the currently supported OSs and their possible EOL dates.
| OpenSUSE 15.3 | GitLab CE / GitLab EE 14.5.0 | x86_64, aarch64 | [OpenSUSE Install Documentation](https://about.gitlab.com/install/#opensuse-leap-15-3) | Nov 2022 | <https://en.opensuse.org/Lifetime> |
| RHEL 8 | GitLab CE / GitLab EE 12.8.1 | x86_64, arm64 | [Use CentOS Install Documentation](https://about.gitlab.com/install/#centos-7) | May 2029 | [RHEL Details](https://access.redhat.com/support/policy/updates/errata/#Life_Cycle_Dates) |
| SLES 12 | GitLab EE 9.0.0 | x86_64 | [Use OpenSUSE Install Documentation](https://about.gitlab.com/install/#opensuse-leap-15-3) | Oct 2027 | <https://www.suse.com/lifecycle/> |
+| SLES 15 | GitLab EE 14.8.0 | x86_64 | [Use OpenSUSE Install Documentation](https://about.gitlab.com/install/#opensuse-leap-15-3) | Dec 2024 | <https://www.suse.com/lifecycle/> |
| Oracle Linux | GitLab CE / GitLab EE 8.14.0 | x86_64 | [Use CentOS Install Documentation](https://about.gitlab.com/install/#centos-7) | Jul 2024 | <https://www.oracle.com/a/ocom/docs/elsp-lifetime-069338.pdf> |
| Scientific Linux | GitLab CE / GitLab EE 8.14.0 | x86_64 | [Use CentOS Install Documentation](https://about.gitlab.com/install/#centos-7) | June 2024 | <https://scientificlinux.org/downloads/sl-versions/sl7/> |
| Ubuntu 18.04 | GitLab CE / GitLab EE 10.7.0 | amd64 | [Ubuntu Install Documentation](https://about.gitlab.com/install/#ubuntu) | April 2023 | <https://wiki.ubuntu.com/Releases> |
diff --git a/doc/administration/packages/container_registry.md b/doc/administration/packages/container_registry.md
index 8b115ca1af4..537840ce785 100644
--- a/doc/administration/packages/container_registry.md
+++ b/doc/administration/packages/container_registry.md
@@ -23,7 +23,7 @@ may or may not be available by default.
The Container Registry is automatically enabled and available on your GitLab domain, port 5050 if:
-- You're using the built-in [Let's Encrypt integration](https://docs.gitlab.com/omnibus/settings/ssl.html#lets-encrypt-integration), and
+- You're using the built-in [Let's Encrypt integration](https://docs.gitlab.com/omnibus/settings/ssl.html#enable-the-lets-encrypt-integration), and
- You're using GitLab 12.5 or later.
Otherwise, the Container Registry is not enabled. To enable it:
@@ -199,7 +199,7 @@ a wildcard certificate if hosted under a subdomain of your existing GitLab
domain, for example, `registry.gitlab.example.com`.
As well as manually generated SSL certificates (explained here), certificates automatically
-generated by Let's Encrypt are also [supported in Omnibus installs](https://docs.gitlab.com/omnibus/settings/ssl.html#host-services).
+generated by Let's Encrypt are also [supported in Omnibus installs](https://docs.gitlab.com/omnibus/settings/ssl.html).
Let's assume that you want the container Registry to be accessible at
`https://registry.gitlab.example.com`.
@@ -322,7 +322,7 @@ the Container Registry by themselves, follow the steps below.
In GitLab, tokens for the Container Registry expire every five minutes.
To increase the token duration:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > CI/CD**.
1. Expand **Container Registry**.
1. For the **Authorization token duration (minutes)**, update the value.
@@ -906,7 +906,7 @@ project.container_repositories.find_each do |repo|
puts repo.attributes
# Start the tag cleanup
- puts Projects::ContainerRepository::CleanupTagsService.new(repo, user, policy.attributes.except("created_at", "updated_at")).execute()
+ puts Projects::ContainerRepository::CleanupTagsService.new(container_repository: repo, current_user: user, params: policy.attributes.except("created_at", "updated_at")).execute
end
```
@@ -1202,7 +1202,7 @@ Before diving in to the following sections, here's some basic troubleshooting:
been synchronized (for example, via NTP).
1. If you are using an S3-backed Registry, double check that the IAM
- permissions and the S3 credentials (including region) are correct. See
+ permissions and the S3 credentials (including region) are correct. See
[the sample IAM policy](https://docs.docker.com/registry/storage-drivers/s3/)
for more details.
@@ -1631,7 +1631,7 @@ wrong. However, since all communications between Docker clients and servers
are done over HTTPS, it's a bit difficult to decrypt the traffic quickly even
if you know the private key. What can we do instead?
-One way would be to disable HTTPS by setting up an
+One way would be to disable HTTPS by setting up an
[insecure Registry](https://docs.docker.com/registry/insecure/). This could introduce a
security hole and is only recommended for local testing. If you have a
production system and can't or don't want to do this, there is another way:
diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md
index e545fb3cd1b..024fb12a51f 100644
--- a/doc/administration/pages/index.md
+++ b/doc/administration/pages/index.md
@@ -156,7 +156,7 @@ Watch the [video tutorial](https://youtu.be/dD8c7WNcc6s) for this configuration.
**Requirements:**
- [Wildcard DNS setup](#dns-configuration)
-- Wildcard TLS certificate
+- TLS certificate. Can be either Wildcard, or any other type meeting the [requirements](../../user/project/pages/custom_domains_ssl_tls_certification/index.md#manual-addition-of-ssltls-certificates).
---
@@ -191,6 +191,12 @@ to use the HTTPS protocol.
WARNING:
Multiple wildcards for one instance is not supported. Only one wildcard per instance can be assigned.
+WARNING:
+GitLab Pages does not update the OAuth application if changes are made to the redirect URI.
+Before you reconfigure, remove the `gitlab_pages` section from `/etc/gitlab/gitlab-secrets.json`,
+then run `gitlab-ctl reconfigure`. For more information, read
+[GitLab Pages does not regenerate OAuth](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/3947).
+
### Wildcard domains with TLS-terminating Load Balancer
**Requirements:**
@@ -368,7 +374,7 @@ world. Custom domains and TLS are supported.
If you don't have IPv6, you can omit the IPv6 address.
-1. If you haven't named your certificate and key `example.io.crt` and `example.io.key` respectively,
+1. If you haven't named your certificate `example.io.crt` and your key `example.io.key`,
then you need to also add the full paths as shown below:
```ruby
@@ -397,7 +403,7 @@ domain as a custom domain to their project.
If your user base is private or otherwise trusted, you can disable the
verification requirement:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Preferences**.
1. Expand **Pages**.
1. Clear the **Require users to prove ownership of custom domains** checkbox.
@@ -414,7 +420,7 @@ sites served under a custom domain.
To enable it:
1. Choose an email address on which you want to receive notifications about expiring domains.
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Preferences**.
1. Expand **Pages**.
1. Enter the email address for receiving notifications and accept Let's Encrypt's Terms of Service.
@@ -467,7 +473,7 @@ pre-existing applications must modify the GitLab Pages OAuth application. Follow
this:
1. Enable [access control](#access-control).
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Applications**.
1. Expand **GitLab Pages**.
1. Clear the `api` scope's checkbox and select the desired scope's checkbox (for example,
@@ -486,7 +492,7 @@ This can be helpful to restrict information published with Pages websites to the
of your instance only.
To do that:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Preferences**.
1. Expand **Pages**.
1. Select the **Disable public access to Pages sites** checkbox.
@@ -523,7 +529,7 @@ For Omnibus, this is fixed by [installing a custom CA in Omnibus GitLab](https:/
> [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/548) in GitLab 14.8.
-If GitLab has been [configured to require mutual TLS](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-2-way-ssl-client-authentication), you need to add the client certificates to Pages:
+If GitLab has been [configured to require mutual TLS](https://docs.gitlab.com/omnibus/settings/ssl.html#enable-2-way-ssl-client-authentication), you need to add the client certificates to Pages:
1. Configure in `/etc/gitlab/gitlab.rb`:
@@ -699,7 +705,7 @@ Prerequisites:
To set the global maximum pages size for a project:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Preferences**.
1. Expand **Pages**.
1. Enter a value under **Maximum size of pages**.
@@ -713,7 +719,7 @@ Prerequisites:
To set the maximum size of each GitLab Pages site in a group, overriding the inherited setting:
-1. On the top bar, select **Menu > Groups** and find your group.
+1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Settings > General**.
1. Expand **Pages**.
1. Enter a value under **Maximum size** in MB.
@@ -727,11 +733,24 @@ Prerequisites:
To set the maximum size of GitLab Pages site in a project, overriding the inherited setting:
-1. On the top bar, select **Menu > Projects** and find your project.
+1. On the top bar, select **Main menu > Projects** and find your project.
1. On the left sidebar, select **Settings > Pages**.
1. Enter a value under **Maximum size of pages** in MB.
1. Select **Save changes**.
+## Set maximum number of GitLab Pages custom domains for a project
+
+Prerequisite:
+
+- You must be an administrator of a self-managed GitLab instance.
+
+To set the maximum number of GitLab Pages custom domains for a project:
+
+1. On the top bar, select **Main menu > Admin**.
+1. On the left sidebar, select **Settings > Preferences**, and expand **Pages**.
+1. Enter a value for **Maximum number of custom domains per project**. Use `0` for unlimited domains.
+1. Select **Save changes**.
+
## Running GitLab Pages on a separate server
You can run the GitLab Pages daemon on a separate server to decrease the load on
@@ -933,7 +952,7 @@ The following settings are:
| `connection` | Various connection options described below. | |
NOTE:
-If you want to stop using and disconnect the NFS server, you need to
+If you want to stop using and disconnect the NFS server, you need to
[explicitly disable local storage](#disable-pages-local-storage), and it's only possible after upgrading to GitLab 13.11.
#### S3-compatible connection settings
@@ -1379,7 +1398,7 @@ Upgrading to an [officially supported operating system](https://about.gitlab.com
This problem comes from the permissions of the GitLab Pages OAuth application. To fix it:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Applications > GitLab Pages**.
1. Edit the application.
1. Under **Scopes**, ensure that the `api` scope is selected.
@@ -1502,3 +1521,12 @@ To fix that:
- Store your deployments locally, by commenting out that line.
1. Save the changes you made to your `gitlab.rb` file, then [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
+
+### 404 error `The page you're looking for could not be found`
+
+If you get a `404 Page Not Found` response from GitLab Pages:
+
+1. Check `.gitlab-ci.yml` contains the job `pages:`.
+1. Check the current project's pipeline to confirm the job `pages:deploy` is being run.
+
+Without the `pages:deploy` job, the updates to your GitLab Pages site are never published.
diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md
index 5ab508dbaca..14ac05e0293 100644
--- a/doc/administration/pages/source.md
+++ b/doc/administration/pages/source.md
@@ -364,9 +364,8 @@ world. Custom domains and TLS are supported.
```
1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in
- order to enable the pages daemon. In `gitlab_pages_options` the
- `-pages-domain`, `-listen-http` and `-listen-https` must match the `host`,
- `external_http` and `external_https` settings that you set above respectively.
+ order to enable the pages daemon. In `gitlab_pages_options`, you must match the
+ `-pages-domain` with `host`, `-listen-http` with `external_http`, and `-listen-https` with `external_https` settings.
The `-root-cert` and `-root-key` settings are the wildcard TLS certificates
of the `example.io` domain:
@@ -486,7 +485,7 @@ The default for the maximum size of unpacked archives per project is 100 MB.
To change this value:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Preferences**.
1. Expand **Pages**.
1. Update the value for **Maximum size of pages (MB)**.
diff --git a/doc/administration/polling.md b/doc/administration/polling.md
index 5699bf78c04..7f1e7a047cf 100644
--- a/doc/administration/polling.md
+++ b/doc/administration/polling.md
@@ -26,7 +26,7 @@ The default value (`1`) is recommended for the majority of GitLab installations.
To adjust the polling interval multiplier:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Preferences**.
1. Expand **Polling interval multiplier**.
1. Set a value for the polling interval multiplier. This multiplier is applied to all resources at
diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md
index a900da52aba..c4401b49180 100644
--- a/doc/administration/raketasks/maintenance.md
+++ b/doc/administration/raketasks/maintenance.md
@@ -326,14 +326,20 @@ database: gitlabhq_production
Database migrations can be stuck in an incomplete state, with a `down`
status in the output of the `sudo gitlab-rake db:migrate:status` command.
-To complete these migrations, use the following Rake task:
+1. To complete these migrations, use the following Rake task:
-```shell
-sudo gitlab-rake db:migrate
-```
+ ```shell
+ sudo gitlab-rake db:migrate
+ ```
+
+1. After the command completes, run `sudo gitlab-rake db:migrate:status` to check if all migrations are completed (have an `up` status).
+
+1. Hot reload `puma` and `sidekiq` services:
-After the command completes, run `sudo gitlab-rake db:migrate:status` to check if all
-migrations are completed (have an `up` status).
+ ```shell
+ sudo gitlab-ctl hup puma
+ sudo gitlab-ctl restart sidekiq
+ ```
## Rebuild database indexes
diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md
index f3a9845d129..00bd71af6c5 100644
--- a/doc/administration/raketasks/project_import_export.md
+++ b/doc/administration/raketasks/project_import_export.md
@@ -45,7 +45,7 @@ Note the following:
compatible as described in the [Version history](../../user/project/settings/import_export.md#version-history).
- The project import option must be enabled:
- 1. On the top bar, select **Menu > Admin**.
+ 1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > General**.
1. Expand **Visibility and access controls**.
1. Under **Import sources**, check the "Project export enabled" option.
diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md
index 27b899dd1b1..fc0ff23c5b1 100644
--- a/doc/administration/raketasks/storage.md
+++ b/doc/administration/raketasks/storage.md
@@ -109,7 +109,7 @@ sudo gitlab-rake gitlab:storage:migrate_to_hashed ID_FROM=50 ID_TO=100
To monitor the progress in GitLab:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
1. Watch how long the `hashed_storage:hashed_storage_project_migrate` queue
takes to finish. After it reaches zero, you can confirm every project
diff --git a/doc/administration/raketasks/uploads/migrate.md b/doc/administration/raketasks/uploads/migrate.md
index c73840cb9ff..216c0875645 100644
--- a/doc/administration/raketasks/uploads/migrate.md
+++ b/doc/administration/raketasks/uploads/migrate.md
@@ -79,7 +79,8 @@ The Rake task uses three parameters to find uploads to migrate:
NOTE:
These parameters are mainly internal to the structure of GitLab, you may want to refer to the task list
-instead below.
+instead below. After running these individual tasks, we recommend that you run the [all-in-one Rake task](#all-in-one-rake-task)
+to migrate any uploads not included in the listed types.
This task also accepts an environment variable which you can use to override
the default batch size:
diff --git a/doc/administration/redis/replication_and_failover.md b/doc/administration/redis/replication_and_failover.md
index c4b83b66738..b775b579fd4 100644
--- a/doc/administration/redis/replication_and_failover.md
+++ b/doc/administration/redis/replication_and_failover.md
@@ -343,7 +343,7 @@ NOTE:
If you are using an external Redis Sentinel instance, be sure
to exclude the `requirepass` parameter from the Sentinel
configuration. This parameter causes clients to report `NOAUTH
-Authentication required.`.
+Authentication required.`.
[Redis Sentinel 3.2.x does not support password authentication](https://github.com/antirez/redis/issues/3279).
Now that the Redis servers are all set up, let's configure the Sentinel
diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md
index fcc32100bc8..5d676dac000 100644
--- a/doc/administration/reference_architectures/10k_users.md
+++ b/doc/administration/reference_architectures/10k_users.md
@@ -18,30 +18,30 @@ full list of reference architectures, see
> - **Test requests per second (RPS) rates:** API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS
> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)**
-| Service | Nodes | Configuration | GCP | AWS | Azure |
-|-----------------------------------------------------|----------------|-------------------------|------------------|----------------|----------------|
-| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| PostgreSQL<sup>1</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | `D8s v3` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | `D16s v3` |
-| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` |
-| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable |
-| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Service | Nodes | Configuration | GCP | AWS | Azure |
+|------------------------------------------|-------|-------------------------|------------------|----------------|-----------|
+| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| PostgreSQL<sup>1</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | `D8s v3` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | `D16s v3` |
+| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` |
+| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Object storage<sup>4</sup> | - | - | - | - | - |
+| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -142,10 +142,12 @@ Before starting, you should take note of the following requirements / guidance f
This reference architecture was built and tested on Google Cloud Platform (GCP) using the
[Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms)
-CPU platform. On different hardware you may find that adjustments, either lower
-or higher, are required for your CPU or node counts. For more information, see
-our [Sysbench](https://github.com/akopytov/sysbench)-based
-[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks).
+CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)).
+
+Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported.
+
+NOTE:
+Any "burstable" instance types are not recommended due to inconsistent performance.
### Supported infrastructure
@@ -186,10 +188,11 @@ To set up GitLab and its components to accommodate up to 10,000 users:
environment.
1. [Configure the object storage](#configure-the-object-storage)
used for shared data objects.
+1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended)
+ to have shared disk storage service as an alternative to Gitaly or object
+ storage.
1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster,
more advanced code search across your entire GitLab instance.
-1. [Configure NFS](#configure-nfs)
- to have shared disk storage service for certain GitLab operations (non Gitaly or Object Storage).
The servers start on the same 10.6.0.0/24 private network range, and can
connect to each other freely on these addresses.
@@ -259,7 +262,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather
than `HTTP(S)` protocol. This will pass the connection to the application node's
NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
-See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Load balancer terminates SSL without backend SSL
@@ -270,7 +273,7 @@ terminating SSL.
Since communication between the load balancer and GitLab will not be secure,
there is some additional configuration needed. See the
-[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl)
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
for details.
### Load balancer terminates SSL with backend SSL
@@ -283,7 +286,7 @@ Traffic will also be secure between the load balancers and NGINX in this
scenario. There is no need to add configuration for proxied SSL since the
connection will be secure all the way. However, configuration will need to be
added to GitLab to configure SSL certificates. See
-[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Readiness checks
@@ -2009,7 +2012,7 @@ On each node perform the following:
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
-1. If you're [using NFS](#configure-nfs):
+1. If you're [using NFS](#configure-nfs-optional):
1. If necessary, install the NFS client utility packages using the following
commands:
@@ -2069,7 +2072,7 @@ On each node perform the following:
When you specify `https` in the `external_url`, as in the previous example,
GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the
certificates aren't present, NGINX will fail to start. For more information, see
-the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https).
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html).
### GitLab Rails post-configuration
@@ -2175,7 +2178,7 @@ To configure the Monitoring node:
## Configure the object storage
GitLab supports using an object storage service for holding numerous types of data.
-It's recommended over [NFS](#configure-nfs) and in general it's better
+It's recommended over [NFS](#configure-nfs-optional) and in general it's better
in larger setups as object storage is typically much more performant, reliable,
and scalable.
@@ -2187,7 +2190,6 @@ GitLab has been tested on a number of object storage providers:
- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction)
-- On-premises hardware and appliances from various storage vendors.
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
There are two ways of specifying object storage configuration in GitLab:
@@ -2197,25 +2199,13 @@ There are two ways of specifying object storage configuration in GitLab:
- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its
own object storage [connection and configuration](../object_storage.md#connection-settings).
-Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up.
-
-For configuring object storage in GitLab 13.1 and earlier, or for storage types not
-supported by consolidated configuration form, refer to the following guides based
-on what features you intend to use:
-
-|Object storage type|Supported by consolidated configuration?|
-|-------------------|----------------------------------------|
-| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No |
-| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes |
-| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes |
-| [Uploads](../uploads.md#using-object-storage) | Yes |
-| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No |
-| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes |
-| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No |
-| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes |
-| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes |
-| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No |
-| [Terraform state files](../terraform_state.md#using-object-storage) | Yes |
+The consolidated form is used in the following examples when available.
+
+NOTE:
+When using the [storage-specific form](../object_storage.md#storage-specific-configuration)
+in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload).
+The previous [background upload](../../development/uploads/index.md#direct-upload) mode,
+which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional).
Using separate buckets for each data type is the recommended approach for GitLab.
This ensures there are no collisions across the various types of data GitLab stores.
@@ -2228,29 +2218,13 @@ in the future.
</a>
</div>
-## Enable incremental logging
+### Enable incremental logging
GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes.
While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs.
-## Configure Advanced Search
-
-You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md)
-for faster, more advanced code search across your entire GitLab instance.
-
-Elasticsearch cluster design and requirements are dependent on your specific
-data. For recommended best practices about how to set up your Elasticsearch
-cluster alongside your instance, read how to
-[choose the optimal cluster configuration](../../integration/advanced_search/elasticsearch.md#guidance-on-choosing-optimal-cluster-configuration).
-
-<div align="right">
- <a type="button" class="btn btn-default" href="#setup-components">
- Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
- </a>
-</div>
-
-## Configure NFS
+## Configure NFS (optional)
[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly)
are recommended over NFS wherever possible for improved performance.
@@ -2258,14 +2232,24 @@ are recommended over NFS wherever possible for improved performance.
See how to [configure NFS](../nfs.md).
WARNING:
-Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
-unavailable from GitLab 15.0. No further enhancements are planned for this feature.
+Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation)
+after the release of GitLab 15.6. No further enhancements are planned for this feature.
Read:
- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
+## Configure Advanced Search
+
+You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md)
+for faster, more advanced code search across your entire GitLab instance.
+
+Elasticsearch cluster design and requirements are dependent on your specific
+data. For recommended best practices about how to set up your Elasticsearch
+cluster alongside your instance, read how to
+[choose the optimal cluster configuration](../../integration/advanced_search/elasticsearch.md#guidance-on-choosing-optimal-cluster-configuration).
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
@@ -2292,6 +2276,10 @@ to be complex. **This setup is only recommended** if you have strong working
knowledge and experience in Kubernetes. The rest of this
section assumes this.
+NOTE:
+**Gitaly Cluster is not supported to be run in Kubernetes**.
+Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details.
+
### Cluster topology
The following tables and diagram detail the hybrid environment using the same formats
@@ -2302,11 +2290,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK
and CPU requirements should translate to most other providers. We hope to update this in the
future with further specific cloud provider details.
-| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
-|-----------------------------------------------|-------|-------------------------|-----------------|--------------|---------------------------------|
-| Webservice | 4 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | 127.5 vCPU, 118 GB memory |
-| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory |
-| Supporting services such as NGINX, Prometheus | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory |
+| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
+|---------------------|-------|-------------------------|-----------------|--------------|---------------------------------|
+| Webservice | 4 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | 127.5 vCPU, 118 GB memory |
+| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory |
+| Supporting services | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory |
- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results)
[Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary.
@@ -2316,25 +2304,25 @@ future with further specific cloud provider details.
Next are the backend components that run on static compute VMs via Omnibus (or External PaaS
services where applicable):
-| Service | Nodes | Configuration | GCP | AWS |
-|------------------------------------------|----------------|-----------------------|------------------|----------------|
-| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| PostgreSQL<sup>1</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` |
-| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable |
+| Service | Nodes | Configuration | GCP | AWS |
+|------------------------------------------|-------|-----------------------|------------------|--------------|
+| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| PostgreSQL<sup>1</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` |
+| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Object storage<sup>4</sup> | - | - | - | - |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -2353,7 +2341,7 @@ card "Kubernetes via Helm Charts" as kubernetes {
collections "**Sidekiq** x4" as sidekiq #ff8dd1
}
- card "**Supporting Services**" as support
+ card "**Supporting Services** x2" as support
}
card "**Internal Load Balancer**" as ilb #9370DB
@@ -2419,29 +2407,42 @@ documents how to apply the calculated configuration to the Helm Chart.
#### Webservice
-Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_.
-Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using
+Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_.
+Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using
the [recommended topology](#cluster-topology) because four worker processes
are created by default and each pod has other small processes running.
For 10,000 users we recommend a total Puma worker count of around 80.
With the [provided recommendations](#cluster-topology) this allows the deployment of up to 20
Webservice pods with 4 workers per pod and 5 pods per node. Expand available resources using
-the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional
+the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional
Webservice pod.
For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources).
#### Sidekiq
-Sidekiq pods should generally have 1 vCPU and 2 GB of memory.
+Sidekiq pods should generally have 0.9 CPU and 2 GB of memory.
[The provided starting point](#cluster-topology) allows the deployment of up to
-14 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory
+14 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory
ratio for each additional pod.
For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources).
+### Supporting
+
+The Supporting Node Pool is designed to house all supporting deployments that don't need to be
+on the Webservice and Sidekiq pools.
+
+This includes various deployments related to the Cloud Provider's implementation and supporting
+GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/).
+
+If you wish to make any additional deployments, such as for Monitoring, it's recommended
+to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed
+specifically to accommodate several additional deployments. However, if your deployments don't fit into the
+pool as given, you can increase the node pool accordingly.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md
index 1a774603863..96b1e541f92 100644
--- a/doc/administration/reference_architectures/1k_users.md
+++ b/doc/administration/reference_architectures/1k_users.md
@@ -72,10 +72,12 @@ Before starting, you should take note of the following requirements / guidance f
This reference architecture was built and tested on Google Cloud Platform (GCP) using the
[Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms)
-CPU platform. On different hardware you may find that adjustments, either lower
-or higher, are required for your CPU or node counts. For more information, see
-our [Sysbench](https://github.com/akopytov/sysbench)-based
-[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks).
+CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)).
+
+Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported.
+
+NOTE:
+Any "burstable" instance types are not recommended due to inconsistent performance.
### Supported infrastructure
diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md
index 51d3e3d39a6..423dbc7abfb 100644
--- a/doc/administration/reference_architectures/25k_users.md
+++ b/doc/administration/reference_architectures/25k_users.md
@@ -18,30 +18,30 @@ full list of reference architectures, see
> - **Test requests per second (RPS) rates:** API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS
> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)**
-| Service | Nodes | Configuration | GCP | AWS | Azure |
-|---------------------------------------------------|----------------|-------------------------|------------------|----------------|----------------|
-| External load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
-| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| PostgreSQL<sup>1</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | `D16s v3` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
-| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | `D32s v3` |
-| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| GitLab Rails | 5 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` |
-| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable |
-| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Service | Nodes | Configuration | GCP | AWS | Azure |
+|------------------------------------------|-------|-------------------------|------------------|--------------|-----------|
+| External load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| PostgreSQL<sup>1</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` | `D16s v3` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | `D32s v3` |
+| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| GitLab Rails | 5 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` |
+| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Object storage<sup>4</sup> | - | - | - | - | - |
+| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -142,10 +142,12 @@ Before starting, you should take note of the following requirements / guidance f
This reference architecture was built and tested on Google Cloud Platform (GCP) using the
[Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms)
-CPU platform. On different hardware you may find that adjustments, either lower
-or higher, are required for your CPU or node counts. For more information, see
-our [Sysbench](https://github.com/akopytov/sysbench)-based
-[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks).
+CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)).
+
+Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported.
+
+NOTE:
+Any "burstable" instance types are not recommended due to inconsistent performance.
### Supported infrastructure
@@ -186,11 +188,11 @@ To set up GitLab and its components to accommodate up to 25,000 users:
environment.
1. [Configure the object storage](#configure-the-object-storage)
used for shared data objects.
+1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended)
+ to have shared disk storage service as an alternative to Gitaly or object
+ storage.
1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster,
more advanced code search across your entire GitLab instance.
-1. [Configure NFS](#configure-nfs)
- to have shared disk storage service for certain GitLab operations (non
- Gitaly or Object Storage).
The servers start on the same 10.6.0.0/24 private network range, and can
connect to each other freely on these addresses.
@@ -262,7 +264,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather
than `HTTP(S)` protocol. This will pass the connection to the application node's
NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
-See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Load balancer terminates SSL without backend SSL
@@ -273,7 +275,7 @@ terminating SSL.
Since communication between the load balancer and GitLab will not be secure,
there is some additional configuration needed. See the
-[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl)
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
for details.
### Load balancer terminates SSL with backend SSL
@@ -285,8 +287,8 @@ end users will see.
Traffic will also be secure between the load balancers and NGINX in this
scenario. There is no need to add configuration for proxied SSL since the
connection will be secure all the way. However, configuration will need to be
-added to GitLab to configure SSL certificates. See
-[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+added to GitLab to configure SSL certificates. See the
+[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Readiness checks
@@ -2015,7 +2017,7 @@ On each node perform the following:
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
-1. If you're [using NFS](#configure-nfs):
+1. If you're [using NFS](#configure-nfs-optional):
1. If necessary, install the NFS client utility packages using the following
commands:
@@ -2074,7 +2076,7 @@ On each node perform the following:
When you specify `https` in the `external_url`, as in the previous example,
GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the
certificates aren't present, NGINX will fail to start. For more information, see
-the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https).
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html).
### GitLab Rails post-configuration
@@ -2179,9 +2181,9 @@ To configure the Monitoring node:
## Configure the object storage
GitLab supports using an object storage service for holding numerous types of data.
-Object storage is also recommended over [NFS](#configure-nfs) and in general
-it's better in larger setups as object storage is typically much more performant,
-reliable, and scalable.
+It's recommended over [NFS](#configure-nfs-optional) and in general it's better
+in larger setups as object storage is typically much more performant, reliable,
+and scalable.
GitLab has been tested on a number of object storage providers:
@@ -2191,7 +2193,6 @@ GitLab has been tested on a number of object storage providers:
- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction)
-- On-premises hardware and appliances from various storage vendors.
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
There are two ways of specifying object storage configuration in GitLab:
@@ -2201,25 +2202,13 @@ There are two ways of specifying object storage configuration in GitLab:
- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its
own object storage [connection and configuration](../object_storage.md#connection-settings).
-Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up.
-
-For configuring object storage in GitLab 13.1 and earlier, or for storage types not
-supported by consolidated configuration form, refer to the following guides based
-on what features you intend to use:
-
-|Object storage type|Supported by consolidated configuration?|
-|-------------------|----------------------------------------|
-| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No |
-| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes |
-| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes |
-| [Uploads](../uploads.md#using-object-storage) | Yes |
-| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No |
-| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes |
-| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No |
-| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes |
-| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes |
-| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No |
-| [Terraform state files](../terraform_state.md#using-object-storage) | Yes |
+The consolidated form is used in the following examples when available.
+
+NOTE:
+When using the [storage-specific form](../object_storage.md#storage-specific-configuration)
+in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload).
+The previous [background upload](../../development/uploads/index.md#direct-upload) mode,
+which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional).
Using separate buckets for each data type is the recommended approach for GitLab.
This ensures there are no collisions across the various types of data GitLab stores.
@@ -2232,12 +2221,28 @@ in the future.
</a>
</div>
-## Enable incremental logging
+### Enable incremental logging
GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes.
While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs.
+## Configure NFS (optional)
+
+[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly)
+are recommended over NFS wherever possible for improved performance.
+
+See how to [configure NFS](../nfs.md).
+
+WARNING:
+Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation)
+after the release of GitLab 15.6. No further enhancements are planned for this feature.
+
+Read:
+
+- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
+- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
+
## Configure Advanced Search
You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md)
@@ -2254,22 +2259,6 @@ cluster alongside your instance, read how to
</a>
</div>
-## Configure NFS
-
-[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly)
-are recommended over NFS wherever possible for improved performance.
-
-See how to [configure NFS](../nfs.md).
-
-WARNING:
-Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
-unavailable from GitLab 15.0. No further enhancements are planned for this feature.
-
-Read:
-
-- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
-- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
-
## Cloud Native Hybrid reference architecture with Helm Charts (alternative)
As an alternative approach, you can also run select components of GitLab as Cloud Native
@@ -2290,6 +2279,10 @@ to be complex. **This setup is only recommended** if you have strong working
knowledge and experience in Kubernetes. The rest of this
section assumes this.
+NOTE:
+**Gitaly Cluster is not supported to be run in Kubernetes**.
+Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details.
+
### Cluster topology
The following tables and diagram detail the hybrid environment using the same formats
@@ -2300,11 +2293,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK
and CPU requirements should translate to most other providers. We hope to update this in the
future with further specific cloud provider details.
-| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
-|-----------------------------------------------|-------|-------------------------|-----------------|-----------------|---------------------------------|
-| Webservice | 7 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | 223 vCPU, 206.5 GB memory |
-| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory |
-| Supporting services such as NGINX, Prometheus | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory |
+| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
+|---------------------|-------|-------------------------|-----------------|--------------|---------------------------------|
+| Webservice | 7 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | 223 vCPU, 206.5 GB memory |
+| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory |
+| Supporting services | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory |
- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results)
[Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary.
@@ -2314,25 +2307,25 @@ future with further specific cloud provider details.
Next are the backend components that run on static compute VMs via Omnibus (or External PaaS
services where applicable):
-| Service | Nodes | Configuration | GCP | AWS |
-|------------------------------------------|----------------|------------------------|------------------|----------------|
-| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| PostgreSQL<sup>1</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6GB memory | `n1-highcpu-4` | `c5.xlarge` |
-| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` |
-| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable |
+| Service | Nodes | Configuration | GCP | AWS |
+|------------------------------------------|-------|------------------------|------------------|--------------|
+| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| PostgreSQL<sup>1</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6GB memory | `n1-highcpu-4` | `c5.xlarge` |
+| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` |
+| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Object storage<sup>4</sup> | - | - | - | - |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -2347,11 +2340,11 @@ card "Kubernetes via Helm Charts" as kubernetes {
card "**External Load Balancer**" as elb #6a9be7
together {
- collections "**Webservice** x4" as gitlab #32CD32
+ collections "**Webservice** x7" as gitlab #32CD32
collections "**Sidekiq** x4" as sidekiq #ff8dd1
}
- card "**Supporting Services**" as support
+ card "**Supporting Services** x2" as support
}
card "**Internal Load Balancer**" as ilb #9370DB
@@ -2417,29 +2410,42 @@ documents how to apply the calculated configuration to the Helm Chart.
#### Webservice
-Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_.
-Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using
+Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_.
+Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using
the [recommended topology](#cluster-topology) because four worker processes
are created by default and each pod has other small processes running.
For 25,000 users we recommend a total Puma worker count of around 140.
With the [provided recommendations](#cluster-topology) this allows the deployment of up to 35
Webservice pods with 4 workers per pod and 5 pods per node. Expand available resources using
-the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional
+the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional
Webservice pod.
For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources).
#### Sidekiq
-Sidekiq pods should generally have 1 vCPU and 2 GB of memory.
+Sidekiq pods should generally have 0.9 CPU and 2 GB of memory.
[The provided starting point](#cluster-topology) allows the deployment of up to
-14 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory
+14 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory
ratio for each additional pod.
For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources).
+### Supporting
+
+The Supporting Node Pool is designed to house all supporting deployments that don't need to be
+on the Webservice and Sidekiq pools.
+
+This includes various deployments related to the Cloud Provider's implementation and supporting
+GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/).
+
+If you wish to make any additional deployments, such as for Monitoring, it's recommended
+to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed
+specifically to accommodate several additional deployments. However, if your deployments don't fit into the
+pool as given, you can increase the node pool accordingly.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md
index 8ca58da55ca..99cc6d47f6a 100644
--- a/doc/administration/reference_architectures/2k_users.md
+++ b/doc/administration/reference_architectures/2k_users.md
@@ -19,22 +19,22 @@ For a full list of reference architectures, see
> - **Test requests per second (RPS) rates:** API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS
> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)**
-| Service | Nodes | Configuration | GCP | AWS | Azure |
-|------------------------------------------|----------------|-------------------------|-----------------|----------------|----------------|
-| Load balancer<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
-| Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` | `D2s v3` |
-| Gitaly | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| GitLab Rails | 2 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` |
-| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable |
-| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Service | Nodes | Configuration | GCP | AWS | Azure |
+|----------------------------|-------|------------------------|-----------------|--------------|----------|
+| Load balancer<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
+| Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` | `D2s v3` |
+| Gitaly | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| GitLab Rails | 2 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` |
+| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Object storage<sup>4</sup> | - | - | - | - | - |
+| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run as reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run as reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
<!-- markdownlint-enable MD029 -->
NOTE:
@@ -78,10 +78,12 @@ Before starting, you should take note of the following requirements / guidance f
This reference architecture was built and tested on Google Cloud Platform (GCP) using the
[Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms)
-CPU platform. On different hardware you may find that adjustments, either lower
-or higher, are required for your CPU or node counts. For more information, see
-our [Sysbench](https://github.com/akopytov/sysbench)-based
-[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks).
+CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)).
+
+Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported.
+
+NOTE:
+Any "burstable" instance types are not recommended due to inconsistent performance.
### Supported infrastructure
@@ -110,11 +112,11 @@ To set up GitLab and its components to accommodate up to 2,000 users:
environment.
1. [Configure the object storage](#configure-the-object-storage) used for
shared data objects.
-1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster,
- more advanced code search across your entire GitLab instance.
1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended)
to have shared disk storage service as an alternative to Gitaly or object
storage.
+1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster,
+ more advanced code search across your entire GitLab instance.
## Configure the external load balancer
@@ -147,7 +149,7 @@ of `HTTP(S)`. This will pass the connection unaltered to the application node's
NGINX service, which has the SSL certificate and listens to port 443.
For details about managing SSL certificates and configuring NGINX, see the
-[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https).
+[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
### Load balancer terminates SSL without backend SSL
@@ -157,7 +159,7 @@ terminating SSL.
Due to communication between the load balancer and GitLab not being secure,
you'll need to complete some additional configuration. For details, see the
-[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl).
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination).
### Load balancer terminates SSL with backend SSL
@@ -169,7 +171,7 @@ Traffic will be secure between the load balancers and NGINX in this scenario,
and there's no need to add a configuration for proxied SSL. However, you'll
need to add a configuration to GitLab to configure SSL certificates. For
details about managing SSL certificates and configuring NGINX, see the
-[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https).
+[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html).
### Readiness checks
@@ -750,7 +752,7 @@ On each node perform the following:
When you specify `https` in the `external_url`, as in the previous example,
GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the
certificates aren't present, NGINX will fail to start. For more information, see
-the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https).
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html).
### GitLab Rails post-configuration
@@ -885,10 +887,10 @@ running [Prometheus](../monitoring/prometheus/index.md) and
## Configure the object storage
-GitLab supports using an object storage service for holding several types of
-data, and is recommended over [NFS](#configure-nfs-optional). In general,
-object storage services are better for larger environments, as object storage
-is typically much more performant, reliable, and scalable.
+GitLab supports using an object storage service for holding numerous types of data.
+It's recommended over [NFS](#configure-nfs-optional) and in general it's better
+in larger setups as object storage is typically much more performant, reliable,
+and scalable.
GitLab has been tested on a number of object storage providers:
@@ -898,7 +900,6 @@ GitLab has been tested on a number of object storage providers:
- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction)
-- On-premises hardware and appliances from various storage vendors.
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
There are two ways of specifying object storage configuration in GitLab:
@@ -908,29 +909,13 @@ There are two ways of specifying object storage configuration in GitLab:
- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its
own object storage [connection and configuration](../object_storage.md#connection-settings).
-Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up.
-
-GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared via NFS on any GitLab Rails and Sidekiq nodes.
-
-In GitLab 13.6 and later, it's also recommended to switch to [Incremental logging](../job_logs.md#incremental-logging-architecture), which uses Redis instead of disk space for temporary caching of job logs. This is required when no NFS node has been deployed.
+The consolidated form is used in the following examples when available.
-For configuring object storage in GitLab 13.1 and earlier, or for storage types not
-supported by consolidated configuration form, refer to the following guides based
-on what features you intend to use:
-
-|Object storage type|Supported by consolidated configuration?|
-|-------------------|----------------------------------------|
-| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No |
-| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes |
-| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes |
-| [Uploads](../uploads.md#using-object-storage) | Yes |
-| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No |
-| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes |
-| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No |
-| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes |
-| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes |
-| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No |
-| [Terraform state files](../terraform_state.md#using-object-storage) | Yes |
+NOTE:
+When using the [storage-specific form](../object_storage.md#storage-specific-configuration)
+in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload).
+The previous [background upload](../../development/uploads/index.md#direct-upload) mode,
+which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional).
Using separate buckets for each data type is the recommended approach for GitLab.
This ensures there are no collisions across the various types of data GitLab stores.
@@ -943,21 +928,11 @@ in the future.
</a>
</div>
-## Configure Advanced Search **(PREMIUM SELF)**
-
-You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md)
-for faster, more advanced code search across your entire GitLab instance.
+### Enable incremental logging
-Elasticsearch cluster design and requirements are dependent on your specific
-data. For recommended best practices about how to set up your Elasticsearch
-cluster alongside your instance, read how to
-[choose the optimal cluster configuration](../../integration/advanced_search/elasticsearch.md#guidance-on-choosing-optimal-cluster-configuration).
+GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes.
-<div align="right">
- <a type="button" class="btn btn-default" href="#setup-components">
- Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
- </a>
-</div>
+While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs.
## Configure NFS (optional)
@@ -968,14 +943,30 @@ possible.
See how to [configure NFS](../nfs.md).
WARNING:
-Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
-unavailable from GitLab 15.0. No further enhancements are planned for this feature.
+Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation)
+after the release of GitLab 15.6. No further enhancements are planned for this feature.
Read:
- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
+## Configure Advanced Search **(PREMIUM SELF)**
+
+You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md)
+for faster, more advanced code search across your entire GitLab instance.
+
+Elasticsearch cluster design and requirements are dependent on your specific
+data. For recommended best practices about how to set up your Elasticsearch
+cluster alongside your instance, read how to
+[choose the optimal cluster configuration](../../integration/advanced_search/elasticsearch.md#guidance-on-choosing-optimal-cluster-configuration).
+
+<div align="right">
+ <a type="button" class="btn btn-default" href="#setup-components">
+ Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
+ </a>
+</div>
+
## Cloud Native Hybrid reference architecture with Helm Charts (alternative)
As an alternative approach, you can also run select components of GitLab as Cloud Native
@@ -998,6 +989,10 @@ to be complex. **This setup is only recommended** if you have strong working
knowledge and experience in Kubernetes. The rest of this
section assumes this.
+NOTE:
+**Gitaly Cluster is not supported to be run in Kubernetes**.
+Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details.
+
### Cluster topology
The following tables and diagram detail the hybrid environment using the same formats
@@ -1008,11 +1003,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK
and CPU requirements should translate to most other providers. We hope to update this in the
future with further specific cloud provider details.
-| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
-|-----------------------------------------------|-------|------------------------|-----------------|--------------|---------------------------------|
-| Webservice | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | 23.7 vCPU, 16.9 GB memory |
-| Sidekiq | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 3.9 vCPU, 11.8 GB memory |
-| Supporting services such as NGINX, Prometheus | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | 1.9 vCPU, 5.5 GB memory |
+| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
+|---------------------|-------|------------------------|-----------------|--------------|---------------------------------|
+| Webservice | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | 23.7 vCPU, 16.9 GB memory |
+| Sidekiq | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.8 vCPU, 25.9 GB memory |
+| Supporting services | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | 1.9 vCPU, 5.5 GB memory |
- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results)
[Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary.
@@ -1022,18 +1017,18 @@ future with further specific cloud provider details.
Next are the backend components that run on static compute VMs via Omnibus (or External PaaS
services where applicable):
-| Service | Nodes | Configuration | GCP | AWS |
-|----------------------------|----------------|------------------------|-----------------|----------------|
-| PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
-| Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` |
-| Gitaly | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Object storage<sup>3</sup> | Not applicable | Not applicable | Not applicable | Not applicable |
+| Service | Nodes | Configuration | GCP | AWS |
+|----------------------------|-------|------------------------|-----------------|-------------|
+| PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
+| Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` |
+| Gitaly | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Object storage<sup>3</sup> | - | - | - | - |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
-3. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+3. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
<!-- markdownlint-enable MD029 -->
NOTE:
@@ -1048,9 +1043,10 @@ card "Kubernetes via Helm Charts" as kubernetes {
together {
collections "**Webservice** x3" as gitlab #32CD32
- card "**Sidekiq**" as sidekiq #ff8dd1
- collections "**Supporting Services** x2" as support
+ collections "**Sidekiq** x2" as sidekiq #ff8dd1
}
+
+ collections "**Supporting Services** x2" as support
}
card "**Gitaly**" as gitaly #FF8C00
@@ -1081,29 +1077,42 @@ documents how to apply the calculated configuration to the Helm Chart.
#### Webservice
-Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_.
-Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using
+Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_.
+Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using
the [recommended topology](#cluster-topology) because two worker processes
are created by default and each pod has other small processes running.
For 2,000 users we recommend a total Puma worker count of around 12.
With the [provided recommendations](#cluster-topology) this allows the deployment of up to 3
Webservice pods with 4 workers per pod and 1 pod per node. Expand available resources using
-the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional
+the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional
Webservice pod.
For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources).
#### Sidekiq
-Sidekiq pods should generally have 1 vCPU and 2 GB of memory.
+Sidekiq pods should generally have 0.9 CPU and 2 GB of memory.
[The provided starting point](#cluster-topology) allows the deployment of up to
-4 Sidekiq pods. Expand available resources using the 1 vCPU to 2 GB memory
+4 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory
ratio for each additional pod.
For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources).
+### Supporting
+
+The Supporting Node Pool is designed to house all supporting deployments that don't need to be
+on the Webservice and Sidekiq pools.
+
+This includes various deployments related to the Cloud Provider's implementation and supporting
+GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/).
+
+If you wish to make any additional deployments, such as for Monitoring, it's recommended
+to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed
+specifically to accommodate several additional deployments. However, if your deployments don't fit into the
+pool as given, you can increase the node pool accordingly.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md
index db81ae7ae3c..5c227e3dc27 100644
--- a/doc/administration/reference_architectures/3k_users.md
+++ b/doc/administration/reference_architectures/3k_users.md
@@ -28,29 +28,29 @@ For a full list of reference architectures, see
> - **Test requests per second (RPS) rates:** API: 60 RPS, Web: 6 RPS, Git (Pull): 6 RPS, Git (Push): 1 RPS
> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)**
-| Service | Nodes | Configuration | GCP | AWS | Azure |
-|--------------------------------------------|----------------|-----------------------|-----------------|----------------|----------------|
-| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
-| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
-| GitLab Rails | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` |
-| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable |
-| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Service | Nodes | Configuration | GCP | AWS | Azure |
+|-------------------------------------------|-------|-----------------------|-----------------|--------------|----------|
+| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
+| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
+| GitLab Rails | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` |
+| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Object storage<sup>4</sup> | - | - | - | - | - |
+| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -148,10 +148,12 @@ Before starting, you should take note of the following requirements / guidance f
This reference architecture was built and tested on Google Cloud Platform (GCP) using the
[Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms)
-CPU platform. On different hardware you may find that adjustments, either lower
-or higher, are required for your CPU or node counts. For more information, see
-our [Sysbench](https://github.com/akopytov/sysbench)-based
-[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks).
+CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)).
+
+Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported.
+
+NOTE:
+Any "burstable" instance types are not recommended due to inconsistent performance.
### Supported infrastructure
@@ -192,11 +194,11 @@ To set up GitLab and its components to accommodate up to 3,000 users:
environment.
1. [Configure the object storage](#configure-the-object-storage)
used for shared data objects.
-1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster,
- more advanced code search across your entire GitLab instance.
1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended)
to have shared disk storage service as an alternative to Gitaly or object
storage.
+1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster,
+ more advanced code search across your entire GitLab instance.
The servers start on the same 10.6.0.0/24 private network range, and can
connect to each other freely on these addresses.
@@ -263,7 +265,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather
than `HTTP(S)` protocol. This will pass the connection to the application node's
NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
-See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Load balancer terminates SSL without backend SSL
@@ -274,7 +276,7 @@ terminating SSL.
Since communication between the load balancer and GitLab will not be secure,
there is some additional configuration needed. See the
-[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl)
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
for details.
### Load balancer terminates SSL with backend SSL
@@ -286,8 +288,8 @@ end users will see.
Traffic will also be secure between the load balancers and NGINX in this
scenario. There is no need to add configuration for proxied SSL since the
connection will be secure all the way. However, configuration will need to be
-added to GitLab to configure SSL certificates. See
-[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+added to GitLab to configure SSL certificates. See the
+[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Readiness checks
@@ -2005,7 +2007,7 @@ On each node perform the following:
When you specify `https` in the `external_url`, as in the previous example,
GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the
certificates aren't present, NGINX will fail to start. For more information, see
-the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https).
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html).
### GitLab Rails post-configuration
@@ -2128,7 +2130,6 @@ GitLab has been tested on a number of object storage providers:
- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction)
-- On-premises hardware and appliances from various storage vendors.
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
There are two ways of specifying object storage configuration in GitLab:
@@ -2138,25 +2139,13 @@ There are two ways of specifying object storage configuration in GitLab:
- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its
own object storage [connection and configuration](../object_storage.md#connection-settings).
-Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up.
-
-For configuring object storage in GitLab 13.1 and earlier, or for storage types not
-supported by consolidated configuration form, refer to the following guides based
-on what features you intend to use:
-
-|Object storage type|Supported by consolidated configuration?|
-|-------------------|----------------------------------------|
-| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No |
-| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes |
-| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes |
-| [Uploads](../uploads.md#using-object-storage) | Yes |
-| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No |
-| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes |
-| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No |
-| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes |
-| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes |
-| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No |
-| [Terraform state files](../terraform_state.md#using-object-storage) | Yes |
+The consolidated form is used in the following examples when available.
+
+NOTE:
+When using the [storage-specific form](../object_storage.md#storage-specific-configuration)
+in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload).
+The previous [background upload](../../development/uploads/index.md#direct-upload) mode,
+which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional).
Using separate buckets for each data type is the recommended approach for GitLab.
This ensures there are no collisions across the various types of data GitLab stores.
@@ -2169,12 +2158,28 @@ in the future.
</a>
</div>
-## Enable incremental logging
+### Enable incremental logging
GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes.
While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs.
+## Configure NFS (optional)
+
+[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly)
+are recommended over NFS wherever possible for improved performance.
+
+See how to [configure NFS](../nfs.md).
+
+WARNING:
+Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation)
+after the release of GitLab 15.6. No further enhancements are planned for this feature.
+
+Read:
+
+- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
+- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
+
## Configure Advanced Search
You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md)
@@ -2191,22 +2196,6 @@ cluster alongside your instance, read how to
</a>
</div>
-## Configure NFS (optional)
-
-[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly)
-are recommended over NFS wherever possible for improved performance.
-
-See how to [configure NFS](../nfs.md).
-
-WARNING:
-Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
-unavailable from GitLab 15.0. No further enhancements are planned for this feature.
-
-Read:
-
-- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
-- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
-
## Supported modifications for lower user counts (HA)
The 3k GitLab reference architecture is the smallest we recommend that achieves High Availability (HA).
@@ -2252,6 +2241,10 @@ to be complex. **This setup is only recommended** if you have strong working
knowledge and experience in Kubernetes. The rest of this
section assumes this.
+NOTE:
+**Gitaly Cluster is not supported to be run in Kubernetes**.
+Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details.
+
### Cluster topology
The following tables and diagram detail the hybrid environment using the same formats
@@ -2262,11 +2255,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK
and CPU requirements should translate to most other providers. We hope to update this in the
future with further specific cloud provider details.
-| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
-|-----------------------------------------------|-------|-------------------------|-----------------|--------------|---------------------------------|
-| Webservice | 2 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | 31.8 vCPU, 24.8 GB memory |
-| Sidekiq | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 11.8 vCPU, 38.9 GB memory |
-| Supporting services such as NGINX, Prometheus | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | 3.9 vCPU, 11.8 GB memory |
+| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
+|---------------------|-------|-------------------------|-----------------|--------------|---------------------------------|
+| Webservice | 2 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | 31.8 vCPU, 24.8 GB memory |
+| Sidekiq | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 11.8 vCPU, 38.9 GB memory |
+| Supporting services | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | 3.9 vCPU, 11.8 GB memory |
- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results)
[Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary.
@@ -2276,24 +2269,24 @@ future with further specific cloud provider details.
Next are the backend components that run on static compute VMs via Omnibus (or External PaaS
services where applicable):
-| Service | Nodes | Configuration | GCP | AWS |
-|-------------------------------------------|----------------|-----------------------|-----------------|----------------|
-| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
-| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable |
+| Service | Nodes | Configuration | GCP | AWS |
+|-------------------------------------------|-------|-----------------------|-----------------|-------------|
+| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
+| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Object storage<sup>4</sup> | - | - | - | - |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -2308,11 +2301,11 @@ card "Kubernetes via Helm Charts" as kubernetes {
card "**External Load Balancer**" as elb #6a9be7
together {
- collections "**Webservice** x4" as gitlab #32CD32
- collections "**Sidekiq** x4" as sidekiq #ff8dd1
+ collections "**Webservice** x2" as gitlab #32CD32
+ collections "**Sidekiq** x3" as sidekiq #ff8dd1
}
- card "**Supporting Services**" as support
+ card "**Supporting Services** x2" as support
}
card "**Internal Load Balancer**" as ilb #9370DB
@@ -2375,29 +2368,42 @@ documents how to apply the calculated configuration to the Helm Chart.
#### Webservice
-Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_.
-Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using
+Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_.
+Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using
the [recommended topology](#cluster-topology) because four worker processes
are created by default and each pod has other small processes running.
For 3,000 users we recommend a total Puma worker count of around 16.
With the [provided recommendations](#cluster-topology) this allows the deployment of up to 4
Webservice pods with 4 workers per pod and 2 pods per node. Expand available resources using
-the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional
+the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional
Webservice pod.
For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources).
#### Sidekiq
-Sidekiq pods should generally have 1 vCPU and 2 GB of memory.
+Sidekiq pods should generally have 0.9 CPU and 2 GB of memory.
[The provided starting point](#cluster-topology) allows the deployment of up to
-8 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory
+8 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory
ratio for each additional pod.
For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources).
+### Supporting
+
+The Supporting Node Pool is designed to house all supporting deployments that don't need to be
+on the Webservice and Sidekiq pools.
+
+This includes various deployments related to the Cloud Provider's implementation and supporting
+GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/).
+
+If you wish to make any additional deployments, such as for Monitoring, it's recommended
+to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed
+specifically to accommodate several additional deployments. However, if your deployments don't fit into the
+pool as given, you can increase the node pool accordingly.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md
index 535e483cb7d..bddec55ba71 100644
--- a/doc/administration/reference_architectures/50k_users.md
+++ b/doc/administration/reference_architectures/50k_users.md
@@ -18,30 +18,30 @@ full list of reference architectures, see
> - **Test requests per second (RPS) rates:** API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS
> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)**
-| Service | Nodes | Configuration | GCP | AWS | Azure |
-|---------------------------------------------------|----------------|-------------------------|------------------|----------------|----------------|
-| External load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` |
-| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| PostgreSQL<sup>1</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | `D32s v3` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` |
-| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` | `D64s v3` |
-| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| GitLab Rails | 12 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` |
-| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable |
-| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Service | Nodes | Configuration | GCP | AWS | Azure |
+|------------------------------------------|-------|-------------------------|------------------|---------------|-----------|
+| External load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` |
+| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| PostgreSQL<sup>1</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` | `D32s v3` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` |
+| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` | `D64s v3` |
+| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| GitLab Rails | 12 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | `F32s v2` |
+| Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Object storage<sup>4</sup> | - | - | - | - | - |
+| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -142,10 +142,12 @@ Before starting, you should take note of the following requirements / guidance f
This reference architecture was built and tested on Google Cloud Platform (GCP) using the
[Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms)
-CPU platform. On different hardware you may find that adjustments, either lower
-or higher, are required for your CPU or node counts. For more information, see
-our [Sysbench](https://github.com/akopytov/sysbench)-based
-[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks).
+CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)).
+
+Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported.
+
+NOTE:
+Any "burstable" instance types are not recommended due to inconsistent performance.
### Supported infrastructure
@@ -186,10 +188,11 @@ To set up GitLab and its components to accommodate up to 50,000 users:
environment.
1. [Configure the object storage](#configure-the-object-storage)
used for shared data objects.
+1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended)
+ to have shared disk storage service as an alternative to Gitaly or object
+ storage.
1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster,
more advanced code search across your entire GitLab instance.
-1. [Configure NFS](#configure-nfs)
- to have shared disk storage service for certain GitLab operations (non Gitaly or Object Storage).
The servers start on the same 10.6.0.0/24 private network range, and can
connect to each other freely on these addresses.
@@ -268,7 +271,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather
than `HTTP(S)` protocol. This will pass the connection to the application node's
NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
-See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Load balancer terminates SSL without backend SSL
@@ -279,7 +282,7 @@ terminating SSL.
Since communication between the load balancer and GitLab will not be secure,
there is some additional configuration needed. See the
-[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl)
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
for details.
### Load balancer terminates SSL with backend SSL
@@ -291,8 +294,8 @@ end users will see.
Traffic will also be secure between the load balancers and NGINX in this
scenario. There is no need to add configuration for proxied SSL since the
connection will be secure all the way. However, configuration will need to be
-added to GitLab to configure SSL certificates. See
-[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+added to GitLab to configure SSL certificates. See the
+[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Readiness checks
@@ -2031,7 +2034,7 @@ On each node perform the following:
1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
-1. If you're [using NFS](#configure-nfs):
+1. If you're [using NFS](#configure-nfs-optional):
1. If necessary, install the NFS client utility packages using the following
commands:
@@ -2090,7 +2093,7 @@ On each node perform the following:
When you specify `https` in the `external_url`, as in the previous example,
GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the
certificates aren't present, NGINX will fail to start. For more information, see
-the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https).
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html).
### GitLab Rails post-configuration
@@ -2195,7 +2198,7 @@ To configure the Monitoring node:
## Configure the object storage
GitLab supports using an object storage service for holding numerous types of data.
-It's recommended over [NFS](#configure-nfs) and in general it's better
+It's recommended over [NFS](#configure-nfs-optional) and in general it's better
in larger setups as object storage is typically much more performant, reliable,
and scalable.
@@ -2207,7 +2210,6 @@ GitLab has been tested on a number of object storage providers:
- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction)
-- On-premises hardware and appliances from various storage vendors.
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
There are two ways of specifying object storage configuration in GitLab:
@@ -2217,25 +2219,13 @@ There are two ways of specifying object storage configuration in GitLab:
- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its
own object storage [connection and configuration](../object_storage.md#connection-settings).
-Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up.
-
-For configuring object storage in GitLab 13.1 and earlier, or for storage types not
-supported by consolidated configuration form, refer to the following guides based
-on what features you intend to use:
-
-|Object storage type|Supported by consolidated configuration?|
-|-------------------|----------------------------------------|
-| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No |
-| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes |
-| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes |
-| [Uploads](../uploads.md#using-object-storage) | Yes |
-| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No |
-| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes |
-| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No |
-| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes |
-| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes |
-| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No |
-| [Terraform state files](../terraform_state.md#using-object-storage) | Yes |
+The consolidated form is used in the following examples when available.
+
+NOTE:
+When using the [storage-specific form](../object_storage.md#storage-specific-configuration)
+in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload).
+The previous [background upload](../../development/uploads/index.md#direct-upload) mode,
+which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional).
Using separate buckets for each data type is the recommended approach for GitLab.
This ensures there are no collisions across the various types of data GitLab stores.
@@ -2248,12 +2238,28 @@ in the future.
</a>
</div>
-## Enable incremental logging
+### Enable incremental logging
GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes.
While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs.
+## Configure NFS (optional)
+
+[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly)
+are recommended over NFS wherever possible for improved performance.
+
+See how to [configure NFS](../nfs.md).
+
+WARNING:
+Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation)
+after the release of GitLab 15.6. No further enhancements are planned for this feature.
+
+Read:
+
+- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
+- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
+
## Configure Advanced Search
You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md)
@@ -2270,22 +2276,6 @@ cluster alongside your instance, read how to
</a>
</div>
-## Configure NFS
-
-[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly)
-are recommended over NFS wherever possible for improved performance.
-
-See how to [configure NFS](../nfs.md).
-
-WARNING:
-Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
-unavailable from GitLab 15.0. No further enhancements are planned for this feature.
-
-Read:
-
-- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
-- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
-
## Cloud Native Hybrid reference architecture with Helm Charts (alternative)
As an alternative approach, you can also run select components of GitLab as Cloud Native
@@ -2306,6 +2296,10 @@ to be complex. **This setup is only recommended** if you have strong working
knowledge and experience in Kubernetes. The rest of this
section assumes this.
+NOTE:
+**Gitaly Cluster is not supported to be run in Kubernetes**.
+Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details.
+
### Cluster topology
The following tables and diagram detail the hybrid environment using the same formats
@@ -2316,11 +2310,11 @@ use Google Cloud's Kubernetes Engine (GKE) or AWS Elastic Kubernetes Service (EK
and CPU requirements should translate to most other providers. We hope to update this in the
future with further specific cloud provider details.
-| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
-|-----------------------------------------------|-------|-------------------------|-----------------|--------------|---------------------------------|
-| Webservice | 16 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `m5.8xlarge` | 510 vCPU, 472 GB memory |
-| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory |
-| Supporting services such as NGINX, Prometheus | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory |
+| Service | Nodes | Configuration | GCP | AWS | Min Allocatable CPUs and Memory |
+|---------------------|-------|-------------------------|-----------------|--------------|---------------------------------|
+| Webservice | 16 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `m5.8xlarge` | 510 vCPU, 472 GB memory |
+| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 15.5 vCPU, 50 GB memory |
+| Supporting services | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | 7.75 vCPU, 25 GB memory |
- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results)
[Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary.
@@ -2330,25 +2324,25 @@ future with further specific cloud provider details.
Next are the backend components that run on static compute VMs via Omnibus (or External PaaS
services where applicable):
-| Service | Nodes | Configuration | GCP | AWS |
-|------------------------------------------|----------------|------------------------|------------------|----------------|
-| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| PostgreSQL<sup>1</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` |
-| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` |
-| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable |
+| Service | Nodes | Configuration | GCP | AWS |
+|------------------------------------------|-------|------------------------|------------------|---------------|
+| Consul<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| PostgreSQL<sup>1</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` |
+| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` |
+| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Object storage<sup>4</sup> | - | - | - | - |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -2363,11 +2357,11 @@ card "Kubernetes via Helm Charts" as kubernetes {
card "**External Load Balancer**" as elb #6a9be7
together {
- collections "**Webservice** x4" as gitlab #32CD32
+ collections "**Webservice** x16" as gitlab #32CD32
collections "**Sidekiq** x4" as sidekiq #ff8dd1
}
- card "**Supporting Services**" as support
+ card "**Supporting Services** x2" as support
}
card "**Internal Load Balancer**" as ilb #9370DB
@@ -2433,29 +2427,42 @@ documents how to apply the calculated configuration to the Helm Chart.
#### Webservice
-Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_.
-Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using
+Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_.
+Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using
the [recommended topology](#cluster-topology) because four worker processes
are created by default and each pod has other small processes running.
For 50,000 users we recommend a total Puma worker count of around 320.
With the [provided recommendations](#cluster-topology) this allows the deployment of up to 80
Webservice pods with 4 workers per pod and 5 pods per node. Expand available resources using
-the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional
+the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional
Webservice pod.
For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources).
#### Sidekiq
-Sidekiq pods should generally have 1 vCPU and 2 GB of memory.
+Sidekiq pods should generally have 0.9 CPU and 2 GB of memory.
[The provided starting point](#cluster-topology) allows the deployment of up to
-14 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory
+14 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory
ratio for each additional pod.
For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources).
+### Supporting
+
+The Supporting Node Pool is designed to house all supporting deployments that don't need to be
+on the Webservice and Sidekiq pools.
+
+This includes various deployments related to the Cloud Provider's implementation and supporting
+GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/).
+
+If you wish to make any additional deployments, such as for Monitoring, it's recommended
+to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed
+specifically to accommodate several additional deployments. However, if your deployments don't fit into the
+pool as given, you can increase the node pool accordingly.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md
index e19440e7660..0e599df7c1f 100644
--- a/doc/administration/reference_architectures/5k_users.md
+++ b/doc/administration/reference_architectures/5k_users.md
@@ -25,29 +25,29 @@ costly-to-operate environment by using the
> - **Test requests per second (RPS) rates:** API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS
> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)**
-| Service | Nodes | Configuration | GCP | AWS | Azure |
-|--------------------------------------------|----------------|-------------------------|-----------------|----------------|----------------|
-| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
-| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | `D8s v3` |
-| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
-| GitLab Rails | 3 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | `F16s v2` |
-| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable | Not applicable |
-| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
+| Service | Nodes | Configuration | GCP | AWS | Azure |
+|-------------------------------------------|-------|-------------------------|-----------------|--------------|----------|
+| External load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
+| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` | `D8s v3` |
+| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
+| GitLab Rails | 3 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | `F16s v2`|
+| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
+| Object storage<sup>4</sup> | - | - | - | - | - |
+| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -145,10 +145,12 @@ Before starting, you should take note of the following requirements / guidance f
This reference architecture was built and tested on Google Cloud Platform (GCP) using the
[Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms)
-CPU platform. On different hardware you may find that adjustments, either lower
-or higher, are required for your CPU or node counts. For more information, see
-our [Sysbench](https://github.com/akopytov/sysbench)-based
-[CPU benchmarks](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks).
+CPU platform as a baseline ([Sysbench benchmark](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Reference-Architectures/GCP-CPU-Benchmarks)).
+
+Newer, similarly sized CPUs are supported and may have improved performance as a result. For Omnibus environments, ARM-based equivalents are also supported.
+
+NOTE:
+Any "burstable" instance types are not recommended due to inconsistent performance.
### Supported infrastructure
@@ -189,12 +191,11 @@ To set up GitLab and its components to accommodate up to 5,000 users:
environment.
1. [Configure the object storage](#configure-the-object-storage)
used for shared data objects.
-1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster,
- more advanced code search across your entire GitLab instance.
1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended)
to have shared disk storage service as an alternative to Gitaly or object
- storage. You can skip this step if you're not using GitLab Pages (which
- requires NFS).
+ storage.
+1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster,
+ more advanced code search across your entire GitLab instance.
The servers start on the same 10.6.0.0/24 private network range, and can
connect to each other freely on these addresses.
@@ -261,7 +262,7 @@ Configure your load balancer to pass connections on port 443 as `TCP` rather
than `HTTP(S)` protocol. This passes the connection to the application node's
NGINX service untouched. NGINX has the SSL certificate and listen on port 443.
-See the [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Load balancer terminates SSL without backend SSL
@@ -272,7 +273,7 @@ terminating SSL.
Since communication between the load balancer and GitLab is not secure,
there is some additional configuration needed. See the
-[NGINX proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl)
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
for details.
### Load balancer terminates SSL with backend SSL
@@ -284,8 +285,8 @@ end users see.
Traffic is also secure between the load balancers and NGINX in this
scenario. There is no need to add configuration for proxied SSL since the
connection is secure all the way. However, configuration needs to be
-added to GitLab to configure SSL certificates. See
-[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https)
+added to GitLab to configure SSL certificates. See the
+[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
for details on managing SSL certificates and configuring NGINX.
### Readiness checks
@@ -2005,7 +2006,7 @@ On each node perform the following:
When you specify `https` in the `external_url`, as in the previous example,
GitLab expects that the SSL certificates are in `/etc/gitlab/ssl/`. If the
certificates aren't present, NGINX fails to start. For more information, see
-the [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https).
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html).
### GitLab Rails post-configuration
@@ -2128,7 +2129,6 @@ GitLab has been tested on a number of object storage providers:
- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- [Azure Blob storage](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction)
-- On-premises hardware and appliances from various storage vendors.
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
There are two ways of specifying object storage configuration in GitLab:
@@ -2138,25 +2138,13 @@ There are two ways of specifying object storage configuration in GitLab:
- [Storage-specific form](../object_storage.md#storage-specific-configuration): Every object defines its
own object storage [connection and configuration](../object_storage.md#connection-settings).
-Starting with GitLab 13.2, consolidated object storage configuration is available. It simplifies your GitLab configuration since the connection details are shared across object types. Refer to [Consolidated object storage configuration](../object_storage.md#consolidated-object-storage-configuration) guide for instructions on how to set it up.
-
-For configuring object storage in GitLab 13.1 and earlier, or for storage types not
-supported by consolidated configuration form, refer to the following guides based
-on what features you intend to use:
-
-|Object storage type|Supported by consolidated configuration?|
-|-------------------|----------------------------------------|
-| [Backups](../../raketasks/backup_gitlab.md#uploading-backups-to-a-remote-cloud-storage) | No |
-| [Job artifacts](../job_artifacts.md#using-object-storage) including archived job logs | Yes |
-| [LFS objects](../lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes |
-| [Uploads](../uploads.md#using-object-storage) | Yes |
-| [Container Registry](../packages/container_registry.md#use-object-storage) (optional feature) | No |
-| [Merge request diffs](../merge_request_diffs.md#using-object-storage) | Yes |
-| [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No |
-| [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes |
-| [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes |
-| [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No |
-| [Terraform state files](../terraform_state.md#using-object-storage) | Yes |
+The consolidated form is used in the following examples when available.
+
+NOTE:
+When using the [storage-specific form](../object_storage.md#storage-specific-configuration)
+in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload).
+The previous [background upload](../../development/uploads/index.md#direct-upload) mode,
+which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional).
Using separate buckets for each data type is the recommended approach for GitLab.
This ensures there are no collisions across the various types of data GitLab stores.
@@ -2169,12 +2157,28 @@ in the future.
</a>
</div>
-## Enable incremental logging
+### Enable incremental logging
GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily on disk in `/var/opt/gitlab/gitlab-ci/builds` by default, even when using consolidated object storage. With default configuration, this directory needs to be shared through NFS on any GitLab Rails and Sidekiq nodes.
While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs.
+## Configure NFS (optional)
+
+[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly)
+are recommended over NFS wherever possible for improved performance.
+
+See how to [configure NFS](../nfs.md).
+
+WARNING:
+Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation)
+after the release of GitLab 15.6. No further enhancements are planned for this feature.
+
+Read:
+
+- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
+- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
+
## Configure Advanced Search
You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md)
@@ -2191,22 +2195,6 @@ cluster alongside your instance, read how to
</a>
</div>
-## Configure NFS (optional)
-
-[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly)
-are recommended over NFS wherever possible for improved performance.
-
-See how to [configure NFS](../nfs.md).
-
-WARNING:
-Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
-unavailable from GitLab 15.0. No further enhancements are planned for this feature.
-
-Read:
-
-- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation).
-- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss).
-
## Cloud Native Hybrid reference architecture with Helm Charts (alternative)
As an alternative approach, you can also run select components of GitLab as Cloud Native
@@ -2227,6 +2215,10 @@ to be complex. **This setup is only recommended** if you have strong working
knowledge and experience in Kubernetes. The rest of this
section assumes this.
+NOTE:
+**Gitaly Cluster is not supported to be run in Kubernetes**.
+Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details.
+
### Cluster topology
The following tables and diagram detail the hybrid environment using the same formats
@@ -2251,24 +2243,24 @@ future with further specific cloud provider details.
Next are the backend components that run on static compute VMs via Omnibus (or External PaaS
services where applicable):
-| Service | Nodes | Configuration | GCP | AWS |
-|-------------------------------------------|----------------|-----------------------|-----------------|----------------|
-| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
-| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` |
-| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Object storage<sup>4</sup> | Not applicable | Not applicable | Not applicable | Not applicable |
+| Service | Nodes | Configuration | GCP | AWS |
+|-------------------------------------------|-------|-----------------------|-----------------|--------------|
+| Redis<sup>2</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
+| Consul<sup>1</sup> + Sentinel<sup>2</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` |
+| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
+| Object storage<sup>4</sup> | - | - | - | - |
<!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix -->
<!-- markdownlint-disable MD029 -->
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work. However, Amazon Aurora is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and Azure Database for PostgreSQL is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61). Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS ElastiCache are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work.
-4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work.
+4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
<!-- markdownlint-enable MD029 -->
@@ -2283,11 +2275,11 @@ card "Kubernetes via Helm Charts" as kubernetes {
card "**External Load Balancer**" as elb #6a9be7
together {
- collections "**Webservice** x4" as gitlab #32CD32
- collections "**Sidekiq** x4" as sidekiq #ff8dd1
+ collections "**Webservice** x5" as gitlab #32CD32
+ collections "**Sidekiq** x3" as sidekiq #ff8dd1
}
- card "**Supporting Services**" as support
+ card "**Supporting Services** x2" as support
}
card "**Internal Load Balancer**" as ilb #9370DB
@@ -2350,29 +2342,42 @@ documents how to apply the calculated configuration to the Helm Chart.
#### Webservice
-Webservice pods typically need about 1 vCPU and 1.25 GB of memory _per worker_.
-Each Webservice pod consumes roughly 4 vCPUs and 5 GB of memory using
+Webservice pods typically need about 1 CPU and 1.25 GB of memory _per worker_.
+Each Webservice pod consumes roughly 4 CPUs and 5 GB of memory using
the [recommended topology](#cluster-topology) because four worker processes
are created by default and each pod has other small processes running.
For 5,000 users we recommend a total Puma worker count of around 40.
With the [provided recommendations](#cluster-topology) this allows the deployment of up to 10
Webservice pods with 4 workers per pod and 2 pods per node. Expand available resources using
-the ratio of 1 vCPU to 1.25 GB of memory _per each worker process_ for each additional
+the ratio of 1 CPU to 1.25 GB of memory _per each worker process_ for each additional
Webservice pod.
For further information on resource usage, see the [Webservice resources](https://docs.gitlab.com/charts/charts/gitlab/webservice/#resources).
#### Sidekiq
-Sidekiq pods should generally have 1 vCPU and 2 GB of memory.
+Sidekiq pods should generally have 0.9 CPU and 2 GB of memory.
[The provided starting point](#cluster-topology) allows the deployment of up to
-8 Sidekiq pods. Expand available resources using the 1 vCPU to 2GB memory
+8 Sidekiq pods. Expand available resources using the 0.9 CPU to 2 GB memory
ratio for each additional pod.
For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources).
+### Supporting
+
+The Supporting Node Pool is designed to house all supporting deployments that don't need to be
+on the Webservice and Sidekiq pools.
+
+This includes various deployments related to the Cloud Provider's implementation and supporting
+GitLab deployments such as NGINX or [GitLab Shell](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/).
+
+If you wish to make any additional deployments, such as for Monitoring, it's recommended
+to deploy these in this pool where possible and not in the Webservice or Sidekiq pools, as the Supporting pool has been designed
+specifically to accommodate several additional deployments. However, if your deployments don't fit into the
+pool as given, you can increase the node pool accordingly.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
diff --git a/doc/administration/repository_checks.md b/doc/administration/repository_checks.md
index a97c8611239..4615499d6fa 100644
--- a/doc/administration/repository_checks.md
+++ b/doc/administration/repository_checks.md
@@ -20,7 +20,7 @@ committed to a repository. GitLab administrators can:
To check a project's repository using GitLab UI:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Projects**.
1. Select the project to check.
1. In the **Repository check** section, select **Trigger repository check**.
@@ -32,7 +32,7 @@ project page in the Admin Area. If the checks fail, see [what to do](#what-to-do
Instead of checking repositories manually, GitLab can be configured to run the checks periodically:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Repository** (`/admin/application_settings/repository`).
1. Expand the **Repository maintenance** section.
1. Enable **Enable repository checks**.
@@ -68,7 +68,7 @@ You can run [`git fsck`](https://git-scm.com/docs/git-fsck) using the command li
1. Run the check. For example:
```shell
- sudo /opt/gitlab/embedded/bin/git -C /var/opt/gitlab/git-data/repositories/@hashed/0b/91/0b91...f9.git fsck
+ sudo -u git /opt/gitlab/embedded/bin/git -C /var/opt/gitlab/git-data/repositories/@hashed/0b/91/0b91...f9.git fsck
```
## What to do if a check failed
@@ -81,7 +81,15 @@ If a repository check fails, locate the error in the [`repocheck.log` file](logs
If periodic repository checks cause false alarms, you can clear all repository check states:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Repository** (`/admin/application_settings/repository`).
1. Expand the **Repository maintenance** section.
1. Select **Clear all repository checks**.
+
+### Error: `failed to parse commit <commit SHA> from object database for commit-graph`
+
+You can see a `failed to parse commit <commit SHA> from object database for commit-graph` error in repository check logs. This error occurs if your `commit-graph` cache is out
+of date. The `commit-graph` cache is an auxiliary cache and is not required for regular Git operations.
+
+While the message can be safely ignored, see the issue [error: Could not read from object database for commit-graph](https://gitlab.com/gitlab-org/gitaly/-/issues/2359)
+for more details.
diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md
index f001ba35bca..482cbd97e37 100644
--- a/doc/administration/repository_storage_paths.md
+++ b/doc/administration/repository_storage_paths.md
@@ -146,7 +146,7 @@ Omnibus stores the repositories in a `repositories` subdirectory of the `git-dat
After you [configure](#configure-repository-storage-paths) multiple repository storage paths, you
can choose where new repositories are stored:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Repository** and expand the **Repository storage**
section.
1. Enter values in the **Storage nodes for new repositories** fields.
diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md
index 7ce629b5d71..865daba9e89 100644
--- a/doc/administration/repository_storage_types.md
+++ b/doc/administration/repository_storage_types.md
@@ -79,7 +79,7 @@ Administrators can look up a project's hashed path from its name or ID using:
To look up a project's hash path in the Admin Area:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Projects** and select the project.
The **Gitaly relative path** is displayed there and looks similar to:
diff --git a/doc/administration/restart_gitlab.md b/doc/administration/restart_gitlab.md
index 6625039504a..e5ec12054b8 100644
--- a/doc/administration/restart_gitlab.md
+++ b/doc/administration/restart_gitlab.md
@@ -102,7 +102,7 @@ depend on those files.
## Installations from source
-If you have followed the official installation guide to
+If you have followed the official installation guide to
[install GitLab from source](../install/installation.md), run the following command to restart GitLab:
```shell
diff --git a/doc/administration/server_hooks.md b/doc/administration/server_hooks.md
index 1a47dc4ccf2..4148abf1bdc 100644
--- a/doc/administration/server_hooks.md
+++ b/doc/administration/server_hooks.md
@@ -30,7 +30,7 @@ alternatives to server hooks include:
To create server hooks for a repository:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. Go to **Overview > Projects** and select the project you want to add a server hook to.
1. On the page that appears, locate the value of **Gitaly relative path**. This path is where server hooks must be located.
- If you are using [hashed storage](repository_storage_types.md#hashed-storage), see
@@ -56,8 +56,7 @@ If the server hook code is properly implemented, it should execute when the Git
## Create global server hooks for all repositories
-To create a Git hook that applies to all repositories, set a global server hook. The default global server hook directory
-is in the GitLab Shell directory. Any server hook added there applies to all repositories, including:
+To create a Git hook that applies to all repositories, set a global server hook. Global server hooks also apply to:
- [Project and group wiki](../user/project/wiki/index.md) repositories. Their storage directory names are in the format
`<id>.wiki.git`.
@@ -66,36 +65,36 @@ is in the GitLab Shell directory. Any server hook added there applies to all rep
### Choose a server hook directory
-Before creating a global server hook, you must choose a directory for it. The default global server hook directory:
+Before creating a global server hook, you must choose a directory for it.
-- For Omnibus GitLab installations is usually `/opt/gitlab/embedded/service/gitlab-shell/hooks`.
-- For an installation from source is usually `/home/git/gitlab-shell/hooks`.
+For Omnibus GitLab installations, the directory is set in `gitlab.rb` under `gitaly['custom_hooks_dir']`. You can either:
-To use a different directory for global server hooks, set `custom_hooks_dir` in Gitaly configuration:
+- Use the default suggestion of the `/var/opt/gitlab/gitaly/custom_hooks` directory by uncommenting it.
+- Add your own setting.
-- For Omnibus installations, set in `gitlab.rb`.
-- For source installations, the configuration location depends on the GitLab version. For:
- - GitLab 13.0 and earlier, set in `gitlab-shell/config.yml`.
- - GitLab 13.1 and later, set in `gitaly/config.toml` under the `[hooks]` section. However, GitLab honors the
- `custom_hooks_dir` value in `gitlab-shell/config.yml` if the value in `gitaly/config.toml` is blank or non-existent.
+For installations from source:
+
+- The directory is set in a configuration file. The location of the configuration file depends on the GitLab version:
+ - For GitLab 13.1 and later, the directory is set in `gitaly/config.toml` under the `[hooks]` section. However,
+ GitLab honors the `custom_hooks_dir` value in `gitlab-shell/config.yml` if the value in `gitaly/config.toml` is blank
+ or non-existent.
+ - For GitLab 13.0 and earlier, the directory set in `gitlab-shell/config.yml`.
+- The default directory is `/home/git/gitlab-shell/hooks`.
### Create the global server hook
To create a global server hook for all repositories:
1. On the GitLab server, go to the configured global server hook directory.
-1. In the configured global server hook directory:
- - To create a single server hook, create a file with a name that matches the hook type. For example, for a
- `pre-receive` server hook, the filename should be `pre-receive` with no extension.
- - To create many server hooks, create a directory for the hooks that matches the hook type. For example, for a
- `pre-receive` server hook, the directory name should be `pre-receive.d`. Put the files for the hook in that directory.
-1. Inside this new directory, add your server hook. Server hooks can be in any programming language. Ensure the
+1. In the configured global server hook directory, create a directory for the hooks that matches the hook type. For example, for a `pre-receive` server hook, the directory name should be `pre-receive.d`.
+1. Inside this new directory, add your server hooks. Server hooks can be in any programming language. Ensure the
[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) at the top reflects the language type. For example, if the
script is in Ruby the shebang is probably `#!/usr/bin/env ruby`.
1. Make the hook file executable, ensure that it's owned by the Git user, and ensure it does not match the backup file
pattern (`*~`).
-If the server hook code is properly implemented, it should execute when the Git hook is next triggered.
+If the server hook code is properly implemented, it should execute when the Git hook is next triggered. Hooks are executed in alphabetical order by filename in the hook type
+subdirectories.
## Chained server hooks
diff --git a/doc/administration/sidekiq/extra_sidekiq_processes.md b/doc/administration/sidekiq/extra_sidekiq_processes.md
index 1cd3771c94d..b774b0d3e14 100644
--- a/doc/administration/sidekiq/extra_sidekiq_processes.md
+++ b/doc/administration/sidekiq/extra_sidekiq_processes.md
@@ -95,7 +95,7 @@ To start multiple processes:
To view the Sidekiq processes in GitLab:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Monitoring > Background Jobs**.
## Negate settings
diff --git a/doc/administration/static_objects_external_storage.md b/doc/administration/static_objects_external_storage.md
index 21949388f19..a288b19f164 100644
--- a/doc/administration/static_objects_external_storage.md
+++ b/doc/administration/static_objects_external_storage.md
@@ -16,7 +16,7 @@ storage such as a content delivery network (CDN).
To configure external storage for static objects:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Repository**.
1. Expand the **External storage for repository static objects** section.
1. Enter the base URL and an arbitrary token. When you [set up external storage](#set-up-external-storage),
diff --git a/doc/administration/system_hooks.md b/doc/administration/system_hooks.md
index cb141a18ae6..0d70744e942 100644
--- a/doc/administration/system_hooks.md
+++ b/doc/administration/system_hooks.md
@@ -56,7 +56,7 @@ for Push and Tag events, but we never display commits.
To create a system hook:
-1. On the top bar, select **Menu > Admin**.
+1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **System Hooks**.
1. Provide the **URL** and **Secret Token**.
1. Select the checkbox next to each optional **Trigger** you want to enable.
diff --git a/doc/administration/terraform_state.md b/doc/administration/terraform_state.md
index 7a8d7774948..5a272025987 100644
--- a/doc/administration/terraform_state.md
+++ b/doc/administration/terraform_state.md
@@ -78,8 +78,8 @@ Terraform state files are stored locally, follow the steps below.
## Using object storage **(FREE SELF)**
-Instead of storing Terraform state files on disk, we recommend the use of
-[one of the supported object storage options](object_storage.md#options).
+Instead of storing Terraform state files on disk, we recommend the use of
+[one of the supported object storage options](object_storage.md#options).
This configuration relies on valid credentials to be configured already.
[Read more about using object storage with GitLab](object_storage.md).
diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
index aa4dbec4f95..40bb15ecb70 100644
--- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
+++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
@@ -28,22 +28,6 @@ mentioned above, we recommend running these scripts under the supervision of a
Support Engineer, who can also verify that they continue to work as they
should and, if needed, update the script for the latest version of GitLab.
-## Find specific methods for an object
-
-```ruby
-Array.methods.select { |m| m.to_s.include? "sing" }
-Array.methods.grep(/sing/)
-```
-
-## Find method source
-
-```ruby
-instance_of_object.method(:foo).source_location
-
-# Example for when we would call project.private?
-project.method(:private?).source_location
-```
-
## Attributes
View available attributes, formatted using pretty print (`pp`).
@@ -78,28 +62,6 @@ Notify.test_email(e, "Test email for #{n}", 'Test email').deliver_now
Notify.test_email(u.email, "Test email for #{u.name}", 'Test email').deliver_now
```
-## Limiting output
-
-Adding a semicolon(`;`) and a follow-up statement at the end of a statement prevents the default implicit return output. This can be used if you are already explicitly printing details and potentially have a lot of return output:
-
-```ruby
-puts ActiveRecord::Base.descendants; :ok
-Project.select(&:pages_deployed?).each {|p| puts p.pages_url }; true
-```
-
-## Get or store the result of last operation
-
-Underscore(`_`) represents the implicit return of the previous statement. You can use this to quickly assign a variable from the output of the previous command:
-
-```ruby
-Project.last
-# => #<Project id:2537 root/discard>>
-project = _
-# => #<Project id:2537 root/discard>>
-project.id
-# => 2537
-```
-
## Open object in `irb`
Sometimes it is easier to go through a method if you are in the context of the object. You can shim into the namespace of `Object` to let you open `irb` in the context of any object:
@@ -115,15 +77,6 @@ irb(#<Project>)> web_url
# => "https://gitlab-example/root/discard"
```
-## Query the database using an ActiveRecord Model
-
-```ruby
-m = Model.where('attribute like ?', 'ex%')
-
-# for example to query the projects
-projects = Project.where('path like ?', 'Oumua%')
-```
-
## View all keys in cache
```ruby
@@ -166,18 +119,6 @@ Benchmark.bm do |x|
end
```
-## Feature flags
-
-### Show all feature flags that are enabled
-
-```ruby
-# Regular output
-Feature.all
-
-# Nice output
-Feature.all.map {|f| [f.name, f.state]}
-```
-
## Projects
### Clear a project's cache
@@ -813,30 +754,6 @@ subgroup.members.map(&:errors).map(&:full_messages)
subgroup.members_and_requesters.map(&:errors).map(&:full_messages)
```
-## Authentication
-
-### Re-enable standard web sign-in form
-
-Re-enable the standard username and password-based sign-in form if it was disabled as a [Sign-in restriction](../../user/admin_area/settings/sign_in_restrictions.md#password-authentication-enabled).
-
-You can use this method when a configured external authentication provider (through SSO or an LDAP configuration) is facing an outage and direct sign-in access to GitLab is required.
-
-```ruby
-Gitlab::CurrentSettings.update!(password_authentication_enabled_for_web: true)
-```
-
-## SCIM
-
-### Find groups using an SQL query
-
-Find and store an array of groups based on an SQL query:
-
-```ruby
-# Finds groups and subgroups that end with '%oup'
-Group.find_by_sql("SELECT * FROM namespaces WHERE name LIKE '%oup'")
-=> [#<Group id:3 @test-group>, #<Group id:4 @template-group/template-subgroup>]
-```
-
## Routes
### Remove redirecting routes
@@ -1173,6 +1090,38 @@ This content has been converted to a Rake task, see [verify database values can
## Geo
+### Reverify all uploads (or any SSF data type which is verified)
+
+1. SSH into a GitLab Rails node in the primary Geo site.
+1. Open [Rails console](../operations/rails_console.md).
+1. Mark all uploads as "pending verification":
+
+ ```ruby
+ Upload.verification_state_table_class.each_batch do |relation|
+ relation.update_all(verification_state: 0)
+ end
+ ```
+
+1. This will cause the primary to start checksumming all Uploads.
+1. When a primary successfully checksums a record, then all secondaries rechecksum as well, and they compare the values.
+
+A similar thing can be done for all Models handled by the [Geo Self-Service Framework](../../development/geo/framework.md) which have implemented verification:
+
+- `LfsObject`
+- `MergeRequestDiff`
+- `Packages::PackageFile`
+- `Terraform::StateVersion`
+- `SnippetRepository`
+- `Ci::PipelineArtifact`
+- `PagesDeployment`
+- `Upload`
+- `Ci::JobArtifact`
+- `Ci::SecureFile`
+
+NOTE:
+`GroupWikiRepository` is not in the previous list since verification is not implemented.
+There is an [issue to implement this functionality in the Admin UI](https://gitlab.com/gitlab-org/gitlab/-/issues/364729).
+
### Artifacts
#### Find failed artifacts
@@ -1362,54 +1311,3 @@ Prints the metrics saved in `conversational_development_index_metrics`.
```shell
rake gitlab:usage_data:generate_and_send
```
-
-## Elasticsearch
-
-### Configuration attributes
-
-Open the rails console (`gitlab rails c`) and run the following command to see all the available attributes:
-
-```ruby
-ApplicationSetting.last.attributes
-```
-
-Among other attributes, the output contains all the settings available in the [Elasticsearch Integration page](../../integration/advanced_search/elasticsearch.md), such as `elasticsearch_indexing`, `elasticsearch_url`, `elasticsearch_replicas`, and `elasticsearch_pause_indexing`.
-
-#### Setting attributes
-
-You can then set anyone of Elasticsearch integration settings by issuing a command similar to:
-
-```ruby
-ApplicationSetting.last.update(elasticsearch_url: '<your ES URL and port>')
-
-#or
-
-ApplicationSetting.last.update(elasticsearch_indexing: false)
-```
-
-#### Getting attributes
-
-You can then check if the settings have been set in the [Elasticsearch Integration page](../../integration/advanced_search/elasticsearch.md) or in the rails console by issuing:
-
-```ruby
-Gitlab::CurrentSettings.elasticsearch_url
-
-#or
-
-Gitlab::CurrentSettings.elasticsearch_indexing
-```
-
-#### Changing the Elasticsearch password
-
-```ruby
-es_url = Gitlab::CurrentSettings.current_application_settings
-
-# Confirm the current ElasticSearch URL
-es_url.elasticsearch_url
-
-# Set the ElasticSearch URL
-es_url.elasticsearch_url = "http://<username>:<password>@your.es.host:<port>"
-
-# Save the change
-es_url.save!
-```
diff --git a/doc/administration/troubleshooting/linux_cheat_sheet.md b/doc/administration/troubleshooting/linux_cheat_sheet.md
index 6ff6e562a7d..c1a428018c2 100644
--- a/doc/administration/troubleshooting/linux_cheat_sheet.md
+++ b/doc/administration/troubleshooting/linux_cheat_sheet.md
@@ -204,7 +204,7 @@ or you can build it from source if you have the Rust compiler.
#### How to use the tool
-First run the tool with `summary` flag to get a summary of the top processes sorted by time spent actively performing tasks.
+First run the tool with `summary` flag to get a summary of the top processes sorted by time spent actively performing tasks.
You can also sort based on total time, # of system calls made, PID #, and # of child processes
using the `-s` or `--sort` flag. The number of results defaults to 25 processes, but
can be changed using the `-c`/`--count` option. See `--help` for full details.
diff --git a/doc/administration/troubleshooting/ssl.md b/doc/administration/troubleshooting/ssl.md
index e1cd92a788f..c5f3f0ed8d1 100644
--- a/doc/administration/troubleshooting/ssl.md
+++ b/doc/administration/troubleshooting/ssl.md
@@ -13,7 +13,7 @@ main SSL documentation:
- [Omnibus SSL Configuration](https://docs.gitlab.com/omnibus/settings/ssl.html).
- [Self-signed certificates or custom Certification Authorities for GitLab Runner](https://docs.gitlab.com/runner/configuration/tls-self-signed.html).
-- [Manually configuring HTTPS](https://docs.gitlab.com/omnibus/settings/nginx.html#manually-configuring-https).
+- [Configure HTTPS manually](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-https-manually).
## Using an internal CA certificate with GitLab
@@ -251,3 +251,13 @@ You must specify that Git should use OpenSSL:
```shell
git config --system http.sslbackend openssl
```
+
+Alternatively, you can ignore SSL verification by running:
+
+WARNING:
+Proceed with caution when [ignoring SSL](https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpsslVerify)
+due to the potential security issues associated with disabling this option at global level. Use this option _only_ when troubleshooting, and reinstate SSL verification immediately after.
+
+```shell
+git config --global http.sslVerify false
+```
diff --git a/doc/administration/troubleshooting/tracing_correlation_id.md b/doc/administration/troubleshooting/tracing_correlation_id.md
index ee59b7c2504..917e27bab70 100644
--- a/doc/administration/troubleshooting/tracing_correlation_id.md
+++ b/doc/administration/troubleshooting/tracing_correlation_id.md
@@ -6,6 +6,6 @@ remove_date: '2022-11-12'
This document was moved to [another location](../logs/tracing_correlation_id.md).
<!-- This redirect file can be deleted after 2022-11-12. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> \ No newline at end of file
diff --git a/doc/administration/user_settings.md b/doc/administration/user_settings.md
index 2e879f8789d..0a3f351c695 100644
--- a/doc/administration/user_settings.md
+++ b/doc/administration/user_settings.md
@@ -8,10 +8,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w
GitLab administrators can modify user settings for the entire GitLab instance.
-## Prevent users from creating top-level groups
+## Prevent new users from creating top-level groups
-By default, new users can create top-level groups. To disable your users'
-ability to create top-level groups:
+By default, new users can create top-level groups. To disable new users'
+ability to create top-level groups (does not affect existing users' setting):
**Omnibus GitLab installations**
@@ -33,6 +33,13 @@ ability to create top-level groups:
1. [Restart GitLab](restart_gitlab.md#installations-from-source).
+### Prevent existing users from creating top-level groups
+
+Administrators can:
+
+- Use the Admin Area to [prevent an existing user from creating top-level groups](../user/admin_area/index.md#prevent-a-user-from-creating-groups).
+- Use the [modify an existing user API endpoint](../api/users.md#user-modification) to change the `can_create_group` setting.
+
## Prevent users from changing their usernames
By default, new users can change their usernames. To disable your users'
diff --git a/doc/administration/whats-new.md b/doc/administration/whats-new.md
index 05c769cf20c..d8003d579ac 100644
--- a/doc/administration/whats-new.md
+++ b/doc/administration/whats-new.md
@@ -6,37 +6,39 @@ info: For assistance with this What's new page, see https://about.gitlab.com/han
# What's new **(FREE)**
-With each monthly release, GitLab includes some of the highlights from the last 10
-GitLab versions in the **What's new** feature. To access it:
+You can view some of the highlights from the last 10
+GitLab versions in the **What's new** feature. It lists new features available in different
+[GitLab tiers](https://about.gitlab.com/pricing/).
-1. In the top navigation bar, select the **{question}** icon.
-1. Select **What's new** from the menu.
-
-The **What's new** describes new features available in multiple
-[GitLab tiers](https://about.gitlab.com/pricing/). While all users can see the
-feature list, the feature list is tailored to your subscription type:
+All users can see the feature list, but the entries might differ depending on the subscription type:
-- Features only available to self-managed installations are not shown on GitLab.com.
- Features only available on GitLab.com are not shown to self-managed installations.
+- Features only available to self-managed installations are not shown on GitLab.com.
-## Self-managed installations
+ NOTE:
+ For self-managed installations, the updated **What's new** is included
+ in the first patch release after a new version, such as `13.10.1`.
-Due to our release post process, the content for **What's new** is not finalized
-when a new version (`.0` release) is cut. The updated **What's new** is included
-in the first patch release, such as `13.10.1`.
+## Access What's new
+
+To access the **What's new** feature:
+
+1. On the top bar, select the **{question}** icon.
+1. Select **What's new** from the menu.
-## Configure What's new variant
+## Configure What's new
-You can configure the What's new variant:
+You can configure **What's new** to display features based on the tier,
+or you can hide it. To configure it:
-1. On the top bar, select **Menu > Admin**.
-1. On the left sidebar, select **Settings > Preferences**, then expand **What's new**.
-1. Choose one of the following options:
+1. On the top bar, select **Main menu > Admin**.
+1. On the left sidebar, select **Settings > Preferences**.
+1. Expand **What's new**, and choose one of the following options:
| Option | Description |
| ------ | ----------- |
- | Enable What's new: All tiers | What's new presents new features from all tiers to help you keep track of all new features. |
- | Enable What's new: Current tier only | What's new presents new features for your current subscription tier, while hiding new features not available to your subscription tier. |
- | Disable What's new | What's new is disabled and can no longer be viewed. |
+ | Enable What's new: All tiers | Presents new features from all tiers. |
+ | Enable What's new: Current tier only | Presents new features for your current subscription tier, and hides new features outside of your tier. |
+ | Disable What's new | Disables this feature, so it no longer displays under the **{question}** icon. |
-1. Save your changes.
+1. Select **Save changes**.