diff options
Diffstat (limited to 'doc/administration')
74 files changed, 479 insertions, 291 deletions
diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index 67a3a3c1539..fa7fa3666b3 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -180,12 +180,6 @@ the steps bellow. Feature.enable(:repository_push_audit_event) ``` -## Retention policy - -On GitLab.com, Audit Event records become subject to deletion after 400 days, or when your license is downgraded to a tier that does not include access to Audit Events. Data that is subject to deletion will be deleted at GitLab's discretion, possibly without additional notice. - -If you require a longer retention period, you should independently archive your Audit Event data, which you can retrieve through the [Audit Events API](../api/audit_events.md). - ## Export to CSV **(PREMIUM ONLY)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1449) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. diff --git a/doc/administration/auditor_users.md b/doc/administration/auditor_users.md index c41065abd17..30493597194 100644 --- a/doc/administration/auditor_users.md +++ b/doc/administration/auditor_users.md @@ -58,7 +58,7 @@ helpful: To create a new Auditor user: 1. Create a new user or edit an existing one by navigating to - **Admin Area > Users**. You will find the option of the access level in + **Admin Area > Users**. The option of the access level is located in the 'Access' section.  diff --git a/doc/administration/auth/google_secure_ldap.md b/doc/administration/auth/google_secure_ldap.md index d30efc6d3cc..37366b00f73 100644 --- a/doc/administration/auth/google_secure_ldap.md +++ b/doc/administration/auth/google_secure_ldap.md @@ -3,3 +3,6 @@ redirect_to: 'ldap/google_secure_ldap.md' --- This document was moved to [another location](ldap/google_secure_ldap.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/auth/how_to_configure_ldap_gitlab_ce/index.md b/doc/administration/auth/how_to_configure_ldap_gitlab_ce/index.md index 40d021e180c..ffce06afb63 100644 --- a/doc/administration/auth/how_to_configure_ldap_gitlab_ce/index.md +++ b/doc/administration/auth/how_to_configure_ldap_gitlab_ce/index.md @@ -3,3 +3,6 @@ redirect_to: '../ldap/index.md' --- This document was moved to [another location](../ldap/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/auth/how_to_configure_ldap_gitlab_ee/index.md b/doc/administration/auth/how_to_configure_ldap_gitlab_ee/index.md index 40d021e180c..ffce06afb63 100644 --- a/doc/administration/auth/how_to_configure_ldap_gitlab_ee/index.md +++ b/doc/administration/auth/how_to_configure_ldap_gitlab_ee/index.md @@ -3,3 +3,6 @@ redirect_to: '../ldap/index.md' --- This document was moved to [another location](../ldap/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/auth/ldap-ee.md b/doc/administration/auth/ldap-ee.md index f5565628af1..6d56654a44b 100644 --- a/doc/administration/auth/ldap-ee.md +++ b/doc/administration/auth/ldap-ee.md @@ -3,3 +3,6 @@ redirect_to: 'ldap/index.md' --- This document was moved to [another location](ldap/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/auth/ldap-troubleshooting.md b/doc/administration/auth/ldap-troubleshooting.md index a553f449abb..1e02755e3e5 100644 --- a/doc/administration/auth/ldap-troubleshooting.md +++ b/doc/administration/auth/ldap-troubleshooting.md @@ -3,3 +3,6 @@ redirect_to: 'ldap/ldap-troubleshooting.md' --- This document was moved to [another location](ldap/ldap-troubleshooting.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index f5565628af1..6d56654a44b 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -3,3 +3,6 @@ redirect_to: 'ldap/index.md' --- This document was moved to [another location](ldap/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md index 9d88d66bf46..cef274d5c97 100644 --- a/doc/administration/auth/ldap/index.md +++ b/doc/administration/auth/ldap/index.md @@ -46,10 +46,10 @@ the LDAP server or share email addresses. ### User deletion **(CORE ONLY)** -If a user is deleted from the LDAP server, they will be blocked in GitLab as -well. Users will be immediately blocked from logging in. However, there is an +If a user is deleted from the LDAP server, they are also blocked in GitLab. +Users are immediately blocked from logging in. However, there is an LDAP check cache time of one hour (see note) which means users that -are already logged in or are using Git over SSH will still be able to access +are already logged in or are using Git over SSH are be able to access GitLab for up to one hour. Manually block the user in the GitLab Admin Area to immediately block all access. @@ -66,7 +66,7 @@ in the application settings. When a user signs in to GitLab with LDAP for the first time, and their LDAP email address is the primary email address of an existing GitLab user, then -the LDAP DN will be associated with the existing user. If the LDAP email +the LDAP DN is associated with the existing user. If the LDAP email attribute is not found in GitLab's database, a new user is created. In other words, if an existing GitLab user wants to enable LDAP sign-in for @@ -94,7 +94,7 @@ for information on the LDAP check Rake task. NOTE: **Note:** The `encryption` value `simple_tls` corresponds to 'Simple TLS' in the LDAP library. `start_tls` corresponds to StartTLS, not to be confused with regular TLS. -Normally, if you specify `simple_tls` it will be on port 636, while `start_tls` (StartTLS) +Normally, if you specify `simple_tls` it is on port 636, while `start_tls` (StartTLS) would be on port 389. `plain` also operates on port 389. Removed values: `tls` was replaced with `start_tls` and `ssl` was replaced with `simple_tls`. LDAP users must have a set email address, regardless of whether or not it's used @@ -167,27 +167,27 @@ production: | Setting | Description | Required | Examples | | ------- | ----------- | -------- | -------- | -| `label` | A human-friendly name for your LDAP server. It will be displayed on your sign-in page. | yes | `'Paris'` or `'Acme, Ltd.'` | +| `label` | A human-friendly name for your LDAP server. It is displayed on your sign-in page. | yes | `'Paris'` or `'Acme, Ltd.'` | | `host` | IP address or domain name of your LDAP server. | yes | `'ldap.mydomain.com'` | | `port` | The port to connect with on your LDAP server. Always an integer, not a string. | yes | `389` or `636` (for SSL) | -| `uid` | LDAP attribute for username. Should be the attribute, not the value that maps to the `uid`. | yes | `'sAMAccountName'`, `'uid'`, `'userPrincipalName'` | -| `bind_dn` | The full DN of the user you will bind with. | no | `'america\momo'` or `'CN=Gitlab,OU=Users,DC=domain,DC=com'` | +| `uid` | LDAP attribute for username. Should be the attribute, not the value that maps to the `uid`. | yes | `'sAMAccountName'` or `'uid'` or `'userPrincipalName'` | +| `bind_dn` | The full DN of the user you bind with. | no | `'america\momo'` or `'CN=Gitlab,OU=Users,DC=domain,DC=com'` | | `password` | The password of the bind user. | no | `'your_great_password'` | | `encryption` | Encryption method. The `method` key is deprecated in favor of `encryption`. | yes | `'start_tls'` or `'simple_tls'` or `'plain'` | | `verify_certificates` | Enables SSL certificate verification if encryption method is `start_tls` or `simple_tls`. Defaults to true. | no | boolean | | `timeout` | Set a timeout, in seconds, for LDAP queries. This helps avoid blocking a request if the LDAP server becomes unresponsive. A value of 0 means there is no timeout. | no | `10` or `30` | | `active_directory` | This setting specifies if LDAP server is Active Directory LDAP server. For non-AD servers it skips the AD specific queries. If your LDAP server is not AD, set this to false. | no | boolean | -| `allow_username_or_email_login` | If enabled, GitLab will ignore everything after the first `@` in the LDAP username submitted by the user on sign-in. If you are using `uid: 'userPrincipalName'` on ActiveDirectory you need to disable this setting, because the userPrincipalName contains an `@`. | no | boolean | -| `block_auto_created_users` | To maintain tight control over the number of active users on your GitLab installation, enable this setting to keep new users blocked until they have been cleared by the admin (default: false). | no | boolean | +| `allow_username_or_email_login` | If enabled, GitLab ignores everything after the first `@` in the LDAP username submitted by the user on sign-in. If you are using `uid: 'userPrincipalName'` on ActiveDirectory you need to disable this setting, because the userPrincipalName contains an `@`. | no | boolean | +| `block_auto_created_users` | To maintain tight control over the number of billable users on your GitLab installation, enable this setting to keep new users blocked until they have been cleared by an administrator (default: false). | no | boolean | | `base` | Base where we can search for users. | yes | `'ou=people,dc=gitlab,dc=example'` or `'DC=mydomain,DC=com'` | | `user_filter` | Filter LDAP users. Format: [RFC 4515](https://tools.ietf.org/search/rfc4515) Note: GitLab does not support `omniauth-ldap`'s custom filter syntax. | no | `'(employeeType=developer)'` or `'(&(objectclass=user)(|(samaccountname=momo)(samaccountname=toto)))'` | -| `lowercase_usernames` | If lowercase_usernames is enabled, GitLab will lower case the username. | no | boolean | +| `lowercase_usernames` | If lowercase_usernames is enabled, GitLab converts the name to lower case. | no | boolean | ### SSL Configuration Settings **(CORE ONLY)** | Setting | Description | Required | Examples | | ------- | ----------- | -------- | -------- | -| `ca_file` | Specifies the path to a file containing a PEM-format CA certificate, e.g. if you need to use an internal CA. | no | `'/etc/ca.pem'` | +| `ca_file` | Specifies the path to a file containing a PEM-format CA certificate, for example, if you need to use an internal CA. | no | `'/etc/ca.pem'` | | `ssl_version` | Specifies the SSL version for OpenSSL to use, if the OpenSSL default is not appropriate. | no | `'TLSv1_1'` | | `ciphers` | Specific SSL ciphers to use in communication with LDAP servers. | no | `'ALL:!EXPORT:!LOW:!aNULL:!eNULL:!SSLv2'` | | `cert` | Client certificate | no | `'-----BEGIN CERTIFICATE----- <REDACTED> -----END CERTIFICATE -----'` | @@ -195,11 +195,11 @@ production: ### Attribute Configuration Settings **(CORE ONLY)** -LDAP attributes that GitLab will use to create an account for the LDAP user. The specified attribute can either be the attribute name as a string (e.g. `'mail'`), or an array of attribute names to try in order (e.g. `['mail', 'email']`). Note that the user's LDAP sign-in will always be the attribute specified as `uid` above. +LDAP attributes that GitLab uses to create an account for the LDAP user. The specified attribute can either be the attribute name as a string (for example, `'mail'`), or an array of attribute names to try in order (for example, `['mail', 'email']`). Note that the user's LDAP sign-in is the attribute specified as `uid` above. | Setting | Description | Required | Examples | | ------- | ----------- | -------- | -------- | -| `username` | The username will be used in paths for the user's own projects (like `gitlab.example.com/username/project`) and when mentioning them in issues, merge request and comments (like `@username`). If the attribute specified for `username` contains an email address, the GitLab username will be the part of the email address before the `@`. | no | `['uid', 'userid', 'sAMAccountName']` | +| `username` | The username is used in paths for the user's own projects (like `gitlab.example.com/username/project`) and when mentioning them in issues, merge request and comments (like `@username`). If the attribute specified for `username` contains an email address, the GitLab username is part of the email address before the `@`. | no | `['uid', 'userid', 'sAMAccountName']` | | `email` | LDAP attribute for user email. | no | `['mail', 'email', 'userPrincipalName']` | | `name` | LDAP attribute for user display name. If no full name could be found at the attribute specified for `name`, the full name is determined using the attributes specified for `first_name` and `last_name`. | no | `'cn'` or `'displayName'` | | `first_name` | LDAP attribute for user first name. | no | `'givenName'` | @@ -335,7 +335,7 @@ an alternative such as SAML is preferred. This allows LDAP to be used for group sync, while also allowing your SAML identity provider to handle additional checks like custom 2FA. -When LDAP web sign in is disabled, users will not see a **LDAP** tab on the sign in page. +When LDAP web sign in is disabled, users don't see an **LDAP** tab on the sign in page. This does not disable [using LDAP credentials for Git access](#git-password-authentication). **Omnibus configuration** @@ -383,7 +383,7 @@ be mandatory and clients cannot be authenticated with the TLS protocol. ## Multiple LDAP servers **(STARTER ONLY)** With GitLab Enterprise Edition Starter, you can configure multiple LDAP servers -that your GitLab instance will connect to. +that your GitLab instance connects to. To add another LDAP server: @@ -391,7 +391,7 @@ To add another LDAP server: 1. Edit them to match the additional LDAP server. Be sure to choose a different provider ID made of letters a-z and numbers 0-9. -This ID will be stored in the database so that GitLab can remember which LDAP +This ID is stored in the database so that GitLab can remember which LDAP server a user belongs to.  @@ -437,7 +437,7 @@ The process executes the following access checks: - Ensure the user is still present in LDAP. - If the LDAP server is Active Directory, ensure the user is active (not - blocked/disabled state). This will only be checked if + blocked/disabled state). This is checked only if `active_directory: true` is set in the LDAP configuration. In Active Directory, a user is marked as disabled/blocked if the user @@ -446,7 +446,7 @@ has bit 2 set. For more information, see <https://ctovswild.com/2009/09/03/bitmask-searches-in-ldap/> The user is set to an `ldap_blocked` state in GitLab if the previous conditions -fail. This means the user won't be able to sign in or push/pull code. +fail. This means the user is not able to sign in or push/pull code. The process also updates the following user information: @@ -495,18 +495,18 @@ sync to run once every 12 hours at the top of the hour. ## Group Sync **(STARTER ONLY)** If your LDAP supports the `memberof` property, when the user signs in for the -first time GitLab will trigger a sync for groups the user should be a member of. +first time GitLab triggers a sync for groups the user should be a member of. That way they don't need to wait for the hourly sync to be granted access to their groups and projects. -A group sync process will run every hour on the hour, and `group_base` must be set +A group sync process runs every hour on the hour, and `group_base` must be set in LDAP configuration for LDAP synchronizations based on group CN to work. This allows GitLab group membership to be automatically updated based on LDAP group members. The `group_base` configuration should be a base LDAP 'container', such as an 'organization' or 'organizational unit', that contains LDAP groups that should be available to GitLab. For example, `group_base` could be -`ou=groups,dc=example,dc=com`. In the configuration file it will look like the +`ou=groups,dc=example,dc=com`. In the configuration file it looks like the following. **Omnibus configuration** @@ -539,7 +539,7 @@ following. 1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. -To take advantage of group sync, group owners or maintainers will need to [create one +To take advantage of group sync, group owners or maintainers need to [create one or more LDAP group links](#adding-group-links). ### Adding group links **(STARTER ONLY)** @@ -550,11 +550,11 @@ For information on adding group links via CNs and filters, refer to [the GitLab As an extension of group sync, you can automatically manage your global GitLab administrators. Specify a group CN for `admin_group` and all members of the -LDAP group will be given administrator privileges. The configuration will look +LDAP group will be given administrator privileges. The configuration looks like the following. NOTE: **Note:** -Administrators will not be synced unless `group_base` is also +Administrators are not synced unless `group_base` is also specified alongside `admin_group`. Also, only specify the CN of the admin group, as opposed to the full DN. @@ -691,10 +691,10 @@ There is a lot going on with group sync 'under the hood'. This section outlines what LDAP queries are executed and what behavior you can expect from group sync. -Group member access will be downgraded from a higher level if their LDAP group +Group member access are downgraded from a higher level if their LDAP group membership changes. For example, if a user has 'Owner' rights in a group and the next group sync reveals they should only have 'Developer' privileges, their -access will be adjusted accordingly. The only exception is if the user is the +access is adjusted accordingly. The only exception is if the user is the *last* owner in a group. Groups need at least one owner to fulfill administrative duties. @@ -716,13 +716,13 @@ are defined as one of the mentioned attributes. This also means GitLab supports Microsoft Active Directory, Apple Open Directory, Open LDAP, and 389 Server. Other LDAP servers should work, too. -Active Directory also supports nested groups. Group sync will recursively -resolve membership if `active_directory: true` is set in the configuration file. +Active Directory also supports nested groups. Group sync recursively +resolves membership if `active_directory: true` is set in the configuration file. ##### Nested group memberships Nested group memberships are resolved only if the nested group -is found within the configured `group_base`. For example, if GitLab sees a +is found in the configured `group_base`. For example, if GitLab sees a nested group with DN `cn=nested_group,ou=special_groups,dc=example,dc=com` but the configured `group_base` is `ou=groups,dc=example,dc=com`, `cn=nested_group` is ignored. @@ -731,7 +731,7 @@ is ignored. - Each LDAP group is queried a maximum of one time with base `group_base` and filter `(cn=<cn_from_group_link>)`. -- If the LDAP group has the `memberuid` attribute, GitLab will execute another +- If the LDAP group has the `memberuid` attribute, GitLab executes another LDAP query per member to obtain each user's full DN. These queries are executed with base `base`, scope 'base object', and a filter depending on whether `user_filter` is set. Filter may be `(uid=<uid_from_group>)` or a @@ -750,9 +750,9 @@ LDAP group links each: - Subsequent syncs (checking membership, no writes) took 15 minutes These metrics are meant to provide a baseline and performance may vary based on -any number of factors. This was a pretty extreme benchmark and most instances will -not have near this many users or groups. Disk speed, database performance, -network and LDAP server response time will affect these metrics. +any number of factors. This was an extreme benchmark and most instances don't +have near this many users or groups. Disk speed, database performance, +network and LDAP server response time affects these metrics. ## Troubleshooting diff --git a/doc/administration/availability/index.md b/doc/administration/availability/index.md index 748373c8941..4f934646ed6 100644 --- a/doc/administration/availability/index.md +++ b/doc/administration/availability/index.md @@ -3,3 +3,6 @@ redirect_to: ../reference_architectures/index.md --- This document was moved to [another location](../reference_architectures/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/build_artifacts.md b/doc/administration/build_artifacts.md index 85ae2946bc8..e42f50b2e54 100644 --- a/doc/administration/build_artifacts.md +++ b/doc/administration/build_artifacts.md @@ -3,3 +3,6 @@ redirect_to: 'job_artifacts.md' --- This document was moved to [job_artifacts](job_artifacts.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/container_registry.md b/doc/administration/container_registry.md index 7a533aa9043..52941556237 100644 --- a/doc/administration/container_registry.md +++ b/doc/administration/container_registry.md @@ -3,3 +3,6 @@ redirect_to: 'packages/container_registry.md' --- This document was moved to [another location](packages/container_registry.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index 4cb8b15e4d8..0580540eef9 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -3,3 +3,6 @@ redirect_to: 'server_hooks.md' --- This document was moved to [another location](server_hooks.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/dependency_proxy.md b/doc/administration/dependency_proxy.md index 4683565998a..c0e0643b5de 100644 --- a/doc/administration/dependency_proxy.md +++ b/doc/administration/dependency_proxy.md @@ -3,3 +3,6 @@ redirect_to: 'packages/dependency_proxy.md' --- This document was moved to [another location](packages/dependency_proxy.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/external_pipeline_validation.md b/doc/administration/external_pipeline_validation.md index cb1d35f24d5..bc1a57e8bcd 100644 --- a/doc/administration/external_pipeline_validation.md +++ b/doc/administration/external_pipeline_validation.md @@ -14,9 +14,9 @@ This is an experimental feature and subject to change without notice. ## Usage -GitLab will send a POST request to the external service URL with the pipeline -data as payload. GitLab will then invalidate the pipeline based on the response -code. If there's an error or the request times out, the pipeline will not be +GitLab sends a POST request to the external service URL with the pipeline +data as payload. GitLab then invalidates the pipeline based on the response +code. If there's an error or the request times out, the pipeline is not invalidated. Response Code Legend: diff --git a/doc/administration/geo/disaster_recovery/promotion_runbook.md b/doc/administration/geo/disaster_recovery/promotion_runbook.md index 7eb6ef01aee..eef1e7ae9dd 100644 --- a/doc/administration/geo/disaster_recovery/promotion_runbook.md +++ b/doc/administration/geo/disaster_recovery/promotion_runbook.md @@ -3,3 +3,6 @@ redirect_to: runbooks/planned_failover_single_node.md --- This document was moved to [another location](runbooks/planned_failover_single_node.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/geo/index.md b/doc/administration/geo/index.md index 02b907ae237..fdea46c30ac 100644 --- a/doc/administration/geo/index.md +++ b/doc/administration/geo/index.md @@ -208,7 +208,9 @@ Omnibus GitLab-managed database. External databases are currently not supported. In some circumstances, like during [upgrades](replication/updating_the_geo_nodes.md) or a [planned failover](disaster_recovery/planned_failover.md), it is desirable to pause replication between the primary and secondary. -Pausing and resuming replication is done via a command line tool from the secondary node. +Pausing and resuming replication is done via a command line tool from the secondary node where the `postgresql` service is enabled. + +If `postgresql` is on a standalone database node, ensure that `gitlab.rb` on that node contains the configuration line `gitlab_rails['geo_node_name'] = 'node_name'`, where `node_name` is the same as the `geo_name_name` on the application node. **To Pause: (from secondary)** diff --git a/doc/administration/geo/replication/database.md b/doc/administration/geo/replication/database.md index da5376190b9..099a73e93d8 100644 --- a/doc/administration/geo/replication/database.md +++ b/doc/administration/geo/replication/database.md @@ -3,3 +3,6 @@ redirect_to: '../setup/database.md' --- This document was moved to [another location](../setup/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/geo/replication/external_database.md b/doc/administration/geo/replication/external_database.md index 0d0cd8a37d5..dfaf6819fa0 100644 --- a/doc/administration/geo/replication/external_database.md +++ b/doc/administration/geo/replication/external_database.md @@ -3,3 +3,6 @@ redirect_to: '../setup/external_database.md' --- This document was moved to [another location](../setup/external_database.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/geo/replication/high_availability.md b/doc/administration/geo/replication/high_availability.md index 214f15b7565..1da4246db1f 100644 --- a/doc/administration/geo/replication/high_availability.md +++ b/doc/administration/geo/replication/high_availability.md @@ -3,3 +3,6 @@ redirect_to: 'multiple_servers.md' --- This document was moved to [another location](multiple_servers.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/geo/replication/index.md b/doc/administration/geo/replication/index.md index 5e9dd73ecc5..955e05491d2 100644 --- a/doc/administration/geo/replication/index.md +++ b/doc/administration/geo/replication/index.md @@ -3,3 +3,6 @@ redirect_to: '../index.md' --- This document was moved to [another location](../index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/geo/replication/multiple_servers.md b/doc/administration/geo/replication/multiple_servers.md index 7d65d2165c5..9d5653bcc72 100644 --- a/doc/administration/geo/replication/multiple_servers.md +++ b/doc/administration/geo/replication/multiple_servers.md @@ -174,6 +174,12 @@ the **primary** database. Use the following as a guide. roles ['geo_secondary_role', 'postgres_role'] ## + ## The unique identifier for the Geo node. + ## This should match the secondary's application node. + ## + gitlab_rails['geo_node_name'] = '<node_name_here>' + + ## ## Secondary address ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node ## - replace '<tracking_database_ip>' with the public or VPC address of your Geo tracking database node diff --git a/doc/administration/geo/setup/database.md b/doc/administration/geo/setup/database.md index 24e55d26997..7fef2a02dd0 100644 --- a/doc/administration/geo/setup/database.md +++ b/doc/administration/geo/setup/database.md @@ -499,6 +499,8 @@ This experimental implementation has the following limitations: For instructions about how to set up Patroni on the primary node, see the [PostgreSQL replication and failover with Omnibus GitLab](../../postgresql/replication_and_failover.md#patroni) page. +If you are currently using `repmgr` on your Geo primary, see [these instructions](#migrating-from-repmgr-to-patroni) for migrating from `repmgr` to Patroni. + A production-ready and secure setup requires at least three Patroni instances on the primary, and a similar configuration on the secondary nodes. Be sure to use password credentials and other database best practices. @@ -549,6 +551,13 @@ patroni['standby_cluster']['primary_slot_name'] = 'geo_secondary' # or the uniqu patroni['replication_password'] = 'PLAIN_TEXT_POSTGRESQL_REPLICATION_PASSWORD' ``` +## Migrating from repmgr to Patroni + +1. Before migrating, it is recommended that there is no replication lag between the primary and secondary sites and that replication is paused. In GitLab 13.2 and later, you can pause and resume replication with `gitlab-ctl geo-replication-pause` and `gitlab-ctl geo-replication-resume` on a Geo secondary database node. +1. Follow the [instructions to migrate repmgr to Patroni](../../postgresql/replication_and_failover.md#switching-from-repmgr-to-patroni). When configuring Patroni on each primary site database node, add `patroni['replicaton_slots'] = { '<slot_name>' => 'physical' }` +to `gitlab.rb` where `<slot_name>` is the name of the replication slot for your Geo secondary. This will ensure that Patroni recognizes the replication slot as permanent and will not drop it upon restarting. +1. If database replication to the secondary was paused before migration, resume replication once Patroni is confirmed working on the primary. + ## Troubleshooting Read the [troubleshooting document](../replication/troubleshooting.md). diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 0d9c761e078..b8af4bf0f6a 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -434,8 +434,8 @@ server (with `gitaly_address`) unless you setup with special ``` NOTE: **Note:** - `/some/local/path` should be set to a local folder that exists, however no data will be stored in - this folder. This will no longer be necessary after + `/some/local/path` should be set to a local folder that exists, however no data is stored in + this folder. This requirement is scheduled to be removed when [this issue](https://gitlab.com/gitlab-org/gitaly/-/issues/1282) is resolved. 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source). @@ -461,7 +461,7 @@ GitLab can reside on the same server as one of many Gitaly servers, but doesn't configuration that mixes local and remote configuration. The following setup is incorrect, because: - All addresses must be reachable from the other Gitaly servers. -- `storage1` will be assigned a Unix socket for `gitaly_address` which is +- `storage1` is assigned a Unix socket for `gitaly_address` which is invalid for some of the Gitaly servers. ```ruby @@ -493,7 +493,7 @@ gitaly['key_path'] = "/etc/gitlab/ssl/key.pem" ``` `path` can only be included for storage shards on the local Gitaly server. -If it's excluded, default Git storage directory will be used for that storage shard. +If it's excluded, default Git storage directory is used for that storage shard. ### Disable Gitaly where not required (optional) @@ -529,12 +529,18 @@ To disable Gitaly on a GitLab server: ## Enable TLS support -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22602) in GitLab 11.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22602) in GitLab 11.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/3160) in GitLab 13.6, outgoing TLS connections to GitLab provide client certificates if configured. Gitaly supports TLS encryption. To communicate with a Gitaly instance that listens for secure connections, you must use `tls://` URL scheme in the `gitaly_address` of the corresponding storage entry in the GitLab configuration. +Gitaly provides the same server certificates as client certificates in TLS +connections to GitLab. This can be used as part of a mutual TLS authentication strategy +when combined with reverse proxies (for example, NGINX) that validate client certificate +to grant access to GitLab. + You must supply your own certificates as this isn't provided automatically. The certificate corresponding to each Gitaly server must be installed on that Gitaly server. @@ -584,7 +590,7 @@ To configure Gitaly with TLS: ``` 1. Copy all Gitaly server certificates (or their certificate authority) to - `/etc/gitlab/trusted-certs` so that Gitaly servers will trust the certificate when calling into themselves + `/etc/gitlab/trusted-certs` so that Gitaly servers trust the certificate when calling into themselves or other Gitaly servers: ```shell @@ -641,8 +647,8 @@ To configure Gitaly with TLS: ``` NOTE: **Note:** - `/some/local/path` should be set to a local folder that exists, however no data will be stored - in this folder. This will no longer be necessary after + `/some/local/path` should be set to a local folder that exists, however no data is stored + in this folder. This requirement is scheduled to be removed when [Gitaly issue #1282](https://gitlab.com/gitlab-org/gitaly/-/issues/1282) is resolved. 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source). @@ -662,7 +668,7 @@ To configure Gitaly with TLS: ``` 1. Copy all Gitaly server certificates (or their certificate authority) to the system trusted - certificates folder so Gitaly server will trust the certificate when calling into itself or other Gitaly + certificates folder so Gitaly server trusts the certificate when calling into itself or other Gitaly servers. ```shell @@ -800,7 +806,7 @@ You can observe the behavior of this queue using the Gitaly logs and Prometheus: NOTE: **Note:** Though the name of the Prometheus metric contains `rate_limiting`, it is a concurrency limiter, not -a rate limiter. If a Gitaly client makes 1000 requests in a row very quickly, concurrency will not +a rate limiter. If a Gitaly client makes 1000 requests in a row very quickly, concurrency does not exceed 1 and the concurrency limiter has no effect. ## Rotate Gitaly authentication token @@ -829,7 +835,7 @@ behavior of your GitLab installation using Prometheus. Use the following Prometh sum(rate(gitaly_authentications_total[5m])) by (enforced, status) ``` -In a system where authentication is configured correctly and where you have live traffic, you will +In a system where authentication is configured correctly and where you have live traffic, you see something like this: ```prometheus @@ -885,7 +891,7 @@ To update to a new Gitaly authentication token, on each Gitaly client **and** Gi ``` If you run your [Prometheus query](#verify-authentication-monitoring) while this change is -being rolled out, you will see non-zero values for the `enforced="false",status="denied"` counter. +being rolled out, you see non-zero values for the `enforced="false",status="denied"` counter. ### Ensure there are no authentication failures @@ -997,7 +1003,7 @@ When GitLab calls a function that has a "Rugged patch", it performs two checks: - Is the feature flag for this patch set in the database? If so, the feature flag setting controls GitLab's use of "Rugged patch" code. - If the feature flag is not set, GitLab tries accessing the filesystem underneath the - Gitaly server directly. If it can, it will use the "Rugged patch": + Gitaly server directly. If it can, it uses the "Rugged patch": - If using Unicorn. - If using Puma and [thread count](../../install/requirements.md#puma-threads) is set to `1`. @@ -1070,7 +1076,7 @@ gitaly-debug -h remote: GitLab: 401 Unauthorized ``` -You will need to sync your `gitlab-secrets.json` file with your Gitaly clients (GitLab +You need to sync your `gitlab-secrets.json` file with your Gitaly clients (GitLab app nodes). ### Client side gRPC logs diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index d091ae5895a..edee79ebee3 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -133,7 +133,7 @@ GitLab](https://about.gitlab.com/install/). - 3 Gitaly nodes (high CPU, high memory, fast storage) - 1 GitLab server -You will need the IP/host address for each node. +You need the IP/host address for each node. 1. `LOAD_BALANCER_SERVER_ADDRESS`: the IP/host address of the load balancer 1. `POSTGRESQL_SERVER_ADDRESS`: the IP/host address of the PostgreSQL server @@ -149,7 +149,7 @@ If you are using Google Cloud Platform, SoftLayer, or any other vendor that prov The communication between components is secured with different secrets, which are described below. Before you begin, generate a unique secret for each, and -make note of it. This will make it easy to replace these placeholder tokens +make note of it. This makes it easy to replace these placeholder tokens with secure tokens as you complete the setup process. 1. `GITLAB_SHELL_SECRET_TOKEN`: this is used by Git hooks to make callback HTTP @@ -164,7 +164,7 @@ with secure tokens as you complete the setup process. 1. `PRAEFECT_SQL_PASSWORD`: this password is used by Praefect to connect to PostgreSQL. -We will note in the instructions below where these secrets are required. +We note in the instructions below where these secrets are required. ### PostgreSQL @@ -184,13 +184,13 @@ failure. For greater fault tolerance, the following options are available: - Use a cloud-managed PostgreSQL service. AWS [Relational Database Service](https://aws.amazon.com/rds/) is recommended. -To complete this section you will need: +To complete this section you need: - 1 Praefect node - 1 PostgreSQL server (PostgreSQL 11 or newer) - An SQL user with permissions to create databases -During this section, we will configure the PostgreSQL server, from the Praefect +During this section, we configure the PostgreSQL server, from the Praefect node, using `psql` which is installed by Omnibus GitLab. 1. SSH into the **Praefect** node and login as root: @@ -207,7 +207,7 @@ node, using `psql` which is installed by Omnibus GitLab. /opt/gitlab/embedded/bin/psql -U postgres -d template1 -h POSTGRESQL_SERVER_ADDRESS ``` - Create a new user `praefect` which will be used by Praefect. Replace + Create a new user `praefect` to be used by Praefect. Replace `PRAEFECT_SQL_PASSWORD` with the strong password you generated in the preparation step. @@ -281,11 +281,10 @@ PostgreSQL instances. Otherwise you should change the configuration parameter NOTE: **Note:** If there are multiple Praefect nodes, complete these steps for **each** node. -To complete this section you will need: +To complete this section you need a [configured PostgreSQL server](#postgresql), including: -- [Configured PostgreSQL server](#postgresql), including: - - IP/host address (`POSTGRESQL_SERVER_ADDRESS`) - - password (`PRAEFECT_SQL_PASSWORD`) +- IP/host address (`POSTGRESQL_SERVER_ADDRESS`) +- Password (`PRAEFECT_SQL_PASSWORD`) Praefect should be run on a dedicated node. Do not run Praefect on the application server, or a Gitaly node. @@ -331,8 +330,8 @@ application server, or a Gitaly node. ``` 1. Configure a strong `auth_token` for **Praefect** by editing - `/etc/gitlab/gitlab.rb`. This will be needed by clients outside the cluster - (like GitLab Shell) to communicate with the Praefect cluster : + `/etc/gitlab/gitlab.rb`. This is needed by clients outside the cluster + (like GitLab Shell) to communicate with the Praefect cluster: ```ruby praefect['auth_token'] = 'PRAEFECT_EXTERNAL_TOKEN' @@ -341,7 +340,7 @@ application server, or a Gitaly node. 1. Configure **Praefect** to connect to the PostgreSQL database by editing `/etc/gitlab/gitlab.rb`. - You will need to replace `POSTGRESQL_SERVER_ADDRESS` with the IP/host address + You need to replace `POSTGRESQL_SERVER_ADDRESS` with the IP/host address of the database, and `PRAEFECT_SQL_PASSWORD` with the strong password set above. @@ -364,7 +363,7 @@ application server, or a Gitaly node. # praefect['database_sslrootcert'] = '/path/to/rootcert' ``` - By default Praefect will refuse to make an unencrypted connection to + By default, Praefect refuses to make an unencrypted connection to PostgreSQL. You can override this by uncommenting the following line: ```ruby @@ -377,7 +376,7 @@ application server, or a Gitaly node. The virtual storage's name must match the configured storage name in GitLab configuration. In a later step, we configure the storage name as `default` so we use `default` here as well. This cluster has three Gitaly nodes `gitaly-1`, - `gitaly-2`, and `gitaly-3`, which will be replicas of each other. + `gitaly-2`, and `gitaly-3`, which are intended to be replicas of each other. CAUTION: **Caution:** If you have data on an already existing storage called @@ -385,7 +384,7 @@ application server, or a Gitaly node. [migrate the data to the Gitaly Cluster storage](#migrate-existing-repositories-to-gitaly-cluster) afterwards. - Replace `PRAEFECT_INTERNAL_TOKEN` with a strong secret, which will be used by + Replace `PRAEFECT_INTERNAL_TOKEN` with a strong secret, which is used by Praefect when communicating with Gitaly nodes in the cluster. This token is distinct from the `PRAEFECT_EXTERNAL_TOKEN`. @@ -555,12 +554,12 @@ To configure Praefect with TLS: NOTE: **Note:** `/some/local/path` should be set to a local folder that exists, however no - data will be stored in this folder. This will no longer be necessary after + data is stored in this folder. This requirement is scheduled to be removed when [this issue](https://gitlab.com/gitlab-org/gitaly/-/issues/1282) is resolved. 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source). 1. Copy all Praefect server certificates, or their certificate authority, to the system - trusted certificates on each Gitaly server so the Praefect server will trust the + trusted certificates on each Gitaly server so the Praefect server trusts the certificate when called by Gitaly servers: ```shell @@ -585,7 +584,7 @@ To configure Praefect with TLS: NOTE: **Note:** Complete these steps for **each** Gitaly node. -To complete this section you will need: +To complete this section you need: - [Configured Praefect node](#praefect) - 3 (or more) servers, with GitLab installed, to be configured as Gitaly nodes. @@ -595,19 +594,19 @@ Every Gitaly server assigned to the Praefect cluster needs to be configured. The configuration is the same as a normal [standalone Gitaly server](index.md), except: -- the storage names are exposed to Praefect, not GitLab -- the secret token is shared with Praefect, not GitLab +- The storage names are exposed to Praefect, not GitLab +- The secret token is shared with Praefect, not GitLab The configuration of all Gitaly nodes in the Praefect cluster can be identical, because we rely on Praefect to route operations correctly. Particular attention should be shown to: -- the `gitaly['auth_token']` configured in this section must match the `token` +- The `gitaly['auth_token']` configured in this section must match the `token` value under `praefect['virtual_storages']` on the Praefect node. This was set in the [previous section](#praefect). This document uses the placeholder `PRAEFECT_INTERNAL_TOKEN` throughout. -- the storage names in `git_data_dirs` configured in this section must match the +- The storage names in `git_data_dirs` configured in this section must match the storage names under `praefect['virtual_storages']` on the Praefect node. This was set in the [previous section](#praefect). This document uses `gitaly-1`, `gitaly-2`, and `gitaly-3` as Gitaly storage names. @@ -659,8 +658,8 @@ documentation](index.md#configure-gitaly-servers). ``` 1. Configure a strong `auth_token` for **Gitaly** by editing - `/etc/gitlab/gitlab.rb`. This will be needed by clients to communicate with - this Gitaly nodes. Typically, this token will be the same for all Gitaly + `/etc/gitlab/gitlab.rb`. This is needed by clients to communicate with + this Gitaly nodes. Typically, this token is the same for all Gitaly nodes. ```ruby @@ -754,7 +753,7 @@ We hope that if you’re managing HA systems like GitLab, you have a load balanc of choice already. Some examples include [HAProxy](https://www.haproxy.org/) (open-source), [Google Internal Load Balancer](https://cloud.google.com/load-balancing/docs/internal/), [AWS Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/), F5 -Big-IP LTM, and Citrix Net Scaler. This documentation will outline what ports +Big-IP LTM, and Citrix Net Scaler. This documentation outlines what ports and protocols you need configure. | LB Port | Backend Port | Protocol | @@ -763,7 +762,7 @@ and protocols you need configure. ### GitLab -To complete this section you will need: +To complete this section you need: - [Configured Praefect node](#praefect) - [Configured Gitaly nodes](#gitaly) @@ -787,15 +786,15 @@ Particular attention should be shown to: 1. Configure the `external_url` so that files could be served by GitLab by proper endpoint access by editing `/etc/gitlab/gitlab.rb`: - You will need to replace `GITLAB_SERVER_URL` with the real external facing + You need to replace `GITLAB_SERVER_URL` with the real external facing URL on which current GitLab instance is serving: ```ruby external_url 'GITLAB_SERVER_URL' ``` -1. Disable the default Gitaly service running on the GitLab host. It won't be needed - as GitLab will connect to the configured cluster. +1. Disable the default Gitaly service running on the GitLab host. It isn't needed + because GitLab connects to the configured cluster. CAUTION: **Caution:** If you have existing data stored on the default Gitaly storage, @@ -809,7 +808,7 @@ Particular attention should be shown to: 1. Add the Praefect cluster as a storage location by editing `/etc/gitlab/gitlab.rb`. - You will need to replace: + You need to replace: - `LOAD_BALANCER_SERVER_ADDRESS` with the IP address or hostname of the load balancer. @@ -828,7 +827,7 @@ Particular attention should be shown to: nodes during a `git push` are properly authenticated by editing `/etc/gitlab/gitlab.rb`: - You will need to replace `GITLAB_SHELL_SECRET_TOKEN` with the real secret. + You need to replace `GITLAB_SHELL_SECRET_TOKEN` with the real secret. ```ruby gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN' @@ -837,7 +836,7 @@ Particular attention should be shown to: 1. Add Prometheus monitoring settings by editing `/etc/gitlab/gitlab.rb`. If Prometheus is enabled on a different node, make edits on that node instead. - You will need to replace: + You need to replace: - `PRAEFECT_HOST` with the IP address or hostname of the Praefect node - `GITALY_HOST` with the IP address or hostname of each Gitaly node @@ -922,7 +921,7 @@ To get started quickly: gitlab-ctl reconfigure ``` -1. Set the Grafana admin password. This command will prompt you to enter a new +1. Set the Grafana admin password. This command prompts you to enter a new password: ```shell @@ -966,7 +965,7 @@ _Up to date_ in this context means that: - The last replication operation is in _completed_ state. If there is no such nodes, or any other error occurs during node selection, the primary -node will be chosen to serve the request. +node is chosen to serve the request. To track distribution of read operations, you can use the `gitaly_praefect_read_distribution` Prometheus counter metric. It has two labels: @@ -1040,9 +1039,9 @@ current primary node is found to be unhealthy. - **PostgreSQL (recommended):** Enabled by default, and equivalent to: `praefect['failover_election_strategy'] = sql`. This configuration - option will allow multiple Praefect nodes to coordinate via the + option allows multiple Praefect nodes to coordinate via the PostgreSQL database to elect a primary Gitaly node. This configuration - will cause Praefect nodes to elect a new primary, monitor its health, + causes Praefect nodes to elect a new primary, monitor its health, and elect a new primary if the current one has not been reachable in 10 seconds by a majority of the Praefect nodes. - **Memory:** Enabled by setting `praefect['failover_election_strategy'] = 'local'` @@ -1051,8 +1050,7 @@ current primary node is found to be unhealthy. be elected. **Do not use with multiple Praefect nodes!** Using with multiple Praefect nodes is likely to result in a split brain. -It is likely that we will implement support for Consul, and a cloud native -strategy in the future. +We are likely to implement support for Consul, and a cloud native, strategy in the future. ## Primary Node Failure diff --git a/doc/administration/gitaly/reference.md b/doc/administration/gitaly/reference.md index 53001b946d8..3ba6783ae3a 100644 --- a/doc/administration/gitaly/reference.md +++ b/doc/administration/gitaly/reference.md @@ -75,7 +75,7 @@ the `gitaly_authentications_total` metric. ### TLS -Gitaly supports TLS encryption. You will need to bring your own certificates as +Gitaly supports TLS encryption. You need to bring your own certificates as this isn't provided automatically. | Name | Type | Required | Description | @@ -128,7 +128,7 @@ The following values can be set in the `[git]` section of the configuration file | Name | Type | Required | Description | | ---- | ---- | -------- | ----------- | -| `bin_path` | string | no | Path to Git binary. If not set, will be resolved using `PATH`. | +| `bin_path` | string | no | Path to Git binary. If not set, is resolved using `PATH`. | | `catfile_cache_size` | integer | no | Maximum number of cached [cat-file processes](#cat-file-cache). Default is `100`. | #### `cat-file` cache @@ -192,8 +192,8 @@ For historical reasons [GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell) contains the Git hooks that allow GitLab to validate and react to Git pushes. Because Gitaly "owns" Git pushes, GitLab Shell must therefore be -installed alongside Gitaly. This will be [simplified in the -future](https://gitlab.com/gitlab-org/gitaly/-/issues/1226). +installed alongside Gitaly. We plan to +[simplify this](https://gitlab.com/gitlab-org/gitaly/-/issues/1226). | Name | Type | Required | Description | | ---- | ---- | -------- | ----------- | @@ -238,9 +238,11 @@ The following values configure logging in Gitaly under the `[logging]` section. While the main Gitaly application logs go to `stdout`, there are some extra log files that go to a configured directory, like the GitLab Shell logs. -GitLab Shell does not support `panic` or `trace` level logs. `panic` will fall -back to `error`, while `trace` will fall back to `debug`. Any other invalid log -levels will default to `info`. +GitLab Shell does not support `panic` or `trace` level logs: + +- `panic` falls back to `error`. +- `trace` falls back to `debug`. +- Any other invalid log levels default to `info`. Example: diff --git a/doc/administration/index.md b/doc/administration/index.md index 0aa94b86371..f667bdb5d2e 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -135,7 +135,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Issue closing pattern](issue_closing_pattern.md): Customize how to close an issue from commit messages. - [Gitaly](gitaly/index.md): Configuring Gitaly, GitLab's Git repository storage service. -- [Default labels](../user/admin_area/labels.md): Create labels that will be automatically added to every new project. +- [Default labels](../user/admin_area/labels.md): Create labels that are automatically added to every new project. - [Restrict the use of public or internal projects](../public_access/public_access.md#restricting-the-use-of-public-or-internal-projects): Restrict the use of visibility levels for users when they create a project or a snippet. - [Custom project templates](../user/admin_area/custom_project_templates.md): Configure a set of projects to be used as custom templates when creating a new project. **(PREMIUM ONLY)** diff --git a/doc/administration/instance_review.md b/doc/administration/instance_review.md index b00586b281b..6af016c873d 100644 --- a/doc/administration/instance_review.md +++ b/doc/administration/instance_review.md @@ -21,5 +21,5 @@ you qualify for a free Instance Review. 1. GitLab redirects you to a form with prefilled data obtained from your instance. 1. Click **Submit** to see the initial report. -A GitLab team member will contact you for further review, to provide suggestions -that will help you improve your usage of GitLab. +You will be contacted by a GitLab team member for further review, to provide suggestions +intended to help you improve your usage of GitLab. diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index a010903013e..aca5b321262 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -118,14 +118,14 @@ This section describes the earlier configuration format. For source installations the following settings are nested under `artifacts:` and then `object_store:`. On Omnibus GitLab installs they are prefixed by `artifacts_object_store_`. -| Setting | Description | Default | -|---------|-------------|---------| -| `enabled` | Enable/disable object storage | `false` | -| `remote_directory` | The bucket name where Artifacts will be stored| | -| `direct_upload` | Set to `true` to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. | `false` | -| `background_upload` | Set to `false` to disable automatic upload. Option may be removed once upload is direct to S3 | `true` | -| `proxy_download` | Set to `true` to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` | -| `connection` | Various connection options described below | | +| Setting | Default | Description | +|---------------------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `enabled` | `false` | Enable/disable object storage | +| `remote_directory` | | The bucket name where Artifacts are stored | +| `direct_upload` | `false` | Set to `true` to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. | +| `background_upload` | `true` | Set to `false` to disable automatic upload. Option may be removed once upload is direct to S3 | +| `proxy_download` | `false` | Set to `true` to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data. | +| `connection` | | Various connection options described below | #### Connection settings @@ -336,7 +336,7 @@ To migrate back to local storage: If [`artifacts:expire_in`](../ci/yaml/README.md#artifactsexpire_in) is used to set an expiry for the artifacts, they are marked for deletion right after that date passes. -Otherwise, they will expire per the [default artifacts expiration setting](../user/admin_area/settings/continuous_integration.md). +Otherwise, they expire per the [default artifacts expiration setting](../user/admin_area/settings/continuous_integration.md). Artifacts are cleaned up by the `expire_build_artifacts_worker` cron job which Sidekiq runs every hour at 50 minutes (`50 * * * *`). @@ -367,7 +367,7 @@ steps below. 1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. -If the `expire` directive is not set explicitly in your pipeline, artifacts will expire per the +If the `expire` directive is not set explicitly in your pipeline, artifacts expire per the default artifacts expiration setting, which you can find in the [CI/CD Admin settings](../user/admin_area/settings/continuous_integration.md). ## Validation for dependencies @@ -444,7 +444,7 @@ reasons are: - The number of jobs run, and hence artifacts generated, is higher than expected. - Job logs are larger than expected, and have accumulated over time. -In these and other cases, you'll need to identify the projects most responsible +In these and other cases, identify the projects most responsible for disk space usage, figure out what types of artifacts are using the most space, and in some cases, manually delete job artifacts to reclaim disk space. @@ -508,7 +508,7 @@ If you need to manually remove job artifacts associated with multiple jobs while 1. Delete job artifacts older than a specific date: NOTE: **Note:** - This step will also erase artifacts that users have chosen to + This step also erases artifacts that users have chosen to ["keep"](../ci/pipelines/job_artifacts.md#browsing-artifacts). ```ruby @@ -552,7 +552,7 @@ If you need to manually remove **all** job artifacts associated with multiple jo builds_with_artifacts = Ci::Build.with_existing_job_artifacts(Ci::JobArtifact.trace) ``` -1. Select the user which will be mentioned in the web UI as erasing the job: +1. Select the user which is mentioned in the web UI as erasing the job: ```ruby admin_user = User.find_by(username: 'username') diff --git a/doc/administration/job_traces.md b/doc/administration/job_traces.md index d0b346a931e..2dbcf5d6705 100644 --- a/doc/administration/job_traces.md +++ b/doc/administration/job_traces.md @@ -3,3 +3,6 @@ redirect_to: 'job_logs.md' --- This document was moved to [another location](job_logs.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/lfs/lfs_administration.md b/doc/administration/lfs/lfs_administration.md index 7ace0ec5a93..a275efce5bb 100644 --- a/doc/administration/lfs/lfs_administration.md +++ b/doc/administration/lfs/lfs_administration.md @@ -3,3 +3,6 @@ redirect_to: 'index.md' --- This document was moved to [another location](index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/lfs/manage_large_binaries_with_git_lfs.md b/doc/administration/lfs/manage_large_binaries_with_git_lfs.md index 4656bccf5e6..69c401acb86 100644 --- a/doc/administration/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/administration/lfs/manage_large_binaries_with_git_lfs.md @@ -3,3 +3,6 @@ redirect_to: '../../topics/git/lfs/index.md' --- This document was moved to [another location](../../topics/git/lfs/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md b/doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md index dd98a50e353..16095c1dbf6 100644 --- a/doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md +++ b/doc/administration/lfs/migrate_from_git_annex_to_git_lfs.md @@ -3,3 +3,6 @@ redirect_to: '../../topics/git/lfs/migrate_from_git_annex_to_git_lfs.md' --- This document was moved to [another location](../../topics/git/lfs/migrate_from_git_annex_to_git_lfs.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/libravatar.md b/doc/administration/libravatar.md index a92e6fade03..5d92a0162bf 100644 --- a/doc/administration/libravatar.md +++ b/doc/administration/libravatar.md @@ -41,7 +41,7 @@ the configuration options as follows: ### Your own Libravatar server If you are [running your own Libravatar service](https://wiki.libravatar.org/running_your_own/), -the URL will be different in the configuration, but you must provide the same +the URL is different in the configuration, but you must provide the same placeholders so GitLab can parse the URL correctly. For example, you host a service on `http://libravatar.example.com` and the diff --git a/doc/administration/load_balancer.md b/doc/administration/load_balancer.md index 410381ff2b0..1ab87fb4b4c 100644 --- a/doc/administration/load_balancer.md +++ b/doc/administration/load_balancer.md @@ -7,17 +7,17 @@ type: reference # Load Balancer for multi-node GitLab -In an multi-node GitLab configuration, you will need a load balancer to route +In an multi-node GitLab configuration, you need a load balancer to route traffic to the application servers. The specifics on which load balancer to use or the exact configuration is beyond the scope of GitLab documentation. We hope that if you're managing HA systems like GitLab you have a load balancer of choice already. Some examples including HAProxy (open-source), F5 Big-IP LTM, -and Citrix Net Scaler. This documentation will outline what ports and protocols +and Citrix Net Scaler. This documentation outlines what ports and protocols you need to use with GitLab. ## SSL -How will you handle SSL in your multi-node environment? There are several different +How do you want to handle SSL in your multi-node environment? There are several different options: - Each application node terminates SSL @@ -29,8 +29,8 @@ options: ### Application nodes terminate SSL Configure your load balancer(s) to pass connections on port 443 as 'TCP' rather -than 'HTTP(S)' protocol. This will pass the connection to the application nodes -NGINX service untouched. NGINX will have the SSL certificate and listen on port 443. +than 'HTTP(S)' protocol. This passes the connection to the application nodes +NGINX service untouched. NGINX has the SSL certificate and listen on port 443. See [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) for details on managing SSL certificates and configuring NGINX. @@ -38,10 +38,10 @@ for details on managing SSL certificates and configuring NGINX. ### Load Balancer(s) terminate SSL without backend SSL Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. -The load balancer(s) will then be responsible for managing SSL certificates and +The load balancer(s) is be responsible for managing SSL certificates and terminating SSL. -Since communication between the load balancer(s) and GitLab will not be secure, +Since communication between the load balancer(s) and GitLab isn't secure, there is some additional configuration needed. See [NGINX Proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) for details. @@ -49,12 +49,12 @@ for details. ### Load Balancer(s) terminate SSL with backend SSL Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. -The load balancer(s) will be responsible for managing SSL certificates that -end users will see. +The load balancer(s) is responsible for managing SSL certificates that +end users see. -Traffic will also be secure between the load balancer(s) and NGINX in this +Traffic is secure between the load balancer(s) and NGINX in this scenario. There is no need to add configuration for proxied SSL since the -connection will be secure all the way. However, configuration will need to be +connection is secure all the way. However, configuration must be added to GitLab to configure SSL certificates. See [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) for details on managing SSL certificates and configuring NGINX. @@ -75,13 +75,13 @@ for details on managing SSL certificates and configuring NGINX. to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the [web terminal](integration/terminal.md) integration guide for more details. -- (*2*): When using HTTPS protocol for port 443, you will need to add an SSL +- (*2*): When using HTTPS protocol for port 443, you must add an SSL certificate to the load balancers. If you wish to terminate SSL at the GitLab application server instead, use TCP protocol. ### GitLab Pages Ports -If you're using GitLab Pages with custom domain support you will need some +If you're using GitLab Pages with custom domain support you need some additional port configurations. GitLab Pages requires a separate virtual IP address. Configure DNS to point the `pages_external_url` from `/etc/gitlab/gitlab.rb` at the new virtual IP address. See the @@ -103,7 +103,7 @@ GitLab Pages requires a separate virtual IP address. Configure DNS to point the Some organizations have policies against opening SSH port 22. In this case, it may be helpful to configure an alternate SSH hostname that allows users -to use SSH on port 443. An alternate SSH hostname will require a new virtual IP address +to use SSH on port 443. An alternate SSH hostname requires a new virtual IP address compared to the other GitLab HTTP configuration above. Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`. @@ -114,7 +114,7 @@ Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`. ## Readiness check -It is strongly recommend that multi-node deployments configure load balancers to use the [readiness check](../user/admin_area/monitoring/health_check.md#readiness) to ensure a node is ready to accept traffic, before routing traffic to it. This is especially important when utilizing Puma, as there is a brief period during a restart where Puma will not accept requests. +It is strongly recommend that multi-node deployments configure load balancers to use the [readiness check](../user/admin_area/monitoring/health_check.md#readiness) to ensure a node is ready to accept traffic, before routing traffic to it. This is especially important when utilizing Puma, as there is a brief period during a restart where Puma doesn't accept requests. <!-- ## Troubleshooting diff --git a/doc/administration/logs.md b/doc/administration/logs.md index e5523ba67aa..156990cc4d6 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -979,6 +979,9 @@ When [troubleshooting](troubleshooting/index.md) issues that aren't localized to previously listed components, it's helpful to simultaneously gather multiple logs and statistics from a GitLab instance. +NOTE: **Note:** +GitLab Support will often ask for one of these, and maintains the required tools. + ### Briefly tail the main logs If the bug or error is readily reproducible, save the main GitLab logs @@ -995,12 +998,9 @@ Conclude the log gathering with <kbd>Ctrl</kbd> + <kbd>C</kbd>. If performance degradations or cascading errors occur that can't readily be attributed to one of the previously listed GitLab components, [GitLabSOS](https://gitlab.com/gitlab-com/support/toolbox/gitlabsos/) -can provide a perspective spanning all of Omnibus GitLab. For more details and instructions +can provide a broader perspective of the GitLab instance. For more details and instructions to run it, read [the GitLabSOS documentation](https://gitlab.com/gitlab-com/support/toolbox/gitlabsos/#gitlabsos). -NOTE: **Note:** -GitLab Support likes to use this custom-made tool. - ### Fast-stats [Fast-stats](https://gitlab.com/gitlab-com/support/toolbox/fast-stats) is a tool diff --git a/doc/administration/maven_packages.md b/doc/administration/maven_packages.md index cdcce79f8f9..690b6785941 100644 --- a/doc/administration/maven_packages.md +++ b/doc/administration/maven_packages.md @@ -3,3 +3,6 @@ redirect_to: 'packages/index.md' --- This document was moved to [another location](packages/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/maven_repository.md b/doc/administration/maven_repository.md index cdcce79f8f9..690b6785941 100644 --- a/doc/administration/maven_repository.md +++ b/doc/administration/maven_repository.md @@ -3,3 +3,6 @@ redirect_to: 'packages/index.md' --- This document was moved to [another location](packages/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/monitoring/gitlab_instance_administration_project/index.md b/doc/administration/monitoring/gitlab_instance_administration_project/index.md index 1235eb2edec..59837dcdb20 100644 --- a/doc/administration/monitoring/gitlab_instance_administration_project/index.md +++ b/doc/administration/monitoring/gitlab_instance_administration_project/index.md @@ -3,3 +3,6 @@ redirect_to: '../gitlab_self_monitoring_project/index.md' --- This document was moved to [another location](../gitlab_self_monitoring_project/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md index fcaf9aaaaee..e4a4e9663ff 100644 --- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md +++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md @@ -14,13 +14,13 @@ health of their GitLab instance. To surface this experience in a native way (similar to how you would interact with an application deployed using GitLab), a base project called "GitLab self monitoring" with [internal visibility](../../../public_access/public_access.md#internal-projects) -will be added under a group called "GitLab Instance Administrators" +is added under a group called "GitLab Instance Administrators" specifically created for visualizing and configuring the monitoring of your GitLab instance. -All administrators at the time of creation of the project and group will be -added as maintainers of the group and project, and as an admin, you'll be able -to add new members to the group to give them maintainer access to the project. +All administrators at the time of creation of the project and group are +added as maintainers of the group and project, and as an admin, you can +add new members to the group to give them maintainer access to the project. This project is used to self monitor your GitLab instance. The metrics dashboard of the project shows some basic resource usage charts, such as CPU and memory usage @@ -34,13 +34,13 @@ metrics exposed by the [GitLab exporter](../prometheus/gitlab_metrics.md#metrics 1. Navigate to **Admin Area > Settings > Metrics and profiling**, and expand the **Self monitoring** section. 1. Toggle the **Create Project** button on. -1. Once your GitLab instance creates the project, you'll see a link to the project in the text above the **Create Project** toggle. You can also find it under **Projects > Your projects**. +1. Once your GitLab instance creates the project, GitLab displays a link to the project in the text above the **Create Project** toggle. You can also find it under **Projects > Your projects**. ## Deleting the self monitoring project CAUTION: **Warning:** -If you delete the self monitoring project, you will lose any changes made to the -project. If you create the project again, it will be created in its default state. +Deleting the self monitoring project removes any changes made to the project. If +you create the project again, it's created in its default state. 1. Navigate to **Admin Area > Settings > Metrics and profiling**, and expand the **Self monitoring** section. 1. Toggle the **Create Project** button off. @@ -63,7 +63,7 @@ You can also ## Connection to Prometheus -The project will be automatically configured to connect to the +The project is automatically configured to connect to the [internal Prometheus](../prometheus/index.md) instance if the Prometheus instance is present (should be the case if GitLab was installed via Omnibus and you haven't disabled it). diff --git a/doc/administration/monitoring/performance/introduction.md b/doc/administration/monitoring/performance/introduction.md index 7ace0ec5a93..a275efce5bb 100644 --- a/doc/administration/monitoring/performance/introduction.md +++ b/doc/administration/monitoring/performance/introduction.md @@ -3,3 +3,6 @@ redirect_to: 'index.md' --- This document was moved to [another location](index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/monitoring/performance/prometheus.md b/doc/administration/monitoring/performance/prometheus.md index f05a420fc19..2978de865e2 100644 --- a/doc/administration/monitoring/performance/prometheus.md +++ b/doc/administration/monitoring/performance/prometheus.md @@ -3,3 +3,6 @@ redirect_to: '../prometheus/index.md' --- This document was moved to [another location](../prometheus/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md b/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md index ae3a3d739b5..8fa2e16dcd0 100644 --- a/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md +++ b/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md @@ -3,3 +3,6 @@ redirect_to: 'gitlab_exporter.md' --- This document was moved to [another location](gitlab_exporter.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index 91f810dc681..22c59c5a7fb 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -10,10 +10,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w > > - Prometheus and the various exporters listed in this page are bundled in the > Omnibus GitLab package. Check each exporter's documentation for the timeline -> they got added. For installations from source you will have to install them -> yourself. Over subsequent releases additional GitLab metrics will be captured. +> they got added. For installations from source you must install them +> yourself. Over subsequent releases additional GitLab metrics are captured. > - Prometheus services are on by default with GitLab 9.0. -> - Prometheus and its exporters don't authenticate users, and will be available +> - Prometheus and its exporters don't authenticate users, and are available > to anyone who can access them. [Prometheus](https://prometheus.io) is a powerful time-series monitoring service, providing a flexible @@ -34,9 +34,9 @@ dashboard tool like [Grafana](https://grafana.com). For installations from source, you must install and configure it yourself. Prometheus and its exporters are on by default, starting with GitLab 9.0. -Prometheus will run as the `gitlab-prometheus` user and listen on +Prometheus runs as the `gitlab-prometheus` user and listen on `http://localhost:9090`. By default, Prometheus is only accessible from the GitLab server itself. -Each exporter will be automatically set up as a +Each exporter is automatically set up as a monitoring target for Prometheus, unless individually disabled. To disable Prometheus and all of its exporters, as well as any added in the future: @@ -179,8 +179,7 @@ The next step is to tell all the other nodes where the monitoring node is: After monitoring using Service Discovery is enabled with `consul['monitoring_service_discovery'] = true`, ensure that `prometheus['scrape_configs']` is not set in `/etc/gitlab/gitlab.rb`. Setting both -`consul['monitoring_service_discovery'] = true` and `prometheus['scrape_configs']` in `/etc/gitlab/gitlab.rb` -will result in errors. +`consul['monitoring_service_discovery'] = true` and `prometheus['scrape_configs']` in `/etc/gitlab/gitlab.rb` results in errors. ### Using an external Prometheus server @@ -234,7 +233,7 @@ To use an external Prometheus server: gitlab_rails['prometheus_address'] = '192.168.0.1:9090' ``` -1. To scrape NGINX metrics, you'll also need to configure NGINX to allow the Prometheus server +1. To scrape NGINX metrics, you must also configure NGINX to allow the Prometheus server IP. For example: ```ruby @@ -399,7 +398,7 @@ The GitLab exporter allows you to measure various GitLab metrics, pulled from Re > - Introduced in GitLab 9.0. > - Pod monitoring introduced in GitLab 9.4. -If your GitLab server is running within Kubernetes, Prometheus will collect metrics from the Nodes and [annotated Pods](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config) in the cluster, including performance data on each container. This is particularly helpful if your CI/CD environments run in the same cluster, as you can use the [Prometheus project integration](../../../user/project/integrations/prometheus.md) to monitor them. +If your GitLab server is running within Kubernetes, Prometheus collects metrics from the Nodes and [annotated Pods](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config) in the cluster, including performance data on each container. This is particularly helpful if your CI/CD environments run in the same cluster, as you can use the [Prometheus project integration](../../../user/project/integrations/prometheus.md) to monitor them. To disable the monitoring of Kubernetes: diff --git a/doc/administration/monitoring/prometheus/node_exporter.md b/doc/administration/monitoring/prometheus/node_exporter.md index fea78a3685c..b8d01f87e9f 100644 --- a/doc/administration/monitoring/prometheus/node_exporter.md +++ b/doc/administration/monitoring/prometheus/node_exporter.md @@ -24,5 +24,5 @@ To enable the node exporter: 1. Save the file, and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -Prometheus will now begin collecting performance data from the node exporter +Prometheus begins collecting performance data from the node exporter exposed at `localhost:9100`. diff --git a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md index ff0cfc65e10..0029a6867c7 100644 --- a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md +++ b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md @@ -26,9 +26,9 @@ To enable the PgBouncer exporter: 1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -Prometheus will now begin collecting performance data from the PgBouncer exporter +Prometheus begins collecting performance data from the PgBouncer exporter exposed at `localhost:9188`. -The PgBouncer exporter will also be enabled by default if the +The PgBouncer exporter is enabled by default if the [`pgbouncer_role`](https://docs.gitlab.com/omnibus/roles/#postgresql-roles) role is enabled. diff --git a/doc/administration/monitoring/prometheus/postgres_exporter.md b/doc/administration/monitoring/prometheus/postgres_exporter.md index f7368556235..b28d541371f 100644 --- a/doc/administration/monitoring/prometheus/postgres_exporter.md +++ b/doc/administration/monitoring/prometheus/postgres_exporter.md @@ -21,12 +21,12 @@ To enable the PostgreSQL Server Exporter: If PostgreSQL Server Exporter is configured on a separate node, make sure that the local address is [listed in `trust_auth_cidr_addresses`](../../postgresql/replication_and_failover.md#network-information) or the - exporter will not be able to connect to the database. + exporter can't connect to the database. 1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -Prometheus will now automatically begin collecting performance data from +Prometheus begins collecting performance data from the PostgreSQL Server Exporter exposed under `localhost:9187`. ## Advanced configuration diff --git a/doc/administration/monitoring/prometheus/redis_exporter.md b/doc/administration/monitoring/prometheus/redis_exporter.md index 41a84f1f3ed..024624f5c67 100644 --- a/doc/administration/monitoring/prometheus/redis_exporter.md +++ b/doc/administration/monitoring/prometheus/redis_exporter.md @@ -25,5 +25,5 @@ To enable the Redis exporter: 1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -Prometheus will now begin collecting performance data from +Prometheus begins collecting performance data from the Redis exporter exposed at `localhost:9121`. diff --git a/doc/administration/monitoring/prometheus/registry_exporter.md b/doc/administration/monitoring/prometheus/registry_exporter.md index 3bf4db8a665..c3cfacd835a 100644 --- a/doc/administration/monitoring/prometheus/registry_exporter.md +++ b/doc/administration/monitoring/prometheus/registry_exporter.md @@ -21,7 +21,7 @@ To enable it: 1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -Prometheus will now automatically begin collecting performance data from +Prometheus automatically begins collecting performance data from the registry exporter exposed under `localhost:5001/metrics`. [← Back to the main Prometheus page](index.md) diff --git a/doc/administration/npm_registry.md b/doc/administration/npm_registry.md index cdcce79f8f9..690b6785941 100644 --- a/doc/administration/npm_registry.md +++ b/doc/administration/npm_registry.md @@ -3,3 +3,6 @@ redirect_to: 'packages/index.md' --- This document was moved to [another location](packages/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/operations.md b/doc/administration/operations.md index 9cd78105bbb..cfabb5ec27d 100644 --- a/doc/administration/operations.md +++ b/doc/administration/operations.md @@ -3,3 +3,6 @@ redirect_to: 'operations/index.md' --- This document was moved to [another location](operations/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/operations/speed_up_ssh.md b/doc/administration/operations/speed_up_ssh.md index 6dc83c42f53..2f3cf40a222 100644 --- a/doc/administration/operations/speed_up_ssh.md +++ b/doc/administration/operations/speed_up_ssh.md @@ -3,3 +3,6 @@ redirect_to: 'fast_ssh_key_lookup.md' --- This document was moved to [another location](fast_ssh_key_lookup.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/packages.md b/doc/administration/packages.md index cdcce79f8f9..690b6785941 100644 --- a/doc/administration/packages.md +++ b/doc/administration/packages.md @@ -3,3 +3,6 @@ redirect_to: 'packages/index.md' --- This document was moved to [another location](packages/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/packages/container_registry.md b/doc/administration/packages/container_registry.md index 541bd99084c..a9ec2764b80 100644 --- a/doc/administration/packages/container_registry.md +++ b/doc/administration/packages/container_registry.md @@ -820,7 +820,7 @@ If you did not change the default location of the configuration file, run: sudo gitlab-ctl registry-garbage-collect ``` -This command will take some time to complete, depending on the amount of +This command takes some time to complete, depending on the amount of layers you have stored. If you changed the location of the Container Registry `config.yml`: @@ -867,7 +867,7 @@ You can perform garbage collection without stopping the Container Registry by pu it in read-only mode and by not using the built-in command. On large instances this could require Container Registry to be in read-only mode for a while. During this time, -you will be able to pull from the Container Registry, but you will not be able to +you are able to pull from the Container Registry, but you are not able to push. By default, the [registry storage path](#configure-storage-for-the-container-registry) @@ -896,7 +896,7 @@ To enable the read-only mode: sudo gitlab-ctl reconfigure ``` - This will set the Container Registry into the read only mode. + This command sets the Container Registry into the read only mode. 1. Next, trigger one of the garbage collect commands: @@ -908,7 +908,7 @@ To enable the read-only mode: sudo /opt/gitlab/embedded/bin/registry garbage-collect -m /var/opt/gitlab/registry/config.yml ``` - This will start the garbage collection, which might take some time to complete. + This command starts the garbage collection, which might take some time to complete. 1. Once done, in `/etc/gitlab/gitlab.rb` change it back to read-write mode: @@ -935,7 +935,7 @@ To enable the read-only mode: Ideally, you want to run the garbage collection of the registry regularly on a weekly basis at a time when the registry is not being in-use. -The simplest way is to add a new crontab job that it will run periodically +The simplest way is to add a new crontab job that it runs periodically once a week. Create a file under `/etc/cron.d/registry-garbage-collect`: @@ -1155,7 +1155,7 @@ curl localhost:5001/debug/vars ### Advanced Troubleshooting -We will use a concrete example in the past to illustrate how to +We use a concrete example to illustrate how to diagnose a problem with the S3 setup. #### Unexpected 403 error during push @@ -1227,14 +1227,14 @@ To verify that the certificates are properly installed, run: mitmproxy --port 9000 ``` -This will run mitmproxy on port `9000`. In another window, run: +This command runs mitmproxy on port `9000`. In another window, run: ```shell curl --proxy http://localhost:9000 https://httpbin.org/status/200 ``` -If everything is set up correctly, you will see information on the mitmproxy window and -no errors from the curl commands. +If everything is set up correctly, information is displayed on the mitmproxy window and +no errors are generated by the curl commands. #### Running the Docker daemon with a proxy @@ -1248,7 +1248,7 @@ export HTTPS_PROXY="https://localhost:9000" docker daemon --debug ``` -This will launch the Docker daemon and proxy all connections through mitmproxy. +This command launches the Docker daemon and proxies all connections through mitmproxy. #### Running the Docker client @@ -1273,4 +1273,4 @@ The above image shows: What does this mean? This strongly suggests that the S3 user does not have the right [permissions to perform a HEAD request](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html). The solution: check the [IAM permissions again](https://docs.docker.com/registry/storage-drivers/s3/). -Once the right permissions were set, the error will go away. +Once the right permissions were set, the error goes away. diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 0ebeb96d247..32812cb1e9a 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -811,3 +811,11 @@ Upgrading to an [officially supported operating system](https://about.gitlab.com This problem comes from the permissions of the GitLab Pages OAuth application. To fix it, go to **Admin > Applications > GitLab Pages** and edit the application. Under **Scopes**, ensure that the `api` scope is selected and save your changes. + +### Workaround in case no wildcard DNS entry can be set + +If the wildcard DNS [prerequisite](#prerequisites) can't be met, you can still use GitLab Pages in a limited fashion: + +1. [Move](../../user/project/settings/index.md#transferring-an-existing-project-into-another-namespace) + all projects you need to use Pages with into a single group namespace, for example `pages`. +1. Configure a [DNS entry](#dns-configuration) without the `*.`-wildcard, for example `pages.example.io`. diff --git a/doc/administration/plugins.md b/doc/administration/plugins.md index dbb733b9b19..f5d4b3aa2ff 100644 --- a/doc/administration/plugins.md +++ b/doc/administration/plugins.md @@ -3,3 +3,6 @@ redirect_to: 'file_hooks.md' --- This document was moved to [File Hooks](file_hooks.md), after the Plugins feature was renamed to File Hooks. + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/pseudonymizer.md b/doc/administration/pseudonymizer.md index 41a7ec087ac..2098708d6d9 100644 --- a/doc/administration/pseudonymizer.md +++ b/doc/administration/pseudonymizer.md @@ -28,7 +28,7 @@ To configure the pseudonymizer, you need to: - Provide a manifest file that describes which fields should be included or pseudonymized ([example `manifest.yml` file](https://gitlab.com/gitlab-org/gitlab/tree/master/config/pseudonymizer.yml)). - A default manifest is provided with the GitLab installation. Using a relative file path will be resolved from the Rails root. + A default manifest is provided with the GitLab installation, using a relative file path that resolves from the Rails root. Alternatively, you can use an absolute file path. - Use an object storage and specify the connection parameters in the `pseudonymizer.upload.connection` configuration option. @@ -100,7 +100,7 @@ sudo gitlab-rake gitlab:db:pseudonymizer sudo -u git -H bundle exec rake gitlab:db:pseudonymizer RAILS_ENV=production ``` -This will produce some CSV files that might be very large, so make sure the +This produces some CSV files that might be very large, so make sure the `PSEUDONYMIZER_OUTPUT_DIR` has sufficient space. As a rule of thumb, at least 10% of the database size is recommended. diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index f61a3107b3d..13f23815760 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -56,7 +56,7 @@ sudo -u git -H bundle exec rake gitlab:git:fsck RAILS_ENV=production Various types of files can be uploaded to a GitLab installation by users. These integrity checks can detect missing files. Additionally, for locally stored files, checksums are generated and stored in the database upon upload, -and these checks will verify them against current files. +and these checks verify them against current files. Currently, integrity checks are supported for the following types of file: @@ -137,8 +137,8 @@ Done! ## LDAP check -The LDAP check Rake task will test the bind DN and password credentials -(if configured) and will list a sample of LDAP users. This task is also +The LDAP check Rake task tests the bind DN and password credentials +(if configured) and lists a sample of LDAP users. This task is also executed as part of the `gitlab:check` task, but can run independently. See [LDAP Rake Tasks - LDAP Check](ldap.md#check) for details. diff --git a/doc/administration/raketasks/github_import.md b/doc/administration/raketasks/github_import.md index d549110c32f..a28cf488218 100644 --- a/doc/administration/raketasks/github_import.md +++ b/doc/administration/raketasks/github_import.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w To retrieve and import GitHub repositories, you need a [GitHub personal access token](https://github.com/settings/tokens). A username should be passed as the second argument to the Rake task, -which will become the owner of the project. You can resume an import +which becomes the owner of the project. You can resume an import with the same command. Bear in mind that the syntax is very specific. Remove any spaces within the argument block and @@ -20,7 +20,7 @@ before/after the brackets. Also, some shells (for example, `zsh`) can interpret ## Caveats If the GitHub [rate limit](https://developer.github.com/v3/#rate-limiting) is reached while importing, -the importing process will wait (`sleep()`) until it can continue importing. +the importing process waits (`sleep()`) until it can continue importing. ## Importing multiple projects @@ -35,8 +35,8 @@ bundle exec rake "import:github[access_token,root,foo/bar]" RAILS_ENV=production ``` In this case, `access_token` is your GitHub personal access token, `root` -is your GitLab username, and `foo/bar` is the new GitLab namespace/project that -will get created from your GitHub project. Subgroups are also possible: `foo/foo/bar`. +is your GitLab username, and `foo/bar` is the new GitLab namespace/project +created from your GitHub project. Subgroups are also possible: `foo/foo/bar`. ## Importing a single project diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index b93442be0a1..5fc95fcb1ae 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -107,8 +107,8 @@ The `gitlab:check` Rake task runs the following Rake tasks: - `gitlab:sidekiq:check` - `gitlab:app:check` -It will check that each component was set up according to the installation guide and suggest fixes -for issues found. This command must be run from your application server and will not work correctly on +It checks that each component was set up according to the installation guide and suggest fixes +for issues found. This command must be run from your application server and doesn't work correctly on component servers like [Gitaly](../gitaly/index.md#run-gitaly-on-its-own-server). You may also have a look at our troubleshooting guides for: @@ -300,7 +300,7 @@ To check the status of specific migrations, you can use the following Rake task: sudo gitlab-rake db:migrate:status ``` -This will output a table with a `Status` of `up` or `down` for +This outputs a table with a `Status` of `up` or `down` for each Migration ID. ```shell @@ -313,7 +313,7 @@ database: gitlabhq_production ## Run incomplete database migrations -Database migrations can be stuck in an incomplete state. That is, they'll have a `down` +Database migrations can be stuck in an incomplete state, with a `down` status in the output of the `sudo gitlab-rake db:migrate:status` command. To complete these migrations, use the following Rake task: diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index d83ab6e5cee..976fcdc8091 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -36,7 +36,7 @@ sudo gitlab-rake gitlab:import_export:version bundle exec rake gitlab:import_export:version RAILS_ENV=production ``` -The current list of DB tables that will be exported can be listed by using the following command: +The current list of DB tables to export can be listed by using the following command: ```shell # Omnibus installations diff --git a/doc/administration/raketasks/uploads/migrate.md b/doc/administration/raketasks/uploads/migrate.md index 075b27fb70d..552a81a765d 100644 --- a/doc/administration/raketasks/uploads/migrate.md +++ b/doc/administration/raketasks/uploads/migrate.md @@ -16,7 +16,7 @@ There is a Rake task for migrating uploads between different storage types. After [configuring the object storage](../../uploads.md#using-object-storage) for GitLab's uploads, use this task to migrate existing uploads from the local storage to the remote storage. -All of the processing will be done in a background worker and requires **no downtime**. +All of the processing is done in a background worker and requires **no downtime**. Read more about using [object storage with GitLab](../../object_storage.md). @@ -133,7 +133,7 @@ migrate your data out of object storage and back into your local storage. CAUTION: **Warning:** **Extended downtime is required** so no new files are created in object storage during -the migration. A configuration setting will be added soon to allow migrating +the migration. A configuration setting is planned to allow migrating from object storage to local files with only a brief moment of downtime for configuration changes. To follow progress, see the [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30979). diff --git a/doc/administration/raketasks/uploads/sanitize.md b/doc/administration/raketasks/uploads/sanitize.md index 4f7c99babd8..2ab2136ad2f 100644 --- a/doc/administration/raketasks/uploads/sanitize.md +++ b/doc/administration/raketasks/uploads/sanitize.md @@ -41,8 +41,8 @@ The Rake task accepts following parameters. | Parameter | Type | Description | |:-------------|:--------|:----------------------------------------------------------------------------------------------------------------------------| -| `start_id` | integer | Only uploads with equal or greater ID will be processed | -| `stop_id` | integer | Only uploads with equal or smaller ID will be processed | +| `start_id` | integer | Only uploads with equal or greater ID are processed | +| `stop_id` | integer | Only uploads with equal or smaller ID are processed | | `dry_run` | boolean | Do not remove EXIF data, only check if EXIF data are present or not. Defaults to `true` | | `sleep_time` | float | Pause for number of seconds after processing each image. Defaults to 0.3 seconds | | `uploader` | string | Run sanitization only for uploads of the given uploader: `FileUploader`, `PersonalFileUploader`, or `NamespaceFileUploader` | @@ -66,7 +66,7 @@ To remove EXIF data on uploads with an ID between 100 and 5000 and pause for 0.1 sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:sanitize:remove_exif[100,5000,false,0.1] 2>&1 | tee exif.log ``` -The output is written into an `exif.log` file because it will probably be long. +The output is written into an `exif.log` file because it is often long. If sanitization fails for an upload, an error message should be in the output of the Rake task. Typical reasons include that the file is missing in the storage or it's not a valid image. diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 9f522e0d599..d1afcd0343e 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -1356,19 +1356,17 @@ and improved designed. [Gitaly](../gitaly/index.md) server node requirements are dependent on data, specifically the number of projects and those projects' sizes. It's recommended -that a Gitaly server node stores no more than 5 TB of data. Although this -reference architecture includes a recommendation for the number of Gitaly server -nodes to use, depending on your storage requirements, you may require additional -Gitaly server nodes. +that a Gitaly server node stores no more than 5 TB of data. Depending on your +repository storage requirements, you may require additional Gitaly server nodes. Due to Gitaly having notable input and output requirements, we strongly -recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should -have a throughput of at least 8,000 input/output operations per second (IOPS) -for read operations and 2,000 IOPS for write operations. These IOPS values are -initial recommendations, and may be adjusted to greater or lesser values -depending on the scale of your environment's workload. If you're running the -environment on a Cloud provider, refer to their documentation about how to -configure IOPS correctly. +recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs +should have a throughput of at least 8,000 +input/output operations per second (IOPS) for read operations and 2,000 IOPS for +write operations. These IOPS values are initial recommendations, and may be +adjusted to greater or lesser values depending on the scale of your +environment's workload. If you're running the environment on a Cloud provider, +refer to their documentation about how to configure IOPS correctly. Be sure to note the following items: @@ -1376,8 +1374,8 @@ Be sure to note the following items: [repository storage paths](../repository_storage_paths.md). - A Gitaly server can host one or more storage paths. - A GitLab server can use one or more Gitaly server nodes. -- Gitaly addresses must be specified to be correctly resolvable for _all_ - Gitaly clients. +- Gitaly addresses must be specified to be correctly resolvable for all Gitaly + clients. - Gitaly servers must not be exposed to the public internet, as Gitaly's network traffic is unencrypted by default. The use of a firewall is highly recommended to restrict access to the Gitaly server. Another option is to @@ -1406,8 +1404,8 @@ On each node: 1. [Download and install](https://about.gitlab.com/install/) the Omnibus GitLab package of your choice. Be sure to follow _only_ installation steps 1 and 2 on the page, and _do not_ provide the `EXTERNAL_URL` value. -1. Edit `/etc/gitlab/gitlab.rb` to configure the storage paths, enable - the network listener, and configure the token: +1. Edit the Gitaly server node's `/etc/gitlab/gitlab.rb` file to configure + storage paths, enable the network listener, and to configure the token: <!-- updates to following example must also be made at diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index b106f7bced1..cf86bf15333 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -1356,19 +1356,17 @@ and improved designed. [Gitaly](../gitaly/index.md) server node requirements are dependent on data, specifically the number of projects and those projects' sizes. It's recommended -that a Gitaly server node stores no more than 5 TB of data. Although this -reference architecture includes a recommendation for the number of Gitaly server -nodes to use, depending on your storage requirements, you may require additional -Gitaly server nodes. +that a Gitaly server node stores no more than 5 TB of data. Depending on your +repository storage requirements, you may require additional Gitaly server nodes. Due to Gitaly having notable input and output requirements, we strongly -recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should -have a throughput of at least 8,000 input/output operations per second (IOPS) -for read operations and 2,000 IOPS for write operations. These IOPS values are -initial recommendations, and may be adjusted to greater or lesser values -depending on the scale of your environment's workload. If you're running the -environment on a Cloud provider, refer to their documentation about how to -configure IOPS correctly. +recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs +should have a throughput of at least 8,000 +input/output operations per second (IOPS) for read operations and 2,000 IOPS for +write operations. These IOPS values are initial recommendations, and may be +adjusted to greater or lesser values depending on the scale of your +environment's workload. If you're running the environment on a Cloud provider, +refer to their documentation about how to configure IOPS correctly. Be sure to note the following items: @@ -1376,8 +1374,8 @@ Be sure to note the following items: [repository storage paths](../repository_storage_paths.md). - A Gitaly server can host one or more storage paths. - A GitLab server can use one or more Gitaly server nodes. -- Gitaly addresses must be specified to be correctly resolvable for _all_ - Gitaly clients. +- Gitaly addresses must be specified to be correctly resolvable for all Gitaly + clients. - Gitaly servers must not be exposed to the public internet, as Gitaly's network traffic is unencrypted by default. The use of a firewall is highly recommended to restrict access to the Gitaly server. Another option is to @@ -1406,8 +1404,8 @@ On each node: 1. [Download and install](https://about.gitlab.com/install/) the Omnibus GitLab package of your choice. Be sure to follow _only_ installation steps 1 and 2 on the page, and _do not_ provide the `EXTERNAL_URL` value. -1. Edit `/etc/gitlab/gitlab.rb` to configure the storage paths, enable - the network listener, and configure the token: +1. Edit the Gitaly server node's `/etc/gitlab/gitlab.rb` file to configure + storage paths, enable the network listener, and to configure the token: <!-- updates to following example must also be made at diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index b5b3e4e0300..c0c1d566255 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -1067,19 +1067,17 @@ and improved designed. [Gitaly](../gitaly/index.md) server node requirements are dependent on data, specifically the number of projects and those projects' sizes. It's recommended -that a Gitaly server node stores no more than 5 TB of data. Although this -reference architecture includes a recommendation for the number of Gitaly server -nodes to use, depending on your storage requirements, you may require additional -Gitaly server nodes. +that a Gitaly server node stores no more than 5 TB of data. Depending on your +repository storage requirements, you may require additional Gitaly server nodes. Due to Gitaly having notable input and output requirements, we strongly -recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should -have a throughput of at least 8,000 input/output operations per second (IOPS) -for read operations and 2,000 IOPS for write operations. These IOPS values are -initial recommendations, and may be adjusted to greater or lesser values -depending on the scale of your environment's workload. If you're running the -environment on a Cloud provider, refer to their documentation about how to -configure IOPS correctly. +recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs +should have a throughput of at least 8,000 +input/output operations per second (IOPS) for read operations and 2,000 IOPS for +write operations. These IOPS values are initial recommendations, and may be +adjusted to greater or lesser values depending on the scale of your +environment's workload. If you're running the environment on a Cloud provider, +refer to their documentation about how to configure IOPS correctly. Be sure to note the following items: @@ -1087,8 +1085,8 @@ Be sure to note the following items: [repository storage paths](../repository_storage_paths.md). - A Gitaly server can host one or more storage paths. - A GitLab server can use one or more Gitaly server nodes. -- Gitaly addresses must be specified to be correctly resolvable for _all_ - Gitaly clients. +- Gitaly addresses must be specified to be correctly resolvable for all Gitaly + clients. - Gitaly servers must not be exposed to the public internet, as Gitaly's network traffic is unencrypted by default. The use of a firewall is highly recommended to restrict access to the Gitaly server. Another option is to @@ -1117,8 +1115,8 @@ On each node: 1. [Download and install](https://about.gitlab.com/install/) the Omnibus GitLab package of your choice. Be sure to follow _only_ installation steps 1 and 2 on the page, and _do not_ provide the `EXTERNAL_URL` value. -1. Edit `/etc/gitlab/gitlab.rb` to configure the storage paths, enable - the network listener, and configure the token: +1. Edit the Gitaly server node's `/etc/gitlab/gitlab.rb` file to configure + storage paths, enable the network listener, and to configure the token: <!-- updates to following example must also be made at diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 152eb9cb90d..3149bbb0934 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -1356,19 +1356,17 @@ and improved designed. [Gitaly](../gitaly/index.md) server node requirements are dependent on data, specifically the number of projects and those projects' sizes. It's recommended -that a Gitaly server node stores no more than 5 TB of data. Although this -reference architecture includes a recommendation for the number of Gitaly server -nodes to use, depending on your storage requirements, you may require additional -Gitaly server nodes. +that a Gitaly server node stores no more than 5 TB of data. Depending on your +repository storage requirements, you may require additional Gitaly server nodes. Due to Gitaly having notable input and output requirements, we strongly -recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should -have a throughput of at least 8,000 input/output operations per second (IOPS) -for read operations and 2,000 IOPS for write operations. These IOPS values are -initial recommendations, and may be adjusted to greater or lesser values -depending on the scale of your environment's workload. If you're running the -environment on a Cloud provider, refer to their documentation about how to -configure IOPS correctly. +recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs +should have a throughput of at least 8,000 +input/output operations per second (IOPS) for read operations and 2,000 IOPS for +write operations. These IOPS values are initial recommendations, and may be +adjusted to greater or lesser values depending on the scale of your +environment's workload. If you're running the environment on a Cloud provider, +refer to their documentation about how to configure IOPS correctly. Be sure to note the following items: @@ -1376,8 +1374,8 @@ Be sure to note the following items: [repository storage paths](../repository_storage_paths.md). - A Gitaly server can host one or more storage paths. - A GitLab server can use one or more Gitaly server nodes. -- Gitaly addresses must be specified to be correctly resolvable for _all_ - Gitaly clients. +- Gitaly addresses must be specified to be correctly resolvable for all Gitaly + clients. - Gitaly servers must not be exposed to the public internet, as Gitaly's network traffic is unencrypted by default. The use of a firewall is highly recommended to restrict access to the Gitaly server. Another option is to @@ -1406,8 +1404,8 @@ On each node: 1. [Download and install](https://about.gitlab.com/install/) the Omnibus GitLab package of your choice. Be sure to follow _only_ installation steps 1 and 2 on the page, and _do not_ provide the `EXTERNAL_URL` value. -1. Edit `/etc/gitlab/gitlab.rb` to configure the storage paths, enable - the network listener, and configure the token: +1. Edit the Gitaly server node's `/etc/gitlab/gitlab.rb` file to configure + storage paths, enable the network listener, and to configure the token: <!-- updates to following example must also be made at diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index f023971bdc0..519e59f8552 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -37,6 +37,63 @@ costly-to-operate environment by using the | Object storage | n/a | n/a | n/a | n/a | n/a | | NFS server (optional, not recommended) | 1 | 4 vCPU, 3.6 GB memory | n1-highcpu-4 | c5.xlarge | F4s v2 | +```mermaid +stateDiagram-v2 + [*] --> LoadBalancer + LoadBalancer --> ApplicationServer + + ApplicationServer --> BackgroundJobs + ApplicationServer --> Gitaly + ApplicationServer --> Redis + ApplicationServer --> PgBouncer + PgBouncer --> Database + ApplicationServer --> ObjectStorage + BackgroundJobs --> ObjectStorage + + ApplicationMonitoring -->ApplicationServer + ApplicationMonitoring -->Redis + ApplicationMonitoring -->PgBouncer + ApplicationMonitoring -->Database + ApplicationMonitoring -->BackgroundJobs + + state Database { + "PG_Primary_Node" + "PG_Secondary_Node_1..2" + } + state Redis { + "R_Primary_Node" + "R_Replica_Node_1..2" + "R_Consul/Sentinel_1..3" + } + + state Gitaly { + "Gitaly_1..2" + } + + state BackgroundJobs { + "Sidekiq_1..4" + } + + state ApplicationServer { + "GitLab_Rails_1..3" + } + + state LoadBalancer { + "LoadBalancer_1" + } + + state ApplicationMonitoring { + "Prometheus" + "Grafana" + } + + state PgBouncer { + "Internal_Load_Balancer" + "PgBouncer_1..3" + + } +``` + The Google Cloud Platform (GCP) architectures were built and tested using the [Intel Xeon E5 v3 (Haswell)](https://cloud.google.com/compute/docs/cpu-platforms) CPU platform. On different hardware you may find that adjustments, either lower @@ -1066,19 +1123,17 @@ and improved designed. [Gitaly](../gitaly/index.md) server node requirements are dependent on data, specifically the number of projects and those projects' sizes. It's recommended -that a Gitaly server node stores no more than 5 TB of data. Although this -reference architecture includes a recommendation for the number of Gitaly server -nodes to use, depending on your storage requirements, you may require additional -Gitaly server nodes. +that a Gitaly server node stores no more than 5 TB of data. Depending on your +repository storage requirements, you may require additional Gitaly server nodes. Due to Gitaly having notable input and output requirements, we strongly -recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs should -have a throughput of at least 8,000 input/output operations per second (IOPS) -for read operations and 2,000 IOPS for write operations. These IOPS values are -initial recommendations, and may be adjusted to greater or lesser values -depending on the scale of your environment's workload. If you're running the -environment on a Cloud provider, refer to their documentation about how to -configure IOPS correctly. +recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs +should have a throughput of at least 8,000 +input/output operations per second (IOPS) for read operations and 2,000 IOPS for +write operations. These IOPS values are initial recommendations, and may be +adjusted to greater or lesser values depending on the scale of your +environment's workload. If you're running the environment on a Cloud provider, +refer to their documentation about how to configure IOPS correctly. Be sure to note the following items: @@ -1086,8 +1141,8 @@ Be sure to note the following items: [repository storage paths](../repository_storage_paths.md). - A Gitaly server can host one or more storage paths. - A GitLab server can use one or more Gitaly server nodes. -- Gitaly addresses must be specified to be correctly resolvable for _all_ - Gitaly clients. +- Gitaly addresses must be specified to be correctly resolvable for all Gitaly + clients. - Gitaly servers must not be exposed to the public internet, as Gitaly's network traffic is unencrypted by default. The use of a firewall is highly recommended to restrict access to the Gitaly server. Another option is to @@ -1116,8 +1171,8 @@ On each node: 1. [Download and install](https://about.gitlab.com/install/) the Omnibus GitLab package of your choice. Be sure to follow _only_ installation steps 1 and 2 on the page, and _do not_ provide the `EXTERNAL_URL` value. -1. Edit `/etc/gitlab/gitlab.rb` to configure the storage paths, enable - the network listener, and configure the token: +1. Edit the Gitaly server node's `/etc/gitlab/gitlab.rb` file to configure + storage paths, enable the network listener, and to configure the token: <!-- updates to following example must also be made at diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 8816d0eecf4..54f3eca204b 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -151,7 +151,27 @@ is recommended. instance to other geographical locations as a read-only fully operational instance that can also be promoted in case of disaster. -## Configuring select components with Cloud Native Helm +## Deviating from the suggested reference architectures + +As a general rule of thumb, the further away you move from the Reference Architectures, +the harder it will be get support for it. With any deviation, you're introducing +a layer of complexity that will add challenges to finding out where potential +issues might lie. + +The reference architectures use the official GitLab Linux packages (Omnibus +GitLab) to install and configure the various components (with one notable exception being the suggested select Cloud Native installation method described below). The components are +installed on separate machines (virtualized or bare metal), with machine hardware +requirements listed in the "Configuration" column and equivalent VM standard sizes listed +in GCP/AWS/Azure columns of each [available reference architecture](#available-reference-architectures). + +Running components on Docker (including Compose) with the same specs should be fine, as Docker is well known in terms of support. +However, it is still an additional layer and may still add some support complexities, such as not being able to run `strace` easily in containers. + +Other technologies, like [Docker swarm](https://docs.docker.com/engine/swarm/) +are not officially supported, but can be implemented at your own risk. In that +case, GitLab Support will not be able to help you. + +### Configuring select components with Cloud Native Helm We also provide [Helm charts](https://docs.gitlab.com/charts/) as a Cloud Native installation method for GitLab. For the reference architectures, select components can be set up in this diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index 16e17458e10..74c6e3d23f7 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -50,7 +50,7 @@ storage2: > structure than Omnibus. In `gitlab.yml` you indicate the path for the > repositories, for example `/home/git/repositories`, while in Omnibus you > indicate `git_data_dirs`, which for the example above would be `/home/git`. -> Then, Omnibus will create a `repositories` directory under that path to use with +> Then, Omnibus creates a `repositories` directory under that path to use with > `gitlab.yml`. > > This little detail matters because while restoring a backup, the current @@ -90,8 +90,7 @@ This example uses NFS. We do not recommend using EFS for storage as it may impac 1. [Restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. NOTE: **Note:** -The [`gitlab_shell: repos_path` entry](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/8-9-stable/config/gitlab.yml.example#L457) in `gitlab.yml` will be -deprecated and replaced by `repositories: storages` in the future, so if you +We plan to replace [`gitlab_shell: repos_path` entry](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/8-9-stable/config/gitlab.yml.example#L457) in `gitlab.yml` with `repositories: storages`. If you are upgrading from a version prior to 8.10, make sure to add the configuration as described in the step above. After you make the changes and confirm they are working, you can remove the `repos_path` line. @@ -112,16 +111,15 @@ working, you can remove the `repos_path` line. Note that Omnibus stores the repositories in a `repositories` subdirectory of the `git-data` directory. -## Choose where new repositories will be stored +## Choose where new repositories are stored Once you set the multiple storage paths, you can choose where new repositories -will be stored under **Admin Area > Settings > Repository > -Repository storage > Storage nodes for new repositories**. +are stored in the Admin Area under **Settings > Repository > Repository storage > Storage nodes for new repositories**. Each storage can be assigned a weight from 0-100. When a new project is created, these -weights are used to determine the storage location the repository will be created on. +weights are used to determine the storage location the repository is created on.  Beginning with GitLab 8.13.4, multiple paths can be chosen. New repositories -will be randomly placed on one of the selected paths. +are randomly placed on one of the selected paths. diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index d2d492c8f1a..2401dfa5812 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -19,7 +19,7 @@ locations that can be: - Accessed via [Gitaly](gitaly/index.md) on its own machine. In GitLab, this is configured in `/etc/gitlab/gitlab.rb` by the `git_data_dirs({})` -configuration hash. The storage layouts discussed here will apply to any shard +configuration hash. The storage layouts discussed here apply to any shard defined in it. The `default` repository shard that is available in any installations @@ -30,22 +30,22 @@ Anything discussed below is expected to be part of that folder. NOTE: **Note:** In GitLab 13.0, hashed storage is enabled by default and the legacy storage is -deprecated. Support for legacy storage will be removed in GitLab 14.0. +deprecated. Support for legacy storage is scheduled to be removed in GitLab 14.0. If you haven't migrated yet, check the [migration instructions](raketasks/storage.md#migrate-to-hashed-storage). The option to choose between hashed and legacy storage in the admin area has been disabled. Hashed storage is the storage behavior we rolled out with 10.0. Instead -of coupling project URL and the folder structure where the repository will be +of coupling project URL and the folder structure where the repository is stored on disk, we are coupling a hash, based on the project's ID. This makes the folder structure immutable, and therefore eliminates any requirement to synchronize state from URLs to disk structure. This means that renaming a group, -user, or project will cost only the database transaction, and will take effect +user, or project costs only the database transaction, and takes effect immediately. The hash also helps to spread the repositories more evenly on the disk, so the -top-level directory will contain less folders than the total amount of top-level +top-level directory contains fewer folders than the total number of top-level namespaces. The hash format is based on the hexadecimal representation of SHA256: @@ -64,7 +64,7 @@ by another folder with the next 2 characters. They are both stored in a special ### Translating hashed storage paths Troubleshooting problems with the Git repositories, adding hooks, and other -tasks will require you translate between the human readable project name +tasks requires you translate between the human readable project name and the hashed storage path. #### From project name to hashed path @@ -102,7 +102,7 @@ To translate from a hashed storage path to a project name: ProjectRepository.find_by(disk_path: '@hashed/b1/7e/b17ef6d19c7a5b1ee83b907c595526dcb1eb06db8227d650d5dda0a9f4ce8cd9').project ``` -The quoted string in that command is the directory tree you'll find on your +The quoted string in that command is the directory tree you can find on your GitLab server. For example, on a default Omnibus installation this would be `/var/opt/gitlab/git-data/repositories/@hashed/b1/7e/b17ef6d19c7a5b1ee83b907c595526dcb1eb06db8227d650d5dda0a9f4ce8cd9.git` with `.git` from the end of the directory name removed. @@ -135,7 +135,7 @@ when housekeeping is run on the source project. ### Hashed storage coverage migration -Files stored in an S3 compatible endpoint will not have the downsides +Files stored in an S3-compatible endpoint do not have the downsides mentioned earlier, if they are not prefixed with `#{namespace}/#{project_name}`, which is true for CI Cache and LFS Objects. @@ -183,7 +183,7 @@ NOTE: **Deprecated:** In GitLab 13.0, hashed storage is enabled by default and the legacy storage is deprecated. If you haven't migrated yet, check the [migration instructions](raketasks/storage.md#migrate-to-hashed-storage). -Support for legacy storage will be removed in GitLab 14.0. If you're on GitLab +Support for legacy storage is scheduled to be removed in GitLab 14.0. If you're on GitLab 13.0 and later, switching new projects to legacy storage is not possible. The option to choose between hashed and legacy storage in the admin area has been disabled. @@ -199,7 +199,7 @@ easy for Administrators to find where the repository is stored. On the other hand this has some drawbacks: -Storage location will concentrate huge amount of top-level namespaces. The +Storage location concentrates a huge number of top-level namespaces. The impact can be reduced by the introduction of [multiple storage paths](repository_storage_paths.md). @@ -209,6 +209,6 @@ an old removed or renamed project sharing the same URL. This means that `mygroup/myproject` from your backup may not be the same original project that is at that same URL today. -Any change in the URL will need to be reflected on disk (when groups / users or +Any change in the URL needs to be reflected on disk (when groups / users or projects are renamed). This can add a lot of load in big installations, especially if using any type of network based filesystem. diff --git a/doc/administration/repository_storages.md b/doc/administration/repository_storages.md index af7a385e5a0..93d0ee3cac9 100644 --- a/doc/administration/repository_storages.md +++ b/doc/administration/repository_storages.md @@ -3,3 +3,6 @@ redirect_to: 'repository_storage_paths.md' --- This document was moved to [another location](repository_storage_paths.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/scaling/index.md b/doc/administration/scaling/index.md index 748373c8941..4f934646ed6 100644 --- a/doc/administration/scaling/index.md +++ b/doc/administration/scaling/index.md @@ -3,3 +3,6 @@ redirect_to: ../reference_architectures/index.md --- This document was moved to [another location](../reference_architectures/index.md). + +<!-- This redirect file can be deleted after February 1, 2021. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/terraform_state.md b/doc/administration/terraform_state.md index 55e166d2bf7..cf30090d45c 100644 --- a/doc/administration/terraform_state.md +++ b/doc/administration/terraform_state.md @@ -68,7 +68,7 @@ The following settings are: | Setting | Description | Default | |---------|-------------|---------| | `enabled` | Enable/disable object storage | `false` | -| `remote_directory` | The bucket name where Terraform state files will be stored | | +| `remote_directory` | The bucket name where Terraform state files are stored | | | `connection` | Various connection options described below | | ### S3-compatible connection settings |