diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2021-01-08 00:10:18 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2021-01-08 00:10:18 +0300 |
commit | fd320d0858df108b47ebadaa546a3c99f0e44cf1 (patch) | |
tree | 10fd76baeaf079ba50392d569c52d3d711169c8b /doc | |
parent | dec7332357ae21be289ac26a4c440cb0b5fa1002 (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
28 files changed, 556 insertions, 942 deletions
diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index 69e17c28e54..afa53b5efa8 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -422,9 +422,9 @@ install the necessary dependencies from step 1, and add the GitLab package repository from step 2. When installing GitLab in the second step, do not supply the `EXTERNAL_URL` value. -#### PostgreSQL primary node +#### PostgreSQL nodes -1. SSH in to the PostgreSQL primary node. +1. SSH in to one of the PostgreSQL nodes. 1. Generate a password hash for the PostgreSQL username/password pair. This assumes you will use the default username of `gitlab` (recommended). The command will request a password and confirmation. Use the value that is output by this command in the next @@ -452,23 +452,33 @@ in the second step, do not supply the `EXTERNAL_URL` value. sudo gitlab-ctl pg-password-md5 gitlab-consul ``` -1. On the primary database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: +1. On every database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul + # Disable all components except PostgreSQL, Patroni, and Consul roles ['postgres_role'] # PostgreSQL configuration postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' + + # Enable Patroni + patroni['enable'] = true + # Set `max_wal_senders` to one more than the number of database nodes in the cluster. + # This is used to prevent replication from using up all of the + # available database connections. + patroni['postgresql']['max_wal_senders'] = 4 + patroni['postgresql']['max_replication_slots'] = 4 + # Incoming recommended value for max connections is 500. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + patroni['postgresql']['max_connections'] = 500 # Disable automatic database migrations gitlab_rails['auto_migrate'] = false # Configure the Consul agent + consul['enable'] = true consul['services'] = %w(postgresql) + ## Enable service discovery for Prometheus + consul['monitoring_service_discovery'] = true # START user configuration # Please set the real values as explained in Required Information section @@ -477,18 +487,9 @@ in the second step, do not supply the `EXTERNAL_URL` value. postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value postgresql['sql_user_password'] = '<postgresql_password_hash>' - # Set `max_wal_senders` to one more than the number of database nodes in the cluster. - # This is used to prevent replication from using up all of the - # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 # Replace XXX.XXX.XXX.XXX/YY with Network Address postgresql['trust_auth_cidr_addresses'] = %w(10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['monitoring_service_discovery'] = true # Set the network addresses that the exporters will listen on for monitoring node_exporter['listen_address'] = '0.0.0.0:9100' @@ -503,70 +504,9 @@ in the second step, do not supply the `EXTERNAL_URL` value. # END user configuration ``` -1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace - the file of the same name on this server. If that file is not on this server, - add the file from your Consul server to this server. - -1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -<div align="right"> - <a type="button" class="btn btn-default" href="#setup-components"> - Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> - </a> -</div> - -#### PostgreSQL secondary nodes - -1. On both the secondary nodes, add the same configuration specified above for the primary node - with an additional setting (`repmgr['master_on_initialization'] = false`) that will inform `gitlab-ctl` that they are standby nodes initially - and there's no need to attempt to register them as a primary node: - - ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul - roles ['postgres_role'] - - # PostgreSQL configuration - postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' - - # Disable automatic database migrations - gitlab_rails['auto_migrate'] = false - - # Configure the Consul agent - consul['services'] = %w(postgresql) - - # Specify if a node should attempt to be primary on initialization. - repmgr['master_on_initialization'] = false - - # Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value - postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' - # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value - postgresql['sql_user_password'] = '<postgresql_password_hash>' - # Set `max_wal_senders` to one more than the number of database nodes in the cluster. - # This is used to prevent replication from using up all of the - # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 - - # Replace with your network addresses - postgresql['trust_auth_cidr_addresses'] = %w(10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['monitoring_service_discovery'] = true - - # Set the network addresses that the exporters will listen on for monitoring - node_exporter['listen_address'] = '0.0.0.0:9100' - postgres_exporter['listen_address'] = '0.0.0.0:9187' - - ## The IPs of the Consul server nodes - ## You can also use FQDNs and intermix them with IPs - consul['configuration'] = { - retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), - } - ``` +PostgreSQL, with Patroni managing its failover, will default to use `pg_rewind` by default to handle conflicts. +Like most failover handling methods, this has a small chance of leading to data loss. +Learn more about the various [Patroni replication methods](../postgresql/replication_and_failover.md#selecting-the-appropriate-patroni-replication-method). 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace the file of the same name on this server. If that file is not on this server, @@ -601,84 +541,25 @@ SSH in to the **primary node**: 1. Exit the database prompt by typing `\q` and Enter. -1. Verify the cluster is initialized with one node: +1. Check the status of the leader and cluster: ```shell - gitlab-ctl repmgr cluster show + gitlab-ctl patroni members ``` The output should be similar to the following: ```plaintext - Role | Name | Upstream | Connection String - ----------+----------|----------|---------------------------------------- - * master | HOSTNAME | | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr + | Cluster | Member | Host | Role | State | TL | Lag in MB | Pending restart | + |---------------|-----------------------------------|-----------|--------|---------|-----|-----------|-----------------| + | postgresql-ha | <PostgreSQL primary hostname> | 10.6.0.21 | Leader | running | 175 | | * | + | postgresql-ha | <PostgreSQL secondary 1 hostname> | 10.6.0.22 | | running | 175 | 0 | * | + | postgresql-ha | <PostgreSQL secondary 2 hostname> | 10.6.0.23 | | running | 175 | 0 | * | ``` -1. Note down the hostname or IP address in the connection string: `host=HOSTNAME`. We will - refer to the hostname in the next section as `<primary_node_name>`. If the value - is not an IP address, it will need to be a resolvable name (via DNS or - `/etc/hosts`) - -SSH in to the **secondary node**: - -1. Set up the repmgr standby: - - ```shell - gitlab-ctl repmgr standby setup <primary_node_name> - ``` - - Do note that this will remove the existing data on the node. The command - has a wait time. - - The output should be similar to the following: - - ```console - Doing this will delete the entire contents of /var/opt/gitlab/postgresql/data - If this is not what you want, hit Ctrl-C now to exit - To skip waiting, rerun with the -w option - Sleeping for 30 seconds - Stopping the database - Removing the data - Cloning the data - Starting the database - Registering the node with the cluster - ok: run: repmgrd: (pid 19068) 0s - ``` - -Before moving on, make sure the databases are configured correctly. Run the -following command on the **primary** node to verify that replication is working -properly and the secondary nodes appear in the cluster: - -```shell -gitlab-ctl repmgr cluster show -``` - -The output should be similar to the following: - -```plaintext -Role | Name | Upstream | Connection String -----------+---------|-----------|------------------------------------------------ -* master | MASTER | | host=<primary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr -``` - -If the 'Role' column for any node says "FAILED", check the +If the 'State' column for any node doesn't say "running", check the [Troubleshooting section](troubleshooting.md) before proceeding. -Also, check that the `repmgr-check-master` command works successfully on each node: - -```shell -su - gitlab-consul -gitlab-ctl repmgr-check-master || echo 'This node is a standby repmgr node' -``` - -This command relies on exit codes to tell Consul whether a particular node is a master -or secondary. The most important thing here is that this command does not produce errors. -If there are errors it's most likely due to incorrect `gitlab-consul` database user permissions. -Check the [Troubleshooting section](troubleshooting.md) before proceeding. - <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> @@ -696,7 +577,7 @@ The following IPs will be used as an example: 1. On each PgBouncer node, edit `/etc/gitlab/gitlab.rb`, and replace `<consul_password_hash>` and `<pgbouncer_password_hash>` with the - password hashes you [set up previously](#postgresql-primary-node): + password hashes you [set up previously](#postgresql-nodes): ```ruby # Disable all components except Pgbouncer and Consul agent @@ -704,15 +585,16 @@ The following IPs will be used as an example: # Configure PgBouncer pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul) - pgbouncer['users'] = { - 'gitlab-consul': { - password: '<consul_password_hash>' - }, - 'pgbouncer': { - password: '<pgbouncer_password_hash>' - } + 'gitlab-consul': { + password: '<consul_password_hash>' + }, + 'pgbouncer': { + password: '<pgbouncer_password_hash>' + } } + # Incoming recommended value for max db connections is 150. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + pgbouncer['max_db_connections'] = 150 # Configure Consul agent consul['watchers'] = %w(postgresql) diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 03e340bdb88..d96e93d4ab4 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -422,9 +422,9 @@ install the necessary dependencies from step 1, and add the GitLab package repository from step 2. When installing GitLab in the second step, do not supply the `EXTERNAL_URL` value. -#### PostgreSQL primary node +#### PostgreSQL nodes -1. SSH in to the PostgreSQL primary node. +1. SSH in to one of the PostgreSQL nodes. 1. Generate a password hash for the PostgreSQL username/password pair. This assumes you will use the default username of `gitlab` (recommended). The command will request a password and confirmation. Use the value that is output by this command in the next @@ -452,23 +452,33 @@ in the second step, do not supply the `EXTERNAL_URL` value. sudo gitlab-ctl pg-password-md5 gitlab-consul ``` -1. On the primary database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: +1. On every database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul + # Disable all components except PostgreSQL, Patroni, and Consul roles ['postgres_role'] # PostgreSQL configuration postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' + + # Enable Patroni + patroni['enable'] = true + # Set `max_wal_senders` to one more than the number of database nodes in the cluster. + # This is used to prevent replication from using up all of the + # available database connections. + patroni['postgresql']['max_wal_senders'] = 4 + patroni['postgresql']['max_replication_slots'] = 4 + # Incoming recommended value for max connections is 500. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + patroni['postgresql']['max_connections'] = 500 # Disable automatic database migrations gitlab_rails['auto_migrate'] = false # Configure the Consul agent + consul['enable'] = true consul['services'] = %w(postgresql) + ## Enable service discovery for Prometheus + consul['monitoring_service_discovery'] = true # START user configuration # Please set the real values as explained in Required Information section @@ -477,18 +487,9 @@ in the second step, do not supply the `EXTERNAL_URL` value. postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value postgresql['sql_user_password'] = '<postgresql_password_hash>' - # Set `max_wal_senders` to one more than the number of database nodes in the cluster. - # This is used to prevent replication from using up all of the - # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 # Replace XXX.XXX.XXX.XXX/YY with Network Address postgresql['trust_auth_cidr_addresses'] = %w(10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['monitoring_service_discovery'] = true # Set the network addresses that the exporters will listen on for monitoring node_exporter['listen_address'] = '0.0.0.0:9100' @@ -503,70 +504,9 @@ in the second step, do not supply the `EXTERNAL_URL` value. # END user configuration ``` -1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace - the file of the same name on this server. If that file is not on this server, - add the file from your Consul server to this server. - -1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -<div align="right"> - <a type="button" class="btn btn-default" href="#setup-components"> - Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> - </a> -</div> - -#### PostgreSQL secondary nodes - -1. On both the secondary nodes, add the same configuration specified above for the primary node - with an additional setting (`repmgr['master_on_initialization'] = false`) that will inform `gitlab-ctl` that they are standby nodes initially - and there's no need to attempt to register them as a primary node: - - ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul - roles ['postgres_role'] - - # PostgreSQL configuration - postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' - - # Disable automatic database migrations - gitlab_rails['auto_migrate'] = false - - # Configure the Consul agent - consul['services'] = %w(postgresql) - - # Specify if a node should attempt to be primary on initialization. - repmgr['master_on_initialization'] = false - - # Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value - postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' - # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value - postgresql['sql_user_password'] = '<postgresql_password_hash>' - # Set `max_wal_senders` to one more than the number of database nodes in the cluster. - # This is used to prevent replication from using up all of the - # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 - - # Replace with your network addresses - postgresql['trust_auth_cidr_addresses'] = %w(10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['monitoring_service_discovery'] = true - - # Set the network addresses that the exporters will listen on for monitoring - node_exporter['listen_address'] = '0.0.0.0:9100' - postgres_exporter['listen_address'] = '0.0.0.0:9187' - - ## The IPs of the Consul server nodes - ## You can also use FQDNs and intermix them with IPs - consul['configuration'] = { - retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), - } - ``` +PostgreSQL, with Patroni managing its failover, will default to use `pg_rewind` by default to handle conflicts. +Like most failover handling methods, this has a small chance of leading to data loss. +Learn more about the various [Patroni replication methods](../postgresql/replication_and_failover.md#selecting-the-appropriate-patroni-replication-method). 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace the file of the same name on this server. If that file is not on this server, @@ -601,84 +541,25 @@ SSH in to the **primary node**: 1. Exit the database prompt by typing `\q` and Enter. -1. Verify the cluster is initialized with one node: +1. Check the status of the leader and cluster: ```shell - gitlab-ctl repmgr cluster show + gitlab-ctl patroni members ``` The output should be similar to the following: ```plaintext - Role | Name | Upstream | Connection String - ----------+----------|----------|---------------------------------------- - * master | HOSTNAME | | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr + | Cluster | Member | Host | Role | State | TL | Lag in MB | Pending restart | + |---------------|-----------------------------------|-----------|--------|---------|-----|-----------|-----------------| + | postgresql-ha | <PostgreSQL primary hostname> | 10.6.0.21 | Leader | running | 175 | | * | + | postgresql-ha | <PostgreSQL secondary 1 hostname> | 10.6.0.22 | | running | 175 | 0 | * | + | postgresql-ha | <PostgreSQL secondary 2 hostname> | 10.6.0.23 | | running | 175 | 0 | * | ``` -1. Note down the hostname or IP address in the connection string: `host=HOSTNAME`. We will - refer to the hostname in the next section as `<primary_node_name>`. If the value - is not an IP address, it will need to be a resolvable name (via DNS or - `/etc/hosts`) - -SSH in to the **secondary node**: - -1. Set up the repmgr standby: - - ```shell - gitlab-ctl repmgr standby setup <primary_node_name> - ``` - - Do note that this will remove the existing data on the node. The command - has a wait time. - - The output should be similar to the following: - - ```console - Doing this will delete the entire contents of /var/opt/gitlab/postgresql/data - If this is not what you want, hit Ctrl-C now to exit - To skip waiting, rerun with the -w option - Sleeping for 30 seconds - Stopping the database - Removing the data - Cloning the data - Starting the database - Registering the node with the cluster - ok: run: repmgrd: (pid 19068) 0s - ``` - -Before moving on, make sure the databases are configured correctly. Run the -following command on the **primary** node to verify that replication is working -properly and the secondary nodes appear in the cluster: - -```shell -gitlab-ctl repmgr cluster show -``` - -The output should be similar to the following: - -```plaintext -Role | Name | Upstream | Connection String -----------+---------|-----------|------------------------------------------------ -* master | MASTER | | host=<primary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr -``` - -If the 'Role' column for any node says "FAILED", check the +If the 'State' column for any node doesn't say "running", check the [Troubleshooting section](troubleshooting.md) before proceeding. -Also, check that the `repmgr-check-master` command works successfully on each node: - -```shell -su - gitlab-consul -gitlab-ctl repmgr-check-master || echo 'This node is a standby repmgr node' -``` - -This command relies on exit codes to tell Consul whether a particular node is a master -or secondary. The most important thing here is that this command does not produce errors. -If there are errors it's most likely due to incorrect `gitlab-consul` database user permissions. -Check the [Troubleshooting section](troubleshooting.md) before proceeding. - <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> @@ -696,7 +577,7 @@ The following IPs will be used as an example: 1. On each PgBouncer node, edit `/etc/gitlab/gitlab.rb`, and replace `<consul_password_hash>` and `<pgbouncer_password_hash>` with the - password hashes you [set up previously](#postgresql-primary-node): + password hashes you [set up previously](#postgresql-nodes): ```ruby # Disable all components except Pgbouncer and Consul agent @@ -704,15 +585,16 @@ The following IPs will be used as an example: # Configure PgBouncer pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul) - pgbouncer['users'] = { - 'gitlab-consul': { - password: '<consul_password_hash>' - }, - 'pgbouncer': { - password: '<pgbouncer_password_hash>' - } + 'gitlab-consul': { + password: '<consul_password_hash>' + }, + 'pgbouncer': { + password: '<pgbouncer_password_hash>' + } } + # Incoming recommended value for max db connections is 150. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + pgbouncer['max_db_connections'] = 150 # Configure Consul agent consul['watchers'] = %w(postgresql) diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 6158fabb8bd..e664d55fa1b 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -271,7 +271,7 @@ further configuration steps. ```ruby # Disable all components except PostgreSQL roles ['postgres_role'] - repmgr['enable'] = false + patroni['enable'] = false consul['enable'] = false prometheus['enable'] = false alertmanager['enable'] = false diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 670b86913cb..e7b151452a5 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -672,9 +672,9 @@ install the necessary dependencies from step 1, and add the GitLab package repository from step 2. When installing GitLab in the second step, do not supply the `EXTERNAL_URL` value. -#### PostgreSQL primary node +#### PostgreSQL nodes -1. SSH in to the PostgreSQL primary node. +1. SSH in to one of the PostgreSQL nodes. 1. Generate a password hash for the PostgreSQL username/password pair. This assumes you will use the default username of `gitlab` (recommended). The command will request a password and confirmation. Use the value that is output by this command in the next @@ -702,114 +702,33 @@ in the second step, do not supply the `EXTERNAL_URL` value. sudo gitlab-ctl pg-password-md5 gitlab-consul ``` -1. On the primary database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: +1. On every database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul + # Disable all components except PostgreSQL, Patroni, and Consul roles ['postgres_role'] # PostgreSQL configuration postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' - # Disable automatic database migrations - gitlab_rails['auto_migrate'] = false - - # Configure the Consul agent - consul['services'] = %w(postgresql) - - # START user configuration - # Please set the real values as explained in Required Information section - # - # Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value - postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' - # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value - postgresql['sql_user_password'] = '<postgresql_password_hash>' + # Enable Patroni + patroni['enable'] = true # Set `max_wal_senders` to one more than the number of database nodes in the cluster. # This is used to prevent replication from using up all of the # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 - - # Replace XXX.XXX.XXX.XXX/YY with Network Address - postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['enable'] = true - consul['monitoring_service_discovery'] = true - - # Set the network addresses that the exporters will listen on for monitoring - node_exporter['listen_address'] = '0.0.0.0:9100' - postgres_exporter['listen_address'] = '0.0.0.0:9187' - postgres_exporter['dbname'] = 'gitlabhq_production' - postgres_exporter['password'] = '<postgresql_password_hash>' - - ## The IPs of the Consul server nodes - ## You can also use FQDNs and intermix them with IPs - consul['configuration'] = { - retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), - } - # - # END user configuration - ``` - -1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -1. You can list the current PostgreSQL primary, secondary nodes status via: - - ```shell - sudo /opt/gitlab/bin/gitlab-ctl repmgr cluster show - ``` - -1. Verify the GitLab services are running: - - ```shell - sudo gitlab-ctl status - ``` - - The output should be similar to the following: - - ```plaintext - run: consul: (pid 30593) 77133s; run: log: (pid 29912) 77156s - run: logrotate: (pid 23449) 3341s; run: log: (pid 29794) 77175s - run: node-exporter: (pid 30613) 77133s; run: log: (pid 29824) 77170s - run: postgres-exporter: (pid 30620) 77132s; run: log: (pid 29894) 77163s - run: postgresql: (pid 30630) 77132s; run: log: (pid 29618) 77181s - run: repmgrd: (pid 30639) 77132s; run: log: (pid 29985) 77150s - ``` - -<div align="right"> - <a type="button" class="btn btn-default" href="#setup-components"> - Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> - </a> -</div> - -#### PostgreSQL secondary nodes - -1. On both the secondary nodes, add the same configuration specified above for the primary node - with an additional setting that will inform `gitlab-ctl` that they are standby nodes initially - and there's no need to attempt to register them as a primary node: - - ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul - roles ['postgres_role'] - - # PostgreSQL configuration - postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' + patroni['postgresql']['max_wal_senders'] = 4 + patroni['postgresql']['max_replication_slots'] = 4 + # Incoming recommended value for max connections is 500. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + patroni['postgresql']['max_connections'] = 500 # Disable automatic database migrations gitlab_rails['auto_migrate'] = false # Configure the Consul agent + consul['enable'] = true consul['services'] = %w(postgresql) - - # Specify if a node should attempt to be primary on initialization. - repmgr['master_on_initialization'] = false + ## Enable service discovery for Prometheus + consul['monitoring_service_discovery'] = true # START user configuration # Please set the real values as explained in Required Information section @@ -818,34 +737,31 @@ in the second step, do not supply the `EXTERNAL_URL` value. postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value postgresql['sql_user_password'] = '<postgresql_password_hash>' - # Set `max_wal_senders` to one more than the number of database nodes in the cluster. - # This is used to prevent replication from using up all of the - # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 # Replace XXX.XXX.XXX.XXX/YY with Network Address - postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['enable'] = true - consul['monitoring_service_discovery'] = true + postgresql['trust_auth_cidr_addresses'] = %w(10.6.0.0/24) # Set the network addresses that the exporters will listen on for monitoring node_exporter['listen_address'] = '0.0.0.0:9100' postgres_exporter['listen_address'] = '0.0.0.0:9187' - postgres_exporter['dbname'] = 'gitlabhq_production' - postgres_exporter['password'] = '<postgresql_password_hash>' ## The IPs of the Consul server nodes ## You can also use FQDNs and intermix them with IPs consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), } + # # END user configuration ``` +PostgreSQL, with Patroni managing its failover, will default to use `pg_rewind` by default to handle conflicts. +Like most failover handling methods, this has a small chance of leading to data loss. +Learn more about the various [Patroni replication methods](../postgresql/replication_and_failover.md#selecting-the-appropriate-patroni-replication-method). + +1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace + the file of the same name on this server. If that file is not on this server, + add the file from your Consul server to this server. + 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. Advanced [configuration options](https://docs.gitlab.com/omnibus/settings/database.html) @@ -876,84 +792,25 @@ SSH in to the **primary node**: 1. Exit the database prompt by typing `\q` and Enter. -1. Verify the cluster is initialized with one node: +1. Check the status of the leader and cluster: ```shell - gitlab-ctl repmgr cluster show + gitlab-ctl patroni members ``` The output should be similar to the following: ```plaintext - Role | Name | Upstream | Connection String - ----------+----------|----------|---------------------------------------- - * master | HOSTNAME | | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr + | Cluster | Member | Host | Role | State | TL | Lag in MB | Pending restart | + |---------------|-----------------------------------|-----------|--------|---------|-----|-----------|-----------------| + | postgresql-ha | <PostgreSQL primary hostname> | 10.6.0.31 | Leader | running | 175 | | * | + | postgresql-ha | <PostgreSQL secondary 1 hostname> | 10.6.0.32 | | running | 175 | 0 | * | + | postgresql-ha | <PostgreSQL secondary 2 hostname> | 10.6.0.33 | | running | 175 | 0 | * | ``` -1. Note down the hostname or IP address in the connection string: `host=HOSTNAME`. We will - refer to the hostname in the next section as `<primary_node_name>`. If the value - is not an IP address, it will need to be a resolvable name (via DNS or - `/etc/hosts`) - -SSH in to the **secondary node**: - -1. Set up the repmgr standby: - - ```shell - gitlab-ctl repmgr standby setup <primary_node_name> - ``` - - Do note that this will remove the existing data on the node. The command - has a wait time. - - The output should be similar to the following: - - ```console - Doing this will delete the entire contents of /var/opt/gitlab/postgresql/data - If this is not what you want, hit Ctrl-C now to exit - To skip waiting, rerun with the -w option - Sleeping for 30 seconds - Stopping the database - Removing the data - Cloning the data - Starting the database - Registering the node with the cluster - ok: run: repmgrd: (pid 19068) 0s - ``` - -Before moving on, make sure the databases are configured correctly. Run the -following command on the **primary** node to verify that replication is working -properly and the secondary nodes appear in the cluster: - -```shell -gitlab-ctl repmgr cluster show -``` - -The output should be similar to the following: - -```plaintext -Role | Name | Upstream | Connection String -----------+---------|-----------|------------------------------------------------ -* master | MASTER | | host=<primary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr -``` - -If the 'Role' column for any node says "FAILED", check the +If the 'State' column for any node doesn't say "running", check the [Troubleshooting section](troubleshooting.md) before proceeding. -Also, check that the `repmgr-check-master` command works successfully on each node: - -```shell -su - gitlab-consul -gitlab-ctl repmgr-check-master || echo 'This node is a standby repmgr node' -``` - -This command relies on exit codes to tell Consul whether a particular node is a master -or secondary. The most important thing here is that this command does not produce errors. -If there are errors it's most likely due to incorrect `gitlab-consul` database user permissions. -Check the [Troubleshooting section](troubleshooting.md) before proceeding. - <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> @@ -971,7 +828,7 @@ The following IPs will be used as an example: 1. On each PgBouncer node, edit `/etc/gitlab/gitlab.rb`, and replace `<consul_password_hash>` and `<pgbouncer_password_hash>` with the - password hashes you [set up previously](#postgresql-primary-node): + password hashes you [set up previously](#postgresql-nodes): ```ruby # Disable all components except Pgbouncer and Consul agent @@ -979,15 +836,16 @@ The following IPs will be used as an example: # Configure PgBouncer pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul) - pgbouncer['users'] = { - 'gitlab-consul': { - password: '<consul_password_hash>' - }, - 'pgbouncer': { - password: '<pgbouncer_password_hash>' - } + 'gitlab-consul': { + password: '<consul_password_hash>' + }, + 'pgbouncer': { + password: '<pgbouncer_password_hash>' + } } + # Incoming recommended value for max db connections is 150. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + pgbouncer['max_db_connections'] = 150 # Configure Consul agent consul['watchers'] = %w(postgresql) diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 91b3b022365..093869d331b 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -422,9 +422,9 @@ install the necessary dependencies from step 1, and add the GitLab package repository from step 2. When installing GitLab in the second step, do not supply the `EXTERNAL_URL` value. -#### PostgreSQL primary node +#### PostgreSQL nodes -1. SSH in to the PostgreSQL primary node. +1. SSH in to one of the PostgreSQL nodes. 1. Generate a password hash for the PostgreSQL username/password pair. This assumes you will use the default username of `gitlab` (recommended). The command will request a password and confirmation. Use the value that is output by this command in the next @@ -452,23 +452,33 @@ in the second step, do not supply the `EXTERNAL_URL` value. sudo gitlab-ctl pg-password-md5 gitlab-consul ``` -1. On the primary database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: +1. On every database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul + # Disable all components except PostgreSQL, Patroni, and Consul roles ['postgres_role'] # PostgreSQL configuration postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' + + # Enable Patroni + patroni['enable'] = true + # Set `max_wal_senders` to one more than the number of database nodes in the cluster. + # This is used to prevent replication from using up all of the + # available database connections. + patroni['postgresql']['max_wal_senders'] = 4 + patroni['postgresql']['max_replication_slots'] = 4 + # Incoming recommended value for max connections is 500. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + patroni['postgresql']['max_connections'] = 500 # Disable automatic database migrations gitlab_rails['auto_migrate'] = false # Configure the Consul agent + consul['enable'] = true consul['services'] = %w(postgresql) + ## Enable service discovery for Prometheus + consul['monitoring_service_discovery'] = true # START user configuration # Please set the real values as explained in Required Information section @@ -477,18 +487,9 @@ in the second step, do not supply the `EXTERNAL_URL` value. postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value postgresql['sql_user_password'] = '<postgresql_password_hash>' - # Set `max_wal_senders` to one more than the number of database nodes in the cluster. - # This is used to prevent replication from using up all of the - # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 # Replace XXX.XXX.XXX.XXX/YY with Network Address postgresql['trust_auth_cidr_addresses'] = %w(10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['monitoring_service_discovery'] = true # Set the network addresses that the exporters will listen on for monitoring node_exporter['listen_address'] = '0.0.0.0:9100' @@ -503,70 +504,9 @@ in the second step, do not supply the `EXTERNAL_URL` value. # END user configuration ``` -1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace - the file of the same name on this server. If that file is not on this server, - add the file from your Consul server to this server. - -1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -<div align="right"> - <a type="button" class="btn btn-default" href="#setup-components"> - Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> - </a> -</div> - -#### PostgreSQL secondary nodes - -1. On both the secondary nodes, add the same configuration specified above for the primary node - with an additional setting (`repmgr['master_on_initialization'] = false`) that will inform `gitlab-ctl` that they are standby nodes initially - and there's no need to attempt to register them as a primary node: - - ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul - roles ['postgres_role'] - - # PostgreSQL configuration - postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' - - # Disable automatic database migrations - gitlab_rails['auto_migrate'] = false - - # Configure the Consul agent - consul['services'] = %w(postgresql) - - # Specify if a node should attempt to be primary on initialization. - repmgr['master_on_initialization'] = false - - # Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value - postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' - # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value - postgresql['sql_user_password'] = '<postgresql_password_hash>' - # Set `max_wal_senders` to one more than the number of database nodes in the cluster. - # This is used to prevent replication from using up all of the - # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 - - # Replace with your network addresses - postgresql['trust_auth_cidr_addresses'] = %w(10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['monitoring_service_discovery'] = true - - # Set the network addresses that the exporters will listen on for monitoring - node_exporter['listen_address'] = '0.0.0.0:9100' - postgres_exporter['listen_address'] = '0.0.0.0:9187' - - ## The IPs of the Consul server nodes - ## You can also use FQDNs and intermix them with IPs - consul['configuration'] = { - retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), - } - ``` +PostgreSQL, with Patroni managing its failover, will default to use `pg_rewind` by default to handle conflicts. +Like most failover handling methods, this has a small chance of leading to data loss. +Learn more about the various [Patroni replication methods](../postgresql/replication_and_failover.md#selecting-the-appropriate-patroni-replication-method). 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace the file of the same name on this server. If that file is not on this server, @@ -601,84 +541,25 @@ SSH in to the **primary node**: 1. Exit the database prompt by typing `\q` and Enter. -1. Verify the cluster is initialized with one node: +1. Check the status of the leader and cluster: ```shell - gitlab-ctl repmgr cluster show + gitlab-ctl patroni members ``` The output should be similar to the following: ```plaintext - Role | Name | Upstream | Connection String - ----------+----------|----------|---------------------------------------- - * master | HOSTNAME | | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr + | Cluster | Member | Host | Role | State | TL | Lag in MB | Pending restart | + |---------------|-----------------------------------|-----------|--------|---------|-----|-----------|-----------------| + | postgresql-ha | <PostgreSQL primary hostname> | 10.6.0.21 | Leader | running | 175 | | * | + | postgresql-ha | <PostgreSQL secondary 1 hostname> | 10.6.0.22 | | running | 175 | 0 | * | + | postgresql-ha | <PostgreSQL secondary 2 hostname> | 10.6.0.23 | | running | 175 | 0 | * | ``` -1. Note down the hostname or IP address in the connection string: `host=HOSTNAME`. We will - refer to the hostname in the next section as `<primary_node_name>`. If the value - is not an IP address, it will need to be a resolvable name (via DNS or - `/etc/hosts`) - -SSH in to the **secondary node**: - -1. Set up the repmgr standby: - - ```shell - gitlab-ctl repmgr standby setup <primary_node_name> - ``` - - Do note that this will remove the existing data on the node. The command - has a wait time. - - The output should be similar to the following: - - ```console - Doing this will delete the entire contents of /var/opt/gitlab/postgresql/data - If this is not what you want, hit Ctrl-C now to exit - To skip waiting, rerun with the -w option - Sleeping for 30 seconds - Stopping the database - Removing the data - Cloning the data - Starting the database - Registering the node with the cluster - ok: run: repmgrd: (pid 19068) 0s - ``` - -Before moving on, make sure the databases are configured correctly. Run the -following command on the **primary** node to verify that replication is working -properly and the secondary nodes appear in the cluster: - -```shell -gitlab-ctl repmgr cluster show -``` - -The output should be similar to the following: - -```plaintext -Role | Name | Upstream | Connection String -----------+---------|-----------|------------------------------------------------ -* master | MASTER | | host=<primary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr -``` - -If the 'Role' column for any node says "FAILED", check the +If the 'State' column for any node doesn't say "running", check the [Troubleshooting section](troubleshooting.md) before proceeding. -Also, check that the `repmgr-check-master` command works successfully on each node: - -```shell -su - gitlab-consul -gitlab-ctl repmgr-check-master || echo 'This node is a standby repmgr node' -``` - -This command relies on exit codes to tell Consul whether a particular node is a master -or secondary. The most important thing here is that this command does not produce errors. -If there are errors it's most likely due to incorrect `gitlab-consul` database user permissions. -Check the [Troubleshooting section](troubleshooting.md) before proceeding. - <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> @@ -696,7 +577,7 @@ The following IPs will be used as an example: 1. On each PgBouncer node, edit `/etc/gitlab/gitlab.rb`, and replace `<consul_password_hash>` and `<pgbouncer_password_hash>` with the - password hashes you [set up previously](#postgresql-primary-node): + password hashes you [set up previously](#postgresql-nodes): ```ruby # Disable all components except Pgbouncer and Consul agent @@ -704,15 +585,16 @@ The following IPs will be used as an example: # Configure PgBouncer pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul) - pgbouncer['users'] = { - 'gitlab-consul': { - password: '<consul_password_hash>' - }, - 'pgbouncer': { - password: '<pgbouncer_password_hash>' - } + 'gitlab-consul': { + password: '<consul_password_hash>' + }, + 'pgbouncer': { + password: '<pgbouncer_password_hash>' + } } + # Incoming recommended value for max db connections is 150. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + pgbouncer['max_db_connections'] = 150 # Configure Consul agent consul['watchers'] = %w(postgresql) diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 6d7511f67db..16ad866a108 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -671,9 +671,9 @@ install the necessary dependencies from step 1, and add the GitLab package repository from step 2. When installing GitLab in the second step, do not supply the `EXTERNAL_URL` value. -#### PostgreSQL primary node +#### PostgreSQL nodes -1. SSH in to the PostgreSQL primary node. +1. SSH in to one of the PostgreSQL nodes. 1. Generate a password hash for the PostgreSQL username/password pair. This assumes you will use the default username of `gitlab` (recommended). The command will request a password and confirmation. Use the value that is output by this command in the next @@ -701,114 +701,33 @@ in the second step, do not supply the `EXTERNAL_URL` value. sudo gitlab-ctl pg-password-md5 gitlab-consul ``` -1. On the primary database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: +1. On every database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul + # Disable all components except PostgreSQL, Patroni, and Consul roles ['postgres_role'] # PostgreSQL configuration postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' - # Disable automatic database migrations - gitlab_rails['auto_migrate'] = false - - # Configure the Consul agent - consul['services'] = %w(postgresql) - - # START user configuration - # Please set the real values as explained in Required Information section - # - # Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value - postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' - # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value - postgresql['sql_user_password'] = '<postgresql_password_hash>' + # Enable Patroni + patroni['enable'] = true # Set `max_wal_senders` to one more than the number of database nodes in the cluster. # This is used to prevent replication from using up all of the # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 - - # Replace XXX.XXX.XXX.XXX/YY with Network Address - postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['enable'] = true - consul['monitoring_service_discovery'] = true - - # Set the network addresses that the exporters will listen on for monitoring - node_exporter['listen_address'] = '0.0.0.0:9100' - postgres_exporter['listen_address'] = '0.0.0.0:9187' - postgres_exporter['dbname'] = 'gitlabhq_production' - postgres_exporter['password'] = '<postgresql_password_hash>' - - ## The IPs of the Consul server nodes - ## You can also use FQDNs and intermix them with IPs - consul['configuration'] = { - retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), - } - # - # END user configuration - ``` - -1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -1. You can list the current PostgreSQL primary, secondary nodes status via: - - ```shell - sudo /opt/gitlab/bin/gitlab-ctl repmgr cluster show - ``` - -1. Verify the GitLab services are running: - - ```shell - sudo gitlab-ctl status - ``` - - The output should be similar to the following: - - ```plaintext - run: consul: (pid 30593) 77133s; run: log: (pid 29912) 77156s - run: logrotate: (pid 23449) 3341s; run: log: (pid 29794) 77175s - run: node-exporter: (pid 30613) 77133s; run: log: (pid 29824) 77170s - run: postgres-exporter: (pid 30620) 77132s; run: log: (pid 29894) 77163s - run: postgresql: (pid 30630) 77132s; run: log: (pid 29618) 77181s - run: repmgrd: (pid 30639) 77132s; run: log: (pid 29985) 77150s - ``` - -<div align="right"> - <a type="button" class="btn btn-default" href="#setup-components"> - Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> - </a> -</div> - -#### PostgreSQL secondary nodes - -1. On both the secondary nodes, add the same configuration specified above for the primary node - with an additional setting that will inform `gitlab-ctl` that they are standby nodes initially - and there's no need to attempt to register them as a primary node: - - ```ruby - # Disable all components except PostgreSQL and Repmgr and Consul - roles ['postgres_role'] - - # PostgreSQL configuration - postgresql['listen_address'] = '0.0.0.0' - postgresql['hot_standby'] = 'on' - postgresql['wal_level'] = 'replica' - postgresql['shared_preload_libraries'] = 'repmgr_funcs' + patroni['postgresql']['max_wal_senders'] = 4 + patroni['postgresql']['max_replication_slots'] = 4 + # Incoming recommended value for max connections is 500. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + patroni['postgresql']['max_connections'] = 500 # Disable automatic database migrations gitlab_rails['auto_migrate'] = false # Configure the Consul agent + consul['enable'] = true consul['services'] = %w(postgresql) - - # Specify if a node should attempt to be primary on initialization. - repmgr['master_on_initialization'] = false + ## Enable service discovery for Prometheus + consul['monitoring_service_discovery'] = true # START user configuration # Please set the real values as explained in Required Information section @@ -817,34 +736,31 @@ in the second step, do not supply the `EXTERNAL_URL` value. postgresql['pgbouncer_user_password'] = '<pgbouncer_password_hash>' # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value postgresql['sql_user_password'] = '<postgresql_password_hash>' - # Set `max_wal_senders` to one more than the number of database nodes in the cluster. - # This is used to prevent replication from using up all of the - # available database connections. - postgresql['max_wal_senders'] = 4 - postgresql['max_replication_slots'] = 4 # Replace XXX.XXX.XXX.XXX/YY with Network Address - postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) - - ## Enable service discovery for Prometheus - consul['enable'] = true - consul['monitoring_service_discovery'] = true + postgresql['trust_auth_cidr_addresses'] = %w(10.6.0.0/24) # Set the network addresses that the exporters will listen on for monitoring node_exporter['listen_address'] = '0.0.0.0:9100' postgres_exporter['listen_address'] = '0.0.0.0:9187' - postgres_exporter['dbname'] = 'gitlabhq_production' - postgres_exporter['password'] = '<postgresql_password_hash>' ## The IPs of the Consul server nodes ## You can also use FQDNs and intermix them with IPs consul['configuration'] = { retry_join: %w(10.6.0.11 10.6.0.12 10.6.0.13), } + # # END user configuration ``` +PostgreSQL, with Patroni managing its failover, will default to use `pg_rewind` by default to handle conflicts. +Like most failover handling methods, this has a small chance of leading to data loss. +Learn more about the various [Patroni replication methods](../postgresql/replication_and_failover.md#selecting-the-appropriate-patroni-replication-method). + +1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace + the file of the same name on this server. If that file is not on this server, + add the file from your Consul server to this server. + 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. Advanced [configuration options](https://docs.gitlab.com/omnibus/settings/database.html) @@ -874,84 +790,25 @@ SSH in to the **primary node**: 1. Exit the database prompt by typing `\q` and Enter. -1. Verify the cluster is initialized with one node: +1. Check the status of the leader and cluster: ```shell - gitlab-ctl repmgr cluster show + gitlab-ctl patroni members ``` The output should be similar to the following: ```plaintext - Role | Name | Upstream | Connection String - ----------+----------|----------|---------------------------------------- - * master | HOSTNAME | | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr + | Cluster | Member | Host | Role | State | TL | Lag in MB | Pending restart | + |---------------|-----------------------------------|-----------|--------|---------|-----|-----------|-----------------| + | postgresql-ha | <PostgreSQL primary hostname> | 10.6.0.31 | Leader | running | 175 | | * | + | postgresql-ha | <PostgreSQL secondary 1 hostname> | 10.6.0.32 | | running | 175 | 0 | * | + | postgresql-ha | <PostgreSQL secondary 2 hostname> | 10.6.0.33 | | running | 175 | 0 | * | ``` -1. Note down the hostname or IP address in the connection string: `host=HOSTNAME`. We will - refer to the hostname in the next section as `<primary_node_name>`. If the value - is not an IP address, it will need to be a resolvable name (via DNS or - `/etc/hosts`) - -SSH in to the **secondary node**: - -1. Set up the repmgr standby: - - ```shell - gitlab-ctl repmgr standby setup <primary_node_name> - ``` - - Do note that this will remove the existing data on the node. The command - has a wait time. - - The output should be similar to the following: - - ```console - Doing this will delete the entire contents of /var/opt/gitlab/postgresql/data - If this is not what you want, hit Ctrl-C now to exit - To skip waiting, rerun with the -w option - Sleeping for 30 seconds - Stopping the database - Removing the data - Cloning the data - Starting the database - Registering the node with the cluster - ok: run: repmgrd: (pid 19068) 0s - ``` - -Before moving on, make sure the databases are configured correctly. Run the -following command on the **primary** node to verify that replication is working -properly and the secondary nodes appear in the cluster: - -```shell -gitlab-ctl repmgr cluster show -``` - -The output should be similar to the following: - -```plaintext -Role | Name | Upstream | Connection String -----------+---------|-----------|------------------------------------------------ -* master | MASTER | | host=<primary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr - standby | STANDBY | MASTER | host=<secondary_node_name> user=gitlab_repmgr dbname=gitlab_repmgr -``` - -If the 'Role' column for any node says "FAILED", check the +If the 'State' column for any node doesn't say "running", check the [Troubleshooting section](troubleshooting.md) before proceeding. -Also, check that the `repmgr-check-master` command works successfully on each node: - -```shell -su - gitlab-consul -gitlab-ctl repmgr-check-master || echo 'This node is a standby repmgr node' -``` - -This command relies on exit codes to tell Consul whether a particular node is a master -or secondary. The most important thing here is that this command does not produce errors. -If there are errors it's most likely due to incorrect `gitlab-consul` database user permissions. -Check the [Troubleshooting section](troubleshooting.md) before proceeding. - <div align="right"> <a type="button" class="btn btn-default" href="#setup-components"> Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i> @@ -969,7 +826,7 @@ The following IPs will be used as an example: 1. On each PgBouncer node, edit `/etc/gitlab/gitlab.rb`, and replace `<consul_password_hash>` and `<pgbouncer_password_hash>` with the - password hashes you [set up previously](#postgresql-primary-node): + password hashes you [set up previously](#postgresql-nodes): ```ruby # Disable all components except Pgbouncer and Consul agent @@ -977,15 +834,16 @@ The following IPs will be used as an example: # Configure PgBouncer pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul) - pgbouncer['users'] = { - 'gitlab-consul': { - password: '<consul_password_hash>' - }, - 'pgbouncer': { - password: '<pgbouncer_password_hash>' - } + 'gitlab-consul': { + password: '<consul_password_hash>' + }, + 'pgbouncer': { + password: '<pgbouncer_password_hash>' + } } + # Incoming recommended value for max db connections is 150. See https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5691. + pgbouncer['max_db_connections'] = 150 # Configure Consul agent consul['watchers'] = %w(postgresql) diff --git a/doc/administration/reference_architectures/troubleshooting.md b/doc/administration/reference_architectures/troubleshooting.md index cd1a2bc91df..8fcdb457d6a 100644 --- a/doc/administration/reference_architectures/troubleshooting.md +++ b/doc/administration/reference_architectures/troubleshooting.md @@ -514,39 +514,24 @@ See the suggested fix [in Geo documentation](../geo/replication/troubleshooting. See the suggested fix [in Geo documentation](../geo/replication/troubleshooting.md#message-log--invalid-ip-mask-md5-name-or-service-not-known). -## Troubleshooting PostgreSQL +## Troubleshooting PostgreSQL with Patroni -In case you are experiencing any issues connecting through PgBouncer, the first place to check is always the logs: +In case you are experiencing any issues connecting through PgBouncer, the first place to check is always the logs for PostgreSQL (which is run through Patroni): ```shell -sudo gitlab-ctl tail postgresql +sudo gitlab-ctl tail patroni ``` -### Consul and PostgreSQL changes not taking effect +### Consul and PostgreSQL with Patroni changes not taking effect Due to the potential impacts, `gitlab-ctl reconfigure` only reloads Consul and PostgreSQL, it will not restart the services. However, not all changes can be activated by reloading. -To restart either service, run `gitlab-ctl restart SERVICE` +To restart either service, run `gitlab-ctl restart consul` or `gitlab-ctl restart patroni` respectively. -For PostgreSQL, it is usually safe to restart the master node by default. Automatic failover defaults to a 1 minute timeout. Provided the database returns before then, nothing else needs to be done. To be safe, you can stop `repmgrd` on the standby nodes first with `gitlab-ctl stop repmgrd`, then start afterwards with `gitlab-ctl start repmgrd`. +For PostgreSQL with Patroni, to prevent the primary node from being failed over automatically, it's safest to stop all secondaries first, then restart the primary and finally restart the secondaries again. On the Consul server nodes, it is important to restart the Consul service in a controlled fashion. Read our [Consul documentation](../consul.md#restart-consul) for instructions on how to restart the service. -### `gitlab-ctl repmgr-check-master` command produces errors - -If this command displays errors about database permissions it is likely that something failed during -install, resulting in the `gitlab-consul` database user getting incorrect permissions. Follow these -steps to fix the problem: - -1. On the master database node, connect to the database prompt - `gitlab-psql -d template1` -1. Delete the `gitlab-consul` user - `DROP USER "gitlab-consul";` -1. Exit the database prompt - `\q` -1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) and the user will be re-added with the proper permissions. -1. Change to the `gitlab-consul` user - `su - gitlab-consul` -1. Try the check command again - `gitlab-ctl repmgr-check-master`. - -Now there should not be errors. If errors still occur then there is another problem. - ### PgBouncer error `ERROR: pgbouncer cannot connect to server` You may get this error when running `gitlab-rake gitlab:db:configure` or you diff --git a/doc/api/appearance.md b/doc/api/appearance.md index b33e41cf271..d03fc94cfcf 100644 --- a/doc/api/appearance.md +++ b/doc/api/appearance.md @@ -60,8 +60,8 @@ PUT /application/appearance | `favicon` | mixed | no | Instance favicon in `.ico` or `.png` format | `new_project_guidelines` | string | no | Markdown text shown on the new project page | `profile_image_guidelines` | string | no | Markdown text shown on the profile page below Public Avatar -| `header_message` | string | no | Message within the system header bar -| `footer_message` | string | no | Message within the system footer bar +| `header_message` | string | no | Message in the system header bar +| `footer_message` | string | no | Message in the system footer bar | `message_background_color` | string | no | Background color for the system header / footer bar | `message_font_color` | string | no | Font color for the system header / footer bar | `email_header_and_footer_enabled` | boolean | no | Add header and footer to all outgoing emails if enabled diff --git a/doc/api/avatar.md b/doc/api/avatar.md index 31f254c7986..ccaa50eedd8 100644 --- a/doc/api/avatar.md +++ b/doc/api/avatar.md @@ -30,7 +30,7 @@ Parameters: | Attribute | Type | Required | Description | |:----------|:--------|:---------|:----------------------------------------------------------------------------------------------------------------------------------------| | `email` | string | yes | Public email address of the user. | -| `size` | integer | no | Single pixel dimension (since images are squares). Only used for avatar lookups at `Gravatar` or at the configured `Libravatar` server. | +| `size` | integer | no | Single pixel dimension (because images are squares). Only used for avatar lookups at `Gravatar` or at the configured `Libravatar` server. | Example request: diff --git a/doc/api/boards.md b/doc/api/boards.md index aff82daa1bf..e86f0d846ed 100644 --- a/doc/api/boards.md +++ b/doc/api/boards.md @@ -426,7 +426,7 @@ POST /projects/:id/boards/:board_id/lists NOTE: Label, assignee and milestone arguments are mutually exclusive, that is, only one of them are accepted in a request. -Check the [Issue Board docs](../user/project/issue_board.md) +Check the [Issue Board documentation](../user/project/issue_board.md) for more information regarding the required license for each list type. ```shell diff --git a/doc/api/container_registry.md b/doc/api/container_registry.md index 5f3dbcb1ab0..df71e0ccb01 100644 --- a/doc/api/container_registry.md +++ b/doc/api/container_registry.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/55978) in GitLab 11.8. -This is the API docs of the [GitLab Container Registry](../user/packages/container_registry/index.md). +This is the API documentation of the [GitLab Container Registry](../user/packages/container_registry/index.md). ## List registry repositories diff --git a/doc/api/job_artifacts.md b/doc/api/job_artifacts.md index 229b90282f0..a6b5e9e4711 100644 --- a/doc/api/job_artifacts.md +++ b/doc/api/job_artifacts.md @@ -33,7 +33,7 @@ To use this in a [`script` definition](../ci/yaml/README.md#script) inside - The `JOB-TOKEN` header with the GitLab-provided `CI_JOB_TOKEN` variable. For example, the following job downloads the artifacts of the job with ID - `42`. Note that the command is wrapped into single quotes since it contains a + `42`. Note that the command is wrapped into single quotes because it contains a colon (`:`): ```yaml @@ -99,7 +99,7 @@ To use this in a [`script` definition](../ci/yaml/README.md#script) inside - The `JOB-TOKEN` header with the GitLab-provided `CI_JOB_TOKEN` variable. For example, the following job downloads the artifacts of the `test` job of the `master` branch. Note that the command is wrapped into single quotes - since it contains a colon (`:`): + because it contains a colon (`:`): ```yaml artifact_download: @@ -130,7 +130,7 @@ Possible response status codes: > Introduced in GitLab 10.0 -Download a single artifact file from a job with a specified ID from within +Download a single artifact file from a job with a specified ID from inside the job's artifacts zipped archive. The file is extracted from the archive and streamed to the client. @@ -165,7 +165,7 @@ Possible response status codes: > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/23538) in GitLab 11.5. Download a single artifact file for a specific job of the latest successful -pipeline for the given reference name from within the job's artifacts archive. +pipeline for the given reference name from inside the job's artifacts archive. The file is extracted from the archive and streamed to the client. In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/201784) and later, artifacts diff --git a/doc/api/members.md b/doc/api/members.md index 05914b50cd7..66439c4e22d 100644 --- a/doc/api/members.md +++ b/doc/api/members.md @@ -468,7 +468,7 @@ DELETE /projects/:id/members/:user_id | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project or group](README.md#namespaced-path-encoding) owned by the authenticated user | | `user_id` | integer | yes | The user ID of the member | -| `unassign_issuables` | boolean | false | Flag indicating if the removed member should be unassigned from any issues or merge requests within given group or project | +| `unassign_issuables` | boolean | false | Flag indicating if the removed member should be unassigned from any issues or merge requests inside a given group or project | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/:id/members/:user_id" diff --git a/doc/api/oauth2.md b/doc/api/oauth2.md index 1607c6910c9..a80a97890ba 100644 --- a/doc/api/oauth2.md +++ b/doc/api/oauth2.md @@ -19,12 +19,12 @@ documentation. This functionality is based on the GitLab currently supports the following authorization flows: -- **Authorization code with [Proof Key for Code Exchange (PKCE)](https://tools.ietf.org/html/rfc7636):** +- **Authorization code with [Proof Key for Code Exchange (PKCE)](https://tools.ietf.org/html/rfc7636):** Most secure. Without PKCE, you'd have to include client secrets on mobile clients, and is recommended for both client and server aoos. -- **Authorization code:** Secure and common flow. Recommended option for secure +- **Authorization code:** Secure and common flow. Recommended option for secure server-side apps. -- **Implicit grant:** Originally designed for user-agent only apps, such as +- **Implicit grant:** Originally designed for user-agent only apps, such as single page web apps running on GitLab Pages). The [IETF](https://tools.ietf.org/html/draft-ietf-oauth-security-topics-09#section-2.1.2) recommends against Implicit grant flow. @@ -159,7 +159,7 @@ The authorization code flow is essentially the same as You should then use `code` to request an access token. -1. Once you have the authorization code you can request an `access_token` using the +1. After you have the authorization code you can request an `access_token` using the code. You can do that by using any HTTP client. In the following example, we are using Ruby's `rest-client`: @@ -189,7 +189,7 @@ You can now make requests to the API with the access token returned. ### Implicit grant flow NOTE: -For a detailed flow diagram, see the [RFC specification](https://tools.ietf.org/html/rfc6749#section-4.2). +For a detailed flow diagram, see the [RFC specification](https://tools.ietf.org/html/rfc6749#section-4.2). WARNING: The Implicit grant flow is inherently insecure. The IETF plans to remove it in diff --git a/doc/api/packages.md b/doc/api/packages.md index 2f588954ae1..a0d966fdd88 100644 --- a/doc/api/packages.md +++ b/doc/api/packages.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Packages API -This is the API docs of [GitLab Packages](../administration/packages/index.md). +This is the API documentation of [GitLab Packages](../administration/packages/index.md). ## List packages @@ -68,7 +68,7 @@ Example response: ] ``` -By default, the `GET` request returns 20 results, since the API is [paginated](README.md#pagination). +By default, the `GET` request returns 20 results, because the API is [paginated](README.md#pagination). ### Within a group @@ -159,7 +159,7 @@ Example response: ] ``` -By default, the `GET` request returns 20 results, since the API is [paginated](README.md#pagination). +By default, the `GET` request returns 20 results, because the API is [paginated](README.md#pagination). The `_links` object contains the following properties: @@ -316,7 +316,7 @@ Example response: ] ``` -By default, the `GET` request returns 20 results, since the API is [paginated](README.md#pagination). +By default, the `GET` request returns 20 results, because the API is [paginated](README.md#pagination). ## Delete a project package diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 00f4cf90481..a6c2339495b 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -69,11 +69,12 @@ It picks reviewers and maintainers from the list at the [engineering projects](https://about.gitlab.com/handbook/engineering/projects/) page, with these behaviors: -1. It doesn't pick people whose [GitLab status](../user/profile/index.md#current-status) - contains the string 'OOO', or the emoji is `:palm_tree:` or `:beach:`. +1. It doesn't pick people whose Slack or [GitLab status](../user/profile/index.md#current-status): + - contains the string 'OOO', 'PTO', 'Parental Leave', or 'Friends and Family' + - emoji is `:palm_tree:`, `:beach:`, `:beach_umbrella:`, `:beach_with_umbrella:`, `:ferris_wheel:`, `:thermometer:`, `:face_with_thermometer:`, `:red_circle:`, `:bulb:`, `:sun_with_face:`. 1. [Trainee maintainers](https://about.gitlab.com/handbook/engineering/workflow/code-review/#trainee-maintainer) are three times as likely to be picked as other reviewers. -1. People whose [GitLab status](../user/profile/index.md#current-status) emoji +1. Team members whose Slack or [GitLab status](../user/profile/index.md#current-status) emoji is `:large_blue_circle:` are more likely to be picked. This applies to both reviewers and trainee maintainers. - Reviewers with `:large_blue_circle:` are two times as likely to be picked as other reviewers. - Trainee maintainers with `:large_blue_circle:` are four times as likely to be picked as other reviewers. diff --git a/doc/development/usage_ping/metrics_dictionary.md b/doc/development/usage_ping/metrics_dictionary.md new file mode 100644 index 00000000000..bae79689f3b --- /dev/null +++ b/doc/development/usage_ping/metrics_dictionary.md @@ -0,0 +1,73 @@ +--- +stage: Growth +group: Product Intelligence +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Metrics Dictionary Guide + +This guide describes Metrics Dictionary and how it's implemented + +## Metrics Definition and validation + +We are using [JSON Schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/schema.json) to validate the metrics definition. + +This process is meant to ensure consistent and valid metrics defined for Usage Ping. All metrics *must*: + +- Comply with the definied [JSON schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/schema.json). +- Have a unique `full_path` . +- Have an owner. + +All metrics are stored in YAML files: + +- [`config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/metrics) + +Each metric is definied in a separate YAML file consisting of a number of fields: + +| Field | Required | Additional information | +|---------------------|----------|----------------------------------------------------------------| +| `name` | yes | | +| `description` | yes | | +| `value_type` | yes | | +| `status` | yes | | +| `default_generation`| yes | Default generation path of the metric. One full_path value. (1) | +| `full_path` | yes | Full path of the metric for one or multiple generations. Path of the metric in Usage Ping payload. (1) | +| `group` | yes | The [group](https://about.gitlab.com/handbook/product/categories/#devops-stages) that owns the metric. | +| `time_frame` | yes | `string`; may be set to a value like "7d" | +| `data_source` | yes | `string`: may be set to a value like `database` or `redis_hll`. | +| `distribution` | yes | The [distribution](https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/#definitions) where the metric applies. | +| `tier` | yes | The [tier]( https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/) where the metric applies. | +| `product_category` | no | The [product category](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/categories.yml) for the metric. | +| `stage` | no | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the metric. | +| `milestone` | no | The milestone when the metric is introduced. | +| `milestone_removed` | no | The milestone when the metric is removed. | +| `introduced_by_url` | no | The URL to the Merge Request that introduced the metric. | + +1. The default generation path is the location of the metric in the Usage Ping payload. + The `full_path` is the list locations for multiple Usage Ping generaations. + +### Example metric definition + +The linked [`uuid`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/license/uuid.yml) +YAML file includes an example metric definition, where the `uuid` metric is the GitLab +instance unique identifier. + +```yaml +name: uuid +description: GitLab instance unique identifier +value_type: string +product_category: collection +stage: growth +status: data_available +default_generation: generation_1 +full_path: + generation_1: uuid + generation_2: license.uuid +milestone: 9.1 +introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1521 +group: group::product intelligence +time_frame: none +data_source: database +distribution: [ee, ce] +tier: ['free', 'starter', 'premium', 'ultimate', 'bronze', 'silver', 'gold'] +``` diff --git a/doc/security/rack_attack.md b/doc/security/rack_attack.md index f159b4f8e21..d80de92501e 100644 --- a/doc/security/rack_attack.md +++ b/doc/security/rack_attack.md @@ -54,11 +54,7 @@ By default, protected paths are: - `/import/github/personal_access_token` - `/admin/session` -This header is included in responses to blocked requests: - -```plaintext -Retry-After: 60 -``` +See [User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md#response-headers) for the headers responded to blocked requests. For example, the following are limited to a maximum 10 requests per minute: diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index bc266216714..5ceff1c02f6 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -7,6 +7,14 @@ type: reference # Account and limit settings **(CORE ONLY)** +## Default projects limit + +You can change the default maximum number of projects that users can create in their personal namespace. +Navigate to **Admin Area > Settings > General**, then expand **Account and Limit**. +You can increase or decrease that `Default projects limit` value. + +- If you set `Default projects limit` to 0, users are not allowed to create projects in their users personal namespace. However, projects can still be created within a group. + ## Max attachment size You can change the maximum file size for attachments in comments and replies in GitLab. diff --git a/doc/user/admin_area/settings/protected_paths.md b/doc/user/admin_area/settings/protected_paths.md index 870735f5be7..a03156511e2 100644 --- a/doc/user/admin_area/settings/protected_paths.md +++ b/doc/user/admin_area/settings/protected_paths.md @@ -28,11 +28,7 @@ GitLab rate limits the following paths with Rack Attack by default: GitLab responds with HTTP status code `429` to POST requests at protected paths that exceed 10 requests per minute per IP address. -This header is included in responses to blocked requests: - -```plaintext -Retry-After: 60 -``` +See [User and IP rate limits](../../admin_area/settings/user_and_ip_rate_limits.md#response-headers) for the headers responded to blocked requests. For example, the following are limited to a maximum 10 requests per minute: diff --git a/doc/user/admin_area/settings/user_and_ip_rate_limits.md b/doc/user/admin_area/settings/user_and_ip_rate_limits.md index 72c05d1115d..7496ef224df 100644 --- a/doc/user/admin_area/settings/user_and_ip_rate_limits.md +++ b/doc/user/admin_area/settings/user_and_ip_rate_limits.md @@ -36,6 +36,25 @@ Retry later It is possible to customize this response text in the admin area. +## Response headers + +> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/731) in GitLab 13.8, the `Rate-Limit` headers. `Retry-After` was introduced in an earlier version. + +When a client exceeds the associated rate limit, the following requests are +blocked. The server may respond with rate-limiting information allowing the +requester to retry after a specific period of time. These information are +attached into the response headers. + +| Header | Example | Description | +|:----------------------|:--------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `RateLimit-Limit` | `60` | The request quota for the client **each minute**. If the rate limit period set in the admin area is different from 1 minute, the value of this header is adjusted to approximately the nearest 60-minute period. | +| `RateLimit-Name` | `throttle_authenticated_web` | Name of the throttle blocking the requests. | +| `RateLimit-Observed` | `67` | Number of requests associated to the client in the time window. | +| `RateLimit-Remaining` | `0` | Remaining quota in the time window. The result of `RateLimit-Limit` - `RateLimit-Remaining`. | +| `RateLimit-Reset` | `30` | An alias of `Retry-After` header. | +| `RateLimit-ResetTime` | `Tue, 05 Jan 2021 11:00:00 GMT` | [RFC2616](https://tools.ietf.org/html/rfc2616#section-3.3.1)-formatted date and time when the request quota is reset. | +| `Retry-After` | `30` | Remaining duration **in seconds** until the quota is reset. This is a [standard HTTP header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After). | + ## Use an HTTP header to bypass rate limiting > [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/622) in GitLab 13.6. diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index 09e38d5048f..674afcb29ee 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -157,6 +157,7 @@ You can use various tools to generate HAR files: - [Insomnia Core](https://insomnia.rest/): API client - [Chrome](https://www.google.com/chrome/): Browser - [Firefox](https://www.mozilla.org/en-US/firefox/): Browser +- [GitLab HAR Recorder](https://gitlab.com/gitlab-org/security-products/har-recorder): Command line WARNING: HAR files may contain sensitive information such as authentication tokens, API keys, and session diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 03434de8d62..8e2768ee5cb 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -704,6 +704,49 @@ security reports without requiring internet access. Alternatively, you can use the variable `SECURE_ANALYZERS_PREFIX` to override the base registry address of the `dast` image. +## On-demand scans + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3. + +An on-demand DAST scan runs outside the DevOps life cycle. Changes in your repository don't trigger +the scan. You must start it manually. + +An on-demand DAST scan: + +- Uses settings in the site profile and scanner profile you select when you run the scan, + instead of those in the `.gitlab-ci.yml` file. +- Is associated with your project's default branch. + +### On-demand scan modes + +An on-demand scan can be run in active or passive mode: + +- _Passive mode_ is the default and runs a ZAP Baseline Scan. +- _Active mode_ runs a ZAP Full Scan which is potentially harmful to the site being scanned. To + minimize the risk of accidental damage, running an active scan requires a [validated site + profile](#site-profile-validation). + +### Run an on-demand DAST scan + +NOTE: +You must have permission to run an on-demand DAST scan against a protected branch. +The default branch is automatically protected. For more information, see +[Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches). + +To run an on-demand DAST scan, you need: + +- A [scanner profile](#create-a-scanner-profile). +- A [site profile](#create-a-site-profile). +- If you are running an active scan the site profile must be [validated](#validate-a-site-profile). + +1. From your project's home page, go to **Security & Compliance > On-demand Scans** in the left sidebar. +1. In **Scanner profile**, select a scanner profile from the dropdown. +1. In **Site profile**, select a site profile from the dropdown. +1. Click **Run scan**. + +The on-demand DAST scan runs and the project's dashboard shows the results. + ## Site profile A site profile describes the attributes of a web site to scan on demand with DAST. A site profile is @@ -714,31 +757,115 @@ A site profile contains the following: - **Profile name**: A name you assign to the site to be scanned. - **Target URL**: The URL that DAST runs against. +## Site profile validation + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233020) in GitLab 13.8. + +Site profile validation reduces the risk of running an active scan against the wrong website. A site +must be validated before an active scan can run against it. The site validation methods are as +follows: + +- _Text file validation_ requires a text file be uploaded to the target site. The text file is + allocated a name and content that is unique to the project. The validation process checks the + file's content. +- _Header validation_ requires the header `Gitlab-On-Demand-DAST` be added to the target site, + with a value unique to the project. The validation process checks that the header is present, and + checks its value. + +Both methods are equivalent in functionality. Use whichever is feasible. + ### Create a site profile To create a site profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. -1. Click **Manage** in the **DAST Profiles** row. -1. Click **New Profile > Site Profile**. -1. Type in a unique **Profile name** and **Target URL** then click **Save profile**. +1. Select **Manage** in the **DAST Profiles** row. +1. Select **New Profile > Site Profile**. +1. Type in a unique **Profile name** and **Target URL** then select **Save profile**. ### Edit a site profile To edit an existing site profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. -1. Click **Manage** in the **DAST Profiles** row. -1. Click **Edit** in the row of the profile to edit. -1. Edit the **Profile name** and **Target URL**, then click **Save profile**. +1. Select **Manage** in the **DAST Profiles** row. +1. Select **Edit** in the row of the profile to edit. +1. Edit the **Profile name** and **Target URL**, then select **Save profile**. ### Delete a site profile To delete an existing site profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. -1. Click **Manage** in the **DAST Profiles** row. -1. Click **{remove}** (Delete profile) in the row of the profile to delete. +1. Select **Manage** in the **DAST Profiles** row. +1. Select **{remove}** (Delete profile) in the row of the profile to delete. + +### Validate a site profile + +To validate a site profile: + +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Select **Manage** in the **DAST Profiles** row. +1. Select **Validate target site** beside the profile to validate. +1. Select the validation method. + 1. For **Text file validation**: + 1. Download the validation file listed in **Step 2**. + 1. Upload the validation file to the host. You can upload the file to the location in + **Step 3** or any location you prefer. + 1. Select **Validate**. + 1. For **Header validation**: + 1. Select the clipboard icon in **Step 2**. + 1. Edit the header of the site to validate, and paste the clipboard content. + 1. Select the input field in **Step 3** and enter the location of the header. + 1. Select **Validate**. + +The site is validated and an active scan can run against it. + +If a validated site profile's target URL is edited, the site is no longer validated. + +#### Validated site profile headers + +The following are code samples of how you could provide the required site profile header in your +application. + +##### Ruby on Rails example for on-demand scan + +Here's how you can add a custom header in a Ruby on Rails application: + +```ruby +class DastWebsiteTargetController < ActionController::Base + def dast_website_target + response.headers['Gitlab-On-Demand-DAST'] = '0dd79c9a-7b29-4e26-a815-eaaf53fcab1c' + head :ok + end +end +``` + +##### Django example for on-demand scan + +Here's how you can add a +[custom header in Django](https://docs.djangoproject.com/en/2.2/ref/request-response/#setting-header-fields): + +```python +class DastWebsiteTargetView(View): + def head(self, *args, **kwargs): + response = HttpResponse() + response['Gitlab-On-Demand-DAST'] = '0dd79c9a-7b29-4e26-a815-eaaf53fcab1c' + + return response +``` + +##### Node (with Express) example for on-demand scan + +Here's how you can add a +[custom header in Node (with Express)](http://expressjs.com/en/5x/api.html#res.append): + +```javascript +app.get('/dast-website-target', function(req, res) { + res.append('Gitlab-On-Demand-DAST', '0dd79c9a-7b29-4e26-a815-eaaf53fcab1c') + res.send('Respond to DAST ping') +}) +``` ## Scanner profile @@ -782,40 +909,6 @@ To delete a scanner profile: 1. Click **Manage** in the **DAST Profiles** row. 1. Click **{remove}** (Delete profile) in the scanner profile's row. -## On-demand scans - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3. - -An on-demand DAST scan runs outside the DevOps life cycle. Changes in your repository don't trigger -the scan. You must start it manually. - -An on-demand DAST scan: - -- Uses settings in the site profile and scanner profile you select when you run the scan, - instead of those in the `.gitlab-ci.yml` file. -- Is associated with your project's default branch. - -### Run an on-demand DAST scan - -NOTE: -You must have permission to run an on-demand DAST scan against a protected branch. -The default branch is automatically protected. For more information, see -[Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches). - -To run an on-demand DAST scan, you need: - -- A [scanner profile](#create-a-scanner-profile). -- A [site profile](#create-a-site-profile). - -1. From your project's home page, go to **Security & Compliance > On-demand Scans** in the left sidebar. -1. Click **Create new DAST scan**. -1. In **Scanner profile**, select a scanner profile from the dropdown. -1. In **Site profile**, select a site profile from the dropdown. -1. Click **Run scan**. - -The on-demand DAST scan runs and the project's dashboard shows the results. - ## Reports The DAST tool outputs a report file in JSON format by default. However, this tool can also generate reports in diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 7aafa52799d..00242057784 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -532,13 +532,10 @@ endpoints](../../user/admin_area/settings/rate_limits_on_raw_endpoints.md). ### Rate limiting responses -The [`Retry-After` -header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) -indicates when the client should retry. +For information on rate limiting responses, see: -Rate limits applied by HAProxy (instead of Cloudflare or the -GitLab application) have `RateLimit-Reset` and `RateLimit-ResetTime` -headers. +- [List of headers on responses to blocked requests](../admin_area/settings/user_and_ip_rate_limits.md#response-headers). +- [Customizable response text](../admin_area/settings/user_and_ip_rate_limits.md#response-text). ### Protected paths throttle @@ -548,11 +545,7 @@ paths that exceed 10 requests per **minute** per IP address. See the source below for which paths are protected. This includes user creation, user confirmation, user sign in, and password reset. -This header is included in responses to blocked requests: - -```plaintext -Retry-After: 60 -``` +[User and IP rate limits](../admin_area/settings/user_and_ip_rate_limits.md#response-headers) includes a list of the headers responded to blocked requests. See [Protected Paths](../admin_area/settings/protected_paths.md) for more details. diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index b6d47b56bce..092c12938e7 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -21,6 +21,11 @@ If you do not have a Composer package, create one and check it in to a repository. This example shows a GitLab repository, but the repository can be any public or private repository. +WARNING: +If you are using a GitLab repository, the project must have been created from +a group's namespace, rather than a user's namespace. Composer packages +[can't be published to projects created from a user's namespace](https://gitlab.com/gitlab-org/gitlab/-/issues/235467). + 1. Create a directory called `my-composer-package` and change to that directory: ```shell diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 38f829c94e8..c16fea1d00a 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -94,7 +94,10 @@ Some features such as [publishing](#publish-an-npm-package) a package is only av ## Authenticate to the Package Registry -To authenticate to the Package Registry, you must use one of the following: +You must authenticate with the Package Registry when the project +is private. Public projects do not require authentication. + +To authenticate, use one of the following: - A [personal access token](../../../user/profile/personal_access_tokens.md) (required for two-factor authentication (2FA)), with the scope set to `api`. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 6cf7e1f30a3..309831b7fb0 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -1347,7 +1347,14 @@ Note that `deployable_id` is the ID of the CI job. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/260347) in GitLab 13.7. -Triggered when a user is added as a group member. +Member events are triggered when: + +- A user is added as a group member +- The access level of a user has changed +- The expiration date for user access has been updated +- A user has been removed from the group + +#### Add member to group **Request Header**: @@ -1375,6 +1382,62 @@ X-Gitlab-Event: Member Hook } ``` +#### Update member access level or expiration date + +**Request Header**: + +```plaintext +X-Gitlab-Event: Member Hook +``` + +**Request Body**: + +```json +{ + "created_at": "2020-12-11T04:57:22Z", + "updated_at": "2020-12-12T08:48:19Z", + "group_name": "webhook-test", + "group_path": "webhook-test", + "group_id": 100, + "user_username": "test_user", + "user_name": "Test User", + "user_email": "testuser@webhooktest.com", + "user_id": 64, + "group_access": "Developer", + "group_plan": null, + "expires_at": "2020-12-20T00:00:00Z", + "event_name": "user_update_for_group" +} +``` + +#### Remove member from group + +**Request Header**: + +```plaintext +X-Gitlab-Event: Member Hook +``` + +**Request Body**: + +```json +{ + "created_at": "2020-12-11T04:57:22Z", + "updated_at": "2020-12-12T08:52:34Z", + "group_name": "webhook-test", + "group_path": "webhook-test", + "group_id": 100, + "user_username": "test_user", + "user_name": "Test User", + "user_email": "testuser@webhooktest.com", + "user_id": 64, + "group_access": "Guest", + "group_plan": null, + "expires_at": "2020-12-14T00:00:00Z", + "event_name": "user_remove_from_group" +} +``` + ### Feature Flag events Triggered when a feature flag is turned on or off. diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md index fd7c58f12b9..4910751ece1 100644 --- a/doc/user/project/new_ci_build_permissions_model.md +++ b/doc/user/project/new_ci_build_permissions_model.md @@ -72,10 +72,26 @@ Let's consider the following scenario: ## Job token -A unique job token is generated for each job and provides the user read -access all projects that would be normally accessible to the user creating that -job. The unique job token does not have any write permissions, but there -is a [proposal to add support](https://gitlab.com/groups/gitlab-org/-/epics/3559). +When a pipeline job is about to run, GitLab generates a unique token and injects it as the +[`CI_JOB_TOKEN` predefined variable](../../ci/variables/predefined_variables.md). +This token can authenticate [API requests](../../api/README.md) +from the job script (Runner) that needs to access the project's resources (for example, when +fetching a job artifact). + +Once the token is authenticated, GitLab identifies the user who triggered the job and uses this user +to authorize access to the resource. Therefore, this user must be assigned to +[a role that has the required privileges](../permissions.md). + +The job token has these limitations: + +- Not all APIs allow job tokens for authentication. See [this list](../../api/README.md#gitlab-ci-job-token) + for available endpoints. +- The token is valid only while the pipeline job runs. Once the job finishes, the token can't be + used for authentication. + +Although a job token is handy to quickly access a project's resources without any configuration, it +sometimes gives extra permissions that aren't necessary. There is [a proposal](https://gitlab.com/groups/gitlab-org/-/epics/3559) +to redesign the feature for more strategic control of the access permissions. If you need your CI pipeline to push to the Package Registry, consider using [deploy tokens](deploy_tokens/index.md). |