diff options
Diffstat (limited to 'doc/administration/postgresql')
-rw-r--r-- | doc/administration/postgresql/database_load_balancing.md | 4 | ||||
-rw-r--r-- | doc/administration/postgresql/external.md | 2 | ||||
-rw-r--r-- | doc/administration/postgresql/index.md | 18 | ||||
-rw-r--r-- | doc/administration/postgresql/multiple_databases.md | 8 | ||||
-rw-r--r-- | doc/administration/postgresql/pgbouncer.md | 18 | ||||
-rw-r--r-- | doc/administration/postgresql/replication_and_failover.md | 43 | ||||
-rw-r--r-- | doc/administration/postgresql/standalone.md | 10 |
7 files changed, 52 insertions, 51 deletions
diff --git a/doc/administration/postgresql/database_load_balancing.md b/doc/administration/postgresql/database_load_balancing.md index cc550dbe389..f8b6be1fb21 100644 --- a/doc/administration/postgresql/database_load_balancing.md +++ b/doc/administration/postgresql/database_load_balancing.md @@ -64,7 +64,7 @@ To enable Database Load Balancing, make sure that: - The HA PostgreSQL setup has one or more secondary nodes replicating the primary. - Each PostgreSQL node is connected with the same credentials and on the same port. -For Omnibus GitLab, you also need PgBouncer configured on each PostgreSQL node to pool +For Linux package installations, you also need PgBouncer configured on each PostgreSQL node to pool all load-balanced connections when [configuring a multi-node setup](replication_and_failover.md). ## Configuring Database Load Balancing @@ -112,7 +112,7 @@ checks a DNS `A` record, using the IPs returned by this record as the addresses for the secondaries. For service discovery to work, all you need is a DNS server and an `A` record containing the IP addresses of your secondaries. -When using Omnibus GitLab the provided [Consul](../consul.md) service works as +When using a Linux package installation, the provided [Consul](../consul.md) service works as a DNS server and returns PostgreSQL addresses via the `postgresql-ha.service.consul` record. For example: diff --git a/doc/administration/postgresql/external.md b/doc/administration/postgresql/external.md index 8f664f9809e..9c44d53213b 100644 --- a/doc/administration/postgresql/external.md +++ b/doc/administration/postgresql/external.md @@ -11,7 +11,7 @@ managed service for PostgreSQL. For example, AWS offers a managed Relational Database Service (RDS) that runs PostgreSQL. Alternatively, you may opt to manage your own PostgreSQL instance or cluster -separate from the Omnibus GitLab package. +separate from the Linux package. If you use a cloud-managed service, or provide your own PostgreSQL instance: diff --git a/doc/administration/postgresql/index.md b/doc/administration/postgresql/index.md index e829397abc1..af0a86c3d72 100644 --- a/doc/administration/postgresql/index.md +++ b/doc/administration/postgresql/index.md @@ -13,32 +13,32 @@ be used with GitLab in one of our [reference architectures](../reference_archite Choose one of the following PostgreSQL configuration options: -### Standalone PostgreSQL using Omnibus GitLab +### Standalone PostgreSQL for Linux package installations -This setup is for when you have installed the -[Omnibus GitLab packages](https://about.gitlab.com/install/) (CE or EE), +This setup is for when you have installed GitLab by using the +[Linux package](https://about.gitlab.com/install/) (CE or EE), to use the bundled PostgreSQL having only its service enabled. -Read how to [set up a standalone PostgreSQL instance](standalone.md) using Omnibus GitLab. +Read how to [set up a standalone PostgreSQL instance](standalone.md) for Linux package installations. ### Provide your own PostgreSQL instance This setup is for when you have installed GitLab using the -[Omnibus GitLab packages](https://about.gitlab.com/install/) (CE or EE), -or installed it [from source](../../install/installation.md), but you want to use +[Linux package](https://about.gitlab.com/install/) (CE or EE), +or [self-compiled](../../install/installation.md) your installation, but you want to use your own external PostgreSQL server. Read how to [set up an external PostgreSQL instance](external.md). -### PostgreSQL replication and failover with Omnibus GitLab **(PREMIUM SELF)** +### PostgreSQL replication and failover for Linux package installations **(PREMIUM SELF)** This setup is for when you have installed GitLab using the -[Omnibus GitLab **Enterprise Edition** (EE) package](https://about.gitlab.com/install/?version=ee). +[Linux **Enterprise Edition** (EE) package](https://about.gitlab.com/install/?version=ee). All the tools that are needed like PostgreSQL, PgBouncer, and Patroni are bundled in the package, so you can use it to set up the whole PostgreSQL infrastructure (primary, replica). -Read how to [set up PostgreSQL replication and failover](replication_and_failover.md) using Omnibus GitLab. +Read how to [set up PostgreSQL replication and failover](replication_and_failover.md) for Linux package installations. ## Related topics diff --git a/doc/administration/postgresql/multiple_databases.md b/doc/administration/postgresql/multiple_databases.md index f03781d0ee2..5dcb080d707 100644 --- a/doc/administration/postgresql/multiple_databases.md +++ b/doc/administration/postgresql/multiple_databases.md @@ -65,7 +65,7 @@ copy the database across. 1. Configure GitLab to [use multiple databases](#set-up-multiple-databases). -### Existing Omnibus installation +### Existing Linux package installations 1. Stop GitLab, except for PostgreSQL: @@ -103,7 +103,7 @@ the other way around. 1. For existing installations, [migrate the data](#migrate-existing-installations) first. -1. [Back up GitLab](../../raketasks/backup_restore.md) +1. [Back up GitLab](../../administration/backup_restore/index.md) in case of unforeseen issues. 1. Stop GitLab: @@ -152,12 +152,12 @@ the other way around. sudo service gitlab restart ``` -### Omnibus GitLab installations +### Linux package installations 1. For existing installations, [migrate the data](#migrate-existing-installations) first. -1. [Back up GitLab](../../raketasks/backup_restore.md) +1. [Back up GitLab](../../administration/backup_restore/index.md) in case of unforeseen issues. 1. Stop GitLab: diff --git a/doc/administration/postgresql/pgbouncer.md b/doc/administration/postgresql/pgbouncer.md index 59aac9141a4..7e46933113b 100644 --- a/doc/administration/postgresql/pgbouncer.md +++ b/doc/administration/postgresql/pgbouncer.md @@ -25,9 +25,9 @@ This content has been moved to a [new location](replication_and_failover.md#conf ## PgBouncer as part of a non-fault-tolerant GitLab installation -1. Generate PGBOUNCER_USER_PASSWORD_HASH with the command `gitlab-ctl pg-password-md5 pgbouncer` +1. Generate `PGBOUNCER_USER_PASSWORD_HASH` with the command `gitlab-ctl pg-password-md5 pgbouncer` -1. Generate SQL_USER_PASSWORD_HASH with the command `gitlab-ctl pg-password-md5 gitlab`. Enter the plaintext SQL_USER_PASSWORD later. +1. Generate `SQL_USER_PASSWORD_HASH` with the command `gitlab-ctl pg-password-md5 gitlab`. Enter the plaintext SQL_USER_PASSWORD later. 1. On your database node, ensure the following is set in your `/etc/gitlab/gitlab.rb` @@ -88,7 +88,7 @@ This content has been moved to a [new location](replication_and_failover.md#conf Do not backup or restore GitLab through a PgBouncer connection: it causes a GitLab outage. -[Read more about this and how to reconfigure backups](../../raketasks/backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer). +[Read more about this and how to reconfigure backups](../../administration/backup_restore/backup_gitlab.md#back-up-and-restore-for-installations-using-pgbouncer). ## Enable Monitoring @@ -119,7 +119,7 @@ If you enable Monitoring, it must be enabled on **all** PgBouncer servers. ## Administrative console -As part of Omnibus GitLab, a command is provided to automatically connect to the +In Linux package installations, a command is provided to automatically connect to the PgBouncer administrative console. See the [PgBouncer documentation](https://www.pgbouncer.org/usage.html#admin-console) for detailed instructions on how to interact with the console. @@ -174,11 +174,11 @@ ote_pid | tls ## Procedure for bypassing PgBouncer -### Omnibus installations +### Linux package installations Some database changes have to be done directly, and not through PgBouncer. -The main affected tasks are [database restores](../../raketasks/backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer) +The main affected tasks are [database restores](../../administration/backup_restore/backup_gitlab.md#back-up-and-restore-for-installations-using-pgbouncer) and [GitLab upgrades with database migrations](../../update/zero_downtime.md#postgresql). 1. To find the primary node, run the following on a database node: @@ -208,8 +208,8 @@ After you've performed the tasks or procedure, switch back to using PgBouncer: ### Helm chart installations -High-availability deployments also need to bypass PgBouncer for the same reasons as Omnibus-based ones. -For this type of installation: +High-availability deployments also need to bypass PgBouncer for the same reasons as Linux package-based ones. +For Helm chart installations: - Database backup and restore tasks are performed by the toolbox container. - Migration tasks are performed by the migrations container. @@ -226,7 +226,7 @@ In specific cases you may want to change the performance-specific and resource-s throughput or to limit resource utilization that could cause memory exhaustion on the database. You can find the parameters and respective documentation on the [official PgBouncer documentation](https://www.pgbouncer.org/config.html). -Listed below are the most relevant ones and their defaults on an Omnibus GitLab installation: +Listed below are the most relevant ones and their defaults on a Linux package installation: - `pgbouncer['max_client_conn']` (default: `2048`, depends on server file descriptor limits) This is the "frontend" pool in PgBouncer: connections from Rails to PgBouncer. diff --git a/doc/administration/postgresql/replication_and_failover.md b/doc/administration/postgresql/replication_and_failover.md index 392f9f2b89c..05ff6fa8a4a 100644 --- a/doc/administration/postgresql/replication_and_failover.md +++ b/doc/administration/postgresql/replication_and_failover.md @@ -4,21 +4,21 @@ group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# PostgreSQL replication and failover with Omnibus GitLab **(PREMIUM SELF)** +# PostgreSQL replication and failover for Linux package installations **(PREMIUM SELF)** If you're a Free user of GitLab self-managed, consider using a cloud-hosted solution. This document doesn't cover installations from source. If a setup with replication and failover isn't what you were looking for, see the [database configuration document](https://docs.gitlab.com/omnibus/settings/database.html) -for the Omnibus GitLab packages. +for the Linux packages. It's recommended to read this document fully before attempting to configure PostgreSQL with replication and failover for GitLab. ## Architecture -The Omnibus GitLab recommended configuration for a PostgreSQL cluster with +The Linux pacakage-recommended configuration for a PostgreSQL cluster with replication failover requires: - A minimum of three PostgreSQL nodes. @@ -73,9 +73,9 @@ sure you have redundant connectivity between all Database and GitLab instances to avoid the network becoming a single point of failure. NOTE: -As of GitLab 13.3, PostgreSQL 12 is shipped with Omnibus GitLab. Clustering for PostgreSQL 12 is supported only with +As of GitLab 13.3, PostgreSQL 12 is shipped with Linux package installations. Clustering for PostgreSQL 12 is supported only with Patroni. See the [Patroni](#patroni) section for further details. Starting with GitLab 14.0, only PostgreSQL 12 is -shipped with Omnibus GitLab, and thus Patroni becomes mandatory for replication and failover. +shipped with Linux package installations, and thus Patroni becomes mandatory for replication and failover. ### Database node @@ -152,7 +152,7 @@ This is why you need: When using default setup, minimum configuration requires: -- `CONSUL_USERNAME`. The default user for Omnibus GitLab is `gitlab-consul` +- `CONSUL_USERNAME`. The default user for Linux package installations is `gitlab-consul` - `CONSUL_DATABASE_PASSWORD`. Password for the database user. - `CONSUL_PASSWORD_HASH`. This is a hash generated out of Consul username/password pair. It can be generated with: @@ -192,7 +192,7 @@ server nodes on hand. You need the following password information for the application's database user: -- `POSTGRESQL_USERNAME`. The default user for Omnibus GitLab is `gitlab` +- `POSTGRESQL_USERNAME`. The default user for Linux package installations is `gitlab` - `POSTGRESQL_USER_PASSWORD`. The password for the database user - `POSTGRESQL_PASSWORD_HASH`. This is a hash generated out of the username/password pair. It can be generated with: @@ -212,7 +212,7 @@ You need the following password information for the Patroni API: When using a default setup, the minimum configuration requires: -- `PGBOUNCER_USERNAME`. The default user for Omnibus GitLab is `pgbouncer` +- `PGBOUNCER_USERNAME`. The default user for Linux package installations is `pgbouncer` - `PGBOUNCER_PASSWORD`. This is a password for PgBouncer service. - `PGBOUNCER_PASSWORD_HASH`. This is a hash generated out of PgBouncer username/password pair. It can be generated with: @@ -230,10 +230,9 @@ Few things to remember about the service itself: - `/etc/gitlab/gitlab.rb`: hashed, and in plain text - `/var/opt/gitlab/pgbouncer/pg_auth`: hashed -### Installing Omnibus GitLab +### Installing the Linux package -First, make sure to [download/install](https://about.gitlab.com/install/) -Omnibus GitLab **on each node**. +First, make sure to [download and install](https://about.gitlab.com/install/) the Linux package **on each node**. Make sure you install the necessary dependencies from step 1, add GitLab package repository from step 2. @@ -373,7 +372,7 @@ patroni['tls_key_password'] = 'private-key-password' # This is the plain-text pa ``` If you are using a self-signed certificate or an internal CA, you need to either disable the TLS verification or pass the certificate of the -internal CA, otherwise you may run into an unexpected error when using the `gitlab-ctl patroni ....` commands. Omnibus ensures that Patroni API +internal CA, otherwise you may run into an unexpected error when using the `gitlab-ctl patroni ....` commands. The Linux package ensures that Patroni API clients honor this configuration. TLS certificate verification is enabled by default. To disable it: @@ -415,7 +414,7 @@ authentication mode (`patroni['tls_client_mode']`), must each have the same valu 1. Make sure you collect [`CONSUL_SERVER_NODES`](#consul-information), [`CONSUL_PASSWORD_HASH`](#consul-information), and [`PGBOUNCER_PASSWORD_HASH`](#pgbouncer-information) before executing the next step. -1. One each node, edit the `/etc/gitlab/gitlab.rb` configuration file and replace values noted in the `# START user configuration` section as below: +1. On each node, edit the `/etc/gitlab/gitlab.rb` configuration file and replace values noted in the `# START user configuration` section as below: ```ruby # Disable all components except PgBouncer and Consul agent @@ -571,7 +570,7 @@ in the Troubleshooting section before proceeding. Do not backup or restore GitLab through a PgBouncer connection: this causes a GitLab outage. -[Read more about this and how to reconfigure backups](../../raketasks/backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer). +[Read more about this and how to reconfigure backups](../../administration/backup_restore/backup_gitlab.md#back-up-and-restore-for-installations-using-pgbouncer). ### Ensure GitLab is running @@ -837,7 +836,7 @@ Read more about the data returned by the replica ### Selecting the appropriate Patroni replication method -[Review the Patroni documentation carefully](https://patroni.readthedocs.io/en/latest/SETTINGS.html#postgresql) +[Review the Patroni documentation carefully](https://patroni.readthedocs.io/en/latest/yaml_configuration.html#postgresql) before making changes as **_some of the options carry a risk of potential data loss if not fully understood_**. The [replication mode](https://patroni.readthedocs.io/en/latest/replication_modes.html) configured determines the amount of tolerable data loss. @@ -845,7 +844,7 @@ configured determines the amount of tolerable data loss. WARNING: Replication is not a backup strategy! There is no replacement for a well-considered and tested backup solution. -Omnibus GitLab defaults [`synchronous_commit`](https://www.postgresql.org/docs/11/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT) to `on`. +Linux package installations default [`synchronous_commit`](https://www.postgresql.org/docs/11/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT) to `on`. ```ruby postgresql['synchronous_commit'] = 'on' @@ -854,7 +853,7 @@ gitlab['geo-postgresql']['synchronous_commit'] = 'on' #### Customizing Patroni failover behavior -Omnibus GitLab exposes several options allowing more control over the [Patroni restoration process](#recovering-the-patroni-cluster). +Linux package installations expose several options allowing more control over the [Patroni restoration process](#recovering-the-patroni-cluster). Each option is shown below with its default value in `/etc/gitlab/gitlab.rb`. @@ -864,7 +863,7 @@ patroni['remove_data_directory_on_rewind_failure'] = false patroni['remove_data_directory_on_diverged_timelines'] = false ``` -[The upstream documentation is always more up to date](https://patroni.readthedocs.io/en/latest/SETTINGS.html#postgresql), but the table below should provide a minimal overview of functionality. +[The upstream documentation is always more up to date](https://patroni.readthedocs.io/en/latest/patroni_configuration.html), but the table below should provide a minimal overview of functionality. |Setting|Overview| |-|-| @@ -996,7 +995,9 @@ You can switch an exiting database cluster to use Patroni instead of repmgr with ### Upgrading PostgreSQL major version in a Patroni cluster -As of GitLab 14.1, PostgreSQL 12.6 and 13.3 are both shipped with Omnibus GitLab by default. As of GitLab 15.0, PostgreSQL 13 is the default. If you want to upgrade to PostgreSQL 13 in versions prior to GitLab 15.0, you must ask for it explicitly. +As of GitLab 14.1, PostgreSQL 12.6 and 13.3 are both shipped with the Linux package by default. As of GitLab 15.0, +PostgreSQL 13 is the default. If you want to upgrade to PostgreSQL 13 in versions prior to GitLab 15.0, you must ask for +it explicitly. WARNING: The procedure for upgrading PostgreSQL in a Patroni cluster is different than when upgrading using repmgr. @@ -1466,9 +1467,9 @@ Workarounds: - If set to enforcing, SELinux may also prevent these operations. Verify the issue is fixed by setting SELinux to permissive. -Patroni first shipped in Omnibus GitLab 13.1, along with a build of Python 3.7. +Patroni first shipped in the Linux package for GitLab 13.1, along with a build of Python 3.7. The code which causes this was removed in Python 3.8: this fix shipped in -[Omnibus GitLab 14.3](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5547) +[the Linux package for GitLab 14.3](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5547) and later, removing the need for a workaround. ### Errors running `gitlab-ctl` diff --git a/doc/administration/postgresql/standalone.md b/doc/administration/postgresql/standalone.md index d00310ecee0..d6f3460e255 100644 --- a/doc/administration/postgresql/standalone.md +++ b/doc/administration/postgresql/standalone.md @@ -4,19 +4,19 @@ group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Standalone PostgreSQL using Omnibus GitLab **(FREE SELF)** +# Standalone PostgreSQL for Linux package installations **(FREE SELF)** If you wish to have your database service hosted separately from your GitLab application servers, you can do this using the PostgreSQL binaries packaged -together with Omnibus GitLab. This is recommended as part of our +together with the Linux package. This is recommended as part of our [reference architecture for up to 2,000 users](../reference_architectures/2k_users.md). ## Setting it up 1. SSH in to the PostgreSQL server. -1. [Download and install](https://about.gitlab.com/install/) the Omnibus GitLab - package you want using *steps 1 and 2* from the GitLab downloads page. - - Do not complete any other steps on the download page. +1. [Download and install](https://about.gitlab.com/install/) the Linux + package you want using *steps 1 and 2* from the GitLab downloads page. Do not complete any other steps on the + download page. 1. Generate a password hash for PostgreSQL. This assumes you are using the default username of `gitlab` (recommended). The command requests a password and confirmation. Use the value that is output by this command in the next |