diff options
Diffstat (limited to 'doc/administration/geo')
29 files changed, 190 insertions, 171 deletions
diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md index d7db48bb6cf..57f8664ba11 100644 --- a/doc/administration/geo/disaster_recovery/background_verification.md +++ b/doc/administration/geo/disaster_recovery/background_verification.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/geo/disaster_recovery/bring_primary_back.md b/doc/administration/geo/disaster_recovery/bring_primary_back.md index 4e68cc39580..833b9a877e9 100644 --- a/doc/administration/geo/disaster_recovery/bring_primary_back.md +++ b/doc/administration/geo/disaster_recovery/bring_primary_back.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -18,8 +18,8 @@ If you have any doubts about the consistency of the data on this site, we recomm ## Configure the former **primary** site to be a **secondary** site -Since the former **primary** site will be out of sync with the current **primary** site, the first step is to bring the former **primary** site up to date. Note, deletion of data stored on disk like -repositories and uploads will not be replayed when bringing the former **primary** site back +Since the former **primary** site is out of sync with the current **primary** site, the first step is to bring the former **primary** site up to date. Note, deletion of data stored on disk like +repositories and uploads is not replayed when bringing the former **primary** site back into sync, which may result in increased disk usage. Alternatively, you can [set up a new **secondary** GitLab instance](../setup/index.md) to avoid this. @@ -34,8 +34,8 @@ To bring the former **primary** site up to date: NOTE: If you [disabled the **primary** site permanently](index.md#step-2-permanently-disable-the-primary-site), - you need to undo those steps now. For Debian/Ubuntu you just need to run - `sudo systemctl enable gitlab-runsvdir`. For CentOS 6, you need to install + you need to undo those steps now. For distributions with systemd, such as Debian/Ubuntu/CentOS7+, you must run + `sudo systemctl enable gitlab-runsvdir`. For distributions without systemd, such as CentOS 6, you need to install the GitLab instance from scratch and set it up as a **secondary** site by following [Setup instructions](../setup/index.md). In this case, you don't need to follow the next step. diff --git a/doc/administration/geo/disaster_recovery/index.md b/doc/administration/geo/disaster_recovery/index.md index baece830318..e5b9a14cfbd 100644 --- a/doc/administration/geo/disaster_recovery/index.md +++ b/doc/administration/geo/disaster_recovery/index.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -335,16 +335,14 @@ a **secondary** with only a single node. Instead, you must do this manually. #### Promoting a **secondary** site with an external PostgreSQL database running GitLab 14.5 and later -The `gitlab-ctl geo promote` command can be used in conjunction with -an external PostgreSQL database, but it can only perform changes on -a **secondary** PostgreSQL database managed by Omnibus. -You must promote the replica database associated with the **secondary** -site first. +The `gitlab-ctl geo promote` command can be used in conjunction with an external PostgreSQL database. +In this case, you must first manually promote the replica database associated +with the **secondary** site: 1. Promote the replica database associated with the **secondary** site. This sets the database to read-write. The instructions vary depending on where your database is hosted: - [Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html#USER_ReadRepl.Promote) - - [Azure PostgreSQL](https://docs.microsoft.com/en-us/azure/postgresql/howto-read-replicas-portal#stop-replication) + - [Azure PostgreSQL](https://docs.microsoft.com/en-us/azure/postgresql/single-server/how-to-read-replicas-portal#stop-replication) - [Google Cloud SQL](https://cloud.google.com/sql/docs/mysql/replication/manage-replicas#promote-replica) - For other external PostgreSQL databases, save the following script in your secondary site, for example `/tmp/geo_promote.sh`, and modify the connection diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md index 57bad6177d9..7d8dd7d5d2a 100644 --- a/doc/administration/geo/disaster_recovery/planned_failover.md +++ b/doc/administration/geo/disaster_recovery/planned_failover.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -143,7 +143,7 @@ If the **primary** site uses custom or self-signed TLS certificates to secure in ### Ensure Geo replication is up-to-date -The maintenance window won't end until Geo replication and verification is +The maintenance window does not end until Geo replication and verification is completely finished. To keep the window as short as possible, you should ensure these processes are close to 100% as possible during active use. @@ -201,7 +201,7 @@ be disabled on the **primary** site: ## Finish replicating and verifying all data NOTE: -GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary site statuses will appear to stop updating and become unhealthy. For more information, see [Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](../replication/troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). +GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary site statuses appears to stop updating and become unhealthy. For more information, see [Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](../replication/troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). 1. If you are manually replicating any data not managed by Geo, trigger the final replication process now. diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md index 59134df8e7f..46b2ccddefd 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md index 25f6999ce43..da26023e4f9 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto diff --git a/doc/administration/geo/glossary.md b/doc/administration/geo/glossary.md index c6b3f26dc67..c9d4ccacae3 100644 --- a/doc/administration/geo/glossary.md +++ b/doc/administration/geo/glossary.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto diff --git a/doc/administration/geo/index.md b/doc/administration/geo/index.md index dc25be825ea..c5472e412d3 100644 --- a/doc/administration/geo/index.md +++ b/doc/administration/geo/index.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -13,7 +13,7 @@ a warm-standby as part of a disaster recovery strategy. WARNING: Geo undergoes significant changes from release to release. Upgrades are -supported and [documented](#updating-geo), but you should ensure that you're +supported and [documented](#upgrading-geo), but you should ensure that you're using the right version of the documentation for your installation. Fetching large repositories can take a long time for teams located far from a single GitLab instance. @@ -53,8 +53,7 @@ Geo provides: ### Gitaly Cluster Geo should not be confused with [Gitaly Cluster](../gitaly/praefect.md). For more information about -the difference between Geo and Gitaly Cluster, see -[How does Gitaly Cluster compare to Geo?](../gitaly/faq.md#how-does-gitaly-cluster-compare-to-geo). +the difference between Geo and Gitaly Cluster, see [Comparison to Geo](../gitaly/index.md#comparison-to-geo). ## How it works @@ -234,9 +233,9 @@ After installing GitLab on the **secondary** sites and performing the initial co For information on configuring Geo, see [Geo configuration](replication/configuration.md). -### Updating Geo +### Upgrading Geo -For information on how to update your Geo sites to the latest GitLab version, see [Updating the Geo sites](replication/updating_the_geo_sites.md). +For information on how to update your Geo sites to the latest GitLab version, see [Upgrading the Geo sites](replication/upgrading_the_geo_sites.md). ### Pausing and resuming replication @@ -252,7 +251,7 @@ WARNING: Pausing and resuming of replication is only supported for Geo installations using an Omnibus GitLab-managed database. External databases are not supported. -In some circumstances, like during [upgrades](replication/updating_the_geo_sites.md) or a [planned failover](disaster_recovery/planned_failover.md), it is desirable to pause replication between the primary and secondary. +In some circumstances, like during [upgrades](replication/upgrading_the_geo_sites.md) or a [planned failover](disaster_recovery/planned_failover.md), it is desirable to pause replication between the primary and secondary. Pausing and resuming replication is done via a command line tool from the node in the secondary site where the `postgresql` service is enabled. diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index adf89c24a4e..043b96478c9 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -384,7 +384,7 @@ when: ## Upgrading Geo -See the [updating the Geo sites document](updating_the_geo_sites.md). +See the [upgrading the Geo sites document](upgrading_the_geo_sites.md). ## Troubleshooting diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md index 83e207e7a8f..8a5aa935d11 100644 --- a/doc/administration/geo/replication/datatypes.md +++ b/doc/administration/geo/replication/datatypes.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -26,7 +26,7 @@ verification methods: | Type | Feature / component | Replication method | Verification method | |:---------|:------------------------------------------------|:--------------------------------------|:-----------------------| | Database | Application data in PostgreSQL | Native | Native | -| Database | Redis | _N/A_ (*1*) | _N/A_ | +| Database | Redis | Not applicable (*1*) | Not applicable | | Database | Elasticsearch | Native | Native | | Database | SSH public keys | PostgreSQL Replication | PostgreSQL Replication | | Git | Project repository | Geo with Gitaly | Gitaly Checksum | @@ -203,8 +203,10 @@ successfully, you must replicate their data using some other means. |[External merge request diffs](../../merge_request_diffs.md) | **Yes** (13.5) | **Yes** (14.6) | Via Object Storage provider if supported. Native Geo support (Beta). | Replication is behind the feature flag `geo_merge_request_diff_replication`, enabled by default. Verification was behind the feature flag `geo_merge_request_diff_verification`, removed in 14.7.| |[Versioned snippets](../../../user/snippets.md#versioned-snippets) | [**Yes** (13.7)](https://gitlab.com/groups/gitlab-org/-/epics/2809) | [**Yes** (14.2)](https://gitlab.com/groups/gitlab-org/-/epics/2810) | No | Verification was implemented behind the feature flag `geo_snippet_repository_verification` in 13.11, and the feature flag was removed in 14.2. | |[GitLab Pages](../../pages/index.md) | [**Yes** (14.3)](https://gitlab.com/groups/gitlab-org/-/epics/589) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (14.6) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_pages_deployment_replication`, enabled by default. Verification was behind the feature flag `geo_pages_deployment_verification`, removed in 14.7. | +|[Incident Metric Images](../../../operations/incident_management/incidents.md#metrics) | [Planned](https://gitlab.com/gitlab-org/gitlab/-/issues/352326) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/362561) | No | | +|[Alert Metric Images](../../../operations/incident_management/alerts.md#metrics-tab) | [Planned](https://gitlab.com/gitlab-org/gitlab/-/issues/352326) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/362561) | No | | |[Server-side Git hooks](../../server_hooks.md) | [Not planned](https://gitlab.com/groups/gitlab-org/-/epics/1867) | No | No | Not planned because of current implementation complexity, low customer interest, and availability of alternatives to hooks. | -|[Elasticsearch integration](../../../integration/elasticsearch.md) | [Not planned](https://gitlab.com/gitlab-org/gitlab/-/issues/1186) | No | No | Not planned because further product discovery is required and Elasticsearch (ES) clusters can be rebuilt. Secondaries use the same ES cluster as the primary. | +|[Elasticsearch integration](../../../integration/advanced_search/elasticsearch.md) | [Not planned](https://gitlab.com/gitlab-org/gitlab/-/issues/1186) | No | No | Not planned because further product discovery is required and Elasticsearch (ES) clusters can be rebuilt. Secondaries use the same ES cluster as the primary. | |[Dependency proxy images](../../../user/packages/dependency_proxy/index.md) | [Not planned](https://gitlab.com/gitlab-org/gitlab/-/issues/259694) | No | No | Blocked by [Geo: Secondary Mimicry](https://gitlab.com/groups/gitlab-org/-/epics/1528). Replication of this cache is not needed for disaster recovery purposes because it can be recreated from external sources. | |[Vulnerability Export](../../../user/application_security/vulnerability_report/#export-vulnerability-details) | [Not planned](https://gitlab.com/groups/gitlab-org/-/epics/3111) | No | No | Not planned because they are ephemeral and sensitive information. They can be regenerated on demand. | diff --git a/doc/administration/geo/replication/disable_geo.md b/doc/administration/geo/replication/disable_geo.md index 0fa469e57cd..5d4ae12c990 100644 --- a/doc/administration/geo/replication/disable_geo.md +++ b/doc/administration/geo/replication/disable_geo.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto diff --git a/doc/administration/geo/replication/docker_registry.md b/doc/administration/geo/replication/docker_registry.md index 4004ef3c17f..855e33d9a51 100644 --- a/doc/administration/geo/replication/docker_registry.md +++ b/doc/administration/geo/replication/docker_registry.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -114,10 +114,10 @@ For each application and Sidekiq node on the **secondary** site: ```ruby gitlab_rails['geo_registry_replication_enabled'] = true - + # Primary registry's hostname and port, it will be used by # the secondary node to directly communicate to primary registry - gitlab_rails['geo_registry_replication_primary_api_url'] = 'https://primary.example.com:5050/' + gitlab_rails['geo_registry_replication_primary_api_url'] = 'https://primary.example.com:5050/' ``` 1. Reconfigure the node for the change to take effect: diff --git a/doc/administration/geo/replication/faq.md b/doc/administration/geo/replication/faq.md index 12b3b382bf7..bdf1771e8a8 100644 --- a/doc/administration/geo/replication/faq.md +++ b/doc/administration/geo/replication/faq.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -32,10 +32,10 @@ To ensure that problems with pipelines (for example, syncs failing too many time the number of concurrent syncs falls below `repos_max_capacity` and there are no new projects waiting to be synced. Geo also has a checksum feature which runs a SHA256 sum across all the Git references to the SHA values. -If the refs don't match between the **primary** site and the **secondary** site, then the **secondary** site will mark that project as dirty and try to resync it. +If the refs don't match between the **primary** site and the **secondary** site, then the **secondary** site marks that project as dirty and try to resync it. So even if we have an outdated tracking database, the validation should activate and find discrepancies in the repository state and resync. -## Can I use Geo in a disaster recovery situation? +## Can you use Geo in a disaster recovery situation? Yes, but there are limitations to what we replicate (see [What data is replicated to a **secondary** site?](#what-data-is-replicated-to-a-secondary-site)). @@ -46,7 +46,7 @@ Read the documentation for [Disaster Recovery](../disaster_recovery/index.md). We currently replicate project repositories, LFS objects, generated attachments and avatars, and the whole database. This means user accounts, -issues, merge requests, groups, project data, and so on, will be available for +issues, merge requests, groups, project data, and so on, are available for query. For more details, see the [supported Geo data types](datatypes.md). @@ -69,6 +69,6 @@ That's totally fine. We use HTTP(s) to fetch repository changes from the **prima Yes. See [Docker Registry for a **secondary** site](docker_registry.md). -## Can I login to a secondary site? +## Can you login to a secondary site? Yes, but secondary sites receive all authentication data (like user accounts and logins) from the primary instance. This means you are re-directed to the primary for authentication and then routed back. diff --git a/doc/administration/geo/replication/geo_validation_tests.md b/doc/administration/geo/replication/geo_validation_tests.md index e4432c41ff1..6540366bf7f 100644 --- a/doc/administration/geo/replication/geo_validation_tests.md +++ b/doc/administration/geo/replication/geo_validation_tests.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/geo/replication/location_aware_git_url.md b/doc/administration/geo/replication/location_aware_git_url.md index 8689fac987f..18103211580 100644 --- a/doc/administration/geo/replication/location_aware_git_url.md +++ b/doc/administration/geo/replication/location_aware_git_url.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -57,34 +57,34 @@ routing configurations.  -1. Click the **Create traffic policy** button. +1. Select the **Create traffic policy** button.  -1. Fill in the **Policy Name** field with `Single Git Host` and click **Next**. +1. Fill in the **Policy Name** field with `Single Git Host` and select **Next**.  1. Leave **DNS type** as `A: IP Address in IPv4 format`. -1. Click **Connect to...** and select **Geolocation rule**. +1. Select **Connect to...** and select **Geolocation rule**.  1. For the first **Location**, leave it as `Default`. -1. Click **Connect to...** and select **New endpoint**. +1. Select **Connect to...** and select **New endpoint**. 1. Choose **Type** `value` and fill it in with `<your **primary** IP address>`. 1. For the second **Location**, choose `Europe`. -1. Click **Connect to...** and select **New endpoint**. +1. Select **Connect to...** and select **New endpoint**. 1. Choose **Type** `value` and fill it in with `<your **secondary** IP address>`.  -1. Click **Create traffic policy**. +1. Select **Create traffic policy**.  1. Fill in **Policy record DNS name** with `git`. -1. Click **Create policy records**. +1. Select **Create policy records**.  diff --git a/doc/administration/geo/replication/multiple_servers.md b/doc/administration/geo/replication/multiple_servers.md index 7b800817461..afa4f4eb552 100644 --- a/doc/administration/geo/replication/multiple_servers.md +++ b/doc/administration/geo/replication/multiple_servers.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto diff --git a/doc/administration/geo/replication/object_storage.md b/doc/administration/geo/replication/object_storage.md index 06f25fa15af..9ea226fd93b 100644 --- a/doc/administration/geo/replication/object_storage.md +++ b/doc/administration/geo/replication/object_storage.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto diff --git a/doc/administration/geo/replication/remove_geo_site.md b/doc/administration/geo/replication/remove_geo_site.md index b69dfe2e723..0d6715a93b7 100644 --- a/doc/administration/geo/replication/remove_geo_site.md +++ b/doc/administration/geo/replication/remove_geo_site.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -44,7 +44,7 @@ Once GitLab has been uninstalled from each node on the **secondary** site, the r ``` NOTE: - Using `gitlab-rails dbconsole` will not work, because managing replication slots requires superuser permissions. + Using `gitlab-rails dbconsole` does not work, because managing replication slots requires superuser permissions. 1. Find the name of the relevant replication slot. This is the slot that is specified with `--slot-name` when running the replicate command: `gitlab-ctl replicate-geo-database`. diff --git a/doc/administration/geo/replication/security_review.md b/doc/administration/geo/replication/security_review.md index df893298f85..bc08b7b71e7 100644 --- a/doc/administration/geo/replication/security_review.md +++ b/doc/administration/geo/replication/security_review.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 5a29c5a3c54..bb7dbef214b 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -398,7 +398,7 @@ where some queries never complete due to being canceled on every replication. These long-running queries are [planned to be removed in the future](https://gitlab.com/gitlab-org/gitlab/-/issues/34269), but as a workaround, we recommend enabling -[hot_standby_feedback](https://www.postgresql.org/docs/10/hot-standby.html#HOT-STANDBY-CONFLICT). +[`hot_standby_feedback`](https://www.postgresql.org/docs/10/hot-standby.html#HOT-STANDBY-CONFLICT). This increases the likelihood of bloat on the **primary** node as it prevents `VACUUM` from removing recently-dead rows. However, it has been used successfully in production on GitLab.com. @@ -767,7 +767,7 @@ The appropriate action sometimes depends on the cause. For example, you can remo In some cases, a file may be determined to be of low value, and so it may be worth deleting the record. -Geo itself is an excellent mitigation for files missing on the primary. If a file disappears on the primary but it was already synced to the secondary, you can grab the secondary's file. In cases like this, the `File is not checksummable` error message will not occur on Geo secondary sites, and only the primary will log this error message. +Geo itself is an excellent mitigation for files missing on the primary. If a file disappears on the primary but it was already synced to the secondary, you can grab the secondary's file. In cases like this, the `File is not checksummable` error message does not occur on Geo secondary sites, and only the primary logs this error message. This problem is more likely to show up in Geo secondary sites which were set up long after the original GitLab site. In this case, Geo is only surfacing an existing problem. @@ -815,6 +815,20 @@ Gitlab::Geo.verification_enabled_replicator_classes.each do |klass| end ``` +### Message: curl 18 transfer closed with outstanding read data remaining & fetch-pack: unexpected disconnect while reading sideband packet + +Unstable networking conditions can cause Gitaly to fail when trying to fetch large repository +data from the primary site. This is more likely to happen if a repository has to be +replicated from scratch between sites. + +Geo retries several times, but if the transmission is consistently interrupted +by network hiccups, an alternative method such as `rsync` can be used to circumvent `git` and +create the initial copy of any repository that fails to be replicated by Geo. + +We recommend transferring each failing repository individually and checking for consistency +after each transfer. Follow the [single target `rsync` instructions](../../operations/moving_repositories.md#single-rsync-to-another-server) +to transfer each affected repository from the primary to the secondary site. + ## Fixing errors during a failover or when promoting a secondary to a primary node The following are possible error messages that might be encountered during failover or @@ -1104,9 +1118,9 @@ If using a load balancer, ensure that the load balancer's URL is set as the `ext ### Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode -In GitLab 13.9 through GitLab 14.3, when [GitLab Maintenance Mode](../../maintenance_mode/index.md) is enabled, the status of Geo secondary sites will stop getting updated. After 10 minutes, the status changes to `Unhealthy`. +In GitLab 13.9 through GitLab 14.3, when [GitLab Maintenance Mode](../../maintenance_mode/index.md) is enabled, the status of Geo secondary sites stops getting updated. After 10 minutes, the status changes to `Unhealthy`. -Geo secondary sites will continue to replicate and verify data, and the secondary sites should still be usable. You can use the [Sync status Rake task](#sync-status-rake-task) to determine the actual status of a secondary site during Maintenance Mode. +Geo secondary sites continue to replicate and verify data, and the secondary sites should still be usable. You can use the [Sync status Rake task](#sync-status-rake-task) to determine the actual status of a secondary site during Maintenance Mode. This bug was [fixed in GitLab 14.4](https://gitlab.com/gitlab-org/gitlab/-/issues/292983). @@ -1129,7 +1143,7 @@ Geo::TrackingBase::SecondaryNotConfigured: Geo secondary database is not configu On a Geo primary site this error can be ignored. -This happens because GitLab is attempting to display registries from the [Geo tracking database](../../../administration/geo/#geo-tracking-database) which doesn't exist on the primary site (only the original projects exist on the primary; no replicated projects are present, therefore no tracking database exists). +This happens because GitLab is attempting to display registries from the [Geo tracking database](../../../administration/geo/#geo-tracking-database) which doesn't exist on the primary site (only the original projects exist on the primary; no replicated projects are present, therefore no tracking database exists). ## Fixing client errors @@ -1148,7 +1162,7 @@ The partial failover to a secondary Geo *site* may be the result of a temporary/ 1. SSH into every Sidekiq, PostgresSQL, Gitaly, and Rails node in the **secondary** site and run one of the following commands: - To promote the secondary node to primary: - + ```shell sudo gitlab-ctl geo promote ``` @@ -1158,8 +1172,8 @@ The partial failover to a secondary Geo *site* may be the result of a temporary/ ```shell sudo gitlab-ctl geo promote --force ``` - -1. Verify you can connect to the newly-promoted **primary** site using the URL used previously for the **secondary** site. + +1. Verify you can connect to the newly-promoted **primary** site using the URL used previously for the **secondary** site. 1. If **successful**, the **secondary** site is now promoted to the **primary** site. If the above steps are **not successful**, proceed through the next steps: diff --git a/doc/administration/geo/replication/tuning.md b/doc/administration/geo/replication/tuning.md index bcff6181296..670459624f3 100644 --- a/doc/administration/geo/replication/tuning.md +++ b/doc/administration/geo/replication/tuning.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -7,7 +7,7 @@ type: howto # Tuning Geo **(PREMIUM SELF)** -You can limit the number of concurrent operations the nodes can run +You can limit the number of concurrent operations the sites can run in the background. ## Changing the sync/verification concurrency values @@ -15,8 +15,8 @@ in the background. On the **primary** site: 1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Geo > Nodes**. -1. Select **Edit** of the secondary node you want to tune. +1. On the left sidebar, select **Geo > Sites**. +1. Select **Edit** of the secondary site you want to tune. 1. Under **Tuning settings**, there are several variables that can be tuned to improve the performance of Geo: @@ -25,7 +25,7 @@ On the **primary** site: - Container repositories synchronization concurrency limit - Verification concurrency limit -Increasing the concurrency values will increase the number of jobs that are scheduled. +Increasing the concurrency values increases the number of jobs that are scheduled. However, this may not lead to more downloads in parallel unless the number of available Sidekiq threads is also increased. For example, if repository synchronization concurrency is increased from 25 to 50, you may also want to increase the number diff --git a/doc/administration/geo/replication/updating_the_geo_sites.md b/doc/administration/geo/replication/upgrading_the_geo_sites.md index e82afe5f0d4..30961de0381 100644 --- a/doc/administration/geo/replication/updating_the_geo_sites.md +++ b/doc/administration/geo/replication/upgrading_the_geo_sites.md @@ -1,31 +1,31 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto --- -# Updating the Geo sites **(PREMIUM SELF)** +# Upgrading the Geo sites **(PREMIUM SELF)** WARNING: Read these sections carefully before updating your Geo sites. Not following -version-specific update steps may result in unexpected downtime. If you have +version-specific upgrade steps may result in unexpected downtime. If you have any specific questions, [contact Support](https://about.gitlab.com/support/#contact-support). -Updating Geo sites involves performing: +Upgrading Geo sites involves performing: -1. [Version-specific update steps](version_specific_updates.md), depending on the - version being updated to or from. -1. [General update steps](#general-update-steps), for all updates. +1. [Version-specific upgrade steps](version_specific_upgrades.md), depending on the + version being upgraded to or from. +1. [General upgrade steps](#general-upgrade-steps), for all upgrades. -## General update steps +## General upgrade steps NOTE: -These general update steps are not intended for multi-site deployments, -and will cause downtime. If you want to avoid downtime, consider using +These general upgrade steps are not intended for multi-site deployments, +and cause downtime. If you want to avoid downtime, consider using [zero downtime upgrades](../../../update/zero_downtime.md#multi-node--ha-deployment-with-geo). -To update the Geo sites when a new GitLab version is released, update **primary** +To upgrade the Geo sites when a new GitLab version is released, upgrade **primary** and all **secondary** sites: 1. **Optional:** [Pause replication on each **secondary** sites.](../index.md#pausing-and-resuming-replication) @@ -34,11 +34,11 @@ and all **secondary** sites: 1. SSH into each node of **secondary** sites. 1. [Upgrade GitLab on each **secondary** site](../../../update/package/index.md#upgrade-using-the-official-repositories). 1. If you paused replication in step 1, [resume replication on each **secondary**](../index.md#pausing-and-resuming-replication) -1. [Test](#check-status-after-updating) **primary** and **secondary** sites, and check version in each. +1. [Test](#check-status-after-upgrading) **primary** and **secondary** sites, and check version in each. -### Check status after updating +### Check status after upgrading -Now that the update process is complete, you may want to check whether +Now that the upgrade process is complete, you may want to check whether everything is working correctly: 1. Run the Geo Rake task on an application node for the primary and secondary sites. Everything should be green: diff --git a/doc/administration/geo/replication/usage.md b/doc/administration/geo/replication/usage.md index d96bfae78a1..fe0e06e7ea4 100644 --- a/doc/administration/geo/replication/usage.md +++ b/doc/administration/geo/replication/usage.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -11,9 +11,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w After you set up the [database replication and configure the Geo nodes](../index.md#setup-instructions), use your closest GitLab site as you would do with the primary one. You can push directly to a **secondary** site (for both HTTP, SSH including -Git LFS), and the request will be proxied to the primary site instead. +Git LFS), and the request is proxied to the primary site instead. -Example of the output you will see when pushing to a **secondary** site: +Example of the output you see when pushing to a **secondary** site: ```shell $ git push @@ -31,7 +31,7 @@ If you're using HTTPS instead of [SSH](../../../user/ssh.md) to push to the seco you can't store credentials in the URL like `user:password@URL`. Instead, you can use a [`.netrc` file](https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html) for Unix-like operating systems or `_netrc` for Windows. In that case, the credentials -will be stored as a plain text. If you're looking for a more secure way to store credentials, +are stored as a plain text. If you're looking for a more secure way to store credentials, you can use [Git Credential Storage](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). ## Fetch Go modules from Geo secondary sites diff --git a/doc/administration/geo/replication/version_specific_updates.md b/doc/administration/geo/replication/version_specific_upgrades.md index 6b617a21be8..f0925bdf87e 100644 --- a/doc/administration/geo/replication/version_specific_updates.md +++ b/doc/administration/geo/replication/version_specific_upgrades.md @@ -1,24 +1,24 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Version-specific update instructions **(PREMIUM SELF)** +# Version-specific upgrade instructions **(PREMIUM SELF)** -Review this page for update instructions for your version. These steps -accompany the [general steps](updating_the_geo_sites.md#general-update-steps) -for updating Geo sites. +Review this page for upgrade instructions for your version. These steps +accompany the [general steps](upgrading_the_geo_sites.md#general-upgrade-steps) +for upgrading Geo sites. -## Updating to 14.9 +## Upgrading to 14.9 -**DO NOT** update to GitLab 14.9.0. Instead, use 14.9.1 or later. +**Do not** upgrade to GitLab 14.9.0. Instead, use 14.9.1 or later. We've discovered an issue with Geo's CI verification feature that may [cause job traces to be lost](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/6664). This issue was fixed in [the GitLab 14.9.1 patch release](https://about.gitlab.com/releases/2022/03/23/gitlab-14-9-1-released/). -If you have already updated to GitLab 14.9.0, you can disable the feature causing the issue by [disabling the `geo_job_artifact_replication` feature flag](../../feature_flags.md#how-to-enable-and-disable-features-behind-flags). +If you have already upgraded to GitLab 14.9.0, you can disable the feature causing the issue by [disabling the `geo_job_artifact_replication` feature flag](../../feature_flags.md#how-to-enable-and-disable-features-behind-flags). -## Updating to 14.2 through 14.7 +## Upgrading to 14.2 through 14.7 There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467) that affects Geo when the GitLab-managed object storage replication is used, causing blob object types to fail synchronization. @@ -33,11 +33,11 @@ results in a loop that consistently fails for all objects stored in object stora For information on how to fix this, see [Troubleshooting - Failed syncs with GitLab-managed object storage replication](troubleshooting.md#failed-syncs-with-gitlab-managed-object-storage-replication). -## Updating to 14.4 +## Upgrading to 14.4 There is [an issue in GitLab 14.4.0 through 14.4.2](../../../update/index.md#1440) that can affect Geo and other features that rely on cronjobs. We recommend upgrading to GitLab 14.4.3 or later. -## Updating to 14.1, 14.2, 14.3 +## Upgrading to 14.1, 14.2, 14.3 ### Multi-arch images @@ -72,13 +72,13 @@ Otherwise, for each **secondary** site, on a Rails application node, open a [Rai end ``` -If you are running a version prior to 14.1 and are using Geo and multi-arch containers in your Container Registry, we recommend [upgrading](updating_the_geo_sites.md) to at least GitLab 14.1. +If you are running a version prior to 14.1 and are using Geo and multi-arch containers in your Container Registry, we recommend [upgrading](upgrading_the_geo_sites.md) to at least GitLab 14.1. ### Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode -GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop updating and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). +GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop upgrading and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). -## Updating to GitLab 14.0/14.1 +## Upgrading to GitLab 14.0/14.1 ### Primary sites can not be removed from the UI @@ -90,13 +90,13 @@ If you are running an affected version and need to remove your Primary site, you ### Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode -GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop updating and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). +GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop upgrading and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). -## Updating to GitLab 13.12 +## Upgrading to GitLab 13.12 -### Secondary sites re-download all LFS files upon update +### Secondary sites re-download all LFS files upon upgrade -We found an issue where [secondary sites re-download all LFS files](https://gitlab.com/gitlab-org/gitlab/-/issues/334550) upon update. This bug: +We found an issue where [secondary sites re-download all LFS files](https://gitlab.com/gitlab-org/gitlab/-/issues/334550) upon upgrade. This bug: - Only applies to Geo secondary sites that have replicated LFS objects. - Is _not_ a data loss risk. @@ -104,9 +104,9 @@ We found an issue where [secondary sites re-download all LFS files](https://gitl - May impact performance for GitLab installations with a large number of LFS files. If you don't have many LFS objects or can stand a bit of churn, then it is safe to let the secondary sites re-download LFS objects. -If you do have many LFS objects, or many Geo secondary sites, or limited bandwidth, or a combination of them all, then we recommend you skip GitLab 13.12.0 through 13.12.6 and update to GitLab 13.12.7 or newer. +If you do have many LFS objects, or many Geo secondary sites, or limited bandwidth, or a combination of them all, then we recommend you skip GitLab 13.12.0 through 13.12.6 and upgrade to GitLab 13.12.7 or newer. -#### If you have already updated to an affected version, and the re-sync is ongoing +#### If you have already upgraded to an affected version, and the re-sync is ongoing You can manually migrate the legacy sync state to the new state column by running the following command in a [Rails console](../../operations/rails_console.md). It should take under a minute: @@ -116,29 +116,29 @@ Geo::LfsObjectRegistry.where(state: 0, success: true).update_all(state: 2) ### Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode -GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop updating and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). +GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop upgrading and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). -## Updating to GitLab 13.11 +## Upgrading to GitLab 13.11 We found an [issue with Git clone/pull through HTTP(s)](https://gitlab.com/gitlab-org/gitlab/-/issues/330787) on Geo secondaries and on any GitLab instance if maintenance mode is enabled. This was caused by a regression in GitLab Workhorse. This is fixed in the [GitLab 13.11.4 patch release](https://about.gitlab.com/releases/2021/05/14/gitlab-13-11-4-released/). To avoid this issue, upgrade to GitLab 13.11.4 or later. ### Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode -GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop updating and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). +GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop upgrading and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). -## Updating to GitLab 13.10 +## Upgrading to GitLab 13.10 ### Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode -GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop updating and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). +GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop upgrading and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). -## Updating to GitLab 13.9 +## Upgrading to GitLab 13.9 -### Error during zero-downtime update: "cannot drop column asset_proxy_whitelist" +### Error during zero-downtime upgrade: "cannot drop column asset_proxy_whitelist" We've detected an issue [with a column rename](https://gitlab.com/gitlab-org/gitlab/-/issues/324160) that prevents upgrades to GitLab 13.9.0, 13.9.1, 13.9.2 and 13.9.3 when following the zero-downtime steps. It is necessary -to perform the following additional steps for the zero-downtime update: +to perform the following additional steps for the zero-downtime upgrade: 1. Before running the final `sudo gitlab-rake db:migrate` command on the deploy node, execute the following queries using the PostgreSQL console (or `sudo gitlab-psql`) @@ -169,27 +169,27 @@ PG::DependentObjectsStillExist: ERROR: cannot drop column asset_proxy_whitelist DETAIL: trigger trigger_0d588df444c8 on table application_settings depends on column asset_proxy_whitelist of table application_settings ``` -To work around this bug, follow the previous steps to complete the update. +To work around this bug, follow the previous steps to complete the upgrade. More details are available [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/324160). ### Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode -GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop updating and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). +GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab Maintenance Mode](../../maintenance_mode/index.md) causes Geo secondary site statuses to appear to stop upgrading and become unhealthy. For more information, see [Troubleshooting - Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). -## Updating to GitLab 13.7 +## Upgrading to GitLab 13.7 We've detected an issue with the `FetchRemove` call used by Geo secondaries. This causes performance issues as we execute reference transaction hooks for -each updated reference. Delay any upgrade attempts until this is in the +each upgraded reference. Delay any upgrade attempts until this is in the [13.7.5 patch release.](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/3002). More details are available [in this issue](https://gitlab.com/gitlab-org/git/-/issues/79). -## Updating to GitLab 13.5 +## Upgrading to GitLab 13.5 GitLab 13.5 has a [regression that prevents viewing a list of container repositories and registries](https://gitlab.com/gitlab-org/gitlab/-/issues/285475) on Geo secondaries. This issue is fixed in GitLab 13.6.1 and later. -## Updating to GitLab 13.3 +## Upgrading to GitLab 13.3 In GitLab 13.3, Geo removed the PostgreSQL [Foreign Data Wrapper](https://www.postgresql.org/docs/11/postgres-fdw.html) dependency for the tracking database. @@ -219,61 +219,61 @@ when using `--force` or `--skip-preflight-checks`, due to [an issue](https://git The [troubleshooting steps](troubleshooting.md#errors-when-using---skip-preflight-checks-or---force) contain a workaround if you run into errors during the failover. -## Updating to GitLab 13.2 +## Upgrading to GitLab 13.2 In GitLab 13.2, promoting a secondary site to a primary while the secondary is paused fails. Do not pause replication before promoting a secondary. If the site is paused, be sure to resume before promoting. To avoid this issue, upgrade to GitLab 13.4 or later. -## Updating to GitLab 13.0 +## Upgrading to GitLab 13.0 Upgrading to GitLab 13.0 requires GitLab 12.10 to already be using PostgreSQL version 11. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). -## Updating to GitLab 12.10 +## Upgrading to GitLab 12.10 -GitLab 12.10 doesn't attempt to update the embedded PostgreSQL server when +GitLab 12.10 doesn't attempt to upgrade the embedded PostgreSQL server when using Geo, because the PostgreSQL upgrade requires downtime for secondaries while reinitializing streaming replication. It must be upgraded manually. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). -## Updating to GitLab 12.9 +## Upgrading to GitLab 12.9 WARNING: GitLab 12.9.0 through GitLab 12.9.3 are affected by [a bug that stops repository verification](https://gitlab.com/gitlab-org/gitlab/-/issues/213523). The issue is fixed in GitLab 12.9.4. Upgrade to GitLab 12.9.4 or later. -By default, GitLab 12.9 attempts to update the embedded PostgreSQL server +By default, GitLab 12.9 attempts to upgrade the embedded PostgreSQL server version from 9.6 to 10.12, which requires downtime on secondaries while reinitializing streaming replication. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). You can temporarily disable this behavior by running the following before -updating: +upgrading: ```shell sudo touch /etc/gitlab/disable-postgresql-upgrade ``` -## Updating to GitLab 12.8 +## Upgrading to GitLab 12.8 -By default, GitLab 12.8 attempts to update the embedded PostgreSQL server +By default, GitLab 12.8 attempts to upgrade the embedded PostgreSQL server version from 9.6 to 10.12, which requires downtime on secondaries while reinitializing streaming replication. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). You can temporarily disable this behavior by running the following before -updating: +upgrading: ```shell sudo touch /etc/gitlab/disable-postgresql-upgrade ``` -## Updating to GitLab 12.7 +## Upgrading to GitLab 12.7 WARNING: Only upgrade to GitLab 12.7.5 or later. Do not upgrade to versions 12.7.0 @@ -281,65 +281,65 @@ through 12.7.4 because there is [an initialization order bug](https://gitlab.com [The fix](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/24021) was shipped in 12.7.5. -By default, GitLab 12.7 attempts to update the embedded PostgreSQL server +By default, GitLab 12.7 attempts to upgrade the embedded PostgreSQL server version from 9.6 to 10.9, which requires downtime on secondaries while reinitializing streaming replication. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). You can temporarily disable this behavior by running the following before -updating: +upgrading: ```shell sudo touch /etc/gitlab/disable-postgresql-upgrade ``` -## Updating to GitLab 12.6 +## Upgrading to GitLab 12.6 -By default, GitLab 12.6 attempts to update the embedded PostgreSQL server +By default, GitLab 12.6 attempts to upgrade the embedded PostgreSQL server version from 9.6 to 10.9, which requires downtime on secondaries while reinitializing streaming replication. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). You can temporarily disable this behavior by running the following before -updating: +upgrading: ```shell sudo touch /etc/gitlab/disable-postgresql-upgrade ``` -## Updating to GitLab 12.5 +## Upgrading to GitLab 12.5 -By default, GitLab 12.5 attempts to update the embedded PostgreSQL server +By default, GitLab 12.5 attempts to upgrade the embedded PostgreSQL server version from 9.6 to 10.9, which requires downtime on secondaries while reinitializing streaming replication. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). You can temporarily disable this behavior by running the following before -updating: +upgrading: ```shell sudo touch /etc/gitlab/disable-postgresql-upgrade ``` -## Updating to GitLab 12.4 +## Upgrading to GitLab 12.4 -By default, GitLab 12.4 attempts to update the embedded PostgreSQL server +By default, GitLab 12.4 attempts to upgrade the embedded PostgreSQL server version from 9.6 to 10.9, which requires downtime on secondaries while reinitializing streaming replication. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). You can temporarily disable this behavior by running the following before -updating: +upgrading: ```shell sudo touch /etc/gitlab/disable-postgresql-upgrade ``` -## Updating to GitLab 12.3 +## Upgrading to GitLab 12.3 WARNING: If the existing PostgreSQL server version is 9.6.x, we recommend upgrading to -GitLab 12.4 or later. By default, GitLab 12.3 attempts to update the embedded +GitLab 12.4 or later. By default, GitLab 12.3 attempts to upgrade the embedded PostgreSQL server version from 9.6 to 10.9. In certain circumstances, it can fail. For more information, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). @@ -349,11 +349,11 @@ requires downtime for secondaries while reinitializing streaming replication. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). -## Updating to GitLab 12.2 +## Upgrading to GitLab 12.2 WARNING: If the existing PostgreSQL server version is 9.6.x, we recommend upgrading to -GitLab 12.4 or later. By default, GitLab 12.2 attempts to update the embedded +GitLab 12.4 or later. By default, GitLab 12.2 attempts to upgrade the embedded PostgreSQL server version from 9.6 to 10.9. In certain circumstances, it can fail. For more information, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). @@ -363,12 +363,12 @@ requires downtime for secondaries while reinitializing streaming replication. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). -GitLab 12.2 includes the following minor PostgreSQL updates: +GitLab 12.2 includes the following minor PostgreSQL upgrades: - To version `9.6.14`, if you run PostgreSQL 9.6. - To version `10.9`, if you run PostgreSQL 10. -This update occurs even if major PostgreSQL updates are disabled. +This upgrade occurs even if major PostgreSQL upgrades are disabled. Before [refreshing Foreign Data Wrapper during a Geo upgrade](../../../update/zero_downtime.md#step-4-run-post-deployment-migrations-and-checks), restart the Geo tracking database: @@ -380,11 +380,11 @@ sudo gitlab-ctl restart geo-postgresql The restart avoids a version mismatch when PostgreSQL tries to load the FDW extension. -## Updating to GitLab 12.1 +## Upgrading to GitLab 12.1 WARNING: If the existing PostgreSQL server version is 9.6.x, we recommend upgrading to -GitLab 12.4 or later. By default, GitLab 12.1 attempts to update the embedded +GitLab 12.4 or later. By default, GitLab 12.1 attempts to upgrade the embedded PostgreSQL server version from 9.6 to 10.9. In certain circumstances, it can fail. For more information, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). @@ -394,7 +394,7 @@ requires downtime for secondaries while reinitializing streaming replication. For the recommended procedure, see the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance). -## Updating to GitLab 12.0 +## Upgrading to GitLab 12.0 WARNING: This version is affected by a [bug that results in new LFS objects not being diff --git a/doc/administration/geo/secondary_proxy/index.md b/doc/administration/geo/secondary_proxy/index.md index 768adab9101..9a1aab8c238 100644 --- a/doc/administration/geo/secondary_proxy/index.md +++ b/doc/administration/geo/secondary_proxy/index.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -10,6 +10,7 @@ type: howto > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5914) in GitLab 14.4 [with a flag](../../feature_flags.md) named `geo_secondary_proxy`. Disabled by default. > - [Enabled by default for unified URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/325732) in GitLab 14.6. > - [Disabled by default for different URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/325732) in GitLab 14.6 [with a flag](../../feature_flags.md) named `geo_secondary_proxy_separate_urls`. +> - [Enabled by default for different URLs](https://gitlab.com/gitlab-org/gitlab/-/issues/346112) in GitLab 15.1. FLAG: On self-managed GitLab, this feature is only available by default for Geo sites using a unified URL. See below to @@ -36,6 +37,9 @@ Use secondary proxying for use-cases including: - Having all Geo sites behind a single URL. - Geographically load-balancing traffic without worrying about write access. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see: [Secondary proxying using geographic load-balancer and AWS Route53](https://www.youtube.com/watch?v=TALLy7__Na8). + ## Set up a unified URL for Geo sites Secondary sites can transparently serve read-write traffic. You can @@ -104,17 +108,19 @@ gitlab: GEO_SECONDARY_PROXY: "0" ``` -## Enable Geo proxying with Separate URLs +## Geo proxying with Separate URLs + +Since GitLab 15.1, Geo secondary proxying is enabled by default for separate URLs also. -The ability to use proxying with separate URLs is still in development. You can follow the -["Geo secondary proxying with separate URLs" epic](https://gitlab.com/groups/gitlab-org/-/epics/6865) -for progress. +There are minor known issues linked in the ["Geo secondary proxying with separate URLs" +epic](https://gitlab.com/groups/gitlab-org/-/epics/6865). You can also add feedback in the epic about any use-cases that +are not possible anymore with proxying enabled. -To try out this feature, enable the `geo_secondary_proxy_separate_urls` feature flag. +If you run into issues, to disable this feature, disable the `geo_secondary_proxy_separate_urls` feature flag. SSH into one node running Rails on your primary Geo site and run: ```shell -sudo gitlab-rails runner "Feature.enable(:geo_secondary_proxy_separate_urls)" +sudo gitlab-rails runner "Feature.disable(:geo_secondary_proxy_separate_urls)" ``` In Kubernetes, you can run the same command in the toolbox pod. Refer to the diff --git a/doc/administration/geo/secondary_proxy/location_aware_external_url.md b/doc/administration/geo/secondary_proxy/location_aware_external_url.md index b5a8d4baa1f..1430e99557f 100644 --- a/doc/administration/geo/secondary_proxy/location_aware_external_url.md +++ b/doc/administration/geo/secondary_proxy/location_aware_external_url.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto diff --git a/doc/administration/geo/setup/database.md b/doc/administration/geo/setup/database.md index 52f4adc4e03..7dfca4c8f73 100644 --- a/doc/administration/geo/setup/database.md +++ b/doc/administration/geo/setup/database.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -177,10 +177,10 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o In most cases, the following addresses are used to configure GitLab Geo: - | Configuration | Address | - |:----------------------------------------|:------------------------------------------------------| - | `postgresql['listen_address']` | **Primary** node's public or VPC private address. | - | `postgresql['md5_auth_cidr_addresses']` | **Secondary** node's public or VPC private addresses. | + | Configuration | Address | + |:----------------------------------------|:----------------------------------------------------------------------| + | `postgresql['listen_address']` | **Primary** node's public or VPC private address. | + | `postgresql['md5_auth_cidr_addresses']` | **Primary** and **Secondary** nodes' public or VPC private addresses. | If you are using Google Cloud Platform, SoftLayer, or any other vendor that provides a virtual private cloud (VPC) you can use the **primary** and **secondary** nodes diff --git a/doc/administration/geo/setup/external_database.md b/doc/administration/geo/setup/external_database.md index 58fd6b28797..7e8ffa829f9 100644 --- a/doc/administration/geo/setup/external_database.md +++ b/doc/administration/geo/setup/external_database.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -51,7 +51,7 @@ developed and tested. We aim to be compatible with most external gitlab-ctl set-geo-primary-node ``` - This command will use your defined `external_url` in `/etc/gitlab/gitlab.rb`. + This command uses your defined `external_url` in `/etc/gitlab/gitlab.rb`. ### Configure the external database to be replicated @@ -64,14 +64,14 @@ To set up an external database, you can either: Given you have a primary node set up on AWS EC2 that uses RDS. You can now just create a read-only replica in a different region and the -replication process will be managed by AWS. Make sure you've set Network ACL, Subnet, and +replication process is managed by AWS. Make sure you've set Network ACL (Access Control List), Subnet, and Security Group according to your needs, so the secondary application node can access the database. The following instructions detail how to create a read-only replica for common cloud providers: - Amazon RDS - [Creating a Read Replica](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html#USER_ReadRepl.Create) -- Azure Database for PostgreSQL - [Create and manage read replicas in Azure Database for PostgreSQL](https://docs.microsoft.com/en-us/azure/postgresql/howto-read-replicas-portal) +- Azure Database for PostgreSQL - [Create and manage read replicas in Azure Database for PostgreSQL](https://docs.microsoft.com/en-us/azure/postgresql/single-server/how-to-read-replicas-portal) - Google Cloud SQL - [Creating read replicas](https://cloud.google.com/sql/docs/postgres/replication/create-replica) Once your read-only replica is set up, you can skip to [configure your secondary application node](#configure-secondary-application-nodes-to-use-the-external-read-replica). @@ -190,7 +190,7 @@ to grant additional roles to your tracking database user (by default, this is `gitlab_geo`): - Amazon RDS requires the [`rds_superuser`](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.Roles) role. -- Azure Database for PostgreSQL requires the [`azure_pg_admin`](https://docs.microsoft.com/en-us/azure/postgresql/howto-create-users#how-to-create-additional-admin-users-in-azure-database-for-postgresql) role. +- Azure Database for PostgreSQL requires the [`azure_pg_admin`](https://docs.microsoft.com/en-us/azure/postgresql/single-server/how-to-create-users#how-to-create-additional-admin-users-in-azure-database-for-postgresql) role. - Google Cloud SQL requires the [`cloudsqlsuperuser`](https://cloud.google.com/sql/docs/postgres/users#default-users) role. This is for the installation of extensions during installation and upgrades. As an alternative, diff --git a/doc/administration/geo/setup/index.md b/doc/administration/geo/setup/index.md index 04a341aa822..f4c21293782 100644 --- a/doc/administration/geo/setup/index.md +++ b/doc/administration/geo/setup/index.md @@ -1,5 +1,5 @@ --- -stage: Enablement +stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto @@ -19,7 +19,7 @@ The steps below should be followed in the order they appear. **Make sure the Git If you installed GitLab using the Omnibus packages (highly recommended): -1. [Install GitLab Enterprise Edition](https://about.gitlab.com/install/) on the nodes that will serve as the **secondary** site. Do not create an account or log in to the new **secondary** site. +1. [Install GitLab Enterprise Edition](https://about.gitlab.com/install/) on the nodes that serve as the **secondary** site. Do not create an account or log in to the new **secondary** site. The **GitLab version must match** across primary and secondary sites. 1. [Add the GitLab License](../../../user/admin_area/license.md) on the **primary** site to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. 1. [Set up the database replication](database.md) (`primary (read-write) <-> secondary (read-only)` topology). 1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). This step is required and needs to be done on **both** the **primary** and **secondary** sites. |