diff options
Diffstat (limited to 'doc/administration/geo/replication')
8 files changed, 72 insertions, 44 deletions
diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md index 166a724f9c1..52a2b1521a9 100644 --- a/doc/administration/geo/replication/datatypes.md +++ b/doc/administration/geo/replication/datatypes.md @@ -47,6 +47,8 @@ verification methods: | Blobs | Container registry _(object storage)_ | Geo with API/Managed/Docker API (*2*) | _Not implemented_ | | Blobs | Package registry _(filesystem)_ | Geo with API | _Not implemented_ | | Blobs | Package registry _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | +| Blobs | Versioned Terraform State _(filesystem)_ | Geo with API | _Not implemented_ | +| Blobs | Versioned Terraform State _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | - (*1*): Redis replication can be used as part of HA with Redis sentinel. It's not used between Geo nodes. - (*2*): Object storage replication can be performed by Geo or by your object storage provider/appliance @@ -160,39 +162,33 @@ replicating data from those features will cause the data to be **lost**. If you wish to use those features on a **secondary** node, or to execute a failover successfully, you must replicate their data using some other means. -| Feature | Replicated (added in GitLab version) | Verified (added in GitLab version) | Notes | -|:------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------|:----------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------| -| Application data in PostgreSQL | **Yes** (10.2) | **Yes** (10.2) | | -| Project repository | **Yes** (10.2) | **Yes** (10.7) | | -| Project wiki repository | **Yes** (10.2) | **Yes** (10.7) | | -| Project designs repository | **Yes** (12.7) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/32467) | | -| Uploads | **Yes** (10.2) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | Verified only on transfer, or manually (*1*) | -| LFS objects | **Yes** (10.2) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8922) | Verified only on transfer, or manually (*1*). Unavailable for new LFS objects in 11.11.x and 12.0.x (*2*). | -| CI job artifacts (other than traces) | **Yes** (10.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8923) | Verified only manually (*1*) | -| Archived traces | **Yes** (10.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8923) | Verified only on transfer, or manually (*1*) | -| Personal snippets | **Yes** (10.2) | **Yes** (10.2) | | -| [Versioned snippets](../../../user/snippets.md#versioned-snippets) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2809) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2810) | | -| Project snippets | **Yes** (10.2) | **Yes** (10.2) | | -| Object pools for forked project deduplication | **Yes** | No | | -| [Server-side Git hooks](../../server_hooks.md) | No | No | | -| [Elasticsearch integration](../../../integration/elasticsearch.md) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/1186) | No | | -| [GitLab Pages](../../pages/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/589) | No | | -| [Container Registry](../../packages/container_registry.md) | **Yes** (12.3) | No | | -| [NPM Registry](../../../user/packages/npm_registry/index.md) | **Yes** (13.2) | No | Behind feature flag `geo_package_file_replication`, enabled by default | -| [Maven Repository](../../../user/packages/maven_repository/index.md) | **Yes** (13.2) | No | Behind feature flag `geo_package_file_replication`, enabled by default | -| [Conan Repository](../../../user/packages/conan_repository/index.md) | **Yes** (13.2) | No | Behind feature flag `geo_package_file_replication`, enabled by default | -| [NuGet Repository](../../../user/packages/nuget_repository/index.md) | **Yes** (13.2) | No | Behind feature flag `geo_package_file_replication`, enabled by default | -| [PyPi Repository](../../../user/packages/pypi_repository/index.md) | **Yes** (13.2) | No | Behind feature flag `geo_package_file_replication`, enabled by default | -| [Composer Repository](../../../user/packages/composer_repository/index.md) | **Yes** (13.2) | No | Behind feature flag `geo_package_file_replication`, enabled by default | -| [External merge request diffs](../../merge_request_diffs.md) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/33817) | No | | -| [Terraform State](../../terraform_state.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/3112)(*3*) | No | | -| [Vulnerability Export](../../../user/application_security/security_dashboard/#export-vulnerabilities) | [No](https://gitlab.com/groups/gitlab-org/-/epics/3111)(*3*) | No | | -| Content in object storage | **Yes** (12.4) | No | | - -- (*1*): The integrity can be verified manually using - [Integrity Check Rake Task](../../raketasks/check.md) on both nodes and comparing - the output between them. -- (*2*): GitLab versions 11.11.x and 12.0.x are affected by [a bug that prevents any new - LFS objects from replicating](https://gitlab.com/gitlab-org/gitlab/-/issues/32696). -- (*3*): If you are using Object Storage, the replication can be performed by the - Object Storage provider if supported. Please see [Geo with Object Storage](object_storage.md) +| Feature | Replicated (added in GitLab version) | Verified (added in GitLab version) | Object Storage replication (please see [Geo with Object Storage](object_storage.md)) | Notes | +|:---------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------|:-------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Application data in PostgreSQL](../../postgresql/index.md) | **Yes** (10.2) | **Yes** (10.2) | No | | +| [Project repository](../../..//user/project/repository/) | **Yes** (10.2) | **Yes** (10.7) | No | | +| [Project wiki repository](../../../user/project/wiki/) | **Yes** (10.2) | **Yes** (10.7) | No | | +| [Uploads](../../uploads.md) | **Yes** (10.2) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | No | Verified only on transfer or manually using [Integrity Check Rake Task](../../raketasks/check.md) on both nodes and comparing the output between them. | +| [LFS objects](../../lfs/index.md) | **Yes** (10.2) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8922) | Via Object Storage provider if supported. Native Geo support (Beta). | Verified only on transfer or manually using [Integrity Check Rake Task](../../raketasks/check.md) on both nodes and comparing the output between them. GitLab versions 11.11.x and 12.0.x are affected by [a bug that prevents any new LFS objects from replicating](https://gitlab.com/gitlab-org/gitlab/-/issues/32696). | +| [Personal snippets](../../../user/snippets.md#personal-snippets) | **Yes** (10.2) | **Yes** (10.2) | No | | +| [Project snippets](../../../user/snippets.md#project-snippets) | **Yes** (10.2) | **Yes** (10.2) | No | | +| [CI job artifacts (other than Job Logs)](../../../ci/pipelines/job_artifacts.md) | **Yes** (10.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8923) | Via Object Storage provider if supported. Native Geo support (Beta) . | Verified only manually using [Integrity Check Rake Task](../../raketasks/check.md) on both nodes and comparing the output between them | +| [Job logs](../../job_logs.md) | **Yes** (10.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8923) | Via Object Storage provider if supported. Native Geo support (Beta). | Verified only on transfer or manually using [Integrity Check Rake Task](../../raketasks/check.md) on both nodes and comparing the output between them | +| [Object pools for forked project deduplication](../../../development/git_object_deduplication.md) | **Yes** | No | No | | +| [Container Registry](../../packages/container_registry.md) | **Yes** (12.3) | No | No | | +| [Content in object storage (beta)](object_storage.md) | **Yes** (12.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) | No | | +| [Project designs repository](../../../user/project/issues/design_management.md) | **Yes** (12.7) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/32467) | Via Object Storage provider if supported. Native Geo support (Beta). | | +| [NPM Registry](../../../user/packages/npm_registry/index.md) | **Yes** (13.2) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default | +| [Maven Repository](../../../user/packages/maven_repository/index.md) | **Yes** (13.2) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default | +| [Conan Repository](../../../user/packages/conan_repository/index.md) | **Yes** (13.2) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default | +| [NuGet Repository](../../../user/packages/nuget_repository/index.md) | **Yes** (13.2) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default | +| [PyPi Repository](../../../user/packages/pypi_repository/index.md) | **Yes** (13.2) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default | +| [Composer Repository](../../../user/packages/composer_repository/index.md) | **Yes** (13.2) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default | +| [Versioned Terraform State](../../terraform_state.md) | **Yes** (13.5) | No | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_terraform_state_version_replication`, enabled by default | +| [External merge request diffs](../../merge_request_diffs.md) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/33817) | No | Via Object Storage provider if supported. Native Geo support (Beta). | | +| [Versioned snippets](../../../user/snippets.md#versioned-snippets) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2809) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2810) | No | | +| [Server-side Git hooks](../../server_hooks.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1867) | No | No | | +| [Elasticsearch integration](../../../integration/elasticsearch.md) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/1186) | No | No | | +| [GitLab Pages](../../pages/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/589) | No | No | | +| [CI Pipeline Artifacts](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/pipeline_artifact.rb) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | No | Via Object Storage provider if supported. Native Geo support (Beta). | Persists additional artifacts after a pipeline completes | +| [Dependency proxy images](../../../user/packages/dependency_proxy/index.md) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/259694) | No | No | Blocked on [Geo: Secondary Mimicry](https://gitlab.com/groups/gitlab-org/-/epics/1528). Note that replication of this cache is not needed for Disaster Recovery purposes because it can be recreated from external sources. | +| [Vulnerability Export](../../../user/application_security/security_dashboard/#export-vulnerabilities) | [Not planned](https://gitlab.com/groups/gitlab-org/-/epics/3111) | No | Via Object Storage provider if supported. Native Geo support (Beta). | Not planned because they are ephemeral and sensitive. They can be regenerated on demand. | diff --git a/doc/administration/geo/replication/disable_geo.md b/doc/administration/geo/replication/disable_geo.md index aed8e5fc3bc..14a11d9c1e3 100644 --- a/doc/administration/geo/replication/disable_geo.md +++ b/doc/administration/geo/replication/disable_geo.md @@ -29,7 +29,7 @@ anymore on these nodes. You can follow our docs to [remove your secondary Geo no If the current node that you want to keep using is a secondary node, you need to first promote it to primary. You can use our steps on [how to promote a secondary node](../disaster_recovery/#step-3-promoting-a-secondary-node) -in order to do that. +to do that. ## Remove the primary node from the UI diff --git a/doc/administration/geo/replication/faq.md b/doc/administration/geo/replication/faq.md index 3892d73b465..f7f391b360e 100644 --- a/doc/administration/geo/replication/faq.md +++ b/doc/administration/geo/replication/faq.md @@ -67,3 +67,7 @@ That's totally fine. We use HTTP(s) to fetch repository changes from the **prima ## Is this possible to set up a Docker Registry for a **secondary** node that mirrors the one on the **primary** node? Yes. See [Docker Registry for a **secondary** node](docker_registry.md). + +## Can I login to a secondary node? + +Yes, but secondary nodes receive all authentication data (like user accounts and logins) from the primary instance. This means you will be re-directed to the primary for authentication and routed back afterwards. diff --git a/doc/administration/geo/replication/geo_validation_tests.md b/doc/administration/geo/replication/geo_validation_tests.md index 8247b8c6336..efd070635cb 100644 --- a/doc/administration/geo/replication/geo_validation_tests.md +++ b/doc/administration/geo/replication/geo_validation_tests.md @@ -114,6 +114,22 @@ The following are GitLab upgrade validation tests we performed. The following are PostgreSQL upgrade validation tests we performed. +### September 2020 + +[Verify PostgreSQL 12 upgrade for Geo installations](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5454): + +- Description: With PostgreSQL 12 available as an opt-in version in GitLab 13.3, we tested upgrading + existing Geo installations from PostgreSQL 11 to 12. We also re-tested fresh installations of GitLab + with Geo after fixes were made to support PostgreSQL 12. These tests were done using a + [nightly build](https://packages.gitlab.com/gitlab/nightly-builds/packages/ubuntu/bionic/gitlab-ee_13.3.6+rnightly.169516.d5209202-0_amd64.deb) + of GitLab 13.4. +- Outcome: Tests were successful for Geo deployments with a single database node on the primary and secondary. + We encountered known issues with repmgr and Patroni managed PostgreSQL clusters on the Geo primary. Using + PostgreSQL 12 with a database cluster on the primary is not recommended until the issues are resolved. +- Known issues for PostgreSQL clusters: + - [Ensure Patroni detects PostgreSQL update](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5423) + - [Allow configuring permanent replication slots in patroni](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5628) + ### August 2020 [Verify Geo installation with PostgreSQL 12](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5453): diff --git a/doc/administration/geo/replication/multiple_servers.md b/doc/administration/geo/replication/multiple_servers.md index cba41c375a3..9828c52ee7d 100644 --- a/doc/administration/geo/replication/multiple_servers.md +++ b/doc/administration/geo/replication/multiple_servers.md @@ -133,7 +133,7 @@ Configure the following services, again using the non-Geo multi-node documentation: - [Configuring Redis for GitLab](../../redis/replication_and_failover.md#example-configuration-for-the-gitlab-application) for multiple nodes. -- [Gitaly](../../high_availability/gitaly.md), which will store data that is +- [Gitaly](../../gitaly/index.md), which will store data that is synchronized from the **primary** node. NOTE: **Note:** diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index f6d6f39fb19..0b6ff867f11 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -386,6 +386,15 @@ This happens when you have added IP addresses without a subnet mask in `postgres To fix this, add the subnet mask in `/etc/gitlab/gitlab.rb` under `postgresql['md5_auth_cidr_addresses']` to respect the CIDR format (i.e. `1.2.3.4/32`). +### Message: `Found data in the gitlabhq_production database!` when running `gitlab-ctl replicate-geo-database` + +This happens if data is detected in the `projects` table. When one or more projects are detected, the operation +is aborted to prevent accidental data loss. To bypass this message, pass the `--force` option to the command. + +In GitLab 13.4, a seed project is added when GitLab is first installed. This makes it necessary to pass `--force` even +on a new Geo secondary node. There is an [issue to account for seed projects](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5618) +when checking the database. + ### Very large repositories never successfully synchronize on the **secondary** node GitLab places a timeout on all repository clones, including project imports @@ -483,8 +492,8 @@ to start again from scratch, there are a few steps that can help you: gitlab-ctl start geo-postgresql ``` - Reconfigure in order to recreate the folders and make sure permissions and ownership - are correctly + Reconfigure to recreate the folders and make sure permissions and ownership + are correct: ```shell gitlab-ctl reconfigure diff --git a/doc/administration/geo/replication/updating_the_geo_nodes.md b/doc/administration/geo/replication/updating_the_geo_nodes.md index b78aeb06ebf..1af2b8d0b88 100644 --- a/doc/administration/geo/replication/updating_the_geo_nodes.md +++ b/doc/administration/geo/replication/updating_the_geo_nodes.md @@ -21,14 +21,17 @@ Updating Geo nodes involves performing: NOTE: **Note:** These general update steps are not intended for [high-availability deployments](https://docs.gitlab.com/omnibus/update/README.html#multi-node--ha-deployment), and will cause downtime. If you want to avoid downtime, consider using [zero downtime updates](https://docs.gitlab.com/omnibus/update/README.html#zero-downtime-updates). +DANGER: **Danger:** +In GitLab 13.2 and later versions, promoting a secondary node to a primary while the secondary is paused fails. We are [investigating the issue](https://gitlab.com/gitlab-org/gitlab/-/issues/225173). Do not pause replication before promoting a secondary. If the node is paused, please resume before promoting. + To update the Geo nodes when a new GitLab version is released, update **primary** and all **secondary** nodes: 1. **Optional:** [Pause replication on each **secondary** node.](../index.md#pausing-and-resuming-replication) 1. Log into the **primary** node. -1. [Update GitLab on the **primary** node using Omnibus](https://docs.gitlab.com/omnibus/update/README.html). +1. [Update GitLab on the **primary** node using Omnibus's Geo-specific steps](https://docs.gitlab.com/omnibus/update/README.html#geo-deployment). 1. Log into each **secondary** node. -1. [Update GitLab on each **secondary** node using Omnibus](https://docs.gitlab.com/omnibus/update/README.html). +1. [Update GitLab on each **secondary** node using Omnibus's Geo-specific steps](https://docs.gitlab.com/omnibus/update/README.html#geo-deployment). 1. If you paused replication in step 1, [resume replication on each **secondary**](../index.md#pausing-and-resuming-replication) 1. [Test](#check-status-after-updating) **primary** and **secondary** nodes, and check version in each. diff --git a/doc/administration/geo/replication/version_specific_updates.md b/doc/administration/geo/replication/version_specific_updates.md index 1ae246e3e61..71facb808ab 100644 --- a/doc/administration/geo/replication/version_specific_updates.md +++ b/doc/administration/geo/replication/version_specific_updates.md @@ -447,8 +447,8 @@ Omnibus is the following: > **IMPORTANT**: With GitLab 9.0, the PostgreSQL version is updated to 9.6 and manual steps are -required in order to update the **secondary** nodes and keep the Streaming -Replication working. Downtime is required, so plan ahead. +required to update the **secondary** nodes and keep the Streaming Replication +working. Downtime is required, so plan ahead. The following steps apply only if you update from a 8.17 GitLab version to 9.0+. For previous versions, update to GitLab 8.17 first before attempting to |