diff options
Diffstat (limited to 'doc/raketasks/backup_restore.md')
| -rw-r--r-- | doc/raketasks/backup_restore.md | 103 |
1 files changed, 89 insertions, 14 deletions
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index e9c9659d4f5..b6f772dee17 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -347,18 +347,21 @@ sudo -u git -H GITLAB_ASSUME_YES=1 bundle exec rake gitlab:backup:restore RAILS_ #### Back up Git repositories concurrently -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37158) in GitLab 13.3. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37158) in GitLab 13.3. +> - [Concurrent restore introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69330) in GitLab 14.3 When using [multiple repository storages](../administration/repository_storage_paths.md), -repositories can be backed up concurrently to help fully use CPU time. The +repositories can be backed up or restored concurrently to help fully use CPU time. The following variables are available to modify the default behavior of the Rake task: - `GITLAB_BACKUP_MAX_CONCURRENCY`: The maximum number of projects to back up at - the same time. Defaults to `1`. + the same time. Defaults to the number of logical CPUs (in GitLab 14.1 and + earlier, defaults to `1`). - `GITLAB_BACKUP_MAX_STORAGE_CONCURRENCY`: The maximum number of projects to back up at the same time on each storage. This allows the repository backups - to be spread across storages. Defaults to `1`. + to be spread across storages. Defaults to `2` (in GitLab 14.1 and earlier, + defaults to `1`). For example, for Omnibus GitLab installations with 4 repository storages: @@ -404,6 +407,67 @@ For Omnibus GitLab packages: 1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect +##### S3 Encrypted Buckets + +AWS supports these [modes for server side encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/serv-side-encryption.html): + +- Amazon S3-Managed Keys (SSE-S3) +- Customer Master Keys (CMKs) Stored in AWS Key Management Service (SSE-KMS) +- Customer-Provided Keys (SSE-C) + +Use your mode of choice with GitLab. Each mode has similar, but slightly +different, configuration methods. + +###### SSE-S3 + +To enable SSE-S3, in the backup storage options set the `server_side_encryption` +field to `AES256`. For example, in Omnibus GitLab: + +```ruby +gitlab_rails['backup_upload_storage_options'] = { + 'server_side_encryption' => 'AES256' +} +``` + +###### SSE-KMS + +To enable SSE-KMS, you'll need the [KMS key via its Amazon Resource Name (ARN) +in the `arn:aws:kms:region:acct-id:key/key-id` format](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html). Under the `backup_upload_storage_options` config setting, set: + +- `server_side_encryption` to `aws:kms`. +- `server_side_encryption_kms_key_id` to the ARN of the key. + +For example, in Omnibus GitLab: + +```ruby +gitlab_rails['backup_upload_storage_options'] = { + 'server_side_encryption' => 'aws:kms', + 'server_side_encryption_kms_key_id' => 'arn:aws:<YOUR KMS KEY ID>:' +} +``` + +###### SSE-C + +SSE-C requires you to set these encryption options: + +- `backup_encryption`: AES256. +- `backup_encryption_key`: Unencoded, 32-byte (256 bits) key. The upload fails if this isn't exactly 32 bytes. + +For example, in Omnibus GitLab: + +```ruby +gitlab_rails['backup_encryption'] = 'AES256' +gitlab_rails['backup_encryption_key'] = '<YOUR 32-BYTE KEY HERE>' +``` + +If the key contains binary characters and cannot be encoded in UTF-8, +instead, specify the key with the `GITLAB_BACKUP_ENCRYPTION_KEY` environment variable. +For example: + +```ruby +gitlab_rails['env'] = { 'GITLAB_BACKUP_ENCRYPTION_KEY' => "\xDE\xAD\xBE\xEF" * 8 } +``` + ##### Digital Ocean Spaces This example can be used for a bucket in Amsterdam (AMS3): @@ -455,15 +519,25 @@ For installations from source: # use_iam_profile: 'true' # The remote 'directory' to store your backups. For S3, this would be the bucket name. remote_directory: 'my.s3.bucket' - # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional - # encryption: 'AES256' - # Turns on AWS Server-Side Encryption with Amazon Customer-Provided Encryption Keys for backups, this is optional - # This should be set to the encryption key for Amazon S3 to use to encrypt or decrypt your data. - # 'encryption' must also be set in order for this to have any effect. - # To avoid storing the key on disk, the key can also be specified via the `GITLAB_BACKUP_ENCRYPTION_KEY` environment variable. - # encryption_key: '<key>' # Specifies Amazon S3 storage class to use for backups, this is optional # storage_class: 'STANDARD' + # + # Turns on AWS Server-Side Encryption with Amazon Customer-Provided Encryption Keys for backups, this is optional + # 'encryption' must be set in order for this to have any effect. + # 'encryption_key' should be set to the 256-bit encryption key for Amazon S3 to use to encrypt or decrypt. + # To avoid storing the key on disk, the key can also be specified via the `GITLAB_BACKUP_ENCRYPTION_KEY` your data. + # encryption: 'AES256' + # encryption_key: '<key>' + # + # + # Turns on AWS Server-Side Encryption with Amazon S3-Managed keys (optional) + # https://docs.aws.amazon.com/AmazonS3/latest/userguide/serv-side-encryption.html + # For SSE-S3, set 'server_side_encryption' to 'AES256'. + # For SS3-KMS, set 'server_side_encryption' to 'aws:kms'. Set + # 'server_side_encryption_kms_key_id' to the ARN of customer master key. + # storage_options: + # server_side_encryption: 'aws:kms' + # server_side_encryption_kms_key_id: 'arn:aws:kms:YOUR-KEY-ID-HERE' ``` 1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) @@ -807,7 +881,7 @@ You can restore a backup only to _the exact same version and type (CE/EE)_ of GitLab that you created it on (for example CE 9.1.0). If your backup is a different version than the current installation, you must -[downgrade your GitLab installation](https://docs.gitlab.com/omnibus/update/README.html#downgrade) +[downgrade your GitLab installation](../update/package/downgrade.md) before restoring the backup. ### Restore prerequisites @@ -1138,7 +1212,8 @@ There are a few possible downsides to this: the last project that gets backed up. - Fork networks should be entirely read-only while the projects inside get backed up to prevent potential changes to the pool repository. -There is an **experimental** script that attempts to automate this process in [Snippet 2149205](https://gitlab.com/-/snippets/2149205). +There is an **experimental** script that attempts to automate this process in +[the Geo team Runbooks project](https://gitlab.com/gitlab-org/geo-team/runbooks/-/tree/main/experimental-online-backup-through-rsync). ## Backup and restore for installations using PgBouncer @@ -1295,7 +1370,7 @@ Be sure to create a full database backup before attempting any changes. #### Disable user two-factor authentication (2FA) Users with 2FA enabled can't sign in to GitLab. In that case, you must -[disable 2FA for everyone](../security/two_factor_authentication.md#disabling-2fa-for-everyone), +[disable 2FA for everyone](../security/two_factor_authentication.md#disable-2fa-for-everyone), after which users must reactivate 2FA. #### Reset CI/CD variables |