diff options
Diffstat (limited to 'doc/raketasks/backup_restore.md')
| -rw-r--r-- | doc/raketasks/backup_restore.md | 202 |
1 files changed, 158 insertions, 44 deletions
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 855f437cd73..bbd2d214fe4 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -5,15 +5,28 @@ An application data backup creates an archive file that contains the database, all repositories and all attachments. -You can only restore a backup to **exactly the same version** of GitLab on which -it was created. The best way to migrate your repositories from one server to -another is through backup restore. +You can only restore a backup to **exactly the same version and type (CE/EE)** +of GitLab on which it was created. The best way to migrate your repositories +from one server to another is through backup restore. ## Backup GitLab provides a simple command line interface to backup your whole installation, and is flexible enough to fit your needs. +### Requirements + +If you're using GitLab with the Omnibus package, you're all set. If you +installed GitLab from source, make sure the following packages are installed: + +* rsync + +If you're using Ubuntu, you could run: + +``` +sudo apt-get install -y rsync +``` + ### Backup timestamp >**Note:** @@ -136,45 +149,80 @@ In the example below we use Amazon S3 for storage, but Fog also lets you use for AWS, Google, OpenStack Swift, Rackspace and Aliyun as well. A local driver is [also available](#uploading-to-locally-mounted-shares). -For omnibus packages, add the following to `/etc/gitlab/gitlab.rb`: +#### Using Amazon S3 -```ruby -gitlab_rails['backup_upload_connection'] = { - 'provider' => 'AWS', - 'region' => 'eu-west-1', - 'aws_access_key_id' => 'AKIAKIAKI', - 'aws_secret_access_key' => 'secret123' - # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty - # ie. 'aws_access_key_id' => '', - # 'use_iam_profile' => 'true' -} -gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' -``` +For Omnibus GitLab packages: + +1. Add the following to `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_upload_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-west-1', + 'aws_access_key_id' => 'AKIAKIAKI', + 'aws_secret_access_key' => 'secret123' + # If using an IAM Profile, don't configure aws_access_key_id & aws_secret_access_key + # 'use_iam_profile' => true + } + gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' + ``` + +1. [Reconfigure GitLab] for the changes to take effect -Make sure to run `sudo gitlab-ctl reconfigure` after editing `/etc/gitlab/gitlab.rb` to reflect the changes. +#### Digital Ocean Spaces + +This example can be used for a bucket in Amsterdam (AMS3). + +1. Add the following to `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_upload_connection'] = { + 'provider' => 'AWS', + 'region' => 'ams3', + 'aws_access_key_id' => 'AKIAKIAKI', + 'aws_secret_access_key' => 'secret123', + 'endpoint' => 'https://ams3.digitaloceanspaces.com' + } + gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' + ``` + +1. [Reconfigure GitLab] for the changes to take effect + +#### Other S3 Providers + +Not all S3 providers are fully-compatible with the Fog library. For example, +if you see `411 Length Required` errors after attempting to upload, you may +need to downgrade the `aws_signature_version` value from the default value to +2 [due to this issue](https://github.com/fog/fog-aws/issues/428). + +--- For installations from source: -```yaml - backup: - # snip - upload: - # Fog storage connection settings, see http://fog.io/storage/ . - connection: - provider: AWS - region: eu-west-1 - aws_access_key_id: AKIAKIAKI - aws_secret_access_key: 'secret123' - # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty - # ie. aws_access_key_id: '' - # use_iam_profile: 'true' - # The remote 'directory' to store your backups. For S3, this would be the bucket name. - remote_directory: 'my.s3.bucket' - # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional - # encryption: 'AES256' - # Specifies Amazon S3 storage class to use for backups, this is optional - # storage_class: 'STANDARD' -``` +1. Edit `home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + # snip + upload: + # Fog storage connection settings, see http://fog.io/storage/ . + connection: + provider: AWS + region: eu-west-1 + aws_access_key_id: AKIAKIAKI + aws_secret_access_key: 'secret123' + # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty + # ie. aws_access_key_id: '' + # use_iam_profile: 'true' + # The remote 'directory' to store your backups. For S3, this would be the bucket name. + remote_directory: 'my.s3.bucket' + # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional + # encryption: 'AES256' + # Specifies Amazon S3 storage class to use for backups, this is optional + # storage_class: 'STANDARD' + ``` + +1. [Restart GitLab] for the changes to take effect If you are uploading your backups to S3 you will probably want to create a new IAM user with restricted access rights. To give the upload user access only for @@ -227,6 +275,50 @@ with the name of your bucket: } ``` +#### Using Google Cloud Storage + +If you want to use Google Cloud Storage to save backups, you'll have to create +an access key from the Google console first: + +1. Go to the storage settings page https://console.cloud.google.com/storage/settings +1. Select "Interoperability" and create an access key +1. Make note of the "Access Key" and "Secret" and replace them in the + configurations below +1. Make sure you already have a bucket created + +For Omnibus GitLab packages: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_upload_connection'] = { + 'provider' => 'Google', + 'google_storage_access_key_id' => 'Access Key', + 'google_storage_secret_access_key' => 'Secret' + } + gitlab_rails['backup_upload_remote_directory'] = 'my.google.bucket' + ``` + +1. [Reconfigure GitLab] for the changes to take effect + +--- + +For installations from source: + +1. Edit `home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + upload: + connection: + provider: 'Google' + google_storage_access_key_id: 'Access Key' + google_storage_secret_access_key: 'Secret' + remote_directory: 'my.google.bucket' + ``` + +1. [Restart GitLab] for the changes to take effect + ### Uploading to locally mounted shares You may also send backups to a mounted share (`NFS` / `CIFS` / `SMB` / etc.) by @@ -270,6 +362,15 @@ For installations from source: remote_directory: 'gitlab_backups' ``` +### Specifying a custom directory for backups + +If you want to group your backups you can pass a `DIRECTORY` environment variable: + +``` +sudo gitlab-rake gitlab:backup:create DIRECTORY=daily +sudo gitlab-rake gitlab:backup:create DIRECTORY=weekly +``` + ### Backup archive permissions The backup archives created by GitLab (`1393513186_2014_02_27_gitlab_backup.tar`) @@ -362,15 +463,15 @@ This is recommended to reduce cron spam. ## Restore -GitLab provides a simple command line interface to backup your whole installation, +GitLab provides a simple command line interface to restore your whole installation, and is flexible enough to fit your needs. The [restore prerequisites section](#restore-prerequisites) includes crucial information. Make sure to read and test the whole restore process at least once before attempting to perform it in a production environment. -You can only restore a backup to **exactly the same version** of GitLab that -you created it on, for example 9.1.0. +You can only restore a backup to **exactly the same version and type (CE/EE)** of +GitLab that you created it on, for example CE 9.1.0. ### Restore prerequisites @@ -395,7 +496,7 @@ more of the following options: - `BACKUP=timestamp_of_backup` - Required if more than one backup exists. Read what the [backup timestamp is about](#backup-timestamp). -- `force=yes` - Do not ask if the authorized_keys file should get regenerated. +- `force=yes` - Does not ask if the authorized_keys file should get regenerated and assumes 'yes' for warning that database tables will be removed. ### Restore for installation from source @@ -437,12 +538,20 @@ Restoring repositories: Deleting tmp directories...[DONE] ``` +Next, restore `/home/git/gitlab/.secret` if necessary as mentioned above. + +Restart GitLab: + +```shell +sudo service gitlab restart +``` + ### Restore for Omnibus installations This procedure assumes that: -- You have installed the **exact same version** of GitLab Omnibus with which the - backup was created. +- You have installed the **exact same version and type (CE/EE)** of GitLab + Omnibus with which the backup was created. - You have run `sudo gitlab-ctl reconfigure` at least once. - GitLab is running. If not, start it using `sudo gitlab-ctl start`. @@ -472,10 +581,12 @@ restore: sudo gitlab-rake gitlab:backup:restore BACKUP=1493107454_2017_04_25_9.1.0 ``` +Next, restore `/etc/gitlab/gitlab-secrets.json` if necessary as mentioned above. + Restart and check GitLab: ```shell -sudo gitlab-ctl start +sudo gitlab-ctl restart sudo gitlab-rake gitlab:check SANITIZE=true ``` @@ -536,3 +647,6 @@ The rake task runs this as the `gitlab` user which does not have the superuser a Those objects have no influence on the database backup/restore but they give this annoying warning. For more information see similar questions on postgresql issue tracker[here](http://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com) and [here](http://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us) as well as [stack overflow](http://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql). + +[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure +[restart GitLab]: ../administration/restart_gitlab.md#installations-from-source |