diff options
Diffstat (limited to 'doc/raketasks/backup_restore.md')
| -rw-r--r-- | doc/raketasks/backup_restore.md | 240 |
1 files changed, 0 insertions, 240 deletions
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md deleted file mode 100644 index 2e41fad89e7..00000000000 --- a/doc/raketasks/backup_restore.md +++ /dev/null @@ -1,240 +0,0 @@ -# Backup restore - - - -## Create a backup of the GitLab system - -A backup creates an archive file that contains the database, all repositories and all attachments. -This archive will be saved in backup_path (see `config/gitlab.yml`). -The filename will be `[TIMESTAMP]_gitlab_backup.tar`. This timestamp can be used to restore an specific backup. -You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. - -``` -# use this command if you've installed GitLab with the Omnibus package -sudo gitlab-rake gitlab:backup:create - -# if you've installed GitLab from source -sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production -``` - -Also you can choose what should be backed up by adding environment variable SKIP. Available options: db, -uploads (attachments), repositories. Use a comma to specify several options at the same time. - -``` -sudo gitlab-rake gitlab:backup:create SKIP=db,uploads -``` - -Example output: - -``` -Dumping database tables: -- Dumping table events... [DONE] -- Dumping table issues... [DONE] -- Dumping table keys... [DONE] -- Dumping table merge_requests... [DONE] -- Dumping table milestones... [DONE] -- Dumping table namespaces... [DONE] -- Dumping table notes... [DONE] -- Dumping table projects... [DONE] -- Dumping table protected_branches... [DONE] -- Dumping table schema_migrations... [DONE] -- Dumping table services... [DONE] -- Dumping table snippets... [DONE] -- Dumping table taggings... [DONE] -- Dumping table tags... [DONE] -- Dumping table users... [DONE] -- Dumping table users_projects... [DONE] -- Dumping table web_hooks... [DONE] -- Dumping table wikis... [DONE] -Dumping repositories: -- Dumping repository abcd... [DONE] -Creating backup archive: $TIMESTAMP_gitlab_backup.tar [DONE] -Deleting tmp directories...[DONE] -Deleting old backups... [SKIPPING] -``` - -## Upload backups to remote (cloud) storage - -Starting with GitLab 7.4 you can let the backup script upload the '.tar' file it creates. -It uses the [Fog library](http://fog.io/) to perform the upload. -In the example below we use Amazon S3 for storage. -But Fog also lets you use [other storage providers](http://fog.io/storage/). - -For omnibus packages: - -```ruby -gitlab_rails['backup_upload_connection'] = { - 'provider' => 'AWS', - 'region' => 'eu-west-1', - 'aws_access_key_id' => 'AKIAKIAKI', - 'aws_secret_access_key' => 'secret123' -} -gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' -``` - -For installations from source: - -```yaml - backup: - # snip - upload: - # Fog storage connection settings, see http://fog.io/storage/ . - connection: - provider: AWS - region: eu-west-1 - aws_access_key_id: AKIAKIAKI - aws_secret_access_key: 'secret123' - # The remote 'directory' to store your backups. For S3, this would be the bucket name. - remote_directory: 'my.s3.bucket' -``` - -If you are uploading your backups to S3 you will probably want to create a new -IAM user with restricted access rights. To give the upload user access only for -uploading backups create the following IAM profile, replacing `my.s3.bucket` -with the name of your bucket: - -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Sid": "Stmt1412062044000", - "Effect": "Allow", - "Action": [ - "s3:AbortMultipartUpload", - "s3:GetBucketAcl", - "s3:GetBucketLocation", - "s3:GetObject", - "s3:GetObjectAcl", - "s3:ListBucketMultipartUploads", - "s3:PutObject", - "s3:PutObjectAcl" - ], - "Resource": [ - "arn:aws:s3:::my.s3.bucket/*" - ] - }, - { - "Sid": "Stmt1412062097000", - "Effect": "Allow", - "Action": [ - "s3:GetBucketLocation", - "s3:ListAllMyBuckets" - ], - "Resource": [ - "*" - ] - }, - { - "Sid": "Stmt1412062128000", - "Effect": "Allow", - "Action": [ - "s3:ListBucket" - ], - "Resource": [ - "arn:aws:s3:::my.s3.bucket" - ] - } - ] -} -``` - -## Storing configuration files - -Please be informed that a backup does not store your configuration files. -If you use an Omnibus package please see the [instructions in the readme to backup your configuration](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#backup-and-restore-omnibus-gitlab-configuration). -If you have a cookbook installation there should be a copy of your configuration in Chef. -If you have an installation from source, please consider backing up your `gitlab.yml` file, any SSL keys and certificates, and your [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). - -## Restore a previously created backup - -You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. - -``` -# Omnibus package installation -sudo gitlab-rake gitlab:backup:restore - -# installation from source -bundle exec rake gitlab:backup:restore RAILS_ENV=production -``` - -Options: - -``` -BACKUP=timestamp_of_backup (required if more than one backup exists) -``` - -Example output: - -``` -Unpacking backup... [DONE] -Restoring database tables: --- create_table("events", {:force=>true}) - -> 0.2231s -[...] -- Loading fixture events...[DONE] -- Loading fixture issues...[DONE] -- Loading fixture keys...[SKIPPING] -- Loading fixture merge_requests...[DONE] -- Loading fixture milestones...[DONE] -- Loading fixture namespaces...[DONE] -- Loading fixture notes...[DONE] -- Loading fixture projects...[DONE] -- Loading fixture protected_branches...[SKIPPING] -- Loading fixture schema_migrations...[DONE] -- Loading fixture services...[SKIPPING] -- Loading fixture snippets...[SKIPPING] -- Loading fixture taggings...[SKIPPING] -- Loading fixture tags...[SKIPPING] -- Loading fixture users...[DONE] -- Loading fixture users_projects...[DONE] -- Loading fixture web_hooks...[SKIPPING] -- Loading fixture wikis...[SKIPPING] -Restoring repositories: -- Restoring repository abcd... [DONE] -Deleting tmp directories...[DONE] -``` - -## Configure cron to make daily backups - -For Omnibus package installations, see https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#scheduling-a-backup . - -For installation from source: -``` -cd /home/git/gitlab -sudo -u git -H editor config/gitlab.yml # Enable keep_time in the backup section to automatically delete old backups -sudo -u git crontab -e # Edit the crontab for the git user -``` - -Add the following lines at the bottom: - -``` -# Create a full backup of the GitLab repositories and SQL database every day at 4am -0 4 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1 -``` - -The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors. -This is recommended to reduce cron spam. - -## Alternative backup strategies - -If your GitLab server contains a lot of Git repository data you may find the GitLab backup script to be too slow. -In this case you can consider using filesystem snapshots as part of your backup strategy. - -Example: Amazon EBS - -> A GitLab server using omnibus-gitlab hosted on Amazon AWS. -> An EBS drive containing an ext4 filesystem is mounted at `/var/opt/gitlab`. -> In this case you could make an application backup by taking an EBS snapshot. -> The backup includes all repositories, uploads and Postgres data. - -Example: LVM snapshots + rsync - -> A GitLab server using omnibus-gitlab, with an LVM logical volume mounted at `/var/opt/gitlab`. -> Replicating the `/var/opt/gitlab` directory using rsync would not be reliable because too many files would change while rsync is running. -> Instead of rsync-ing `/var/opt/gitlab`, we create a temporary LVM snapshot, which we mount as a read-only filesystem at `/mnt/gitlab_backup`. -> Now we can have a longer running rsync job which will create a consistent replica on the remote server. -> The replica includes all repositories, uploads and Postgres data. - -If you are running GitLab on a virtualized server you can possibly also create VM snapshots of the entire GitLab server. -It is not uncommon however for a VM snapshot to require you to power down the server, so this approach is probably of limited practical use. |