| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
only.
|
|
This commit fixes a typo where the mode argument to FileUtils.mkdir()
would be passed in decimal rather than octal format, yielding bad
permissions.
|
|
Since version ruby-2.2.0, method `File::exists?` is deprecated.
|
|
Set permissions of backup dir to g+s
Closes #20188
|
|
|
|
Closes #12710
|
|
Colorize is a gem licensed under the GPLv2, so we can’t use it in GitLab without relicensing GitLab under the terms of the GPL. Rainbow is licensed under the MIT license and does the exact same thing as Colorize, so Rainbow was added in place of Colorize.
The syntax is slightly different for Rainbow vs. Colorize, and was updated in accordance.
The gem is still a dependency of Spinach, so it’s included in the development/test environments, but won’t be packaged with the actual product, and therefore doesn’t require we relicense the product.
An attempt at relicensing Colorize was made, but didn’t succeed as the library owner never responded.
Rainbow library: https://github.com/sickill/rainbow
Relevant issue regarding licensing in GitLab's gems: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3775
|
|
This is idempotent, so there's no harm calling it if the directory
already exists.
Closes #12710
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Offloads uploading to GitLab Workhorse
- Use /authorize request for fast uploading
- Added backup recipes for artifacts
- Support download acceleration using X-Sendfile
|
|
Closes #3311
|
|
|
|
|
|
|
|
|
|
Documentation elsewhere refers to this internal path, let's keep
it.
|
|
|
|
By using light gzip compression we can save a lot of disk IO during
the backup.
|
|
During the backup we create an intermediate copy of two directories:
builds and uploads. Instead of creating many small files with 'cp
-r', we now use tar (and fast gzip) to create single intermediate
files. This saves on disk IO and disk space while creating a backup.
|
|
This adds support for AWS S3 SSE with S3 managed keys, this means the
data is encrypted at rest and the encryption is handled transparently to
the end user as well as in the AWS Console. This is optional and not
required to make S3 uploads work.
|
|
|
|
|
|
|
|
|
|
The change in baa157926d432f404a41c31ad6514ff8d5366269 broke backup
restore fucnctionality. This would not lead to data loss, but it
prevented the restore script from working. This bug exists only in
7.14.0 release candidate versions, not in 7.13.
Reported in https://github.com/gitlabhq/gitlabhq/issues/9571 .
|
|
command line.
|
|
|
|
|
|
This sidesteps problems with running 'chmod' on some CIFS mounts.
|
|
backup-archive-permissions
|
|
The existing behavior of the backups is to overwrite whatever data
was still there in the scratch directories. This broke when we added
a 'gzip' step because 'gzip database.sql' will fail if 'database.sql.gz'
already exists. Doing 'rm -f database.sql.gz' before the 'gzip'
avoids this failure.
|
|
Use native Postgres database cleaning during backup restore
We were using hacks to drop tables etc during a Postgres backup
restore. With this change, we let pg_dump insert the DROP TABLE
statements it needs at the start of the SQL dump.
See merge request !1891
|
|
We were using hacks to drop tables etc during a Postgres backup
restore. With this change, we let pg_dump insert the DROP TABLE
statements it needs at the start of the SQL dump.
|
|
This change helps system administrators who want to replicate
GitLab backup files without needing root permissions.
|
|
|
|
Avoid "cannot copy directory ... to itself" error on restore (on Docker?)
rake gitlab:backup:restore fails for me in my Docker-hosted Gitlab-CE instance; during the restore, any existing "uploads" directory is backed up by [this code](https://gitlab.com/gitlab-org/gitlab-ce/blob/833bc30/lib/backup/uploads.rb#L23) --
```ruby
def backup_existing_uploads_dir
timestamped_uploads_path = File.join(app_uploads_dir, '..', "uploads.#{Time.now.to_i}")
if File.exists?(app_uploads_dir)
FileUtils.mv(app_uploads_dir, timestamped_uploads_path)
end
end
```
When this executes for me, the ```FileUtils.mv``` parameters are "/home/git/gitlab/public/uploads" and "/home/git/gitlab/public/uploads/../uploads.1407019546"; an exception is raised, producing this double stacktrace:
```
ArgumentError: cannot copy directory /home/git/gitlab/public/uploads to itself /home/git/gitlab/public/uploads/../uploads.1407019546
/home/git/gitlab/lib/backup/uploads.rb:26:in `backup_existing_uploads_dir'
/home/git/gitlab/lib/backup/uploads.rb:18:in `restore'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:73:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:30:in `block (3 levels) in <top (required)>'
Errno::EXDEV: Invalid cross-device link @ sys_fail2 - (/home/git/gitlab/public/uploads, /home/git/gitlab/public/uploads/../uploads.1407019546)
/home/git/gitlab/lib/backup/uploads.rb:26:in `backup_existing_uploads_dir'
/home/git/gitlab/lib/backup/uploads.rb:18:in `restore'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:73:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:30:in `block (3 levels) in <top (required)>'
Tasks: TOP => gitlab:backup:uploads:restore
(See full trace by running task with --trace)
```
I'm guessing from the first message that ```mv``` walks the destination path to ensure that we're not moving the source into itself -- it doesn't get as far as interpreting the '..', but throws when it sees that the destination appears to start with the source path.
The second stacktrace I have no clue about - maybe it's AUFS- or Docker-related?
I attempted to reproduce this separately with the omnibus distribution in a fresh Ubuntu 14.04 install without Docker involved, and was unable to - backup and restore worked fine. I then tested my theory by FileUtils.expand_path-ing the destination in my own Docker setup code, and that made the problem go away, so that's what this merge request does.
(I'm using backups created and restored on gitlab-ce 7-1-stable, at facfec4b2; this is on Ubuntu 14.04 with Docker 1.1.1)
I know I'd look askance at a PR without tests for an unreproducable problem, but even if this is rejected, I'm submitting it anyway because maybe someone else will Google it and find it useful. I'm happy to do more work to improve this if you have suggestions.
See merge request !165
|
|
|
|
|
|
|
|
When creating backup tar files, only change permissions on the `db`,
`uploads`, and `repositories` directories, not their contents.
|
|
|
