diff options
author | Gabriel Mazetto <brodock@gmail.com> | 2018-06-27 06:01:09 +0300 |
---|---|---|
committer | Gabriel Mazetto <brodock@gmail.com> | 2018-06-27 14:01:25 +0300 |
commit | fbc687c032bad39c7ea4ce576216290c46a263b4 (patch) | |
tree | bd93391e6413e44a7215349b812cb1aaab794765 | |
parent | 869d645069f46585b73b261c79aefece2fd87b08 (diff) |
Improve Hashed Storage documentation for rollback
Fixed storage coverage table with additional information
and wrote down implementationd details from few entities.
-rw-r--r-- | doc/administration/repository_storage_types.md | 68 |
1 files changed, 66 insertions, 2 deletions
diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index 39bd19ac851..087fe729b28 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -82,6 +82,46 @@ To migrate your existing projects to the new storage type, check the specific [rake tasks]: raketasks/storage.md#migrate-existing-projects-to-hashed-storage [storage-paths]: repository_storage_types.md +#### Rollback + +There is no automated rollback implemented. Below are the steps required to rollback +from each storage migration. + +The rollback has to be performed in the reverse order. To get into "Legacy" state, +you need to rollback Attachments first, then Project. + +Also note that if Geo is enabled, after the migration was triggered, an event is generated +to replicate the operation on any Secondary node. That means the on disk changes will also +need to be performed on these nodes as well. Database changes will propagate without issues. + +You must make sure the migration event was already processed or otherwise it may migrate +the files back to Hashed state again. + +##### Attachments + +To rollback single Attachment migration, rename `aa/bb/abcdef1234567890...` folder back to `namespace/project`. + +Both folder names can be generated by the `FileUploader.absolute_base_dir(project)`, you +just need to switch the version from the `project` back to the previous one. + +```ruby +project.storage_version +# => 2 + +FileUploader.absolute_base_dir(project) +# => "/opt/gitlab/embedded/service/gitlab-rails/public/uploads/@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35" + +project.storage_version = 1 + +FileUploader.absolute_base_dir(project) +# => "/opt/gitlab/embedded/service/gitlab-rails/public/uploads/gitlab/gitlab-shell-renamed" +``` + +##### Project + +To rollback single Project migration, move `@hashed/aa/bb/aabbcdef1234567890abcdef.git` and `@hashed/aa/bb/aabbcdef1234567890abcdef.wiki.git` +back to `namespace/project.git` and `namespace/project.wiki.git` respectively and switch the version from the `project` back to `null`. + ### Hashed Storage coverage We are incrementally moving every storable object in GitLab to the Hashed @@ -100,6 +140,30 @@ which is true for CI Cache and LFS Objects. | Pages | Yes | No | - | - | | Docker Registry | Yes | No | - | - | | CI Build Logs | No | No | - | - | -| CI Artifacts | No | No | Yes (Premium) | - | +| CI Artifacts | No | No | Yes | 9.4 / 10.6 | | CI Cache | No | No | Yes | - | -| LFS Objects | Yes | No | Yes (Premium) | - | +| LFS Objects | Yes | Similar | Yes | 10.0 / 10.7 | + +#### Implementation Details + +##### Avatars + +Each file is stored in a folder with its `id` from the database. The filename is always `avatar.png` for user avatars. +When avatar is replaced, `Upload` model is destroyed and a new one takes place with different `id`. + +##### CI Artifacts + +CI Artifacts are S3 compatible since **9.4** (GitLab Premium), and available in GitLab Core since **10.6**. + +##### LFS Objects + +LFS Objects implements a similar storage pattern using 2 chars, 2 level folders, following git own implementation: + +```ruby +"shared/lfs-objects/#{oid[0..1}/#{oid[2..3]}/#{oid[4..-1]}" + +# Based on object `oid`: `8909029eb962194cfb326259411b22ae3f4a814b5be4f80651735aeef9f3229c`, path will be: +"shared/lfs-objects/89/09/029eb962194cfb326259411b22ae3f4a814b5be4f80651735aeef9f3229c" +``` + +They are also S3 compatible since **10.0** (GitLab Premium), and available in GitLab Core since **10.7**. |