diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-11-17 14:33:21 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-11-17 14:33:21 +0300 |
commit | 7021455bd1ed7b125c55eb1b33c5a01f2bc55ee0 (patch) | |
tree | 5bdc2229f5198d516781f8d24eace62fc7e589e9 /doc/user/project/settings/import_export.md | |
parent | 185b095e93520f96e9cfc31d9c3e69b498cdab7c (diff) |
Add latest changes from gitlab-org/gitlab@15-6-stable-eev15.6.0-rc42
Diffstat (limited to 'doc/user/project/settings/import_export.md')
-rw-r--r-- | doc/user/project/settings/import_export.md | 165 |
1 files changed, 0 insertions, 165 deletions
diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index a47873f5179..3c12bb9b80f 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -254,168 +254,3 @@ and the exports between them are compatible. - [Project import/export administration Rake tasks](../../../administration/raketasks/project_import_export.md) - [Group import/export](../../group/settings/import_export.md) - [Group import/export API](../../../api/group_import_export.md) - -## Troubleshooting - -### Project fails to import due to mismatch - -If the [shared runners enablement](../../../ci/runners/runners_scope.md#enable-shared-runners-for-a-project) -does not match between the exported project, and the project import, the project fails to import. -Review [issue 276930](https://gitlab.com/gitlab-org/gitlab/-/issues/276930), and either: - -- Ensure shared runners are enabled in both the source and destination projects. -- Disable shared runners on the parent group when you import the project. - -### Import workarounds for large repositories - -[Maximum import size limitations](#import-a-project-and-its-data) -can prevent an import from being successful. If changing the import limits is not possible, you can -try one of the workarounds listed here. - -#### Workaround option 1 - -The following local workflow can be used to temporarily -reduce the repository size for another import attempt: - -1. Create a temporary working directory from the export: - - ```shell - EXPORT=<filename-without-extension> - - mkdir "$EXPORT" - tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/ - cd "$EXPORT"/ - git clone project.bundle - - # Prevent interference with recreating an importable file later - mv project.bundle ../"$EXPORT"-original.bundle - mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz - - git switch --create smaller-tmp-main - ``` - -1. To reduce the repository size, work on this `smaller-tmp-main` branch: - [identify and remove large files](../repository/reducing_the_repo_size_using_git.md) - or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase) - to reduce the number of commits. - - ```shell - # Reduce the .git/objects/pack/ file size - cd project - git reflog expire --expire=now --all - git gc --prune=now --aggressive - - # Prepare recreating an importable file - git bundle create ../project.bundle <default-branch-name> - cd .. - mv project/ ../"$EXPORT"-project - cd .. - - # Recreate an importable file - tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ . - ``` - -1. Import this new, smaller file into GitLab. -1. In a full clone of the original repository, - use `git remote set-url origin <new-url> && git push --force --all` - to complete the import. -1. Update the imported repository's - [branch protection rules](../protected_branches.md) and - its [default branch](../repository/branches/default.md), and - delete the temporary, `smaller-tmp-main` branch, and - the local, temporary data. - -#### Workaround option 2 - -NOTE: -This workaround does not account for LFS objects. - -Rather than attempting to push all changes at once, this workaround: - -- Separates the project import from the Git Repository import -- Incrementally pushes the repository to GitLab - -1. Make a local clone of the repository to migrate. In a later step, you push this clone outside of - the project export. -1. Download the export and remove the `project.bundle` (which contains the Git repository): - - ```shell - tar -czvf new_export.tar.gz --exclude='project.bundle' @old_export.tar.gz - ``` - -1. Import the export without a Git repository. It asks you to confirm to import without a - repository. -1. Save this bash script as a file and run it after adding the appropriate origin. - - ```shell - #!/bin/sh - - # ASSUMPTIONS: - # - The GitLab location is "origin" - # - The default branch is "main" - # - This will attempt to push in chunks of 500MB (dividing the total size by 500MB). - # Decrease this size to push in smaller chunks if you still receive timeouts. - - git gc - SIZE=$(git count-objects -v 2> /dev/null | grep size-pack | awk '{print $2}') - - # Be conservative... and try to push 2GB at a time - # (given this assumes each commit is the same size - which is wrong) - BATCHES=$(($SIZE / 500000)) - TOTAL_COMMITS=$(git rev-list --count HEAD) - if (( BATCHES > TOTAL_COMMITS )); then - BATCHES=$TOTAL_COMMITS - fi - - INCREMENTS=$(( ($TOTAL_COMMITS / $BATCHES) - 1 )) - - for (( BATCH=BATCHES; BATCH>=1; BATCH-- )) - do - COMMIT_NUM=$(( $BATCH - $INCREMENTS )) - COMMIT_SHA=$(git log -n $COMMIT_NUM --format=format:%H | tail -1) - git push -u origin ${COMMIT_SHA}:refs/heads/main - done - git push -u origin main - git push -u origin --all - git push -u origin --tags - ``` - -### Manually execute export steps - -Exports sometimes fail without giving enough information to troubleshoot. In these cases, it can be -helpful to [open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session) -and loop through [all the defined exporters](https://gitlab.com/gitlab-org/gitlab/-/blob/b67a5b5a12498d57cd877023b7385f7251e57de8/app/services/projects/import_export/export_service.rb#L65). -Execute each line individually, rather than pasting the entire block at once, so you can see any -errors each command returns. - -```shell -# User needs to have permission to export -u = User.find_by_username('someuser') -p = Project.find_by_full_path('some/project') -e = Projects::ImportExport::ExportService.new(p,u) - -e.send(:version_saver).send(:save) -e.send(:repo_saver).send(:save) -## continue using `e.send(:exporter_name).send(:save)` going through the list of exporters - -# The following line should show you the export_path similar to /var/opt/gitlab/gitlab-rails/shared/tmp/gitlab_exports/@hashed/49/94/4994.... -s = Gitlab::ImportExport::Saver.new(exportable: p, shared:p.import_export_shared) - -# To try and upload use: -s.send(:compress_and_save) -s.send(:save_upload) -``` - -### Import using the REST API fails when using a group access token - -[Group access tokens](../../group/settings/group_access_tokens.md) -don't work for project or group import operations. When a group access token initiates an import, -the import fails with this message: - -```plaintext -Error adding importer user to Project members. -Validation failed: User project bots cannot be added to other groups / projects -``` - -To use [Import REST API](../../../api/project_import_export.md), -pass regular user account credentials such as [personal access tokens](../../profile/personal_access_tokens.md). |