diff options
Diffstat (limited to 'doc/user/project/settings/import_export.md')
| -rw-r--r-- | doc/user/project/settings/import_export.md | 175 |
1 files changed, 93 insertions, 82 deletions
diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 3715eef08e0..958246908bc 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -1,46 +1,70 @@ --- stage: Manage -group: Import +group: Import and Integrate info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" --- # Migrating projects using file exports **(FREE)** Existing projects on any self-managed GitLab instance or GitLab.com can be exported to a file and -then imported into a new GitLab instance. You can also: +then imported into another GitLab instance. You can also copy GitLab projects to another location with more automation by +[migrating groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). -- [Migrate groups](../../group/import/index.md) using the preferred method. -- [Migrate groups using file exports](../../group/settings/import_export.md). +## Preserving user contributions -GitLab maps user contributions correctly when an admin access token is used to perform the import. +Preserving user contribution depends on meeting the following requirements: -Consequently, migrating projects using file exports does not map user contributions correctly when you are importing -projects from a self-managed instance to GitLab.com. +### Migrating from GitLab self-managed to GitLab.com -Instead, all GitLab user associations (such as comment author) are changed to the user importing the project. For more -information, see the prerequisites and important notes in these sections: +When migrating projects by using file exports, an administrator's access token is required for user contributions to map correctly. -- [Export a project and its data](../settings/import_export.md#export-a-project-and-its-data). -- [Import the project](../settings/import_export.md#import-a-project-and-its-data). +Therefore, user contributions never map correctly when importing file exports from a self-managed instance to GitLab.com. Instead, all GitLab user associations (such as +comment author) are changed to the user importing the project. To preserve contribution history, do one of the following: -To preserve contribution history, [migrate using direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). +- [Migrate by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). +- Consider paid GitLab [migration services](https://about.gitlab.com/services/migration/). -If you migrate from GitLab.com to self-managed GitLab, an administrator can create users on the self-managed GitLab instance. +### Migrating to GitLab self-managed + +To ensure GitLab maps users and their contributions correctly: + +- The owner of the project's top-level group should export the project so that the information of all members (direct and inherited) with access to the project can be included in the exported file. Project maintainers and owners can initiate the project export. However, only direct members of a project are then exported. +- An administrator must perform the import with an administrator access token. +- Required users must exist on the destination GitLab instance. An administrator can create confirmed users either in bulk in a Rails console or one by one in the UI. +- Users must [set a public email in their profiles](../../profile/index.md#set-your-public-email) on the source GitLab instance that matches their primary email + address on the destination GitLab instance. You can also manually add users' public emails by + [editing project export files](#edit-project-export-files). + +When the email of an existing user matches the email of an imported user, that user is added as a [direct member](../members/index.md) to the imported project. + +If any of the previous conditions are not met, user contributions are not mapped correctly. Instead, all GitLab user associations are changed to the user who performed the import. +That user becomes an author of merge requests created by other users. Supplementary comments mentioning original authors are: + +- Added for comments, merge request approvals, linked tasks, and items. +- Not added for the merge request or issue creator, added or removed labels, and merged-by information. + +## Edit project export files + +You can add or remove data from export files. For example, you can: + +- Manually add users public emails to the `project_members.ndjson` file. +- Trim CI pipelines by removing lines from the `ci_pipelines.ndjson` file. + +To edit a project export file: + +1. Extract the exported `.tar.gz` file. +1. Edit the appropriate file . For example, `tree/project/project_members.ndjson`. +1. Compress the files back to a `.tar.gz` file. + +You can also make sure that all members were exported by checking the `project_members.ndjson` file. ## Compatibility -FLAG: -On self-managed GitLab by default project, file exports are in NDJSON format. To make GitLab produce project file -exports in JSON format, ask an administrator to [disable the feature flags](../../../administration/feature_flags.md) -named `project_export_as_ndjson`. To allow GitLab to import project file exports in JSON format, ask an administrator to -[disable the feature flags](../../../administration/feature_flags.md) named `project_import_ndjson`. On GitLab.com, -project file exports are in NDJSON format only. +> Support for JSON-formatted project file exports [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/389888) in GitLab 15.11. -Project file exports are in NDJSON format. Before version 14.0, GitLab produced project file exports in JSON format. -To support transitions, you can still import JSON-formatted project file exports if you configure the relevant feature -flags. +Project file exports are in NDJSON format. -From GitLab 13.0, GitLab can import project file exports that were exported from a version of GitLab up to two +You can import project file exports that were exported from a version of GitLab up to two [minor](../../../policy/maintenance.md#versioning) versions behind, which is similar to our process for [security releases](../../../policy/maintenance.md#security-releases). @@ -85,7 +109,6 @@ Prerequisites: - Review the list of [items that are exported](#items-that-are-exported). Not all items are exported. - You must have at least the Maintainer role for the project. -- Users must [set a public email](../../profile/index.md#set-your-public-email) in the source GitLab instance that matches one of their verified emails in the target GitLab instance for the user mapping to work correctly. To export a project and its data, follow these steps: @@ -103,76 +126,86 @@ moved to your configured `uploads_directory`. Every 24 hours, a worker deletes t ### Items that are exported -The [`import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) -file for projects lists many of the items exported and imported when migrating projects using file exports. View this file in the branch -for your version of GitLab to see the list of items relevant to you. For example, -[`import_export.yml` on the `14-10-stable-ee` branch](https://gitlab.com/gitlab-org/gitlab/-/blob/14-10-stable-ee/lib/gitlab/import_export/project/import_export.yml). +Exported project items depend on the version of GitLab you use. To determine if a +specific project item is exported: -Migrating projects with file exports uses the same export and import mechanisms as creating projects from templates at the [group](../../group/custom_project_templates.md) and -[instance](../../admin_area/custom_project_templates.md) levels. Therefore, the list of exported items is the same. +1. Check the [`exporters` array](https://gitlab.com/gitlab-org/gitlab/blob/0a60d6dcfa7b809cf4fe0c2e239f406014d92e34/app/services/projects/import_export/export_service.rb#L25-28). +1. Check the [`project/import_export.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/import_export/project/import_export.yml) file for projects for your GitLab version (for example, `<https://gitlab.com/gitlab-org/gitlab/-/blob/15-9-stable-ee/lib/gitlab/import_export/project/import_export.yml>` for GitLab 15.9). -Items that are exported include: +For a quick overview, items that are exported include: - Project and wiki repositories - Project uploads - Project configuration, excluding integrations - Issues - Issue comments - - Issue iteration ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96184) in 15.4) - - Issue resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Issue resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Issue resource iteration events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue iterations ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96184) in GitLab 15.4) + - Issue resource state events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue resource milestone events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Issue resource iteration events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - Merge requests - Merge request diffs - Merge request comments - - Merge request resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) - - Merge request multiple assignees ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) - - Merge request reviewers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) - - Merge request approvers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request resource state events ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4) + - Merge request multiple assignees ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request reviewers ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) + - Merge request approvers ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3) +- Commit comments ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/391601) in GitLab 15.10) - Labels - Milestones - Snippets - Time tracking and other project entities -- Design Management files and data +- Design management files and data - LFS objects - Issue boards - Pipelines history -- Push Rules -- Awards -- Group members are exported as project members, as long as the user has the Maintainer role in the - exported project's group, or is an administrator +- Push rules +- Emoji reactions +- Group members as long as the user has the Maintainer role in the + exported project's group or is an administrator + +### Items that are not exported Items that are **not** exported include: - [Child pipeline history](https://gitlab.com/gitlab-org/gitlab/-/issues/221088) +- Pipeline triggers - Build traces and artifacts - Package and container registry images - CI/CD variables -- Pipeline triggers - Webhooks - Any encrypted tokens - [Number of required approvals](https://gitlab.com/gitlab-org/gitlab/-/issues/221087) - Repository size limits - Deploy keys allowed to push to protected branches -- Secure Files -- [Activity logs for Git-related events](https://gitlab.com/gitlab-org/gitlab/-/issues/214700). For example, pushing and creating tags +- Secure files +- [Activity logs for Git-related events](https://gitlab.com/gitlab-org/gitlab/-/issues/214700) (for example, pushing and creating tags) +- Security policies associated with your project + +Migrating projects with file exports uses the same export and import mechanisms as creating projects from templates at the [group](../../group/custom_project_templates.md) and +[instance](../../admin_area/custom_project_templates.md) levels. Therefore, the list of exported items is the same. ## Import a project and its data -> Default maximum import file size [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. +> Default maximum import file size [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. Administrators of self-managed instances can [set maximum import file size](#set-maximum-import-file-size). On GitLab.com, the value is [set to 5 GB](../../gitlab_com/index.md#account-and-limit-settings). + +You can import a project and its data. WARNING: Only import projects from sources you trust. If you import a project from an untrusted source, it may be possible for an attacker to steal your sensitive data. -Prerequisites: +### Prerequisites + +> Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. - You must have [exported the project and its data](#export-a-project-and-its-data). - Compare GitLab versions and ensure you are importing to a GitLab version that is the same or later than the GitLab version you exported to. - Review [compatibility](#compatibility) for any issues. -- At least the Maintainer role on the destination group to migrate to. Using the Developer role for this purpose was - [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387891) in GitLab 15.8 and will be removed in GitLab 16.0. +- At least the Maintainer role on the destination group to migrate to. + +### Import a project To import a project: @@ -200,14 +233,9 @@ Deploy keys aren't imported. To use deploy keys, you must enable them in your im ### Import large projects **(FREE SELF)** -If you have a larger project, consider using a Rake task as described in the [developer documentation](../../../development/import_project.md#importing-via-a-rake-task). +If you have a larger project, consider [using a Rake task](../../../administration/raketasks/project_import_export.md#import-large-projects). -## Automate group and project import **(PREMIUM)** - -For information on automating user, group, and project import API calls, see -[Automate group and project import](../import/index.md#automate-group-and-project-import). - -## Maximum import file size +## Set maximum import file size **(FREE SELF)** Administrators can set the maximum import file size one of two ways: @@ -216,34 +244,15 @@ Administrators can set the maximum import file size one of two ways: The default is `0` (unlimited). -For the GitLab.com setting, see the [Account and limit settings](../../gitlab_com/index.md#account-and-limit-settings) -section of the GitLab.com settings page. - -## Map users for import - -Imported users can be mapped by their public email addresses on self-managed instances, if an administrator (not an owner) does the import. - -- The project must be exported by a project or group member with the Owner role. -- Public email addresses are not set by default. Users must [set it in their profiles](../../profile/index.md#set-your-public-email) - for mapping to work correctly. -- For contributions to be mapped correctly, users must be an existing member of the namespace, - or they can be added as a member of the project. Otherwise, a supplementary comment is left to mention that the original - author and the merge requests, notes, or issues that are owned by the importer. -- Imported users are set as [direct members](../members/index.md) - in the imported project. - -For project migration imports performed over GitLab.com groups, preserving author information is -possible through a [professional services engagement](https://about.gitlab.com/services/migration/). - ## Rate limits To help avoid abuse, by default, users are rate limited to: -| Request Type | Limit | -| ---------------- | ----- | -| Export | 6 projects per minute | -| Download export | 1 download per group per minute | -| Import | 6 projects per minute | +| Request type | Limit | +|:----------------|:--------------------------------| +| Export | 6 projects per minute | +| Download export | 1 download per group per minute | +| Import | 6 projects per minute | ## Related topics @@ -251,3 +260,5 @@ To help avoid abuse, by default, users are rate limited to: - [Project import and export administration Rake tasks](../../../administration/raketasks/project_import_export.md) - [Migrating GitLab groups](../../group/import/index.md) - [Group import and export API](../../../api/group_import_export.md) +- [Migrate groups by direct transfer](../../group/import/index.md#migrate-groups-by-direct-transfer-recommended). +- [Migrate groups by using file exports](../../group/import/index.md#migrate-groups-by-uploading-an-export-file-deprecated). |