Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2022-11-10 21:11:08 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2022-11-10 21:11:08 +0300
commit4e50b9ed316f3c99693a41274babcd67c63ee640 (patch)
tree21f0f3a7f73a7dd51b06ba62ad9cfa3eaab85c9f /doc
parent609943de5e2ee6c3bf0f09d3fb1d5fc38ed5a4ed (diff)
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r--doc/api/graphql/reference/index.md1
-rw-r--r--doc/development/ee_features.md15
-rw-r--r--doc/development/import_export.md108
-rw-r--r--doc/development/performance.md2
-rw-r--r--doc/development/testing_guide/best_practices.md7
-rw-r--r--doc/user/group/import/index.md8
-rw-r--r--doc/user/project/settings/import_export.md167
-rw-r--r--doc/user/project/settings/import_export_troubleshooting.md280
8 files changed, 302 insertions, 286 deletions
diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md
index 2750ad6a2cd..834e044110c 100644
--- a/doc/api/graphql/reference/index.md
+++ b/doc/api/graphql/reference/index.md
@@ -11061,6 +11061,7 @@ Represents a code quality degradation on the pipeline.
| <a id="codequalitydegradationline"></a>`line` | [`Int!`](#int) | Line on which the code quality degradation occurred. |
| <a id="codequalitydegradationpath"></a>`path` | [`String!`](#string) | Relative path to the file containing the code quality degradation. |
| <a id="codequalitydegradationseverity"></a>`severity` | [`CodeQualityDegradationSeverity!`](#codequalitydegradationseverity) | Status of the degradation (BLOCKER, CRITICAL, MAJOR, MINOR, INFO, UNKNOWN). |
+| <a id="codequalitydegradationweburl"></a>`webUrl` | [`String`](#string) | URL to the file along with line number. |
### `Commit`
diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md
index 162b313b531..14df73b8779 100644
--- a/doc/development/ee_features.md
+++ b/doc/development/ee_features.md
@@ -135,10 +135,17 @@ version of the product:
1. Enable **Allow use of licensed EE features** to make licensed EE features available to projects
only if the project namespace's plan includes the feature.
- 1. Visit **Admin > Settings > General**.
- 1. Expand **Account and limit**.
- 1. Select the **Allow use of licensed EE features** checkbox.
- 1. Click **Save changes**.
+ 1. Visit **Admin > Settings > General**.
+ 1. Expand **Account and limit**.
+ 1. Select the **Allow use of licensed EE features** checkbox.
+ 1. Click **Save changes**.
+
+1. Ensure that the group for which you want to test the EE feature, is actually using an EE plan:
+ 1. On the top bar, select **Main menu > Admin**.
+ 1. On the left sidebar, select **Overview > Groups**.
+ 1. Identify the group you want to modify, and select **Edit**.
+ 1. Scroll to **Permissions and group features**. For **Plan**, select `Ultimate`.
+ 1. Select **Save changes**.
### Run CI pipelines in a FOSS context
diff --git a/doc/development/import_export.md b/doc/development/import_export.md
index f864dd3b678..9aab7f38dbb 100644
--- a/doc/development/import_export.md
+++ b/doc/development/import_export.md
@@ -6,116 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Import/Export development documentation
-Troubleshooting and general development guidelines and tips for the [Import/Export feature](../user/project/settings/import_export.md).
+General development guidelines and tips for the [Import/Export feature](../user/project/settings/import_export.md).
<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> This document is originally based on the [Import/Export 201 presentation available on YouTube](https://www.youtube.com/watch?v=V3i1OfExotE).
-## Troubleshooting commands
-
-Finds information about the status of the import and further logs using the JID:
-
-```ruby
-# Rails console
-Project.find_by_full_path('group/project').import_state.slice(:jid, :status, :last_error)
-> {"jid"=>"414dec93f941a593ea1a6894", "status"=>"finished", "last_error"=>nil}
-```
-
-```shell
-# Logs
-grep JID /var/log/gitlab/sidekiq/current
-grep "Import/Export error" /var/log/gitlab/sidekiq/current
-grep "Import/Export backtrace" /var/log/gitlab/sidekiq/current
-tail /var/log/gitlab/gitlab-rails/importer.log
-```
-
-## Troubleshooting performance issues
-
-Read through the current performance problems using the Import/Export below.
-
-### OOM errors
-
-Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](../administration/operations/sidekiq_memory_killer.md):
-
-```shell
-SIDEKIQ_MEMORY_KILLER_MAX_RSS = 2000000
-SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS = 3000000
-SIDEKIQ_MEMORY_KILLER_GRACE_TIME = 900
-```
-
-An import status `started`, and the following Sidekiq logs signal a memory issue:
-
-```shell
-WARN: Work still in progress <struct with JID>
-```
-
-### Timeouts
-
-Timeout errors occur due to the `Gitlab::Import::StuckProjectImportJobsWorker` marking the process as failed:
-
-```ruby
-module Gitlab
- module Import
- class StuckProjectImportJobsWorker
- include Gitlab::Import::StuckImportJob
- # ...
- end
- end
-end
-
-module Gitlab
- module Import
- module StuckImportJob
- # ...
- IMPORT_JOBS_EXPIRATION = 15.hours.to_i
- # ...
- def perform
- stuck_imports_without_jid_count = mark_imports_without_jid_as_failed!
- stuck_imports_with_jid_count = mark_imports_with_jid_as_failed!
-
- track_metrics(stuck_imports_with_jid_count, stuck_imports_without_jid_count)
- end
- # ...
- end
- end
-end
-```
-
-```shell
-Marked stuck import jobs as failed. JIDs: xyz
-```
-
-```plaintext
- +-----------+ +-----------------------------------+
- |Export Job |--->| Calls ActiveRecord `as_json` and |
- +-----------+ | `to_json` on all project models |
- +-----------------------------------+
-
- +-----------+ +-----------------------------------+
- |Import Job |--->| Loads all JSON in memory, then |
- +-----------+ | inserts into the DB in batches |
- +-----------------------------------+
-```
-
-### Problems and solutions
-
-| Problem | Possible solutions |
-| -------- | -------- |
-| [Slow JSON](https://gitlab.com/gitlab-org/gitlab/-/issues/25251) loading/dumping models from the database | [split the worker](https://gitlab.com/gitlab-org/gitlab/-/issues/25252) |
-| | Batch export
-| | Optimize SQL
-| | Move away from `ActiveRecord` callbacks (difficult)
-| High memory usage (see also some [analysis](https://gitlab.com/gitlab-org/gitlab/-/issues/18857) | DB Commit sweet spot that uses less memory |
-| | [Netflix Fast JSON API](https://github.com/Netflix/fast_jsonapi) may help |
-| | Batch reading/writing to disk and any SQL
-
-### Temporary solutions
-
-While the performance problems are not tackled, there is a process to workaround
-importing big projects, using a foreground import:
-
-[Foreground import](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/5384) of big projects for customers.
-(Using the import template in the [infrastructure tracker](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues))
-
## Security
The Import/Export feature is constantly updated (adding new things to export), however
diff --git a/doc/development/performance.md b/doc/development/performance.md
index e1918bf2e22..3881fad0528 100644
--- a/doc/development/performance.md
+++ b/doc/development/performance.md
@@ -21,7 +21,7 @@ consistent performance of GitLab. Refer to the [Index](#performance-documentatio
- [Query performance guidelines](database/query_performance.md)
- [Pagination performance guidelines](../development/database/pagination_performance_guidelines.md)
- [Keyset pagination performance](../development/database/keyset_pagination.md#performance)
- - [Troubleshooting import/export performance issues](../development/import_export.md#troubleshooting-performance-issues)
+ - [Troubleshooting import/export performance issues](../user/project/settings/import_export_troubleshooting.md#troubleshooting-performance-issues)
- [Pipelines performance in the `gitlab` project](pipelines/performance.md)
- Frontend:
- [Performance guidelines and monitoring](../development/fe_guide/performance.md)
diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md
index 61f60567418..58f6b02c1f1 100644
--- a/doc/development/testing_guide/best_practices.md
+++ b/doc/development/testing_guide/best_practices.md
@@ -1440,9 +1440,10 @@ GitLab uses [factory_bot](https://github.com/thoughtbot/factory_bot) as a test f
resulting record to pass validation.
- When instantiating from a factory, don't supply attributes that aren't
required by the test.
-- Prefer [implicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#implicit-definition)
- or [explicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#explicit-definition)
- association definitions instead of using `create` / `build` for association setup.
+- Prefer [implicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#implicit-definition),
+ [explicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#explicit-definition), or
+ [inline](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#inline-definition) associations
+ over `create` / `build` for association setup in callbacks.
See [issue #262624](https://gitlab.com/gitlab-org/gitlab/-/issues/262624) for further context.
- Factories don't have to be limited to `ActiveRecord` objects.
[See example](https://gitlab.com/gitlab-org/gitlab-foss/commit/0b8cefd3b2385a21cfed779bd659978c0402766d).
diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md
index ee50cfcf182..df0c12c1569 100644
--- a/doc/user/group/import/index.md
+++ b/doc/user/group/import/index.md
@@ -4,7 +4,7 @@ group: Import
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Migrating groups **(FREE)**
+# Migrating groups with GitLab Migration **(FREE)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/249160) in GitLab 13.7 for group resources [with a flag](../../feature_flags.md) named `bulk_import`. Disabled by default.
> - Group items [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/338985) in GitLab 14.3.
@@ -18,15 +18,15 @@ this feature, ask an administrator to [enable the feature flag](../../../adminis
`bulk_import_projects`. On GitLab.com, migrating group resources is available but migrating project resources is not
available.
-Users with the Owner role on a top-level group can migrate it to:
+Users with the Owner role on a top-level group can use GitLab Migration to migrate the group to:
- Another top-level group.
- The subgroup of any existing top-level group.
- Another GitLab instance, including GitLab.com.
-Migrating groups using the method documented here is not the same as [migrating groups using file exports](../settings/import_export.md).
+Migrating groups using GitLab Migration is not the same as [migrating groups using file exports](../settings/import_export.md).
Importing and exporting groups using file exports requires you to export a group to a file and then import that file in
-another GitLab instance. Migrating groups using the method documented here automates this step.
+another GitLab instance. Migrating groups using GitLab Migration automates this step.
## Import your groups into GitLab
diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md
index 86416f439e7..3c12bb9b80f 100644
--- a/doc/user/project/settings/import_export.md
+++ b/doc/user/project/settings/import_export.md
@@ -254,170 +254,3 @@ and the exports between them are compatible.
- [Project import/export administration Rake tasks](../../../administration/raketasks/project_import_export.md)
- [Group import/export](../../group/settings/import_export.md)
- [Group import/export API](../../../api/group_import_export.md)
-
-## Troubleshooting
-
-### Project fails to import due to mismatch
-
-If the [shared runners enablement](../../../ci/runners/runners_scope.md#enable-shared-runners-for-a-project)
-does not match between the exported project, and the project import, the project fails to import.
-Review [issue 276930](https://gitlab.com/gitlab-org/gitlab/-/issues/276930), and either:
-
-- Ensure shared runners are enabled in both the source and destination projects.
-- Disable shared runners on the parent group when you import the project.
-
-### Import workarounds for large repositories
-
-[Maximum import size limitations](#import-a-project-and-its-data)
-can prevent an import from being successful. If changing the import limits is not possible, you can
-try one of the workarounds listed here.
-
-#### Workaround option 1
-
-The following local workflow can be used to temporarily
-reduce the repository size for another import attempt:
-
-1. Create a temporary working directory from the export:
-
- ```shell
- EXPORT=<filename-without-extension>
-
- mkdir "$EXPORT"
- tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/
- cd "$EXPORT"/
- git clone project.bundle
-
- # Prevent interference with recreating an importable file later
- mv project.bundle ../"$EXPORT"-original.bundle
- mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz
-
- git switch --create smaller-tmp-main
- ```
-
-1. To reduce the repository size, work on this `smaller-tmp-main` branch:
- [identify and remove large files](../repository/reducing_the_repo_size_using_git.md)
- or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase)
- to reduce the number of commits.
-
- ```shell
- # Reduce the .git/objects/pack/ file size
- cd project
- git reflog expire --expire=now --all
- git gc --prune=now --aggressive
-
- # Prepare recreating an importable file
- git bundle create ../project.bundle <default-branch-name>
- cd ..
- mv project/ ../"$EXPORT"-project
- cd ..
-
- # Recreate an importable file
- tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ .
- ```
-
-1. Import this new, smaller file into GitLab.
-1. In a full clone of the original repository,
- use `git remote set-url origin <new-url> && git push --force --all`
- to complete the import.
-1. Update the imported repository's
- [branch protection rules](../protected_branches.md) and
- its [default branch](../repository/branches/default.md), and
- delete the temporary, `smaller-tmp-main` branch, and
- the local, temporary data.
-
-#### Workaround option 2
-
-NOTE:
-This workaround does not account for LFS objects.
-
-Rather than attempting to push all changes at once, this workaround:
-
-- Separates the project import from the Git Repository import
-- Incrementally pushes the repository to GitLab
-
-1. Make a local clone of the repository to migrate. In a later step, you push this clone outside of
- the project export.
-1. Download the export and remove the `project.bundle` (which contains the Git repository):
-
- ```shell
- tar -czvf new_export.tar.gz --exclude='project.bundle' @old_export.tar.gz
- ```
-
-1. Import the export without a Git repository. It asks you to confirm to import without a
- repository.
-1. Save this bash script as a file and run it after adding the appropriate origin.
-
- ```shell
- #!/bin/sh
-
- # ASSUMPTIONS:
- # - The GitLab location is "origin"
- # - The default branch is "main"
- # - This will attempt to push in chunks of 500MB (dividing the total size by 500MB).
- # Decrease this size to push in smaller chunks if you still receive timeouts.
-
- git gc
- SIZE=$(git count-objects -v 2> /dev/null | grep size-pack | awk '{print $2}')
-
- # Be conservative... and try to push 2GB at a time
- # (given this assumes each commit is the same size - which is wrong)
- BATCHES=$(($SIZE / 500000))
- TOTAL_COMMITS=$(git rev-list --count HEAD)
- if (( BATCHES > TOTAL_COMMITS )); then
- BATCHES=$TOTAL_COMMITS
- fi
-
- INCREMENTS=$(( ($TOTAL_COMMITS / $BATCHES) - 1 ))
-
- for (( BATCH=BATCHES; BATCH>=1; BATCH-- ))
- do
- COMMIT_NUM=$(( $BATCH - $INCREMENTS ))
- COMMIT_SHA=$(git log -n $COMMIT_NUM --format=format:%H | tail -1)
- git push -u origin ${COMMIT_SHA}:refs/heads/main
- done
- git push -u origin main
- git push -u origin --all
- git push -u origin --tags
- ```
-
-### Manually execute export steps
-
-You usually export a project through [the web interface](#export-a-project-and-its-data) or through [the API](../../../api/project_import_export.md). Exporting using these
-methods can sometimes fail without giving enough information to troubleshoot. In these cases,
-[open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
-Execute each line individually, rather than pasting the entire block at once, so you can see any
-errors each command returns.
-
-```shell
-# User needs to have permission to export
-u = User.find_by_username('someuser')
-p = Project.find_by_full_path('some/project')
-e = Projects::ImportExport::ExportService.new(p,u)
-
-e.send(:version_saver).send(:save)
-e.send(:repo_saver).send(:save)
-## continue using `e.send(:exporter_name).send(:save)` going through the list of exporters
-
-# The following line should show you the export_path similar to /var/opt/gitlab/gitlab-rails/shared/tmp/gitlab_exports/@hashed/49/94/4994....
-s = Gitlab::ImportExport::Saver.new(exportable: p, shared:p.import_export_shared)
-
-# To try and upload use:
-s.send(:compress_and_save)
-s.send(:save_upload)
-```
-
-After the project is successfully uploaded, the exported project is located in a `.tar.gz` file in `/var/opt/gitlab/gitlab-rails/uploads/-/system/import_export_upload/export_file/`.
-
-### Import using the REST API fails when using a group access token
-
-[Group access tokens](../../group/settings/group_access_tokens.md)
-don't work for project or group import operations. When a group access token initiates an import,
-the import fails with this message:
-
-```plaintext
-Error adding importer user to Project members.
-Validation failed: User project bots cannot be added to other groups / projects
-```
-
-To use [Import REST API](../../../api/project_import_export.md),
-pass regular user account credentials such as [personal access tokens](../../profile/personal_access_tokens.md).
diff --git a/doc/user/project/settings/import_export_troubleshooting.md b/doc/user/project/settings/import_export_troubleshooting.md
new file mode 100644
index 00000000000..81ceba7139b
--- /dev/null
+++ b/doc/user/project/settings/import_export_troubleshooting.md
@@ -0,0 +1,280 @@
+---
+stage: Manage
+group: Import
+info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments"
+---
+
+# Troubleshooting file export project migrations
+
+If you have problems with [migrating projects using file exports](import_export.md), see the possible solutions below.
+
+## Troubleshooting commands
+
+Finds information about the status of the import and further logs using the JID:
+
+```ruby
+# Rails console
+Project.find_by_full_path('group/project').import_state.slice(:jid, :status, :last_error)
+> {"jid"=>"414dec93f941a593ea1a6894", "status"=>"finished", "last_error"=>nil}
+```
+
+```shell
+# Logs
+grep JID /var/log/gitlab/sidekiq/current
+grep "Import/Export error" /var/log/gitlab/sidekiq/current
+grep "Import/Export backtrace" /var/log/gitlab/sidekiq/current
+tail /var/log/gitlab/gitlab-rails/importer.log
+```
+
+## Project fails to import due to mismatch
+
+If the [shared runners enablement](../../../ci/runners/runners_scope.md#enable-shared-runners-for-a-project)
+does not match between the exported project, and the project import, the project fails to import.
+Review [issue 276930](https://gitlab.com/gitlab-org/gitlab/-/issues/276930), and either:
+
+- Ensure shared runners are enabled in both the source and destination projects.
+- Disable shared runners on the parent group when you import the project.
+
+## Import workarounds for large repositories
+
+[Maximum import size limitations](import_export.md#import-a-project-and-its-data)
+can prevent an import from being successful. If changing the import limits is not possible, you can
+try one of the workarounds listed here.
+
+### Workaround option 1
+
+The following local workflow can be used to temporarily
+reduce the repository size for another import attempt:
+
+1. Create a temporary working directory from the export:
+
+ ```shell
+ EXPORT=<filename-without-extension>
+
+ mkdir "$EXPORT"
+ tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/
+ cd "$EXPORT"/
+ git clone project.bundle
+
+ # Prevent interference with recreating an importable file later
+ mv project.bundle ../"$EXPORT"-original.bundle
+ mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz
+
+ git switch --create smaller-tmp-main
+ ```
+
+1. To reduce the repository size, work on this `smaller-tmp-main` branch:
+ [identify and remove large files](../repository/reducing_the_repo_size_using_git.md)
+ or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase)
+ to reduce the number of commits.
+
+ ```shell
+ # Reduce the .git/objects/pack/ file size
+ cd project
+ git reflog expire --expire=now --all
+ git gc --prune=now --aggressive
+
+ # Prepare recreating an importable file
+ git bundle create ../project.bundle <default-branch-name>
+ cd ..
+ mv project/ ../"$EXPORT"-project
+ cd ..
+
+ # Recreate an importable file
+ tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ .
+ ```
+
+1. Import this new, smaller file into GitLab.
+1. In a full clone of the original repository,
+ use `git remote set-url origin <new-url> && git push --force --all`
+ to complete the import.
+1. Update the imported repository's
+ [branch protection rules](../protected_branches.md) and
+ its [default branch](../repository/branches/default.md), and
+ delete the temporary, `smaller-tmp-main` branch, and
+ the local, temporary data.
+
+### Workaround option 2
+
+NOTE:
+This workaround does not account for LFS objects.
+
+Rather than attempting to push all changes at once, this workaround:
+
+- Separates the project import from the Git Repository import
+- Incrementally pushes the repository to GitLab
+
+1. Make a local clone of the repository to migrate. In a later step, you push this clone outside of
+ the project export.
+1. Download the export and remove the `project.bundle` (which contains the Git repository):
+
+ ```shell
+ tar -czvf new_export.tar.gz --exclude='project.bundle' @old_export.tar.gz
+ ```
+
+1. Import the export without a Git repository. It asks you to confirm to import without a
+ repository.
+1. Save this bash script as a file and run it after adding the appropriate origin.
+
+ ```shell
+ #!/bin/sh
+
+ # ASSUMPTIONS:
+ # - The GitLab location is "origin"
+ # - The default branch is "main"
+ # - This will attempt to push in chunks of 500MB (dividing the total size by 500MB).
+ # Decrease this size to push in smaller chunks if you still receive timeouts.
+
+ git gc
+ SIZE=$(git count-objects -v 2> /dev/null | grep size-pack | awk '{print $2}')
+
+ # Be conservative... and try to push 2GB at a time
+ # (given this assumes each commit is the same size - which is wrong)
+ BATCHES=$(($SIZE / 500000))
+ TOTAL_COMMITS=$(git rev-list --count HEAD)
+ if (( BATCHES > TOTAL_COMMITS )); then
+ BATCHES=$TOTAL_COMMITS
+ fi
+
+ INCREMENTS=$(( ($TOTAL_COMMITS / $BATCHES) - 1 ))
+
+ for (( BATCH=BATCHES; BATCH>=1; BATCH-- ))
+ do
+ COMMIT_NUM=$(( $BATCH - $INCREMENTS ))
+ COMMIT_SHA=$(git log -n $COMMIT_NUM --format=format:%H | tail -1)
+ git push -u origin ${COMMIT_SHA}:refs/heads/main
+ done
+ git push -u origin main
+ git push -u origin --all
+ git push -u origin --tags
+ ```
+
+## Manually execute export steps
+
+You usually export a project through [the web interface](import_export.md#export-a-project-and-its-data) or through [the API](../../../api/project_import_export.md). Exporting using these
+methods can sometimes fail without giving enough information to troubleshoot. In these cases,
+[open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
+Execute each line individually, rather than pasting the entire block at once, so you can see any
+errors each command returns.
+
+```shell
+# User needs to have permission to export
+u = User.find_by_username('someuser')
+p = Project.find_by_full_path('some/project')
+e = Projects::ImportExport::ExportService.new(p,u)
+
+e.send(:version_saver).send(:save)
+e.send(:repo_saver).send(:save)
+## continue using `e.send(:exporter_name).send(:save)` going through the list of exporters
+
+# The following line should show you the export_path similar to /var/opt/gitlab/gitlab-rails/shared/tmp/gitlab_exports/@hashed/49/94/4994....
+s = Gitlab::ImportExport::Saver.new(exportable: p, shared:p.import_export_shared)
+
+# To try and upload use:
+s.send(:compress_and_save)
+s.send(:save_upload)
+```
+
+After the project is successfully uploaded, the exported project is located in a `.tar.gz` file in `/var/opt/gitlab/gitlab-rails/uploads/-/system/import_export_upload/export_file/`.
+
+## Import using the REST API fails when using a group access token
+
+[Group access tokens](../../group/settings/group_access_tokens.md)
+don't work for project or group import operations. When a group access token initiates an import,
+the import fails with this message:
+
+```plaintext
+Error adding importer user to Project members.
+Validation failed: User project bots cannot be added to other groups / projects
+```
+
+To use [Import REST API](../../../api/project_import_export.md),
+pass regular user account credentials such as [personal access tokens](../../profile/personal_access_tokens.md).
+
+## Troubleshooting performance issues
+
+Read through the current performance problems using the Import/Export below.
+
+### OOM errors
+
+Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](../../../administration/sidekiq/sidekiq_memory_killer.md):
+
+```shell
+SIDEKIQ_MEMORY_KILLER_MAX_RSS = 2000000
+SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS = 3000000
+SIDEKIQ_MEMORY_KILLER_GRACE_TIME = 900
+```
+
+An import status `started`, and the following Sidekiq logs signal a memory issue:
+
+```shell
+WARN: Work still in progress <struct with JID>
+```
+
+### Timeouts
+
+Timeout errors occur due to the `Gitlab::Import::StuckProjectImportJobsWorker` marking the process as failed:
+
+```ruby
+module Gitlab
+ module Import
+ class StuckProjectImportJobsWorker
+ include Gitlab::Import::StuckImportJob
+ # ...
+ end
+ end
+end
+
+module Gitlab
+ module Import
+ module StuckImportJob
+ # ...
+ IMPORT_JOBS_EXPIRATION = 15.hours.to_i
+ # ...
+ def perform
+ stuck_imports_without_jid_count = mark_imports_without_jid_as_failed!
+ stuck_imports_with_jid_count = mark_imports_with_jid_as_failed!
+
+ track_metrics(stuck_imports_with_jid_count, stuck_imports_without_jid_count)
+ end
+ # ...
+ end
+ end
+end
+```
+
+```shell
+Marked stuck import jobs as failed. JIDs: xyz
+```
+
+```plaintext
+ +-----------+ +-----------------------------------+
+ |Export Job |--->| Calls ActiveRecord `as_json` and |
+ +-----------+ | `to_json` on all project models |
+ +-----------------------------------+
+
+ +-----------+ +-----------------------------------+
+ |Import Job |--->| Loads all JSON in memory, then |
+ +-----------+ | inserts into the DB in batches |
+ +-----------------------------------+
+```
+
+### Problems and solutions
+
+| Problem | Possible solutions |
+| -------- | -------- |
+| [Slow JSON](https://gitlab.com/gitlab-org/gitlab/-/issues/25251) loading/dumping models from the database | [split the worker](https://gitlab.com/gitlab-org/gitlab/-/issues/25252) |
+| | Batch export
+| | Optimize SQL
+| | Move away from `ActiveRecord` callbacks (difficult)
+| High memory usage (see also some [analysis](https://gitlab.com/gitlab-org/gitlab/-/issues/18857) | DB Commit sweet spot that uses less memory |
+| | [Netflix Fast JSON API](https://github.com/Netflix/fast_jsonapi) may help |
+| | Batch reading/writing to disk and any SQL
+
+### Temporary solutions
+
+While the performance problems are not tackled, there is a process to workaround
+importing big projects, using a foreground import:
+
+[Foreground import](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/5384) of big projects for customers.
+(Using the import template in the [infrastructure tracker](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues))