diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-11-10 21:11:08 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-11-10 21:11:08 +0300 |
commit | 4e50b9ed316f3c99693a41274babcd67c63ee640 (patch) | |
tree | 21f0f3a7f73a7dd51b06ba62ad9cfa3eaab85c9f /doc | |
parent | 609943de5e2ee6c3bf0f09d3fb1d5fc38ed5a4ed (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r-- | doc/api/graphql/reference/index.md | 1 | ||||
-rw-r--r-- | doc/development/ee_features.md | 15 | ||||
-rw-r--r-- | doc/development/import_export.md | 108 | ||||
-rw-r--r-- | doc/development/performance.md | 2 | ||||
-rw-r--r-- | doc/development/testing_guide/best_practices.md | 7 | ||||
-rw-r--r-- | doc/user/group/import/index.md | 8 | ||||
-rw-r--r-- | doc/user/project/settings/import_export.md | 167 | ||||
-rw-r--r-- | doc/user/project/settings/import_export_troubleshooting.md | 280 |
8 files changed, 302 insertions, 286 deletions
diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index 2750ad6a2cd..834e044110c 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -11061,6 +11061,7 @@ Represents a code quality degradation on the pipeline. | <a id="codequalitydegradationline"></a>`line` | [`Int!`](#int) | Line on which the code quality degradation occurred. | | <a id="codequalitydegradationpath"></a>`path` | [`String!`](#string) | Relative path to the file containing the code quality degradation. | | <a id="codequalitydegradationseverity"></a>`severity` | [`CodeQualityDegradationSeverity!`](#codequalitydegradationseverity) | Status of the degradation (BLOCKER, CRITICAL, MAJOR, MINOR, INFO, UNKNOWN). | +| <a id="codequalitydegradationweburl"></a>`webUrl` | [`String`](#string) | URL to the file along with line number. | ### `Commit` diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index 162b313b531..14df73b8779 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -135,10 +135,17 @@ version of the product: 1. Enable **Allow use of licensed EE features** to make licensed EE features available to projects only if the project namespace's plan includes the feature. - 1. Visit **Admin > Settings > General**. - 1. Expand **Account and limit**. - 1. Select the **Allow use of licensed EE features** checkbox. - 1. Click **Save changes**. + 1. Visit **Admin > Settings > General**. + 1. Expand **Account and limit**. + 1. Select the **Allow use of licensed EE features** checkbox. + 1. Click **Save changes**. + +1. Ensure that the group for which you want to test the EE feature, is actually using an EE plan: + 1. On the top bar, select **Main menu > Admin**. + 1. On the left sidebar, select **Overview > Groups**. + 1. Identify the group you want to modify, and select **Edit**. + 1. Scroll to **Permissions and group features**. For **Plan**, select `Ultimate`. + 1. Select **Save changes**. ### Run CI pipelines in a FOSS context diff --git a/doc/development/import_export.md b/doc/development/import_export.md index f864dd3b678..9aab7f38dbb 100644 --- a/doc/development/import_export.md +++ b/doc/development/import_export.md @@ -6,116 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Import/Export development documentation -Troubleshooting and general development guidelines and tips for the [Import/Export feature](../user/project/settings/import_export.md). +General development guidelines and tips for the [Import/Export feature](../user/project/settings/import_export.md). <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> This document is originally based on the [Import/Export 201 presentation available on YouTube](https://www.youtube.com/watch?v=V3i1OfExotE). -## Troubleshooting commands - -Finds information about the status of the import and further logs using the JID: - -```ruby -# Rails console -Project.find_by_full_path('group/project').import_state.slice(:jid, :status, :last_error) -> {"jid"=>"414dec93f941a593ea1a6894", "status"=>"finished", "last_error"=>nil} -``` - -```shell -# Logs -grep JID /var/log/gitlab/sidekiq/current -grep "Import/Export error" /var/log/gitlab/sidekiq/current -grep "Import/Export backtrace" /var/log/gitlab/sidekiq/current -tail /var/log/gitlab/gitlab-rails/importer.log -``` - -## Troubleshooting performance issues - -Read through the current performance problems using the Import/Export below. - -### OOM errors - -Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](../administration/operations/sidekiq_memory_killer.md): - -```shell -SIDEKIQ_MEMORY_KILLER_MAX_RSS = 2000000 -SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS = 3000000 -SIDEKIQ_MEMORY_KILLER_GRACE_TIME = 900 -``` - -An import status `started`, and the following Sidekiq logs signal a memory issue: - -```shell -WARN: Work still in progress <struct with JID> -``` - -### Timeouts - -Timeout errors occur due to the `Gitlab::Import::StuckProjectImportJobsWorker` marking the process as failed: - -```ruby -module Gitlab - module Import - class StuckProjectImportJobsWorker - include Gitlab::Import::StuckImportJob - # ... - end - end -end - -module Gitlab - module Import - module StuckImportJob - # ... - IMPORT_JOBS_EXPIRATION = 15.hours.to_i - # ... - def perform - stuck_imports_without_jid_count = mark_imports_without_jid_as_failed! - stuck_imports_with_jid_count = mark_imports_with_jid_as_failed! - - track_metrics(stuck_imports_with_jid_count, stuck_imports_without_jid_count) - end - # ... - end - end -end -``` - -```shell -Marked stuck import jobs as failed. JIDs: xyz -``` - -```plaintext - +-----------+ +-----------------------------------+ - |Export Job |--->| Calls ActiveRecord `as_json` and | - +-----------+ | `to_json` on all project models | - +-----------------------------------+ - - +-----------+ +-----------------------------------+ - |Import Job |--->| Loads all JSON in memory, then | - +-----------+ | inserts into the DB in batches | - +-----------------------------------+ -``` - -### Problems and solutions - -| Problem | Possible solutions | -| -------- | -------- | -| [Slow JSON](https://gitlab.com/gitlab-org/gitlab/-/issues/25251) loading/dumping models from the database | [split the worker](https://gitlab.com/gitlab-org/gitlab/-/issues/25252) | -| | Batch export -| | Optimize SQL -| | Move away from `ActiveRecord` callbacks (difficult) -| High memory usage (see also some [analysis](https://gitlab.com/gitlab-org/gitlab/-/issues/18857) | DB Commit sweet spot that uses less memory | -| | [Netflix Fast JSON API](https://github.com/Netflix/fast_jsonapi) may help | -| | Batch reading/writing to disk and any SQL - -### Temporary solutions - -While the performance problems are not tackled, there is a process to workaround -importing big projects, using a foreground import: - -[Foreground import](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/5384) of big projects for customers. -(Using the import template in the [infrastructure tracker](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues)) - ## Security The Import/Export feature is constantly updated (adding new things to export), however diff --git a/doc/development/performance.md b/doc/development/performance.md index e1918bf2e22..3881fad0528 100644 --- a/doc/development/performance.md +++ b/doc/development/performance.md @@ -21,7 +21,7 @@ consistent performance of GitLab. Refer to the [Index](#performance-documentatio - [Query performance guidelines](database/query_performance.md) - [Pagination performance guidelines](../development/database/pagination_performance_guidelines.md) - [Keyset pagination performance](../development/database/keyset_pagination.md#performance) - - [Troubleshooting import/export performance issues](../development/import_export.md#troubleshooting-performance-issues) + - [Troubleshooting import/export performance issues](../user/project/settings/import_export_troubleshooting.md#troubleshooting-performance-issues) - [Pipelines performance in the `gitlab` project](pipelines/performance.md) - Frontend: - [Performance guidelines and monitoring](../development/fe_guide/performance.md) diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 61f60567418..58f6b02c1f1 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -1440,9 +1440,10 @@ GitLab uses [factory_bot](https://github.com/thoughtbot/factory_bot) as a test f resulting record to pass validation. - When instantiating from a factory, don't supply attributes that aren't required by the test. -- Prefer [implicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#implicit-definition) - or [explicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#explicit-definition) - association definitions instead of using `create` / `build` for association setup. +- Prefer [implicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#implicit-definition), + [explicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#explicit-definition), or + [inline](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#inline-definition) associations + over `create` / `build` for association setup in callbacks. See [issue #262624](https://gitlab.com/gitlab-org/gitlab/-/issues/262624) for further context. - Factories don't have to be limited to `ActiveRecord` objects. [See example](https://gitlab.com/gitlab-org/gitlab-foss/commit/0b8cefd3b2385a21cfed779bd659978c0402766d). diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index ee50cfcf182..df0c12c1569 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -4,7 +4,7 @@ group: Import info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Migrating groups **(FREE)** +# Migrating groups with GitLab Migration **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/249160) in GitLab 13.7 for group resources [with a flag](../../feature_flags.md) named `bulk_import`. Disabled by default. > - Group items [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/338985) in GitLab 14.3. @@ -18,15 +18,15 @@ this feature, ask an administrator to [enable the feature flag](../../../adminis `bulk_import_projects`. On GitLab.com, migrating group resources is available but migrating project resources is not available. -Users with the Owner role on a top-level group can migrate it to: +Users with the Owner role on a top-level group can use GitLab Migration to migrate the group to: - Another top-level group. - The subgroup of any existing top-level group. - Another GitLab instance, including GitLab.com. -Migrating groups using the method documented here is not the same as [migrating groups using file exports](../settings/import_export.md). +Migrating groups using GitLab Migration is not the same as [migrating groups using file exports](../settings/import_export.md). Importing and exporting groups using file exports requires you to export a group to a file and then import that file in -another GitLab instance. Migrating groups using the method documented here automates this step. +another GitLab instance. Migrating groups using GitLab Migration automates this step. ## Import your groups into GitLab diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 86416f439e7..3c12bb9b80f 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -254,170 +254,3 @@ and the exports between them are compatible. - [Project import/export administration Rake tasks](../../../administration/raketasks/project_import_export.md) - [Group import/export](../../group/settings/import_export.md) - [Group import/export API](../../../api/group_import_export.md) - -## Troubleshooting - -### Project fails to import due to mismatch - -If the [shared runners enablement](../../../ci/runners/runners_scope.md#enable-shared-runners-for-a-project) -does not match between the exported project, and the project import, the project fails to import. -Review [issue 276930](https://gitlab.com/gitlab-org/gitlab/-/issues/276930), and either: - -- Ensure shared runners are enabled in both the source and destination projects. -- Disable shared runners on the parent group when you import the project. - -### Import workarounds for large repositories - -[Maximum import size limitations](#import-a-project-and-its-data) -can prevent an import from being successful. If changing the import limits is not possible, you can -try one of the workarounds listed here. - -#### Workaround option 1 - -The following local workflow can be used to temporarily -reduce the repository size for another import attempt: - -1. Create a temporary working directory from the export: - - ```shell - EXPORT=<filename-without-extension> - - mkdir "$EXPORT" - tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/ - cd "$EXPORT"/ - git clone project.bundle - - # Prevent interference with recreating an importable file later - mv project.bundle ../"$EXPORT"-original.bundle - mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz - - git switch --create smaller-tmp-main - ``` - -1. To reduce the repository size, work on this `smaller-tmp-main` branch: - [identify and remove large files](../repository/reducing_the_repo_size_using_git.md) - or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase) - to reduce the number of commits. - - ```shell - # Reduce the .git/objects/pack/ file size - cd project - git reflog expire --expire=now --all - git gc --prune=now --aggressive - - # Prepare recreating an importable file - git bundle create ../project.bundle <default-branch-name> - cd .. - mv project/ ../"$EXPORT"-project - cd .. - - # Recreate an importable file - tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ . - ``` - -1. Import this new, smaller file into GitLab. -1. In a full clone of the original repository, - use `git remote set-url origin <new-url> && git push --force --all` - to complete the import. -1. Update the imported repository's - [branch protection rules](../protected_branches.md) and - its [default branch](../repository/branches/default.md), and - delete the temporary, `smaller-tmp-main` branch, and - the local, temporary data. - -#### Workaround option 2 - -NOTE: -This workaround does not account for LFS objects. - -Rather than attempting to push all changes at once, this workaround: - -- Separates the project import from the Git Repository import -- Incrementally pushes the repository to GitLab - -1. Make a local clone of the repository to migrate. In a later step, you push this clone outside of - the project export. -1. Download the export and remove the `project.bundle` (which contains the Git repository): - - ```shell - tar -czvf new_export.tar.gz --exclude='project.bundle' @old_export.tar.gz - ``` - -1. Import the export without a Git repository. It asks you to confirm to import without a - repository. -1. Save this bash script as a file and run it after adding the appropriate origin. - - ```shell - #!/bin/sh - - # ASSUMPTIONS: - # - The GitLab location is "origin" - # - The default branch is "main" - # - This will attempt to push in chunks of 500MB (dividing the total size by 500MB). - # Decrease this size to push in smaller chunks if you still receive timeouts. - - git gc - SIZE=$(git count-objects -v 2> /dev/null | grep size-pack | awk '{print $2}') - - # Be conservative... and try to push 2GB at a time - # (given this assumes each commit is the same size - which is wrong) - BATCHES=$(($SIZE / 500000)) - TOTAL_COMMITS=$(git rev-list --count HEAD) - if (( BATCHES > TOTAL_COMMITS )); then - BATCHES=$TOTAL_COMMITS - fi - - INCREMENTS=$(( ($TOTAL_COMMITS / $BATCHES) - 1 )) - - for (( BATCH=BATCHES; BATCH>=1; BATCH-- )) - do - COMMIT_NUM=$(( $BATCH - $INCREMENTS )) - COMMIT_SHA=$(git log -n $COMMIT_NUM --format=format:%H | tail -1) - git push -u origin ${COMMIT_SHA}:refs/heads/main - done - git push -u origin main - git push -u origin --all - git push -u origin --tags - ``` - -### Manually execute export steps - -You usually export a project through [the web interface](#export-a-project-and-its-data) or through [the API](../../../api/project_import_export.md). Exporting using these -methods can sometimes fail without giving enough information to troubleshoot. In these cases, -[open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session) -Execute each line individually, rather than pasting the entire block at once, so you can see any -errors each command returns. - -```shell -# User needs to have permission to export -u = User.find_by_username('someuser') -p = Project.find_by_full_path('some/project') -e = Projects::ImportExport::ExportService.new(p,u) - -e.send(:version_saver).send(:save) -e.send(:repo_saver).send(:save) -## continue using `e.send(:exporter_name).send(:save)` going through the list of exporters - -# The following line should show you the export_path similar to /var/opt/gitlab/gitlab-rails/shared/tmp/gitlab_exports/@hashed/49/94/4994.... -s = Gitlab::ImportExport::Saver.new(exportable: p, shared:p.import_export_shared) - -# To try and upload use: -s.send(:compress_and_save) -s.send(:save_upload) -``` - -After the project is successfully uploaded, the exported project is located in a `.tar.gz` file in `/var/opt/gitlab/gitlab-rails/uploads/-/system/import_export_upload/export_file/`. - -### Import using the REST API fails when using a group access token - -[Group access tokens](../../group/settings/group_access_tokens.md) -don't work for project or group import operations. When a group access token initiates an import, -the import fails with this message: - -```plaintext -Error adding importer user to Project members. -Validation failed: User project bots cannot be added to other groups / projects -``` - -To use [Import REST API](../../../api/project_import_export.md), -pass regular user account credentials such as [personal access tokens](../../profile/personal_access_tokens.md). diff --git a/doc/user/project/settings/import_export_troubleshooting.md b/doc/user/project/settings/import_export_troubleshooting.md new file mode 100644 index 00000000000..81ceba7139b --- /dev/null +++ b/doc/user/project/settings/import_export_troubleshooting.md @@ -0,0 +1,280 @@ +--- +stage: Manage +group: Import +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +--- + +# Troubleshooting file export project migrations + +If you have problems with [migrating projects using file exports](import_export.md), see the possible solutions below. + +## Troubleshooting commands + +Finds information about the status of the import and further logs using the JID: + +```ruby +# Rails console +Project.find_by_full_path('group/project').import_state.slice(:jid, :status, :last_error) +> {"jid"=>"414dec93f941a593ea1a6894", "status"=>"finished", "last_error"=>nil} +``` + +```shell +# Logs +grep JID /var/log/gitlab/sidekiq/current +grep "Import/Export error" /var/log/gitlab/sidekiq/current +grep "Import/Export backtrace" /var/log/gitlab/sidekiq/current +tail /var/log/gitlab/gitlab-rails/importer.log +``` + +## Project fails to import due to mismatch + +If the [shared runners enablement](../../../ci/runners/runners_scope.md#enable-shared-runners-for-a-project) +does not match between the exported project, and the project import, the project fails to import. +Review [issue 276930](https://gitlab.com/gitlab-org/gitlab/-/issues/276930), and either: + +- Ensure shared runners are enabled in both the source and destination projects. +- Disable shared runners on the parent group when you import the project. + +## Import workarounds for large repositories + +[Maximum import size limitations](import_export.md#import-a-project-and-its-data) +can prevent an import from being successful. If changing the import limits is not possible, you can +try one of the workarounds listed here. + +### Workaround option 1 + +The following local workflow can be used to temporarily +reduce the repository size for another import attempt: + +1. Create a temporary working directory from the export: + + ```shell + EXPORT=<filename-without-extension> + + mkdir "$EXPORT" + tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/ + cd "$EXPORT"/ + git clone project.bundle + + # Prevent interference with recreating an importable file later + mv project.bundle ../"$EXPORT"-original.bundle + mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz + + git switch --create smaller-tmp-main + ``` + +1. To reduce the repository size, work on this `smaller-tmp-main` branch: + [identify and remove large files](../repository/reducing_the_repo_size_using_git.md) + or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase) + to reduce the number of commits. + + ```shell + # Reduce the .git/objects/pack/ file size + cd project + git reflog expire --expire=now --all + git gc --prune=now --aggressive + + # Prepare recreating an importable file + git bundle create ../project.bundle <default-branch-name> + cd .. + mv project/ ../"$EXPORT"-project + cd .. + + # Recreate an importable file + tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ . + ``` + +1. Import this new, smaller file into GitLab. +1. In a full clone of the original repository, + use `git remote set-url origin <new-url> && git push --force --all` + to complete the import. +1. Update the imported repository's + [branch protection rules](../protected_branches.md) and + its [default branch](../repository/branches/default.md), and + delete the temporary, `smaller-tmp-main` branch, and + the local, temporary data. + +### Workaround option 2 + +NOTE: +This workaround does not account for LFS objects. + +Rather than attempting to push all changes at once, this workaround: + +- Separates the project import from the Git Repository import +- Incrementally pushes the repository to GitLab + +1. Make a local clone of the repository to migrate. In a later step, you push this clone outside of + the project export. +1. Download the export and remove the `project.bundle` (which contains the Git repository): + + ```shell + tar -czvf new_export.tar.gz --exclude='project.bundle' @old_export.tar.gz + ``` + +1. Import the export without a Git repository. It asks you to confirm to import without a + repository. +1. Save this bash script as a file and run it after adding the appropriate origin. + + ```shell + #!/bin/sh + + # ASSUMPTIONS: + # - The GitLab location is "origin" + # - The default branch is "main" + # - This will attempt to push in chunks of 500MB (dividing the total size by 500MB). + # Decrease this size to push in smaller chunks if you still receive timeouts. + + git gc + SIZE=$(git count-objects -v 2> /dev/null | grep size-pack | awk '{print $2}') + + # Be conservative... and try to push 2GB at a time + # (given this assumes each commit is the same size - which is wrong) + BATCHES=$(($SIZE / 500000)) + TOTAL_COMMITS=$(git rev-list --count HEAD) + if (( BATCHES > TOTAL_COMMITS )); then + BATCHES=$TOTAL_COMMITS + fi + + INCREMENTS=$(( ($TOTAL_COMMITS / $BATCHES) - 1 )) + + for (( BATCH=BATCHES; BATCH>=1; BATCH-- )) + do + COMMIT_NUM=$(( $BATCH - $INCREMENTS )) + COMMIT_SHA=$(git log -n $COMMIT_NUM --format=format:%H | tail -1) + git push -u origin ${COMMIT_SHA}:refs/heads/main + done + git push -u origin main + git push -u origin --all + git push -u origin --tags + ``` + +## Manually execute export steps + +You usually export a project through [the web interface](import_export.md#export-a-project-and-its-data) or through [the API](../../../api/project_import_export.md). Exporting using these +methods can sometimes fail without giving enough information to troubleshoot. In these cases, +[open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session) +Execute each line individually, rather than pasting the entire block at once, so you can see any +errors each command returns. + +```shell +# User needs to have permission to export +u = User.find_by_username('someuser') +p = Project.find_by_full_path('some/project') +e = Projects::ImportExport::ExportService.new(p,u) + +e.send(:version_saver).send(:save) +e.send(:repo_saver).send(:save) +## continue using `e.send(:exporter_name).send(:save)` going through the list of exporters + +# The following line should show you the export_path similar to /var/opt/gitlab/gitlab-rails/shared/tmp/gitlab_exports/@hashed/49/94/4994.... +s = Gitlab::ImportExport::Saver.new(exportable: p, shared:p.import_export_shared) + +# To try and upload use: +s.send(:compress_and_save) +s.send(:save_upload) +``` + +After the project is successfully uploaded, the exported project is located in a `.tar.gz` file in `/var/opt/gitlab/gitlab-rails/uploads/-/system/import_export_upload/export_file/`. + +## Import using the REST API fails when using a group access token + +[Group access tokens](../../group/settings/group_access_tokens.md) +don't work for project or group import operations. When a group access token initiates an import, +the import fails with this message: + +```plaintext +Error adding importer user to Project members. +Validation failed: User project bots cannot be added to other groups / projects +``` + +To use [Import REST API](../../../api/project_import_export.md), +pass regular user account credentials such as [personal access tokens](../../profile/personal_access_tokens.md). + +## Troubleshooting performance issues + +Read through the current performance problems using the Import/Export below. + +### OOM errors + +Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](../../../administration/sidekiq/sidekiq_memory_killer.md): + +```shell +SIDEKIQ_MEMORY_KILLER_MAX_RSS = 2000000 +SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS = 3000000 +SIDEKIQ_MEMORY_KILLER_GRACE_TIME = 900 +``` + +An import status `started`, and the following Sidekiq logs signal a memory issue: + +```shell +WARN: Work still in progress <struct with JID> +``` + +### Timeouts + +Timeout errors occur due to the `Gitlab::Import::StuckProjectImportJobsWorker` marking the process as failed: + +```ruby +module Gitlab + module Import + class StuckProjectImportJobsWorker + include Gitlab::Import::StuckImportJob + # ... + end + end +end + +module Gitlab + module Import + module StuckImportJob + # ... + IMPORT_JOBS_EXPIRATION = 15.hours.to_i + # ... + def perform + stuck_imports_without_jid_count = mark_imports_without_jid_as_failed! + stuck_imports_with_jid_count = mark_imports_with_jid_as_failed! + + track_metrics(stuck_imports_with_jid_count, stuck_imports_without_jid_count) + end + # ... + end + end +end +``` + +```shell +Marked stuck import jobs as failed. JIDs: xyz +``` + +```plaintext + +-----------+ +-----------------------------------+ + |Export Job |--->| Calls ActiveRecord `as_json` and | + +-----------+ | `to_json` on all project models | + +-----------------------------------+ + + +-----------+ +-----------------------------------+ + |Import Job |--->| Loads all JSON in memory, then | + +-----------+ | inserts into the DB in batches | + +-----------------------------------+ +``` + +### Problems and solutions + +| Problem | Possible solutions | +| -------- | -------- | +| [Slow JSON](https://gitlab.com/gitlab-org/gitlab/-/issues/25251) loading/dumping models from the database | [split the worker](https://gitlab.com/gitlab-org/gitlab/-/issues/25252) | +| | Batch export +| | Optimize SQL +| | Move away from `ActiveRecord` callbacks (difficult) +| High memory usage (see also some [analysis](https://gitlab.com/gitlab-org/gitlab/-/issues/18857) | DB Commit sweet spot that uses less memory | +| | [Netflix Fast JSON API](https://github.com/Netflix/fast_jsonapi) may help | +| | Batch reading/writing to disk and any SQL + +### Temporary solutions + +While the performance problems are not tackled, there is a process to workaround +importing big projects, using a foreground import: + +[Foreground import](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/5384) of big projects for customers. +(Using the import template in the [infrastructure tracker](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues)) |