Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2022-11-17 14:33:21 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2022-11-17 14:33:21 +0300
commit7021455bd1ed7b125c55eb1b33c5a01f2bc55ee0 (patch)
tree5bdc2229f5198d516781f8d24eace62fc7e589e9 /doc
parent185b095e93520f96e9cfc31d9c3e69b498cdab7c (diff)
Add latest changes from gitlab-org/gitlab@15-6-stable-eev15.6.0-rc42
Diffstat (limited to 'doc')
-rw-r--r--doc/.markdownlint/require_helper.js14
-rw-r--r--doc/.markdownlint/rules/tabs_blank_lines.js26
-rw-r--r--doc/.markdownlint/rules/tabs_title_markup.js31
-rw-r--r--doc/.markdownlint/rules/tabs_title_text.js23
-rw-r--r--doc/.markdownlint/rules/tabs_wrapper_tags.js21
-rw-r--r--doc/.vale/gitlab/Admin.yml13
-rw-r--r--doc/.vale/gitlab/AlertBoxStyle.yml8
-rw-r--r--doc/.vale/gitlab/BadPlurals.yml6
-rw-r--r--doc/.vale/gitlab/BadgeCapitalization.yml4
-rw-r--r--doc/.vale/gitlab/British.yml4
-rw-r--r--doc/.vale/gitlab/CIConfigFile.yml4
-rw-r--r--doc/.vale/gitlab/CodeblockFences.yml4
-rw-r--r--doc/.vale/gitlab/CurlStringsQuoted.yml4
-rw-r--r--doc/.vale/gitlab/CurrentStatus.yml6
-rw-r--r--doc/.vale/gitlab/DefaultBranch.yml8
-rw-r--r--doc/.vale/gitlab/Dropdown.yml6
-rw-r--r--doc/.vale/gitlab/EOLWhitespace.yml4
-rw-r--r--doc/.vale/gitlab/ElementDescriptors.yml4
-rw-r--r--doc/.vale/gitlab/FirstPerson.yml6
-rw-r--r--doc/.vale/gitlab/FutureTense.yml6
-rw-r--r--doc/.vale/gitlab/HeadingContent.yml6
-rw-r--r--doc/.vale/gitlab/HeadingDepth.yml2
-rw-r--r--doc/.vale/gitlab/InclusionAbleism.yml6
-rw-r--r--doc/.vale/gitlab/InclusionCultural.yml6
-rw-r--r--doc/.vale/gitlab/InclusionGender.yml6
-rw-r--r--doc/.vale/gitlab/InternalLinkCase.yml6
-rw-r--r--doc/.vale/gitlab/InternalLinkExtension.yml6
-rw-r--r--doc/.vale/gitlab/InternalLinkFormat.yml10
-rw-r--r--doc/.vale/gitlab/LatinTerms.yml6
-rw-r--r--doc/.vale/gitlab/Markdown_emoji.yml4
-rw-r--r--doc/.vale/gitlab/MeaningfulLinkWords.yml6
-rw-r--r--doc/.vale/gitlab/MergeConflictMarkers.yml4
-rw-r--r--doc/.vale/gitlab/MultiLineLinks.yml6
-rw-r--r--doc/.vale/gitlab/NonStandardQuotes.yml6
-rw-r--r--doc/.vale/gitlab/OutdatedVersions.yml6
-rw-r--r--doc/.vale/gitlab/OxfordComma.yml4
-rw-r--r--doc/.vale/gitlab/Possessive.yml6
-rw-r--r--doc/.vale/gitlab/ReadingLevel.yml4
-rw-r--r--doc/.vale/gitlab/ReferenceLinks.yml6
-rw-r--r--doc/.vale/gitlab/RelativeLinks.yml8
-rw-r--r--doc/.vale/gitlab/RelativeLinksDoubleSlashes.yml6
-rw-r--r--doc/.vale/gitlab/Repetition.yml4
-rw-r--r--doc/.vale/gitlab/SentenceLength.yml4
-rw-r--r--doc/.vale/gitlab/SentenceSpacing.yml4
-rw-r--r--doc/.vale/gitlab/Simplicity.yml6
-rw-r--r--doc/.vale/gitlab/Spelling.yml4
-rw-r--r--doc/.vale/gitlab/SubstitutionSuggestions.yml33
-rw-r--r--doc/.vale/gitlab/SubstitutionWarning.yml36
-rw-r--r--doc/.vale/gitlab/Substitutions.yml4
-rw-r--r--doc/.vale/gitlab/ToDo.yml6
-rw-r--r--doc/.vale/gitlab/UnclearAntecedent.yml4
-rw-r--r--doc/.vale/gitlab/Units.yml15
-rw-r--r--doc/.vale/gitlab/Uppercase.yml6
-rw-r--r--doc/.vale/gitlab/VersionText.yml4
-rw-r--r--doc/.vale/gitlab/VersionTextSingleLine.yml4
-rw-r--r--doc/.vale/gitlab/Wordy.yml14
-rw-r--r--doc/.vale/gitlab/spelling-exceptions.txt3
-rw-r--r--doc/.vale/vale-json.tmpl65
-rw-r--r--doc/administration/audit_event_streaming.md5
-rw-r--r--doc/administration/audit_events.md41
-rw-r--r--doc/administration/auth/authentiq.md2
-rw-r--r--doc/administration/auth/jwt.md2
-rw-r--r--doc/administration/auth/ldap/google_secure_ldap.md2
-rw-r--r--doc/administration/auth/ldap/index.md6
-rw-r--r--doc/administration/auth/ldap/ldap-troubleshooting.md18
-rw-r--r--doc/administration/auth/ldap/ldap_synchronization.md10
-rw-r--r--doc/administration/auth/oidc.md4
-rw-r--r--doc/administration/auth/smartcard.md2
-rw-r--r--doc/administration/cicd.md2
-rw-r--r--doc/administration/compliance.md4
-rw-r--r--doc/administration/configure.md6
-rw-r--r--doc/administration/consul.md10
-rw-r--r--doc/administration/environment_variables.md2
-rw-r--r--doc/administration/feature_flags.md4
-rw-r--r--doc/administration/file_hooks.md6
-rw-r--r--doc/administration/geo/disaster_recovery/background_verification.md2
-rw-r--r--doc/administration/geo/disaster_recovery/index.md2
-rw-r--r--doc/administration/geo/disaster_recovery/planned_failover.md4
-rw-r--r--doc/administration/geo/index.md10
-rw-r--r--doc/administration/geo/replication/configuration.md7
-rw-r--r--doc/administration/geo/replication/datatypes.md3
-rw-r--r--doc/administration/geo/replication/disable_geo.md4
-rw-r--r--doc/administration/geo/replication/docker_registry.md11
-rw-r--r--doc/administration/geo/replication/faq.md10
-rw-r--r--doc/administration/geo/replication/geo_validation_tests.md4
-rw-r--r--doc/administration/geo/replication/location_aware_git_url.md2
-rw-r--r--doc/administration/geo/replication/remove_geo_site.md5
-rw-r--r--doc/administration/geo/replication/security_review.md10
-rw-r--r--doc/administration/geo/replication/troubleshooting.md429
-rw-r--r--doc/administration/geo/secondary_proxy/location_aware_external_url.md37
-rw-r--r--doc/administration/geo/setup/database.md14
-rw-r--r--doc/administration/geo/setup/external_database.md4
-rw-r--r--doc/administration/git_protocol.md4
-rw-r--r--doc/administration/gitaly/configure_gitaly.md67
-rw-r--r--doc/administration/gitaly/index.md56
-rw-r--r--doc/administration/gitaly/monitoring.md3
-rw-r--r--doc/administration/gitaly/praefect.md40
-rw-r--r--doc/administration/gitaly/recovery.md62
-rw-r--r--doc/administration/gitaly/reference.md3
-rw-r--r--doc/administration/gitaly/troubleshooting.md30
-rw-r--r--doc/administration/housekeeping.md229
-rw-r--r--doc/administration/incoming_email.md2
-rw-r--r--doc/administration/index.md5
-rw-r--r--doc/administration/instance_limits.md20
-rw-r--r--doc/administration/integration/kroki.md2
-rw-r--r--doc/administration/integration/mailgun.md2
-rw-r--r--doc/administration/issue_closing_pattern.md2
-rw-r--r--doc/administration/job_artifacts.md127
-rw-r--r--doc/administration/lfs/index.md5
-rw-r--r--doc/administration/libravatar.md2
-rw-r--r--doc/administration/logs.md11
-rw-r--r--doc/administration/logs/index.md42
-rw-r--r--doc/administration/logs/tracing_correlation_id.md2
-rw-r--r--doc/administration/maintenance_mode/index.md6
-rw-r--r--doc/administration/monitoring/gitlab_self_monitoring_project/index.md30
-rw-r--r--doc/administration/monitoring/ip_allowlist.md3
-rw-r--r--doc/administration/monitoring/performance/grafana_configuration.md6
-rw-r--r--doc/administration/monitoring/performance/index.md2
-rw-r--r--doc/administration/monitoring/prometheus/gitlab_metrics.md19
-rw-r--r--doc/administration/nfs.md10
-rw-r--r--doc/administration/object_storage.md10
-rw-r--r--doc/administration/operations/moving_repositories.md11
-rw-r--r--doc/administration/operations/rails_console.md36
-rw-r--r--doc/administration/package_information/defaults.md4
-rw-r--r--doc/administration/package_information/omnibus_packages.md2
-rw-r--r--doc/administration/package_information/postgresql_versions.md1
-rw-r--r--doc/administration/package_information/signed_packages.md2
-rw-r--r--doc/administration/packages/container_registry.md7
-rw-r--r--doc/administration/packages/dependency_proxy.md9
-rw-r--r--doc/administration/packages/index.md5
-rw-r--r--doc/administration/pages/index.md12
-rw-r--r--doc/administration/pages/source.md2
-rw-r--r--doc/administration/polling.md4
-rw-r--r--doc/administration/postgresql/database_load_balancing.md1
-rw-r--r--doc/administration/postgresql/replication_and_failover.md4
-rw-r--r--doc/administration/raketasks/uploads/migrate.md5
-rw-r--r--doc/administration/redis/replication_and_failover.md4
-rw-r--r--doc/administration/redis/replication_and_failover_external.md2
-rw-r--r--doc/administration/redis/troubleshooting.md4
-rw-r--r--doc/administration/reference_architectures/10k_users.md178
-rw-r--r--doc/administration/reference_architectures/1k_users.md15
-rw-r--r--doc/administration/reference_architectures/25k_users.md165
-rw-r--r--doc/administration/reference_architectures/2k_users.md195
-rw-r--r--doc/administration/reference_architectures/3k_users.md160
-rw-r--r--doc/administration/reference_architectures/50k_users.md168
-rw-r--r--doc/administration/reference_architectures/5k_users.md169
-rw-r--r--doc/administration/reference_architectures/index.md17
-rw-r--r--doc/administration/reference_architectures/troubleshooting.md11
-rw-r--r--doc/administration/repository_storage_paths.md4
-rw-r--r--doc/administration/server_hooks.md14
-rw-r--r--doc/administration/sidekiq/sidekiq_memory_killer.md2
-rw-r--r--doc/administration/snippets/index.md2
-rw-r--r--doc/administration/system_hooks.md28
-rw-r--r--doc/administration/terraform_state.md5
-rw-r--r--doc/administration/troubleshooting/debug.md11
-rw-r--r--doc/administration/troubleshooting/defcon.md11
-rw-r--r--doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md663
-rw-r--r--doc/administration/troubleshooting/group_saml_scim.md11
-rw-r--r--doc/administration/troubleshooting/index.md8
-rw-r--r--doc/administration/troubleshooting/kubernetes_cheat_sheet.md11
-rw-r--r--doc/administration/troubleshooting/linux_cheat_sheet.md4
-rw-r--r--doc/administration/troubleshooting/log_parsing.md11
-rw-r--r--doc/administration/troubleshooting/navigating_gitlab_via_rails_console.md11
-rw-r--r--doc/administration/troubleshooting/postgresql.md146
-rw-r--r--doc/administration/troubleshooting/ssl.md266
-rw-r--r--doc/administration/troubleshooting/test_environments.md4
-rw-r--r--doc/administration/user_settings.md2
-rw-r--r--doc/api/award_emoji.md10
-rw-r--r--doc/api/broadcast_messages.md13
-rw-r--r--doc/api/commits.md72
-rw-r--r--doc/api/container_registry.md2
-rw-r--r--doc/api/dependency_proxy.md2
-rw-r--r--doc/api/deploy_tokens.md2
-rw-r--r--doc/api/deployments.md13
-rw-r--r--doc/api/environments.md4
-rw-r--r--doc/api/error_tracking.md2
-rw-r--r--doc/api/events.md2
-rw-r--r--doc/api/feature_flag_specs.md6
-rw-r--r--doc/api/feature_flag_user_lists.md8
-rw-r--r--doc/api/feature_flags.md8
-rw-r--r--doc/api/features.md25
-rw-r--r--doc/api/freeze_periods.md34
-rw-r--r--doc/api/geo_nodes.md39
-rw-r--r--doc/api/graphql/custom_emoji.md43
-rw-r--r--doc/api/graphql/getting_started.md22
-rw-r--r--doc/api/graphql/index.md28
-rw-r--r--doc/api/graphql/reference/index.md880
-rw-r--r--doc/api/group_iterations.md2
-rw-r--r--doc/api/group_level_variables.md7
-rw-r--r--doc/api/group_protected_environments.md6
-rw-r--r--doc/api/groups.md4
-rw-r--r--doc/api/index.md2
-rw-r--r--doc/api/instance_level_ci_variables.md17
-rw-r--r--doc/api/integrations.md2
-rw-r--r--doc/api/issues.md4
-rw-r--r--doc/api/iterations.md2
-rw-r--r--doc/api/job_artifacts.md4
-rw-r--r--doc/api/lint.md2
-rw-r--r--doc/api/members.md3
-rw-r--r--doc/api/merge_request_approvals.md17
-rw-r--r--doc/api/merge_requests.md136
-rw-r--r--doc/api/merge_trains.md70
-rw-r--r--doc/api/metadata.md4
-rw-r--r--doc/api/milestones.md8
-rw-r--r--doc/api/oauth2.md2
-rw-r--r--doc/api/openapi/openapi.yaml642
-rw-r--r--doc/api/openapi/openapi_v2.yaml343
-rw-r--r--doc/api/openapi/v4/access_requests.yaml381
-rw-r--r--doc/api/openapi/v4/access_tokens.yaml170
-rw-r--r--doc/api/openapi/v4/metadata.yaml43
-rw-r--r--doc/api/openapi/v4/version.yaml28
-rw-r--r--doc/api/packages.md2
-rw-r--r--doc/api/packages/composer.md2
-rw-r--r--doc/api/packages/conan.md14
-rw-r--r--doc/api/packages/debian.md4
-rw-r--r--doc/api/packages/debian_group_distributions.md2
-rw-r--r--doc/api/packages/debian_project_distributions.md2
-rw-r--r--doc/api/packages/go_proxy.md2
-rw-r--r--doc/api/packages/helm.md2
-rw-r--r--doc/api/packages/maven.md2
-rw-r--r--doc/api/packages/npm.md6
-rw-r--r--doc/api/packages/nuget.md10
-rw-r--r--doc/api/packages/pypi.md8
-rw-r--r--doc/api/packages/rubygems.md2
-rw-r--r--doc/api/packages/terraform-modules.md2
-rw-r--r--doc/api/pipeline_triggers.md2
-rw-r--r--doc/api/pipelines.md61
-rw-r--r--doc/api/product_analytics.md2
-rw-r--r--doc/api/project_level_variables.md7
-rw-r--r--doc/api/projects.md46
-rw-r--r--doc/api/protected_branches.md124
-rw-r--r--doc/api/protected_environments.md4
-rw-r--r--doc/api/protected_tags.md3
-rw-r--r--doc/api/releases/index.md65
-rw-r--r--doc/api/releases/links.md20
-rw-r--r--doc/api/remote_mirrors.md31
-rw-r--r--doc/api/repositories.md2
-rw-r--r--doc/api/repository_files.md104
-rw-r--r--doc/api/runners.md101
-rw-r--r--doc/api/scim.md10
-rw-r--r--doc/api/secure_files.md34
-rw-r--r--doc/api/settings.md6
-rw-r--r--doc/api/status_checks.md10
-rw-r--r--doc/api/suggestions.md54
-rw-r--r--doc/api/templates/dockerfiles.md2
-rw-r--r--doc/api/templates/gitignores.md2
-rw-r--r--doc/api/templates/gitlab_ci_ymls.md2
-rw-r--r--doc/api/templates/licenses.md2
-rw-r--r--doc/api/users.md40
-rw-r--r--doc/api/vulnerability_exports.md21
-rw-r--r--doc/architecture/blueprints/_template.md7
-rw-r--r--doc/architecture/blueprints/ci_data_decay/index.md60
-rw-r--r--doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md48
-rw-r--r--doc/architecture/blueprints/ci_pipeline_components/index.md58
-rw-r--r--doc/architecture/blueprints/cloud_native_build_logs/index.md38
-rw-r--r--doc/architecture/blueprints/cloud_native_gitlab_pages/index.md47
-rw-r--r--doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md46
-rw-r--r--doc/architecture/blueprints/consolidating_groups_and_projects/index.md40
-rw-r--r--doc/architecture/blueprints/container_registry_metadata_database/index.md33
-rw-r--r--doc/architecture/blueprints/database/scalability/patterns/read_mostly.md2
-rw-r--r--doc/architecture/blueprints/database/scalability/patterns/time_decay.md2
-rw-r--r--doc/architecture/blueprints/database_scaling/size-limits.md2
-rw-r--r--doc/architecture/blueprints/database_testing/index.md35
-rw-r--r--doc/architecture/blueprints/feature_flags_development/index.md35
-rw-r--r--doc/architecture/blueprints/gitlab_to_kubernetes_communication/index.md37
-rw-r--r--doc/architecture/blueprints/graphql_api/index.md51
-rw-r--r--doc/architecture/blueprints/image_resizing/index.md12
-rw-r--r--doc/architecture/blueprints/object_storage/index.md32
-rw-r--r--doc/architecture/blueprints/pods/images/iteration0-organizations-introduction.pngbin0 -> 67160 bytes
-rw-r--r--doc/architecture/blueprints/pods/images/pods-and-fulfillment.pngbin0 -> 75803 bytes
-rw-r--r--doc/architecture/blueprints/pods/images/term-cluster.pngbin0 -> 63268 bytes
-rw-r--r--doc/architecture/blueprints/pods/images/term-organization.pngbin0 -> 7150 bytes
-rw-r--r--doc/architecture/blueprints/pods/images/term-pod.png (renamed from doc/architecture/blueprints/pods/term-pod.png)bin16104 -> 16104 bytes
-rw-r--r--doc/architecture/blueprints/pods/images/term-top-level-namespace.png (renamed from doc/architecture/blueprints/pods/term-top-level-namespace.png)bin11451 -> 11451 bytes
-rw-r--r--doc/architecture/blueprints/pods/index.md122
-rw-r--r--doc/architecture/blueprints/pods/iteration0-organizations-introduction.pngbin326285 -> 0 bytes
-rw-r--r--doc/architecture/blueprints/pods/pods-feature-data-migration.md82
-rw-r--r--doc/architecture/blueprints/pods/pods-feature-database-sequences.md94
-rw-r--r--doc/architecture/blueprints/pods/pods-feature-git-access.md163
-rw-r--r--doc/architecture/blueprints/pods/pods-feature-graphql.md94
-rw-r--r--doc/architecture/blueprints/pods/pods-feature-organizations.md58
-rw-r--r--doc/architecture/blueprints/pods/pods-feature-router-endpoints-classification.md46
-rw-r--r--doc/architecture/blueprints/pods/pods-feature-template.md29
-rw-r--r--doc/architecture/blueprints/pods/proposal-stateless-router-with-buffering-requests.md648
-rw-r--r--doc/architecture/blueprints/pods/proposal-stateless-router-with-routes-learning.md672
-rw-r--r--doc/architecture/blueprints/pods/term-cluster.pngbin271291 -> 0 bytes
-rw-r--r--doc/architecture/blueprints/pods/term-organization.pngbin22575 -> 0 bytes
-rw-r--r--doc/architecture/blueprints/rate_limiting/index.md84
-rw-r--r--doc/architecture/blueprints/runner_scaling/index.md47
-rw-r--r--doc/architecture/blueprints/runner_tokens/index.md227
-rw-r--r--doc/architecture/blueprints/work_items/index.md66
-rw-r--r--doc/architecture/index.md6
-rw-r--r--doc/ci/chatops/index.md2
-rw-r--r--doc/ci/ci_cd_for_external_repos/bitbucket_integration.md27
-rw-r--r--doc/ci/ci_cd_for_external_repos/github_integration.md2
-rw-r--r--doc/ci/cloud_deployment/index.md2
-rw-r--r--doc/ci/cloud_services/azure/index.md20
-rw-r--r--doc/ci/directed_acyclic_graph/index.md2
-rw-r--r--doc/ci/docker/using_docker_build.md4
-rw-r--r--doc/ci/docker/using_docker_images.md2
-rw-r--r--doc/ci/enable_or_disable_ci.md2
-rw-r--r--doc/ci/environments/deployment_approvals.md19
-rw-r--r--doc/ci/environments/deployment_safety.md64
-rw-r--r--doc/ci/environments/environments_dashboard.md2
-rw-r--r--doc/ci/environments/incremental_rollouts.md2
-rw-r--r--doc/ci/environments/index.md15
-rw-r--r--doc/ci/environments/protected_environments.md4
-rw-r--r--doc/ci/examples/authenticating-with-hashicorp-vault/index.md32
-rw-r--r--doc/ci/examples/deployment/composer-npm-deploy.md2
-rw-r--r--doc/ci/examples/end_to_end_testing_webdriverio/index.md2
-rw-r--r--doc/ci/examples/laravel_with_gitlab_and_envoy/index.md8
-rw-r--r--doc/ci/examples/semantic-release.md2
-rw-r--r--doc/ci/git_submodules.md15
-rw-r--r--doc/ci/interactive_web_terminal/index.md2
-rw-r--r--doc/ci/jobs/ci_job_token.md4
-rw-r--r--doc/ci/jobs/index.md4
-rw-r--r--doc/ci/jobs/job_control.md31
-rw-r--r--doc/ci/migration/circleci.md2
-rw-r--r--doc/ci/migration/jenkins.md8
-rw-r--r--doc/ci/pipelines/cicd_minutes.md12
-rw-r--r--doc/ci/pipelines/downstream_pipelines.md370
-rw-r--r--doc/ci/pipelines/img/merge_train_cancel_v12_0.pngbin7654 -> 0 bytes
-rw-r--r--doc/ci/pipelines/img/merge_train_failure.pngbin23899 -> 0 bytes
-rw-r--r--doc/ci/pipelines/img/merge_train_immediate_merge_v12_6.pngbin18607 -> 0 bytes
-rw-r--r--doc/ci/pipelines/img/merge_train_position_v12_0.pngbin6152 -> 0 bytes
-rw-r--r--doc/ci/pipelines/img/merge_train_start_v12_0.pngbin3548 -> 0 bytes
-rw-r--r--doc/ci/pipelines/img/merge_train_start_when_pipeline_succeeds_v12_0.pngbin4337 -> 0 bytes
-rw-r--r--doc/ci/pipelines/index.md9
-rw-r--r--doc/ci/pipelines/job_artifacts.md70
-rw-r--r--doc/ci/pipelines/merge_trains.md306
-rw-r--r--doc/ci/pipelines/pipeline_architectures.md7
-rw-r--r--doc/ci/pipelines/pipeline_efficiency.md2
-rw-r--r--doc/ci/pipelines/schedules.md2
-rw-r--r--doc/ci/pipelines/settings.md86
-rw-r--r--doc/ci/quick_start/index.md147
-rw-r--r--doc/ci/resource_groups/index.md3
-rw-r--r--doc/ci/review_apps/index.md70
-rw-r--r--doc/ci/runners/configure_runners.md57
-rw-r--r--doc/ci/runners/index.md2
-rw-r--r--doc/ci/runners/runners_scope.md19
-rw-r--r--doc/ci/runners/saas/linux_saas_runner.md2
-rw-r--r--doc/ci/runners/saas/macos_saas_runner.md2
-rw-r--r--doc/ci/secrets/index.md12
-rw-r--r--doc/ci/services/index.md42
-rw-r--r--doc/ci/testing/code_quality.md14
-rw-r--r--doc/ci/testing/load_performance_testing.md2
-rw-r--r--doc/ci/testing/test_coverage_visualization.md2
-rw-r--r--doc/ci/testing/unit_test_report_examples.md3
-rw-r--r--doc/ci/triggers/index.md8
-rw-r--r--doc/ci/troubleshooting.md83
-rw-r--r--doc/ci/variables/index.md111
-rw-r--r--doc/ci/variables/predefined_variables.md2
-rw-r--r--doc/ci/variables/where_variables_can_be_used.md4
-rw-r--r--doc/ci/yaml/artifacts_reports.md2
-rw-r--r--doc/ci/yaml/index.md149
-rw-r--r--doc/cloud_seed/index.md3
-rw-r--r--doc/development/api_graphql_styleguide.md65
-rw-r--r--doc/development/application_limits.md2
-rw-r--r--doc/development/audit_event_guide/index.md12
-rw-r--r--doc/development/backend/ruby_style_guide.md87
-rw-r--r--doc/development/cached_queries.md2
-rw-r--r--doc/development/cascading_settings.md2
-rw-r--r--doc/development/changelog.md4
-rw-r--r--doc/development/chatops_on_gitlabcom.md4
-rw-r--r--doc/development/cicd/index.md2
-rw-r--r--doc/development/cicd/templates.md2
-rw-r--r--doc/development/code_review.md24
-rw-r--r--doc/development/contributing/design.md4
-rw-r--r--doc/development/contributing/index.md2
-rw-r--r--doc/development/contributing/merge_request_workflow.md4
-rw-r--r--doc/development/contributing/style_guides.md115
-rw-r--r--doc/development/database/adding_database_indexes.md36
-rw-r--r--doc/development/database/background_migrations.md4
-rw-r--r--doc/development/database/batched_background_migrations.md13
-rw-r--r--doc/development/database/database_debugging.md8
-rw-r--r--doc/development/database/database_migration_pipeline.md52
-rw-r--r--doc/development/database/loose_foreign_keys.md8
-rw-r--r--doc/development/database/not_null_constraints.md10
-rw-r--r--doc/development/database/query_recorder.md9
-rw-r--r--doc/development/database/single_table_inheritance.md2
-rw-r--r--doc/development/database/table_partitioning.md19
-rw-r--r--doc/development/database_review.md13
-rw-r--r--doc/development/deprecation_guidelines/index.md6
-rw-r--r--doc/development/development_processes.md2
-rw-r--r--doc/development/distributed_tracing.md2
-rw-r--r--doc/development/documentation/drawers.md59
-rw-r--r--doc/development/documentation/feature_flags.md7
-rw-r--r--doc/development/documentation/index.md67
-rw-r--r--doc/development/documentation/restful_api_styleguide.md4
-rw-r--r--doc/development/documentation/review_apps.md55
-rw-r--r--doc/development/documentation/site_architecture/deployment_process.md16
-rw-r--r--doc/development/documentation/site_architecture/global_nav.md2
-rw-r--r--doc/development/documentation/styleguide/index.md113
-rw-r--r--doc/development/documentation/styleguide/word_list.md122
-rw-r--r--doc/development/documentation/testing.md4
-rw-r--r--doc/development/documentation/topic_types/concept.md6
-rw-r--r--doc/development/documentation/topic_types/index.md69
-rw-r--r--doc/development/documentation/topic_types/reference.md6
-rw-r--r--doc/development/documentation/topic_types/task.md13
-rw-r--r--doc/development/documentation/topic_types/troubleshooting.md19
-rw-r--r--doc/development/documentation/topic_types/tutorial.md102
-rw-r--r--doc/development/documentation/versions.md2
-rw-r--r--doc/development/documentation/workflow.md2
-rw-r--r--doc/development/ee_features.md38
-rw-r--r--doc/development/event_store.md6
-rw-r--r--doc/development/fe_guide/graphql.md6
-rw-r--r--doc/development/fe_guide/merge_request_widget_extensions.md2
-rw-r--r--doc/development/fe_guide/registry_architecture.md2
-rw-r--r--doc/development/fe_guide/style/scss.md2
-rw-r--r--doc/development/fe_guide/view_component.md10
-rw-r--r--doc/development/fe_guide/vue.md3
-rw-r--r--doc/development/fe_guide/vue3_migration.md229
-rw-r--r--doc/development/fe_guide/vuex.md2
-rw-r--r--doc/development/feature_development.md1
-rw-r--r--doc/development/feature_flags/controls.md6
-rw-r--r--doc/development/feature_flags/index.md2
-rw-r--r--doc/development/fips_compliance.md4
-rw-r--r--doc/development/gemfile.md2
-rw-r--r--doc/development/github_importer.md23
-rw-r--r--doc/development/gitlab_flavored_markdown/specification_guide/index.md267
-rw-r--r--doc/development/go_guide/index.md2
-rw-r--r--doc/development/graphql_guide/pagination.md4
-rw-r--r--doc/development/i18n/externalization.md30
-rw-r--r--doc/development/i18n/merging_translations.md2
-rw-r--r--doc/development/i18n/proofreader.md1
-rw-r--r--doc/development/import_export.md108
-rw-r--r--doc/development/integrations/index.md4
-rw-r--r--doc/development/integrations/jira_connect.md2
-rw-r--r--doc/development/integrations/secure.md4
-rw-r--r--doc/development/integrations/secure_partner_integration.md2
-rw-r--r--doc/development/internal_api/index.md4
-rw-r--r--doc/development/kubernetes.md8
-rw-r--r--doc/development/licensed_feature_availability.md11
-rw-r--r--doc/development/merge_request_concepts/widget_extensions.md11
-rw-r--r--doc/development/migration_style_guide.md34
-rw-r--r--doc/development/multi_version_compatibility.md2
-rw-r--r--doc/development/packages/cleanup_policies.md122
-rw-r--r--doc/development/packages/debian_repository.md2
-rw-r--r--doc/development/packages/dependency_proxy.md4
-rw-r--r--doc/development/packages/index.md20
-rw-r--r--doc/development/packages/new_format_development.md2
-rw-r--r--doc/development/packages/settings.md2
-rw-r--r--doc/development/packages/structure.md2
-rw-r--r--doc/development/performance.md4
-rw-r--r--doc/development/pipelines.md874
-rw-r--r--doc/development/pipelines/index.md631
-rw-r--r--doc/development/pipelines/internals.md216
-rw-r--r--doc/development/pipelines/performance.md151
-rw-r--r--doc/development/prometheus_metrics.md2
-rw-r--r--doc/development/rake_tasks.md14
-rw-r--r--doc/development/reusing_abstractions.md3
-rw-r--r--doc/development/routing.md2
-rw-r--r--doc/development/rubocop_development_guide.md114
-rw-r--r--doc/development/sec/analyzer_development_guide.md20
-rw-r--r--doc/development/sec/img/primary_identifier_changed_v15_6.pngbin0 -> 12665 bytes
-rw-r--r--doc/development/sec/index.md90
-rw-r--r--doc/development/sec/security_report_ingestion_overview.md74
-rw-r--r--doc/development/secure_coding_guidelines.md8
-rw-r--r--doc/development/service_ping/implement.md132
-rw-r--r--doc/development/service_ping/index.md4
-rw-r--r--doc/development/service_ping/metrics_instrumentation.md15
-rw-r--r--doc/development/service_ping/metrics_lifecycle.md7
-rw-r--r--doc/development/service_ping/review_guidelines.md1
-rw-r--r--doc/development/service_ping/troubleshooting.md42
-rw-r--r--doc/development/service_ping/usage_data.md1
-rw-r--r--doc/development/shell_commands.md2
-rw-r--r--doc/development/sidekiq/index.md25
-rw-r--r--doc/development/sidekiq/worker_attributes.md2
-rw-r--r--doc/development/snowplow/event_dictionary_guide.md36
-rw-r--r--doc/development/snowplow/implementation.md24
-rw-r--r--doc/development/snowplow/index.md17
-rw-r--r--doc/development/snowplow/review_guidelines.md2
-rw-r--r--doc/development/snowplow/schemas.md19
-rw-r--r--doc/development/spam_protection_and_captcha/exploratory_testing.md2
-rw-r--r--doc/development/sql.md4
-rw-r--r--doc/development/stage_group_observability/dashboards/index.md4
-rw-r--r--doc/development/stage_group_observability/dashboards/stage_group_dashboard.md2
-rw-r--r--doc/development/testing_guide/best_practices.md27
-rw-r--r--doc/development/testing_guide/contract/consumer_tests.md2
-rw-r--r--doc/development/testing_guide/end_to_end/best_practices.md4
-rw-r--r--doc/development/testing_guide/end_to_end/feature_flags.md2
-rw-r--r--doc/development/testing_guide/end_to_end/index.md23
-rw-r--r--doc/development/testing_guide/end_to_end/rspec_metadata_tests.md2
-rw-r--r--doc/development/testing_guide/end_to_end/style_guide.md8
-rw-r--r--doc/development/testing_guide/flaky_tests.md139
-rw-r--r--doc/development/testing_guide/frontend_testing.md332
-rw-r--r--doc/development/testing_guide/index.md7
-rw-r--r--doc/development/testing_guide/review_apps.md70
-rw-r--r--doc/development/utilities.md18
-rw-r--r--doc/development/windows.md2
-rw-r--r--doc/development/work_items.md31
-rw-r--r--doc/development/workhorse/index.md2
-rw-r--r--doc/drawers/advanced_search_syntax.md45
-rw-r--r--doc/gitlab-basics/add-file.md2
-rw-r--r--doc/gitlab-basics/command-line-commands.md2
-rw-r--r--doc/gitlab-basics/start-using-git.md2
-rw-r--r--doc/install/aws/gitlab_hybrid_on_aws.md2
-rw-r--r--doc/install/aws/manual_install_aws.md4
-rw-r--r--doc/install/docker.md6
-rw-r--r--doc/install/google_cloud_platform/index.md2
-rw-r--r--doc/install/installation.md26
-rw-r--r--doc/install/migrate/compare_sm_to_saas.md4
-rw-r--r--doc/install/relative_url.md2
-rw-r--r--doc/install/requirements.md6
-rw-r--r--doc/integration/advanced_search/elasticsearch.md2
-rw-r--r--doc/integration/advanced_search/elasticsearch_troubleshooting.md4
-rw-r--r--doc/integration/bitbucket.md2
-rw-r--r--doc/integration/gitlab.md9
-rw-r--r--doc/integration/gitpod.md2
-rw-r--r--doc/integration/index.md108
-rw-r--r--doc/integration/jenkins_deprecated.md13
-rw-r--r--doc/integration/jira/configure.md2
-rw-r--r--doc/integration/jira/index.md99
-rw-r--r--doc/integration/jira/jira_server_configuration.md2
-rw-r--r--doc/integration/jira/troubleshooting.md110
-rw-r--r--doc/integration/oauth_provider.md16
-rw-r--r--doc/integration/omniauth.md66
-rw-r--r--doc/integration/saml.md18
-rw-r--r--doc/integration/slash_commands.md2
-rw-r--r--doc/integration/vault.md10
-rw-r--r--doc/legal/corporate_contributor_license_agreement.md9
-rw-r--r--doc/legal/developer_certificate_of_origin.md1
-rw-r--r--doc/legal/individual_contributor_license_agreement.md9
-rw-r--r--doc/operations/error_tracking.md19
-rw-r--r--doc/operations/feature_flags.md2
-rw-r--r--doc/operations/incident_management/alerts.md89
-rw-r--r--doc/operations/incident_management/img/timeline_event_for_severity_change_v15_6.pngbin0 -> 10377 bytes
-rw-r--r--doc/operations/incident_management/incident_timeline_events.md11
-rw-r--r--doc/operations/incident_management/incidents.md2
-rw-r--r--doc/operations/incident_management/integrations.md4
-rw-r--r--doc/operations/incident_management/linked_resources.md8
-rw-r--r--doc/operations/incident_management/paging.md4
-rw-r--r--doc/operations/index.md30
-rw-r--r--doc/operations/metrics/dashboards/index.md4
-rw-r--r--doc/operations/metrics/dashboards/templating_variables.md10
-rw-r--r--doc/operations/metrics/index.md4
-rw-r--r--doc/operations/product_analytics.md45
-rw-r--r--doc/operations/tracing.md51
-rw-r--r--doc/raketasks/backup_gitlab.md2
-rw-r--r--doc/raketasks/cleanup.md6
-rw-r--r--doc/raketasks/index.md2
-rw-r--r--doc/security/crime_vulnerability.md2
-rw-r--r--doc/security/index.md2
-rw-r--r--doc/security/information_exclusivity.md2
-rw-r--r--doc/security/password_length_limits.md46
-rw-r--r--doc/security/password_storage.md25
-rw-r--r--doc/security/passwords_for_integrated_authentication_methods.md2
-rw-r--r--doc/security/ssh_keys_restrictions.md2
-rw-r--r--doc/security/two_factor_authentication.md8
-rw-r--r--doc/security/unlock_user.md4
-rw-r--r--doc/security/user_email_confirmation.md2
-rw-r--r--doc/security/user_file_uploads.md2
-rw-r--r--doc/security/webhooks.md2
-rw-r--r--doc/subscriptions/gitlab_com/index.md39
-rw-r--r--doc/subscriptions/gitlab_dedicated/index.md2
-rw-r--r--doc/subscriptions/index.md18
-rw-r--r--doc/subscriptions/self_managed/index.md39
-rw-r--r--doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md11
-rw-r--r--doc/topics/autodevops/customize.md3
-rw-r--r--doc/topics/autodevops/multiple_clusters_auto_devops.md2
-rw-r--r--doc/topics/autodevops/stages.md2
-rw-r--r--doc/topics/awesome_co.md2
-rw-r--r--doc/topics/git/how_to_install_git/index.md2
-rw-r--r--doc/topics/git/lfs/index.md18
-rw-r--r--doc/topics/git/lfs/migrate_to_git_lfs.md4
-rw-r--r--doc/topics/git/merge_conflicts.md72
-rw-r--r--doc/topics/git/numerous_undo_possibilities_in_git/index.md2
-rw-r--r--doc/topics/git/useful_git_commands.md2
-rw-r--r--doc/topics/release_your_application.md2
-rw-r--r--doc/tutorials/agile_sprint.md101
-rw-r--r--doc/tutorials/index.md7
-rw-r--r--doc/tutorials/make_your_first_git_commit.md18
-rw-r--r--doc/tutorials/move_personal_project_to_a_group.md2
-rw-r--r--doc/update/deprecations.md148
-rw-r--r--doc/update/index.md32
-rw-r--r--doc/update/patch_versions.md5
-rw-r--r--doc/update/removals.md24
-rw-r--r--doc/update/upgrading_from_ce_to_ee.md3
-rw-r--r--doc/update/upgrading_from_source.md11
-rw-r--r--doc/update/zero_downtime.md2
-rw-r--r--doc/user/admin_area/appearance.md2
-rw-r--r--doc/user/admin_area/broadcast_messages.md2
-rw-r--r--doc/user/admin_area/custom_project_templates.md2
-rw-r--r--doc/user/admin_area/diff_limits.md2
-rw-r--r--doc/user/admin_area/email_from_gitlab.md2
-rw-r--r--doc/user/admin_area/geo_nodes.md11
-rw-r--r--doc/user/admin_area/geo_sites.md2
-rw-r--r--doc/user/admin_area/img/minimum_password_length_settings_v12_6.pngbin10385 -> 0 bytes
-rw-r--r--doc/user/admin_area/index.md8
-rw-r--r--doc/user/admin_area/labels.md2
-rw-r--r--doc/user/admin_area/license.md17
-rw-r--r--doc/user/admin_area/license_file.md80
-rw-r--r--doc/user/admin_area/moderate_users.md81
-rw-r--r--doc/user/admin_area/monitoring/health_check.md2
-rw-r--r--doc/user/admin_area/reporting/git_abuse_rate_limit.md27
-rw-r--r--doc/user/admin_area/review_abuse_reports.md2
-rw-r--r--doc/user/admin_area/settings/account_and_limit_settings.md18
-rw-r--r--doc/user/admin_area/settings/continuous_integration.md16
-rw-r--r--doc/user/admin_area/settings/deprecated_api_rate_limits.md4
-rw-r--r--doc/user/admin_area/settings/email.md2
-rw-r--r--doc/user/admin_area/settings/external_authorization.md2
-rw-r--r--doc/user/admin_area/settings/files_api_rate_limits.md4
-rw-r--r--doc/user/admin_area/settings/floc.md2
-rw-r--r--doc/user/admin_area/settings/help_page.md4
-rw-r--r--doc/user/admin_area/settings/index.md4
-rw-r--r--doc/user/admin_area/settings/instance_template_repository.md8
-rw-r--r--doc/user/admin_area/settings/package_registry_rate_limits.md2
-rw-r--r--doc/user/admin_area/settings/sign_in_restrictions.md75
-rw-r--r--doc/user/admin_area/settings/sign_up_restrictions.md4
-rw-r--r--doc/user/admin_area/settings/terms.md2
-rw-r--r--doc/user/admin_area/settings/third_party_offers.md2
-rw-r--r--doc/user/admin_area/settings/usage_statistics.md4
-rw-r--r--doc/user/admin_area/settings/user_and_ip_rate_limits.md2
-rw-r--r--doc/user/admin_area/settings/visibility_and_access_controls.md28
-rw-r--r--doc/user/analytics/ci_cd_analytics.md18
-rw-r--r--doc/user/analytics/dora_metrics.md127
-rw-r--r--doc/user/analytics/index.md89
-rw-r--r--doc/user/application_security/api_fuzzing/create_har_files.md4
-rw-r--r--doc/user/application_security/api_fuzzing/index.md261
-rw-r--r--doc/user/application_security/configuration/index.md2
-rw-r--r--doc/user/application_security/container_scanning/index.md44
-rw-r--r--doc/user/application_security/dast/browser_based.md6
-rw-r--r--doc/user/application_security/dast/dast_troubleshooting.md2
-rw-r--r--doc/user/application_security/dast/index.md1438
-rw-r--r--doc/user/application_security/dast/proxy-based.md1247
-rw-r--r--doc/user/application_security/dast_api/index.md246
-rw-r--r--doc/user/application_security/dependency_scanning/index.md21
-rw-r--r--doc/user/application_security/get-started-security.md2
-rw-r--r--doc/user/application_security/iac_scanning/index.md158
-rw-r--r--doc/user/application_security/img/secure_tools_and_cicd_stages.pngbin42240 -> 11849 bytes
-rw-r--r--doc/user/application_security/index.md12
-rw-r--r--doc/user/application_security/policies/index.md2
-rw-r--r--doc/user/application_security/policies/scan-execution-policies.md12
-rw-r--r--doc/user/application_security/policies/scan-result-policies.md8
-rw-r--r--doc/user/application_security/sast/analyzers.md10
-rw-r--r--doc/user/application_security/sast/index.md214
-rw-r--r--doc/user/application_security/secret_detection/index.md99
-rw-r--r--doc/user/application_security/security_dashboard/index.md2
-rw-r--r--doc/user/application_security/terminology/index.md2
-rw-r--r--doc/user/application_security/vulnerabilities/severities.md4
-rw-r--r--doc/user/application_security/vulnerability_report/index.md4
-rw-r--r--doc/user/award_emojis.md26
-rw-r--r--doc/user/clusters/agent/ci_cd_workflow.md55
-rw-r--r--doc/user/clusters/agent/gitops/helm.md40
-rw-r--r--doc/user/clusters/agent/index.md7
-rw-r--r--doc/user/clusters/agent/install/index.md4
-rw-r--r--doc/user/clusters/agent/troubleshooting.md21
-rw-r--r--doc/user/clusters/management_project.md2
-rw-r--r--doc/user/clusters/management_project_template.md1
-rw-r--r--doc/user/compliance/compliance_report/index.md8
-rw-r--r--doc/user/crm/index.md24
-rw-r--r--doc/user/discussions/index.md30
-rw-r--r--doc/user/feature_highlight.md11
-rw-r--r--doc/user/free_user_limit.md28
-rw-r--r--doc/user/group/access_and_permissions.md20
-rw-r--r--doc/user/group/clusters/index.md2
-rw-r--r--doc/user/group/compliance_frameworks.md262
-rw-r--r--doc/user/group/contribution_analytics/index.md10
-rw-r--r--doc/user/group/custom_project_templates.md2
-rw-r--r--doc/user/group/epics/epic_boards.md6
-rw-r--r--doc/user/group/epics/index.md10
-rw-r--r--doc/user/group/epics/manage_epics.md56
-rw-r--r--doc/user/group/import/index.md10
-rw-r--r--doc/user/group/insights/index.md2
-rw-r--r--doc/user/group/issues_analytics/index.md2
-rw-r--r--doc/user/group/iterations/index.md6
-rw-r--r--doc/user/group/manage.md253
-rw-r--r--doc/user/group/planning_hierarchy/index.md4
-rw-r--r--doc/user/group/reporting/git_abuse_rate_limit.md36
-rw-r--r--doc/user/group/repositories_analytics/index.md4
-rw-r--r--doc/user/group/roadmap/img/roadmap_blocked_icon_v15_5.pngbin25453 -> 8248 bytes
-rw-r--r--doc/user/group/roadmap/index.md2
-rw-r--r--doc/user/group/saml_sso/group_sync.md8
-rw-r--r--doc/user/group/saml_sso/img/member_enterprise_badge_v14_0.pngbin31502 -> 0 bytes
-rw-r--r--doc/user/group/saml_sso/index.md28
-rw-r--r--doc/user/group/saml_sso/scim_setup.md50
-rw-r--r--doc/user/group/saml_sso/troubleshooting.md38
-rw-r--r--doc/user/group/saml_sso/troubleshooting_scim.md156
-rw-r--r--doc/user/group/subgroups/index.md2
-rw-r--r--doc/user/group/value_stream_analytics/index.md2
-rw-r--r--doc/user/infrastructure/clusters/connect/new_civo_cluster.md2
-rw-r--r--doc/user/infrastructure/clusters/connect/new_eks_cluster.md2
-rw-r--r--doc/user/infrastructure/clusters/connect/new_gke_cluster.md6
-rw-r--r--doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md2
-rw-r--r--doc/user/infrastructure/clusters/manage/management_project_applications/vault.md12
-rw-r--r--doc/user/infrastructure/iac/index.md4
-rw-r--r--doc/user/infrastructure/iac/terraform_state.md2
-rw-r--r--doc/user/infrastructure/iac/troubleshooting.md10
-rw-r--r--doc/user/markdown.md18
-rw-r--r--doc/user/packages/composer_repository/index.md59
-rw-r--r--doc/user/packages/conan_repository/index.md114
-rw-r--r--doc/user/packages/container_registry/index.md12
-rw-r--r--doc/user/packages/container_registry/reduce_container_registry_data_transfer.md4
-rw-r--r--doc/user/packages/container_registry/reduce_container_registry_storage.md2
-rw-r--r--doc/user/packages/debian_repository/index.md2
-rw-r--r--doc/user/packages/dependency_proxy/index.md4
-rw-r--r--doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md2
-rw-r--r--doc/user/packages/generic_packages/index.md2
-rw-r--r--doc/user/packages/go_proxy/index.md2
-rw-r--r--doc/user/packages/harbor_container_registry/index.md2
-rw-r--r--doc/user/packages/helm_repository/index.md4
-rw-r--r--doc/user/packages/index.md2
-rw-r--r--doc/user/packages/infrastructure_registry/index.md4
-rw-r--r--doc/user/packages/maven_repository/index.md206
-rw-r--r--doc/user/packages/npm_registry/index.md103
-rw-r--r--doc/user/packages/nuget_repository/index.md46
-rw-r--r--doc/user/packages/package_registry/index.md2
-rw-r--r--doc/user/packages/package_registry/reduce_package_registry_storage.md2
-rw-r--r--doc/user/packages/pypi_repository/index.md145
-rw-r--r--doc/user/packages/rubygems_registry/index.md2
-rw-r--r--doc/user/packages/terraform_module_registry/index.md12
-rw-r--r--doc/user/packages/workflows/build_packages.md504
-rw-r--r--doc/user/packages/workflows/project_registry.md2
-rw-r--r--doc/user/packages/workflows/working_with_monorepos.md2
-rw-r--r--doc/user/permissions.md43
-rw-r--r--doc/user/product_analytics/index.md48
-rw-r--r--doc/user/profile/account/create_accounts.md16
-rw-r--r--doc/user/profile/account/two_factor_authentication.md4
-rw-r--r--doc/user/profile/active_sessions.md2
-rw-r--r--doc/user/profile/contributions_calendar.md136
-rw-r--r--doc/user/profile/img/contributions_calendar_v15_6.pngbin0 -> 60252 bytes
-rw-r--r--doc/user/profile/index.md81
-rw-r--r--doc/user/profile/notifications.md49
-rw-r--r--doc/user/profile/personal_access_tokens.md36
-rw-r--r--doc/user/profile/preferences.md4
-rw-r--r--doc/user/profile/user_passwords.md3
-rw-r--r--doc/user/project/badges.md2
-rw-r--r--doc/user/project/clusters/add_existing_cluster.md2
-rw-r--r--doc/user/project/clusters/deploy_to_cluster.md2
-rw-r--r--doc/user/project/clusters/kubernetes_pod_logs.md12
-rw-r--r--doc/user/project/clusters/runbooks/index.md4
-rw-r--r--doc/user/project/import/clearcase.md2
-rw-r--r--doc/user/project/import/github.md143
-rw-r--r--doc/user/project/import/tfvc.md2
-rw-r--r--doc/user/project/integrations/gitlab_slack_application.md64
-rw-r--r--doc/user/project/integrations/hangouts_chat.md2
-rw-r--r--doc/user/project/integrations/mlflow_client.md81
-rw-r--r--doc/user/project/integrations/prometheus_library/nginx_ingress.md2
-rw-r--r--doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md2
-rw-r--r--doc/user/project/integrations/servicenow.md4
-rw-r--r--doc/user/project/integrations/slack.md16
-rw-r--r--doc/user/project/integrations/slack_slash_commands.md2
-rw-r--r--doc/user/project/integrations/unify_circuit.md2
-rw-r--r--doc/user/project/integrations/webex_teams.md2
-rw-r--r--doc/user/project/integrations/webhook_events.md19
-rw-r--r--doc/user/project/integrations/webhooks.md24
-rw-r--r--doc/user/project/issue_board.md8
-rw-r--r--doc/user/project/issues/confidential_issues.md6
-rw-r--r--doc/user/project/issues/design_management.md2
-rw-r--r--doc/user/project/issues/due_dates.md2
-rw-r--r--doc/user/project/issues/managing_issues.md46
-rw-r--r--doc/user/project/issues/related_issues.md2
-rw-r--r--doc/user/project/labels.md32
-rw-r--r--doc/user/project/members/index.md58
-rw-r--r--doc/user/project/merge_requests/approvals/index.md2
-rw-r--r--doc/user/project/merge_requests/authorization_for_merge_requests.md2
-rw-r--r--doc/user/project/merge_requests/changes.md6
-rw-r--r--doc/user/project/merge_requests/conflicts.md2
-rw-r--r--doc/user/project/merge_requests/drafts.md2
-rw-r--r--doc/user/project/merge_requests/img/conflict_ui_v14_0.pngbin8371 -> 0 bytes
-rw-r--r--doc/user/project/merge_requests/img/conflict_ui_v15_6.pngbin0 -> 14089 bytes
-rw-r--r--doc/user/project/merge_requests/index.md95
-rw-r--r--doc/user/project/merge_requests/revert_changes.md2
-rw-r--r--doc/user/project/merge_requests/reviews/data_usage.md2
-rw-r--r--doc/user/project/merge_requests/reviews/index.md2
-rw-r--r--doc/user/project/merge_requests/reviews/suggestions.md7
-rw-r--r--doc/user/project/merge_requests/squash_and_merge.md2
-rw-r--r--doc/user/project/merge_requests/versions.md8
-rw-r--r--doc/user/project/milestones/burndown_and_burnup_charts.md2
-rw-r--r--doc/user/project/milestones/index.md2
-rw-r--r--doc/user/project/ml/experiment_tracking/img/candidates.pngbin0 -> 62281 bytes
-rw-r--r--doc/user/project/ml/experiment_tracking/img/experiments.pngbin0 -> 45022 bytes
-rw-r--r--doc/user/project/ml/experiment_tracking/index.md76
-rw-r--r--doc/user/project/pages/custom_domains_ssl_tls_certification/index.md2
-rw-r--r--doc/user/project/pages/getting_started/pages_ci_cd_template.md10
-rw-r--r--doc/user/project/pages/getting_started/pages_forked_sample_project.md4
-rw-r--r--doc/user/project/pages/getting_started/pages_from_scratch.md6
-rw-r--r--doc/user/project/pages/getting_started/pages_new_project_template.md4
-rw-r--r--doc/user/project/pages/getting_started/pages_ui.md4
-rw-r--r--doc/user/project/pages/img/remove_pages_v15_3.pngbin4432 -> 0 bytes
-rw-r--r--doc/user/project/pages/index.md2
-rw-r--r--doc/user/project/pages/introduction.md67
-rw-r--r--doc/user/project/pages/pages_access_control.md2
-rw-r--r--doc/user/project/pages/public_folder.md2
-rw-r--r--doc/user/project/protected_branches.md2
-rw-r--r--doc/user/project/protected_tags.md2
-rw-r--r--doc/user/project/quick_actions.md14
-rw-r--r--doc/user/project/releases/index.md14
-rw-r--r--doc/user/project/releases/release_fields.md4
-rw-r--r--doc/user/project/remote_development/index.md12
-rw-r--r--doc/user/project/repository/branches/index.md2
-rw-r--r--doc/user/project/repository/gpg_signed_commits/index.md16
-rw-r--r--doc/user/project/repository/index.md2
-rw-r--r--doc/user/project/repository/jupyter_notebooks/index.md12
-rw-r--r--doc/user/project/repository/mirror/index.md3
-rw-r--r--doc/user/project/repository/push_rules.md4
-rw-r--r--doc/user/project/repository/reducing_the_repo_size_using_git.md45
-rw-r--r--doc/user/project/repository/web_editor.md33
-rw-r--r--doc/user/project/repository/x509_signed_commits/index.md5
-rw-r--r--doc/user/project/requirements/index.md6
-rw-r--r--doc/user/project/settings/import_export.md165
-rw-r--r--doc/user/project/settings/import_export_troubleshooting.md280
-rw-r--r--doc/user/project/settings/index.md9
-rw-r--r--doc/user/project/web_ide/index.md13
-rw-r--r--doc/user/project/working_with_projects.md6
-rw-r--r--doc/user/public_access.md12
-rw-r--r--doc/user/reserved_names.md4
-rw-r--r--doc/user/search/advanced_search.md38
-rw-r--r--doc/user/search/global_search/advanced_search_syntax.md55
-rw-r--r--doc/user/shortcuts.md3
-rw-r--r--doc/user/ssh.md25
-rw-r--r--doc/user/tasks.md43
-rw-r--r--doc/user/usage_quotas.md99
-rw-r--r--doc/user/workspace/index.md2
814 files changed, 18472 insertions, 10929 deletions
diff --git a/doc/.markdownlint/require_helper.js b/doc/.markdownlint/require_helper.js
new file mode 100644
index 00000000000..7d06cf67419
--- /dev/null
+++ b/doc/.markdownlint/require_helper.js
@@ -0,0 +1,14 @@
+/**
+ * Look up the global node modules directory.
+ *
+ * Because we install markdownlint packages globally
+ * in the Docker image where this runs, we need to
+ * provide the path to the global install location
+ * when referencing global functions from our own node
+ * modules.
+ *
+ * Image:
+ * https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/dockerfiles/gitlab-docs-lint-markdown.Dockerfile
+ */
+const { execSync } = require('child_process');
+module.exports.globalPath = execSync('yarn global dir').toString().trim() + '/node_modules/';
diff --git a/doc/.markdownlint/rules/tabs_blank_lines.js b/doc/.markdownlint/rules/tabs_blank_lines.js
new file mode 100644
index 00000000000..e0e2c1a0a9b
--- /dev/null
+++ b/doc/.markdownlint/rules/tabs_blank_lines.js
@@ -0,0 +1,26 @@
+const { globalPath } = require('../require_helper');
+const {
+ forEachLine,
+ getLineMetadata,
+ isBlankLine,
+} = require(`${globalPath}/markdownlint-rule-helpers`);
+
+module.exports = {
+ names: ['tabs-blank-lines'],
+ description: 'Tab elements must be surrounded by blank lines',
+ tags: ['gitlab-docs', 'tabs'],
+ function: (params, onError) => {
+ const tabElements = ['::Tabs', '::EndTabs', ':::TabTitle'];
+ forEachLine(getLineMetadata(params), (line, lineIndex) => {
+ const lineHasTab = tabElements.includes(line.split(' ')[0]);
+ const prevLine = params.lines[lineIndex - 1];
+ const nextLine = params.lines[lineIndex + 1];
+
+ if (lineHasTab && (!isBlankLine(prevLine) || !isBlankLine(nextLine))) {
+ onError({
+ lineNumber: lineIndex + 1,
+ });
+ }
+ });
+ },
+};
diff --git a/doc/.markdownlint/rules/tabs_title_markup.js b/doc/.markdownlint/rules/tabs_title_markup.js
new file mode 100644
index 00000000000..9c1de1e630d
--- /dev/null
+++ b/doc/.markdownlint/rules/tabs_title_markup.js
@@ -0,0 +1,31 @@
+const { globalPath } = require('../require_helper');
+const { forEachLine, getLineMetadata } = require(`${globalPath}/markdownlint-rule-helpers`);
+
+module.exports = {
+ names: ['tabs-title-markup'],
+ description: 'Incorrect number of colon characters for tag',
+ information: new URL('https://docs.gitlab.com/ee/development/documentation/styleguide/#tabs'),
+ tags: ['gitlab-docs', 'tabs'],
+ function: (params, onError) => {
+ // Note the correct number of colons in each tab tag type.
+ const wrapperColons = 2;
+ const titleColons = 3;
+
+ forEachLine(getLineMetadata(params), (line, lineIndex) => {
+ // Get the number of colons in this line.
+ const colonCount = [...line].filter((x) => x === ':').length;
+
+ // Throw an error in the case of a mismatch.
+ if (
+ ((line.includes(':Tabs') || line.includes(':EndTabs')) && colonCount !== wrapperColons) ||
+ (line.includes(':TabTitle') && colonCount !== titleColons)
+ ) {
+ const correctColonCount = line.includes(':TabTitle') ? wrapperColons : titleColons;
+ onError({
+ lineNumber: lineIndex + 1,
+ detail: `Actual: ${colonCount}; Expected: ${correctColonCount}`,
+ });
+ }
+ });
+ },
+};
diff --git a/doc/.markdownlint/rules/tabs_title_text.js b/doc/.markdownlint/rules/tabs_title_text.js
new file mode 100644
index 00000000000..672aa70f562
--- /dev/null
+++ b/doc/.markdownlint/rules/tabs_title_text.js
@@ -0,0 +1,23 @@
+const { globalPath } = require('../require_helper');
+const {
+ forEachLine,
+ getLineMetadata,
+ isBlankLine,
+} = require(`${globalPath}/markdownlint-rule-helpers`);
+
+module.exports = {
+ names: ['tabs-title-text'],
+ description: 'Tab without title text',
+ information: new URL('https://docs.gitlab.com/ee/development/documentation/styleguide/#tabs'),
+ tags: ['gitlab-docs', 'tabs'],
+ function: (params, onError) => {
+ forEachLine(getLineMetadata(params), (line, lineIndex) => {
+ if (!isBlankLine(line) && line.replace(':::TabTitle', '').trim() === '') {
+ onError({
+ lineNumber: lineIndex + 1,
+ detail: 'Expected: :::TabTitle <your title here>; Actual: :::TabTitle',
+ });
+ }
+ });
+ },
+};
diff --git a/doc/.markdownlint/rules/tabs_wrapper_tags.js b/doc/.markdownlint/rules/tabs_wrapper_tags.js
new file mode 100644
index 00000000000..beacec0b737
--- /dev/null
+++ b/doc/.markdownlint/rules/tabs_wrapper_tags.js
@@ -0,0 +1,21 @@
+module.exports = {
+ names: ['tabs-wrapper-tags'],
+ description: 'Unequal number of tab start and end tags',
+ information: new URL('https://docs.gitlab.com/ee/development/documentation/styleguide/#tabs'),
+ tags: ['gitlab-docs', 'tabs'],
+ function: function rule(params, onError) {
+ const tabStarts = params.lines.filter((line) => line === '::Tabs');
+ const tabEnds = params.lines.filter((line) => line === '::EndTabs');
+
+ if (tabStarts.length !== tabEnds.length) {
+ const errorIndex =
+ params.lines.indexOf('::Tabs') > 0
+ ? params.lines.indexOf('::Tabs')
+ : params.lines.indexOf('::EndTabs');
+ onError({
+ lineNumber: errorIndex + 1,
+ detail: `Opening tags: ${tabStarts.length}; Closing tags: ${tabEnds.length}`,
+ });
+ }
+ },
+};
diff --git a/doc/.vale/gitlab/Admin.yml b/doc/.vale/gitlab/Admin.yml
deleted file mode 100644
index 78a86e27456..00000000000
--- a/doc/.vale/gitlab/Admin.yml
+++ /dev/null
@@ -1,13 +0,0 @@
----
-# Suggestion: gitlab.Admin
-#
-# Checks for "admin" and recommends using the full word instead. "Admin Area" is OK.
-#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
-extends: substitution
-message: 'Verify this use of the word "admin". Can it be updated to "administration", "administrator", "administer", or "Admin Area"?'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html
-level: suggestion
-ignorecase: false
-swap:
- '[Aa]dmin ?\w*': '(?:Admin( Area| Mode)?|[Aa]dminist(ration|rator|rators|er|rative|ering|ered))'
diff --git a/doc/.vale/gitlab/AlertBoxStyle.yml b/doc/.vale/gitlab/AlertBoxStyle.yml
index 5912c4707fd..0a4514fa3c8 100644
--- a/doc/.vale/gitlab/AlertBoxStyle.yml
+++ b/doc/.vale/gitlab/AlertBoxStyle.yml
@@ -3,13 +3,13 @@
#
# Makes sure alert boxes are used with block quotes. Checks for 3 formatting issues:
#
-# - Alert boxes inside a block quote (">")
+# - Alert boxes inside a block quote ('>')
# - Alert boxes with the note text on the same line
-# - Alert boxes using words other than "NOTE" or "WARNING"
+# - Alert boxes using words other than 'NOTE' or 'WARNING'
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Alert box "%s" must use the formatting in the style guide.'
+message: "Update the format of the '%s' alert box. View the style guide for details."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#alert-boxes
level: error
nonword: true
diff --git a/doc/.vale/gitlab/BadPlurals.yml b/doc/.vale/gitlab/BadPlurals.yml
index 533805c67b0..9cdb8661708 100644
--- a/doc/.vale/gitlab/BadPlurals.yml
+++ b/doc/.vale/gitlab/BadPlurals.yml
@@ -1,11 +1,11 @@
---
# Warning: gitlab.BadPlurals
#
-# Don't write plural words with the '(s)' construction. "HTTP(S)" is acceptable.
+# Don't write plural words with the '(s)' construction. 'HTTP(S)' is acceptable.
#
-# For a list of all options, see https://docs.errata.ai/vale/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Rewrite "%s" to be plural, without parentheses.'
+message: "Rewrite '%s' to be plural without parentheses."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html#s
level: warning
ignorecase: true
diff --git a/doc/.vale/gitlab/BadgeCapitalization.yml b/doc/.vale/gitlab/BadgeCapitalization.yml
index 33425693d53..6e77c3fe822 100644
--- a/doc/.vale/gitlab/BadgeCapitalization.yml
+++ b/doc/.vale/gitlab/BadgeCapitalization.yml
@@ -3,9 +3,9 @@
#
# Verifies that badges are not mixed case, which won't render properly.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Badge "%s" must be capitalized.'
+message: "Capitalize the '%s' badge."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#product-tier-badges
level: error
scope: raw
diff --git a/doc/.vale/gitlab/British.yml b/doc/.vale/gitlab/British.yml
index c63cbd917c7..432ed302e11 100644
--- a/doc/.vale/gitlab/British.yml
+++ b/doc/.vale/gitlab/British.yml
@@ -3,9 +3,9 @@
#
# Checks that US spelling is used instead of British spelling.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'Use the US spelling "%s" instead of the British "%s".'
+message: "Use the US spelling '%s' instead of the British '%s'."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#language
level: error
ignorecase: true
diff --git a/doc/.vale/gitlab/CIConfigFile.yml b/doc/.vale/gitlab/CIConfigFile.yml
index 5a65e51ea16..4d2ba454410 100644
--- a/doc/.vale/gitlab/CIConfigFile.yml
+++ b/doc/.vale/gitlab/CIConfigFile.yml
@@ -3,9 +3,9 @@
#
# Checks that the `.gitlab-ci.yml` file is referenced properly.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'The CI/CD configuration file should be exactly: `.gitlab-ci.yml`'
+message: "Change the file name to be exactly '.gitlab-ci.yml'."
link: https://docs.gitlab.com/ee/development/documentation/versions.html
level: error
scope: raw
diff --git a/doc/.vale/gitlab/CodeblockFences.yml b/doc/.vale/gitlab/CodeblockFences.yml
index 6c2d94e7b8d..9d5efe7f60a 100644
--- a/doc/.vale/gitlab/CodeblockFences.yml
+++ b/doc/.vale/gitlab/CodeblockFences.yml
@@ -3,9 +3,9 @@
#
# Ensures all codeblock language tags use the full name, not aliases.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Syntax highlighting hint "%s" must be one of: yaml, ruby, plaintext, markdown, javascript, shell, golang, python, dockerfile, or typescript.'
+message: "Instead of '%s' for the code block, use yaml, ruby, plaintext, markdown, javascript, shell, golang, python, dockerfile, or typescript."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#code-blocks
level: error
scope: raw
diff --git a/doc/.vale/gitlab/CurlStringsQuoted.yml b/doc/.vale/gitlab/CurlStringsQuoted.yml
index a59fe64d990..efe7aa23832 100644
--- a/doc/.vale/gitlab/CurlStringsQuoted.yml
+++ b/doc/.vale/gitlab/CurlStringsQuoted.yml
@@ -3,9 +3,9 @@
#
# Ensures all code blocks using `curl` wrap URL strings in quotation marks.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'For consistency across all cURL examples, always wrap the URL in double quotes ("): %s'
+message: "For the cURL example, use double quotes around the URL: %s"
link: https://docs.gitlab.com/ee/development/documentation/restful_api_styleguide.html#curl-commands
level: error
scope: code
diff --git a/doc/.vale/gitlab/CurrentStatus.yml b/doc/.vale/gitlab/CurrentStatus.yml
index 040fd3d8a8f..57b95dcf4ac 100644
--- a/doc/.vale/gitlab/CurrentStatus.yml
+++ b/doc/.vale/gitlab/CurrentStatus.yml
@@ -3,12 +3,12 @@
#
# Checks for words that indicate a product or feature may change in the future.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Avoid words like "%s" when you write about future features. Our documentation is about the current state of the product.'
+message: "Remove '%s'. The documentation reflects the current state of the product."
level: suggestion
ignorecase: true
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/#promising-features-in-future-versions
+link: https://docs.gitlab.com/ee/development/documentation/versions.html#promising-features-in-future-versions
tokens:
- currently
- yet
diff --git a/doc/.vale/gitlab/DefaultBranch.yml b/doc/.vale/gitlab/DefaultBranch.yml
index 4bc68433c6d..86c627bcfe3 100644
--- a/doc/.vale/gitlab/DefaultBranch.yml
+++ b/doc/.vale/gitlab/DefaultBranch.yml
@@ -1,14 +1,14 @@
---
# Warning: gitlab.DefaultBranch
#
-# Do not refer to the default branch as the "master" branch, if possible.
+# Do not refer to the default branch as the 'master' branch, if possible.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Use "default branch" or `main` instead of `master`, when possible.'
+message: "Use 'default branch' or `main` instead of `master`, when possible."
level: warning
ignorecase: true
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html#default-branch
scope: raw
raw:
- '\`master\`'
diff --git a/doc/.vale/gitlab/Dropdown.yml b/doc/.vale/gitlab/Dropdown.yml
index 691d44d1a48..c656d1209f5 100644
--- a/doc/.vale/gitlab/Dropdown.yml
+++ b/doc/.vale/gitlab/Dropdown.yml
@@ -3,11 +3,11 @@
#
# Catches many ways the phrase 'dropdown list' can be fumbled.
#
-# For a list of all options, see https://errata-ai.github.io/vale/styles/
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Use "dropdown list".'
+message: "Use 'dropdown list'."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html#dropdown-list
-level: suggestion
+level: warning
ignorecase: true
tokens:
- drop-down( [\w]*)?
diff --git a/doc/.vale/gitlab/EOLWhitespace.yml b/doc/.vale/gitlab/EOLWhitespace.yml
index e160b706014..153786443cc 100644
--- a/doc/.vale/gitlab/EOLWhitespace.yml
+++ b/doc/.vale/gitlab/EOLWhitespace.yml
@@ -3,9 +3,9 @@
#
# Checks that there is no useless whitespace at the end of lines.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Lines should not end with whitespace characters.'
+message: "Remove whitespace characters from the end of the line."
link: https://docs.gitlab.com/ee/development/documentation/versions.html
level: warning
scope: raw
diff --git a/doc/.vale/gitlab/ElementDescriptors.yml b/doc/.vale/gitlab/ElementDescriptors.yml
index 36f1202aef1..f3573f5ce65 100644
--- a/doc/.vale/gitlab/ElementDescriptors.yml
+++ b/doc/.vale/gitlab/ElementDescriptors.yml
@@ -3,9 +3,9 @@
#
# Suggests the correct way to describe elements in a form.
#
-# For a list of all options, see https://errata-ai.github.io/vale/styles/
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'When describing elements, %s "%s".'
+message: "When describing elements, %s '%s'."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#language
level: suggestion
ignorecase: true
diff --git a/doc/.vale/gitlab/FirstPerson.yml b/doc/.vale/gitlab/FirstPerson.yml
index 6fc93d9515c..e97b886b5ed 100644
--- a/doc/.vale/gitlab/FirstPerson.yml
+++ b/doc/.vale/gitlab/FirstPerson.yml
@@ -3,12 +3,12 @@
#
# Checks for use of first person pronouns.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: '"%s" is a first-person pronoun. Use second- or third-person pronouns (like we, you, us, one) instead.'
+message: "Instead of '%s', speak directly to the reader. Use 'you' or re-write to remove."
level: warning
ignorecase: true
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#usage-list
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html
tokens:
- '\bI[ ,;:?!"]|\bI\x27.{1,2}'
- me
diff --git a/doc/.vale/gitlab/FutureTense.yml b/doc/.vale/gitlab/FutureTense.yml
index 64e79612fff..d1484b20008 100644
--- a/doc/.vale/gitlab/FutureTense.yml
+++ b/doc/.vale/gitlab/FutureTense.yml
@@ -3,12 +3,12 @@
#
# Checks for use of future tense in sentences. Present tense is strongly preferred.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Avoid using future tense: "%s". Use present tense instead.'
+message: "Instead of future tense '%s', use present tense."
ignorecase: true
level: warning
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#usage-list
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html#future-tense
raw:
- "(going to( |\n|[[:punct:]])[a-zA-Z]*|"
- "will( |\n|[[:punct:]])[a-zA-Z]*|"
diff --git a/doc/.vale/gitlab/HeadingContent.yml b/doc/.vale/gitlab/HeadingContent.yml
index c2dd2a5c6c2..31ac3022934 100644
--- a/doc/.vale/gitlab/HeadingContent.yml
+++ b/doc/.vale/gitlab/HeadingContent.yml
@@ -3,12 +3,12 @@
#
# Checks for generic, unhelpful subheadings.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Rename the subheading "%s", or re-purpose the content elsewhere.'
+message: "Rename the heading '%s', or re-purpose the content elsewhere."
level: warning
scope: heading
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/#headings-1
+link: https://docs.gitlab.com/ee/development/documentation/topic_types/concept.html#concept-headings
ignorecase: false
tokens:
- How it works
diff --git a/doc/.vale/gitlab/HeadingDepth.yml b/doc/.vale/gitlab/HeadingDepth.yml
index 466ab317226..7a3e5b4b552 100644
--- a/doc/.vale/gitlab/HeadingDepth.yml
+++ b/doc/.vale/gitlab/HeadingDepth.yml
@@ -5,7 +5,7 @@
#
# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'The subheading "%s" is nested too deeply. Headings deeper than H5 suggest the section or page should be refactored.'
+message: "Refactor the section or page to avoid headings greater than H5."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#headings-in-markdown
level: warning
scope: raw
diff --git a/doc/.vale/gitlab/InclusionAbleism.yml b/doc/.vale/gitlab/InclusionAbleism.yml
index 1ebb6a97cf8..7419430c8a2 100644
--- a/doc/.vale/gitlab/InclusionAbleism.yml
+++ b/doc/.vale/gitlab/InclusionAbleism.yml
@@ -3,10 +3,10 @@
#
# Suggests alternatives for words that foster ableism.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'Use inclusive language. Consider "%s" instead of "%s".'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#inclusive-language
+message: "Use inclusive language. Consider '%s' instead of '%s'."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html
level: suggestion
ignorecase: true
swap:
diff --git a/doc/.vale/gitlab/InclusionCultural.yml b/doc/.vale/gitlab/InclusionCultural.yml
index 6f63d1725fc..6de838e7f25 100644
--- a/doc/.vale/gitlab/InclusionCultural.yml
+++ b/doc/.vale/gitlab/InclusionCultural.yml
@@ -3,10 +3,10 @@
#
# Suggests alternatives for words that are culturally inappropriate.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'Use inclusive language. Consider "%s" instead of "%s".'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#inclusive-language
+message: "Use inclusive language. Consider '%s' instead of '%s'."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html
level: warning
ignorecase: true
swap:
diff --git a/doc/.vale/gitlab/InclusionGender.yml b/doc/.vale/gitlab/InclusionGender.yml
index 313a1cc42dc..ce8861b6a09 100644
--- a/doc/.vale/gitlab/InclusionGender.yml
+++ b/doc/.vale/gitlab/InclusionGender.yml
@@ -3,10 +3,10 @@
#
# Suggests alternatives for words that are gender-specific.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'Use inclusive language. Consider "%s" instead of "%s".'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#inclusive-language
+message: "Use inclusive language. Consider '%s' instead of '%s'."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html
level: suggestion
ignorecase: true
swap:
diff --git a/doc/.vale/gitlab/InternalLinkCase.yml b/doc/.vale/gitlab/InternalLinkCase.yml
index c00e633426b..fded735812a 100644
--- a/doc/.vale/gitlab/InternalLinkCase.yml
+++ b/doc/.vale/gitlab/InternalLinkCase.yml
@@ -3,10 +3,10 @@
#
# Checks that anchor fragments on internal links are in lower-case.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Links to subheadings in GitLab docs must be in lower-case: "%s"'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links-to-internal-documentation
+message: "Use lowercase for the anchor link."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#anchor-links
level: error
scope: raw
raw:
diff --git a/doc/.vale/gitlab/InternalLinkExtension.yml b/doc/.vale/gitlab/InternalLinkExtension.yml
index 52142b50dfc..364263f90c8 100644
--- a/doc/.vale/gitlab/InternalLinkExtension.yml
+++ b/doc/.vale/gitlab/InternalLinkExtension.yml
@@ -3,10 +3,10 @@
#
# Checks that internal links have .md extenstion and not .html extension.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Link "%s" must link directly to a file and use the .md file extension.'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links-to-internal-documentation
+message: "Link to a file and use the .md file extension instead of .html."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links
level: error
scope: raw
raw:
diff --git a/doc/.vale/gitlab/InternalLinkFormat.yml b/doc/.vale/gitlab/InternalLinkFormat.yml
index b9ee83b7f5c..be09a020846 100644
--- a/doc/.vale/gitlab/InternalLinkFormat.yml
+++ b/doc/.vale/gitlab/InternalLinkFormat.yml
@@ -1,13 +1,13 @@
---
# Error: gitlab.InternalLinkFormat
#
-# Checks that internal link paths don't start with "./", which is not needed.
+# Checks that internal link paths don't start with './', which is not needed.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Link "%s" must not start with "./".'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links-to-internal-documentation
+message: "Edit the link so it does not start with './'."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links
level: error
scope: raw
raw:
- - '\[.+\]\(\.\/.*?\)'
+ - '\[[^\]]+\]\(\.\/.*?\)'
diff --git a/doc/.vale/gitlab/LatinTerms.yml b/doc/.vale/gitlab/LatinTerms.yml
index 12553a801a1..0bac0448bb1 100644
--- a/doc/.vale/gitlab/LatinTerms.yml
+++ b/doc/.vale/gitlab/LatinTerms.yml
@@ -3,10 +3,10 @@
#
# Checks for use of Latin terms.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'Use "%s" instead of "%s", but consider rewriting the sentence.'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#usage-list
+message: "Use '%s' instead of '%s', but consider rewriting the sentence."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html
level: warning
nonword: true
ignorecase: true
diff --git a/doc/.vale/gitlab/Markdown_emoji.yml b/doc/.vale/gitlab/Markdown_emoji.yml
index ac0dab2d69d..9873fb8becd 100644
--- a/doc/.vale/gitlab/Markdown_emoji.yml
+++ b/doc/.vale/gitlab/Markdown_emoji.yml
@@ -3,9 +3,9 @@
#
# Check for use of GLFM emoji syntax (https://docs.gitlab.com/ee/user/markdown.html#emojis), which doesn't render correctly in documentation.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'This appears to be GLFM emoji syntax. Replace "%s" with GitLab SVGs or Unicode emojis.'
+message: "Replace '%s' with GitLab SVGs or Unicode emojis."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/#gitlab-svg-icons
level: warning
scope: text
diff --git a/doc/.vale/gitlab/MeaningfulLinkWords.yml b/doc/.vale/gitlab/MeaningfulLinkWords.yml
index 4a255e5aae4..6fb41c7ce95 100644
--- a/doc/.vale/gitlab/MeaningfulLinkWords.yml
+++ b/doc/.vale/gitlab/MeaningfulLinkWords.yml
@@ -3,13 +3,13 @@
#
# Checks for the presence of semantically unhelpful words in link text.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Improve SEO and accessibility by rewriting "%s" in the link text.'
+message: "Improve SEO and accessibility by rewriting '%s' in the link text."
level: warning
scope: link
ignorecase: true
-link: https://about.gitlab.com/handbook/communication/#writing-style-guidelines
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#text-for-links
tokens:
- here
- this page
diff --git a/doc/.vale/gitlab/MergeConflictMarkers.yml b/doc/.vale/gitlab/MergeConflictMarkers.yml
index 4f216ac34c5..54e044f195d 100644
--- a/doc/.vale/gitlab/MergeConflictMarkers.yml
+++ b/doc/.vale/gitlab/MergeConflictMarkers.yml
@@ -3,9 +3,9 @@
#
# Checks for the presence of merge conflict markers.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Merge conflict marker "%s" found.'
+message: "Remove the merge conflict marker '%s'."
link: https://docs.gitlab.com/ee/development/code_review.html#merging-a-merge-request
level: error
scope: raw
diff --git a/doc/.vale/gitlab/MultiLineLinks.yml b/doc/.vale/gitlab/MultiLineLinks.yml
index 64ad017f16c..32fe38277dc 100644
--- a/doc/.vale/gitlab/MultiLineLinks.yml
+++ b/doc/.vale/gitlab/MultiLineLinks.yml
@@ -3,10 +3,10 @@
#
# Checks that links are all on a single line.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Link "%s" must be on a single line, even if very long.'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#basic-link-criteria
+message: "Put the full link on one line, even if the link is very long."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links
level: error
scope: raw
raw:
diff --git a/doc/.vale/gitlab/NonStandardQuotes.yml b/doc/.vale/gitlab/NonStandardQuotes.yml
index f31d615eb19..6161a4cc000 100644
--- a/doc/.vale/gitlab/NonStandardQuotes.yml
+++ b/doc/.vale/gitlab/NonStandardQuotes.yml
@@ -3,12 +3,12 @@
#
# Use only standard single and double quotes, not left or right quotes.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Use standard single quotes or double quotes only. Do not use left or right quotes.'
+message: "Use standard single quotes or double quotes only. Do not use left or right quotes."
level: warning
ignorecase: true
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#punctuation
scope: raw
raw:
- '[‘’“”]'
diff --git a/doc/.vale/gitlab/OutdatedVersions.yml b/doc/.vale/gitlab/OutdatedVersions.yml
index f25de44ad17..10fbaa0a676 100644
--- a/doc/.vale/gitlab/OutdatedVersions.yml
+++ b/doc/.vale/gitlab/OutdatedVersions.yml
@@ -3,10 +3,10 @@
#
# Checks for references to versions of GitLab that are no longer supported.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Can this reference to "%s" be refactored?'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#gitlab-versions
+message: "If possible, remove the reference to '%s'."
+link: https://docs.gitlab.com/ee/development/documentation/versions.html
level: suggestion
nonword: true
ignorecase: true
diff --git a/doc/.vale/gitlab/OxfordComma.yml b/doc/.vale/gitlab/OxfordComma.yml
index 1716145b26a..81a9ae5c1fc 100644
--- a/doc/.vale/gitlab/OxfordComma.yml
+++ b/doc/.vale/gitlab/OxfordComma.yml
@@ -3,9 +3,9 @@
#
# Checks for the lack of an Oxford comma. In some cases, will catch overly complex sentence structures with lots of commas.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Use a comma before the last "and" or "or" in a list of four or more items.'
+message: "Use a comma before the last 'and' or 'or' in a list of four or more items."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#punctuation
level: warning
raw:
diff --git a/doc/.vale/gitlab/Possessive.yml b/doc/.vale/gitlab/Possessive.yml
index 92ae66543a2..64c9481ac28 100644
--- a/doc/.vale/gitlab/Possessive.yml
+++ b/doc/.vale/gitlab/Possessive.yml
@@ -3,11 +3,11 @@
#
# The word GitLab should not be used in the possessive form.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: "Rewrite '%s' to not use 's."
+message: "Remove 's from %s."
level: error
ignorecase: true
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#trademark
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html#gitlab
tokens:
- GitLab's
diff --git a/doc/.vale/gitlab/ReadingLevel.yml b/doc/.vale/gitlab/ReadingLevel.yml
index cd7597ee8dc..a1ddebec1ea 100644
--- a/doc/.vale/gitlab/ReadingLevel.yml
+++ b/doc/.vale/gitlab/ReadingLevel.yml
@@ -4,8 +4,10 @@
# Checks the Flesch-Kincaid reading level.
#
# https://docs.errata.ai/vale/styles#metric
+#
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: metric
-message: "The grade level - %s - refers to how hard the content is to understand. Aim for 8th grade or lower by using shorter sentences and words."
+message: "The grade level is %s. Aim for 8th grade or lower by using shorter sentences and words."
link: https://docs.gitlab.com/ee/development/documentation/testing.html#vale-readability-score
level: suggestion
formula: |
diff --git a/doc/.vale/gitlab/ReferenceLinks.yml b/doc/.vale/gitlab/ReferenceLinks.yml
index ca9948844f8..d9f20fa1bd6 100644
--- a/doc/.vale/gitlab/ReferenceLinks.yml
+++ b/doc/.vale/gitlab/ReferenceLinks.yml
@@ -3,10 +3,10 @@
#
# Checks for reference-style links that should be converted to inline links.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Link "%s" must be inline.'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#basic-link-criteria
+message: "Put this link inline with the rest of the text."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links
level: error
scope: raw
raw:
diff --git a/doc/.vale/gitlab/RelativeLinks.yml b/doc/.vale/gitlab/RelativeLinks.yml
index 14ffc4bd0b8..6d46e432e08 100644
--- a/doc/.vale/gitlab/RelativeLinks.yml
+++ b/doc/.vale/gitlab/RelativeLinks.yml
@@ -3,11 +3,11 @@
#
# Checks for the presence of absolute hyperlinks that should be relative.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Link "%s" must be a relative link with a .md extension.'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links-to-internal-documentation
+message: "Use a relative link instead of a URL, and ensure the file name ends in .md and not .html."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links
level: error
scope: raw
raw:
- - '\[.+\]\(https?:\/\/docs\.gitlab\.com\/[ce]e.*\)'
+ - '\[[^\]]+\]\(https?:\/\/docs\.gitlab\.com\/[ce]e.*?\)'
diff --git a/doc/.vale/gitlab/RelativeLinksDoubleSlashes.yml b/doc/.vale/gitlab/RelativeLinksDoubleSlashes.yml
index ce6ce8b5691..6a94c7f927a 100644
--- a/doc/.vale/gitlab/RelativeLinksDoubleSlashes.yml
+++ b/doc/.vale/gitlab/RelativeLinksDoubleSlashes.yml
@@ -3,10 +3,10 @@
#
# Checks for the presence of double slashes in relative URLs.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Relative links must not include a double slash.'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links-to-internal-documentation
+message: "Remove the double slash from this relative link."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#links
level: error
scope: raw
raw:
diff --git a/doc/.vale/gitlab/Repetition.yml b/doc/.vale/gitlab/Repetition.yml
index c4b0cc14192..cdeb29e7d45 100644
--- a/doc/.vale/gitlab/Repetition.yml
+++ b/doc/.vale/gitlab/Repetition.yml
@@ -3,9 +3,9 @@
#
# Checks for duplicate words, like `the the` or `and and`.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: repetition
-message: '"%s" is repeated.'
+message: "Remove this duplicate word: '%s'."
level: error
alpha: true
tokens:
diff --git a/doc/.vale/gitlab/SentenceLength.yml b/doc/.vale/gitlab/SentenceLength.yml
index c60df1803e2..69b0d27072e 100644
--- a/doc/.vale/gitlab/SentenceLength.yml
+++ b/doc/.vale/gitlab/SentenceLength.yml
@@ -3,9 +3,9 @@
#
# Counts words in a sentence and alerts if a sentence exceeds 25 words.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: occurrence
-message: 'Shorter sentences improve readability (max 25 words).'
+message: "Improve readability by using fewer than 25 words in this sentence."
scope: sentence
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#language
level: warning
diff --git a/doc/.vale/gitlab/SentenceSpacing.yml b/doc/.vale/gitlab/SentenceSpacing.yml
index 0288abe834f..9ca685b00b8 100644
--- a/doc/.vale/gitlab/SentenceSpacing.yml
+++ b/doc/.vale/gitlab/SentenceSpacing.yml
@@ -3,9 +3,9 @@
#
# Checks for incorrect spacing (no spaces, or more than one space) around punctuation.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: '"%s" must contain one and only one space.'
+message: "Remove the extra space: '%s'"
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#punctuation
level: error
nonword: true
diff --git a/doc/.vale/gitlab/Simplicity.yml b/doc/.vale/gitlab/Simplicity.yml
index 44e78f89c20..89169c1aa46 100644
--- a/doc/.vale/gitlab/Simplicity.yml
+++ b/doc/.vale/gitlab/Simplicity.yml
@@ -3,12 +3,12 @@
#
# Checks for words implying ease of use, to avoid cognitive dissonance for frustrated users.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Avoid words like "%s" that imply ease of use, because the user may find this action hard.'
+message: "Remove '%s'. Be precise instead of subjective."
level: suggestion
ignorecase: true
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#usage-list
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html
tokens:
- easy
- easily
diff --git a/doc/.vale/gitlab/Spelling.yml b/doc/.vale/gitlab/Spelling.yml
index 4ebaf7bfb70..92c5cb13b29 100644
--- a/doc/.vale/gitlab/Spelling.yml
+++ b/doc/.vale/gitlab/Spelling.yml
@@ -8,9 +8,9 @@
# Commands, like `git clone` must use backticks, and must not be added to the
# exceptions.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: spelling
-message: 'Spelling check: "%s"?'
+message: "Check the spelling of '%s'. If the spelling is correct, add this word to the spelling exception list."
level: warning
ignore:
- gitlab/spelling-exceptions.txt
diff --git a/doc/.vale/gitlab/SubstitutionSuggestions.yml b/doc/.vale/gitlab/SubstitutionSuggestions.yml
index 4b77def065d..21cabf1e0a7 100644
--- a/doc/.vale/gitlab/SubstitutionSuggestions.yml
+++ b/doc/.vale/gitlab/SubstitutionSuggestions.yml
@@ -4,25 +4,26 @@
# Suggests better options for frequently misused terms that are often - but not always - incorrect.
# SubstitutionWarning.yml and Substitutions.yml also exist.
#
-# For a list of all options, see https://errata-ai.github.io/vale/styles/
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'Consider %s instead of "%s".'
+message: "Consider '%s' instead of '%s'."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html
level: suggestion
ignorecase: true
swap:
- active user: '"billable user"'
- active users: '"billable users"'
- docs: '"documentation"'
- e-mail: '"email"'
- GLFM: '"GitLab Flavored Markdown"'
- it is recommended: '"we recommend"'
+ active user: "billable user"
+ active users: "billable users"
+ docs: "documentation"
+ e-mail: "email"
+ GLFM: "GitLab Flavored Markdown"
+ it is recommended: "you should"
+ we recommend: "you should"
navigate: go
- OAuth2: '"OAuth 2.0"'
- once that: '"after that"'
- once the: '"after the"'
- once you: '"after you"'
- since: '"because" or "after"'
- sub-group: '"subgroup"'
- sub-groups: '"subgroups"'
- within: '"in"'
+ OAuth2: "OAuth 2.0"
+ once that: "after that"
+ once the: "after the"
+ once you: "after you"
+ since: "because' or 'after"
+ sub-group: "subgroup"
+ sub-groups: "subgroups"
+ within: "in"
diff --git a/doc/.vale/gitlab/SubstitutionWarning.yml b/doc/.vale/gitlab/SubstitutionWarning.yml
index d17015b97fd..8d6c18c1520 100644
--- a/doc/.vale/gitlab/SubstitutionWarning.yml
+++ b/doc/.vale/gitlab/SubstitutionWarning.yml
@@ -4,26 +4,26 @@
# Checks for misused terms or common shorthand that should never be used at GitLab, but can't be flagged as errors.
# Substitutions.yml and SubstitionSuggestions.yml also exist.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'If possible, use "%s" instead of "%s".'
+message: "If possible, use %s instead of '%s'."
link: https://about.gitlab.com/handbook/communication/#top-misused-terms
level: warning
ignorecase: true
swap:
- air(?:-| )?gapped: offline environment
- bullet: list item
- click: select
- code base: codebase
- config: configuration
- deselect: clear
- deselected: cleared
- distro: distribution
- file name: filename
- filesystem: file system
- GFM: GLFM
- info: information
- n/a: not applicable
- repo: repository
- timezone: time zone
- utilize: use
+ air(?:-| )?gapped: "offline environment"
+ bullet: "list item"
+ click: "select"
+ code base: "codebase"
+ config: "configuration"
+ deselect: "clear"
+ deselected: "cleared"
+ distro: "distribution"
+ file name: "filename"
+ filesystem: "file system"
+ GFM: "'GitLab Flavored Markdown' or 'GitHub Flavored Markdown'"
+ info: "information"
+ n/a: "not applicable"
+ repo: "repository"
+ timezone: "time zone"
+ utilize: "use"
diff --git a/doc/.vale/gitlab/Substitutions.yml b/doc/.vale/gitlab/Substitutions.yml
index af426fa7e06..92791486491 100644
--- a/doc/.vale/gitlab/Substitutions.yml
+++ b/doc/.vale/gitlab/Substitutions.yml
@@ -4,9 +4,9 @@
# Checks for misused terms that should never be used at GitLab.
# SubstitutionWarning.yml and SubstitionSuggestions.yml also exist.
#
-# For a list of all options, see https://docs.errata.ai/vale/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'Use "%s" instead of "%s".'
+message: "Use '%s' instead of '%s'."
link: https://about.gitlab.com/handbook/communication/#top-misused-terms
level: error
ignorecase: true
diff --git a/doc/.vale/gitlab/ToDo.yml b/doc/.vale/gitlab/ToDo.yml
index a3725cb7f6b..079f13baa28 100644
--- a/doc/.vale/gitlab/ToDo.yml
+++ b/doc/.vale/gitlab/ToDo.yml
@@ -3,10 +3,10 @@
#
# You should not use "To Do", unless it refers to the UI element.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: 'Use "to-do item" in most cases, or "Add a to do" if referring to the UI button.'
-link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#feature-names
+message: "Use 'to-do item' in most cases, or 'Add a to do' if referring to the UI button."
+link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html#to-do-item
level: warning
ignorecase: false
swap:
diff --git a/doc/.vale/gitlab/UnclearAntecedent.yml b/doc/.vale/gitlab/UnclearAntecedent.yml
index 5f238598d9f..e5d43b6ab7d 100644
--- a/doc/.vale/gitlab/UnclearAntecedent.yml
+++ b/doc/.vale/gitlab/UnclearAntecedent.yml
@@ -3,9 +3,9 @@
#
# Checks for words that need a noun for clarity.
#
-# For a list of all options, see https://docs.errata.ai/vale/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: "'%s' is not precise. Try rewriting with a specific subject and verb."
+message: "Instead of '%s', try starting this sentence with a specific subject and verb."
link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html#this-these-that-those
level: warning
ignorecase: false
diff --git a/doc/.vale/gitlab/Units.yml b/doc/.vale/gitlab/Units.yml
new file mode 100644
index 00000000000..4211fdee38b
--- /dev/null
+++ b/doc/.vale/gitlab/Units.yml
@@ -0,0 +1,15 @@
+---
+# Suggestion: gitlab.Units
+#
+# Recommends a space between a number and a unit of measure.
+#
+# For a list of all options, see https://vale.sh/docs/topics/styles/
+extends: existence
+message: "Add a space between the number and the unit in '%s'."
+link: 'https://docs.gitlab.com/ee/development/documentation/styleguide/'
+nonword: true
+level: suggestion
+ignorecase: true
+tokens:
+ - \d+(?:B|kB|KiB|MB|MiB|GB|GiB|TB|TiB)
+ - \d+(?:ns|ms|μs|s|min|h|d)
diff --git a/doc/.vale/gitlab/Uppercase.yml b/doc/.vale/gitlab/Uppercase.yml
index dc05aa05730..f53e1c72dcb 100644
--- a/doc/.vale/gitlab/Uppercase.yml
+++ b/doc/.vale/gitlab/Uppercase.yml
@@ -3,9 +3,9 @@
#
# Checks for use of all uppercase letters with unknown reason.
#
-# For a list of all options, see https://docs.errata.ai/vale/styles.
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: conditional
-message: "'%s' is uppercase. Use lowercase or `backticks` if possible. Otherwise add this word to the rule's exception list."
+message: "Instead of uppercase for '%s', use lowercase or backticks (`) if possible. Otherwise, add this word or acronym to the rule's exception list."
link: https://docs.gitlab.com/ee/development/documentation/testing.html#vale-uppercase-acronym-test
level: warning
ignorecase: false
@@ -25,6 +25,7 @@ exceptions:
- ARN
- ASCII
- ASG
+ - AST
- AWS
- BMP
- BSD
@@ -144,6 +145,7 @@ exceptions:
- NOTE
- NPM
- NTP
+ - OCI
- OKD
- ONLY
- OSS
diff --git a/doc/.vale/gitlab/VersionText.yml b/doc/.vale/gitlab/VersionText.yml
index 571fba52ab7..4fd80bc3f4e 100644
--- a/doc/.vale/gitlab/VersionText.yml
+++ b/doc/.vale/gitlab/VersionText.yml
@@ -9,9 +9,9 @@
# - `> Introduced in GitLab 14.0.
# - `> Removed in GitLab 15.0.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'This introduced-in section is not formatted correctly. Each entry must start with `> -` and long entries must be on one line.'
+message: "Start each entry with `> -`. Keep long entries on one line."
link: https://docs.gitlab.com/ee/development/documentation/versions.html
level: error
scope: raw
diff --git a/doc/.vale/gitlab/VersionTextSingleLine.yml b/doc/.vale/gitlab/VersionTextSingleLine.yml
index f76574bcf8a..552ccb9951e 100644
--- a/doc/.vale/gitlab/VersionTextSingleLine.yml
+++ b/doc/.vale/gitlab/VersionTextSingleLine.yml
@@ -3,9 +3,9 @@
#
# Verifies that single-item version notes don't have a hyphen.
#
-# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: existence
-message: 'Version text with only a single item must not start with a hyphen.'
+message: "Do not use a hyphen '-' in version text if there is only a single item."
link: https://docs.gitlab.com/ee/development/documentation/versions.html#add-a-version-history-item
level: error
scope: raw
diff --git a/doc/.vale/gitlab/Wordy.yml b/doc/.vale/gitlab/Wordy.yml
index 7888d16dadb..45546435ee9 100644
--- a/doc/.vale/gitlab/Wordy.yml
+++ b/doc/.vale/gitlab/Wordy.yml
@@ -3,15 +3,15 @@
#
# Suggests shorter versions of wordy phrases.
#
-# For a list of all options, see https://docs.errata.ai/vale/styles
+# For a list of all options, see https://vale.sh/docs/topics/styles/
extends: substitution
-message: '%s "%s".'
+message: "%s"
link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html
level: suggestion
ignorecase: true
swap:
- in order to: "Be concise: use 'to' rather than"
- needs? to: "Rewrite the sentence, or use 'must', instead of"
- note that: "Be concise: rewrite the sentence to not use"
- please: "Remove this word from the sentence: "
- respectively: "Rewrite the sentence to be more precise, instead of using "
+ note that: "Remove the phrase 'note that'."
+ please: "Use 'please' only if we've inconvenienced the user."
+ respectively: "Remove 'respectively' and list each option instead."
+ and so on: "Remove 'and so on'. Try to use 'like' and provide examples instead."
+ in order to: "Remove 'in order' and leave 'to'."
diff --git a/doc/.vale/gitlab/spelling-exceptions.txt b/doc/.vale/gitlab/spelling-exceptions.txt
index 5d268fdb241..0fc7c9703ac 100644
--- a/doc/.vale/gitlab/spelling-exceptions.txt
+++ b/doc/.vale/gitlab/spelling-exceptions.txt
@@ -546,6 +546,7 @@ relicensing
remediations
renderers
replicables
+rpcs
repmgr
repmgrd
repurposing
@@ -570,6 +571,7 @@ roadmap
roadmaps
rollout
rollouts
+RSpec
rsync
rsynced
rsyncing
@@ -613,6 +615,7 @@ Silverlight
Sisense
Sitespeed
skippable
+skopeo
Slack
Slackbot
Slony
diff --git a/doc/.vale/vale-json.tmpl b/doc/.vale/vale-json.tmpl
index ed3c3259df3..7969cb704a0 100644
--- a/doc/.vale/vale-json.tmpl
+++ b/doc/.vale/vale-json.tmpl
@@ -1,47 +1,28 @@
{{- /* Modify Vale's output https://docs.errata.ai/vale/cli#--output */ -}}
-{{- /* Keep track of our various counts */ -}}
+{{- $fileIndexes := len .Files -}}
+{{- $fileIndexes = sub $fileIndexes 1 -}}
-{{- $e := 0 -}}
-{{- $w := 0 -}}
-{{- $s := 0 -}}
-{{- $f := 0 -}}
-
-{{- /* Range over the linted files */ -}}
-
-{{- range .Files}}
-
-{{- $f = add1 $f -}}
-{{- $path := .Path -}}
-
-{{- /* Range over the file's alerts */ -}}
[
-
-{{- range $idx, $a := .Alerts -}}
-
-{{- $error := "" -}}
-{{- if eq .Severity "error" -}}
- {{- $error = .Severity -}}
- {{- $e = add1 $e -}}
-{{- else if eq .Severity "warning" -}}
- {{- $error = .Severity -}}
- {{- $w = add1 $w -}}
-{{- else -}}
- {{- $error = .Severity -}}
- {{- $s = add1 $s -}}
-{{- end}}
-
-{{- /* Variables setup */ -}}
-
-{{- $path = $path -}}
-{{- $loc := printf "%d" .Line -}}
-{{- $check := printf "%s" .Check -}}
-{{- $message := printf "%s" .Message -}}
-{{- $link := printf "%s" .Link -}}
-{{- if $idx -}},{{- end -}}
-
-{{- /* Output */ -}}
-
+ {{- /* Range over the linted files */ -}}
+ {{- range $idx1, $a := .Files -}}
+ {{- $path := .Path -}}
+
+ {{/* Range over the file's alerts */}}
+ {{- range $idx2, $b := .Alerts -}}
+ {{- $error := "info" -}}
+ {{- if eq .Severity "error" -}}
+ {{- $error = "blocker" -}}
+ {{- else if eq .Severity "warning" -}}
+ {{- $error = "major" -}}
+ {{- end}}
+
+ {{- /* Variables setup */ -}}
+ {{- $loc := printf "%d" .Line -}}
+ {{- $message := printf "%s" .Message -}}
+ {{- if $idx2 -}},{{- end -}}
+
+ {{/* Output */}}
{
"description": "{{ $message }}",
"fingerprint": "{{ $path }}-{{ $loc }}",
@@ -53,6 +34,6 @@
}
}
}
-{{end -}}
-{{end -}}
+ {{- end}}{{- if (lt $idx1 $fileIndexes) -}},{{- end -}}
+ {{- end}}
]
diff --git a/doc/administration/audit_event_streaming.md b/doc/administration/audit_event_streaming.md
index ca60b6142fe..0af1af12a60 100644
--- a/doc/administration/audit_event_streaming.md
+++ b/doc/administration/audit_event_streaming.md
@@ -312,10 +312,7 @@ Streamed audit events have a predictable schema in the body of the response.
> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/357211) in GitLab 15.0.
> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/357211) in GitLab 15.1 by default.
> - [Added `details.author_class` field](https://gitlab.com/gitlab-org/gitlab/-/issues/363876) in GitLab 15.3.
-
-FLAG:
-On self-managed GitLab, by default this feature is available. To hide the
-feature, ask an administrator to [disable the feature flag](feature_flags.md) named `audit_event_streaming_git_operations`.
+> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101583) in GitLab 15.6. Feature flag `audit_event_streaming_git_operations` removed.
Streaming audit events can be sent when signed-in users push, pull, or clone a project's remote Git repositories:
diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md
index 11600bc2da6..0aa0d163972 100644
--- a/doc/administration/audit_events.md
+++ b/doc/administration/audit_events.md
@@ -4,37 +4,22 @@ group: Compliance
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Audit Events **(PREMIUM)**
+# Audit events **(PREMIUM)**
-GitLab offers a way to view the changes made within the GitLab server for owners and administrators
-on a [paid plan](https://about.gitlab.com/pricing/).
+Use audit events to track important events, including who performed the related action and when.
+You can use audit events to track, for example:
-GitLab system administrators can also view all audit events by accessing the [`audit_json.log` file](logs/index.md#audit_jsonlog).
-The JSON audit log does not include events that are [only streamed](../development/audit_event_guide/index.md#event-streaming).
+- Who changed the permission level of a particular user for a GitLab project, and when.
+- Who added a new user or removed a user, and when.
-You can:
+The GitLab API, database, and `audit_json.log` record many audit events. Some audit events are only available through
+[streaming audit events](audit_event_streaming.md).
-- Generate an [audit report](audit_reports.md) of audit events.
-- [Stream audit events](audit_event_streaming.md) to an external endpoint.
+You can also generate an [audit report](audit_reports.md) of audit events.
-## Overview
-
-**Audit Events** is a tool for GitLab owners and administrators
-to track important events such as who performed certain actions and the
-time they happened. For example, these actions could be a change to a user
-permission level, who added a new user, or who removed a user.
-
-## Use cases
-
-- Check who changed the permission level of a particular
- user for a GitLab project.
-- Track which users have access to a certain group of projects
- in GitLab, and who gave them that permission level.
-
-## Retention policy
-
-There is no retention policy in place for audit events.
-See the [Specify a retention period for audit events](https://gitlab.com/groups/gitlab-org/-/epics/7917) for more information.
+NOTE:
+You can't configure a retention policy for audit events, but epic
+[7917](https://gitlab.com/groups/gitlab-org/-/epics/7917) proposes to change this.
## List of events
@@ -115,7 +100,7 @@ From there, you can see the following actions:
- Instance administrator started or stopped impersonation of a group member. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300961) in GitLab 14.8.
- Group deploy token was successfully created, revoked, or deleted. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353452) in GitLab 14.9.
- Failed attempt to create a group deploy token. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353452) in GitLab 14.9.
-- [IP restrictions](../user/group/access_and_permissions.md#restrict-group-access-by-ip-address) changed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/358986) in GitLab 15.0.
+- [IP restrictions](../user/group/access_and_permissions.md#restrict-access-to-groups-by-ip-address) changed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/358986) in GitLab 15.0.
- Changes to push rules. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227629) in GitLab 15.0.
- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356152) in GitLab 15.1, changes to the following merge request approvals settings:
- Prevent approval by author.
@@ -124,6 +109,7 @@ From there, you can see the following actions:
- Require user password to approve.
- Remove all approvals when commits are added to the source branch.
- Changes to streaming audit destination custom HTTP headers. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/366350) in GitLab 15.3.
+- Group had a security policy project linked, changed, or unlinked. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/377877) in GitLab 15.6)
Group events can also be accessed via the [Group Audit Events API](../api/audit_events.md#group-audit-events)
@@ -194,6 +180,7 @@ From there, you can see the following actions:
- Squash commit message template is updated ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355805) in GitLab 15.0)
- Default description template for merge requests is updated ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355805) in GitLab 15.0)
- Project was scheduled for deletion due to inactivity ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85689) in GitLab 15.0)
+- Project had a security policy project linked, changed, or unlinked ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/377877) in GitLab 15.6)
Project events can also be accessed via the [Project Audit Events API](../api/audit_events.md#project-audit-events).
diff --git a/doc/administration/auth/authentiq.md b/doc/administration/auth/authentiq.md
index 1ac62b06fe7..d51601439f9 100644
--- a/doc/administration/auth/authentiq.md
+++ b/doc/administration/auth/authentiq.md
@@ -95,6 +95,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/administration/auth/jwt.md b/doc/administration/auth/jwt.md
index c7e7253ef72..c1e76d1c2ed 100644
--- a/doc/administration/auth/jwt.md
+++ b/doc/administration/auth/jwt.md
@@ -87,6 +87,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/administration/auth/ldap/google_secure_ldap.md b/doc/administration/auth/ldap/google_secure_ldap.md
index 2077c6f7baf..01197fdacdf 100644
--- a/doc/administration/auth/ldap/google_secure_ldap.md
+++ b/doc/administration/auth/ldap/google_secure_ldap.md
@@ -225,6 +225,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md
index 3bb9350e960..6243f3da2d2 100644
--- a/doc/administration/auth/ldap/index.md
+++ b/doc/administration/auth/ldap/index.md
@@ -179,7 +179,7 @@ These configuration settings are available:
| `allow_username_or_email_login` | If enabled, GitLab ignores everything after the first `@` in the LDAP username submitted by the user on sign-in. If you are using `uid: 'userPrincipalName'` on ActiveDirectory you must disable this setting, because the userPrincipalName contains an `@`. | **{dotted-circle}** No | boolean |
| `block_auto_created_users` | To maintain tight control over the number of billable users on your GitLab installation, enable this setting to keep new users blocked until they have been cleared by an administrator (default: false). | **{dotted-circle}** No | boolean |
| `base` | Base where we can search for users. | **{check-circle}** Yes | `'ou=people,dc=gitlab,dc=example'` or `'DC=mydomain,DC=com'` |
-| `user_filter` | Filter LDAP users. Format: [RFC 4515](https://tools.ietf.org/search/rfc4515) Note: GitLab does not support `omniauth-ldap`'s custom filter syntax. | **{dotted-circle}** No | For examples, read [Examples of user filters](#examples-of-user-filters). |
+| `user_filter` | Filter LDAP users. Format: [RFC 4515](https://www.rfc-editor.org/rfc/rfc4515.html) Note: GitLab does not support `omniauth-ldap`'s custom filter syntax. | **{dotted-circle}** No | For examples, read [Examples of user filters](#examples-of-user-filters). |
| `lowercase_usernames` | If enabled, GitLab converts the name to lower case. | **{dotted-circle}** No | boolean |
| `retry_empty_result_with_codes` | An array of LDAP query response code that attempt to retry the operation if the result/content is empty. For Google Secure LDAP, set this value to `[80]`. | **{dotted-circle}** No | `[80]` |
@@ -281,7 +281,7 @@ This example results in a sign-in page with the following tabs:
To limit all GitLab access to a subset of the LDAP users on your LDAP server, first narrow the
configured `base`. However, to further filter users if
-necessary, you can set up an LDAP user filter. The filter must comply with [RFC 4515](https://tools.ietf.org/search/rfc4515).
+necessary, you can set up an LDAP user filter. The filter must comply with [RFC 4515](https://www.rfc-editor.org/rfc/rfc4515.html).
- Example user filter for Omnibus GitLab instances:
@@ -336,7 +336,7 @@ The `user_filter` DN can contain special characters. For example:
```
These characters must be escaped as documented in
- [RFC 4515](https://tools.ietf.org/search/rfc4515).
+ [RFC 4515](https://www.rfc-editor.org/rfc/rfc4515.html).
- Escape commas with `\2C`. For example:
diff --git a/doc/administration/auth/ldap/ldap-troubleshooting.md b/doc/administration/auth/ldap/ldap-troubleshooting.md
index 0ec482648a9..499c3c64af7 100644
--- a/doc/administration/auth/ldap/ldap-troubleshooting.md
+++ b/doc/administration/auth/ldap/ldap-troubleshooting.md
@@ -196,7 +196,7 @@ same user) has the email `email@example.com` set as a secondary email, which
is throwing this error.
We can check where this conflicting email address is coming from using the
-[rails console](#rails-console). Once in the console, run the following:
+[rails console](#rails-console). In the console, run the following:
```ruby
# This searches for an email among the primary AND secondary emails
@@ -546,7 +546,7 @@ this entry, it could be due to a mismatched DN stored in GitLab. See
```shell
User with DN `uid=john0,ou=people,dc=example,dc=com` should have access
to 'my_group' group but there is no user in GitLab with that
-identity. Membership will be updated once the user signs in for
+identity. Membership will be updated when the user signs in for
the first time.
```
@@ -556,7 +556,7 @@ Finally, the following entry says syncing has finished for this group:
Finished syncing all providers for 'my_group' group
```
-Once all the configured group links have been synchronized, GitLab looks
+When all the configured group links have been synchronized, GitLab looks
for any Administrators or External users to sync:
```shell
@@ -614,6 +614,16 @@ ldap_group.member_dns
ldap_group.member_uids
```
+#### LDAP synchronization does not remove group creator from group
+
+[LDAP synchronization](ldap_synchronization.md) should remove an LDAP group's creator
+from that group, if that user does not exist in the group. If running LDAP synchronization
+does not do this:
+
+1. Add the user to the LDAP group.
+1. Wait until LDAP group synchronization has finished running.
+1. Remove the user from the LDAP group.
+
### User DN or/and email have changed
When an LDAP user is created in GitLab, their LDAP DN is stored for later reference.
@@ -735,7 +745,7 @@ To resolve this error, you must apply a new license to the GitLab instance witho
1. Remove or comment out the GitLab configuration lines for all non-primary LDAP servers.
1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) so that it temporarily uses only one LDAP server.
-1. Enter the [Rails console and add the license key](../../troubleshooting/gitlab_rails_cheat_sheet.md#add-a-license-through-the-console).
+1. Enter the [Rails console and add the license key](../../../user/admin_area/license_file.md#add-a-license-through-the-console).
1. Re-enable the additional LDAP servers in the GitLab configuration and reconfigure GitLab again.
## Debugging Tools
diff --git a/doc/administration/auth/ldap/ldap_synchronization.md b/doc/administration/auth/ldap/ldap_synchronization.md
index af2b1400670..02b04861844 100644
--- a/doc/administration/auth/ldap/ldap_synchronization.md
+++ b/doc/administration/auth/ldap/ldap_synchronization.md
@@ -182,16 +182,18 @@ group, GitLab revokes their `admin` role when syncing.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1793) in GitLab 12.0.
-"Lock memberships to LDAP synchronization" setting allows instance administrators
-to lock down user abilities to invite new members to a group.
+GitLab administrators can prevent group members from inviting new members to subgroups that have their membership synchronized with LDAP.
-When enabled, the following applies:
+Global group membership lock only applies to subgroups of the top-level group where LDAP synchronization is configured. No user can modify the
+membership of a top-level group configured for LDAP synchronization.
+
+When global group memberships lock is enabled:
- Only an administrator can manage memberships of any group including access levels.
- Users are not allowed to share a project with other groups or invite members to
a project created in a group.
-To enable it, you must:
+To enable global group memberships lock:
1. [Configure LDAP](index.md#configure-ldap).
1. On the top bar, select **Main menu > Admin**.
diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md
index bb263512e06..1f73d8bff38 100644
--- a/doc/administration/auth/oidc.md
+++ b/doc/administration/auth/oidc.md
@@ -154,7 +154,7 @@ different providers with Omnibus GitLab.
### Configure Google
-See the [Google documentation](https://developers.google.com/identity/protocols/oauth2/openid-connect)
+See the [Google documentation](https://developers.google.com/identity/openid-connect/openid-connect)
for more details:
```ruby
@@ -502,7 +502,7 @@ For your app, complete the following steps on Casdoor:
ensure the Casdoor app has the following
`Redirect URI`: `https://gitlab.example.com/users/auth/openid_connect/callback`.
-See the [Casdoor documentation](https://casdoor.org/docs/integration/gitlab) for more details.
+See the [Casdoor documentation](https://casdoor.org/docs/integration/ruby/gitlab) for more details.
Example Omnibus GitLab configuration (file path: `/etc/gitlab/gitlab.rb`):
diff --git a/doc/administration/auth/smartcard.md b/doc/administration/auth/smartcard.md
index 11117e8a74c..5b6d299f171 100644
--- a/doc/administration/auth/smartcard.md
+++ b/doc/administration/auth/smartcard.md
@@ -342,6 +342,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/administration/cicd.md b/doc/administration/cicd.md
index 6899b572e8f..ad0671d4c13 100644
--- a/doc/administration/cicd.md
+++ b/doc/administration/cicd.md
@@ -101,6 +101,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md
index ad345461776..ee7476ed6d4 100644
--- a/doc/administration/compliance.md
+++ b/doc/administration/compliance.md
@@ -48,9 +48,9 @@ settings and automation to ensure that whatever a compliance team has configured
stays configured and working correctly. These features can help you automate
compliance:
-- [**Compliance frameworks**](../user/group/manage.md#compliance-frameworks) (for groups): Create a custom
+- [**Compliance frameworks**](../user/group/compliance_frameworks.md) (for groups): Create a custom
compliance framework at the group level to describe the type of compliance requirements any child project needs to follow.
-- [**Compliance pipelines**](../user/group/manage.md#configure-a-compliance-pipeline) (for groups): Define a
+- [**Compliance pipelines**](../user/group/compliance_frameworks.md#configure-a-compliance-pipeline) (for groups): Define a
pipeline configuration to run for any projects with a given compliance framework.
## Audit management
diff --git a/doc/administration/configure.md b/doc/administration/configure.md
index 40f03d25e8d..91fec753f7e 100644
--- a/doc/administration/configure.md
+++ b/doc/administration/configure.md
@@ -30,7 +30,7 @@ The tables lists features on the left and provides their capabilities to the rig
## Geo Capabilities
-If your availability needs to span multiple zones or multiple locations, please read about [Geo](geo/index.md).
+If your availability needs to span multiple zones or multiple locations, read about [Geo](geo/index.md).
| | Availability | Recoverability | Data Resiliency | Performance | Risks/Trade-offs|
|-|--------------|----------------|-----------------|-------------|-----------------|
@@ -46,5 +46,5 @@ The following table outlines failure modes and mitigation paths for the product
| Single Gitaly Node + Geo Secondary | Downtime - Must restore from backup, can perform a manual failover to secondary | Downtime - Must restore from Backup, errors could have propagated to secondary | Manual intervention - failover to Geo secondary | |
| Sharded Gitaly Install | Partial Downtime - Only repositories on impacted node affected, must restore from backup | Partial Downtime - Only repositories on impacted node affected, must restore from backup | Downtime - Must wait for outage to end | |
| Sharded Gitaly Install + Geo Secondary | Partial Downtime - Only repositories on impacted node affected, must restore from backup, could perform manual failover to secondary for impacted repositories | Partial Downtime - Only repositories on impacted node affected, must restore from backup, errors could have propagated to secondary | Manual intervention - failover to Geo secondary | |
-| Gitaly Cluster Install* | No Downtime - will swap repository primary to another node after 10 seconds | Not applicable; All writes are voted on by multiple Gitaly Cluster nodes | Downtime - Must wait for outage to end | Snapshot backups for Gitaly Cluster nodes not supported at this time |
-| Gitaly Cluster Install* + Geo Secondary | No Downtime - will swap repository primary to another node after 10 seconds | Not applicable; All writes are voted on by multiple Gitaly Cluster nodes | Manual intervention - failover to Geo secondary | Snapshot backups for Gitaly Cluster nodes not supported at this time |
+| Gitaly Cluster Install* | No Downtime - swaps repository primary to another node after 10 seconds | Not applicable; All writes are voted on by multiple Gitaly Cluster nodes | Downtime - Must wait for outage to end | Snapshot backups for Gitaly Cluster nodes not supported at this time |
+| Gitaly Cluster Install* + Geo Secondary | No Downtime - swaps repository primary to another node after 10 seconds | Not applicable; All writes are voted on by multiple Gitaly Cluster nodes | Manual intervention - failover to Geo secondary | Snapshot backups for Gitaly Cluster nodes not supported at this time |
diff --git a/doc/administration/consul.md b/doc/administration/consul.md
index a6f76882c4d..965231db440 100644
--- a/doc/administration/consul.md
+++ b/doc/administration/consul.md
@@ -8,7 +8,7 @@ type: reference
# How to set up Consul **(PREMIUM SELF)**
A Consul cluster consists of both
-[server and client agents](https://www.consul.io/docs/agent).
+[server and client agents](https://developer.hashicorp.com/consul/docs/agent).
The servers run on their own nodes and the clients run on other nodes that in
turn communicate with the servers.
@@ -99,7 +99,7 @@ Consul nodes communicate using the raft protocol. If the current leader goes
offline, there must be a leader election. A leader node must exist to facilitate
synchronization across the cluster. If too many nodes go offline at the same time,
the cluster loses quorum and doesn't elect a leader due to
-[broken consensus](https://www.consul.io/docs/architecture/consensus).
+[broken consensus](https://developer.hashicorp.com/consul/docs/architecture/consensus).
Consult the [troubleshooting section](#troubleshooting-consul) if the cluster is not
able to recover after the upgrade. The [outage recovery](#outage-recovery) may
@@ -148,7 +148,7 @@ you follow the Consul [outage recovery](#outage-recovery) process.
To be safe, it's recommended that you only restart Consul in one node at a time to
ensure the cluster remains intact. For larger clusters, it is possible to restart
multiple nodes at a time. See the
-[Consul consensus document](https://www.consul.io/docs/architecture/consensus#deployment-table)
+[Consul consensus document](https://developer.hashicorp.com/consul/docs/architecture/consensus#deployment-table)
for the number of failures it can tolerate. This is the number of simultaneous
restarts it can sustain.
@@ -161,7 +161,7 @@ sudo gitlab-ctl restart consul
### Consul nodes unable to communicate
By default, Consul attempts to
-[bind](https://www.consul.io/docs/agent/config/config-files#bind_addr) to `0.0.0.0`, but
+[bind](https://developer.hashicorp.com/consul/docs/agent/config/config-files#bind_addr) to `0.0.0.0`, but
it advertises the first private IP address on the node for other Consul nodes
to communicate with it. If the other nodes cannot communicate with a node on
this address, then the cluster has a failed status.
@@ -249,5 +249,5 @@ Shortly after that, the client agents should rejoin as well.
If you have taken advantage of Consul to store other data and want to restore
the failed node, follow the
-[Consul guide](https://learn.hashicorp.com/tutorials/consul/recovery-outage)
+[Consul guide](https://developer.hashicorp.com/consul/tutorials/datacenter-operations/recovery-outage)
to recover a failed cluster.
diff --git a/doc/administration/environment_variables.md b/doc/administration/environment_variables.md
index beaef7afe57..d2edc3085f1 100644
--- a/doc/administration/environment_variables.md
+++ b/doc/administration/environment_variables.md
@@ -50,5 +50,5 @@ To set environment variables, follow [these instructions](https://docs.gitlab.co
It's possible to preconfigure the GitLab Docker image by adding the environment
variable `GITLAB_OMNIBUS_CONFIG` to the `docker run` command.
-For more information, see the [Pre-configure Docker container](https://docs.gitlab.com/omnibus/docker/#pre-configure-docker-container)
+For more information, see the [Pre-configure Docker container](../install/docker.md#pre-configure-docker-container)
section of the Omnibus GitLab documentation.
diff --git a/doc/administration/feature_flags.md b/doc/administration/feature_flags.md
index 4f8f7c22a55..f2a40b60536 100644
--- a/doc/administration/feature_flags.md
+++ b/doc/administration/feature_flags.md
@@ -118,10 +118,10 @@ Some feature flags can be enabled or disabled on a per project basis:
Feature.enable(:<feature flag>, Project.find(<project id>))
```
-For example, to enable the [`:product_analytics`](../operations/product_analytics.md) feature flag for project `1234`:
+For example, to enable the `:my_awesome_feature` feature flag for project `1234`:
```ruby
-Feature.enable(:product_analytics, Project.find(1234))
+Feature.enable(:my_awesome_feature, Project.find(1234))
```
`Feature.enable` and `Feature.disable` always return `true`, even if the application doesn't use the flag:
diff --git a/doc/administration/file_hooks.md b/doc/administration/file_hooks.md
index 8fafb258593..f8b55a7c680 100644
--- a/doc/administration/file_hooks.md
+++ b/doc/administration/file_hooks.md
@@ -7,10 +7,8 @@ type: reference
# File hooks **(FREE SELF)**
-> Renamed feature from Plugins to File hooks in GitLab 12.8.
-
-With custom file hooks, GitLab administrators can introduce custom integrations
-without modifying the GitLab source code.
+Use custom file hooks (not to be confused with [server hooks](server_hooks.md) or [system hooks](system_hooks.md)),
+to introduce custom integrations without modifying the GitLab source code.
A file hook runs on each event. You can filter events or projects
in a file hook's code, and create many file hooks as you need. Each file hook is
diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md
index 699fe6851eb..99ac95cd536 100644
--- a/doc/administration/geo/disaster_recovery/background_verification.md
+++ b/doc/administration/geo/disaster_recovery/background_verification.md
@@ -122,7 +122,7 @@ be resynced with a back-off period. If you want to reset them manually, this
Rake task marks projects where verification has failed or the checksum mismatch
to be resynced without the back-off period:
-Run the appropriate commands on a **Rails node on the primary** site.
+Run the appropriate commands on a **Rails node on the secondary** site.
For repositories:
diff --git a/doc/administration/geo/disaster_recovery/index.md b/doc/administration/geo/disaster_recovery/index.md
index 156a87614e6..dfa8d09e6ef 100644
--- a/doc/administration/geo/disaster_recovery/index.md
+++ b/doc/administration/geo/disaster_recovery/index.md
@@ -758,7 +758,7 @@ If you are running GitLab 14.4 and earlier:
To promote the **secondary** cluster to a **primary** cluster, update `role: secondary` to `role: primary`.
- If the cluster remains as a primary site, you can remove the entire `psql` section; it refers to the tracking database and is ignored whilst the cluster is acting as a primary site.
+ If the cluster remains as a primary site, you can remove the entire `psql` section; it refers to the tracking database and is ignored while the cluster is acting as a primary site.
Update the cluster with the new configuration:
diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md
index 60101f5af8e..80707afacca 100644
--- a/doc/administration/geo/disaster_recovery/planned_failover.md
+++ b/doc/administration/geo/disaster_recovery/planned_failover.md
@@ -14,12 +14,12 @@ downtime.
As replication between Geo sites is asynchronous, a planned failover requires
a maintenance window in which updates to the **primary** site are blocked. The
-length of this window is determined by your replication capacity - once the
+length of this window is determined by your replication capacity - when the
**secondary** site is completely synchronized with the **primary** site, the failover can occur without
data loss.
This document assumes you already have a fully configured, working Geo setup.
-Please read it and the [Disaster Recovery](index.md) failover
+Read this document and the [Disaster Recovery](index.md) failover
documentation in full before proceeding. Planned failover is a major operation,
and if performed incorrectly, there is a high risk of data loss. Consider
rehearsing the procedure until you are comfortable with the necessary steps and
diff --git a/doc/administration/geo/index.md b/doc/administration/geo/index.md
index a336f5ff8bc..68fd0c63e37 100644
--- a/doc/administration/geo/index.md
+++ b/doc/administration/geo/index.md
@@ -120,18 +120,18 @@ The following are required to run Geo:
- [CentOS](https://www.centos.org) 7.4 or later
- [Ubuntu](https://ubuntu.com) 16.04 or later
- PostgreSQL 12 or 13 with [Streaming Replication](https://wiki.postgresql.org/wiki/Streaming_Replication)
- - Note,[PostgreSQL 12 is deprecated](../../update/deprecations.md#postgresql-12-deprecated) and will be removed in GitLab 16.0.
+ - Note,[PostgreSQL 12 is deprecated](../../update/deprecations.md#postgresql-12-deprecated) and is removed in GitLab 16.0.
- Git 2.9 or later
- Git-lfs 2.4.2 or later on the user side when using LFS
- All sites must run [the same GitLab and PostgreSQL versions](setup/database.md#postgresql-replication).
-- If using different operating system versions between Geo sites, [check OS locale data compatibility](replication/troubleshooting.md#check-os-locale-data-compatibility) across Geo sites.
+- If using different operating system versions between Geo sites, [check OS locale data compatibility](replication/troubleshooting.md#check-os-locale-data-compatibility) across Geo sites.
Additionally, check the GitLab [minimum requirements](../../install/requirements.md),
-and we recommend you use the latest version of GitLab for a better experience.
+and use the latest version of GitLab for a better experience.
### Firewall rules
-The following table lists basic ports that must be open between the **primary** and **secondary** sites for Geo. To simplify failovers, we recommend opening ports in both directions.
+The following table lists basic ports that must be open between the **primary** and **secondary** sites for Geo. To simplify failovers, you should open ports in both directions.
| Source site | Source port | Destination site | Destination port | Protocol |
|-------------|-------------|------------------|------------------|-------------|
@@ -306,7 +306,7 @@ For an example of how to set up a location-aware Git remote URL with AWS Route53
### Backfill
-Once a **secondary** site is set up, it starts replicating missing data from
+When a **secondary** site is set up, it starts replicating missing data from
the **primary** site in a process known as **backfill**. You can monitor the
synchronization process on each Geo site from the **primary** site's **Geo Nodes**
dashboard in your browser.
diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md
index fa74f16cdc8..55c5d3784c2 100644
--- a/doc/administration/geo/replication/configuration.md
+++ b/doc/administration/geo/replication/configuration.md
@@ -12,7 +12,7 @@ type: howto
NOTE:
This is the final step in setting up a **secondary** Geo site. Stages of the
setup process must be completed in the documented order.
-If not, [complete all prior stages](../setup/index.md#using-omnibus-gitlab) before procceed.
+If not, [complete all prior stages](../setup/index.md#using-omnibus-gitlab) before proceeding.
Make sure you [set up the database replication](../setup/database.md), and [configured fast lookup of authorized SSH keys](../../operations/fast_ssh_key_lookup.md) in **both primary and secondary sites**.
@@ -239,8 +239,9 @@ keys must be manually replicated to the **secondary** site.
If any of the checks fail, check the [troubleshooting documentation](troubleshooting.md).
-Once added to the Geo administration page and restarted, the **secondary** site automatically starts
-replicating missing data from the **primary** site in a process known as **backfill**.
+After the **secondary** site is added to the Geo administration page and restarted,
+the site automatically starts replicating missing data from the **primary** site
+in a process known as **backfill**.
Meanwhile, the **primary** site starts to notify each **secondary** site of any changes, so
that the **secondary** site can act on those notifications immediately.
diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md
index 566df2ee509..0198d2a63e8 100644
--- a/doc/administration/geo/replication/datatypes.md
+++ b/doc/administration/geo/replication/datatypes.md
@@ -201,8 +201,7 @@ successfully, you must replicate their data using some other means.
|[CI job artifacts](../../../ci/pipelines/job_artifacts.md) | **Yes** (10.4) | **Yes** (14.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Verification is behind the feature flag `geo_job_artifact_replication`, enabled by default in 14.10. |
|[CI Pipeline Artifacts](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/pipeline_artifact.rb) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Persists additional artifacts after a pipeline completes. |
|[CI Secure Files](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/secure_file.rb) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [No](object_storage.md#verification-of-files-in-object-storage) | Verification is behind the feature flag `geo_ci_secure_file_replication`, enabled by default in 15.3. |
-|[Container Registry](../../packages/container_registry.md) | **Yes** (12.3)* | No | No | No | Replication is behind feature flag `geo_container_repository_replication`, enabled by default.
-Requires additional configuration. See [instructions](container_registry.md) to set up the Container Registry replication. |
+|[Container Registry](../../packages/container_registry.md) | **Yes** (12.3)* | No | No | No | Replication is behind feature flag `geo_container_repository_replication`, enabled by default. Requires additional configuration. See [instructions](container_registry.md) to set up the Container Registry replication. |
|[Infrastructure Registry](../../../user/packages/infrastructure_registry/index.md) | **Yes** (14.0) | **Yes** (14.0) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Behind feature flag `geo_package_file_replication`, enabled by default. |
|[Project designs repository](../../../user/project/issues/design_management.md) | **Yes** (12.7) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/32467) | N/A | N/A | Designs also require replication of LFS objects and Uploads. |
|[Package Registry](../../../user/packages/package_registry/index.md) | **Yes** (13.2) | **Yes** (13.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Behind feature flag `geo_package_file_replication`, enabled by default. |
diff --git a/doc/administration/geo/replication/disable_geo.md b/doc/administration/geo/replication/disable_geo.md
index 3230a92136f..c42130a62a7 100644
--- a/doc/administration/geo/replication/disable_geo.md
+++ b/doc/administration/geo/replication/disable_geo.md
@@ -24,8 +24,8 @@ To disable Geo, follow these steps:
## Remove all secondary Geo sites
-To disable Geo, you need to first remove all your secondary Geo sites, which means replication will not happen
-anymore on these sites. You can follow our docs to [remove your secondary Geo sites](remove_geo_site.md).
+To disable Geo, you need to first remove all your secondary Geo sites, which means replication does not happen
+anymore on these sites. You can follow our documentation to [remove your secondary Geo sites](remove_geo_site.md).
If the current site that you want to keep using is a secondary site, you need to first promote it to primary.
You can use our steps on [how to promote a secondary site](../disaster_recovery/index.md#step-3-promoting-a-secondary-site)
diff --git a/doc/administration/geo/replication/docker_registry.md b/doc/administration/geo/replication/docker_registry.md
deleted file mode 100644
index d0af6f2a66f..00000000000
--- a/doc/administration/geo/replication/docker_registry.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: 'container_registry.md'
-remove_date: '2022-10-29'
----
-
-This document was moved to [another location](container_registry.md).
-
-<!-- This redirect file can be deleted after <2022-10-29>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> \ No newline at end of file
diff --git a/doc/administration/geo/replication/faq.md b/doc/administration/geo/replication/faq.md
index 81afcc19bb1..311cdeee5b9 100644
--- a/doc/administration/geo/replication/faq.md
+++ b/doc/administration/geo/replication/faq.md
@@ -22,7 +22,7 @@ For each project to sync:
1. Geo issues a `git fetch geo --mirror` to get the latest information from the **primary** site.
If there are no changes, the sync is fast. Otherwise, it has to pull the latest commits.
-1. The **secondary** site updates the tracking database to store the fact that it has synced projects A, B, C, and so on.
+1. The **secondary** site updates the tracking database to store the fact that it has synced projects by name.
1. Repeat until all projects are synced.
When someone pushes a commit to the **primary** site, it generates an event in the GitLab database that the repository has changed.
@@ -45,8 +45,8 @@ Read the documentation for [Disaster Recovery](../disaster_recovery/index.md).
## What data is replicated to a **secondary** site?
We currently replicate project repositories, LFS objects, generated
-attachments and avatars, and the whole database. This means user accounts,
-issues, merge requests, groups, project data, and so on, are available for
+attachments and avatars, and the whole database. This means information such as user accounts,
+issues, merge requests, groups, and project data are available for
query.
For more details, see the [supported Geo data types](datatypes.md).
@@ -58,8 +58,8 @@ Pushing directly to a **secondary** site (for both HTTP and SSH, including Git L
## How long does it take to have a commit replicated to a **secondary** site?
All replication operations are asynchronous and are queued to be dispatched. Therefore, it depends on a lot of
-factors including the amount of traffic, how big your commit is, the
-connectivity between your sites, your hardware, and so on.
+factors such as the amount of traffic, how big your commit is, the
+connectivity between your sites, and your hardware.
## What if the SSH server runs at a different port?
diff --git a/doc/administration/geo/replication/geo_validation_tests.md b/doc/administration/geo/replication/geo_validation_tests.md
index 8fa5a45b579..f09422d1e26 100644
--- a/doc/administration/geo/replication/geo_validation_tests.md
+++ b/doc/administration/geo/replication/geo_validation_tests.md
@@ -29,7 +29,7 @@ The following are GitLab upgrade validation tests we performed.
[Switch from repmgr to Patroni on a Geo primary site](https://gitlab.com/gitlab-org/gitlab/-/issues/224652):
- Description: Tested switching from repmgr to Patroni on a multi-node Geo primary site. Used [the orchestrator tool](https://gitlab.com/gitlab-org/gitlab-orchestrator) to deploy a Geo installation with 3 database nodes managed by repmgr. With this approach, we were also able to address a related issue for [verifying a Geo installation with Patroni and PostgreSQL 11](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5113).
-- Outcome: Partial success. We enabled Patroni on the primary site and set up database replication on the secondary site. However, we found that Patroni would delete the secondary site's replication slot whenever Patroni was restarted. Another issue is that when Patroni elects a new leader in the cluster, the secondary site will fail to automatically follow the new leader. Until these issues are resolved, we cannot officially support and recommend Patroni for Geo installations.
+- Outcome: Partial success. We enabled Patroni on the primary site and set up database replication on the secondary site. However, we found that Patroni would delete the secondary site's replication slot whenever Patroni was restarted. Another issue is that when Patroni elects a new leader in the cluster, the secondary site fails to automatically follow the new leader. Until these issues are resolved, we cannot officially support and recommend Patroni for Geo installations.
- Follow up issues/actions:
- [Investigate permanent replication slot for Patroni with Geo single node secondary](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5528)
@@ -213,7 +213,7 @@ The following are additional validation tests we performed.
[Validate Object storage replication using GCP based object storage](https://gitlab.com/gitlab-org/gitlab/-/issues/351464):
- Description: Tested the average time it takes for a single image to replicate from the primary object storage location to the secondary when using GCP based object storage replication and [GitLab based object storage replication](object_storage.md#enabling-gitlab-managed-object-storage-replication). This was tested by uploading a 1mb image to a project on the primary site every second for 60 seconds. The time was then measured until a image was available on the secondary site. This was achieved using a [Ruby Script](https://gitlab.com/gitlab-org/quality/geo-replication-tester).
-- Outcome: GCP handles replication differently than other Cloud Providers. In GCP, the process is to a create single bucket that is either multi, dual or single region based. This means that the bucket will automatically store replicas in a region based on the option chosen. Even when using multi region, this will still only replicate within a single continent, the options being America, Europe, or Asia. At current there doesn't seem to be any way to replicate objects between continents using GCP based replication. For Geo managed replication the average time when replicating within the same region was 6 seconds, and when replicating cross region this rose to just 9 seconds.
+- Outcome: GCP handles replication differently than other Cloud Providers. In GCP, the process is to a create single bucket that is either multi, dual, or single region based. This means that the bucket automatically stores replicas in a region based on the option chosen. Even when using multi region, this only replicates in a single continent, the options being America, Europe, or Asia. At current there doesn't seem to be any way to replicate objects between continents using GCP based replication. For Geo managed replication the average time when replicating in the same region was 6 seconds, and when replicating cross region this rose to just 9 seconds.
## Other tests
diff --git a/doc/administration/geo/replication/location_aware_git_url.md b/doc/administration/geo/replication/location_aware_git_url.md
index e0e113eebbd..dbe543f5a62 100644
--- a/doc/administration/geo/replication/location_aware_git_url.md
+++ b/doc/administration/geo/replication/location_aware_git_url.md
@@ -31,7 +31,7 @@ In this example, we have already set up:
- `primary.example.com` as a Geo **primary** site.
- `secondary.example.com` as a Geo **secondary** site.
-We will create a `git.example.com` subdomain that will automatically direct
+We create a `git.example.com` subdomain that automatically directs
requests:
- From Europe to the **secondary** site.
diff --git a/doc/administration/geo/replication/remove_geo_site.md b/doc/administration/geo/replication/remove_geo_site.md
index 62b1d9fdf7b..4b9f31dc08c 100644
--- a/doc/administration/geo/replication/remove_geo_site.md
+++ b/doc/administration/geo/replication/remove_geo_site.md
@@ -14,7 +14,8 @@ type: howto
1. Select the **Remove** button for the **secondary** site you want to remove.
1. Confirm by selecting **Remove** when the prompt appears.
-Once removed from the Geo administration page, you must stop and uninstall the **secondary** site. For each node on your secondary Geo site:
+After the **secondary** site is removed from the Geo administration page, you must
+stop and uninstall this site. For each node on your secondary Geo site:
1. Stop GitLab:
@@ -35,7 +36,7 @@ Once removed from the Geo administration page, you must stop and uninstall the *
sudo rpm --erase gitlab-ee
```
-Once GitLab has been uninstalled from each node on the **secondary** site, the replication slot must be dropped from the **primary** site's database as follows:
+When GitLab has been uninstalled from each node on the **secondary** site, the replication slot must be dropped from the **primary** site's database as follows:
1. On the **primary** site's database node, start a PostgreSQL console session:
diff --git a/doc/administration/geo/replication/security_review.md b/doc/administration/geo/replication/security_review.md
index 0231da53b9c..afe831dcb9c 100644
--- a/doc/administration/geo/replication/security_review.md
+++ b/doc/administration/geo/replication/security_review.md
@@ -25,8 +25,8 @@ from [owasp.org](https://owasp.org/).
### What data does the application receive, produce, and process?
- Geo streams almost all data held by a GitLab instance between sites. This
- includes full database replication, most files (user-uploaded attachments,
- and so on) and repository + wiki data. In a typical configuration, this will
+ includes full database replication, most files such as user-uploaded attachments,
+ and repository + wiki data. In a typical configuration, this will
happen across the public Internet, and be TLS-encrypted.
- PostgreSQL replication is TLS-encrypted.
- See also: [only TLSv1.2 should be supported](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/2948)
@@ -37,7 +37,7 @@ from [owasp.org](https://owasp.org/).
private projects. Geo replicates them all indiscriminately. "Selective sync"
exists for files and repositories (but not database content), which would permit
only less-sensitive projects to be replicated to a **secondary** site if desired.
-- See also: [GitLab data classification policy](https://about.gitlab.com/handbook/engineering/security/data-classification-standard.html).
+- See also: [GitLab data classification policy](https://about.gitlab.com/handbook/security/data-classification-standard.html).
### What data backup and retention requirements have been defined for the application?
@@ -59,8 +59,8 @@ from [owasp.org](https://owasp.org/).
(notably a HTTP/HTTPS web application, and HTTP/HTTPS or SSH Git repository
access), but is constrained to read-only activities. The principal use case is
envisioned to be cloning Git repositories from the **secondary** site in favor of the
- **primary** site, but end-users may use the GitLab web interface to view projects,
- issues, merge requests, snippets, and so on.
+ **primary** site, but end-users may use the GitLab web interface to view information like projects,
+ issues, merge requests, and snippets.
### What security expectations do the end‐users have?
diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md
index 3f16c1552ad..fa668091c90 100644
--- a/doc/administration/geo/replication/troubleshooting.md
+++ b/doc/administration/geo/replication/troubleshooting.md
@@ -12,8 +12,9 @@ miss a step.
Here is a list of steps you should take to attempt to fix problem:
1. Perform [basic troubleshooting](#basic-troubleshooting).
-1. Fix any [replication errors](#fixing-replication-errors).
+1. Fix any [PostgreSQL database replication errors](#fixing-postgresql-database-replication-errors).
1. Fix any [common](#fixing-common-errors) errors.
+1. Fix any [non-PostgreSQL replication failures](#fixing-non-postgresql-replication-failures).
## Basic troubleshooting
@@ -131,6 +132,8 @@ http://secondary.example.com/
To find more details about failed items, check
[the `gitlab-rails/geo.log` file](../../logs/log_parsing.md#find-most-common-geo-sync-errors)
+If you notice replication or verification failures, you can try to [resolve them](#fixing-non-postgresql-replication-failures).
+
### Check if PostgreSQL replication is working
To check if PostgreSQL replication is working, check if:
@@ -185,6 +188,41 @@ This machine's Geo node name matches a database record ... no
Learn more about recommended site names in the description of the Name field in
[Geo Admin Area Common Settings](../../../user/admin_area/geo_sites.md#common-settings).
+### Reverify all uploads (or any SSF data type which is verified)
+
+1. SSH into a GitLab Rails node in the primary Geo site.
+1. Open [Rails console](../../operations/rails_console.md).
+1. Mark all uploads as "pending verification":
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+ ```ruby
+ Upload.verification_state_table_class.each_batch do |relation|
+ relation.update_all(verification_state: 0)
+ end
+ ```
+
+1. This will cause the primary to start checksumming all Uploads.
+1. When a primary successfully checksums a record, then all secondaries rechecksum as well, and they compare the values.
+
+A similar thing can be done for all Models handled by the [Geo Self-Service Framework](../../../development/geo/framework.md) which have implemented verification:
+
+- `LfsObject`
+- `MergeRequestDiff`
+- `Packages::PackageFile`
+- `Terraform::StateVersion`
+- `SnippetRepository`
+- `Ci::PipelineArtifact`
+- `PagesDeployment`
+- `Upload`
+- `Ci::JobArtifact`
+- `Ci::SecureFile`
+
+NOTE:
+`GroupWikiRepository` is not in the previous list since verification is not implemented.
+There is an [issue to implement this functionality in the Admin Area UI](https://gitlab.com/gitlab-org/gitlab/-/issues/364729).
+
### Message: `WARNING: oldest xmin is far in the past` and `pg_wal` size growing
If a replication slot is inactive,
@@ -311,52 +349,41 @@ sudo gitlab-rake gitlab:geo:check
When performing a PostgreSQL major version (9 > 10) update this is expected. Follow
the [initiate-the-replication-process](../setup/database.md#step-3-initiate-the-replication-process).
-### Repository verification failures
+### Message: Machine clock is synchronized ... Exception
-[Start a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
-to gather the following, basic troubleshooting information.
+The Rake task attempts to verify that the server clock is synchronized with NTP. Synchronized clocks
+are required for Geo to function correctly. As an example, for security, when the server time on the
+primary site and secondary site differ by about a minute or more, requests between Geo sites
+will fail. If this check task fails to complete due to a reason other than mismatching times, it
+does not necessarily mean that Geo will not work.
-WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+The Ruby gem which performs the check is hard coded with `pool.ntp.org` as its reference time source.
-#### Get the number of verification failed repositories
+- Exception message `Machine clock is synchronized ... Exception: Timeout::Error`
-```ruby
-Geo::ProjectRegistry.verification_failed('repository').count
-```
+ This issue occurs when your server cannot access the host `pool.ntp.org`.
-#### Find the verification failed repositories
+- Exception message `Machine clock is synchronized ... Exception: No route to host - recvfrom(2)`
-```ruby
-Geo::ProjectRegistry.verification_failed('repository')
-```
+ This issue occurs when the hostname `pool.ntp.org` resolves to a server which does not provide a time service.
-#### Find repositories that failed to sync
+There is [an issue open](https://gitlab.com/gitlab-org/gitlab/-/issues/381422) for this dependency on `pool.ntp.org`.
-```ruby
-Geo::ProjectRegistry.sync_failed('repository')
-```
-
-### Resync repositories
-
-[Start a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
-to enact the following, basic troubleshooting steps.
-
-#### Queue up all repositories for resync. Sidekiq handles each sync
+To workaround this, do one of the following:
-```ruby
-Geo::ProjectRegistry.update_all(resync_repository: true, resync_wiki: true)
-```
+- Add entries in `/etc/hosts` for `pool.ntp.org` to direct the request to valid local time servers.
+ This fixes the long timeout and the timeout error.
+- Direct the check to any valid IP address. This resolves the timeout issue, but the check will fail
+ with the `No route to host` error, as noted above.
-#### Sync individual repository now
+[Cloud native GitLab deployments](https://docs.gitlab.com/charts/advanced/geo/#set-the-geo-primary-site)
+generate an error because containers in Kubernetes do not have access to the host clock:
-```ruby
-project = Project.find_by_full_path('<group/project>')
-
-Geo::RepositorySyncService.new(project).execute
+```plaintext
+Machine clock is synchronized ... Exception: getaddrinfo: Servname not supported for ai_socktype
```
-## Fixing replication errors
+## Fixing PostgreSQL database replication errors
The following sections outline troubleshooting steps for fixing replication
error messages (indicated by `Database replication working? ... no` in the
@@ -469,7 +496,7 @@ This happens because the PostgreSQL certificate that the Omnibus GitLab package
the Common Name `PostgreSQL`, but the replication is connecting to a different host and GitLab attempts to use
the `verify-full` SSL mode by default.
-In order to fix this, you can either:
+To fix this issue, you can either:
- Use the `--sslmode=verify-ca` argument with the `replicate-geo-database` command.
- For an already replicated database, change `sslmode=verify-full` to `sslmode=verify-ca`
@@ -837,120 +864,6 @@ This behavior affects only the following data types through GitLab 14.6:
to make Geo visibly surface data loss risks. The sync/verification loop is
therefore short-circuited. `last_sync_failure` is now set to `The file is missing on the Geo primary site`.
-### Blob types
-
-- `Ci::JobArtifact`
-- `Ci::PipelineArtifact`
-- `Ci::SecureFile`
-- `LfsObject`
-- `MergeRequestDiff`
-- `Packages::PackageFile`
-- `PagesDeployment`
-- `Terraform::StateVersion`
-- `Upload`
-
-`Packages::PackageFile` is used in the following
-[Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
-examples, but things generally work the same for the other types.
-
-WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
-
-#### The Replicator
-
-The main kinds of classes are Registry, Model, and Replicator. If you have an instance of one of these classes, you can get the others. The Registry and Model mostly manage PostgreSQL DB state. The Replicator knows how to replicate/verify (or it can call a service to do it):
-
-```ruby
-model_record = Packages::PackageFile.last
-model_record.replicator.registry.replicator.model_record # just showing that these methods exist
-```
-
-#### Replicate a package file, synchronously, given an ID
-
-```ruby
-model_record = Packages::PackageFile.find(id)
-model_record.replicator.send(:download)
-```
-
-#### Replicate a package file, synchronously, given a registry ID
-
-```ruby
-registry = Geo::PackageFileRegistry.find(registry_id)
-registry.replicator.send(:download)
-```
-
-#### Verify package files on the secondary manually
-
-This iterates over all package files on the secondary, looking at the
-`verification_checksum` stored in the database (which came from the primary)
-and then calculate this value on the secondary to check if they match. This
-does not change anything in the UI:
-
-```ruby
-# Run on secondary
-status = {}
-
-Packages::PackageFile.find_each do |package_file|
- primary_checksum = package_file.verification_checksum
- secondary_checksum = Packages::PackageFile.hexdigest(package_file.file.path)
- verification_status = (primary_checksum == secondary_checksum)
-
- status[verification_status.to_s] ||= []
- status[verification_status.to_s] << package_file.id
-end
-
-# Count how many of each value we get
-status.keys.each {|key| puts "#{key} count: #{status[key].count}"}
-
-# See the output in its entirety
-status
-```
-
-### Repository types newer than project/wiki repositories
-
-- `SnippetRepository`
-- `GroupWikiRepository`
-
-`SnippetRepository` is used in the examples below, but things generally work the same for the other Repository types.
-
-#### The Replicator
-
-The main kinds of classes are Registry, Model, and Replicator. If you have an instance of one of these classes, you can get the others. The Registry and Model mostly manage PostgreSQL DB state. The Replicator knows how to replicate/verify (or it can call a service to do it).
-
-```ruby
-model_record = SnippetRepository.last
-model_record.replicator.registry.replicator.model_record # just showing that these methods exist
-```
-
-#### Replicate a snippet repository, synchronously, given an ID
-
-```ruby
-model_record = SnippetRepository.find(id)
-model_record.replicator.send(:sync_repository)
-```
-
-#### Replicate a snippet repository, synchronously, given a registry ID
-
-```ruby
-registry = Geo::SnippetRepositoryRegistry.find(registry_id)
-registry.replicator.send(:sync_repository)
-```
-
-### Find failed artifacts
-
-[Start a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
-to run the following commands:
-
-```ruby
-Geo::JobArtifactRegistry.failed
-```
-
-#### Find `ID` of synced artifacts that are missing on primary
-
-```ruby
-Geo::JobArtifactRegistry.synced.missing_on_primary.pluck(:artifact_id)
-```
-
#### Failed syncs with GitLab-managed object storage replication
There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467)
@@ -1218,7 +1131,8 @@ If you set up a new secondary from scratch, you must also [remove the old site f
The most common problems that prevent the database from replicating correctly are:
-- **Secondary** sites cannot reach the **primary** site. Check credentials, [firewall rules](../index.md#firewall-rules), and so on.
+- **Secondary** sites cannot reach the **primary** site. Check credentials and
+ [firewall rules](../index.md#firewall-rules).
- SSL certificate problems. Make sure you copied `/etc/gitlab/gitlab-secrets.json` from the **primary** site.
- Database storage disk is full.
- Database replication slot is misconfigured.
@@ -1320,6 +1234,217 @@ To fix this issue, set the primary site's internal URL to a URL that is:
GeoNode.where(primary: true).first.update!(internal_url: "https://unique.url.for.primary.site")
```
+## Fixing non-PostgreSQL replication failures
+
+If you notice replication failures in `Admin > Geo > Sites` or the [Sync status Rake task](#sync-status-rake-task), you can try to resolve the failures with the following general steps:
+
+1. Geo will automatically retry failures. If the failures are new and few in number, or if you suspect the root cause is already resolved, then you can wait to see if the failures go away.
+1. If failures were present for a long time, then many retries have already occurred, and the interval between automatic retries has increased to up to 4 hours depending on the type of failure. If you suspect the root cause is already resolved, you can [manually retry replication or verification](#manually-retry-replication-or-verification).
+1. If the failures persist, use the following sections to try to resolve them.
+
+### Manually retry replication or verification
+
+Project Git repositories and Project Wiki Git repositories have the ability in `Admin > Geo > Replication` to `Resync all`, `Reverify all`, or for a single resource, `Resync` or `Reverify`.
+
+Adding this ability to other data types is proposed in issue [364725](https://gitlab.com/gitlab-org/gitlab/-/issues/364725).
+
+The following sections describe how to use internal application commands in the [Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session) to cause replication or verification immediately.
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+### Blob types
+
+- `Ci::JobArtifact`
+- `Ci::PipelineArtifact`
+- `Ci::SecureFile`
+- `LfsObject`
+- `MergeRequestDiff`
+- `Packages::PackageFile`
+- `PagesDeployment`
+- `Terraform::StateVersion`
+- `Upload`
+
+`Packages::PackageFile` is used in the following
+[Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
+examples, but things generally work the same for the other types.
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+#### The Replicator
+
+The main kinds of classes are Registry, Model, and Replicator. If you have an instance of one of these classes, you can get the others. The Registry and Model mostly manage PostgreSQL DB state. The Replicator knows how to replicate/verify (or it can call a service to do it):
+
+```ruby
+model_record = Packages::PackageFile.last
+model_record.replicator.registry.replicator.model_record # just showing that these methods exist
+```
+
+#### Replicate a package file, synchronously, given an ID
+
+```ruby
+model_record = Packages::PackageFile.find(id)
+model_record.replicator.send(:download)
+```
+
+#### Replicate a package file, synchronously, given a registry ID
+
+```ruby
+registry = Geo::PackageFileRegistry.find(registry_id)
+registry.replicator.send(:download)
+```
+
+#### Verify package files on the secondary manually
+
+This iterates over all package files on the secondary, looking at the
+`verification_checksum` stored in the database (which came from the primary)
+and then calculate this value on the secondary to check if they match. This
+does not change anything in the UI:
+
+```ruby
+# Run on secondary
+status = {}
+
+Packages::PackageFile.find_each do |package_file|
+ primary_checksum = package_file.verification_checksum
+ secondary_checksum = Packages::PackageFile.hexdigest(package_file.file.path)
+ verification_status = (primary_checksum == secondary_checksum)
+
+ status[verification_status.to_s] ||= []
+ status[verification_status.to_s] << package_file.id
+end
+
+# Count how many of each value we get
+status.keys.each {|key| puts "#{key} count: #{status[key].count}"}
+
+# See the output in its entirety
+status
+```
+
+### Reverify all uploads (or any SSF data type which is verified)
+
+1. SSH into a GitLab Rails node in the primary Geo site.
+1. Open [Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session).
+1. Mark all uploads as "pending verification":
+
+ ```ruby
+ Upload.verification_state_table_class.each_batch do |relation|
+ relation.update_all(verification_state: 0)
+ end
+ ```
+
+1. This will cause the primary to start checksumming all Uploads.
+1. When a primary successfully checksums a record, then all secondaries rechecksum as well, and they compare the values.
+
+For other SSF data types replace `Upload` in the command above with the desired model class.
+
+NOTE:
+There is an [issue to implement this functionality in the Admin Area UI](https://gitlab.com/gitlab-org/gitlab/-/issues/364729).
+
+### Repository types, except for project or project wiki repositories
+
+- `SnippetRepository`
+- `GroupWikiRepository`
+
+`SnippetRepository` is used in the examples below, but things generally work the same for the other Repository types.
+
+[Start a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
+to enact the following, basic troubleshooting steps.
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+#### The Replicator
+
+The main kinds of classes are Registry, Model, and Replicator. If you have an instance of one of these classes, you can get the others. The Registry and Model mostly manage PostgreSQL DB state. The Replicator knows how to replicate/verify (or it can call a service to do it).
+
+```ruby
+model_record = SnippetRepository.last
+model_record.replicator.registry.replicator.model_record # just showing that these methods exist
+```
+
+#### Replicate a snippet repository, synchronously, given an ID
+
+```ruby
+model_record = SnippetRepository.find(id)
+model_record.replicator.send(:sync_repository)
+```
+
+#### Replicate a snippet repository, synchronously, given a registry ID
+
+```ruby
+registry = Geo::SnippetRepositoryRegistry.find(registry_id)
+registry.replicator.send(:sync_repository)
+```
+
+### Find failed artifacts
+
+[Start a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
+to run the following commands:
+
+```ruby
+Geo::JobArtifactRegistry.failed
+```
+
+#### Find `ID` of synced artifacts that are missing on primary
+
+```ruby
+Geo::JobArtifactRegistry.synced.missing_on_primary.pluck(:artifact_id)
+```
+
+### Project or project wiki repositories
+
+#### Find repository verification failures
+
+[Start a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
+to gather the following, basic troubleshooting information.
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+##### Get the number of verification failed repositories
+
+```ruby
+Geo::ProjectRegistry.verification_failed('repository').count
+```
+
+##### Find the verification failed repositories
+
+```ruby
+Geo::ProjectRegistry.verification_failed('repository')
+```
+
+##### Find repositories that failed to sync
+
+```ruby
+Geo::ProjectRegistry.sync_failed('repository')
+```
+
+#### Resync project and project wiki repositories
+
+[Start a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
+to enact the following, basic troubleshooting steps.
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+##### Queue up all repositories for resync
+
+When you run this, Sidekiq handles each sync.
+
+```ruby
+Geo::ProjectRegistry.update_all(resync_repository: true, resync_wiki: true)
+```
+
+##### Sync individual repository now
+
+```ruby
+project = Project.find_by_full_path('<group/project>')
+
+Geo::RepositorySyncService.new(project).execute
+```
+
## Fixing client errors
### Authorization errors from LFS HTTP(S) client requests
@@ -1390,10 +1515,6 @@ If the above steps are **not successful**, proceed through the next steps:
1. Verify you can connect to the newly-promoted **primary** site using the URL used previously for the **secondary** site.
1. If successful, the **secondary** site is now promoted to the **primary** site.
-## Additional tools
-
-There are useful snippets for manipulating Geo internals in the [GitLab Rails Cheat Sheet](../../troubleshooting/gitlab_rails_cheat_sheet.md#geo). For example, you can find how to manually sync or verify a replicable in Rails console.
-
## Check OS locale data compatibility
If different operating systems or different operating system versions are deployed across Geo sites, we recommend that you perform a locale data compatibility check setting up Geo.
diff --git a/doc/administration/geo/secondary_proxy/location_aware_external_url.md b/doc/administration/geo/secondary_proxy/location_aware_external_url.md
index b983230a314..b71b813ee9f 100644
--- a/doc/administration/geo/secondary_proxy/location_aware_external_url.md
+++ b/doc/administration/geo/secondary_proxy/location_aware_external_url.md
@@ -15,10 +15,6 @@ advantage of closer Geo sites as they move.
With [Geo proxying for secondary sites](index.md) web and Git requests are proxied
from **secondary** sites to the **primary**.
-Though these instructions use [AWS Route53](https://aws.amazon.com/route53/),
-other services such as [Cloudflare](https://www.cloudflare.com/) can be used
-as well.
-
## Prerequisites
This example creates a `gitlab.example.com` subdomain that automatically directs
@@ -36,17 +32,20 @@ For this example, you need:
- A working GitLab **primary** site that is accessible at `gitlab.example.com` _and_ `primary.example.com`.
- A working GitLab **secondary** site.
-- A Route53 Hosted Zone managing your domain for the Route53 setup.
+- A DNS zone managing your domain. Although the following instructions use
+ [AWS Route53](https://aws.amazon.com/route53/)
+ and [GCP cloud DNS](https://cloud.google.com/dns/), other services such as
+ [Cloudflare](https://www.cloudflare.com/) can be used as well.
If you haven't yet set up a Geo _primary_ site and _secondary_ site, see the
[Geo setup instructions](../index.md#setup-instructions).
## AWS Route53
-### Create a traffic policy
+In this example, you use a Route53 Hosted Zone managing your domain for the Route53 setup.
In a Route53 Hosted Zone, traffic policies can be used to set up a variety of
-routing configurations.
+routing configurations. To create a traffic policy:
1. Go to the
[Route53 dashboard](https://console.aws.amazon.com/route53/home) and select
@@ -78,6 +77,30 @@ routing configurations.
You have successfully set up a single host, like `gitlab.example.com`, which
distributes traffic to your Geo sites by geolocation.
+## GCP
+
+In this example, you create a GCP Cloud DNS zone managing your domain.
+
+When creating Geo-Based record sets, GCP applies a nearest match for the source region when the source of the traffic doesn't match any policy items exactly. To create a Geo-Based record set:
+
+1. Select **Network Services** > **Cloud DNS**.
+1. Select the Zone configured for your domain.
+1. Select **Add Record Set**.
+1. Enter the DNS Name for your Location-aware public URL e.g. `gitlab.example.com`.
+1. Select the **Routing Policy**: **Geo-Based**.
+1. Select **Add Managed RRData**.
+ 1. Select **Source Region**: **us-central1**.
+ 1. Enter your `<**primary** IP address>`.
+ 1. Select **Done**.
+1. Select **Add Managed RRData**.
+ 1. Select **Source Region**: **europe-west1**.
+ 1. Enter your `<**secondary** IP address>`.
+ 1. Select **Done**.
+1. Select **Create**.
+
+You have successfully set up a single host, like `gitlab.example.com`, which
+distributes traffic to your Geo sites using a location-aware URL.
+
## Enable Geo proxying for secondary sites
After setting up a single URL to use for all Geo sites, continue with the [steps to enable Geo proxying for secondary sites](index.md).
diff --git a/doc/administration/geo/setup/database.md b/doc/administration/geo/setup/database.md
index 8ea8d6c4d8e..86caf5306b5 100644
--- a/doc/administration/geo/setup/database.md
+++ b/doc/administration/geo/setup/database.md
@@ -478,6 +478,12 @@ data before running `pg_basebackup`.
Each Geo **secondary** site must have its own unique replication slot name.
Using the same slot name between two secondaries breaks PostgreSQL replication.
+ NOTE:
+ Replication slot names must only contain lowercase letters, numbers, and the underscore character.
+
+ When prompted, enter the _plaintext_ password you set up for the `gitlab_replicator`
+ user in the first step.
+
```shell
gitlab-ctl replicate-geo-database \
--slot-name=<secondary_site_name> \
@@ -486,12 +492,6 @@ data before running `pg_basebackup`.
```
NOTE:
- Replication slot names must only contain lowercase letters, numbers, and the underscore character.
-
- When prompted, enter the _plaintext_ password you set up for the `gitlab_replicator`
- user in the first step.
-
- NOTE:
If you have generated custom PostgreSQL certificates, you will want to use
`--sslmode=verify-full` (or omit the `sslmode` line entirely), to benefit from the extra
validation of the full host name in the certificate CN / SAN for additional security.
@@ -619,7 +619,7 @@ If you still haven't [migrated from repmgr to Patroni](#migrating-from-repmgr-to
1. Before migrating, we recommend that there is no replication lag between the **primary** and **secondary** sites and that replication is paused. In GitLab 13.2 and later, you can pause and resume replication with `gitlab-ctl geo-replication-pause` and `gitlab-ctl geo-replication-resume` on a Geo secondary database node.
1. Follow the [instructions to migrate repmgr to Patroni](../../postgresql/replication_and_failover.md#switching-from-repmgr-to-patroni). When configuring Patroni on each **primary** site database node, add `patroni['replication_slots'] = { '<slot_name>' => 'physical' }`
to `gitlab.rb` where `<slot_name>` is the name of the replication slot for your **secondary** site. This ensures that Patroni recognizes the replication slot as permanent and not drop it upon restarting.
-1. If database replication to the **secondary** site was paused before migration, resume replication once Patroni is confirmed working on the **primary** site.
+1. If database replication to the **secondary** site was paused before migration, resume replication after Patroni is confirmed working on the **primary** site.
### Migrating a single PostgreSQL node to Patroni
diff --git a/doc/administration/geo/setup/external_database.md b/doc/administration/geo/setup/external_database.md
index eabed7c10f3..0fefc11f078 100644
--- a/doc/administration/geo/setup/external_database.md
+++ b/doc/administration/geo/setup/external_database.md
@@ -62,7 +62,7 @@ developed and tested. We aim to be compatible with most external
To set up an external database, you can either:
-- Set up [streaming replication](https://www.postgresql.org/docs/12/warm-standby.html#STREAMING-REPLICATION-SLOTS) yourself (for example Amazon RDS, bare metal not managed by Omnibus, and so on).
+- Set up [streaming replication](https://www.postgresql.org/docs/12/warm-standby.html#STREAMING-REPLICATION-SLOTS) yourself (for example Amazon RDS, or bare metal not managed by Omnibus).
- Perform the Omnibus configuration manually as follows.
#### Leverage your cloud provider's tools to replicate the primary database
@@ -78,7 +78,7 @@ cloud providers:
- Azure Database for PostgreSQL - [Create and manage read replicas in Azure Database for PostgreSQL](https://learn.microsoft.com/en-us/azure/postgresql/single-server/how-to-read-replicas-portal)
- Google Cloud SQL - [Creating read replicas](https://cloud.google.com/sql/docs/postgres/replication/create-replica)
-Once your read-only replica is set up, you can skip to [configure your secondary site](#configure-secondary-site-to-use-the-external-read-replica)
+When your read-only replica is set up, you can skip to [configure your secondary site](#configure-secondary-site-to-use-the-external-read-replica)
#### Manually configure the primary database for replication
diff --git a/doc/administration/git_protocol.md b/doc/administration/git_protocol.md
index f900c5a6867..349a92de51e 100644
--- a/doc/administration/git_protocol.md
+++ b/doc/administration/git_protocol.md
@@ -25,7 +25,7 @@ From the server side, if we want to configure SSH we need to set the `sshd`
server to accept the `GIT_PROTOCOL` environment.
In installations using [GitLab Helm Charts](https://docs.gitlab.com/charts/)
-and [All-in-one Docker image](https://docs.gitlab.com/omnibus/docker/), the SSH
+and [All-in-one Docker image](../install/docker.md), the SSH
service is already configured to accept the `GIT_PROTOCOL` environment. Users
need not do anything more.
@@ -36,7 +36,7 @@ the SSH configuration of your server manually by adding this line to the `/etc/s
AcceptEnv GIT_PROTOCOL
```
-Once configured, restart the SSH daemon for the change to take effect:
+When you have configured the SSH daemon, restart it for the change to take effect:
```shell
# CentOS 6 / RHEL 6
diff --git a/doc/administration/gitaly/configure_gitaly.md b/doc/administration/gitaly/configure_gitaly.md
index e4aef2db9a8..e7a6fc35ced 100644
--- a/doc/administration/gitaly/configure_gitaly.md
+++ b/doc/administration/gitaly/configure_gitaly.md
@@ -57,21 +57,6 @@ The process for setting up Gitaly on its own server is:
1. [Configure Gitaly clients](#configure-gitaly-clients).
1. [Disable Gitaly where not required](#disable-gitaly-where-not-required-optional) (optional).
-When running Gitaly on its own server, note the following regarding GitLab versions:
-
-- From GitLab 11.4, Gitaly was able to serve all Git requests without requiring a shared NFS mount
- for Git repository data, except for the
- [Elasticsearch indexer](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer).
-- From GitLab 11.8, the Elasticsearch indexer also uses Gitaly for data access. NFS can still be
- leveraged for redundancy on block-level Git data, but should be mounted only on the Gitaly
- servers.
-- From GitLab 11.8 to 12.2, it is possible to use Elasticsearch in a Gitaly setup that doesn't use
- NFS. To use Elasticsearch in these versions, the
- [repository indexer](../../integration/advanced_search/elasticsearch.md#elasticsearch-repository-indexer)
- must be enabled in your GitLab configuration.
-- [In GitLab 12.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/6481), the new indexer is
- the default and no configuration is required.
-
### Network architecture
The following list depicts the network architecture of Gitaly:
@@ -1100,8 +1085,8 @@ benefit from it. It is orthogonal to:
- The transport (HTTP or SSH).
- Git protocol version (v0 or v2).
-- The type of fetch (full clones, incremental fetches, shallow clones,
- partial clones, and so on).
+- The type of fetch, such as full clones, incremental fetches, shallow clones,
+ or partial clones.
The strength of this cache is its ability to deduplicate concurrent
identical fetches. It:
@@ -1309,18 +1294,38 @@ following keys (in this example, to disable the `hasDotgit` consistency check):
- In [GitLab 15.3](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6800) and later:
```ruby
+ ignored_blobs = "/etc/gitlab/instance_wide_ignored_git_blobs.txt"
+
gitaly['gitconfig'] = [
+
+ # Populate a file with one unabbreviated SHA-1 per line.
+ # See https://git-scm.com/docs/git-config#Documentation/git-config.txt-fsckskipList
+ { key: "fsck.skipList", value: ignored_blobs },
+ { key: "fetch.fsck.skipList", value: ignored_blobs },
+ { key: "receive.fsck.skipList", value: ignored_blobs },
+
{ key: "fsck.hasDotgit", value: "ignore" },
{ key: "fetch.fsck.hasDotgit", value: "ignore" },
- { key: "receive.fsck.hasDotgit", value: "ignore "},
+ { key: "receive.fsck.hasDotgit", value: "ignore" },
+ { key: "fsck.missingSpaceBeforeEmail", value: "ignore" },
]
```
- In GitLab 15.2 and earlier (legacy method):
```ruby
- ignored_git_errors = ["hasDotgit = ignore"]
+ ignored_git_errors = [
+ "hasDotgit = ignore",
+ "missingSpaceBeforeEmail = ignore",
+ ]
omnibus_gitconfig['system'] = {
+
+ # Populate a file with one unabbreviated SHA-1 per line.
+ # See https://git-scm.com/docs/git-config#Documentation/git-config.txt-fsckskipList
+ "fsck.skipList" => ignored_blobs
+ "fetch.fsck.skipList" => ignored_blobs,
+ "receive.fsck.skipList" => ignored_blobs,
+
"fsck" => ignored_git_errors,
"fetch.fsck" => ignored_git_errors,
"receive.fsck" => ignored_git_errors,
@@ -1342,6 +1347,30 @@ value = "ignore"
[[git.config]]
key = "receive.fsck.hasDotgit"
value = "ignore"
+
+[[git.config]]
+key = "fsck.missingSpaceBeforeEmail"
+value = "ignore"
+
+[[git.config]]
+key = "fetch.fsck.missingSpaceBeforeEmail"
+value = "ignore"
+
+[[git.config]]
+key = "receive.fsck.missingSpaceBeforeEmail"
+value = "ignore"
+
+[[git.config]]
+key = "fsck.skipList"
+value = "/etc/gitlab/instance_wide_ignored_git_blobs.txt"
+
+[[git.config]]
+key = "fetch.fsck.skipList"
+value = "/etc/gitlab/instance_wide_ignored_git_blobs.txt"
+
+[[git.config]]
+key = "receive.fsck.skipList"
+value = "/etc/gitlab/instance_wide_ignored_git_blobs.txt"
```
## Configure commit signing for GitLab UI commits
diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md
index 96447239116..1b7e1f0f7c6 100644
--- a/doc/administration/gitaly/index.md
+++ b/doc/administration/gitaly/index.md
@@ -45,7 +45,7 @@ repository storage is either:
## Before deploying Gitaly Cluster
Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management.
-Before deploying Gitaly Cluster, please review:
+Before deploying Gitaly Cluster, review:
- Existing [known issues](#known-issues).
- [Snapshot limitations](#snapshot-backup-and-recovery-limitations).
@@ -66,13 +66,14 @@ Contact your Technical Account Manager or customer support if you have any quest
### Known issues
The following table outlines current known issues impacting the use of Gitaly Cluster. For
-the current status of these issues, please refer to the referenced issues and epics.
+the current status of these issues, refer to the referenced issues and epics.
| Issue | Summary | How to avoid |
|:--------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------|
| Gitaly Cluster + Geo - Issues retrying failed syncs | If Gitaly Cluster is used on a Geo secondary site, repositories that have failed to sync could continue to fail when Geo tries to resync them. Recovering from this state requires assistance from support to run manual steps. | No known solution prior to GitLab 15.0. In GitLab 15.0 to 15.2, enable the [`gitaly_praefect_generated_replica_paths` feature flag](#praefect-generated-replica-paths-gitlab-150-and-later). In GitLab 15.3, the feature flag is enabled by default. |
| Praefect unable to insert data into the database due to migrations not being applied after an upgrade | If the database is not kept up to date with completed migrations, then the Praefect node is unable to perform normal operation. | Make sure the Praefect database is up and running with all migrations completed (For example: `/opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-migrate-status` should show a list of all applied migrations). Consider [requesting upgrade assistance](https://about.gitlab.com/support/scheduling-upgrade-assistance/) so your upgrade plan can be reviewed by support. |
-| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup, it's best to [shut down GitLab](../read_only_gitlab.md#shut-down-the-gitlab-ui), snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. |
+| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind results in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup:<br/><br/>1. [Shut down GitLab](../read_only_gitlab.md#shut-down-the-gitlab-ui).<br/>2. Snapshot all Gitaly Cluster nodes at the same time.<br/>3. Take a database dump of the Praefect database. |
+| Rebuilding or replacing an existing Gitaly Cluster node | There is no way to replace existing nodes in place because the Praefect database is relied on to determine the current state of each Gitaly node. Changing how Gitaly Cluster stores repositories is proposed in issue [4218](https://gitlab.com/gitlab-org/gitaly/-/issues/4218). Turning on [repository verification](praefect.md#repository-verification) is proposed in issue [4429](https://gitlab.com/gitlab-org/gitaly/-/issues/4429) so Praefect can detect that data is missing and needs to replicated to a new Gitaly node. | No known solution at this time. |
### Snapshot backup and recovery limitations
@@ -83,11 +84,11 @@ during a restore, we recommend using the [official backup and restore Rake tasks
The [incremental backup method](../../raketasks/backup_gitlab.md#incremental-repository-backups)
can be used to speed up Gitaly Cluster backups.
-If you are unable to use either method, please contact customer support for restoration help.
+If you are unable to use either method, contact customer support for restoration help.
### What to do if you are on Gitaly Cluster experiencing an issue or limitation
-Please contact customer support for immediate help in restoration or recovery.
+Contact customer support for immediate help in restoration or recovery.
## Directly accessing repositories
@@ -219,8 +220,7 @@ The availability objectives for Gitaly clusters assuming a single node failure a
second. Failover requires ten consecutive failed health checks on each
Praefect node.
- Faster outage detection, to improve this speed to less than 1 second,
- is tracked [in this issue](https://gitlab.com/gitlab-org/gitaly/-/issues/2608).
+Improvements to RPO and RTO are proposed in epic [8903](https://gitlab.com/groups/gitlab-org/-/epics/8903).
WARNING:
If complete cluster failure occurs, disaster recovery plans should be executed. These can affect the
@@ -320,9 +320,7 @@ follow the [hashed storage](../repository_storage_types.md#hashed-storage) schem
> - [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/4218) in GitLab 15.0 [with a flag](../feature_flags.md) named `gitaly_praefect_generated_replica_paths`. Disabled by default.
> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitaly/-/issues/4218) in GitLab 15.2.
> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4809) in GitLab 15.3.
-
-FLAG:
-On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../feature_flags.md) named `gitaly_praefect_generated_replica_paths`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only.
+> - [Generally available](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4941) in GitLab 15.6. Feature flag `gitaly_praefect_generated_replica_paths` removed.
When Gitaly Cluster creates a repository, it assigns the repository a unique and permanent ID called the _repository ID_. The repository ID is
internal to Gitaly Cluster and doesn't relate to any IDs elsewhere in GitLab. If a repository is removed from Gitaly Cluster and later moved
@@ -410,7 +408,7 @@ relative path of the repository in the metadata store.
### Moving beyond NFS
Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be unavailable starting
-November 22, 2022. Please see our [statement of support](https://about.gitlab.com/support/statement-of-support/#gitaly-and-nfs)
+November 22, 2022. See our [statement of support](https://about.gitlab.com/support/statement-of-support/#gitaly-and-nfs)
for more details.
[Network File System (NFS)](https://en.wikipedia.org/wiki/Network_File_System)
@@ -470,12 +468,6 @@ including [horizontally distributing reads](https://gitlab.com/groups/gitlab-org
#### Distributed reads
-> - Introduced in GitLab 13.1 in [beta](../../policy/alpha-beta-support.md#beta-features) with feature flag `gitaly_distributed_reads` set to disabled.
-> - [Made generally available and enabled by default](https://gitlab.com/gitlab-org/gitaly/-/issues/2951) in GitLab 13.3.
-> - [Disabled by default](https://gitlab.com/gitlab-org/gitaly/-/issues/3178) in GitLab 13.5.
-> - [Enabled by default](https://gitlab.com/gitlab-org/gitaly/-/issues/3334) in GitLab 13.8.
-> - [Feature flag removed](https://gitlab.com/gitlab-org/gitaly/-/issues/3383) in GitLab 13.11.
-
Gitaly Cluster supports distribution of read operations across Gitaly nodes that are configured for
the [virtual storage](#virtual-storage).
@@ -496,31 +488,21 @@ You can [monitor distribution of reads](monitoring.md#monitor-gitaly-cluster) us
#### Strong consistency
-> - Introduced in GitLab 13.1 in [alpha](../../policy/alpha-beta-support.md#alpha-features), disabled by default.
-> - Entered [beta](../../policy/alpha-beta-support.md#beta-features) in GitLab 13.2, disabled by default.
-> - In GitLab 13.3, disabled unless primary-wins voting strategy is disabled.
-> - From GitLab 13.4, enabled by default.
-> - From GitLab 13.5, you must use Git v2.28.0 or higher on Gitaly nodes to enable strong consistency.
-> - From GitLab 13.6, primary-wins voting strategy and the `gitaly_reference_transactions_primary_wins` feature flag was removed.
-> - From GitLab 14.0, [Gitaly Cluster only supports strong consistency](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/3575), and the `gitaly_reference_transactions` feature flag was removed.
+> - In GitLab 13.6 to 13.12, strong consistency must be manually configured. Refer to [the 13.12 documentation](https://docs.gitlab.com/13.12/ee/administration/gitaly/praefect.html#strong-consistency).
+> - In GitLab 14.0, strong consistency is the primary replication method.
Gitaly Cluster provides strong consistency by writing changes synchronously to all healthy, up-to-date replicas. If a
replica is outdated or unhealthy at the time of the transaction, the write is asynchronously replicated to it.
+Strong consistency is the primary replication method. A subset of operations still use replication jobs
+(eventual consistency) instead of strong consistency. Refer to the
+[strong consistency epic](https://gitlab.com/groups/gitlab-org/-/epics/1189) for more information.
+
If strong consistency is unavailable, Gitaly Cluster guarantees eventual consistency. In this case. Gitaly Cluster
replicates all writes to secondary Gitaly nodes after the write to the primary Gitaly node has occurred.
-Strong consistency:
-
-- Is the primary replication method in GitLab 14.0 and later. A subset of operations still use replication jobs
- (eventual consistency) instead of strong consistency. Refer to the
- [strong consistency epic](https://gitlab.com/groups/gitlab-org/-/epics/1189) for more information.
-- Must be configured in GitLab versions 13.1 to 13.12. For configuration information, refer to either:
- - Documentation on your GitLab instance at `/help`.
- - The [13.12 documentation](https://docs.gitlab.com/13.12/ee/administration/gitaly/praefect.html#strong-consistency).
-- Is unavailable in GitLab 13.0 and earlier.
-
-For more information on monitoring strong consistency, see the Gitaly Cluster [Prometheus metrics documentation](monitoring.md#monitor-gitaly-cluster).
+For more information on monitoring strong consistency, see the Gitaly Cluster
+[Prometheus metrics documentation](monitoring.md#monitor-gitaly-cluster).
#### Replication factor
@@ -598,7 +580,7 @@ To migrate to Gitaly Cluster:
1. Create the required storage. Refer to
[repository storage recommendations](praefect.md#repository-storage-recommendations).
1. Create and configure [Gitaly Cluster](praefect.md).
-1. Configure the existing Gitaly instance [to use TPC](praefect.md#use-tcp-for-existing-gitlab-instances), if not already configured that way.
+1. Configure the existing Gitaly instance [to use TCP](praefect.md#use-tcp-for-existing-gitlab-instances), if not already configured that way.
1. [Move the repositories](../operations/moving_repositories.md#move-repositories). To migrate to
Gitaly Cluster, existing repositories stored outside Gitaly Cluster must be moved. There is no
automatic migration but the moves can be scheduled with the GitLab API.
@@ -718,4 +700,4 @@ The second facet presents the only real solution. For this, we developed
## NFS deprecation notice
Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
-unavailable beginning November 22, 2022. For further information, please see our [NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation) documentation.
+unavailable beginning November 22, 2022. For further information, see our [NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation) documentation.
diff --git a/doc/administration/gitaly/monitoring.md b/doc/administration/gitaly/monitoring.md
index 9b7acd536b3..8d4f30c7c20 100644
--- a/doc/administration/gitaly/monitoring.md
+++ b/doc/administration/gitaly/monitoring.md
@@ -152,9 +152,6 @@ The following metrics are available from the `/metrics` endpoint:
for replication to complete after the replication job starts. Available in GitLab 12.10 and later.
- `gitaly_praefect_replication_delay_bucket`, a histogram measuring how much time passes between
when the replication job is created and when it starts. Available in GitLab 12.10 and later.
-- `gitaly_praefect_node_latency_bucket`, a histogram measuring the latency in Gitaly returning
- health check information to Praefect. This indicates Praefect connection saturation. Available in
- GitLab 12.10 and later.
- `gitaly_praefect_connections_total`, the total number of connections to Praefect. [Introduced](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4220) in GitLab 14.7.
To monitor [strong consistency](index.md#strong-consistency), you can use the following Prometheus metrics:
diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md
index e3b198d1012..8cf32a6aaa3 100644
--- a/doc/administration/gitaly/praefect.md
+++ b/doc/administration/gitaly/praefect.md
@@ -28,6 +28,9 @@ The minimum recommended configuration for a Gitaly Cluster requires:
- 3 Praefect nodes
- 3 Gitaly nodes (1 primary, 2 secondary)
+You should configure an odd number of Gitaly nodes so that transactions have a tie-breaker in case one of the
+Gitaly nodes fails in a mutating RPC call.
+
See the [design document](https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/design_ha.md)
for implementation details.
@@ -155,6 +158,14 @@ We note in the instructions below where these secrets are required.
NOTE:
Omnibus GitLab installations can use `gitlab-secrets.json` for `GITLAB_SHELL_SECRET_TOKEN`.
+### Customize time server setting
+
+By default, Gitaly and Praefect nodes use the time server at `pool.ntp.org` for time synchronization checks. You can customize this setting by adding the
+following to `gitlab.rb` on each node:
+
+- `gitaly['env'] = { "NTP_HOST" => "ntp.example.com" }`, for Gitaly nodes.
+- `praefect['env'] = { "NTP_HOST" => "ntp.example.com" }`, for Praefect nodes.
+
### PostgreSQL
NOTE:
@@ -285,7 +296,7 @@ praefect['database_direct_dbname'] = 'praefect_production'
#praefect['database_direct_sslrootcert'] = '...'
```
-Once configured, this connection is automatically used for the
+When configured, this connection is automatically used for the
[SQL LISTEN](https://www.postgresql.org/docs/11/sql-listen.html) feature and
allows Praefect to receive notifications from PostgreSQL for cache invalidation.
@@ -584,11 +595,7 @@ Updates to example must be made at:
}
```
- NOTE:
- In [GitLab 13.8 and earlier](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4988),
- Gitaly nodes were configured directly under the virtual storage, and not under the `nodes` key.
-
-1. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2013) in GitLab 13.1 and later, enable [distribution of reads](index.md#distributed-reads).
+1. Enable [distribution of reads](index.md#distributed-reads).
1. Save the changes to `/etc/gitlab/gitlab.rb` and
[reconfigure Praefect](../restart_gitlab.md#omnibus-gitlab-reconfigure):
@@ -641,8 +648,6 @@ Updates to example must be made at:
#### Enable TLS support
-> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/1698) in GitLab 13.2.
-
Praefect supports TLS encryption. To communicate with a Praefect instance that listens
for secure connections, you must:
@@ -1369,7 +1374,8 @@ We recommend using [repository-specific primary nodes](#repository-specific-prim
### Repository-specific primary nodes
-> [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/3492) in GitLab 13.12.
+> - [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/3492) in GitLab 13.12, with primary elections run when Praefect starts or the cluster's consensus of a Gitaly node's health changes.
+> - [Changed](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/3543) in GitLab 14.1, primary elections are run lazily.
Gitaly Cluster supports electing repository-specific primary Gitaly nodes. Repository-specific
Gitaly primary nodes are enabled in `/etc/gitlab/gitlab.rb` by setting
@@ -1386,14 +1392,8 @@ The `per_repository` election strategy solves this problem by electing a primary
repository. Combined with [configurable replication factors](#configure-replication-factor), you can
horizontally scale storage capacity and distribute write load across Gitaly nodes.
-Primary elections are run:
-
-- In GitLab 14.1 and later, lazily. This means that Praefect doesn't immediately elect
- a new primary node if the current one is unhealthy. A new primary is elected if it is
- necessary to serve a request while the current primary is unavailable.
-- In GitLab 13.12 to GitLab 14.0 when:
- - Praefect starts up.
- - The cluster's consensus of a Gitaly node's health changes.
+Primary elections are run lazily. Praefect doesn't immediately elect a new primary node if the current
+one is unhealthy. A new primary is elected if a request must be served while the current primary is unavailable.
A valid primary node candidate is a Gitaly node that:
@@ -1422,9 +1422,9 @@ To migrate existing clusters:
1. Praefect nodes didn't historically keep database records of every repository stored on the cluster. When
the `per_repository` election strategy is configured, Praefect expects to have database records of
- each repository. A [background database migration](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2749) is
- included in GitLab 13.6 and later to create any missing database records for repositories. Before migrating,
- check Praefect's logs to verify that the database migration ran.
+ each repository. A [background database migration](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2749)
+ creates any missing database records for repositories. Before migrating, check Praefect's logs to verify
+ that the database migration ran.
Check Praefect's logs for `repository importer finished` message. The `virtual_storages` field contains
the names of virtual storages and whether they've had any missing database records created.
diff --git a/doc/administration/gitaly/recovery.md b/doc/administration/gitaly/recovery.md
index b51454aa44e..56894f3e963 100644
--- a/doc/administration/gitaly/recovery.md
+++ b/doc/administration/gitaly/recovery.md
@@ -11,40 +11,15 @@ recovery and has Praefect tracking database tools.
## Primary node failure
-Gitaly Cluster recovers from a failing primary Gitaly node by promoting a healthy secondary as the
-new primary.
+> - Introduced in GitLab 13.0, Gitaly Cluster, elects the secondary with the least unreplicated writes from the primary to be the new primary. There can still be some unreplicated writes, so [data loss can occur](#check-for-data-loss).
+> - Primary node failure recovery support added in GitLab 14.1.
-In GitLab 14.1 and later, Gitaly Cluster:
+Gitaly Cluster recovers from a failing primary Gitaly node by promoting a healthy secondary as the new primary. Gitaly
+Cluster:
- Elects a healthy secondary with a fully up to date copy of the repository as the new primary.
- Repository becomes unavailable if there are no fully up to date copies of it on healthy secondaries.
-To minimize data loss in GitLab 13.0 to 14.0, Gitaly Cluster:
-
-- Switches repositories that are outdated on the new primary to [read-only mode](#read-only-mode).
-- Elects the secondary with the least unreplicated writes from the primary to be the new
- primary. Because there can still be some unreplicated writes,
- [data loss can occur](#check-for-data-loss).
-
-### Read-only mode
-
-> - Introduced in GitLab 13.0 as [generally available](../../policy/alpha-beta-support.md#generally-available-ga).
-> - Between GitLab 13.0 and GitLab 13.2, read-only mode applied to the whole virtual storage and occurred whenever failover occurred.
-> - [In GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitaly/-/issues/2862), read-only mode applies on a per-repository basis and only occurs if a new primary is out of date. If the failed primary contained unreplicated writes, [data loss can occur](#check-for-data-loss).
-> - Removed in GitLab 14.1. Instead, repositories [become unavailable](#unavailable-repositories).
-
-When Gitaly Cluster switches to a new primary in GitLab 13.0 to 14.0, repositories enter read-only mode if they are
-out-of-date. This can happen after failing over to an outdated secondary. Read-only mode eases data recovery efforts by
-preventing writes that may conflict with the unreplicated writes on other nodes.
-
-To enable writes again in GitLab 13.0 to 14.0, an administrator can:
-
-1. [Check](#check-for-data-loss) for data loss.
-1. Attempt to [recover](#data-recovery) missing data.
-1. Either [enable writes](#enable-writes-or-accept-data-loss) in the virtual storage or
- [accept data loss](#enable-writes-or-accept-data-loss) if necessary, depending on the version of
- GitLab.
-
### Unavailable repositories
> - From GitLab 13.0 through 14.0, repositories became read-only if they were outdated on the primary but fully up to date on a healthy secondary. `dataloss` sub-command displays read-only repositories by default through these versions.
@@ -144,9 +119,7 @@ Virtual storage: default
#### Unavailable replicas of available repositories
-NOTE:
-In GitLab 14.0 and earlier, the flag is `-partially-replicated` and the output shows any repositories with assigned nodes with outdated
-copies.
+> Introduced in GitLab 14.0, flag renamed from `-partially-replicated` and behavior changed.
To also list information of repositories which are available but are unavailable from some of the assigned nodes,
use the `-partially-unavailable` flag.
@@ -209,26 +182,17 @@ WARNING:
`accept-dataloss` causes permanent data loss by overwriting other versions of the repository.
Data [recovery efforts](#data-recovery) must be performed before using it.
-Praefect provides the following subcommands to re-enable writes or accept data loss:
-
-- In GitLab 13.2 and earlier, `enable-writes` to re-enable virtual storage for writes after
- data recovery attempts:
+Praefect provides the following subcommands to re-enable writes or accept data loss. If it is not possible to bring one
+of the up-to-date nodes back online, you might have to accept data loss:
- ```shell
- sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml enable-writes -virtual-storage <virtual-storage>
- ```
-
-- In GitLab 13.3 and later, if it is not possible to bring one of the up to date nodes back
- online, you may have to accept data loss:
-
- ```shell
- sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml accept-dataloss -virtual-storage <virtual-storage> -repository <relative-path> -authoritative-storage <storage-name>
- ```
+```shell
+sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml accept-dataloss -virtual-storage <virtual-storage> -repository <relative-path> -authoritative-storage <storage-name>
+```
- When accepting data loss, Praefect:
+When accepting data loss, Praefect:
- 1. Marks the chosen copy of the repository as the latest version.
- 1. Replicates the copy to the other assigned Gitaly nodes.
+1. Marks the chosen copy of the repository as the latest version.
+1. Replicates the copy to the other assigned Gitaly nodes.
This process overwrites any other copy of the repository so care must be taken.
diff --git a/doc/administration/gitaly/reference.md b/doc/administration/gitaly/reference.md
index 3bf1e3136c0..8f7dc688e56 100644
--- a/doc/administration/gitaly/reference.md
+++ b/doc/administration/gitaly/reference.md
@@ -192,8 +192,7 @@ For historical reasons
[GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell) contains
the Git hooks that allow GitLab to validate and react to Git pushes.
Because Gitaly "owns" Git pushes, GitLab Shell must therefore be
-installed alongside Gitaly. We plan to
-[simplify this](https://gitlab.com/gitlab-org/gitaly/-/issues/1226).
+installed alongside Gitaly.
| Name | Type | Required | Description |
| ---- | ---- | -------- | ----------- |
diff --git a/doc/administration/gitaly/troubleshooting.md b/doc/administration/gitaly/troubleshooting.md
index 1c5f0d43864..7edce840396 100644
--- a/doc/administration/gitaly/troubleshooting.md
+++ b/doc/administration/gitaly/troubleshooting.md
@@ -41,17 +41,6 @@ The `gitaly-debug` command provides "production debugging" tools for Gitaly and
performance. It is intended to help production engineers and support
engineers investigate Gitaly performance problems.
-If you're using GitLab 11.6 or newer, this tool should be installed on
-your GitLab or Gitaly server already at `/opt/gitlab/embedded/bin/gitaly-debug`.
-If you're investigating an older GitLab version you can compile this
-tool offline and copy the executable to your server:
-
-```shell
-git clone https://gitlab.com/gitlab-org/gitaly.git
-cd cmd/gitaly-debug
-GOOS=linux GOARCH=amd64 go build -o gitaly-debug
-```
-
To see the help page of `gitaly-debug` for a list of supported sub-commands, run:
```shell
@@ -144,9 +133,8 @@ If you have Prometheus set up to scrape your Gitaly process, you can see
request rates and error codes for individual RPCs in `gitaly-ruby` by
querying `grpc_client_handled_total`.
-- In theory, this metric does not differentiate between `gitaly-ruby` and other RPCs.
-- In practice from GitLab 11.9, all gRPC calls made by Gitaly itself are internal calls from the
- main Gitaly process to one of its `gitaly-ruby` sidecars.
+All gRPC calls made by `gitaly-ruby` itself are internal calls from the main Gitaly process to one of its `gitaly-ruby`
+sidecars.
Assuming your `grpc_client_handled_total` counter only observes Gitaly,
the following query shows you RPCs are (most likely) internally
@@ -335,7 +323,7 @@ You might see the following in Gitaly and Praefect logs:
}
```
-This is a GRPC call
+This is a gRPC call
[error response code](https://grpc.github.io/grpc/core/md_doc_statuscodes.html).
If this error occurs, even though
@@ -359,7 +347,7 @@ necessary because [this issue](https://gitlab.com/gitlab-org/gitaly/-/issues/252
If this error occurs even though file permissions are correct, it's likely that the Gitaly node is
experiencing [clock drift](https://en.wikipedia.org/wiki/Clock_drift).
-Please ensure that the GitLab and Gitaly nodes are synchronized and use an NTP time
+Ensure that the GitLab and Gitaly nodes are synchronized and use an NTP time
server to keep them synchronized if possible.
### Health check warnings
@@ -604,6 +592,16 @@ For each replica, the following metadata is available:
| `Valid Primary` | Indicates whether the replica is fit to serve as the primary node. If the repository's primary is not a valid primary, a failover occurs on the next write to the repository if there is another replica that is a valid primary. A replica is a valid primary if:<br><br>- It is stored on a healthy Gitaly node.<br>- It is fully up to date.<br>- It is not targeted by a pending deletion job from decreasing replication factor.<br>- It is assigned. |
| `Verified At` | Indicates last successful verification of the replica by the [verification worker](praefect.md#repository-verification). If the replica has not yet been verified, `unverified` is displayed in place of the last successful verification time. Introduced in GitLab 15.0. |
+#### Command fails with 'repository not found'
+
+If the supplied value for `-virtual-storage` is incorrect, the command returns the following error:
+
+```plaintext
+get metadata: rpc error: code = NotFound desc = repository not found
+```
+
+The documented examples specify `-virtual-storage default`. Check the Praefect server setting `praefect['virtual_storages']` in `/etc/gitlab/gitlab.rb`.
+
### Check that repositories are in sync
Is [some cases](index.md#known-issues) the Praefect database can get out of sync with the underlying Gitaly nodes. To check that
diff --git a/doc/administration/housekeeping.md b/doc/administration/housekeeping.md
index 15287b917e7..2d3e937e047 100644
--- a/doc/administration/housekeeping.md
+++ b/doc/administration/housekeeping.md
@@ -1,43 +1,144 @@
---
stage: Systems
-group: Distribution
+group: Gitaly
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Housekeeping **(FREE SELF)**
-GitLab supports and automates housekeeping tasks in your current repository such as:
+GitLab supports and automates housekeeping tasks in Git repositories to ensure
+that they can be served as efficiently as possible. Housekeeping tasks include:
-- Compressing Git objects.
+- Compressing Git objects and revisions.
- Removing unreachable objects.
+- Removing stale data like lock files.
+- Maintaining data structures that improve performance.
+- Updating object pools to improve object deduplication across forks.
-## Configure housekeeping
+WARNING:
+Do not manually execute Git commands to perform housekeeping in Git
+repositories that are controlled by GitLab. Doing so may lead to corrupt
+repositories and data loss.
+
+## Housekeeping strategy
+
+Gitaly can perform housekeeping tasks in a Git repository in two ways:
+
+- [Eager housekeeping](#eager-housekeeping) executes specific housekeeping tasks
+ independent of the state a repository is in.
+- [Heuristical housekeeping](#heuristical-housekeeping) executes housekeeping
+ tasks based on a set of heuristics that determine what housekeeping tasks need
+ to be executed based on the repository state.
+
+### Eager housekeeping
+
+The "eager" housekeeping strategy executes housekeeping tasks in a repository
+independent of the repository state. This is the default strategy as used by the
+[manual trigger](#manual-trigger) and the [push-based trigger](#push-based-trigger).
+
+The eager housekeeping strategy is controlled by the GitLab application.
+Depending on the trigger that caused the housekeeping job to run, GitLab asks
+Gitaly to perform specific housekeeping tasks. Gitaly performs these tasks even
+if the repository is in an optimized state. As a result, this strategy can be
+inefficient in large repositories where performing the housekeeping tasks may
+be slow.
+
+### Heuristical housekeeping
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/2634) in GitLab 14.9 for the [manual trigger](#manual-trigger) and the [push-based trigger](#push-based-trigger) [with a flag](feature_flags.md) named `optimized_housekeeping`. Disabled by default.
+> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/353607) in GitLab 14.10.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available for the [manual trigger](#manual-trigger) and the [push-based trigger](#push-based-trigger).
+To make it available, ask an administrator to [enable the feature flag](feature_flags.md) named `optimized_housekeeping`.
+
+The heuristical (or "opportunistic") housekeeping strategy analyzes the
+repository's state and executes housekeeping tasks only when it finds one or
+more data structures are insufficiently optimized. This is the strategy used by
+[scheduled housekeeping](#scheduled-housekeeping). It can optionally be enabled
+for the [manual trigger](#manual-trigger) and the [push-based trigger](#push-based-trigger)
+by enabling the `optimized_housekeeping` feature flag.
+
+Heuristical housekeeping uses the following information to decide on the tasks
+it needs to run:
+
+- The number of loose and stale objects.
+- The number of packfiles that contain already-compressed objects.
+- The number of loose references.
+- The presence of a commit-graph.
+
+The decision whether any of the analyzed data structures need to be optimized is
+based on the size of the repository:
+
+- Objects are repacked frequently the bigger the total size of all objects.
+- References are repacked less frequently the more references there are in
+ total.
+
+Gitaly does this to offset the fact that optimizing those data structures takes
+more time the bigger they get. It is especially important in large
+monorepositories (which receive a lot of traffic) to avoid optimizing them too
+frequently.
+
+## Running housekeeping tasks
+
+There are different ways in which GitLab runs housekeeping tasks:
-GitLab automatically runs `git gc` and `git repack` on repositories after Git pushes:
+- A project's administrator can [manually trigger](#manual-trigger) repository
+ housekeeping tasks.
+- GitLab can automatically schedule housekeeping tasks [after a number of Git pushes](#push-based-trigger).
+- GitLab can [schedule a job](#scheduled-housekeeping) that runs housekeeping
+ tasks for all repositories in a configurable time frame.
+
+### Manual trigger
+
+Administrators of repositories can manually trigger housekeeping tasks in a
+repository. In general this is not required as GitLab knows to automatically run
+housekeeping tasks. The manual trigger can be useful when either:
+
+- A repository is known to require housekeeping.
+- Automated push-based scheduling of housekeeping tasks has been disabled.
+
+To trigger housekeeping tasks manually:
+
+1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, select **Settings > General**.
+1. Expand **Advanced**.
+1. Select **Run housekeeping**.
+
+This starts an asynchronous background worker for the project's repository. The
+background worker executes `git gc`, which performs a number of optimizations.
+
+### Push-based trigger
+
+GitLab automatically runs repository housekeeping tasks after a configured
+number of pushes:
- [`git gc`](https://git-scm.com/docs/git-gc) runs a number of housekeeping tasks such as:
- Compressing Git objects to reduce disk space and increase performance.
- Removing unreachable objects that may have been created from changes to the repository, like force-overwriting branches.
- [`git repack`](https://git-scm.com/docs/git-repack) either:
- - Runs an incremental repack, according to a [configured period](#housekeeping-options). This
+ - Runs an incremental repack, according to a [configured period](#configure-push-based-maintenance). This
packs all loose objects into a new packfile and prunes the now-redundant loose objects.
- - Runs a full repack, according to a [configured period](#housekeeping-options). This repacks all
+ - Runs a full repack, according to a [configured period](#configure-push-based-maintenance). This repacks all
packfiles and loose objects into a single new packfile, and deletes the old now-redundant loose
objects and packfiles. It also optionally creates bitmaps for the new packfile.
+- [`git pack-refs`](https://git-scm.com/docs/git-pack-refs) compresses references
+ stored as loose files into a single file.
+
+#### Configure push-based maintenance
-You can change how often this happens or turn it off:
+You can change how often these tasks run when pushes occur, or you can turn
+them off entirely:
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > Repository**.
1. Expand **Repository maintenance**.
-1. In the **Housekeeping** section, configure the [housekeeping options](#housekeeping-options).
+1. In the **Housekeeping** section, configure the housekeeping options.
1. Select **Save changes**.
-### Housekeeping options
-
The following housekeeping options are available:
-- **Enable automatic repository housekeeping**: Regularly run `git repack` and `git gc`. If you
+- **Enable automatic repository housekeeping**: Regularly run housekeeping tasks. If you
keep this setting disabled for a long time, Git repository access on your GitLab server becomes
slower and your repositories use more disk space.
- **Incremental repack period**: Number of Git pushes after which an incremental `git repack` is
@@ -60,30 +161,80 @@ Housekeeping also [removes unreferenced LFS files](../raketasks/cleanup.md#remov
from your project on the same schedule as the `git gc` operation, freeing up storage space for your
project.
-WARNING:
-Running `git gc` or `git repack` commands manually in the
-[repository folder](repository_storage_types.md#from-project-name-to-hashed-path)
-is discouraged. If the created pack files get incorrect access rights (that is, owned by the wrong user)
-browsing to the project page might result in `404` and `503` errors.
-
-## How housekeeping handles pool repositories
-
-Housekeeping for pool repositories is handled differently from standard repositories. It is
-ultimately performed by the Gitaly RPC `FetchIntoObjectPool`.
-
-This is the current call stack by which it is invoked:
-
-1. `Repositories::HousekeepingService#execute_gitlab_shell_gc`
-1. `Projects::GitGarbageCollectWorker#perform`
-1. `Projects::GitDeduplicationService#fetch_from_source`
-1. `ObjectPool#fetch`
-1. `ObjectPoolService#fetch`
-1. `Gitaly::FetchIntoObjectPoolRequest`
-
-To manually invoke it from a [Rails console](operations/rails_console.md) if needed, you can call
-`project.pool_repository.object_pool.fetch`. This is a potentially long-running task, though Gitaly
-times out in about 8 hours.
-
-WARNING:
-Do not run `git prune` or `git gc` in pool repositories! This can cause data loss in "real"
-repositories that depend on the pool in question.
+### Scheduled housekeeping
+
+While GitLab automatically performs housekeeping tasks based on the number of
+pushes, it does not maintain repositories that don't receive any pushes at all.
+As a result, inactive repositories or repositories that are only getting read
+requests may not benefit from improvements in the repository housekeeping
+strategy.
+
+Administrators can enable a background job that performs housekeeping in all
+repositories at a customizable interval to remedy this situation. This
+background job processes all repositories hosted by a Gitaly node in a random
+order and eagerly performs housekeeping tasks on them. The Gitaly node will stop
+processing repositories if it takes longer than the configured interval.
+
+#### Configure scheduled housekeeping
+
+Background maintenance of Git repositories is configured in Gitaly. By default,
+Gitaly performs background repository maintenance every day at 12:00 noon for a
+duration of 10 minutes.
+
+You can change this default in Gitaly configuration. The following snippet
+enables daily background repository maintenance starting at 23:00 for 1 hour
+for the `default` storage:
+
+```toml
+[daily_maintenance]
+start_hour = 23
+start_minute = 00
+duration = 1h
+storages = ["default"]
+```
+
+Use the following snippet to completely disable background repository
+maintenance:
+
+```toml
+[daily_maintenance]
+disabled = true
+```
+
+## Object pool repositories
+
+Object pool repositories are used by GitLab to deduplicate objects across forks
+of a repository. When creating the first fork, we:
+
+1. Create an object pool repository that contains all objects of the repository
+ that is about to be forked.
+1. Link the repository to this new object pool via Git's altenates mechanism.
+1. Repack the repository so that it uses objects from the object pool. It thus
+ can drop its own copy of the objects.
+
+Any forks of this repository can now link against the object pool and thus only
+have to keep objects that diverge from the primary repository.
+
+GitLab needs to perform special housekeeping operations in object pools:
+
+- Gitaly cannot ever delete unreachable objects from object pools because they
+ might be used by any of the forks that are connected to it.
+- Gitaly must keep all objects reachable due to the same reason. Object pools
+ thus maintain references to unreachable "dangling" objects so that they don't
+ ever get deleted.
+- GitLab must update object pools regularly to pull in new objects that have
+ been added in the primary repository. Otherwise, an object pool will become
+ increasingly inefficient at deduplicating objects.
+
+These housekeeping operations are performed by the specialized
+`FetchIntoObjectPool` RPC that handles all of these special tasks while also
+executing the regular housekeeping tasks we execute for normal Git
+repositories.
+
+Object pools are getting optimized automatically whenever the primary member is
+getting garbage collected. Therefore, the cadence can be configured using the
+same Git GC period in that project.
+
+If you need to manually invoke the RPC from a [Rails console](operations/rails_console.md),
+you can call `project.pool_repository.object_pool.fetch`. This is a potentially
+long-running task, though Gitaly times out after about 8 hours.
diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md
index 4959bacaaa4..433956bb066 100644
--- a/doc/administration/incoming_email.md
+++ b/doc/administration/incoming_email.md
@@ -41,7 +41,7 @@ in the mailbox for `user@example.com` . It is supported by providers such as
Gmail, Google Apps, Yahoo! Mail, Outlook.com, and iCloud, as well as the
[Postfix mail server](reply_by_email_postfix_setup.md), which you can run on-premises.
Microsoft Exchange Server [does not support sub-addressing](#microsoft-exchange-server),
-and Microsoft Office 365 [does not support sub-addressing by default](#microsoft-office-365)
+and Microsoft Office 365 [does not support sub-addressing by default](#microsoft-office-365).
NOTE:
If your provider or server supports email sub-addressing, we recommend using it.
diff --git a/doc/administration/index.md b/doc/administration/index.md
index 7d684daf5a6..95db2b7a2e0 100644
--- a/doc/administration/index.md
+++ b/doc/administration/index.md
@@ -185,7 +185,7 @@ Learn how to install, configure, update, and maintain your GitLab instance.
## Git configuration options
-- [Server hooks](server_hooks.md): Server hooks (on the file system) for when webhooks aren't enough.
+- [Git server hooks](server_hooks.md): Git server hooks (on the file system) for when webhooks aren't enough. Previously called server hooks.
- [Git LFS configuration](lfs/index.md): Learn how to configure LFS for GitLab.
- [Housekeeping](housekeeping.md): Keep your Git repositories tidy and fast.
- [Configuring Git Protocol v2](git_protocol.md): Git protocol version 2 support.
@@ -232,12 +232,9 @@ who are aware of the risks.
- [Troubleshooting Kubernetes](https://docs.gitlab.com/charts/troubleshooting/kubernetes_cheat_sheet.html)
- [Troubleshooting PostgreSQL](troubleshooting/postgresql.md)
- [Guide to test environments](troubleshooting/test_environments.md) (for Support Engineers)
-- [GitLab Rails console commands](troubleshooting/gitlab_rails_cheat_sheet.md) (for Support Engineers)
- [Troubleshooting SSL](troubleshooting/ssl.md)
- Related links:
- [GitLab Developer Documentation](../development/index.md)
- [Repairing and recovering broken Git repositories](https://git.seveas.net/repairing-and-recovering-broken-git-repositories.html)
- [Testing with OpenSSL](https://www.feistyduck.com/library/openssl-cookbook/online/testing-with-openssl/index.html)
- [`strace` zine](https://wizardzines.com/zines/strace/)
-- GitLab.com-specific resources:
- - [Example group SAML and SCIM configurations](../user/group/saml_sso/example_saml_config.md)
diff --git a/doc/administration/instance_limits.md b/doc/administration/instance_limits.md
index 3f44b53a6d5..546c4667220 100644
--- a/doc/administration/instance_limits.md
+++ b/doc/administration/instance_limits.md
@@ -212,12 +212,20 @@ It's possible that this limit changes to a lower number in the future.
## Size of commit titles and descriptions
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292039) in GitLab 13.9
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292039) in GitLab 13.9.
-Commits with arbitrarily large messages may be pushed to GitLab, but when
-displaying commits, titles (the first line of the commit message)
-limits to 1KiB, and descriptions (the rest of the message) limits to
-1MiB.
+Commits with arbitrarily large messages may be pushed to GitLab, but the following
+display limits apply:
+
+- **Title** - The first line of the commit message. Limited to 1 KiB.
+- **Description** - The rest of the commit message. Limited to 1 MiB.
+
+When a commit is pushed, GitLab processes the title and description to replace
+references to issues (`#123`) and merge requests (`!123`) with links to the
+issues and merge requests.
+
+When a branch with a large number of commits is pushed, only the last 100 commits
+are processed.
## Number of issues in the milestone overview
@@ -359,7 +367,7 @@ header. Such emails don't create comments on issues or merge requests.
## Amount of data sent from Sentry through Error Tracking
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14926) in GitLab 12.6.
+> [Limiting all Sentry responses](https://gitlab.com/gitlab-org/gitlab/-/issues/356448) introduced in GitLab 15.6.
Sentry payloads sent to GitLab have a 1 MB maximum limit, both for security reasons
and to limit memory consumption.
diff --git a/doc/administration/integration/kroki.md b/doc/administration/integration/kroki.md
index aa029be90e7..491eeb002e4 100644
--- a/doc/administration/integration/kroki.md
+++ b/doc/administration/integration/kroki.md
@@ -34,7 +34,7 @@ The [`yuzutech/kroki`](https://hub.docker.com/r/yuzutech/kroki) image contains t
<!-- vale gitlab.Spelling = NO -->
- [Bytefield](https://bytefield-svg.deepsymmetry.org/)
-- [Ditaa](http://ditaa.sourceforge.net)
+- [Ditaa](https://ditaa.sourceforge.net)
- [Erd](https://github.com/BurntSushi/erd)
- [GraphViz](https://www.graphviz.org/)
- [Nomnoml](https://github.com/skanaar/nomnoml)
diff --git a/doc/administration/integration/mailgun.md b/doc/administration/integration/mailgun.md
index baf9e8c8a3b..d78cd9e1796 100644
--- a/doc/administration/integration/mailgun.md
+++ b/doc/administration/integration/mailgun.md
@@ -48,5 +48,5 @@ you're ready to enable the Mailgun integration:
1. Select the **Enable Mailgun** checkbox.
1. Enter the Mailgun HTTP webhook signing key as described in
[the Mailgun documentation](https://documentation.mailgun.com/en/latest/user_manual.html#webhooks-1) and
- shown in the [API security](https://app.mailgun.com/app/account/security/api_keys) section for your Mailgun account.
+ shown in the API security (`https://app.mailgun.com/app/account/security/api_keys`) section for your Mailgun account.
1. Select **Save changes**.
diff --git a/doc/administration/issue_closing_pattern.md b/doc/administration/issue_closing_pattern.md
index d10f5320109..e9150ae0650 100644
--- a/doc/administration/issue_closing_pattern.md
+++ b/doc/administration/issue_closing_pattern.md
@@ -17,7 +17,7 @@ in the project's default branch.
## Change the issue closing pattern
-In order to change the pattern you need to have access to the server that GitLab
+To change the pattern, you must have access to the server that GitLab
is installed on.
The default pattern can be located in [`gitlab.yml.example`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/gitlab.yml.example)
diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md
index 1e9f20ded55..d79f322c3f5 100644
--- a/doc/administration/job_artifacts.md
+++ b/doc/administration/job_artifacts.md
@@ -200,7 +200,8 @@ sudo -u git -H bundle exec rake gitlab:artifacts:migrate RAILS_ENV=production
You can optionally track progress and verify that all job artifacts migrated successfully using the
[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database):
-- `sudo gitlab-rails dbconsole` for Omnibus GitLab instances.
+- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier.
+- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later.
- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances.
Verify `objectstg` below (where `store=2`) has count of all job artifacts:
@@ -301,7 +302,7 @@ I/O. It instead inspects the metadata file which contains all the relevant
information. This is especially important when there is a lot of artifacts, or
an archive is a very large file.
-When clicking on a specific file, [GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) extracts it
+When selecting a specific file, [GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) extracts it
from the archive and the download begins. This implementation saves space,
memory and disk I/O.
@@ -315,12 +316,132 @@ reasons are:
- Users have configured job artifacts expiration to be longer than necessary.
- The number of jobs run, and hence artifacts generated, is higher than expected.
- Job logs are larger than expected, and have accumulated over time.
+- The file system might run out of inodes because
+ [empty directories are left behind by artifact housekeeping](https://gitlab.com/gitlab-org/gitlab/-/issues/17465).
+ [The Rake task for _orphaned_ artifact files](../raketasks/cleanup.md#remove-orphan-artifact-files)
+ removes these.
+- Artifact files might be left on disk and not deleted by housekeeping. Run the
+ [Rake task for _orphaned_ artifact files](../raketasks/cleanup.md#remove-orphan-artifact-files)
+ to remove these. This script should always find work to do, as it also removes empty directories (see above).
+- [Artifact housekeeping was changed significantly](#artifacts-housekeeping-disabled-in-gitlab-146-to-152),
+ and you might need to enable a feature flag to used the updated system.
In these and other cases, identify the projects most responsible
for disk space usage, figure out what types of artifacts are using the most
space, and in some cases, manually delete job artifacts to reclaim disk space.
-One possible first step is to [clean up _orphaned_ artifact files](../raketasks/cleanup.md#remove-orphan-artifact-files).
+#### Artifacts housekeeping disabled in GitLab 14.6 to 15.2
+
+Artifact housekeeping was significantly changed in GitLab 14.10, and the changes
+were back ported to GitLab 14.6 and later. The updated housekeeping must be
+enabled with feature flags [until GitLab 15.3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92931).
+
+To check if the feature flags are enabled:
+
+1. Start a [Rails console](operations/rails_console.md#starting-a-rails-console-session).
+
+1. Check if the feature flags are enabled.
+
+ - GitLab 14.10 and earlier:
+
+ ```ruby
+ Feature.enabled?(:ci_detect_wrongly_expired_artifacts, default_enabled: :yaml)
+ Feature.enabled?(:ci_update_unlocked_job_artifacts, default_enabled: :yaml)
+ Feature.enabled?(:ci_destroy_unlocked_job_artifacts, default_enabled: :yaml)
+ ```
+
+ - GitLab 15.00 and later:
+
+ ```ruby
+ Feature.enabled?(:ci_detect_wrongly_expired_artifacts)
+ Feature.enabled?(:ci_update_unlocked_job_artifacts)
+ Feature.enabled?(:ci_destroy_unlocked_job_artifacts)
+ ```
+
+1. If any of the feature flags are disabled, enable them:
+
+ ```ruby
+ Feature.enable(:ci_detect_wrongly_expired_artifacts)
+ Feature.enable(:ci_update_unlocked_job_artifacts)
+ Feature.enable(:ci_destroy_unlocked_job_artifacts)
+ ```
+
+These changes include switching artifacts from `unlocked` to `locked` if
+they [should be retained](../ci/pipelines/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs).
+
+Artifacts created before this feature was introduced have a status of `unknown`. After they expire,
+these artifacts are not processed by the new housekeeping jobs.
+
+You can check the database to confirm if your instance has artifacts with the `unknown` status:
+
+1. Start a database console, on Omnibus:
+
+ ```shell
+ sudo gitlab-psql
+ ```
+
+1. Run this query:
+
+ ```sql
+ select expire_at, file_type, locked, count(*) from ci_job_artifacts
+ where expire_at is not null and
+ file_type != 3
+ group by expire_at, file_type, locked having count(*) > 1;
+ ```
+
+If records are returned, then there are artifacts which the housekeeping job
+is unable to process. For example:
+
+```plaintext
+ expire_at | file_type | locked | count
+-------------------------------+-----------+--------+--------
+ 2021-06-21 22:00:00+00 | 1 | 2 | 73614
+ 2021-06-21 22:00:00+00 | 2 | 2 | 73614
+ 2021-06-21 22:00:00+00 | 4 | 2 | 3522
+ 2021-06-21 22:00:00+00 | 9 | 2 | 32
+ 2021-06-21 22:00:00+00 | 12 | 2 | 163
+```
+
+Artifacts with locked status `2` are `unknown`. Check
+[issue #346261](https://gitlab.com/gitlab-org/gitlab/-/issues/346261#note_1028871458)
+for more details.
+
+The Sidekiq worker that processes all `unknown` artifacts is enabled by default in
+GitLab 15.3 and later. It analyzes the artifacts returned by the above database query and
+determines which should be `locked` or `unlocked`. Artifacts are then deleted
+by that worker if needed.
+
+The worker can be enabled on self-managed instances running GitLab 14.10 and later:
+
+1. Start a [Rails console](operations/rails_console.md#starting-a-rails-console-session).
+
+1. Check if the feature is enabled.
+
+ - GitLab 14.10:
+
+ ```ruby
+ Feature.enabled?(:ci_job_artifacts_backlog_work, default_enabled: :yaml)
+ ```
+
+ - GitLab 15.0 and later:
+
+ ```ruby
+ Feature.enabled?(:ci_job_artifacts_backlog_work)
+ ```
+
+1. Enable the feature, if needed:
+
+ ```ruby
+ Feature.enable(:ci_job_artifacts_backlog_work)
+ ```
+
+The worker processes 10,000 `unknown` artifacts every seven minutes, or roughly two million
+in 24 hours.
+
+There is a related `ci_job_artifacts_backlog_large_loop_limit` feature flag
+which causes the worker to process `unknown` artifacts
+[in batches that are five times larger](https://gitlab.com/gitlab-org/gitlab/-/issues/356319).
+This flag is not recommended for use on self-managed instances.
#### List projects and builds with artifacts with a specific expiration (or no expiration)
diff --git a/doc/administration/lfs/index.md b/doc/administration/lfs/index.md
index 1b01c665ab8..8fdc98bd12a 100644
--- a/doc/administration/lfs/index.md
+++ b/doc/administration/lfs/index.md
@@ -174,10 +174,11 @@ For installations from source:
RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:lfs:migrate
```
-You can optionally track progress and verify that all packages migrated successfully using the
+You can optionally track progress and verify that all LFS objects migrated successfully using the
[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database):
-- `sudo gitlab-rails dbconsole` for Omnibus GitLab instances.
+- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier.
+- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later.
- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances.
Verify `objectstg` below (where `store=2`) has count of all LFS objects:
diff --git a/doc/administration/libravatar.md b/doc/administration/libravatar.md
index 5b2334bff8a..802a3be46fa 100644
--- a/doc/administration/libravatar.md
+++ b/doc/administration/libravatar.md
@@ -134,6 +134,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/administration/logs.md b/doc/administration/logs.md
deleted file mode 100644
index 7264cf6db98..00000000000
--- a/doc/administration/logs.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: 'logs/index.md'
-remove_date: '2022-10-23'
----
-
-This document was moved to [another location](logs/index.md).
-
-<!-- This redirect file can be deleted after <2022-10-23>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/administration/logs/index.md b/doc/administration/logs/index.md
index 2bcda759442..b0631c52a47 100644
--- a/doc/administration/logs/index.md
+++ b/doc/administration/logs/index.md
@@ -340,6 +340,12 @@ associated SSH key can download the project in question by using a `git fetch` o
- `params`: Key-value pairs passed in a query string or HTTP body (sensitive parameters, such as passwords and tokens, are filtered out)
- `ua`: The User-Agent of the requester
+NOTE:
+As of [`Grape Logging`](https://github.com/aserafin/grape_logging) v1.8.4,
+the `view_duration_s` is calculated by [`duration_s - db_duration_s`](https://github.com/aserafin/grape_logging/blob/v1.8.4/lib/grape_logging/middleware/request_logger.rb#L117-L119).
+Therefore, `view_duration_s` can be affected by multiple different factors, like read-write
+process on Redis or external HTTP, not only the serialization process.
+
## `application.log`
Depending on your installation method, this file is located at:
@@ -417,44 +423,16 @@ like this example:
}
```
-## `kubernetes.log`
+## `kubernetes.log` (DEPRECATED)
+
+> [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5.
Depending on your installation method, this file is located at:
- Omnibus GitLab: `/var/log/gitlab/gitlab-rails/kubernetes.log`
- Installations from source: `/home/git/gitlab/log/kubernetes.log`
-It logs information related to the Kubernetes Integration, including errors
-during installing cluster applications on your managed Kubernetes
-clusters.
-
-Each line contains JSON that can be ingested by services like Elasticsearch and Splunk.
-Line breaks have been added to the following example for clarity:
-
-```json
-{
- "severity":"ERROR",
- "time":"2018-11-23T15:14:54.652Z",
- "exception":"Kubeclient::HttpError",
- "error_code":401,
- "service":"Clusters::Applications::CheckInstallationProgressService",
- "app_id":14,
- "project_ids":[1],
- "group_ids":[],
- "message":"Unauthorized"
-}
-{
- "severity":"ERROR",
- "time":"2018-11-23T15:42:11.647Z",
- "exception":"Kubeclient::HttpError",
- "error_code":null,
- "service":"Clusters::Applications::InstallService",
- "app_id":2,
- "project_ids":[19],
- "group_ids":[],
- "message":"SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate)"
-}
-```
+It logs information related to [certificate-based clusters](../../user/project/clusters/index.md), such as connectivity errors. Each line contains JSON that can be ingested by services like Elasticsearch and Splunk.
## `git_json.log`
diff --git a/doc/administration/logs/tracing_correlation_id.md b/doc/administration/logs/tracing_correlation_id.md
index f651455a088..906dcd3cea9 100644
--- a/doc/administration/logs/tracing_correlation_id.md
+++ b/doc/administration/logs/tracing_correlation_id.md
@@ -103,7 +103,7 @@ sudo gitlab-ctl tail gitlab-rails/production_json.log | grep '"username":"bob"'
## Searching your logs for the correlation ID
-Once you have the correlation ID you can start searching for relevant log
+When you have the correlation ID you can start searching for relevant log
entries. You can filter the lines by the correlation ID itself.
Combining a `find` and `grep` should be sufficient to find the entries you are looking for.
diff --git a/doc/administration/maintenance_mode/index.md b/doc/administration/maintenance_mode/index.md
index 12f3c4c1cc3..9adb8ce2cd9 100644
--- a/doc/administration/maintenance_mode/index.md
+++ b/doc/administration/maintenance_mode/index.md
@@ -8,9 +8,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2149) in GitLab 13.9.
-Maintenance Mode allows administrators to reduce write operations to a minimum while maintenance tasks are performed. The main goal is to block all external actions that change the internal state, including the PostgreSQL database, but especially files, Git repositories, Container repositories, and so on.
+Maintenance Mode allows administrators to reduce write operations to a minimum while maintenance tasks are performed. The main goal is to block all external actions that change the internal state, including the PostgreSQL database, but especially files, Git repositories, and Container repositories.
-Once Maintenance Mode is enabled, in-progress actions finish relatively quickly since no new actions are coming in, and internal state changes are minimal.
+When Maintenance Mode is enabled, in-progress actions finish relatively quickly since no new actions are coming in, and internal state changes are minimal.
In that state, various maintenance tasks are easier, and services can be stopped completely or be
further degraded for a much shorter period of time than might otherwise be needed. For example, stopping cron jobs and draining queues should be fairly quick.
@@ -150,7 +150,7 @@ is turned off.
Deployments don't go through because pipelines are unfinished.
-It is recommended to disable auto deploys during Maintenance Mode, and enable them once it is disabled.
+It is recommended to disable auto deploys during Maintenance Mode, and enable them when it is disabled.
#### Terraform integration
diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md
index ad864255f02..35dc64a0594 100644
--- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md
+++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md
@@ -30,31 +30,31 @@ As an administrator, you can add new members to the group to give them the Maint
This project can be used to:
-- Self monitor your GitLab instance. The metrics dashboard of the project shows some basic resource
+- Self-monitor your GitLab instance. The metrics dashboard of the project shows some basic resource
usage charts, such as CPU and memory usage of each server in
[Omnibus GitLab](https://docs.gitlab.com/omnibus/) installations.
- Also configure your own [custom metrics](../../../operations/metrics/index.md#adding-custom-metrics)
using metrics exposed by the [GitLab exporter](../prometheus/gitlab_metrics.md#metrics-available).
-## Create the self monitoring project
+## Create the self-monitoring project
1. On the top bar, select **Main menu > Admin**.
-1. On the left sidebar, select **Settings > Metrics and profiling** and expand **Self monitoring**.
-1. Toggle **Self monitoring** on.
+1. On the left sidebar, select **Settings > Metrics and profiling** and expand **Self-monitoring**.
+1. Toggle **Self-monitoring** on.
1. After your GitLab instance creates the project, GitLab displays a link to the
- project in the text above the **Self monitoring** toggle. You can also find it
+ project in the text above the **Self-monitoring** toggle. You can also find it
from the top bar by selecting **Main menu > Projects**.
-## Delete the self monitoring project
+## Delete the self-monitoring project
WARNING:
-Deleting the self monitoring project removes any changes made to the project. If
+Deleting the self-monitoring project removes any changes made to the project. If
you create the project again, it's created in its default state.
1. On the top bar, select **Main menu > Admin**.
-1. On the left sidebar, go to **Settings > Metrics and profiling** and expand **Self monitoring**.
-1. Toggle **Self monitoring** off.
-1. In the confirmation dialog that opens, select **Delete self monitoring project**.
+1. On the left sidebar, go to **Settings > Metrics and profiling** and expand **Self-monitoring**.
+1. Toggle **Self-monitoring** off.
+1. In the confirmation dialog that opens, select **Delete self-monitoring project**.
It can take a few seconds for it to be deleted.
1. After the project is deleted, GitLab displays a message confirming your action.
@@ -66,7 +66,7 @@ panels, provide a regular expression in the **Instance label regex** field.
The dashboard uses metrics available in
[Omnibus GitLab](https://docs.gitlab.com/omnibus/) installations.
-![GitLab self monitoring overview dashboard](img/self_monitoring_overview_dashboard.png)
+![GitLab self-monitoring overview dashboard](img/self_monitoring_overview_dashboard.png)
You can also
[create your own dashboards](../../../operations/metrics/dashboards/index.md).
@@ -85,12 +85,12 @@ you [configure it manually](../../../user/project/integrations/prometheus.md#man
You can [add a Prometheus integration](../../../operations/incident_management/integrations.md)
to GitLab to receive notifications of any alerts.
-Once the integration is setup, you can
+When the integration is set up, you can
[take action on incoming alerts](../../../operations/metrics/alerts.md#trigger-actions-from-alerts).
-## Add custom metrics to the self monitoring project
+## Add custom metrics to the self-monitoring project
-You can add custom metrics in the self monitoring project by:
+You can add custom metrics in the self-monitoring project by:
1. [Duplicating](../../../operations/metrics/dashboards/index.md#duplicate-a-gitlab-defined-dashboard) the overview dashboard.
1. [Editing](../../../operations/metrics/index.md) the newly created dashboard file and configuring it with [dashboard YAML properties](../../../operations/metrics/dashboards/yaml.md).
@@ -118,4 +118,4 @@ If this returns true, the first administrator user is an external user.
If you face this issue, you can temporarily
[make the administrator user a non-external user](../../../user/permissions.md#external-users)
and then try to create the project.
-Once the project is created, the administrator user can be changed back to an external user.
+After the project is created, the administrator user can be changed back to an external user.
diff --git a/doc/administration/monitoring/ip_allowlist.md b/doc/administration/monitoring/ip_allowlist.md
index 400c70d0fde..94d6876b16f 100644
--- a/doc/administration/monitoring/ip_allowlist.md
+++ b/doc/administration/monitoring/ip_allowlist.md
@@ -6,9 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# IP whitelist **(FREE SELF)**
-NOTE:
-We intend to [rename IP whitelist as `IP allowlist`](https://gitlab.com/groups/gitlab-org/-/epics/3478).
-
GitLab provides some [monitoring endpoints](../../user/admin_area/monitoring/health_check.md)
that provide health check information when probed.
diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md
index a003a3f25bc..92e9672cdb6 100644
--- a/doc/administration/monitoring/performance/grafana_configuration.md
+++ b/doc/administration/monitoring/performance/grafana_configuration.md
@@ -133,15 +133,15 @@ However, you should **not** reinstate your old data _except_ under one of the fo
If you require access to your old Grafana data but don't meet one of these criteria, you may consider:
1. Reinstating it temporarily.
-1. [Exporting the dashboards](https://grafana.com/docs/grafana/latest/dashboards/export-import/#exporting-a-dashboard) you need.
-1. Refreshing the data and [re-importing your dashboards](https://grafana.com/docs/grafana/latest/dashboards/export-import/#import-dashboard).
+1. [Exporting the dashboards](https://grafana.com/docs/grafana/latest/dashboards/manage-dashboards/#export-and-import-dashboards) you need.
+1. Refreshing the data and [re-importing your dashboards](https://grafana.com/docs/grafana/latest/dashboards/manage-dashboards/#export-and-import-dashboards).
WARNING:
These actions pose a temporary vulnerability while your old Grafana data is in use.
Deciding to take any of these actions should be weighed carefully with your need to access
existing data and dashboards.
-For more information and further mitigation details, please refer to our
+For more information and further mitigation details, refer to our
[blog post on the security release](https://about.gitlab.com/releases/2019/08/12/critical-security-release-gitlab-12-dot-1-dot-6-released/).
Read more on:
diff --git a/doc/administration/monitoring/performance/index.md b/doc/administration/monitoring/performance/index.md
index 0bea0836191..89f1270532a 100644
--- a/doc/administration/monitoring/performance/index.md
+++ b/doc/administration/monitoring/performance/index.md
@@ -41,7 +41,7 @@ Two types of metrics are collected:
Transaction metrics are metrics that can be associated with a single
transaction. This includes statistics such as the transaction duration, timings
-of any executed SQL queries, time spent rendering HAML views, and so on. These metrics
+of any executed SQL queries, and time spent rendering HAML views. These metrics
are collected for every Rack request and Sidekiq job processed.
### Sampled Metrics
diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md
index e8bdacb0e14..d3b28fd15bc 100644
--- a/doc/administration/monitoring/prometheus/gitlab_metrics.md
+++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md
@@ -44,6 +44,8 @@ The following metrics are available:
| `gitlab_ci_pipeline_size_builds` | Histogram | 13.1 | Total number of builds within a pipeline grouped by a pipeline source | `source` |
| `gitlab_ci_runner_authentication_success_total` | Counter | 15.2 | Total number of times that runner authentication has succeeded | `type` |
| `gitlab_ci_runner_authentication_failure_total` | Counter | 15.2 | Total number of times that runner authentication has failed
+| `gitlab_ghost_user_migration_lag_seconds` | Gauge | 15.6 | The waiting time in seconds of the oldest scheduled record for ghost user migration | |
+| `gitlab_ghost_user_migration_scheduled_records_total` | Gauge | 15.6 | The total number of scheduled ghost user migrations | |
| `job_waiter_started_total` | Counter | 12.9 | Number of batches of jobs started where a web request is waiting for the jobs to complete | `worker` |
| `job_waiter_timeouts_total` | Counter | 12.9 | Number of batches of jobs that timed out where a web request is waiting for the jobs to complete | `worker` |
| `gitlab_ci_active_jobs` | Histogram | 14.2 | Count of active jobs when pipeline is created | |
@@ -148,6 +150,8 @@ The following metrics are available:
| `gitlab_ci_build_trace_errors_total` | Counter | 14.4 | Total amount of different error types on a build trace | `error_reason` |
| `gitlab_presentable_object_cacheless_render_real_duration_seconds` | Histogram | 15.3 | Duration of real time spent caching and representing specific web request objects | `controller`, `action` |
| `cached_object_operations_total` | Counter | 15.3 | Total number of objects cached for specific web requests | `controller`, `action` |
+| `redis_hit_miss_operations_total` | Counter | 15.6 | Total number of Redis cache hits and misses | `cache_hit`, `caller_id`, `cache_identifier`, `feature_category`, `backing_resource` |
+| `redis_cache_generation_duration_seconds` | Histogram | 15.6 | Time to generate Redis cache | `cache_hit`, `caller_id`, `cache_identifier`, `feature_category`, `backing_resource` |
## Metrics controlled by a feature flag
@@ -155,7 +159,6 @@ The following metrics can be controlled by feature flags:
| Metric | Feature Flag |
|:---------------------------------------------------------------|:-------------------------------------------------------------------|
-| `gitlab_method_call_duration_seconds` | `prometheus_metrics_method_instrumentation` |
| `gitlab_view_rendering_duration_seconds` | `prometheus_metrics_view_instrumentation` |
## Praefect metrics
@@ -318,6 +321,16 @@ configuration option in `gitlab.yml`. These metrics are served from the
| `geo_ci_secure_files_verification_total` | Gauge | 15.3 | Number of secure files verifications tried on secondary | `url` |
| `geo_ci_secure_files_verified` | Gauge | 15.3 | Number of secure files verified on secondary | `url` |
| `geo_ci_secure_files_verification_failed` | Gauge | 15.3 | Number of secure files verifications failed on secondary | `url` |
+| `geo_dependency_proxy_blob` | Gauge | 15.6 | Number of dependency proxy blobs on primary | |
+| `geo_dependency_proxy_blob_checksum_total` | Gauge | 15.6 | Number of dependency proxy blobs tried to checksum on primary | |
+| `geo_dependency_proxy_blob_checksummed` | Gauge | 15.6 | Number of dependency proxy blobs successfully checksummed on primary | |
+| `geo_dependency_proxy_blob_checksum_failed` | Gauge | 15.6 | Number of dependency proxy blobs failed to calculate the checksum on primary | |
+| `geo_dependency_proxy_blob_synced` | Gauge | 15.6 | Number of dependency proxy blobs synced on secondary | |
+| `geo_dependency_proxy_blob_failed` | Gauge | 15.6 | Number of dependency proxy blobs failed to sync on secondary | |
+| `geo_dependency_proxy_blob_registry` | Gauge | 15.6 | Number of dependency proxy blobs in the registry | |
+| `geo_dependency_proxy_blob_verification_total` | Gauge | 15.6 | Number of dependency proxy blobs verifications tried on secondary | |
+| `geo_dependency_proxy_blob_verified` | Gauge | 15.6 | Number of dependency proxy blobs verified on secondary | |
+| `geo_dependency_proxy_blob_verification_failed` | Gauge | 15.6 | Number of dependency proxy blobs verifications failed on secondary | |
## Database load balancing metrics **(PREMIUM SELF)**
@@ -374,6 +387,8 @@ Some basic Ruby runtime metrics are available:
| `ruby_process_cpu_seconds_total` | Gauge | 12.0 | Total amount of CPU time per process |
| `ruby_process_max_fds` | Gauge | 12.0 | Maximum number of open file descriptors per process |
| `ruby_process_resident_memory_bytes` | Gauge | 12.0 | Memory usage by process (RSS/Resident Set Size) |
+| `ruby_process_resident_anon_memory_bytes`| Gauge | 15.6 | Anonymous memory usage by process (RSS/Resident Set Size) |
+| `ruby_process_resident_file_memory_bytes`| Gauge | 15.6 | File-backed memory usage by process (RSS/Resident Set Size) |
| `ruby_process_unique_memory_bytes` | Gauge | 13.0 | Memory usage by process (USS/Unique Set Size) |
| `ruby_process_proportional_memory_bytes` | Gauge | 13.0 | Memory usage by process (PSS/Proportional Set Size) |
| `ruby_process_start_time_seconds` | Gauge | 12.0 | UNIX timestamp of process start time |
@@ -399,7 +414,7 @@ These client metrics are meant to complement Redis server metrics.
These metrics are broken down per
[Redis instance](https://docs.gitlab.com/omnibus/settings/redis.html#running-with-multiple-redis-instances).
These metrics all have a `storage` label which indicates the Redis
-instance (`cache`, `shared_state`, and so on).
+instance. For example, `cache` or `shared_state`.
| Metric | Type | Since | Description |
|:--------------------------------- |:------- |:----- |:----------- |
diff --git a/doc/administration/nfs.md b/doc/administration/nfs.md
index 9072bd1f344..85f35a1b188 100644
--- a/doc/administration/nfs.md
+++ b/doc/administration/nfs.md
@@ -10,7 +10,7 @@ type: reference
NFS can be used as an alternative for object storage but this isn't typically
recommended for performance reasons.
-For data objects such as LFS, Uploads, Artifacts, and so on, an [Object Storage service](object_storage.md)
+For data objects such as LFS, Uploads, and Artifacts, an [Object Storage service](object_storage.md)
is recommended over NFS where possible, due to better performance.
When eliminating the usage of NFS, there are [additional steps you need to take](object_storage.md#other-alternatives-to-file-system-storage)
in addition to moving to Object Storage.
@@ -44,7 +44,7 @@ GitLab support is unable to continue with the investigation if both:
- The date of the request is on or after the release of GitLab version 15.6.
- Support Engineers and Management determine that all reasonable non-NFS root causes have been exhausted.
-If the issue is reproducible, or if it happens intermittently but regularly, GitLab Support can investigate providing the issue reproduces without the use of NFS. In order to reproduce without NFS, the affected repositories should be migrated to a different Gitaly shard, such as Gitaly cluster or a standalone Gitaly VM, backed with block storage.
+If the issue is reproducible, or if it happens intermittently but regularly, GitLab Support can investigate providing the issue reproduces without the use of NFS. To reproduce without NFS, the affected repositories should be migrated to a different Gitaly shard, such as Gitaly cluster or a standalone Gitaly VM, backed with block storage.
### Why remove NFS for Git repository data
@@ -52,7 +52,7 @@ If the issue is reproducible, or if it happens intermittently but regularly, Git
NFS is not well-suited to a workload consisting of many small files, like Git repositories. NFS does provide a number of configuration options designed to improve performance. However, over time, a number of these mount options have proven to result in inconsistencies across multiple nodes mounting the NFS volume, up to and including data loss. Addressing these inconsistencies consume extraordinary development and support engineer time that hamper our ability to develop [Gitaly Cluster](gitaly/praefect.md), our purpose-built solution to addressing the deficiencies of NFS in this environment.
-Please note that Gitaly Cluster provides highly-available Git repository storage. If this is not a requirement, single-node Gitaly backed by block storage is a suitable substitute.
+Gitaly Cluster provides highly-available Git repository storage. If this is not a requirement, single-node Gitaly backed by block storage is a suitable substitute.
Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be
unavailable from GitLab 15.0. No further enhancements are planned for this feature.
@@ -338,7 +338,7 @@ following are the 4 locations need to be shared:
| -------- | ----------- | --------------------- |
| `/var/opt/gitlab/git-data` | Git repository data. This accounts for a large portion of your data | `git_data_dirs({"default" => { "path" => "/var/opt/gitlab/git-data"} })`
| `/var/opt/gitlab/gitlab-rails/uploads` | User uploaded attachments | `gitlab_rails['uploads_directory'] = '/var/opt/gitlab/gitlab-rails/uploads'`
-| `/var/opt/gitlab/gitlab-rails/shared` | Build artifacts, GitLab Pages, LFS objects, temp files, and so on. If you're using LFS this may also account for a large portion of your data | `gitlab_rails['shared_path'] = '/var/opt/gitlab/gitlab-rails/shared'`
+| `/var/opt/gitlab/gitlab-rails/shared` | Objects such as build artifacts, GitLab Pages, LFS objects, and temp files. If you're using LFS this may also account for a large portion of your data | `gitlab_rails['shared_path'] = '/var/opt/gitlab/gitlab-rails/shared'`
| `/var/opt/gitlab/gitlab-ci/builds` | GitLab CI/CD build traces | `gitlab_ci['builds_directory'] = '/var/opt/gitlab/gitlab-ci/builds'`
Other GitLab directories should not be shared between nodes. They contain
@@ -352,7 +352,7 @@ are empty before attempting a restore. Read more about the
## Testing NFS
-Once you've set up the NFS server and client, you can verify NFS is configured correctly
+When you've set up the NFS server and client, you can verify NFS is configured correctly
by testing the following commands:
```shell
diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md
index 0e85635b3d2..d2c9c35148c 100644
--- a/doc/administration/object_storage.md
+++ b/doc/administration/object_storage.md
@@ -18,7 +18,7 @@ GitLab has been tested by vendors and customers on a number of object storage pr
- [Amazon S3](https://aws.amazon.com/s3/)
- [Google Cloud Storage](https://cloud.google.com/storage)
- [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces)
-- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
+- [Oracle Cloud Infrastructure](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatible mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- [Azure Blob storage](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction)
- On-premises hardware and appliances from various storage vendors, whose list is not officially established.
@@ -88,8 +88,8 @@ Object storage for <object type> must have a bucket specified
If you want to use local storage for specific object types, you can
[selectively disable object storages](#selectively-disabling-object-storage).
-Most types of objects, such as CI artifacts, LFS files, upload
-attachments, and so on can be saved in object storage by specifying a single
+Most types of objects, such as CI artifacts, LFS files, and upload
+attachments can be saved in object storage by specifying a single
credential for object storage with multiple buckets.
When the consolidated form is:
@@ -279,7 +279,7 @@ Here are the valid connection parameters for GCS:
| `google_project` | GCP project name. | `gcp-project-12345` |
| `google_json_key_location` | JSON key path. | `/path/to/gcp-project-12345-abcde.json` |
| `google_json_key_string` | JSON key string. | `{ "type": "service_account", "project_id": "example-project-382839", ... }` |
-| `google_application_default` | Set to `true` to use [Google Cloud Application Default Credentials](https://cloud.google.com/docs/authentication/production#automatically) to locate service account credentials. | |
+| `google_application_default` | Set to `true` to use [Google Cloud Application Default Credentials](https://cloud.google.com/docs/authentication#adc) to locate service account credentials. | |
GitLab reads the value of `google_json_key_location`, then `google_json_key_string`, and finally, `google_application_default`.
It uses the first of these settings that has a value.
@@ -492,7 +492,7 @@ To migrate existing local data to object storage see the following guides:
Prior to GitLab 13.2:
- Object storage configuration for all types of objects such as CI/CD artifacts, LFS
- files, upload attachments, and so on had to be configured independently.
+ files, and upload attachments had to be configured independently.
- Object store connection parameters such as passwords and endpoint URLs had to be
duplicated for each type.
diff --git a/doc/administration/operations/moving_repositories.md b/doc/administration/operations/moving_repositories.md
index 75078568c44..96c1fcc422d 100644
--- a/doc/administration/operations/moving_repositories.md
+++ b/doc/administration/operations/moving_repositories.md
@@ -376,14 +376,3 @@ sudo -u git -H bundle exec rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\
/home/git/repositories \
/mnt/gitlab/repositories
```
-
-## Troubleshooting
-
-See the following for information on troubleshooting repository moves.
-
-### Repository move fails for archived projects
-
-Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/363670),
-[archived projects](../../user/project/settings/index.md#advanced-project-settings) fail to move even though the data is cloned
-by Gitaly. Make sure archived projects are
-[unarchived](../../user/project/settings/index.md#unarchive-a-project) before initiating a move.
diff --git a/doc/administration/operations/rails_console.md b/doc/administration/operations/rails_console.md
index 1ef985b8938..efaf480c6df 100644
--- a/doc/administration/operations/rails_console.md
+++ b/doc/administration/operations/rails_console.md
@@ -59,6 +59,40 @@ you may run in the console. To turn off logging again, run:
ActiveRecord::Base.logger = nil
```
+## Attributes
+
+View available attributes, formatted using pretty print (`pp`).
+
+For example, determine what attributes contain users' names and email addresses:
+
+```ruby
+u = User.find_by_username('someuser')
+pp u.attributes
+```
+
+Partial output:
+
+```plaintext
+{"id"=>1234,
+ "email"=>"someuser@example.com",
+ "sign_in_count"=>99,
+ "name"=>"S User",
+ "username"=>"someuser",
+ "first_name"=>nil,
+ "last_name"=>nil,
+ "bot_type"=>nil}
+```
+
+Then make use of the attributes, [testing SMTP, for example](https://docs.gitlab.com/omnibus/settings/smtp.html#testing-the-smtp-configuration):
+
+```ruby
+e = u.email
+n = u.name
+Notify.test_email(e, "Test email for #{n}", 'Test email').deliver_now
+#
+Notify.test_email(u.email, "Test email for #{u.name}", 'Test email').deliver_now
+```
+
## Disable database statement timeout
You can disable the PostgreSQL statement timeout for the current Rails console
@@ -702,7 +736,7 @@ ApplicationSetting.current
### Open object in `irb`
WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
Sometimes it is easier to go through a method if you are in the context of the object. You can shim into the namespace of `Object` to let you open `irb` in the context of any object:
diff --git a/doc/administration/package_information/defaults.md b/doc/administration/package_information/defaults.md
index c6a33ed7ba9..717e5ca31c9 100644
--- a/doc/administration/package_information/defaults.md
+++ b/doc/administration/package_information/defaults.md
@@ -68,8 +68,8 @@ over a network which will require, based on implementation, ports `111` and
`2049` to be open.
NOTE:
-In some cases, the GitLab Registry will be automatically enabled by default. Please see [our documentation](../packages/container_registry.md) for more details
+In some cases, the GitLab Registry will be automatically enabled by default. See [our documentation](../packages/container_registry.md) for more details.
- [^Consul-notes]: If using additional Consul functionality, more ports may need to be opened. See the [official documentation](https://www.consul.io/docs/install/ports#ports-table) for the list.
+ [^Consul-notes]: If using additional Consul functionality, more ports may need to be opened. See the [official documentation](https://developer.hashicorp.com/consul/docs/install/ports#ports-table) for the list.
[^Sidekiq-health]: If Sidekiq health check settings are not set, they will default to the Sidekiq metrics exporter settings. This default is deprecated and is set to be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/347509).
diff --git a/doc/administration/package_information/omnibus_packages.md b/doc/administration/package_information/omnibus_packages.md
index ec406f8b458..7b3892ace70 100644
--- a/doc/administration/package_information/omnibus_packages.md
+++ b/doc/administration/package_information/omnibus_packages.md
@@ -72,7 +72,7 @@ Some drawbacks of a package with bundled dependencies:
1. Duplication with possibly existing software.
1. Less flexibility in configuration.
-## Why would I install an omnibus package when I can use a system package?
+## Why would you install an omnibus package when you can use a system package?
The answer can be simplified to: less maintenance required. Instead of handling
multiple packages that *can* break existing functionality if the versions are
diff --git a/doc/administration/package_information/postgresql_versions.md b/doc/administration/package_information/postgresql_versions.md
index 6409c5fdbc9..c1e9f7320ea 100644
--- a/doc/administration/package_information/postgresql_versions.md
+++ b/doc/administration/package_information/postgresql_versions.md
@@ -30,6 +30,7 @@ Read more about update policies and warnings in the PostgreSQL
| GitLab version | PostgreSQL versions | Default version for fresh installs | Default version for upgrades | Notes |
| -------------- | --------------------- | ---------------------------------- | ---------------------------- | ----- |
+| 15.6 | 12.12, 13.8 | 13.8 | 12.12 | For upgrades, users can manually upgrade to 13.8 following the [upgrade documentation](https://docs.gitlab.com/omnibus/settings/database.html#gitlab-150-and-later). |
| 15.0 | 12.10, 13.6 | 13.6 | 12.10 | For upgrades, users can manually upgrade to 13.6 following the [upgrade documentation](https://docs.gitlab.com/omnibus/settings/database.html#gitlab-150-and-later). |
| 14.1 | 12.7, 13.3 | 12.7 | 12.7 | PostgreSQL 13 available for fresh installations if not using [Geo](../geo/index.md#requirements-for-running-geo) or [Patroni](../postgresql/index.md#postgresql-replication-and-failover-with-omnibus-gitlab).
| 14.0 | 12.7 | 12.7 | 12.7 | HA installations with repmgr are no longer supported and are prevented from upgrading to Omnibus GitLab 14.0 |
diff --git a/doc/administration/package_information/signed_packages.md b/doc/administration/package_information/signed_packages.md
index 34c8148e807..e4566be7534 100644
--- a/doc/administration/package_information/signed_packages.md
+++ b/doc/administration/package_information/signed_packages.md
@@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Package Signatures **(FREE SELF)**
-As of the release of GitLab 9.5 on August 22, 2017, GitLab provides signed Omnibus GitLab packages for RPM and DEB based distributions. This means that all packages provided on <https://packages.gitlab.com> are signed, starting with `9.5.0`, and all future versions of supported branches (for example `9.3.x` and `9.4.x` after August 22, 2017). Any package version prior to August 22, 2017, will not be signed. Please pass the appropriate argument to your package manager. (Example: `yum --nogpgcheck`)
+As of the release of GitLab 9.5 on August 22, 2017, GitLab provides signed Omnibus GitLab packages for RPM and DEB based distributions. This means that all packages provided on <https://packages.gitlab.com> are signed, starting with `9.5.0`, and all future versions of supported branches (for example `9.3.x` and `9.4.x` after August 22, 2017). Any package version prior to August 22, 2017, will not be signed. Pass the appropriate argument to your package manager. (Example: `yum --nogpgcheck`)
Omnibus GitLab packages produced by GitLab are created via the [Omnibus](https://github.com/chef/omnibus) tool, for which GitLab has added DEB signing via `debsigs` in [our own fork](https://gitlab.com/gitlab-org/omnibus). This addition, combined with the existing functionality of RPM signing, allows GitLab to provide signed packages for all supported distributions using DEB or RPM.
diff --git a/doc/administration/packages/container_registry.md b/doc/administration/packages/container_registry.md
index d04e3217f57..2623f2afd8d 100644
--- a/doc/administration/packages/container_registry.md
+++ b/doc/administration/packages/container_registry.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -198,7 +198,8 @@ docker login gitlab.example.com:5050
When the Registry is configured to use its own domain, you need a TLS
certificate for that specific domain (for example, `registry.example.com`). You might need
a wildcard certificate if hosted under a subdomain of your existing GitLab
-domain, for example, `registry.gitlab.example.com`.
+domain. For example, `*.gitlab.example.com`, is a wildcard that matches `registry.gitlab.example.com`,
+and is distinct from `*.example.com`.
As well as manually generated SSL certificates (explained here), certificates automatically
generated by Let's Encrypt are also [supported in Omnibus installs](https://docs.gitlab.com/omnibus/settings/ssl.html).
@@ -1156,7 +1157,7 @@ blobs start being deleted is anything permanent done.
## Configuring GitLab and Registry to run on separate nodes (Omnibus GitLab)
By default, package assumes that both services are running on the same node.
-In order to get GitLab and Registry to run on a separate nodes, separate configuration
+To get GitLab and Registry to run on a separate nodes, separate configuration
is necessary for Registry and GitLab.
### Configuring Registry
diff --git a/doc/administration/packages/dependency_proxy.md b/doc/administration/packages/dependency_proxy.md
index 789863e8ed0..f6eb6f85274 100644
--- a/doc/administration/packages/dependency_proxy.md
+++ b/doc/administration/packages/dependency_proxy.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -215,11 +215,12 @@ For installations from source:
RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:dependency_proxy:migrate
```
-You can optionally track progress and verify that all packages migrated successfully using the
+You can optionally track progress and verify that all Dependency Proxy blobs and manifests migrated successfully using the
[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database):
-- For Omnibus GitLab instances: `sudo gitlab-rails dbconsole`
-- For installations from source: `sudo -u git -H psql -d gitlabhq_production`
+- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier.
+- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later.
+- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances.
Verify that `objectstg` (where `file_store = '2'`) has the count of all Dependency Proxy blobs and
manifests for each respective query:
diff --git a/doc/administration/packages/index.md b/doc/administration/packages/index.md
index a7ab0fb3246..74d835eb744 100644
--- a/doc/administration/packages/index.md
+++ b/doc/administration/packages/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -231,7 +231,8 @@ RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:packages:migrate
You can optionally track progress and verify that all packages migrated successfully using the
[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database):
-- `sudo gitlab-rails dbconsole` for Omnibus GitLab instances.
+- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier.
+- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later.
- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances.
Verify `objectstg` below (where `file_store = '2'`) has count of all packages:
diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md
index 922f9a27aad..3d31491a9d2 100644
--- a/doc/administration/pages/index.md
+++ b/doc/administration/pages/index.md
@@ -260,6 +260,7 @@ control over how the Pages daemon runs and serves content in your environment.
| `gitlab_id` | The OAuth application public ID. Leave blank to automatically fill when Pages authenticates with GitLab. |
| `gitlab_secret` | The OAuth application secret. Leave blank to automatically fill when Pages authenticates with GitLab. |
| `auth_scope` | The OAuth application scope to use for authentication. Must match GitLab Pages OAuth application settings. Leave blank to use `api` scope by default. |
+| `auth_cookie_session_timeout` | Authentication cookie session timeout in seconds (default: 600s). A value of `0` means the cookie is deleted after the browser session ends. |
| `gitlab_server` | Server to use for authentication when access control is enabled; defaults to GitLab `external_url`. |
| `headers` | Specify any additional http headers that should be sent to the client with each response. Multiple headers can be given as an array, header and value as one string, for example `['my-header: myvalue', 'my-other-header: my-other-value']` |
| `enable_disk` | Allows the GitLab Pages daemon to serve content from disk. Shall be disabled if shared disk storage isn't available. |
@@ -512,7 +513,7 @@ internet connectivity is gated by a proxy. To use a proxy for GitLab Pages:
### Using a custom Certificate Authority (CA)
-When using certificates issued by a custom CA, [Access Control](../../user/project/pages/pages_access_control.md#gitlab-pages-access-control) and
+When using certificates issued by a custom CA, [Access Control](../../user/project/pages/pages_access_control.md) and
the [online view of HTML job artifacts](../../ci/pipelines/job_artifacts.md#download-job-artifacts)
fails to work if the custom CA is not recognized.
@@ -820,8 +821,8 @@ database encryption. Proceed with caution.
It's possible to run GitLab Pages on multiple servers if you wish to distribute
the load. You can do this through standard load balancing practices such as
-configuring your DNS server to return multiple IPs for your Pages server,
-configuring a load balancer to work at the IP level, and so on. If you wish to
+configuring your DNS server to return multiple IPs for your Pages server, or
+configuring a load balancer to work at the IP level. If you wish to
set up GitLab Pages on multiple servers, perform the above procedure for each
Pages server.
@@ -1073,7 +1074,8 @@ sudo gitlab-rake gitlab:pages:deployments:migrate_to_object_storage
You can track progress and verify that all Pages deployments migrated successfully using the
[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database):
-- `sudo gitlab-rails dbconsole` for Omnibus GitLab instances.
+- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier.
+- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later.
- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances.
Verify `objectstg` below (where `store=2`) has count of all Pages deployments:
@@ -1213,7 +1215,7 @@ the section below.
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md
index 52556809845..e122d49a963 100644
--- a/doc/administration/pages/source.md
+++ b/doc/administration/pages/source.md
@@ -459,7 +459,7 @@ Pages access control is disabled by default. To enable it:
auth-server=<URL of the GitLab instance>
```
-1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control).
+1. Users can now configure it in their [projects' settings](../../user/project/pages/pages_access_control.md).
## Change storage path
diff --git a/doc/administration/polling.md b/doc/administration/polling.md
index 11f26f081cb..deb6e89183d 100644
--- a/doc/administration/polling.md
+++ b/doc/administration/polling.md
@@ -6,8 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Polling interval multiplier **(FREE SELF)**
-The GitLab UI polls for updates for different resources (issue notes, issue titles, pipeline
-statuses, and so on) on a schedule appropriate to the resource.
+The GitLab UI polls for updates for different resources (such as issue notes, issue titles, and pipeline
+statuses) on a schedule appropriate to the resource.
Adjust the multiplier on these schedules to adjust how frequently the GitLab UI polls for updates. If
you set the multiplier to:
diff --git a/doc/administration/postgresql/database_load_balancing.md b/doc/administration/postgresql/database_load_balancing.md
index 6bf36ef4794..805c4b2023f 100644
--- a/doc/administration/postgresql/database_load_balancing.md
+++ b/doc/administration/postgresql/database_load_balancing.md
@@ -130,6 +130,7 @@ record. For example:
| `interval` | The minimum time in seconds between checking the DNS record. | 60 |
| `disconnect_timeout` | The time in seconds after which an old connection is closed, after the list of hosts was updated. | 120 |
| `use_tcp` | Lookup DNS resources using TCP instead of UDP | false |
+| `max_replica_pools` | The maximum number of replicas each Rails process connects to. This is useful if you run a lot of Postgres replicas and a lot of Rails processes because without this limit every Rails process connects to every replica by default. The default behavior is unlimited if not set. | nil |
If `record_type` is set to `SRV`, then GitLab continues to use round-robin algorithm
and ignores the `weight` and `priority` in the record. Since `SRV` records usually
diff --git a/doc/administration/postgresql/replication_and_failover.md b/doc/administration/postgresql/replication_and_failover.md
index 0ee48047944..ee90b120d05 100644
--- a/doc/administration/postgresql/replication_and_failover.md
+++ b/doc/administration/postgresql/replication_and_failover.md
@@ -606,7 +606,7 @@ Here is a list and description of each machine and the assigned IP:
- `10.6.0.33`: PostgreSQL 3
- `10.6.0.41`: GitLab application
-All passwords are set to `toomanysecrets`. Please do not use this password or derived hashes and the `external_url` for GitLab is `http://gitlab.example.com`.
+All passwords are set to `toomanysecrets`. Do not use this password or derived hashes and the `external_url` for GitLab is `http://gitlab.example.com`.
After the initial configuration, if a failover occurs, the PostgresSQL leader node changes to one of the available secondaries until it is failed back.
@@ -957,7 +957,7 @@ For further details, see [Patroni documentation on this subject](https://patroni
### Switching from repmgr to Patroni
WARNING:
-Switching from repmgr to Patroni is straightforward, the other way around is *not*. Rolling back from Patroni to repmgr can be complicated and may involve deletion of data directory. If you need to do that, please contact GitLab support.
+Switching from repmgr to Patroni is straightforward, the other way around is *not*. Rolling back from Patroni to repmgr can be complicated and may involve deletion of data directory. If you need to do that, contact GitLab support.
You can switch an exiting database cluster to use Patroni instead of repmgr with the following steps:
diff --git a/doc/administration/raketasks/uploads/migrate.md b/doc/administration/raketasks/uploads/migrate.md
index b6f14bc6fa4..1f6e7fda082 100644
--- a/doc/administration/raketasks/uploads/migrate.md
+++ b/doc/administration/raketasks/uploads/migrate.md
@@ -42,10 +42,11 @@ gitlab-rake "gitlab:uploads:migrate:all"
sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate:all
```
-You can optionally track progress and verify that all packages migrated successfully using the
+You can optionally track progress and verify that all uploads migrated successfully using the
[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database):
-- `sudo gitlab-rails dbconsole` for Omnibus GitLab instances.
+- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier.
+- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later.
- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances.
Verify `objectstg` below (where `store=2`) has count of all artifacts:
diff --git a/doc/administration/redis/replication_and_failover.md b/doc/administration/redis/replication_and_failover.md
index 1c2515099fe..2ba19aa6f0a 100644
--- a/doc/administration/redis/replication_and_failover.md
+++ b/doc/administration/redis/replication_and_failover.md
@@ -66,7 +66,7 @@ When a **Primary** fails to respond, it's the application's responsibility
(in our case GitLab) to handle timeout and reconnect (querying a **Sentinel**
for a new **Primary**).
-To get a better understanding on how to correctly set up Sentinel, please read
+To get a better understanding on how to correctly set up Sentinel, read
the [Redis Sentinel](https://redis.io/docs/manual/sentinel/) documentation first, as
failing to configure it correctly can lead to data loss or can bring your
whole cluster down, invalidating the failover effort.
@@ -350,7 +350,7 @@ Now that the Redis servers are all set up, let's configure the Sentinel
servers.
If you are not sure if your Redis servers are working and replicating
-correctly, please read the [Troubleshooting Replication](troubleshooting.md#troubleshooting-redis-replication)
+correctly, read the [Troubleshooting Replication](troubleshooting.md#troubleshooting-redis-replication)
and fix it before proceeding with Sentinel setup.
You must have at least `3` Redis Sentinel servers, and they need to
diff --git a/doc/administration/redis/replication_and_failover_external.md b/doc/administration/redis/replication_and_failover_external.md
index 7904fb1ded8..23c9ce33c2d 100644
--- a/doc/administration/redis/replication_and_failover_external.md
+++ b/doc/administration/redis/replication_and_failover_external.md
@@ -64,7 +64,7 @@ settings outlined in
We cannot stress enough the importance of reading the
[replication and failover](replication_and_failover.md) documentation of the
Omnibus Redis HA as it provides some invaluable information to the configuration
-of Redis. Please proceed to read it before going forward with this guide.
+of Redis. Read it before going forward with this guide.
Before proceeding on setting up the new Redis instances, here are some
requirements:
diff --git a/doc/administration/redis/troubleshooting.md b/doc/administration/redis/troubleshooting.md
index e568daed961..29407f65efd 100644
--- a/doc/administration/redis/troubleshooting.md
+++ b/doc/administration/redis/troubleshooting.md
@@ -26,8 +26,8 @@ Start Redis troubleshooting with a basic Redis activity check:
1. Open a terminal on your GitLab server.
1. Run `gitlab-redis-cli --stat` and observe the output while it runs.
-1. Go to your GitLab UI and browse to a handful of pages. Any page works, like
- group or project overviews, issues, files in repositories, and so on.
+1. Go to your GitLab UI and browse to a handful of pages. Any page works, such as
+ group or project overviews, issues, or files in repositories.
1. Check the `stat` output again and verify that the values for `keys`, `clients`,
`requests`, and `connections` increases as you browse. If the numbers go up,
basic Redis functionality is working and GitLab can connect to it.
diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md
index 45939b48f78..a2463c6ff88 100644
--- a/doc/administration/reference_architectures/10k_users.md
+++ b/doc/administration/reference_architectures/10k_users.md
@@ -28,7 +28,7 @@ full list of reference architectures, see
| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` |
| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
@@ -50,6 +50,7 @@ full list of reference architectures, see
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
@@ -158,10 +159,45 @@ Any "burstable" instance types are not recommended due to inconsistent performan
### Supported infrastructure
-As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services, or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section. However, this does not constitute a guarantee for every potential permutation.
+As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services,
+or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section.
+However, this does not constitute a guarantee for every potential permutation.
See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
+### Additional workloads
+
+The Reference Architectures have been [designed and tested](index.md#validation-and-test-results) for standard GitLab setups with
+good headroom in mind to cover most scenarios. However, if any additional workloads are being added on the nodes,
+such as security software, you may still need to adjust the specs accordingly to compensate.
+
+This also applies for some GitLab features where it's possible to run custom scripts, for example [server hooks](../server_hooks.md).
+
+As a general rule, it's recommended to have robust monitoring in place to measure the impact of
+any additional workloads to inform any changes needed to be made.
+
+### Large repositories
+
+The Reference Architectures were tested with repositories of varying sizes that follow best practices.
+
+However, large repositories or monorepos (several gigabytes or more) can **significantly** impact the performance
+of Git and in turn the environment itself if best practices aren't being followed such as not storing
+binary or blob files in LFS. Repositories are at the core of any environment the consequences can be wide-ranging
+when they are not optimized. Some examples of this impact include [Git packing operations](https://git-scm.com/book/en/v2/Git-Internals-Packfiles)
+taking longer and consuming high CPU / Memory resources or Git checkouts taking longer that affect both users and
+CI pipelines alike.
+
+As such, large repositories come with notable cost and typically will require more resources to handle,
+significantly so in some cases. It's therefore **strongly** recommended then to review large repositories
+to ensure they maintain good repo health and reduce their size wherever possible.
+
+NOTE:
+If best practices aren't followed and large repositories are present on the environment,
+increased Gitaly specs may be required to ensure stable performance.
+
+Refer to the [Managing large repositories documentation](../../user/project/repository/managing_large_repositories.md)
+for more information and guidance.
+
### Praefect PostgreSQL
It's worth noting that at this time [Praefect requires its own database server](../gitaly/praefect.md#postgresql) and
@@ -241,8 +277,7 @@ In a multi-node GitLab configuration, you'll need a load balancer to route
traffic to the application servers. The specifics on which load balancer to use
or its exact configuration is beyond the scope of GitLab documentation. We assume
that if you're managing multi-node systems like GitLab, you already have a load
-balancer of choice and that the routing methods used are distributing calls evenly
-between all nodes. Some load balancer examples include HAProxy (open-source),
+balancer of choice. Some load balancer examples include HAProxy (open-source),
F5 Big-IP LTM, and Citrix Net Scaler. This documentation outline the ports and
protocols needed for use with GitLab.
@@ -250,47 +285,13 @@ This architecture has been tested and validated with [HAProxy](https://www.hapro
as the load balancer. Although other load balancers with similar feature sets
could also be used, those load balancers have not been validated.
-The next question is how you will handle SSL in your environment.
-There are several different options:
+### Balancing algorithm
-- [The application node terminates SSL](#application-node-terminates-ssl).
-- [The load balancer terminates SSL without backend SSL](#load-balancer-terminates-ssl-without-backend-ssl)
- and communication is not secure between the load balancer and the application node.
-- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
- and communication is *secure* between the load balancer and the application node.
-
-### Application node terminates SSL
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
-Configure your load balancer to pass connections on port 443 as `TCP` rather
-than `HTTP(S)` protocol. This will pass the connection to the application node's
-NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
-
-See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
-
-### Load balancer terminates SSL without backend SSL
-
-Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
-The load balancer will then be responsible for managing SSL certificates and
-terminating SSL.
-
-Since communication between the load balancer and GitLab will not be secure,
-there is some additional configuration needed. See the
-[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
-for details.
-
-### Load balancer terminates SSL with backend SSL
-
-Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
-The load balancers will be responsible for managing SSL certificates that
-end users will see.
-
-Traffic will also be secure between the load balancers and NGINX in this
-scenario. There is no need to add configuration for proxied SSL since the
-connection will be secure all the way. However, configuration will need to be
-added to GitLab to configure SSL certificates. See
-the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
### Readiness checks
@@ -351,6 +352,50 @@ Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`.
| ------- | ------------ | -------- |
| 443 | 22 | TCP |
+### SSL
+
+The next question is how you will handle SSL in your environment.
+There are several different options:
+
+- [The application node terminates SSL](#application-node-terminates-ssl).
+- [The load balancer terminates SSL without backend SSL](#load-balancer-terminates-ssl-without-backend-ssl)
+ and communication is not secure between the load balancer and the application node.
+- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
+ and communication is *secure* between the load balancer and the application node.
+
+#### Application node terminates SSL
+
+Configure your load balancer to pass connections on port 443 as `TCP` rather
+than `HTTP(S)` protocol. This will pass the connection to the application node's
+NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
+
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
+#### Load balancer terminates SSL without backend SSL
+
+Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
+The load balancer will then be responsible for managing SSL certificates and
+terminating SSL.
+
+Since communication between the load balancer and GitLab will not be secure,
+there is some additional configuration needed. See the
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
+for details.
+
+#### Load balancer terminates SSL with backend SSL
+
+Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
+The load balancers will be responsible for managing SSL certificates that
+end users will see.
+
+Traffic will also be secure between the load balancers and NGINX in this
+scenario. There is no need to add configuration for proxied SSL since the
+connection will be secure all the way. However, configuration will need to be
+added to GitLab to configure SSL certificates. See
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
@@ -415,8 +460,14 @@ backend praefect
```
Refer to your preferred Load Balancer's documentation for further guidance.
-Also ensure that the routing methods used are distributing calls evenly across
-all nodes.
+
+### Balancing algorithm
+
+We recommend that a least-connection-based load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
+
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
@@ -1168,6 +1219,12 @@ NOTE:
Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster).
For implementations with sharded Gitaly, use the same Gitaly specs. Follow the [separate Gitaly documentation](../gitaly/configure_gitaly.md) instead of this section.
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
+
The recommended cluster setup includes the following components:
- 3 Gitaly nodes: Replicated storage of Git repositories.
@@ -1475,9 +1532,15 @@ The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have
requirements that are dependent on data and load.
NOTE:
-The Reference Architecture specs have been designed with good headroom in mind
-but for Gitaly, increased specs or additional
-Gitaly Cluster arrays may be required for notably large data sets or load.
+Increased specs for Gitaly nodes may be required in some circumstances such as
+significantly large repositories or if any [additional workloads](#additional-workloads),
+such as [server hooks](../server_hooks.md), have been added.
+
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
Due to Gitaly having notable input and output requirements, we strongly
recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs
@@ -1790,7 +1853,7 @@ Updates to example must be made at:
gitlab_rails['auto_migrate'] = false
# Sidekiq
- sidekiqp['enable'] = true
+ sidekiq['enable'] = true
sidekiq['listen_address'] = "0.0.0.0"
# Set number of Sidekiq queue processes to the same number as available CPUs
@@ -2155,7 +2218,7 @@ GitLab has been tested on a number of object storage providers:
- [Amazon S3](https://aws.amazon.com/s3/)
- [Google Cloud Storage](https://cloud.google.com/storage)
- [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces)
-- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
+- [Oracle Cloud Infrastructure](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
@@ -2237,6 +2300,10 @@ compute deployments. With this, _stateless_ components can benefit from cloud na
workload management benefits while _stateful_ components are deployed in compute VMs
with Omnibus to benefit from increased permanence.
+Refer to the Helm charts [Advanced configuration](https://docs.gitlab.com/charts/advanced/)
+documentation for setup instructions including guidance on what GitLab secrets to sync
+between Kubernetes and the backend components.
+
NOTE:
This is an **advanced** setup. Running services in Kubernetes is well known
to be complex. **This setup is only recommended** if you have strong working
@@ -2279,7 +2346,7 @@ services where applicable):
| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Gitaly<sup>5</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 16 vCPU, 60 GB memory | `n1-standard-16` | `m5.4xlarge` |
| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Object storage<sup>4</sup> | - | - | - | - |
@@ -2297,6 +2364,7 @@ services where applicable):
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
@@ -2403,7 +2471,7 @@ ratio for each additional pod.
For further information on resource usage, see the [Sidekiq resources](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/#resources).
-### Supporting
+#### Supporting
The Supporting Node Pool is designed to house all supporting deployments that don't need to be
on the Webservice and Sidekiq pools.
@@ -2416,6 +2484,12 @@ to deploy these in this pool where possible and not in the Webservice or Sidekiq
specifically to accommodate several additional deployments. However, if your deployments don't fit into the
pool as given, you can increase the node pool accordingly.
+## Secrets
+
+When setting up a Cloud Native Hybrid environment, it's worth noting that several secrets should be synced from backend VMs from the `/etc/gitlab/gitlab-secrets.json` file into Kubernetes.
+
+For this setup specifically, the [GitLab Rails](https://docs.gitlab.com/charts/installation/secrets.html#gitlab-rails-secret) and [GitLab Shell](https://docs.gitlab.com/charts/installation/secrets.html#gitlab-rails-secret) secrets should be synced.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md
index a8e0e23512f..2a9636b6e05 100644
--- a/doc/administration/reference_architectures/1k_users.md
+++ b/doc/administration/reference_architectures/1k_users.md
@@ -82,10 +82,23 @@ Any "burstable" instance types are not recommended due to inconsistent performan
### Supported infrastructure
-As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP, Azure) and their services, or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section. However, this does not constitute a guarantee for every potential permutation.
+As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services,
+or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section.
+However, this does not constitute a guarantee for every potential permutation.
See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
+### Additional workloads
+
+The Reference Architectures have been [designed and tested](index.md#validation-and-test-results) for standard GitLab setups with
+good headroom in mind to cover most scenarios. However, if any additional workloads are being added on the nodes,
+such as security software, you may still need to adjust the specs accordingly to compensate.
+
+This also applies for some GitLab features where it's possible to run custom scripts, for example [server hooks](../server_hooks.md).
+
+As a general rule, it's recommended to have robust monitoring in place to measure the impact of
+any additional workloads to inform any changes needed to be made.
+
### Swap
In addition to the stated configurations, we recommend having at least 2 GB of
diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md
index 7d67ac48b73..84eba01fe11 100644
--- a/doc/administration/reference_architectures/25k_users.md
+++ b/doc/administration/reference_architectures/25k_users.md
@@ -28,7 +28,7 @@ full list of reference architectures, see
| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` |
| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` |
| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
@@ -50,6 +50,7 @@ full list of reference architectures, see
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
@@ -158,10 +159,45 @@ Any "burstable" instance types are not recommended due to inconsistent performan
### Supported infrastructure
-As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services, or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section. However, this does not constitute a guarantee for every potential permutation.
+As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP, Azure) and their services,
+or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section.
+However, this does not constitute a guarantee for every potential permutation.
See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
+### Additional workloads
+
+The Reference Architectures have been [designed and tested](index.md#validation-and-test-results) for standard GitLab setups with
+good headroom in mind to cover most scenarios. However, if any additional workloads are being added on the nodes,
+such as security software, you may still need to adjust the specs accordingly to compensate.
+
+This also applies for some GitLab features where it's possible to run custom scripts, for example [server hooks](../server_hooks.md).
+
+As a general rule, it's recommended to have robust monitoring in place to measure the impact of
+any additional workloads to inform any changes needed to be made.
+
+### Large repositories
+
+The Reference Architectures were tested with repositories of varying sizes that follow best practices.
+
+However, large repositories or monorepos (several gigabytes or more) can **significantly** impact the performance
+of Git and in turn the environment itself if best practices aren't being followed such as not storing
+binary or blob files in LFS. Repositories are at the core of any environment the consequences can be wide-ranging
+when they are not optimized. Some examples of this impact include [Git packing operations](https://git-scm.com/book/en/v2/Git-Internals-Packfiles)
+taking longer and consuming high CPU / Memory resources or Git checkouts taking longer that affect both users and
+CI pipelines alike.
+
+As such, large repositories come with notable cost and typically will require more resources to handle,
+significantly so in some cases. It's therefore **strongly** recommended then to review large repositories
+to ensure they maintain good repo health and reduce their size wherever possible.
+
+NOTE:
+If best practices aren't followed and large repositories are present on the environment,
+increased Gitaly specs may be required to ensure stable performance.
+
+Refer to the [Managing large repositories documentation](../../user/project/repository/managing_large_repositories.md)
+for more information and guidance.
+
### Praefect PostgreSQL
It's worth noting that at this time [Praefect requires its own database server](../gitaly/praefect.md#postgresql) and
@@ -243,8 +279,7 @@ In a multi-node GitLab configuration, you'll need a load balancer to route
traffic to the application servers. The specifics on which load balancer to use
or its exact configuration is beyond the scope of GitLab documentation. We assume
that if you're managing multi-node systems like GitLab, you already have a load
-balancer of choice and that the routing methods used are distributing calls evenly
-between all nodes. Some load balancer examples include HAProxy (open-source),
+balancer of choice. Some load balancer examples include HAProxy (open-source),
F5 Big-IP LTM, and Citrix Net Scaler. This documentation outline the ports and
protocols needed for use with GitLab.
@@ -261,38 +296,13 @@ There are several different options:
- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
and communication is *secure* between the load balancer and the application node.
-### Application node terminates SSL
-
-Configure your load balancer to pass connections on port 443 as `TCP` rather
-than `HTTP(S)` protocol. This will pass the connection to the application node's
-NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
-
-See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
-
-### Load balancer terminates SSL without backend SSL
-
-Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
-The load balancer will then be responsible for managing SSL certificates and
-terminating SSL.
-
-Since communication between the load balancer and GitLab will not be secure,
-there is some additional configuration needed. See the
-[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
-for details.
+### Balancing algorithm
-### Load balancer terminates SSL with backend SSL
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
-Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
-The load balancers will be responsible for managing SSL certificates that
-end users will see.
-
-Traffic will also be secure between the load balancers and NGINX in this
-scenario. There is no need to add configuration for proxied SSL since the
-connection will be secure all the way. However, configuration will need to be
-added to GitLab to configure SSL certificates. See the
-[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
### Readiness checks
@@ -353,6 +363,50 @@ Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`.
| ------- | ------------ | -------- |
| 443 | 22 | TCP |
+### SSL
+
+The next question is how you will handle SSL in your environment.
+There are several different options:
+
+- [The application node terminates SSL](#application-node-terminates-ssl).
+- [The load balancer terminates SSL without backend SSL](#load-balancer-terminates-ssl-without-backend-ssl)
+ and communication is not secure between the load balancer and the application node.
+- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
+ and communication is *secure* between the load balancer and the application node.
+
+#### Application node terminates SSL
+
+Configure your load balancer to pass connections on port 443 as `TCP` rather
+than `HTTP(S)` protocol. This will pass the connection to the application node's
+NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
+
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
+#### Load balancer terminates SSL without backend SSL
+
+Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
+The load balancer will then be responsible for managing SSL certificates and
+terminating SSL.
+
+Since communication between the load balancer and GitLab will not be secure,
+there is some additional configuration needed. See the
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
+for details.
+
+#### Load balancer terminates SSL with backend SSL
+
+Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
+The load balancers will be responsible for managing SSL certificates that
+end users will see.
+
+Traffic will also be secure between the load balancers and NGINX in this
+scenario. There is no need to add configuration for proxied SSL since the
+connection will be secure all the way. However, configuration will need to be
+added to GitLab to configure SSL certificates. See
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
@@ -417,8 +471,20 @@ backend praefect
```
Refer to your preferred Load Balancer's documentation for further guidance.
-Also ensure that the routing methods used are distributing calls evenly across
-all nodes.
+
+### Balancing algorithm
+
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
+
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
+
+<div align="right">
+ <a type="button" class="btn btn-default" href="#setup-components">
+ Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
+ </a>
+</div>
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
@@ -1173,6 +1239,12 @@ NOTE:
Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster).
For implementations with sharded Gitaly, use the same Gitaly specs. Follow the [separate Gitaly documentation](../gitaly/configure_gitaly.md) instead of this section.
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
+
The recommended cluster setup includes the following components:
- 3 Gitaly nodes: Replicated storage of Git repositories.
@@ -1478,9 +1550,15 @@ The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have
requirements that are dependent on data and load.
NOTE:
-The Reference Architecture specs have been designed with good headroom in mind
-but for Gitaly, increased specs or additional
-Gitaly Cluster arrays may be required for notably large data sets or load.
+Increased specs for Gitaly nodes may be required in some circumstances such as
+significantly large repositories or if any [additional workloads](#additional-workloads),
+such as [server hooks](../server_hooks.md), have been added.
+
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
Due to Gitaly having notable input and output requirements, we strongly
recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs
@@ -2159,7 +2237,7 @@ GitLab has been tested on a number of object storage providers:
- [Amazon S3](https://aws.amazon.com/s3/)
- [Google Cloud Storage](https://cloud.google.com/storage)
- [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces)
-- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
+- [Oracle Cloud Infrastructure](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
@@ -2241,6 +2319,10 @@ compute deployments. With this, _stateless_ components can benefit from cloud na
workload management benefits while _stateful_ components are deployed in compute VMs
with Omnibus to benefit from increased permanence.
+Refer to the Helm charts [Advanced configuration](https://docs.gitlab.com/charts/advanced/)
+documentation for setup instructions including guidance on what GitLab secrets to sync
+between Kubernetes and the backend components.
+
NOTE:
This is an **advanced** setup. Running services in Kubernetes is well known
to be complex. **This setup is only recommended** if you have strong working
@@ -2283,7 +2365,7 @@ services where applicable):
| Internal load balancing node<sup>3</sup> | 1 | 4 vCPU, 3.6GB memory | `n1-highcpu-4` | `c5.xlarge` |
| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Gitaly<sup>5</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 32 vCPU, 120 GB memory | `n1-standard-32` | `m5.8xlarge` |
| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Object storage<sup>4</sup> | - | - | - | - |
@@ -2301,6 +2383,7 @@ services where applicable):
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md
index 61ea435f63f..1acae93f764 100644
--- a/doc/administration/reference_architectures/2k_users.md
+++ b/doc/administration/reference_architectures/2k_users.md
@@ -25,7 +25,7 @@ For a full list of reference architectures, see
| Load balancer<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
| PostgreSQL<sup>1</sup> | 1 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` | `D2s v3` |
| Redis<sup>2</sup> | 1 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | `m5.large` | `D2s v3` |
-| Gitaly | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
+| Gitaly<sup>5</sup> | 1 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` | `D4s v3` |
| GitLab Rails | 2 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` |
| Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` |
| Object storage<sup>4</sup> | - | - | - | - | - |
@@ -35,13 +35,14 @@ For a full list of reference architectures, see
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
- [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work.
- [Google AlloyDB](https://cloud.google.com/alloydb) and [Amazon RDS Multi-AZ DB clusters](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) have not been tested and are not recommended. Both solutions are specifically not expected to work with GitLab Geo.
- - [Amazon Aurora](https://aws.amazon.com/rds/aurora/) is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and [Azure Database for PostgreSQL](https://azure.microsoft.com/en-gb/services/postgresql/) is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61).
+ - [Amazon Aurora](https://aws.amazon.com/rds/aurora/) is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and [Azure Database for PostgreSQL](https://azure.microsoft.com/en-gb/products/postgresql/#overview) is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61).
- Consul is primarily used for Omnibus PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
- [Google Memorystore](https://cloud.google.com/memorystore) and [Amazon ElastiCache](https://aws.amazon.com/elasticache/) are known to work.
3. Can be optionally run on reputable third-party load balancing services (LB PaaS). See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
+5. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
@@ -94,10 +95,45 @@ Any "burstable" instance types are not recommended due to inconsistent performan
### Supported infrastructure
-As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP, Azure) and their services, or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section. However, this does not constitute a guarantee for every potential permutation.
+As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services,
+or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section.
+However, this does not constitute a guarantee for every potential permutation.
See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
+### Additional workloads
+
+The Reference Architectures have been [designed and tested](index.md#validation-and-test-results) for standard GitLab setups with
+good headroom in mind to cover most scenarios. However, if any additional workloads are being added on the nodes,
+such as security software, you may still need to adjust the specs accordingly to compensate.
+
+This also applies for some GitLab features where it's possible to run custom scripts, for example [server hooks](../server_hooks.md).
+
+As a general rule, it's recommended to have robust monitoring in place to measure the impact of
+any additional workloads to inform any changes needed to be made.
+
+### Large repositories
+
+The Reference Architectures were tested with repositories of varying sizes that follow best practices.
+
+However, large repositories or monorepos (several gigabytes or more) can **significantly** impact the performance
+of Git and in turn the environment itself if best practices aren't being followed such as not storing
+binary or blob files in LFS. Repositories are at the core of any environment the consequences can be wide-ranging
+when they are not optimized. Some examples of this impact include [Git packing operations](https://git-scm.com/book/en/v2/Git-Internals-Packfiles)
+taking longer and consuming high CPU / Memory resources or Git checkouts taking longer that affect both users and
+CI pipelines alike.
+
+As such, large repositories come with notable cost and typically will require more resources to handle,
+significantly so in some cases. It's therefore **strongly** recommended then to review large repositories
+to ensure they maintain good repo health and reduce their size wherever possible.
+
+NOTE:
+If best practices aren't followed and large repositories are present on the environment,
+increased Gitaly specs may be required to ensure stable performance.
+
+Refer to the [Managing large repositories documentation](../../user/project/repository/managing_large_repositories.md)
+for more information and guidance.
+
## Setup components
To set up GitLab and its components to accommodate up to 2,000 users:
@@ -127,8 +163,7 @@ In a multi-node GitLab configuration, you'll need a load balancer to route
traffic to the application servers. The specifics on which load balancer to use
or its exact configuration is beyond the scope of GitLab documentation. We assume
that if you're managing multi-node systems like GitLab, you already have a load
-balancer of choice and that the routing methods used are distributing calls evenly
-between all nodes. Some load balancer examples include HAProxy (open-source),
+balancer of choice. Some load balancer examples include HAProxy (open-source),
F5 Big-IP LTM, and Citrix Net Scaler. This documentation outline the ports and
protocols needed for use with GitLab.
@@ -145,36 +180,13 @@ several different options:
- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
and communication is *secure* between the load balancer and the application node.
-### Application node terminates SSL
-
-Configure your load balancer to pass connections on port 443 as `TCP` instead
-of `HTTP(S)`. This will pass the connection unaltered to the application node's
-NGINX service, which has the SSL certificate and listens to port 443.
-
-For details about managing SSL certificates and configuring NGINX, see the
-[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-
-### Load balancer terminates SSL without backend SSL
-
-Configure your load balancer to use the `HTTP(S)` protocol instead of `TCP`.
-The load balancer will be responsible for both managing SSL certificates and
-terminating SSL.
-
-Due to communication between the load balancer and GitLab not being secure,
-you'll need to complete some additional configuration. For details, see the
-[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination).
-
-### Load balancer terminates SSL with backend SSL
+### Balancing algorithm
-Configure your load balancers (or single balancer, if you have only one) to use
-the `HTTP(S)` protocol rather than `TCP`. The load balancers will be
-responsible for the managing SSL certificates for end users.
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
-Traffic will be secure between the load balancers and NGINX in this scenario,
-and there's no need to add a configuration for proxied SSL. However, you'll
-need to add a configuration to GitLab to configure SSL certificates. For
-details about managing SSL certificates and configuring NGINX, see the
-[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html).
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
### Readiness checks
@@ -186,56 +198,99 @@ connect.
### Ports
-The basic load balancer ports you should use are described in the following
-table:
+The basic ports to be used are shown in the table below.
-| Port | Backend Port | Protocol |
+| LB Port | Backend Port | Protocol |
| ------- | ------------ | ------------------------ |
| 80 | 80 | HTTP (*1*) |
| 443 | 443 | TCP or HTTPS (*1*) (*2*) |
| 22 | 22 | TCP |
-- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals-deprecated) support
- requires your load balancer to correctly handle WebSocket connections.
- When using HTTP or HTTPS proxying, your load balancer must be configured
- to pass through the `Connection` and `Upgrade` hop-by-hop headers. For
- details, see the [web terminal](../integration/terminal.md) integration guide.
-- (*2*): When using the HTTPS protocol for port 443, you'll need to add an SSL
- certificate to the load balancers. If you need to terminate SSL at the
- GitLab application server, use the TCP protocol.
+- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals-deprecated) support requires
+ your load balancer to correctly handle WebSocket connections. When using
+ HTTP or HTTPS proxying, this means your load balancer must be configured
+ to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the
+ [web terminal](../integration/terminal.md) integration guide for
+ more details.
+- (*2*): When using HTTPS protocol for port 443, you will need to add an SSL
+ certificate to the load balancers. If you wish to terminate SSL at the
+ GitLab application server instead, use TCP protocol.
If you're using GitLab Pages with custom domain support you will need some
-additional port configurations. GitLab Pages requires a separate virtual IP
-address. Configure DNS to point the `pages_external_url` from
-`/etc/gitlab/gitlab.rb` to the new virtual IP address. For more information,
-see the [GitLab Pages documentation](../pages/index.md).
+additional port configurations.
+GitLab Pages requires a separate virtual IP address. Configure DNS to point the
+`pages_external_url` from `/etc/gitlab/gitlab.rb` at the new virtual IP address. See the
+[GitLab Pages documentation](../pages/index.md) for more information.
-| Port | Backend Port | Protocol |
+| LB Port | Backend Port | Protocol |
| ------- | ------------- | --------- |
| 80 | Varies (*1*) | HTTP |
| 443 | Varies (*1*) | TCP (*2*) |
- (*1*): The backend port for GitLab Pages depends on the
`gitlab_pages['external_http']` and `gitlab_pages['external_https']`
- settings. For details, see the [GitLab Pages documentation](../pages/index.md).
-- (*2*): Port 443 for GitLab Pages must use the TCP protocol. Users can
- configure custom domains with custom SSL, which wouldn't be possible if SSL
- was terminated at the load balancer.
+ setting. See [GitLab Pages documentation](../pages/index.md) for more details.
+- (*2*): Port 443 for GitLab Pages should always use the TCP protocol. Users can
+ configure custom domains with custom SSL, which would not be possible
+ if SSL was terminated at the load balancer.
#### Alternate SSH Port
Some organizations have policies against opening SSH port 22. In this case,
-it may be helpful to configure an alternate SSH hostname that instead allows
-users to use SSH over port 443. An alternate SSH hostname requires a new
-virtual IP address compared to the previously described GitLab HTTP
-configuration.
+it may be helpful to configure an alternate SSH hostname that allows users
+to use SSH on port 443. An alternate SSH hostname will require a new virtual IP address
+compared to the other GitLab HTTP configuration above.
-Configure DNS for an alternate SSH hostname, such as `altssh.gitlab.example.com`:
+Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`.
| LB Port | Backend Port | Protocol |
| ------- | ------------ | -------- |
| 443 | 22 | TCP |
+### SSL
+
+The next question is how you will handle SSL in your environment.
+There are several different options:
+
+- [The application node terminates SSL](#application-node-terminates-ssl).
+- [The load balancer terminates SSL without backend SSL](#load-balancer-terminates-ssl-without-backend-ssl)
+ and communication is not secure between the load balancer and the application node.
+- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
+ and communication is *secure* between the load balancer and the application node.
+
+#### Application node terminates SSL
+
+Configure your load balancer to pass connections on port 443 as `TCP` rather
+than `HTTP(S)` protocol. This will pass the connection to the application node's
+NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
+
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
+#### Load balancer terminates SSL without backend SSL
+
+Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
+The load balancer will then be responsible for managing SSL certificates and
+terminating SSL.
+
+Since communication between the load balancer and GitLab will not be secure,
+there is some additional configuration needed. See the
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
+for details.
+
+#### Load balancer terminates SSL with backend SSL
+
+Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
+The load balancers will be responsible for managing SSL certificates that
+end users will see.
+
+Traffic will also be secure between the load balancers and NGINX in this
+scenario. There is no need to add configuration for proxied SSL since the
+connection will be secure all the way. However, configuration will need to be
+added to GitLab to configure SSL certificates. See
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
@@ -407,9 +462,15 @@ are supported and can be added if needed.
specifically the number of projects and those projects' sizes.
NOTE:
-The Reference Architecture specs have been designed with good headroom in mind
-but for Gitaly, increased specs or switching to Gitaly Cluster
-may be required for notably large data sets or load.
+Increased specs for Gitaly nodes may be required in some circumstances such as
+significantly large repositories or if any [additional workloads](#additional-workloads),
+such as [server hooks](../server_hooks.md), have been added.
+
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
Due to Gitaly having notable input and output requirements, we strongly
recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs
@@ -878,7 +939,7 @@ GitLab has been tested on a number of object storage providers:
- [Amazon S3](https://aws.amazon.com/s3/)
- [Google Cloud Storage](https://cloud.google.com/storage)
- [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces)
-- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
+- [Oracle Cloud Infrastructure](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- [Azure Blob storage](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction)
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
@@ -962,7 +1023,9 @@ compute deployments. With this, _stateless_ components can benefit from cloud na
workload management benefits while _stateful_ components are deployed in compute VMs
with Omnibus to benefit from increased permanence.
-The 2,000 reference architecture is not a highly-available setup. To achieve HA, you can follow a modified [3K reference architecture](3k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative).
+Refer to the Helm charts [Advanced configuration](https://docs.gitlab.com/charts/advanced/)
+documentation for setup instructions including guidance on what GitLab secrets to sync
+between Kubernetes and the backend components.
NOTE:
This is an **advanced** setup. Running services in Kubernetes is well known
@@ -971,6 +1034,10 @@ knowledge and experience in Kubernetes. The rest of this
section assumes this.
NOTE:
+The 2,000 reference architecture is not a highly-available setup. To achieve HA,
+you can follow a modified [3K reference architecture](3k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative).
+
+NOTE:
**Gitaly Cluster is not supported to be run in Kubernetes**.
Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more details.
@@ -1010,7 +1077,7 @@ services where applicable):
1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
- [Google Cloud SQL](https://cloud.google.com/sql/docs/postgres/high-availability#normal) and [Amazon RDS](https://aws.amazon.com/rds/) are known to work.
- [Google AlloyDB](https://cloud.google.com/alloydb) and [Amazon RDS Multi-AZ DB clusters](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) have not been tested and are not recommended. Both solutions are specifically not expected to work with GitLab Geo.
- - [Amazon Aurora](https://aws.amazon.com/rds/aurora/) is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and [Azure Database for PostgreSQL](https://azure.microsoft.com/en-gb/services/postgresql/) is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61).
+ - [Amazon Aurora](https://aws.amazon.com/rds/aurora/) is **incompatible** with load balancing enabled by default in [14.4.0](../../update/index.md#1440), and [Azure Database for PostgreSQL](https://azure.microsoft.com/en-gb/products/postgresql/#overview) is **not recommended** due to [performance issues](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61).
- Consul is primarily used for Omnibus PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However, Consul is also used optionally by Prometheus for Omnibus auto host discovery.
2. Can be optionally run on reputable third-party external PaaS Redis solutions. See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
- [Google Memorystore](https://cloud.google.com/memorystore) and [Amazon ElastiCache](https://aws.amazon.com/elasticache/) are known to work.
diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md
index 7484fafe1b0..4fc6af3f72e 100644
--- a/doc/administration/reference_architectures/3k_users.md
+++ b/doc/administration/reference_architectures/3k_users.md
@@ -37,7 +37,7 @@ For a full list of reference architectures, see
| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
@@ -59,6 +59,7 @@ For a full list of reference architectures, see
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
@@ -164,10 +165,45 @@ Any "burstable" instance types are not recommended due to inconsistent performan
### Supported infrastructure
-As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services, or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section. However, this does not constitute a guarantee for every potential permutation.
+As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services,
+or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section.
+However, this does not constitute a guarantee for every potential permutation.
See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
+### Additional workloads
+
+The Reference Architectures have been [designed and tested](index.md#validation-and-test-results) for standard GitLab setups with
+good headroom in mind to cover most scenarios. However, if any additional workloads are being added on the nodes,
+such as security software, you may still need to adjust the specs accordingly to compensate.
+
+This also applies for some GitLab features where it's possible to run custom scripts, for example [server hooks](../server_hooks.md).
+
+As a general rule, it's recommended to have robust monitoring in place to measure the impact of
+any additional workloads to inform any changes needed to be made.
+
+### Large repositories
+
+The Reference Architectures were tested with repositories of varying sizes that follow best practices.
+
+However, large repositories or monorepos (several gigabytes or more) can **significantly** impact the performance
+of Git and in turn the environment itself if best practices aren't being followed such as not storing
+binary or blob files in LFS. Repositories are at the core of any environment the consequences can be wide-ranging
+when they are not optimized. Some examples of this impact include [Git packing operations](https://git-scm.com/book/en/v2/Git-Internals-Packfiles)
+taking longer and consuming high CPU / Memory resources or Git checkouts taking longer that affect both users and
+CI pipelines alike.
+
+As such, large repositories come with notable cost and typically will require more resources to handle,
+significantly so in some cases. It's therefore **strongly** recommended then to review large repositories
+to ensure they maintain good repo health and reduce their size wherever possible.
+
+NOTE:
+If best practices aren't followed and large repositories are present on the environment,
+increased Gitaly specs may be required to ensure stable performance.
+
+Refer to the [Managing large repositories documentation](../../user/project/repository/managing_large_repositories.md)
+for more information and guidance.
+
### Praefect PostgreSQL
It's worth noting that at this time [Praefect requires its own database server](../gitaly/praefect.md#postgresql) and
@@ -244,8 +280,7 @@ In a multi-node GitLab configuration, you'll need a load balancer to route
traffic to the application servers. The specifics on which load balancer to use
or its exact configuration is beyond the scope of GitLab documentation. We assume
that if you're managing multi-node systems like GitLab, you already have a load
-balancer of choice and that the routing methods used are distributing calls evenly
-between all nodes. Some load balancer examples include HAProxy (open-source),
+balancer of choice. Some load balancer examples include HAProxy (open-source),
F5 Big-IP LTM, and Citrix Net Scaler. This documentation outline the ports and
protocols needed for use with GitLab.
@@ -262,38 +297,13 @@ There are several different options:
- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
and communication is *secure* between the load balancer and the application node.
-### Application node terminates SSL
+### Balancing algorithm
-Configure your load balancer to pass connections on port 443 as `TCP` rather
-than `HTTP(S)` protocol. This will pass the connection to the application node's
-NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
-
-See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
-
-### Load balancer terminates SSL without backend SSL
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
-Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
-The load balancer will then be responsible for managing SSL certificates and
-terminating SSL.
-
-Since communication between the load balancer and GitLab will not be secure,
-there is some additional configuration needed. See the
-[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
-for details.
-
-### Load balancer terminates SSL with backend SSL
-
-Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
-The load balancers will be responsible for managing SSL certificates that
-end users will see.
-
-Traffic will also be secure between the load balancers and NGINX in this
-scenario. There is no need to add configuration for proxied SSL since the
-connection will be secure all the way. However, configuration will need to be
-added to GitLab to configure SSL certificates. See the
-[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
### Readiness checks
@@ -354,6 +364,50 @@ Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`.
| ------- | ------------ | -------- |
| 443 | 22 | TCP |
+### SSL
+
+The next question is how you will handle SSL in your environment.
+There are several different options:
+
+- [The application node terminates SSL](#application-node-terminates-ssl).
+- [The load balancer terminates SSL without backend SSL](#load-balancer-terminates-ssl-without-backend-ssl)
+ and communication is not secure between the load balancer and the application node.
+- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
+ and communication is *secure* between the load balancer and the application node.
+
+#### Application node terminates SSL
+
+Configure your load balancer to pass connections on port 443 as `TCP` rather
+than `HTTP(S)` protocol. This will pass the connection to the application node's
+NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
+
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
+#### Load balancer terminates SSL without backend SSL
+
+Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
+The load balancer will then be responsible for managing SSL certificates and
+terminating SSL.
+
+Since communication between the load balancer and GitLab will not be secure,
+there is some additional configuration needed. See the
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
+for details.
+
+#### Load balancer terminates SSL with backend SSL
+
+Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
+The load balancers will be responsible for managing SSL certificates that
+end users will see.
+
+Traffic will also be secure between the load balancers and NGINX in this
+scenario. There is no need to add configuration for proxied SSL since the
+connection will be secure all the way. However, configuration will need to be
+added to GitLab to configure SSL certificates. See
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
@@ -418,8 +472,14 @@ backend praefect
```
Refer to your preferred Load Balancer's documentation for further guidance.
-Also ensure that the routing methods used are distributing calls evenly across
-all nodes.
+
+### Balancing algorithm
+
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
+
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
@@ -1114,6 +1174,12 @@ NOTE:
Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster).
For implementations with sharded Gitaly, use the same Gitaly specs. Follow the [separate Gitaly documentation](../gitaly/configure_gitaly.md) instead of this section.
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
+
The recommended cluster setup includes the following components:
- 3 Gitaly nodes: Replicated storage of Git repositories.
@@ -1418,9 +1484,15 @@ The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have
requirements that are dependent on data and load.
NOTE:
-The Reference Architecture specs have been designed with good headroom in mind
-but for Gitaly, increased specs or additional
-Gitaly Cluster arrays may be required for notably large data sets or load.
+Increased specs for Gitaly nodes may be required in some circumstances such as
+significantly large repositories or if any [additional workloads](#additional-workloads),
+such as [server hooks](../server_hooks.md), have been added.
+
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
Due to Gitaly having notable input and output requirements, we strongly
recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs
@@ -2112,7 +2184,7 @@ GitLab has been tested on a number of object storage providers:
- [Amazon S3](https://aws.amazon.com/s3/)
- [Google Cloud Storage](https://cloud.google.com/storage)
- [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces)
-- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
+- [Oracle Cloud Infrastructure](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
@@ -2197,7 +2269,6 @@ but with smaller performance requirements, several modifications can be consider
- PostgreSQL and PgBouncer: PgBouncer nodes could be removed and instead be enabled on PostgreSQL nodes with the Internal Load Balancer pointing to them. However, to enable [Database Load Balancing](../postgresql/database_load_balancing.md), a separate PgBouncer array is still required.
- Reducing the node counts: Some node types do not need consensus and can run with fewer nodes (but more than one for redundancy). This will also lead to reduced performance.
- GitLab Rails and Sidekiq: Stateless services don't have a minimum node count. Two are enough for redundancy.
- - Gitaly and Praefect: A quorum is not strictly necessary. Two Gitaly nodes and two Praefect nodes are enough for redundancy.
- PostgreSQL and PgBouncer: A quorum is not strictly necessary. Two PostgreSQL nodes and two PgBouncer nodes are enough for redundancy.
- Running select components in reputable Cloud PaaS solutions: Select components of the GitLab setup can instead be run on Cloud Provider PaaS solutions. By doing this, additional dependent components can also be removed:
- PostgreSQL: Can be run on reputable Cloud PaaS solutions such as Google Cloud SQL or Amazon RDS. In this setup, the PgBouncer and Consul nodes are no longer required:
@@ -2219,6 +2290,10 @@ compute deployments. With this, _stateless_ components can benefit from cloud na
workload management benefits while _stateful_ components are deployed in compute VMs
with Omnibus to benefit from increased permanence.
+Refer to the Helm charts [Advanced configuration](https://docs.gitlab.com/charts/advanced/)
+documentation for setup instructions including guidance on what GitLab secrets to sync
+between Kubernetes and the backend components.
+
NOTE:
This is an **advanced** setup. Running services in Kubernetes is well known
to be complex. **This setup is only recommended** if you have strong working
@@ -2260,7 +2335,7 @@ services where applicable):
| PostgreSQL<sup>1</sup> | 3 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Gitaly<sup>5</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Object storage<sup>4</sup> | - | - | - | - |
@@ -2278,6 +2353,7 @@ services where applicable):
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md
index 88fc3649b3f..ca159d62f1f 100644
--- a/doc/administration/reference_architectures/50k_users.md
+++ b/doc/administration/reference_architectures/50k_users.md
@@ -28,7 +28,7 @@ full list of reference architectures, see
| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` |
| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` |
| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
@@ -50,6 +50,7 @@ full list of reference architectures, see
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
@@ -158,10 +159,45 @@ Any "burstable" instance types are not recommended due to inconsistent performan
### Supported infrastructure
-As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services, or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section. However, this does not constitute a guarantee for every potential permutation.
+As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP, Azure) and their services,
+or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section.
+However, this does not constitute a guarantee for every potential permutation.
See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
+### Additional workloads
+
+The Reference Architectures have been [designed and tested](index.md#validation-and-test-results) for standard GitLab setups with
+good headroom in mind to cover most scenarios. However, if any additional workloads are being added on the nodes,
+such as security software, you may still need to adjust the specs accordingly to compensate.
+
+This also applies for some GitLab features where it's possible to run custom scripts, for example [server hooks](../server_hooks.md).
+
+As a general rule, it's recommended to have robust monitoring in place to measure the impact of
+any additional workloads to inform any changes needed to be made.
+
+### Large repositories
+
+The Reference Architectures were tested with repositories of varying sizes that follow best practices.
+
+However, large repositories or monorepos (Several gigabytes or more) can **significantly** impact the performance
+of Git and in turn the environment itself if best practices aren't being followed such as not storing
+binary or blob files in LFS. Repositories are at the core of any environment the consequences can be wide-ranging
+when they are not optimized. Some examples of this impact include [Git packing operations](https://git-scm.com/book/en/v2/Git-Internals-Packfiles)
+taking longer and consuming high CPU / Memory resources or Git checkouts taking longer that affect both users and
+CI pipelines alike.
+
+As such, large repositories come with notable cost and typically will require more resources to handle,
+significantly so in some cases. It's therefore **strongly** recommended then to review large repositories
+to ensure they maintain good repo health and reduce their size wherever possible.
+
+NOTE:
+If best practices aren't followed and large repositories are present on the environment,
+increased Gitaly specs may be required to ensure stable performance.
+
+Refer to the [Managing large repositories documentation](../../user/project/repository/managing_large_repositories.md)
+for more information and guidance.
+
### Praefect PostgreSQL
It's worth noting that at this time [Praefect requires its own database server](../gitaly/praefect.md#postgresql) and
@@ -250,8 +286,7 @@ In a multi-node GitLab configuration, you'll need a load balancer to route
traffic to the application servers. The specifics on which load balancer to use
or its exact configuration is beyond the scope of GitLab documentation. We assume
that if you're managing multi-node systems like GitLab, you already have a load
-balancer of choice and that the routing methods used are distributing calls evenly
-between all nodes. Some load balancer examples include HAProxy (open-source),
+balancer of choice. Some load balancer examples include HAProxy (open-source),
F5 Big-IP LTM, and Citrix Net Scaler. This documentation outline the ports and
protocols needed for use with GitLab.
@@ -259,47 +294,13 @@ This architecture has been tested and validated with [HAProxy](https://www.hapro
as the load balancer. Although other load balancers with similar feature sets
could also be used, those load balancers have not been validated.
-The next question is how you will handle SSL in your environment.
-There are several different options:
-
-- [The application node terminates SSL](#application-node-terminates-ssl).
-- [The load balancer terminates SSL without backend SSL](#load-balancer-terminates-ssl-without-backend-ssl)
- and communication is not secure between the load balancer and the application node.
-- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
- and communication is *secure* between the load balancer and the application node.
-
-### Application node terminates SSL
-
-Configure your load balancer to pass connections on port 443 as `TCP` rather
-than `HTTP(S)` protocol. This will pass the connection to the application node's
-NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
-
-See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
-
-### Load balancer terminates SSL without backend SSL
+### Balancing algorithm
-Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
-The load balancer will then be responsible for managing SSL certificates and
-terminating SSL.
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
-Since communication between the load balancer and GitLab will not be secure,
-there is some additional configuration needed. See the
-[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
-for details.
-
-### Load balancer terminates SSL with backend SSL
-
-Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
-The load balancers will be responsible for managing SSL certificates that
-end users will see.
-
-Traffic will also be secure between the load balancers and NGINX in this
-scenario. There is no need to add configuration for proxied SSL since the
-connection will be secure all the way. However, configuration will need to be
-added to GitLab to configure SSL certificates. See the
-[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
### Readiness checks
@@ -360,6 +361,50 @@ Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`.
| ------- | ------------ | -------- |
| 443 | 22 | TCP |
+### SSL
+
+The next question is how you will handle SSL in your environment.
+There are several different options:
+
+- [The application node terminates SSL](#application-node-terminates-ssl).
+- [The load balancer terminates SSL without backend SSL](#load-balancer-terminates-ssl-without-backend-ssl)
+ and communication is not secure between the load balancer and the application node.
+- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
+ and communication is *secure* between the load balancer and the application node.
+
+#### Application node terminates SSL
+
+Configure your load balancer to pass connections on port 443 as `TCP` rather
+than `HTTP(S)` protocol. This will pass the connection to the application node's
+NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
+
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
+#### Load balancer terminates SSL without backend SSL
+
+Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
+The load balancer will then be responsible for managing SSL certificates and
+terminating SSL.
+
+Since communication between the load balancer and GitLab will not be secure,
+there is some additional configuration needed. See the
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
+for details.
+
+#### Load balancer terminates SSL with backend SSL
+
+Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
+The load balancers will be responsible for managing SSL certificates that
+end users will see.
+
+Traffic will also be secure between the load balancers and NGINX in this
+scenario. There is no need to add configuration for proxied SSL since the
+connection will be secure all the way. However, configuration will need to be
+added to GitLab to configure SSL certificates. See
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
@@ -424,8 +469,14 @@ backend praefect
```
Refer to your preferred Load Balancer's documentation for further guidance.
-Also ensure that the routing methods used are distributing calls evenly across
-all nodes.
+
+### Balancing algorithm
+
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
+
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
@@ -1181,6 +1232,12 @@ NOTE:
Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster).
For implementations with sharded Gitaly, use the same Gitaly specs. Follow the [separate Gitaly documentation](../gitaly/configure_gitaly.md) instead of this section.
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
+
The recommended cluster setup includes the following components:
- 3 Gitaly nodes: Replicated storage of Git repositories.
@@ -1488,9 +1545,15 @@ The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have
requirements that are dependent on data and load.
NOTE:
-The Reference Architecture specs have been designed with good headroom in mind
-but for Gitaly, increased specs or additional
-Gitaly Cluster arrays may be required for notably large data sets or load.
+Increased specs for Gitaly nodes may be required in some circumstances such as
+significantly large repositories or if any [additional workloads](#additional-workloads),
+such as [server hooks](../server_hooks.md), have been added.
+
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
Due to Gitaly having notable input and output requirements, we strongly
recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs
@@ -2176,7 +2239,7 @@ GitLab has been tested on a number of object storage providers:
- [Amazon S3](https://aws.amazon.com/s3/)
- [Google Cloud Storage](https://cloud.google.com/storage)
- [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces)
-- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
+- [Oracle Cloud Infrastructure](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
@@ -2258,6 +2321,10 @@ compute deployments. With this, _stateless_ components can benefit from cloud na
workload management benefits while _stateful_ components are deployed in compute VMs
with Omnibus to benefit from increased permanence.
+Refer to the Helm charts [Advanced configuration](https://docs.gitlab.com/charts/advanced/)
+documentation for setup instructions including guidance on what GitLab secrets to sync
+between Kubernetes and the backend components.
+
NOTE:
This is an **advanced** setup. Running services in Kubernetes is well known
to be complex. **This setup is only recommended** if you have strong working
@@ -2300,7 +2367,7 @@ services where applicable):
| Internal load balancing node<sup>3</sup> | 1 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` |
| Redis/Sentinel - Cache<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| Redis/Sentinel - Persistent<sup>2</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
-| Gitaly<sup>5</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 64 vCPU, 240 GB memory | `n1-standard-64` | `m5.16xlarge` |
| Praefect<sup>5</sup> | 3 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Object storage<sup>4</sup> | - | - | - | - |
@@ -2318,6 +2385,7 @@ services where applicable):
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md
index c8cf35a2e59..a2b92f9c300 100644
--- a/doc/administration/reference_architectures/5k_users.md
+++ b/doc/administration/reference_architectures/5k_users.md
@@ -34,7 +34,7 @@ costly-to-operate environment by using the
| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` |
| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Sidekiq | 4 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | `m5.large` |
@@ -56,6 +56,7 @@ costly-to-operate environment by using the
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
@@ -161,10 +162,45 @@ Any "burstable" instance types are not recommended due to inconsistent performan
### Supported infrastructure
-As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services, or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section. However, this does not constitute a guarantee for every potential permutation.
+As a general guidance, GitLab should run on most infrastructure such as reputable Cloud Providers (AWS, GCP) and their services,
+or self managed (ESXi) that meet both the specs detailed above, as well as any requirements in this section.
+However, this does not constitute a guarantee for every potential permutation.
See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information.
+### Additional workloads
+
+The Reference Architectures have been [designed and tested](index.md#validation-and-test-results) for standard GitLab setups with
+good headroom in mind to cover most scenarios. However, if any additional workloads are being added on the nodes,
+such as security software, you may still need to adjust the specs accordingly to compensate.
+
+This also applies for some GitLab features where it's possible to run custom scripts, for example [server hooks](../server_hooks.md).
+
+As a general rule, it's recommended to have robust monitoring in place to measure the impact of
+any additional workloads to inform any changes needed to be made.
+
+### Large repositories
+
+The Reference Architectures were tested with repositories of varying sizes that follow best practices.
+
+However, large repositories or monorepos (Several gigabytes or more) can **significantly** impact the performance
+of Git and in turn the environment itself if best practices aren't being followed such as not storing
+binary or blob files in LFS. Repositories are at the core of any environment the consequences can be wide-ranging
+when they are not optimized. Some examples of this impact include [Git packing operations](https://git-scm.com/book/en/v2/Git-Internals-Packfiles)
+taking longer and consuming high CPU / Memory resources or Git checkouts taking longer that affect both users and
+CI pipelines alike.
+
+As such, large repositories come with notable cost and typically will require more resources to handle,
+significantly so in some cases. It's therefore **strongly** recommended then to review large repositories
+to ensure they maintain good repo health and reduce their size wherever possible.
+
+NOTE:
+If best practices aren't followed and large repositories are present on the environment,
+increased Gitaly specs may be required to ensure stable performance.
+
+Refer the [Managing large repositories documentation](../../user/project/repository/managing_large_repositories.md)
+for more information and guidance.
+
### Praefect PostgreSQL
It's worth noting that at this time [Praefect requires its own database server](../gitaly/praefect.md#postgresql) and
@@ -237,12 +273,11 @@ The following list includes descriptions of each server and its assigned IP:
## Configure the external load balancer
-In a multi-node GitLab configuration, you need a load balancer to route
+In a multi-node GitLab configuration, you'll need a load balancer to route
traffic to the application servers. The specifics on which load balancer to use
or its exact configuration is beyond the scope of GitLab documentation. We assume
that if you're managing multi-node systems like GitLab, you already have a load
-balancer of choice and that the routing methods used are distributing calls evenly
-between all nodes. Some load balancer examples include HAProxy (open-source),
+balancer of choice. Some load balancer examples include HAProxy (open-source),
F5 Big-IP LTM, and Citrix Net Scaler. This documentation outline the ports and
protocols needed for use with GitLab.
@@ -259,45 +294,20 @@ There are several different options:
- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
and communication is *secure* between the load balancer and the application node.
-### Application node terminates SSL
-
-Configure your load balancer to pass connections on port 443 as `TCP` rather
-than `HTTP(S)` protocol. This passes the connection to the application node's
-NGINX service untouched. NGINX has the SSL certificate and listen on port 443.
+### Balancing algorithm
-See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
-### Load balancer terminates SSL without backend SSL
-
-Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
-The load balancer is then responsible for managing SSL certificates and
-terminating SSL.
-
-Since communication between the load balancer and GitLab is not secure,
-there is some additional configuration needed. See the
-[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
-for details.
-
-### Load balancer terminates SSL with backend SSL
-
-Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
-The load balancers are responsible for managing SSL certificates that
-end users see.
-
-Traffic is also secure between the load balancers and NGINX in this
-scenario. There is no need to add configuration for proxied SSL since the
-connection is secure all the way. However, configuration needs to be
-added to GitLab to configure SSL certificates. See the
-[HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
-for details on managing SSL certificates and configuring NGINX.
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
### Readiness checks
Ensure the external load balancer only routes to working services with built
in monitoring endpoints. The [readiness checks](../../user/admin_area/monitoring/health_check.md)
all require [additional configuration](../monitoring/ip_allowlist.md)
-on the nodes being checked, otherwise, the external load balancer is not able to
+on the nodes being checked, otherwise, the external load balancer will not be able to
connect.
### Ports
@@ -316,11 +326,11 @@ The basic ports to be used are shown in the table below.
to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the
[web terminal](../integration/terminal.md) integration guide for
more details.
-- (*2*): When using HTTPS protocol for port 443, you need to add an SSL
+- (*2*): When using HTTPS protocol for port 443, you will need to add an SSL
certificate to the load balancers. If you wish to terminate SSL at the
GitLab application server instead, use TCP protocol.
-If you're using GitLab Pages with custom domain support you need some
+If you're using GitLab Pages with custom domain support you will need some
additional port configurations.
GitLab Pages requires a separate virtual IP address. Configure DNS to point the
`pages_external_url` from `/etc/gitlab/gitlab.rb` at the new virtual IP address. See the
@@ -342,7 +352,7 @@ GitLab Pages requires a separate virtual IP address. Configure DNS to point the
Some organizations have policies against opening SSH port 22. In this case,
it may be helpful to configure an alternate SSH hostname that allows users
-to use SSH on port 443. An alternate SSH hostname requires a new virtual IP address
+to use SSH on port 443. An alternate SSH hostname will require a new virtual IP address
compared to the other GitLab HTTP configuration above.
Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`.
@@ -351,6 +361,50 @@ Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`.
| ------- | ------------ | -------- |
| 443 | 22 | TCP |
+### SSL
+
+The next question is how you will handle SSL in your environment.
+There are several different options:
+
+- [The application node terminates SSL](#application-node-terminates-ssl).
+- [The load balancer terminates SSL without backend SSL](#load-balancer-terminates-ssl-without-backend-ssl)
+ and communication is not secure between the load balancer and the application node.
+- [The load balancer terminates SSL with backend SSL](#load-balancer-terminates-ssl-with-backend-ssl)
+ and communication is *secure* between the load balancer and the application node.
+
+#### Application node terminates SSL
+
+Configure your load balancer to pass connections on port 443 as `TCP` rather
+than `HTTP(S)` protocol. This will pass the connection to the application node's
+NGINX service untouched. NGINX will have the SSL certificate and listen on port 443.
+
+See the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
+#### Load balancer terminates SSL without backend SSL
+
+Configure your load balancer to use the `HTTP(S)` protocol rather than `TCP`.
+The load balancer will then be responsible for managing SSL certificates and
+terminating SSL.
+
+Since communication between the load balancer and GitLab will not be secure,
+there is some additional configuration needed. See the
+[proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-a-reverse-proxy-or-load-balancer-ssl-termination)
+for details.
+
+#### Load balancer terminates SSL with backend SSL
+
+Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'.
+The load balancers will be responsible for managing SSL certificates that
+end users will see.
+
+Traffic will also be secure between the load balancers and NGINX in this
+scenario. There is no need to add configuration for proxied SSL since the
+connection will be secure all the way. However, configuration will need to be
+added to GitLab to configure SSL certificates. See
+the [HTTPS documentation](https://docs.gitlab.com/omnibus/settings/ssl.html)
+for details on managing SSL certificates and configuring NGINX.
+
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
Back to setup components <i class="fa fa-angle-double-up" aria-hidden="true"></i>
@@ -415,8 +469,14 @@ backend praefect
```
Refer to your preferred Load Balancer's documentation for further guidance.
-Also ensure that the routing methods used are distributing calls evenly across
-all nodes.
+
+### Balancing algorithm
+
+We recommend that a least-connection load balancing algorithm or equivalent
+is used wherever possible to ensure equal spread of calls to the nodes and good performance.
+
+We don't recommend the use of round-robin algorithms as they are known to not
+spread connections equally in practice.
<div align="right">
<a type="button" class="btn btn-default" href="#setup-components">
@@ -1110,6 +1170,12 @@ NOTE:
Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster).
For implementations with sharded Gitaly, use the same Gitaly specs. Follow the [separate Gitaly documentation](../gitaly/configure_gitaly.md) instead of this section.
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
+
The recommended cluster setup includes the following components:
- 3 Gitaly nodes: Replicated storage of Git repositories.
@@ -1415,9 +1481,15 @@ The [Gitaly](../gitaly/index.md) server nodes that make up the cluster have
requirements that are dependent on data and load.
NOTE:
-The Reference Architecture specs have been designed with good headroom in mind
-but for Gitaly, increased specs or additional
-Gitaly Cluster arrays may be required for notably large data sets or load.
+Increased specs for Gitaly nodes may be required in some circumstances such as
+significantly large repositories or if any [additional workloads](#additional-workloads),
+such as [server hooks](../server_hooks.md), have been added.
+
+NOTE:
+Gitaly has been designed and tested with repositories of varying sizes that follow best practices.
+However, large repositories or monorepos not following these practices can significantly
+impact Gitaly performance and requirements.
+Refer to the [Large Repositories](#large-repositories) for more info.
Due to Gitaly having notable input and output requirements, we strongly
recommend that all Gitaly nodes use solid-state drives (SSDs). These SSDs
@@ -2111,7 +2183,7 @@ GitLab has been tested on a number of object storage providers:
- [Amazon S3](https://aws.amazon.com/s3/)
- [Google Cloud Storage](https://cloud.google.com/storage)
- [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces)
-- [Oracle Cloud Infrastructure](https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
+- [Oracle Cloud Infrastructure](https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm)
- [OpenStack Swift (S3 compatibility mode)](https://docs.openstack.org/swift/latest/s3_compat.html)
- MinIO. We have [a guide to deploying this](https://docs.gitlab.com/charts/advanced/external-object-storage/minio.html) within our Helm Chart documentation.
@@ -2193,6 +2265,10 @@ compute deployments. With this, _stateless_ components can benefit from cloud na
workload management benefits while _stateful_ components are deployed in compute VMs
with Omnibus to benefit from increased permanence.
+Refer to the Helm charts [Advanced configuration](https://docs.gitlab.com/charts/advanced/)
+documentation for setup instructions including guidance on what GitLab secrets to sync
+between Kubernetes and the backend components.
+
NOTE:
This is an **advanced** setup. Running services in Kubernetes is well known
to be complex. **This setup is only recommended** if you have strong working
@@ -2234,7 +2310,7 @@ services where applicable):
| PostgreSQL<sup>1</sup> | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | `m5.xlarge` |
| PgBouncer<sup>1</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Internal load balancing node<sup>3</sup> | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
-| Gitaly<sup>5</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` |
+| Gitaly<sup>5 6</sup> | 3 | 8 vCPU, 30 GB memory | `n1-standard-8` | `m5.2xlarge` |
| Praefect<sup>5</sup> | 3 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Praefect PostgreSQL<sup>1</sup> | 1+ | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` |
| Object storage<sup>4</sup> | - | - | - | - |
@@ -2252,6 +2328,7 @@ services where applicable):
- [Google Cloud Load Balancing](https://cloud.google.com/load-balancing) and [Amazon Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) are known to work.
4. Should be run on reputable Cloud Provider or Self Managed solutions. More information can be found in the [Configure the object storage](#configure-the-object-storage) section.
5. Gitaly Cluster provides the benefits of fault tolerance, but comes with additional complexity of setup and management. Review the existing [technical limitations and considerations before deploying Gitaly Cluster](../gitaly/index.md#before-deploying-gitaly-cluster). If you want sharded Gitaly, use the same specs listed above for `Gitaly`.
+6. Gitaly has been designed and tested with repositories of varying sizes that follow best practices. However, large repositories or monorepos that don't follow these practices can significantly impact Gitaly requirements. Refer to the [Large Repositories](#large-repositories) for more info.
<!-- markdownlint-enable MD029 -->
NOTE:
diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md
index 2cf9f2a1e83..467cc332e25 100644
--- a/doc/administration/reference_architectures/index.md
+++ b/doc/administration/reference_architectures/index.md
@@ -43,11 +43,6 @@ The following Cloud Native Hybrid reference architectures, where select recommen
- [Up to 25,000 users](25k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative)
- [Up to 50,000 users](50k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative)
-A GitLab [Premium or Ultimate](https://about.gitlab.com/pricing/#self-managed) license is required
-to get assistance from Support with troubleshooting the [2,000 users](2k_users.md)
-and higher reference architectures.
-[Read more about our definition of scaled architectures](https://about.gitlab.com/support/definitions/#definition-of-scaled-architecture).
-
## Deciding which architecture to use
The Reference Architectures are designed to strike a balance between two important factors--performance and resilience.
@@ -109,18 +104,18 @@ Below you can find the above guidance in the form of a decision tree. It's recom
```mermaid
%%{init: { 'theme': 'base' } }%%
-graph TD
+graph TD
L1A(<b>What Reference Architecture should I use?</b>) --> L2A(More than 3000 users?)
L2A -->|No| L3A("<a href=#do-you-need-high-availability-ha>Do you need HA?</a><br>(or Zero-Downtime Upgrades)") --> |Yes| L4A><b>Recommendation</b><br><br>3K architecture with HA<br>including supported modifications]
L3A -->|No| L4B><b>Recommendation</b><br><br>Architecture closest to user<br>count with Backups]
L2A -->|Yes| L3B[Do you have experience with<br/>and want additional resilience<br/>with select components in Kubernetes?]
L3B -->|No| L4C><b>Recommendation</b><br><br>Architecture closest to user<br>count with HA]
L3B -->|Yes| L4D><b>Recommendation</b><br><br>Cloud Native Hybrid architecture<br>closest to user count]
-
+
L5A("<a href=#gitlab-geo-cross-regional-distribution-disaster-recovery>Do you need cross regional distribution or disaster recovery?"</a>) --> |Yes| L6A><b>Additional Recommendation</b><br><br> GitLab Geo]
L4A -.- L5A
- L4B -.- L5A
- L4C -.- L5A
+ L4B -.- L5A
+ L4C -.- L5A
L4D -.- L5A
classDef default fill:#FCA326
@@ -211,7 +206,7 @@ When selecting a database service, it should run a standard, performant, and [su
Several cloud provider services are known not to support the above or have been found to have other issues and aren't recommended:
- [Amazon Aurora](https://aws.amazon.com/rds/aurora/) is incompatible and not supported. See [14.4.0](../../update/index.md#1440) for more details.
-- [Azure Database for PostgreSQL Single Server](https://azure.microsoft.com/en-gb/services/postgresql/#overview) (Single / Flexible) is **strongly not recommended** for use due to notable performance / stability issues or missing functionality. See [Recommendation Notes for Azure](#recommendation-notes-for-azure) for more details.
+- [Azure Database for PostgreSQL Single Server](https://azure.microsoft.com/en-gb/products/postgresql/#overview) (Single / Flexible) is **strongly not recommended** for use due to notable performance / stability issues or missing functionality. See [Recommendation Notes for Azure](#recommendation-notes-for-azure) for more details.
- [Google AlloyDB](https://cloud.google.com/alloydb) and [Amazon RDS Multi-AZ DB clusters](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html) have not been tested and are not recommended. Both solutions are specifically not expected to work with GitLab Geo.
### Recommendation notes for Azure
@@ -222,7 +217,7 @@ In addition to the above, you should be aware of the additional specific guidanc
- **We outright strongly do not recommend [Azure Database for PostgreSQL Single Server](https://learn.microsoft.com/en-us/azure/postgresql/single-server/overview-single-server)** specifically due to significant performance and stability issues found. **For GitLab 14.0 and higher the service is not supported** due to it only supporting up to PostgreSQL 11.
- A new service, [Azure Database for Postgres Flexible Server](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/) has been released but due to it missing some functionality we don't recommend it at this time.
-- [Azure Blob Storage](https://azure.microsoft.com/en-gb/services/storage/blobs/) has been found to have performance limits that can impact production use at certain times. However, this has only been seen in larger architectures.
+- [Azure Blob Storage](https://azure.microsoft.com/en-gb/products/storage/blobs/) has been found to have performance limits that can impact production use at certain times. However, this has only been seen in larger architectures.
## Validation and test results
diff --git a/doc/administration/reference_architectures/troubleshooting.md b/doc/administration/reference_architectures/troubleshooting.md
deleted file mode 100644
index a9bf035a528..00000000000
--- a/doc/administration/reference_architectures/troubleshooting.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../configure.md'
-remove_date: '2022-10-24'
----
-
-This document was moved to [another location](../configure.md).
-
-<!-- This redirect file can be deleted after <2022-10-24>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> \ No newline at end of file
diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md
index 492c5306b26..808659132f9 100644
--- a/doc/administration/repository_storage_paths.md
+++ b/doc/administration/repository_storage_paths.md
@@ -45,8 +45,8 @@ For more information on:
WARNING:
The following information is for configuring GitLab to directly access repositories. This
-configuration option is deprecated in favor of using [Gitaly](gitaly/index.md) and is scheduled to
-[be removed in GitLab 14.0](https://gitlab.com/gitlab-org/gitaly/-/issues/1690).
+configuration option is deprecated in favor of using [Gitaly](gitaly/index.md). Gitaly issue
+[1690](https://gitlab.com/gitlab-org/gitaly/-/issues/1690) proposes to remove this configuration option.
To configure repository storage paths:
diff --git a/doc/administration/server_hooks.md b/doc/administration/server_hooks.md
index 6ab4f476e5e..448becb32dc 100644
--- a/doc/administration/server_hooks.md
+++ b/doc/administration/server_hooks.md
@@ -5,16 +5,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w
disqus_identifier: 'https://docs.gitlab.com/ee/administration/custom_hooks.html'
---
-# Server hooks **(FREE SELF)**
+# Git server hooks **(FREE SELF)**
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196051) in GitLab 12.8 replacing Custom Hooks.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196051) in GitLab 12.8 replacing Custom Hooks.
+> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/372991) from server hooks to Git server hooks in GitLab 15.6.
-Server hooks run custom logic on the GitLab server. Users can use them to run Git-related tasks such as:
+Git server hooks (not to be confused with [system hooks](system_hooks.md) or [file hooks](file_hooks.md)) run custom logic
+on the GitLab server. You can use them to run Git-related tasks such as:
- Enforcing specific commit policies.
- Performing tasks based on the state of the repository.
-Server hooks use `pre-receive`, `post-receive`, and `update`
+Git server hooks use `pre-receive`, `post-receive`, and `update`
[Git server-side hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_server_side_hooks).
GitLab administrators configure server hooks on the file system of the GitLab server. If you don't have file system access,
@@ -46,7 +48,7 @@ To create server hooks for a repository:
- To create many server hooks, create a directory for the hooks that matches the hook type. For example, for a
`pre-receive` server hook, the directory name should be `pre-receive.d`. Put the files for the hook in that directory.
1. Make the server hook files executable and ensure that they are owned by the Git user.
-1. Write the code to make the server hook function as expected. Server hooks can be in any programming language. Ensure
+1. Write the code to make the server hook function as expected. Git server hooks can be in any programming language. Ensure
the [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) at the top reflects the language type. For
example, if the script is in Ruby the shebang is probably `#!/usr/bin/env ruby`.
1. Make the hook file executable, ensure that it's owned by the Git user, and ensure it does not match the backup file
@@ -87,7 +89,7 @@ To create a global server hook for all repositories:
1. On the GitLab server, go to the configured global server hook directory.
1. In the configured global server hook directory, create a directory for the hooks that matches the hook type. For example, for a `pre-receive` server hook, the directory name should be `pre-receive.d`.
-1. Inside this new directory, add your server hooks. Server hooks can be in any programming language. Ensure the
+1. Inside this new directory, add your server hooks. Git server hooks can be in any programming language. Ensure the
[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) at the top reflects the language type. For example, if the
script is in Ruby the shebang is probably `#!/usr/bin/env ruby`.
1. Make the hook file executable, ensure that it's owned by the Git user, and ensure it does not match the backup file
diff --git a/doc/administration/sidekiq/sidekiq_memory_killer.md b/doc/administration/sidekiq/sidekiq_memory_killer.md
index bbf95bc45ce..0876f98621d 100644
--- a/doc/administration/sidekiq/sidekiq_memory_killer.md
+++ b/doc/administration/sidekiq/sidekiq_memory_killer.md
@@ -64,5 +64,5 @@ The MemoryKiller is controlled using environment variables.
If the process hard shutdown/restart is not performed by Sidekiq,
the Sidekiq process is forcefully terminated after
- `Sidekiq.options[:timeout] + 2` seconds. An external supervision mechanism
+ `Sidekiq[:timeout] + 2` seconds. An external supervision mechanism
(for example, runit) must restart Sidekiq afterwards.
diff --git a/doc/administration/snippets/index.md b/doc/administration/snippets/index.md
index 89a571946af..7bf828afedd 100644
--- a/doc/administration/snippets/index.md
+++ b/doc/administration/snippets/index.md
@@ -26,7 +26,7 @@ content changes.
### Snippets size limit configuration
This setting is not available through the [Admin Area settings](../../user/admin_area/settings/index.md).
-In order to configure this setting, use either the Rails console
+To configure this setting, use either the Rails console
or the [Application settings API](../../api/settings.md).
NOTE:
diff --git a/doc/administration/system_hooks.md b/doc/administration/system_hooks.md
index 56e73150616..038c26a9c2e 100644
--- a/doc/administration/system_hooks.md
+++ b/doc/administration/system_hooks.md
@@ -7,7 +7,8 @@ type: reference
# System hooks **(FREE SELF)**
-Your GitLab instance can perform HTTP POST requests on the following events:
+System hooks (not to be confused with [server hooks](server_hooks.md) or [file hooks](file_hooks.md)) perform HTTP POST
+requests and are triggered on the following events:
- `group_create`
- `group_destroy`
@@ -31,21 +32,18 @@ Your GitLab instance can perform HTTP POST requests on the following events:
- `user_update_for_group`
- `user_update_for_team`
-The triggers for most of these are self-explanatory, but `project_update` and
-`project_rename` deserve some clarification: `project_update` is fired any time
-an attribute of a project is changed (including name, description, and tags)
-_unless_ the `path` attribute is also changed. In that case, a `project_rename`
-is triggered instead (so that, for instance, if all you care about is the
-repository URL, you can just listen for `project_rename`).
+The triggers for most of these are self-explanatory, but `project_update` and `project_rename` require clarification:
-`user_failed_login` is sent whenever a _blocked_ user attempts to sign in and is
-denied access.
+- `project_update` triggers when an attribute of a project is changed (including name, description, and tags)
+ **except** when the `path` attribute is also changed.
+- `project_rename` triggers when an attribute of a project (including `path`) is changed. If you only care about the
+ repository URL, just listen for `project_rename`.
-System hooks can be used, for example, for logging or changing information in an
-LDAP server.
+`user_failed_login` is sent whenever a **blocked** user attempts to sign in and is denied access.
-In addition to these default events, you can enable triggers for other events,
-such as push events, and disable the `repository_update` event
+As an example, use system hooks for logging or changing information in an LDAP server.
+
+You can also enable triggers for other events, such as push events, and disable the `repository_update` event
when you create a system hook.
NOTE:
@@ -136,7 +134,7 @@ X-Gitlab-Event: System Hook
```
Note that `project_rename` is not triggered if the namespace changes.
-Please refer to `group_rename` and `user_rename` for that case.
+Refer to `group_rename` and `user_rename` for that case.
**Project transferred:**
@@ -766,6 +764,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/administration/terraform_state.md b/doc/administration/terraform_state.md
index 61fda91ba71..02a95e28747 100644
--- a/doc/administration/terraform_state.md
+++ b/doc/administration/terraform_state.md
@@ -135,10 +135,11 @@ For GitLab 13.8 and earlier versions, you can use a workaround for the Rake task
end
```
-You can optionally track progress and verify that all packages migrated successfully using the
+You can optionally track progress and verify that all Terraform state files migrated successfully using the
[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database):
-- `sudo gitlab-rails dbconsole` for Omnibus GitLab instances.
+- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier.
+- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later.
- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances.
Verify `objectstg` below (where `file_store=2`) has count of all states:
diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md
deleted file mode 100644
index d5019f1aa4a..00000000000
--- a/doc/administration/troubleshooting/debug.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../reference_architectures/troubleshooting.md'
-remove_date: '2022-10-19'
----
-
-This document was moved to [another location](../reference_architectures/troubleshooting.md).
-
-<!-- This redirect file can be deleted after 2022-10-19. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/administration/troubleshooting/defcon.md b/doc/administration/troubleshooting/defcon.md
deleted file mode 100644
index f2554f523f0..00000000000
--- a/doc/administration/troubleshooting/defcon.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../../ci/troubleshooting.md#disaster-recovery'
-remove_date: '2022-10-04'
----
-
-This document was moved to [another location](../../ci/troubleshooting.md#disaster-recovery).
-
-<!-- This redirect file can be deleted after <2022-10-04>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
index 20ce52a9094..6993d48b450 100644
--- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
+++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
@@ -1,660 +1,11 @@
---
-stage: Systems
-group: Distribution
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: 'index.md'
+remove_date: '2023-02-01'
---
-# GitLab Rails Console Cheat Sheet **(FREE SELF)**
+This document was moved to [another location](index.md).
-This is the GitLab Support Team's collection of information regarding the GitLab Rails
-console, for use while troubleshooting. It is listed here for transparency,
-and for users with experience with these tools. If you are currently
-having an issue with GitLab, it is highly recommended that you first check
-our guide on [our Rails console](../operations/rails_console.md),
-and your [support options](https://about.gitlab.com/support/), before attempting to use
-this information.
-
-WARNING:
-Some of these scripts could be damaging if not run correctly,
-or under the right conditions. We highly recommend running them under the
-guidance of a Support Engineer, or running them in a test environment with a
-backup of the instance ready to be restored, just in case.
-
-WARNING:
-As GitLab changes, changes to the code are inevitable,
-and so some scripts may not work as they once used to. These are not kept
-up-to-date as these scripts/commands were added as they were found/needed. As
-mentioned above, we recommend running these scripts under the supervision of a
-Support Engineer, who can also verify that they continue to work as they
-should and, if needed, update the script for the latest version of GitLab.
-
-## Attributes
-
-View available attributes, formatted using pretty print (`pp`).
-
-For example, determine what attributes contain users' names and email addresses:
-
-```ruby
-u = User.find_by_username('someuser')
-pp u.attributes
-```
-
-Partial output:
-
-```plaintext
-{"id"=>1234,
- "email"=>"someuser@example.com",
- "sign_in_count"=>99,
- "name"=>"S User",
- "username"=>"someuser",
- "first_name"=>nil,
- "last_name"=>nil,
- "bot_type"=>nil}
-```
-
-Then make use of the attributes, [testing SMTP, for example](https://docs.gitlab.com/omnibus/settings/smtp.html#testing-the-smtp-configuration):
-
-```ruby
-e = u.email
-n = u.name
-Notify.test_email(e, "Test email for #{n}", 'Test email').deliver_now
-#
-Notify.test_email(u.email, "Test email for #{u.name}", 'Test email').deliver_now
-```
-
-## Imports and exports
-
-### Import a project
-
-```ruby
-# Find the project and get the error
-p = Project.find_by_full_path('<username-or-group>/<project-name>')
-
-p.import_error
-
-# To finish the import on GitLab running version before 11.6
-p.import_finish
-
-# To finish the import on GitLab running version 11.6 or after
-p.import_state.mark_as_failed("Failed manually through console.")
-```
-
-### Rename imported repository
-
-In a specific situation, an imported repository needed to be renamed. The Support
-Team was informed of a backup restore that failed on a single repository, which created
-the project with an empty repository. The project was successfully restored to a development
-instance, then exported, and imported into a new project under a different name.
-
-The Support Team was able to transfer the incorrectly named imported project into the
-correctly named empty project using the steps below.
-
-Move the new repository to the empty repository:
-
-```shell
-mv /var/opt/gitlab/git-data/repositories/<group>/<new-project> /var/opt/gitlab/git-data/repositories/<group>/<empty-project>
-```
-
-Make sure the permissions are correct:
-
-```shell
-chown -R git:git <path-to-directory>.git
-```
-
-Clear the cache:
-
-```shell
-sudo gitlab-rake cache:clear
-```
-
-### Export a project
-
-It's typically recommended to export a project through [the web interface](../../user/project/settings/import_export.md#export-a-project-and-its-data) or through [the API](../../api/project_import_export.md). In situations where this is not working as expected, it may be preferable to export a project directly via the Rails console:
-
-```ruby
-user = User.find_by_username('<username>')
-# Sufficient permissions needed
-# Read https://docs.gitlab.com/ee/user/permissions.html#project-members-permissions
-
-project = Project.find_by_full_path('<username-or-group>/<project-name')
-Projects::ImportExport::ExportService.new(project, user).execute
-```
-
-If this all runs successfully, you see an output like the following before being returned to the Rails console prompt:
-
-```ruby
-=> nil
-```
-
-The exported project is located in a `.tar.gz` file in `/var/opt/gitlab/gitlab-rails/uploads/-/system/import_export_upload/export_file/`.
-
-If this fails, [enable verbose logging](../operations/rails_console.md#looking-up-database-persisted-objects),
-repeat the above procedure after,
-and report the output to
-[GitLab Support](https://about.gitlab.com/support/).
-
-## Mirrors
-
-### Find mirrors with "bad decrypt" errors
-
-This content has been converted to a Rake task, see [verify database values can be decrypted using the current secrets](../raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets).
-
-### Transfer mirror users and tokens to a single service account
-
-This content has been moved to [Troubleshooting Repository mirroring](../../user/project/repository/mirror/index.md#transfer-mirror-users-and-tokens-to-a-single-service-account-in-rails-console).
-
-## Users
-
-### Create new user
-
-```ruby
-u = User.new(username: 'test_user', email: 'test@example.com', name: 'Test User', password: 'password', password_confirmation: 'password')
-u.skip_confirmation! # Use it only if you wish user to be automatically confirmed. If skipped, user receives confirmation e-mail
-u.save!
-```
-
-### Skip reconfirmation
-
-```ruby
-user = User.find_by_username('<username>')
-user.skip_reconfirmation!
-```
-
-### Disable 2fa for single user
-
-**In GitLab 13.5 and later:**
-
-Use the code under [Disable 2FA | For a single user](../../security/two_factor_authentication.md#for-a-single-user) so that the target user
-is notified that 2FA has been disabled.
-
-**In GitLab 13.4 and earlier:**
-
-```ruby
-user = User.find_by_username('<username>')
-user.disable_two_factor!
-```
-
-### Active users & Historical users
-
-```ruby
-# Active users on the instance, now
-User.active.count
-
-# Users taking a seat on the instance
-User.billable.count
-
-# The historical max on the instance as of the past year
-::HistoricalData.max_historical_user_count(from: 1.year.ago.beginning_of_day, to: Time.current.end_of_day)
-```
-
-Using cURL and jq (up to a max 100, see [Pagination](../../api/index.md#pagination)):
-
-```shell
-curl --silent --header "Private-Token: ********************" \
- "https://gitlab.example.com/api/v4/users?per_page=100&active" | jq --compact-output '.[] | [.id,.name,.username]'
-```
-
-### Update Daily Billable & Historical users
-
-```ruby
-# Forces recount of historical (max) users
-::HistoricalDataWorker.new.perform
-
-# Forces recount of daily billable users
-identifier = Analytics::UsageTrends::Measurement.identifiers[:billable_users]
-::Analytics::UsageTrends::CounterJobWorker.new.perform(identifier, User.minimum(:id), User.maximum(:id), Time.zone.now)
-```
-
-### Block or Delete Users that have no projects or groups
-
-```ruby
-users = User.where('id NOT IN (select distinct(user_id) from project_authorizations)')
-
-# How many users are removed?
-users.count
-
-# If that count looks sane:
-
-# You can either block the users:
-users.each { |user| user.blocked? ? nil : user.block! }
-
-# Or you can delete them:
- # need 'current user' (your user) for auditing purposes
-current_user = User.find_by(username: '<your username>')
-
-users.each do |user|
- DeleteUserWorker.perform_async(current_user.id, user.id)
-end
-```
-
-### Deactivate Users that have no recent activity
-
-```ruby
-days_inactive = 90
-inactive_users = User.active.where("last_activity_on <= ?", days_inactive.days.ago)
-
-inactive_users.each do |user|
- puts "user '#{user.username}': #{user.last_activity_on}"
- user.deactivate!
-end
-```
-
-### Block Users that have no recent activity
-
-```ruby
-days_inactive = 90
-inactive_users = User.active.where("last_activity_on <= ?", days_inactive.days.ago)
-
-inactive_users.each do |user|
- puts "user '#{user.username}': #{user.last_activity_on}"
- user.block!
-end
-```
-
-### Find a user's max permissions for project/group
-
-```ruby
-user = User.find_by_username 'username'
-project = Project.find_by_full_path 'group/project'
-user.max_member_access_for_project project.id
-```
-
-```ruby
-user = User.find_by_username 'username'
-group = Group.find_by_full_path 'group'
-user.max_member_access_for_group group.id
-```
-
-## Merge requests
-
-### Close a merge request
-
-```ruby
-u = User.find_by_username('<username>')
-p = Project.find_by_full_path('<namespace/project>')
-m = p.merge_requests.find_by(iid: <iid>)
-MergeRequests::CloseService.new(project: p, current_user: u).execute(m)
-```
-
-### Delete a merge request
-
-```ruby
-u = User.find_by_username('<username>')
-p = Project.find_by_full_path('<namespace/project>')
-m = p.merge_requests.find_by(iid: <iid>)
-Issuable::DestroyService.new(project: m.project, current_user: u).execute(m)
-```
-
-### Rebase manually
-
-```ruby
-u = User.find_by_username('<username>')
-p = Project.find_by_full_path('<namespace/project>')
-m = p.merge_requests.find_by(iid: <iid>)
-MergeRequests::RebaseService.new(project: m.target_project, current_user: u).execute(m)
-```
-
-### Set a merge request as merged
-
-Use when a merge request was accepted and the changes merged into the Git repository,
-but the merge request still shows as open.
-
-If the changes are not merged yet, this action causes the merge request to
-incorrectly show `merged into <branch-name>`.
-
-```ruby
-u = User.find_by_username('<username>')
-p = Project.find_by_full_path('<namespace/project>')
-m = p.merge_requests.find_by(iid: <iid>)
-MergeRequests::PostMergeService.new(project: p, current_user: u).execute(m)
-```
-
-## CI
-
-### Cancel stuck pending pipelines
-
-For more information, see the [confidential issue](../../user/project/issues/confidential_issues.md)
-`https://gitlab.com/gitlab-com/support-forum/issues/2449#note_41929707`.
-
-```ruby
-Ci::Pipeline.where(project_id: p.id).where(status: 'pending').count
-Ci::Pipeline.where(project_id: p.id).where(status: 'pending').each {|p| p.cancel if p.stuck?}
-Ci::Pipeline.where(project_id: p.id).where(status: 'pending').count
-```
-
-### Remove artifacts more than a week old
-
-This section has been moved to the [job artifacts troubleshooting documentation](../job_artifacts.md#delete-job-artifacts-from-jobs-completed-before-a-specific-date).
-
-### Find reason failure (for when build trace is empty) (Introduced in 10.3.0)
-
-See <https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41111>.
-
-```ruby
-build = Ci::Build.find(78420)
-
-build.failure_reason
-
-build.dependencies.each do |d| { puts "status: #{d.status}, finished at: #{d.finished_at},
- completed: #{d.complete?}, artifacts_expired: #{d.artifacts_expired?}, erased: #{d.erased?}" }
-```
-
-### Try CI integration
-
-```ruby
-p = Project.find_by_full_path('<project_path>')
-m = project.merge_requests.find_by(iid: )
-m.project.try(:ci_integration)
-```
-
-### Validate the `.gitlab-ci.yml`
-
-```ruby
-project = Project.find_by_full_path 'group/project'
-content = project.repository.gitlab_ci_yml_for(project.repository.root_ref_sha)
-Gitlab::Ci::Lint.new(project: project, current_user: User.first).validate(content)
-```
-
-### Disable AutoDevOps on Existing Projects
-
-```ruby
-Project.all.each do |p|
- p.auto_devops_attributes={"enabled"=>"0"}
- p.save
-end
-```
-
-### Obtain runners registration token
-
-```ruby
-Gitlab::CurrentSettings.current_application_settings.runners_registration_token
-```
-
-### Seed runners registration token
-
-```ruby
-appSetting = Gitlab::CurrentSettings.current_application_settings
-appSetting.set_runners_registration_token('<new-runners-registration-token>')
-appSetting.save!
-```
-
-### Run pipeline schedules manually
-
-You can run pipeline schedules manually through the Rails console to reveal any errors that are usually not visible.
-
-```ruby
-# schedule_id can be obtained from Edit Pipeline Schedule page
-schedule = Ci::PipelineSchedule.find_by(id: <schedule_id>)
-
-# Select the user that you want to run the schedule for
-user = User.find_by_username('<username>')
-
-# Run the schedule
-ps = Ci::CreatePipelineService.new(schedule.project, user, ref: schedule.ref).execute!(:schedule, ignore_skip_ci: true, save_on_errors: false, schedule: schedule)
-```
-
-## License
-
-### See current license information
-
-```ruby
-# License information (name, company, email address)
-License.current.licensee
-
-# Plan:
-License.current.plan
-
-# Uploaded:
-License.current.created_at
-
-# Started:
-License.current.starts_at
-
-# Expires at:
-License.current.expires_at
-
-# Is this a trial license?
-License.current.trial?
-
-# License ID for lookup on CustomersDot
-License.current.license_id
-
-# License data in Base64-encoded ASCII format
-License.current.data
-```
-
-### Check if a project feature is available on the instance
-
-Features listed in <https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/license.rb>.
-
-```ruby
-License.current.feature_available?(:jira_dev_panel_integration)
-```
-
-### Check if a project feature is available in a project
-
-Features listed in [`license.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/license.rb).
-
-```ruby
-p = Project.find_by_full_path('<group>/<project>')
-p.feature_available?(:jira_dev_panel_integration)
-```
-
-### Add a license through the console
-
-```ruby
-key = "<key>"
-license = License.new(data: key)
-license.save
-License.current # check to make sure it applied
-```
-
-This is needed for example in a known edge-case with
-[expired license and multiple LDAP servers](../auth/ldap/ldap-troubleshooting.md#expired-license-causes-errors-with-multiple-ldap-servers).
-
-### Remove licenses
-
-To clean up the [License History table](../../user/admin_area/license_file.md#view-license-details-and-history):
-
-```ruby
-TYPE = :trial?
-# or :expired?
-
-License.select(&TYPE).each(&:destroy!)
-
-# or even License.all.each(&:destroy!)
-```
-
-## Registry
-
-### Registry Disk Space Usage by Project
-
-Find this content in the [Container Registry troubleshooting documentation](../packages/container_registry.md#registry-disk-space-usage-by-project).
-
-### Run the Cleanup policy now
-
-Find this content in the [Container Registry troubleshooting documentation](../packages/container_registry.md#run-the-cleanup-policy-now).
-
-## Sidekiq
-
-This content has been moved to [Troubleshooting Sidekiq](sidekiq.md).
-
-## LFS
-
-### Get information about LFS objects and associated project
-
-```ruby
-o = LfsObject.find_by(oid: "<oid>")
-p = Project.find(LfsObjectsProject.find_by_lfs_object_id(o.id).project_id)
-```
-
-You can then delete these records from the database with:
-
-```ruby
-LfsObjectsProject.find_by_lfs_object_id(o.id).destroy
-o.destroy
-```
-
-You would also want to combine this with deleting the LFS file in the LFS storage
-area on disk. It remains to be seen exactly how or whether the deletion is useful, however.
-
-## Decryption Problems
-
-### Bad Decrypt Script (for encrypted variables)
-
-This content has been converted to a Rake task, see [verify database values can be decrypted using the current secrets](../raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets).
-
-As an example of repairing, if `ProjectImportData Bad count:` is detected and the decision is made to delete the
-encrypted credentials to allow manual reentry:
-
-```ruby
- # Find the ids of the corrupt ProjectImportData objects
- total = 0
- bad = []
- ProjectImportData.find_each do |data|
- begin
- total += 1
- data.credentials
- rescue => e
- bad << data.id
- end
- end
-
- puts "Bad count: #{bad.count} / #{total}"
-
- # See the bad ProjectImportData ids
- bad
-
- # Remove the corrupted credentials
- import_data = ProjectImportData.where(id: bad)
- import_data.each do |data|
- data.update_columns({ encrypted_credentials: nil, encrypted_credentials_iv: nil, encrypted_credentials_salt: nil})
- end
-```
-
-If `User OTP Secret Bad count:` is detected. For each user listed disable/enable
-two-factor authentication.
-
-The following script searches in some of the tables for encrypted tokens that are
-causing decryption errors, and update or reset as needed:
-
-```shell
-wget -O /tmp/encrypted-tokens.rb https://gitlab.com/snippets/1876342/raw
-gitlab-rails runner /tmp/encrypted-tokens.rb
-```
-
-### Decrypt Script for encrypted tokens
-
-This content has been converted to a Rake task, see [verify database values can be decrypted using the current secrets](../raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets).
-
-## Geo
-
-### Reverify all uploads (or any SSF data type which is verified)
-
-1. SSH into a GitLab Rails node in the primary Geo site.
-1. Open [Rails console](../operations/rails_console.md).
-1. Mark all uploads as "pending verification":
-
- ```ruby
- Upload.verification_state_table_class.each_batch do |relation|
- relation.update_all(verification_state: 0)
- end
- ```
-
-1. This will cause the primary to start checksumming all Uploads.
-1. When a primary successfully checksums a record, then all secondaries rechecksum as well, and they compare the values.
-
-A similar thing can be done for all Models handled by the [Geo Self-Service Framework](../../development/geo/framework.md) which have implemented verification:
-
-- `LfsObject`
-- `MergeRequestDiff`
-- `Packages::PackageFile`
-- `Terraform::StateVersion`
-- `SnippetRepository`
-- `Ci::PipelineArtifact`
-- `PagesDeployment`
-- `Upload`
-- `Ci::JobArtifact`
-- `Ci::SecureFile`
-
-NOTE:
-`GroupWikiRepository` is not in the previous list since verification is not implemented.
-There is an [issue to implement this functionality in the Admin UI](https://gitlab.com/gitlab-org/gitlab/-/issues/364729).
-
-### Artifacts
-
-Moved to [Geo replication troubleshooting](../geo/replication/troubleshooting.md#find-failed-artifacts).
-
-### Repository verification failures
-
-Moved to [Geo replication troubleshooting](../geo/replication/troubleshooting.md#repository-verification-failures).
-
-### Resync repositories
-
-Moved to [Geo replication troubleshooting](../geo/replication/troubleshooting.md#resync-repositories).
-
-### Blob types
-
-Moved to [Geo replication troubleshooting](../geo/replication/troubleshooting.md#blob-types).
-
-## Generate Service Ping
-
-The [Service Ping Guide](../../development/service_ping/index.md) in our developer documentation
-has more information about Service Ping.
-
-### Generate or get the cached Service Ping
-
-```ruby
-Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values, cached: true)
-```
-
-### Generate a fresh new Service Ping
-
-This also refreshes the cached Service Ping displayed in the Admin Area
-
-```ruby
-Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)
-```
-
-### Generate and print
-
-Generates Service Ping data in JSON format.
-
-```shell
-rake gitlab:usage_data:generate
-```
-
-Generates Service Ping data in YAML format:
-
-```shell
-rake gitlab:usage_data:dump_sql_in_yaml
-```
-
-### Generate and send Service Ping
-
-Prints the metrics saved in `conversational_development_index_metrics`.
-
-```shell
-rake gitlab:usage_data:generate_and_send
-```
-
-## GraphQL
-
-Call a [GraphQL](../../api/graphql/getting_started.md) endpoint through the Rails console:
-
-```ruby
-query = <<~EOQ
-query securityGetProjects($search: String!) {
- projects(search: $search) {
- nodes {
- path
- }
- }
-}
-EOQ
-
-variables = { "search": "gitlab" }
-
-result = GitlabSchema.execute(query, variables: variables, context: { current_user: current_user })
-result.to_h
-```
+<!-- This redirect file can be deleted after 2023-02-01. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/administration/troubleshooting/group_saml_scim.md b/doc/administration/troubleshooting/group_saml_scim.md
deleted file mode 100644
index b5187504231..00000000000
--- a/doc/administration/troubleshooting/group_saml_scim.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../../user/group/saml_sso/example_saml_config.md'
-remove_date: '2022-10-29'
----
-
-This document was moved to [another location](../../user/group/saml_sso/example_saml_config.md).
-
-<!-- This redirect file can be deleted after <2022-10-29>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/administration/troubleshooting/index.md b/doc/administration/troubleshooting/index.md
index ce548f9e100..db572f202ea 100644
--- a/doc/administration/troubleshooting/index.md
+++ b/doc/administration/troubleshooting/index.md
@@ -9,19 +9,21 @@ info: To determine the technical writer assigned to the Stage/Group associated w
This page documents a collection of resources to help you troubleshoot a GitLab
installation.
+This list is not necessarily comprehensive. If you don't find what you're looking
+for in this list, you should search the documentation.
+
## Troubleshooting guides
- [SSL](ssl.md)
- [Geo](../geo/replication/troubleshooting.md)
-- [GitLab Rails console cheat sheet](gitlab_rails_cheat_sheet.md)
-- [Example group SAML and SCIM configurations](../../user/group/saml_sso/example_saml_config.md)
+- [SAML](../../user/group/saml_sso/troubleshooting.md)
- [Kubernetes cheat sheet](https://docs.gitlab.com/charts/troubleshooting/kubernetes_cheat_sheet.html)
- [Linux cheat sheet](linux_cheat_sheet.md)
- [Parsing GitLab logs with `jq`](../logs/log_parsing.md)
- [Diagnostics tools](diagnostics_tools.md)
Some feature documentation pages also have a troubleshooting section at the end
-that you can check for feature-specific help.
+that you can check for feature-specific help, including helpful Rails commands.
If you need a testing environment to troubleshoot, see the
[apps for a testing environment](test_environments.md).
diff --git a/doc/administration/troubleshooting/kubernetes_cheat_sheet.md b/doc/administration/troubleshooting/kubernetes_cheat_sheet.md
deleted file mode 100644
index 15ec8d5940b..00000000000
--- a/doc/administration/troubleshooting/kubernetes_cheat_sheet.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: 'https://docs.gitlab.com/charts/troubleshooting/kubernetes_cheat_sheet.html'
-remove_date: '2022-10-05'
----
-
-This document was moved to [another location](https://docs.gitlab.com/charts/troubleshooting/kubernetes_cheat_sheet.html).
-
-<!-- This redirect file can be deleted after 2022-10-05. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/administration/troubleshooting/linux_cheat_sheet.md b/doc/administration/troubleshooting/linux_cheat_sheet.md
index 0c6fcd31d1d..90cd1e24c79 100644
--- a/doc/administration/troubleshooting/linux_cheat_sheet.md
+++ b/doc/administration/troubleshooting/linux_cheat_sheet.md
@@ -210,7 +210,7 @@ using the `-s` or `--sort` flag. The number of results defaults to 25 processes,
can be changed using the `-c`/`--count` option. See `--help` for full details.
```shell
-$ ./strace-parser sidekiq_trace.txt summary -c15 -s=pid
+$ ./strace-parser sidekiq_trace.txt summary -c15 -s=pid
Top 15 PIDs by PID #
-----------
@@ -244,7 +244,7 @@ processes using the `-p`/`--pid` for a specific process, or `-s`/`--stats` flags
a sorted list. `--stats` takes the same sorting and count options as summary.
```shell
-./strace-parser sidekiq_trace.txt p 16815
+./strace-parser sidekiq_trace.txt p 16815
PID 16815
diff --git a/doc/administration/troubleshooting/log_parsing.md b/doc/administration/troubleshooting/log_parsing.md
deleted file mode 100644
index 929a49494be..00000000000
--- a/doc/administration/troubleshooting/log_parsing.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../logs/log_parsing.md'
-remove_date: '2022-10-24'
----
-
-This document was moved to [another location](../logs/log_parsing.md).
-
-<!-- This redirect file can be deleted after <2022-10-24>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/administration/troubleshooting/navigating_gitlab_via_rails_console.md b/doc/administration/troubleshooting/navigating_gitlab_via_rails_console.md
deleted file mode 100644
index 09a5cb8d185..00000000000
--- a/doc/administration/troubleshooting/navigating_gitlab_via_rails_console.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../operations/rails_console.md'
-remove_date: '2022-10-05'
----
-
-This document was moved to [another location](../operations/rails_console.md).
-
-<!-- This redirect file can be deleted after <2022-10-05>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/administration/troubleshooting/postgresql.md b/doc/administration/troubleshooting/postgresql.md
index 8a76d4f88ab..829fed38060 100644
--- a/doc/administration/troubleshooting/postgresql.md
+++ b/doc/administration/troubleshooting/postgresql.md
@@ -58,59 +58,6 @@ This section is for links to information elsewhere in the GitLab documentation.
some of which is absolutely not for production use. Including:
- Understanding EXPLAIN plans.
-### Troubleshooting/Fixes
-
-- [GitLab database requirements](../../install/requirements.md#database),
- including
- - Support for MySQL was removed in GitLab 12.1; [migrate to PostgreSQL](../../update/mysql_to_postgresql.md).
- - Required extension: `pg_trgm`
- - Required extension: `btree_gist`
-
-- Errors like this in the `production/sidekiq` log; see:
- [Set `default_transaction_isolation` into read committed](https://docs.gitlab.com/omnibus/settings/database.html#set-default_transaction_isolation-into-read-committed):
-
- ```plaintext
- ActiveRecord::StatementInvalid PG::TRSerializationFailure: ERROR: could not serialize access due to concurrent update
- ```
-
-- PostgreSQL HA [replication slot errors](https://docs.gitlab.com/omnibus/settings/database.html#troubleshooting-upgrades-in-an-ha-cluster):
-
- ```plaintext
- pg_basebackup: could not create temporary replication slot "pg_basebackup_12345": ERROR: all replication slots are in use
- HINT: Free one or increase max_replication_slots.
- ```
-
-- Geo [replication errors](../geo/replication/troubleshooting.md#fixing-replication-errors) including:
-
- ```plaintext
- ERROR: replication slots can only be used if max_replication_slots > 0
-
- FATAL: could not start WAL streaming: ERROR: replication slot "geo_secondary_my_domain_com" does not exist
-
- Command exceeded allowed execution time
-
- PANIC: could not write to file 'pg_xlog/xlogtemp.123': No space left on device
- ```
-
-- [Checking Geo configuration](../geo/replication/troubleshooting.md), including:
- - Reconfiguring hosts/ports.
- - Checking and fixing user/password mappings.
-
-- [Common Geo errors](../geo/replication/troubleshooting.md#fixing-common-errors).
-
-- Mismatch in `pg_dump` and `psql` versions:
-
- ```plaintext
- Dumping PostgreSQL database gitlabhq_production ... pg_dump: error: server version: 13.3; pg_dump version: 14.2
- pg_dump: error: aborting because of server version mismatch
- ```
-
- To fix this, see [Backup and restore a non-packaged PostgreSQL database](https://docs.gitlab.com/omnibus/settings/database.html#backup-and-restore-a-non-packaged-postgresql-database).
-
-- Deploying PostgreSQL on Azure Database for PostgreSQL - Flexible Server may result in an error stating `extension "btree_gist" is not allow-listed for "azure_pg_admin" users in Azure Database for PostgreSQL`
-
- To resolve the above error, [allow-list the extension](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-extensions#how-to-use-postgresql-extensions) prior to install.
-
## Support topics
### Database deadlocks
@@ -236,3 +183,96 @@ To temporarily change the statement timeout:
1. Perform the action for which you need a different timeout
(for example the backup or the Rails command).
1. Revert the edit in `/var/opt/gitlab/gitlab-rails/etc/database.yml`.
+
+## Troubleshooting
+
+### Database is not accepting commands to avoid wraparound data loss
+
+This error likely means that AUTOVACUUM is failing to complete its run:
+
+```plaintext
+ERROR: database is not accepting commands to avoid wraparound data loss in database "gitlabhq_production"
+```
+
+To resolve the error, run `VACUUM` manually:
+
+1. Stop GitLab with the command `gitlab-ctl stop`.
+1. Place the database in single-user mode with the command:
+
+ ```shell
+ /opt/gitlab/embedded/bin/postgres --single -D /var/opt/gitlab/postgresql/data gitlabhq_production
+ ```
+
+1. In the `backend>` prompt, run `VACUUM;`. This command can take several minutes to complete.
+1. Wait for the command to complete, then press <kbd>Control</kbd> + <kbd>D</kbd> to exit.
+1. Start GitLab with the command `gitlab-ctl start`.
+
+### GitLab database requirements
+
+The [database requirements](../../install/requirements.md#database) for GitLab include:
+
+- Support for MySQL was removed in GitLab 12.1; [migrate to PostgreSQL](../../update/mysql_to_postgresql.md).
+- Review and install the [required extension list](../../install/postgresql_extensions.md).
+
+### Serialization errors in the `production/sidekiq` log
+
+If you receive errors like this example in your `production/sidekiq` log, read
+about [setting `default_transaction_isolation` into read committed](https://docs.gitlab.com/omnibus/settings/database.html#set-default_transaction_isolation-into-read-committed) to fix the problem:
+
+```plaintext
+ActiveRecord::StatementInvalid PG::TRSerializationFailure: ERROR: could not serialize access due to concurrent update
+```
+
+### PostgreSQL replication slot errors
+
+If you receive errors like this example, read about how to resolve PostgreSQL HA
+[replication slot errors](https://docs.gitlab.com/omnibus/settings/database.html#troubleshooting-upgrades-in-an-ha-cluster):
+
+```plaintext
+pg_basebackup: could not create temporary replication slot "pg_basebackup_12345": ERROR: all replication slots are in use
+HINT: Free one or increase max_replication_slots.
+```
+
+### Geo replication errors
+
+If you receive errors like this example, read about how to resolve
+[Geo replication errors](../geo/replication/troubleshooting.md#fixing-postgresql-database-replication-errors):
+
+```plaintext
+ERROR: replication slots can only be used if max_replication_slots > 0
+
+FATAL: could not start WAL streaming: ERROR: replication slot "geo_secondary_my_domain_com" does not exist
+
+Command exceeded allowed execution time
+
+PANIC: could not write to file 'pg_xlog/xlogtemp.123': No space left on device
+```
+
+### Review Geo configuration and common errors
+
+When troubleshooting problems with Geo, you should:
+
+- Review [common Geo errors](../geo/replication/troubleshooting.md#fixing-common-errors).
+- [Review your Geo configuration](../geo/replication/troubleshooting.md), including:
+ - Reconfiguring hosts and ports.
+ - Reviewing and fixing the user and password mappings.
+
+### Mismatch in `pg_dump` and `psql` versions
+
+If you receive errors like this example, read about how to
+[back up and restore a non-packaged PostgreSQL database](https://docs.gitlab.com/omnibus/settings/database.html#backup-and-restore-a-non-packaged-postgresql-database):
+
+```plaintext
+Dumping PostgreSQL database gitlabhq_production ... pg_dump: error: server version: 13.3; pg_dump version: 14.2
+pg_dump: error: aborting because of server version mismatch
+```
+
+### Extension `btree_gist` is not allow-listed
+
+Deploying PostgreSQL on an Azure Database for PostgreSQL - Flexible Server may result in this error:
+
+```plaintext
+extension "btree_gist" is not allow-listed for "azure_pg_admin" users in Azure Database for PostgreSQL
+```
+
+To resolve this error, [allow-list the extension](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-extensions#how-to-use-postgresql-extensions) prior to install.
diff --git a/doc/administration/troubleshooting/ssl.md b/doc/administration/troubleshooting/ssl.md
index 245ff9f4982..bf8e6b45fde 100644
--- a/doc/administration/troubleshooting/ssl.md
+++ b/doc/administration/troubleshooting/ssl.md
@@ -1,263 +1,11 @@
---
-stage: Systems
-group: Distribution
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-type: reference
+redirect_to: 'https://docs.gitlab.com/omnibus/settings/ssl/ssl_troubleshooting.html'
+remove_date: '2023-10-27'
---
-# Troubleshooting SSL **(FREE SELF)**
+This document was moved to [another location](https://docs.gitlab.com/omnibus/settings/ssl/ssl_troubleshooting.html).
-This page contains a list of common SSL-related errors and scenarios that you
-may encounter while working with GitLab. It should serve as an addition to the
-main SSL documentation:
-
-- [Omnibus SSL Configuration](https://docs.gitlab.com/omnibus/settings/ssl.html).
-- [Self-signed certificates or custom Certification Authorities for GitLab Runner](https://docs.gitlab.com/runner/configuration/tls-self-signed.html).
-- [Configure HTTPS manually](https://docs.gitlab.com/omnibus/settings/ssl.html#configure-https-manually).
-
-## Using an internal CA certificate with GitLab
-
-After configuring a GitLab instance with an internal CA certificate, you might
-not be able to access it by using various CLI tools. You may experience the
-following issues:
-
-- `curl` fails:
-
- ```shell
- curl "https://gitlab.domain.tld"
- curl: (60) SSL certificate problem: unable to get local issuer certificate
- More details here: https://curl.haxx.se/docs/sslcerts.html
- ```
-
-- Testing by using the [rails console](../operations/rails_console.md#starting-a-rails-console-session)
- also fails:
-
- ```ruby
- uri = URI.parse("https://gitlab.domain.tld")
- http = Net::HTTP.new(uri.host, uri.port)
- http.use_ssl = true
- http.verify_mode = 1
- response = http.request(Net::HTTP::Get.new(uri.request_uri))
- ...
- Traceback (most recent call last):
- 1: from (irb):5
- OpenSSL::SSL::SSLError (SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate))
- ```
-
-- The error `SSL certificate problem: unable to get local issuer certificate`
- is displayed when setting up a [mirror](../../user/project/repository/mirror/index.md)
- from this GitLab instance.
-- `openssl` works when specifying the path to the certificate:
-
- ```shell
- /opt/gitlab/embedded/bin/openssl s_client -CAfile /root/my-cert.crt -connect gitlab.domain.tld:443 -servername gitlab.domain.tld
- ```
-
-If you have the previously described issues, add your certificate to
-`/etc/gitlab/trusted-certs`, and then run `sudo gitlab-ctl reconfigure`.
-
-## X.509 key values mismatch error
-
-After configuring your instance with a certificate bundle, NGINX may display
-the following error message:
-
-`SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch`
-
-This error message means that the server certificate and key you have provided
-don't match. You can confirm this by running the following command and then
-comparing the output:
-
-```shell
-openssl rsa -noout -modulus -in path/to/your/.key | openssl md5
-openssl x509 -noout -modulus -in path/to/your/.crt | openssl md5
-```
-
-The following is an example of an md5 output between a matching key and
-certificate. Note the matching md5 hashes:
-
-```shell
-$ openssl rsa -noout -modulus -in private.key | openssl md5
-4f49b61b25225abeb7542b29ae20e98c
-$ openssl x509 -noout -modulus -in public.crt | openssl md5
-4f49b61b25225abeb7542b29ae20e98c
-```
-
-This is an opposing output with a non-matching key and certificate which shows
-different md5 hashes:
-
-```shell
-$ openssl rsa -noout -modulus -in private.key | openssl md5
-d418865077299af27707b1d1fa83cd99
-$ openssl x509 -noout -modulus -in public.crt | openssl md5
-4f49b61b25225abeb7542b29ae20e98c
-```
-
-If the two outputs differ like the previous example, there's a mismatch between
-the certificate and key. Contact the provider of the SSL certificate for
-further support.
-
-## Using GitLab Runner with a GitLab instance configured with internal CA certificate or self-signed certificate
-
-Besides getting the errors mentioned in
-[Using an internal CA certificate with GitLab](ssl.md#using-an-internal-ca-certificate-with-gitlab),
-your CI pipelines may get stuck in `Pending` status. In the runner logs you may
-see the following error message:
-
-```shell
-Dec 6 02:43:17 runner-host01 gitlab-runner[15131]: #033[0;33mWARNING: Checking for jobs... failed
-#033[0;m #033[0;33mrunner#033[0;m=Bfkz1fyb #033[0;33mstatus#033[0;m=couldn't execute POST against
-https://gitlab.domain.tld/api/v4/jobs/request: Post https://gitlab.domain.tld/api/v4/jobs/request:
-x509: certificate signed by unknown authority
-```
-
-Follow the details in [Self-signed certificates or custom Certification Authorities for GitLab Runner](https://docs.gitlab.com/runner/configuration/tls-self-signed.html).
-
-## Mirroring a remote GitLab repository that uses a self-signed SSL certificate
-
-When configuring a local GitLab instance to [mirror a repository](../../user/project/repository/mirror/index.md)
-from a remote GitLab instance that uses a self-signed certificate, you may see
-the `SSL certificate problem: self signed certificate` error message in the
-user interface.
-
-The cause of the issue can be confirmed by checking if:
-
-- `curl` fails:
-
- ```shell
- $ curl "https://gitlab.domain.tld"
- curl: (60) SSL certificate problem: self signed certificate
- More details here: https://curl.haxx.se/docs/sslcerts.html
- ```
-
-- Testing by using the Rails console also fails:
-
- ```ruby
- uri = URI.parse("https://gitlab.domain.tld")
- http = Net::HTTP.new(uri.host, uri.port)
- http.use_ssl = true
- http.verify_mode = 1
- response = http.request(Net::HTTP::Get.new(uri.request_uri))
- ...
- Traceback (most recent call last):
- 1: from (irb):5
- OpenSSL::SSL::SSLError (SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate))
- ```
-
-To fix this problem:
-
-- Add the self-signed certificate from the remote GitLab instance to the
- `/etc/gitlab/trusted-certs` directory on the local GitLab instance, and then
- run `sudo gitlab-ctl reconfigure` as per the instructions for
- [installing custom public certificates](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates).
-- If your local GitLab instance was installed using the Helm Charts, you can
- [add your self-signed certificate to your GitLab instance](https://docs.gitlab.com/runner/install/kubernetes.html#providing-a-custom-certificate-for-accessing-gitlab).
-
-You may also get another error message when trying to mirror a repository from
-a remote GitLab instance that uses a self-signed certificate:
-
-```shell
-2:Fetching remote upstream failed: fatal: unable to access &amp;#39;https://gitlab.domain.tld/root/test-repo/&amp;#39;:
-SSL: unable to obtain common name from peer certificate
-```
-
-In this case, the problem can be related to the certificate itself:
-
-1. Validate that your self-signed certificate isn't missing a common name. If it
- is, regenerate a valid certificate
-1. Add the certificate to `/etc/gitlab/trusted-certs`.
-1. Run `sudo gitlab-ctl reconfigure`.
-
-## Unable to perform Git operations due to an internal or self-signed certificate
-
-If your GitLab instance is using a self-signed certificate, or if the
-certificate is signed by an internal certificate authority (CA), you might
-experience the following errors when attempting to perform Git operations:
-
-```shell
-$ git clone https://gitlab.domain.tld/group/project.git
-Cloning into 'project'...
-fatal: unable to access 'https://gitlab.domain.tld/group/project.git/': SSL certificate problem: self signed certificate
-```
-
-```shell
-$ git clone https://gitlab.domain.tld/group/project.git
-Cloning into 'project'...
-fatal: unable to access 'https://gitlab.domain.tld/group/project.git/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
-```
-
-To fix this problem:
-
-- If possible, use SSH remotes for all Git operations. This is considered more
- secure and convenient to use.
-- If you must use HTTPS remotes, you can try the following:
- - Copy the self-signed certificate or the internal root CA certificate to a
- local directory (for example, `~/.ssl`) and configure Git to trust your
- certificate:
-
- ```shell
- git config --global http.sslCAInfo ~/.ssl/gitlab.domain.tld.crt
- ```
-
- - Disable SSL verification in your Git client. This is intended as a
- temporary measure, as it could be considered a security risk.
-
- ```shell
- git config --global http.sslVerify false
- ```
-
-## SSL_connect wrong version number
-
-A misconfiguration may result in:
-
-- `gitlab-rails/exceptions_json.log` entries containing:
-
- ```plaintext
- "exception.class":"Excon::Error::Socket","exception.message":"SSL_connect returned=1 errno=0 state=error: wrong version number (OpenSSL::SSL::SSLError)",
- "exception.class":"Excon::Error::Socket","exception.message":"SSL_connect returned=1 errno=0 state=error: wrong version number (OpenSSL::SSL::SSLError)",
- ```
-
-- `gitlab-workhorse/current` containing:
-
- ```plaintext
- http: server gave HTTP response to HTTPS client
- http: server gave HTTP response to HTTPS client
- ```
-
-- `gitlab-rails/sidekiq.log` or `sidekiq/current` containing:
-
- ```plaintext
- message: SSL_connect returned=1 errno=0 state=error: wrong version number (OpenSSL::SSL::SSLError)
- message: SSL_connect returned=1 errno=0 state=error: wrong version number (OpenSSL::SSL::SSLError)
- ```
-
-Some of these errors come from the Excon Ruby gem, and could be generated in
-circumstances where GitLab is configured to initiate an HTTPS session to a
-remote server that is serving only HTTP.
-
-One scenario is that you're using [object storage](../object_storage.md), which
-isn't served under HTTPS. GitLab is misconfigured and attempts a TLS handshake,
-but the object storage responds with plain HTTP.
-
-## `schannel: SEC_E_UNTRUSTED_ROOT`
-
-If you're on Windows and get the following error:
-
-```plaintext
-Fatal: unable to access 'https://gitlab.domain.tld/group/project.git': schannel: SEC_E_UNTRUSTED_ROOT (0x80090325) - The certificate chain was issued by an authority that is not trusted."
-```
-
-You must specify that Git should use OpenSSL:
-
-```shell
-git config --system http.sslbackend openssl
-```
-
-Alternatively, you can ignore SSL verification by running:
-
-WARNING:
-Proceed with caution when [ignoring SSL](https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpsslVerify)
-due to the potential security issues associated with disabling this option at global level. Use this option _only_ when troubleshooting, and reinstate SSL verification immediately after.
-
-```shell
-git config --global http.sslVerify false
-```
+<!-- This redirect file can be deleted after <2023-01-25>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/administration/troubleshooting/test_environments.md b/doc/administration/troubleshooting/test_environments.md
index a249d5bd133..c4d191f8c0f 100644
--- a/doc/administration/troubleshooting/test_environments.md
+++ b/doc/administration/troubleshooting/test_environments.md
@@ -20,13 +20,13 @@ are only available internally at GitLab.
## Docker
The following were tested on Docker containers running in the cloud. Support Engineers,
-please see [these docs](https://gitlab.com/gitlab-com/dev-resources/tree/master/dev-resources#running-docker-containers)
+see [these docs](https://gitlab.com/gitlab-com/dev-resources/tree/master/dev-resources#running-docker-containers)
on how to run Docker containers on `dev-resources`. Other setups haven't been tested,
but contributions are welcome.
### GitLab
-Please see [our official Docker installation method](../../install/docker.md)
+See [our official Docker installation method](../../install/docker.md)
for how to run GitLab on Docker.
### SAML
diff --git a/doc/administration/user_settings.md b/doc/administration/user_settings.md
index a767132db0f..c96a6311208 100644
--- a/doc/administration/user_settings.md
+++ b/doc/administration/user_settings.md
@@ -14,7 +14,7 @@ By default, new users can create top-level groups. To disable new users'
ability to create top-level groups (does not affect existing users' setting), GitLab administrators can modify this setting:
- In GitLab 15.5 and later, using either:
- - The [GitLab UI](../user/admin_area/settings/account_and_limit_settings.md#prevent-users-from-creating-top-level-groups).
+ - The [GitLab UI](../user/admin_area/settings/account_and_limit_settings.md#prevent-new-users-from-creating-top-level-groups).
- The [application setting API](../api/settings.md#change-application-settings).
- In GitLab 15.4 and earlier, in a configuration file by following the steps in this section.
diff --git a/doc/api/award_emoji.md b/doc/api/award_emoji.md
index ca6761ed6be..9d0b8945c53 100644
--- a/doc/api/award_emoji.md
+++ b/doc/api/award_emoji.md
@@ -4,7 +4,7 @@ group: Project Management
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Award emoji API **(FREE)**
+# Award emojis API **(FREE)**
An [awarded emoji](../user/award_emojis.md) tells a thousand words.
@@ -15,11 +15,11 @@ We call GitLab objects on which you can award an emoji "awardables". You can awa
- [Merge requests](../user/project/merge_requests/index.md) ([API](merge_requests.md)).
- [Snippets](../user/snippets.md) ([API](snippets.md)).
-Emojis can also [be awarded](../user/award_emojis.md#award-emoji-for-comments) on comments (also known as notes). See also [Notes API](notes.md).
+Emojis can also [be awarded](../user/award_emojis.md#award-emojis-for-comments) on comments (also known as notes). See also [Notes API](notes.md).
## Issues, merge requests, and snippets
-See [Award Emoji on Comments](#award-emoji-on-comments) for information on using these endpoints with comments.
+See [Award emojis on comments](#award-emojis-on-comments) for information on using these endpoints with comments.
### List an awardable's award emojis
@@ -201,7 +201,7 @@ Parameters:
curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/issues/80/award_emoji/344"
```
-## Award Emoji on Comments
+## Award emojis on comments
Comments (also known as notes) are a sub-resource of issues, merge requests, and snippets.
@@ -366,7 +366,7 @@ Parameters:
| `id` | integer/string | yes | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
| `issue_iid` | integer | yes | Internal ID of an issue. |
| `note_id` | integer | yes | ID of a comment (note). |
-| `award_id` | integer | yes | ID of an award_emoji. |
+| `award_id` | integer | yes | ID of an award emoji. |
Example request:
diff --git a/doc/api/broadcast_messages.md b/doc/api/broadcast_messages.md
index cc777a8bf53..d91557523a9 100644
--- a/doc/api/broadcast_messages.md
+++ b/doc/api/broadcast_messages.md
@@ -6,7 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Broadcast Messages API **(FREE SELF)**
-> `target_access_levels` [introduced](https://gitlab.com/gitlab-org/growth/team-tasks/-/issues/461) in GitLab 14.8 [with a flag](../administration/feature_flags.md) named `role_targeted_broadcast_messages`. Disabled by default.
+- > `target_access_levels` [introduced](https://gitlab.com/gitlab-org/growth/team-tasks/-/issues/461) in GitLab 14.8 [with a flag](../administration/feature_flags.md) named `role_targeted_broadcast_messages`. Disabled by default.
+- > `color` parameter [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95829) in GitLab 15.6.
Broadcast messages API operates on [broadcast messages](../user/admin_area/broadcast_messages.md).
@@ -37,7 +38,6 @@ Example response:
"message":"Example broadcast message",
"starts_at":"2016-08-24T23:21:16.078Z",
"ends_at":"2016-08-26T23:21:16.080Z",
- "color":"#E75E40",
"font":"#FFFFFF",
"id":1,
"active": false,
@@ -76,7 +76,6 @@ Example response:
"message":"Deploy in progress",
"starts_at":"2016-08-24T23:21:16.078Z",
"ends_at":"2016-08-26T23:21:16.080Z",
- "color":"#cecece",
"font":"#FFFFFF",
"id":1,
"active":false,
@@ -102,7 +101,6 @@ Parameters:
| `message` | string | yes | Message to display. |
| `starts_at` | datetime | no | Starting time (defaults to current time in UTC). Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) |
| `ends_at` | datetime | no | Ending time (defaults to one hour from current time in UTC). Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) |
-| `color` | string | no | Background color hex code. |
| `font` | string | no | Foreground color hex code. |
| `target_access_levels` | array of integers | no | Target access levels (roles) of the broadcast message.|
| `target_path` | string | no | Target path of the broadcast message. |
@@ -121,7 +119,7 @@ following levels are valid:
Example request:
```shell
-curl --data "message=Deploy in progress&color=#cecece&target_access_levels[]=10&target_access_levels[]=30" \
+curl --data "message=Deploy in progress&target_access_levels[]=10&target_access_levels[]=30" \
--header "PRIVATE-TOKEN: <your_access_token>" \
"https://gitlab.example.com/api/v4/broadcast_messages"
```
@@ -133,7 +131,6 @@ Example response:
"message":"Deploy in progress",
"starts_at":"2016-08-26T00:41:35.060Z",
"ends_at":"2016-08-26T01:41:35.060Z",
- "color":"#cecece",
"font":"#FFFFFF",
"id":1,
"active": true,
@@ -160,7 +157,6 @@ Parameters:
| `message` | string | no | Message to display. |
| `starts_at` | datetime | no | Starting time (UTC). Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) |
| `ends_at` | datetime | no | Ending time (UTC). Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) |
-| `color` | string | no | Background color hex code. |
| `font` | string | no | Foreground color hex code. |
| `target_access_levels` | array of integers | no | Target access levels (roles) of the broadcast message.|
| `target_path` | string | no | Target path of the broadcast message. |
@@ -179,7 +175,7 @@ following levels are valid:
Example request:
```shell
-curl --request PUT --data "message=Update message&color=#000" \
+curl --request PUT --data "message=Update message" \
--header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/broadcast_messages/1"
```
@@ -190,7 +186,6 @@ Example response:
"message":"Update message",
"starts_at":"2016-08-26T00:41:35.060Z",
"ends_at":"2016-08-26T01:41:35.060Z",
- "color":"#000",
"font":"#FFFFFF",
"id":1,
"active": true,
diff --git a/doc/api/commits.md b/doc/api/commits.md
index f08fe5ba881..3fe77dd5f43 100644
--- a/doc/api/commits.md
+++ b/doc/api/commits.md
@@ -58,7 +58,7 @@ Example response:
"parent_ids": [
"6104942438c14ec7bd21c6cd5bd995272b3faff6"
],
- "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746"
+ "web_url": "https://gitlab.example.com/janedoe/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746"
},
{
"id": "6104942438c14ec7bd21c6cd5bd995272b3faff6",
@@ -73,7 +73,7 @@ Example response:
"parent_ids": [
"ae1d9fb46aa2b07ee9836d49862ec4e2c46fbbba"
],
- "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746"
+ "web_url": "https://gitlab.example.com/janedoe/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746"
}
]
```
@@ -173,7 +173,7 @@ Example response:
"total": 4
},
"status": null,
- "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746"
+ "web_url": "https://gitlab.example.com/janedoe/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746"
}
```
@@ -253,7 +253,7 @@ Example response:
"total": 25
},
"status": "running",
- "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/6104942438c14ec7bd21c6cd5bd995272b3faff6"
+ "web_url": "https://gitlab.example.com/janedoe/gitlab-foss/-/commit/6104942438c14ec7bd21c6cd5bd995272b3faff6"
}
```
@@ -331,7 +331,7 @@ Example response:
"parent_ids": [
"a738f717824ff53aebad8b090c1b79a14f2bd9e8"
],
- "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/8b090c1b79a14f2bd9e8a738f717824ff53aebad"
+ "web_url": "https://gitlab.example.com/janedoe/gitlab-foss/-/commit/8b090c1b79a14f2bd9e8a738f717824ff53aebad"
}
```
@@ -401,7 +401,7 @@ Example response:
"committer_name":"Administrator",
"committer_email":"admin@example.com",
"committed_date":"2018-11-08T15:55:26.000Z",
- "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/commit/8b090c1b79a14f2bd9e8a738f717824ff53aebad"
+ "web_url": "https://gitlab.example.com/janedoe/gitlab-foss/-/commit/8b090c1b79a14f2bd9e8a738f717824ff53aebad"
}
```
@@ -536,7 +536,7 @@ POST /projects/:id/repository/commits/:sha/comments
```shell
curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
- --form "note=Nice picture man\!" --form "path=dudeism.md" --form "line=11" --form "line_type=new" \
+ --form "note=Nice picture\!" --form "path=README.md" --form "line=11" --form "line_type=new" \
"https://gitlab.example.com/api/v4/projects/17/repository/commits/18f3e63d05582537db6d183d9d557be09e1f90c8/comments"
```
@@ -545,18 +545,18 @@ Example response:
```json
{
"author" : {
- "web_url" : "https://gitlab.example.com/thedude",
- "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png",
- "username" : "thedude",
+ "web_url" : "https://gitlab.example.com/janedoe",
+ "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/jane-doe-400-400.png",
+ "username" : "janedoe",
"state" : "active",
- "name" : "Jeff Lebowski",
+ "name" : "Jane Doe",
"id" : 28
},
"created_at" : "2016-01-19T09:44:55.600Z",
"line_type" : "new",
- "path" : "dudeism.md",
+ "path" : "README.md",
"line" : 11,
- "note" : "Nice picture man!"
+ "note" : "Nice picture!"
}
```
@@ -590,15 +590,15 @@ Example response:
{
"id": 334686748,
"type": null,
- "body": "I'm the Dude, so that's what you call me.",
+ "body": "Nice piece of code!",
"attachment": null,
"author" : {
"id" : 28,
- "name" : "Jeff Lebowski",
- "username" : "thedude",
- "web_url" : "https://gitlab.example.com/thedude",
+ "name" : "Jane Doe",
+ "username" : "janedoe",
+ "web_url" : "https://gitlab.example.com/janedoe",
"state" : "active",
- "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png"
+ "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/jane-doe-400-400.png"
},
"created_at": "2020-04-30T18:48:11.432Z",
"updated_at": "2020-04-30T18:48:11.432Z",
@@ -655,16 +655,16 @@ Example response:
"name" : "bundler:audit",
"allow_failure" : true,
"author" : {
- "username" : "thedude",
+ "username" : "janedoe",
"state" : "active",
- "web_url" : "https://gitlab.example.com/thedude",
- "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png",
+ "web_url" : "https://gitlab.example.com/janedoe",
+ "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/jane-doe-400-400.png",
"id" : 28,
- "name" : "Jeff Lebowski"
+ "name" : "Jane Doe"
},
"description" : null,
"sha" : "18f3e63d05582537db6d183d9d557be09e1f90c8",
- "target_url" : "https://gitlab.example.com/thedude/gitlab-foss/builds/91",
+ "target_url" : "https://gitlab.example.com/janedoe/gitlab-foss/builds/91",
"finished_at" : null,
"id" : 91,
"ref" : "master"
@@ -675,18 +675,18 @@ Example response:
"allow_failure" : false,
"status" : "pending",
"created_at" : "2016-01-19T08:40:25.832Z",
- "target_url" : "https://gitlab.example.com/thedude/gitlab-foss/builds/90",
+ "target_url" : "https://gitlab.example.com/janedoe/gitlab-foss/builds/90",
"id" : 90,
"finished_at" : null,
"ref" : "master",
"sha" : "18f3e63d05582537db6d183d9d557be09e1f90c8",
"author" : {
"id" : 28,
- "name" : "Jeff Lebowski",
- "username" : "thedude",
- "web_url" : "https://gitlab.example.com/thedude",
+ "name" : "Jane Doe",
+ "username" : "janedoe",
+ "web_url" : "https://gitlab.example.com/janedoe",
"state" : "active",
- "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png"
+ "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/jane-doe-400-400.png"
},
"description" : null
},
@@ -724,10 +724,10 @@ Example response:
```json
{
"author" : {
- "web_url" : "https://gitlab.example.com/thedude",
- "name" : "Jeff Lebowski",
- "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png",
- "username" : "thedude",
+ "web_url" : "https://gitlab.example.com/janedoe",
+ "name" : "Jane Doe",
+ "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/jane-doe-400-400.png",
+ "username" : "janedoe",
"state" : "active",
"id" : 28
},
@@ -781,10 +781,10 @@ Example response:
"upvotes":0,
"downvotes":0,
"author" : {
- "web_url" : "https://gitlab.example.com/thedude",
- "name" : "Jeff Lebowski",
- "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png",
- "username" : "thedude",
+ "web_url" : "https://gitlab.example.com/janedoe",
+ "name" : "Jane Doe",
+ "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/jane-doe-400-400.png",
+ "username" : "janedoe",
"state" : "active",
"id" : 28
},
diff --git a/doc/api/container_registry.md b/doc/api/container_registry.md
index 6e8911df098..5b11c324802 100644
--- a/doc/api/container_registry.md
+++ b/doc/api/container_registry.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/dependency_proxy.md b/doc/api/dependency_proxy.md
index 05e144f5810..b7eade83bb8 100644
--- a/doc/api/dependency_proxy.md
+++ b/doc/api/dependency_proxy.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/deploy_tokens.md b/doc/api/deploy_tokens.md
index 5c4c99f3ba3..dff3ef05bb0 100644
--- a/doc/api/deploy_tokens.md
+++ b/doc/api/deploy_tokens.md
@@ -222,7 +222,7 @@ Parameters:
| Attribute | Type | Required | Description |
|:---------------|:---------------|:-----------------------|:------------|
-| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
+| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding). |
| `active` | boolean | **{dotted-circle}** No | Limit by active status. |
Example request:
diff --git a/doc/api/deployments.md b/doc/api/deployments.md
index 9618d57013f..daf2b635855 100644
--- a/doc/api/deployments.md
+++ b/doc/api/deployments.md
@@ -23,7 +23,7 @@ GET /projects/:id/deployments
| `updated_after` | datetime | no | Return deployments updated after the specified date. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). |
| `updated_before` | datetime | no | Return deployments updated before the specified date. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). |
| `environment` | string | no | The [name of the environment](../ci/environments/index.md) to filter deployments by. |
-| `status` | string | no | The status to filter deployments by. One of `created`, `running`, `success`, `failed`, `canceled`, `blocked`.
+| `status` | string | no | The status to filter deployments by. One of `created`, `running`, `success`, `failed`, `canceled`, or `blocked`.
```shell
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/deployments"
@@ -133,7 +133,7 @@ Example response:
"tag": false,
"project": {
"ci_job_token_scope_enabled": false
- },
+ },
"user": {
"id": 1,
"name": "Administrator",
@@ -487,7 +487,11 @@ curl --request "DELETE" --header "PRIVATE-TOKEN: <your_access_token>" "https://g
Example responses:
```json
-{ "message": "202 Accepted" }
+{ "message": "204 Deployment destroyed" }
+```
+
+```json
+{ "message": "403 Forbidden" }
```
```json
@@ -503,8 +507,7 @@ Example responses:
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35739) in GitLab 12.7.
NOTE:
-Not all deployments can be associated with merge requests.
-Please see
+Not all deployments can be associated with merge requests. See
[Track what merge requests were deployed to an environment](../ci/environments/index.md#track-newly-included-merge-requests-per-deployment)
for more information.
diff --git a/doc/api/environments.md b/doc/api/environments.md
index 0f969ea4fd3..89b4bb6a1de 100644
--- a/doc/api/environments.md
+++ b/doc/api/environments.md
@@ -20,7 +20,7 @@ GET /projects/:id/environments
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
| `name` | string | no | Return the environment with this name. Mutually exclusive with `search` |
| `search` | string | no | Return list of environments matching the search criteria. Mutually exclusive with `name` |
-| `states` | string | no | List all environments that match a specific state. Accepted values: `available`, `stopping` or `stopped`. If no state value given, returns all environments. |
+| `states` | string | no | List all environments that match a specific state. Accepted values: `available`, `stopping`, or `stopped`. If no state value given, returns all environments. |
```shell
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/environments?name=review%2Ffix-foo"
@@ -279,7 +279,7 @@ Example response:
}
```
-## Edit an existing environment
+## Update an existing environment
Updates an existing environment's name and/or `external_url`.
diff --git a/doc/api/error_tracking.md b/doc/api/error_tracking.md
index c8f795ed88c..4d6d5e50245 100644
--- a/doc/api/error_tracking.md
+++ b/doc/api/error_tracking.md
@@ -1,6 +1,6 @@
---
stage: Monitor
-group: Respond
+group: Observability
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/events.md b/doc/api/events.md
index 632f573be47..320c7a531e8 100644
--- a/doc/api/events.md
+++ b/doc/api/events.md
@@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
### Actions
-See [User contribution events](../user/profile/index.md#user-contribution-events) for available types for the `action` parameter.
+See [User contribution events](../user/profile/contributions_calendar.md#user-contribution-events) for available types for the `action` parameter.
These options are in lowercase.
### Target Types
diff --git a/doc/api/feature_flag_specs.md b/doc/api/feature_flag_specs.md
index b549c790aaa..960d00278d6 100644
--- a/doc/api/feature_flag_specs.md
+++ b/doc/api/feature_flag_specs.md
@@ -2,11 +2,13 @@
stage: Release
group: Release
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+remove_date: '2023-02-14'
+redirect_to: 'feature_flags.md'
---
-# Feature Flag Specs API **(PREMIUM)**
+# Feature Flag Specs API (removed) **(PREMIUM)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9566) in GitLab 12.5.
This API was removed in [GitLab 14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/213369).
-Please use [the new API](feature_flags.md) instead.
+Use [the new API](feature_flags.md) instead.
diff --git a/doc/api/feature_flag_user_lists.md b/doc/api/feature_flag_user_lists.md
index ef65da64668..ca4ca755d08 100644
--- a/doc/api/feature_flag_user_lists.md
+++ b/doc/api/feature_flag_user_lists.md
@@ -70,8 +70,8 @@ POST /projects/:id/feature_flags_user_lists
| Attribute | Type | Required | Description |
| ------------------- | ---------------- | ---------- | ---------------------------------------------------------------------------------------|
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
-| `name` | string | yes | The name of the feature flag. |
-| `user_xids` | string | yes | A comma-separated list of user IDs. |
+| `name` | string | yes | The name of the list. |
+| `user_xids` | string | yes | A comma-separated list of external user IDs. |
```shell
curl "https://gitlab.example.com/api/v4/projects/1/feature_flags_user_lists" \
@@ -142,8 +142,8 @@ PUT /projects/:id/feature_flags_user_lists/:iid
| ------------------- | ---------------- | ---------- | ---------------------------------------------------------------------------------------|
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
| `iid` | integer/string | yes | The internal ID of the project's feature flag user list. |
-| `name` | string | no | The name of the feature flag. |
-| `user_xids` | string | no | A comma-separated list of user IDs. |
+| `name` | string | no | The name of the list. |
+| `user_xids` | string | no | A comma-separated list of external user IDs. |
```shell
curl "https://gitlab.example.com/api/v4/projects/1/feature_flags_user_lists/1" \
diff --git a/doc/api/feature_flags.md b/doc/api/feature_flags.md
index c4d766a6319..1aec1006610 100644
--- a/doc/api/feature_flags.md
+++ b/doc/api/feature_flags.md
@@ -144,14 +144,14 @@ POST /projects/:id/feature_flags
| ------------------- | ---------------- | ---------- | ---------------------------------------------------------------------------------------|
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
| `name` | string | yes | The name of the feature flag. |
-| `version` | string | yes | The version of the feature flag. Must be `new_version_flag`. Omit or set to `legacy_flag` to create a Legacy feature flag. |
+| `version` | string | yes | The version of the feature flag. Must be `new_version_flag`. Omit to create a Legacy feature flag. |
| `description` | string | no | The description of the feature flag. |
| `active` | boolean | no | The active state of the flag. Defaults to true. [Supported](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38350) in GitLab 13.3 and later. |
| `strategies` | JSON | no | The feature flag [strategies](../operations/feature_flags.md#feature-flag-strategies). |
| `strategies:name` | JSON | no | The strategy name. Can be `default`, `gradualRolloutUserId`, `userWithId`, or `gitlabUserList`. In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/36380) and later, can be [`flexibleRollout`](https://docs.getunleash.io/user_guide/activation_strategy#gradual-rollout). |
| `strategies:parameters` | JSON | no | The strategy parameters. |
| `strategies:scopes` | JSON | no | The scopes for the strategy. |
-| `strategies:scopes:environment_scope` | string | no | The environment spec for the scope. |
+| `strategies:scopes:environment_scope` | string | no | The environment scope of the scope. |
```shell
curl "https://gitlab.example.com/api/v4/projects/1/feature_flags" \
@@ -213,8 +213,8 @@ PUT /projects/:id/feature_flags/:feature_flag_name
| `strategies:name` | JSON | no | The strategy name. |
| `strategies:parameters` | JSON | no | The strategy parameters. |
| `strategies:scopes` | JSON | no | The scopes for the strategy. |
-| `strategies:scopes:id` | JSON | no | The scopes ID. |
-| `strategies:scopes:environment_scope` | string | no | The environment spec for the scope. |
+| `strategies:scopes:id` | JSON | no | The environment scope ID. |
+| `strategies:scopes:environment_scope` | string | no | The environment scope of the scope. |
```shell
curl "https://gitlab.example.com/api/v4/projects/1/feature_flags/awesome_feature" \
diff --git a/doc/api/features.md b/doc/api/features.md
index 6f3af683020..819405bea77 100644
--- a/doc/api/features.md
+++ b/doc/api/features.md
@@ -111,20 +111,21 @@ percentage of time.
POST /features/:name
```
-| Attribute | Type | Required | Description |
-| --------- | ---- | -------- | ----------- |
-| `name` | string | yes | Name of the feature to create or update |
-| `value` | integer/string | yes | `true` or `false` to enable/disable, or an integer for percentage of time |
-| `key` | string | no | `percentage_of_actors` or `percentage_of_time` (default) |
-| `feature_group` | string | no | A Feature group name |
-| `user` | string | no | A GitLab username or comma-separated multiple usernames |
-| `group` | string | no | A GitLab group's path, for example `gitlab-org`, or comma-separated multiple group paths |
-| `namespace` | string | no | A GitLab group or user namespace's path, for example `john-doe`, or comma-separated multiple namespace paths. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353117) in GitLab 15.0. |
-| `project` | string | no | A projects path, for example `gitlab-org/gitlab-foss`, or comma-separated multiple project paths |
-| `force` | boolean | no | Skip feature flag validation checks, such as a YAML definition |
+| Attribute | Type | Required | Description |
+|-----------------|----------------|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `name` | string | yes | Name of the feature to create or update |
+| `value` | integer/string | yes | `true` or `false` to enable/disable, or an integer for percentage of time |
+| `key` | string | no | `percentage_of_actors` or `percentage_of_time` (default) |
+| `feature_group` | string | no | A Feature group name |
+| `user` | string | no | A GitLab username or comma-separated multiple usernames |
+| `group` | string | no | A GitLab group's path, for example `gitlab-org`, or comma-separated multiple group paths |
+| `namespace` | string | no | A GitLab group or user namespace's path, for example `john-doe`, or comma-separated multiple namespace paths. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353117) in GitLab 15.0. |
+| `project` | string | no | A projects path, for example `gitlab-org/gitlab-foss`, or comma-separated multiple project paths |
+| `repository` | string | no | A repository path, for example `gitlab-org/gitlab-test.git`, `gitlab-org/gitlab-test.wiki.git`, , `snippets/21.git`, to name a few. Use comma to separate multiple repository paths |
+| `force` | boolean | no | Skip feature flag validation checks, such as a YAML definition |
You can enable or disable a feature for a `feature_group`, a `user`,
-a `group`, a `namespace` and a `project` in a single API call.
+a `group`, a `namespace`, a `project`, and a `repository` in a single API call.
```shell
curl --data "value=30" --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/features/new_library"
diff --git a/doc/api/freeze_periods.md b/doc/api/freeze_periods.md
index 36d069df607..9d2989229ae 100644
--- a/doc/api/freeze_periods.md
+++ b/doc/api/freeze_periods.md
@@ -16,9 +16,9 @@ You can use the Freeze Periods API to manipulate GitLab [Freeze Period](../user/
Only users with Maintainer [permissions](../user/permissions.md) can
interact with the Freeze Period API endpoints.
-## List Freeze Periods
+## List freeze periods
-Paginated list of Freeze Periods, sorted by `created_at` in ascending order.
+Paginated list of freeze periods, sorted by `created_at` in ascending order.
```plaintext
GET /projects/:id/freeze_periods
@@ -49,9 +49,9 @@ Example response:
]
```
-## Get a Freeze Period by a `freeze_period_id`
+## Get a freeze period by a `freeze_period_id`
-Get a Freeze Period for the given `freeze_period_id`.
+Get a freeze period for the given `freeze_period_id`.
```plaintext
GET /projects/:id/freeze_periods/:freeze_period_id
@@ -60,7 +60,7 @@ GET /projects/:id/freeze_periods/:freeze_period_id
| Attribute | Type | Required | Description |
| ------------- | -------------- | -------- | ----------------------------------------------------------------------------------- |
| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
-| `freeze_period_id` | string | yes | The database ID of the Freeze Period. |
+| `freeze_period_id` | integer | yes | The ID of the freeze period. |
Example request:
@@ -81,9 +81,9 @@ Example response:
}
```
-## Create a Freeze Period
+## Create a freeze period
-Create a Freeze Period.
+Create a freeze period.
```plaintext
POST /projects/:id/freeze_periods
@@ -92,8 +92,8 @@ POST /projects/:id/freeze_periods
| Attribute | Type | Required | Description |
| -------------------| --------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
-| `freeze_start` | string | yes | Start of the Freeze Period in [cron](https://crontab.guru/) format. |
-| `freeze_end` | string | yes | End of the Freeze Period in [cron](https://crontab.guru/) format. |
+| `freeze_start` | string | yes | Start of the freeze period in [cron](https://crontab.guru/) format. |
+| `freeze_end` | string | yes | End of the freeze period in [cron](https://crontab.guru/) format. |
| `cron_timezone` | string | no | The time zone for the cron fields, defaults to UTC if not provided. |
Example request:
@@ -117,9 +117,9 @@ Example response:
}
```
-## Update a Freeze Period
+## Update a freeze period
-Update a Freeze Period for the given `freeze_period_id`.
+Update a freeze period for the given `freeze_period_id`.
```plaintext
PUT /projects/:id/freeze_periods/:freeze_period_id
@@ -128,9 +128,9 @@ PUT /projects/:id/freeze_periods/:freeze_period_id
| Attribute | Type | Required | Description |
| ------------- | --------------- | -------- | ----------------------------------------------------------------------------------------------------------- |
| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
-| `freeze_period_id` | integer or string | yes | The database ID of the Freeze Period. |
-| `freeze_start` | string | no | Start of the Freeze Period in [cron](https://crontab.guru/) format. |
-| `freeze_end` | string | no | End of the Freeze Period in [cron](https://crontab.guru/) format. |
+| `freeze_period_id` | integer | yes | The ID of the freeze period. |
+| `freeze_start` | string | no | Start of the freeze period in [cron](https://crontab.guru/) format. |
+| `freeze_end` | string | no | End of the freeze period in [cron](https://crontab.guru/) format. |
| `cron_timezone` | string | no | The time zone for the cron fields. |
Example request:
@@ -154,9 +154,9 @@ Example response:
}
```
-## Delete a Freeze Period
+## Delete a freeze period
-Delete a Freeze Period for the given `freeze_period_id`.
+Delete a freeze period for the given `freeze_period_id`.
```plaintext
DELETE /projects/:id/freeze_periods/:freeze_period_id
@@ -165,7 +165,7 @@ DELETE /projects/:id/freeze_periods/:freeze_period_id
| Attribute | Type | Required | Description |
| ------------- | -------------- | -------- | ----------------------------------------------------------------------------------- |
| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
-| `freeze_period_id` | string | yes | The database ID of the Freeze Period. |
+| `freeze_period_id` | integer | yes | The ID of the freeze period. |
Example request:
diff --git a/doc/api/geo_nodes.md b/doc/api/geo_nodes.md
index fe25b6661a0..00380e1624b 100644
--- a/doc/api/geo_nodes.md
+++ b/doc/api/geo_nodes.md
@@ -504,6 +504,19 @@ Example response:
"ci_secure_files_synced_in_percentage": "100.00%",
"ci_secure_files_verified_in_percentage": "100.00%",
"ci_secure_files_synced_missing_on_primary_count": 0,
+ "dependency_proxy_blobs_count": 5,
+ "dependency_proxy_blobs_checksum_total_count": 5,
+ "dependency_proxy_blobs_checksummed_count": 5,
+ "dependency_proxy_blobs_checksum_failed_count": 0,
+ "dependency_proxy_blobs_synced_count": 5,
+ "dependency_proxy_blobs_failed_count": 0,
+ "dependency_proxy_blobs_registry_count": 5,
+ "dependency_proxy_blobs_verification_total_count": 5,
+ "dependency_proxy_blobs_verified_count": 5,
+ "dependency_proxy_blobs_verification_failed_count": 0,
+ "dependency_proxy_blobs_synced_in_percentage": "100.00%",
+ "dependency_proxy_blobs_verified_in_percentage": "100.00%",
+ "dependency_proxy_blobs_synced_missing_on_primary_count": 0,
"container_repositories_count": 5,
"container_repositories_synced_count": 5,
"container_repositories_failed_count": 0,
@@ -675,6 +688,19 @@ Example response:
"job_artifacts_synced_in_percentage": "100.00%",
"job_artifacts_verified_in_percentage": "100.00%",
"job_artifacts_synced_missing_on_primary_count": 0,
+ "dependency_proxy_blobs_count": 5,
+ "dependency_proxy_blobs_checksum_total_count": 5,
+ "dependency_proxy_blobs_checksummed_count": 5,
+ "dependency_proxy_blobs_checksum_failed_count": 0,
+ "dependency_proxy_blobs_synced_count": 5,
+ "dependency_proxy_blobs_failed_count": 0,
+ "dependency_proxy_blobs_registry_count": 5,
+ "dependency_proxy_blobs_verification_total_count": 5,
+ "dependency_proxy_blobs_verified_count": 5,
+ "dependency_proxy_blobs_verification_failed_count": 0,
+ "dependency_proxy_blobs_synced_in_percentage": "100.00%",
+ "dependency_proxy_blobs_verified_in_percentage": "100.00%",
+ "dependency_proxy_blobs_synced_missing_on_primary_count": 0,
"container_repositories_count": 5,
"container_repositories_synced_count": 5,
"container_repositories_failed_count": 0,
@@ -856,6 +882,19 @@ Example response:
"ci_secure_files_synced_in_percentage": "100.00%",
"ci_secure_files_verified_in_percentage": "100.00%",
"ci_secure_files_synced_missing_on_primary_count": 0,
+ "dependency_proxy_blobs_count": 5,
+ "dependency_proxy_blobs_checksum_total_count": 5,
+ "dependency_proxy_blobs_checksummed_count": 5,
+ "dependency_proxy_blobs_checksum_failed_count": 0,
+ "dependency_proxy_blobs_synced_count": 5,
+ "dependency_proxy_blobs_failed_count": 0,
+ "dependency_proxy_blobs_registry_count": 5,
+ "dependency_proxy_blobs_verification_total_count": 5,
+ "dependency_proxy_blobs_verified_count": 5,
+ "dependency_proxy_blobs_verification_failed_count": 0,
+ "dependency_proxy_blobs_synced_in_percentage": "100.00%",
+ "dependency_proxy_blobs_verified_in_percentage": "100.00%",
+ "dependency_proxy_blobs_synced_missing_on_primary_count": 0,
"container_repositories_count": 5,
"container_repositories_synced_count": 5,
"container_repositories_failed_count": 0,
diff --git a/doc/api/graphql/custom_emoji.md b/doc/api/graphql/custom_emoji.md
index ea90b02a069..e2e8bce4290 100644
--- a/doc/api/graphql/custom_emoji.md
+++ b/doc/api/graphql/custom_emoji.md
@@ -6,17 +6,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Use custom emojis with GraphQL **(FREE)**
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37911) in GitLab 13.6
-> - [Deployed behind a feature flag](../../user/feature_flags.md), disabled by default.
-> - Enabled on GitLab.com.
-> - Recommended for production use.
-> - To use in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-custom-emoji-api). **(FREE SELF)**
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37911) in GitLab 13.6 [with a flag](../../administration/feature_flags.md) named `custom_emoji`. Disabled by default.
+> - Enabled on GitLab.com in GitLab 14.0.
-This in-development feature might not be available for your use. There can be
-[risks when enabling features still in development](../../administration/feature_flags.md#risks-when-enabling-features-still-in-development).
-Refer to this feature's version history for more details.
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `custom_emoji`.
+On GitLab.com, this feature is available.
+This feature is ready for production use.
-To use custom emoji in comments and descriptions, you can add them to a group using the GraphQL API.
+To use custom emojis in comments and descriptions, you can add them to a group using the GraphQL API.
Parameters:
@@ -30,15 +28,17 @@ Parameters:
```graphql
mutation CreateCustomEmoji($groupPath: ID!) {
- createCustomEmoji(input: {groupPath: $groupPath, name: "party-parrot", file: "https://cultofthepartyparrot.com/parrots/hd/parrot.gif", external: true}) {
+ createCustomEmoji(input: {groupPath: $groupPath, name: "party-parrot", url: "https://cultofthepartyparrot.com/parrots/hd/parrot.gif"}) {
clientMutationId
- name
+ customEmoji {
+ name
+ }
errors
}
}
```
-After adding custom emoji to the group, members can use it in the same way as other emoji in the comments.
+After adding a custom emoji to the group, members can use it in the same way as other emojis in the comments.
## Get custom emoji for a group
@@ -90,22 +90,3 @@ For more information on:
- GraphQL specific entities, such as Fragments and Interfaces, see the official
[GraphQL documentation](https://graphql.org/learn/).
- Individual attributes, see the [GraphQL API Resources](reference/index.md).
-
-## Enable or disable custom emoji API **(FREE SELF)**
-
-Custom emoji is under development but ready for production use. It is
-deployed behind a feature flag that is **disabled by default**.
-[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md)
-can enable it.
-
-To enable it:
-
-```ruby
-Feature.enable(:custom_emoji)
-```
-
-To disable it:
-
-```ruby
-Feature.disable(:custom_emoji)
-```
diff --git a/doc/api/graphql/getting_started.md b/doc/api/graphql/getting_started.md
index 20fb2f030f2..1945f528d67 100644
--- a/doc/api/graphql/getting_started.md
+++ b/doc/api/graphql/getting_started.md
@@ -18,6 +18,7 @@ The examples documented here can be run using:
- The command line.
- GraphiQL.
+- Rails console.
### Command line
@@ -73,6 +74,27 @@ NOTE:
If you are running GitLab 12.0, enable the `graphql`
[feature flag](../features.md#set-or-create-a-feature).
+### Rails console **(FREE SELF)**
+
+GraphQL queries can be run in a [Rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session). For example, to search projects:
+
+```ruby
+query = <<~EOQ
+query securityGetProjects($search: String!) {
+ projects(search: $search) {
+ nodes {
+ path
+ }
+ }
+}
+EOQ
+
+variables = { "search": "gitlab" }
+
+result = GitlabSchema.execute(query, variables: variables, context: { current_user: current_user })
+result.to_h
+```
+
## Queries and mutations
The GitLab GraphQL API can be used to perform:
diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md
index 40e1ed115a3..4cf296ac1f3 100644
--- a/doc/api/graphql/index.md
+++ b/doc/api/graphql/index.md
@@ -69,12 +69,13 @@ However, GitLab sometimes changes the GraphQL API in a way that is not backward-
can include removing or renaming fields, arguments, or other parts of the schema.
When creating a breaking change, GitLab follows a [deprecation and removal process](#deprecation-and-removal-process).
-Learn more about [breaking changes](../../development/deprecation_guidelines/index.md).
+To avoid having a breaking change affect your integrations, you should
+familiarize yourself with the [deprecation and removal process](#deprecation-and-removal-process) and
+frequently [verify your API calls against the future breaking-change schema](#verify-against-the-future-breaking-change-schema).
Fields behind a feature flag and disabled by default do not follow the deprecation and removal process, and can be removed at any time without notice.
-To avoid having a breaking change affect your integrations, you should
-familiarize yourself with the deprecation and removal process.
+Learn more about [breaking changes](../../development/deprecation_guidelines/index.md).
WARNING:
GitLab makes all attempts to follow the [deprecation and removal process](#deprecation-and-removal-process).
@@ -82,10 +83,22 @@ On rare occasions, GitLab might make immediate breaking changes to the GraphQL
API to patch critical security or performance concerns if the deprecation
process would pose significant risk.
+### Verify against the future breaking-change schema
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353642) in GitLab 15.6.
+
+You can make calls against the GraphQL API as if all deprecated items were already removed.
+This way, you can verify API calls ahead of a [breaking-change release](#deprecation-and-removal-process)
+before the items are actually removed from the schema.
+
+To make these calls, add a
+`remove_deprecated=true` query parameter to the GitLab GraphQL API endpoint (for example,
+`https://gitlab.com/api/graphql?remove_deprecated=true` for GitLab SaaS GraphQL).
+
### Deprecation and removal process
The deprecation and removal process for the GitLab GraphQL API aligns with the wider GitLab
-[deprecation process](https://about.gitlab.com/handbook/product/gitlab-the-product/#breaking-changes-deprecations-and-removing-features).
+[deprecation process](https://about.gitlab.com/handbook/product/gitlab-the-product/#deprecations-removals-and-breaking-changes).
Parts of the schema marked for removal from the GitLab GraphQL API are first
[deprecated](https://about.gitlab.com/handbook/product/gitlab-the-product/#deprecation)
@@ -99,12 +112,13 @@ Items are marked as deprecated in:
- The [deprecation feature removal schedule](../../update/deprecations.md), which is linked from release posts.
- Introspection queries of the GraphQL API.
+The deprecation message provides an alternative for the deprecated schema item,
+if applicable.
+
NOTE:
If you use the GraphQL API, we recommend you remove the deprecated schema from your GraphQL
API calls as soon as possible to avoid experiencing breaking changes.
-
-The deprecation message provides an alternative for the deprecated schema item,
-if applicable.
+You should [verify your API calls against the schema without the deprecated schema items](#verify-against-the-future-breaking-change-schema).
#### Deprecation example
diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md
index 11a60671008..b1f9d6ceae1 100644
--- a/doc/api/graphql/reference/index.md
+++ b/doc/api/graphql/reference/index.md
@@ -214,6 +214,57 @@ Returns [`Issue`](#issue).
| ---- | ---- | ----------- |
| <a id="queryissueid"></a>`id` | [`IssueID!`](#issueid) | Global ID of the issue. |
+### `Query.issues`
+
+Issues visible by the current user. Returns null if the `root_level_issues_query` feature flag is disabled.
+
+WARNING:
+**Introduced** in 15.6.
+This feature is in Alpha. It can be changed or removed at any time.
+
+Returns [`IssueConnection`](#issueconnection).
+
+This field returns a [connection](#connections). It accepts the
+four standard [pagination arguments](#connection-pagination-arguments):
+`before: String`, `after: String`, `first: Int`, `last: Int`.
+
+#### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="queryissuesassigneeid"></a>`assigneeId` | [`String`](#string) | ID of a user assigned to the issues. Wildcard values "NONE" and "ANY" are supported. |
+| <a id="queryissuesassigneeusername"></a>`assigneeUsername` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.11. Use `assigneeUsernames`. |
+| <a id="queryissuesassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. |
+| <a id="queryissuesauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. |
+| <a id="queryissuesclosedafter"></a>`closedAfter` | [`Time`](#time) | Issues closed after this date. |
+| <a id="queryissuesclosedbefore"></a>`closedBefore` | [`Time`](#time) | Issues closed before this date. |
+| <a id="queryissuesconfidential"></a>`confidential` | [`Boolean`](#boolean) | Filter for confidential issues. If "false", excludes confidential issues. If "true", returns only confidential issues. |
+| <a id="queryissuescreatedafter"></a>`createdAfter` | [`Time`](#time) | Issues created after this date. |
+| <a id="queryissuescreatedbefore"></a>`createdBefore` | [`Time`](#time) | Issues created before this date. |
+| <a id="queryissuescrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. |
+| <a id="queryissuescrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. |
+| <a id="queryissuesepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. |
+| <a id="queryissueshealthstatusfilter"></a>`healthStatusFilter` | [`HealthStatusFilter`](#healthstatusfilter) | Health status of the issue, "none" and "any" values are supported. |
+| <a id="queryissuesiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". |
+| <a id="queryissuesiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. |
+| <a id="queryissuesin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. |
+| <a id="queryissuesincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. |
+| <a id="queryissuesiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. |
+| <a id="queryissuesiterationwildcardid"></a>`iterationWildcardId` | [`IterationWildcardId`](#iterationwildcardid) | Filter by iteration ID wildcard. |
+| <a id="queryissueslabelname"></a>`labelName` | [`[String]`](#string) | Labels applied to this issue. |
+| <a id="queryissuesmilestonetitle"></a>`milestoneTitle` | [`[String]`](#string) | Milestone applied to this issue. |
+| <a id="queryissuesmilestonewildcardid"></a>`milestoneWildcardId` | [`MilestoneWildcardId`](#milestonewildcardid) | Filter issues by milestone ID wildcard. |
+| <a id="queryissuesmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. Wildcard values "NONE" and "ANY" are supported. |
+| <a id="queryissuesnot"></a>`not` | [`NegatedIssueFilterInput`](#negatedissuefilterinput) | Negated arguments. |
+| <a id="queryissuesor"></a>`or` | [`UnionedIssueFilterInput`](#unionedissuefilterinput) | List of arguments with inclusive OR. |
+| <a id="queryissuessearch"></a>`search` | [`String`](#string) | Search query for title or description. |
+| <a id="queryissuessort"></a>`sort` | [`IssueSort`](#issuesort) | Sort issues by this criteria. |
+| <a id="queryissuesstate"></a>`state` | [`IssuableState`](#issuablestate) | Current state of this issue. |
+| <a id="queryissuestypes"></a>`types` | [`[IssueType!]`](#issuetype) | Filter issues by the given issue types. |
+| <a id="queryissuesupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Issues updated after this date. |
+| <a id="queryissuesupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Issues updated before this date. |
+| <a id="queryissuesweight"></a>`weight` | [`String`](#string) | Weight applied to the issue, "none" and "any" values are supported. |
+
### `Query.iteration`
Find an iteration.
@@ -595,7 +646,7 @@ Returns [`Vulnerability`](#vulnerability).
### `Query.workItem`
-Find a work item. Returns `null` if `work_items` feature flag is disabled.
+Find a work item.
WARNING:
**Introduced** in 15.1.
@@ -673,6 +724,7 @@ Input type: `AdminSidekiqQueuesDeleteJobsInput`
| <a id="mutationadminsidekiqqueuesdeletejobsrootnamespace"></a>`rootNamespace` | [`String`](#string) | Delete jobs matching root_namespace in the context metadata. |
| <a id="mutationadminsidekiqqueuesdeletejobssubscriptionplan"></a>`subscriptionPlan` | [`String`](#string) | Delete jobs matching subscription_plan in the context metadata. |
| <a id="mutationadminsidekiqqueuesdeletejobsuser"></a>`user` | [`String`](#string) | Delete jobs matching user in the context metadata. |
+| <a id="mutationadminsidekiqqueuesdeletejobsuserid"></a>`userId` | [`String`](#string) | Delete jobs matching user_id in the context metadata. |
| <a id="mutationadminsidekiqqueuesdeletejobsworkerclass"></a>`workerClass` | [`String`](#string) | Delete jobs with the given worker class. |
#### Fields
@@ -778,6 +830,26 @@ Input type: `ArtifactDestroyInput`
| <a id="mutationartifactdestroyclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. |
| <a id="mutationartifactdestroyerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. |
+### `Mutation.auditEventsStreamingDestinationEventsAdd`
+
+Input type: `AuditEventsStreamingDestinationEventsAddInput`
+
+#### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="mutationauditeventsstreamingdestinationeventsaddclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. |
+| <a id="mutationauditeventsstreamingdestinationeventsadddestinationid"></a>`destinationId` | [`AuditEventsExternalAuditEventDestinationID!`](#auditeventsexternalauditeventdestinationid) | Destination id. |
+| <a id="mutationauditeventsstreamingdestinationeventsaddeventtypefilters"></a>`eventTypeFilters` | [`[String!]!`](#string) | List of event type filters to add for streaming. |
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="mutationauditeventsstreamingdestinationeventsaddclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. |
+| <a id="mutationauditeventsstreamingdestinationeventsadderrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. |
+| <a id="mutationauditeventsstreamingdestinationeventsaddeventtypefilters"></a>`eventTypeFilters` | [`[String!]`](#string) | Event type filters present. |
+
### `Mutation.auditEventsStreamingHeadersCreate`
Input type: `AuditEventsStreamingHeadersCreateInput`
@@ -4198,6 +4270,25 @@ Input type: `PipelineScheduleDeleteInput`
| <a id="mutationpipelinescheduledeleteclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. |
| <a id="mutationpipelinescheduledeleteerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. |
+### `Mutation.pipelineScheduleTakeOwnership`
+
+Input type: `PipelineScheduleTakeOwnershipInput`
+
+#### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="mutationpipelinescheduletakeownershipclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. |
+| <a id="mutationpipelinescheduletakeownershipid"></a>`id` | [`CiPipelineScheduleID!`](#cipipelinescheduleid) | ID of the pipeline schedule to mutate. |
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="mutationpipelinescheduletakeownershipclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. |
+| <a id="mutationpipelinescheduletakeownershiperrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. |
+| <a id="mutationpipelinescheduletakeownershippipelineschedule"></a>`pipelineSchedule` | [`PipelineSchedule`](#pipelineschedule) | Updated pipeline schedule ownership. |
+
### `Mutation.projectCiCdSettingsUpdate`
Input type: `ProjectCiCdSettingsUpdateInput`
@@ -4863,6 +4954,10 @@ Input type: `TerraformStateUnlockInput`
### `Mutation.timelineEventCreate`
+WARNING:
+**Introduced** in 15.6.
+This feature is in Alpha. It can be changed or removed at any time.
+
Input type: `TimelineEventCreateInput`
#### Arguments
@@ -4873,6 +4968,7 @@ Input type: `TimelineEventCreateInput`
| <a id="mutationtimelineeventcreateincidentid"></a>`incidentId` | [`IssueID!`](#issueid) | Incident ID of the timeline event. |
| <a id="mutationtimelineeventcreatenote"></a>`note` | [`String!`](#string) | Text note of the timeline event. |
| <a id="mutationtimelineeventcreateoccurredat"></a>`occurredAt` | [`Time!`](#time) | Timestamp of when the event occurred. |
+| <a id="mutationtimelineeventcreatetimelineeventtagnames"></a>`timelineEventTagNames` | [`[String!]`](#string) | Tags for the incident timeline event. |
#### Fields
@@ -4920,6 +5016,26 @@ Input type: `TimelineEventPromoteFromNoteInput`
| <a id="mutationtimelineeventpromotefromnoteerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. |
| <a id="mutationtimelineeventpromotefromnotetimelineevent"></a>`timelineEvent` | [`TimelineEventType`](#timelineeventtype) | Timeline event. |
+### `Mutation.timelineEventTagCreate`
+
+Input type: `TimelineEventTagCreateInput`
+
+#### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="mutationtimelineeventtagcreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. |
+| <a id="mutationtimelineeventtagcreatename"></a>`name` | [`String!`](#string) | Name of the tag. |
+| <a id="mutationtimelineeventtagcreateprojectpath"></a>`projectPath` | [`ID!`](#id) | Project to create the timeline event tag in. |
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="mutationtimelineeventtagcreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. |
+| <a id="mutationtimelineeventtagcreateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. |
+| <a id="mutationtimelineeventtagcreatetimelineeventtag"></a>`timelineEventTag` | [`TimelineEventTagType`](#timelineeventtagtype) | Timeline event tag. |
+
### `Mutation.timelineEventUpdate`
Input type: `TimelineEventUpdateInput`
@@ -5758,7 +5874,7 @@ Input type: `VulnerabilityRevertToDetectedInput`
### `Mutation.workItemCreate`
-Creates a work item. Available only when feature flag `work_items` is enabled.
+Creates a work item.
WARNING:
**Introduced** in 15.1.
@@ -5774,6 +5890,7 @@ Input type: `WorkItemCreateInput`
| <a id="mutationworkitemcreateconfidential"></a>`confidential` | [`Boolean`](#boolean) | Sets the work item confidentiality. |
| <a id="mutationworkitemcreatedescription"></a>`description` | [`String`](#string) | Description of the work item. |
| <a id="mutationworkitemcreatehierarchywidget"></a>`hierarchyWidget` | [`WorkItemWidgetHierarchyCreateInput`](#workitemwidgethierarchycreateinput) | Input for hierarchy widget. |
+| <a id="mutationworkitemcreatemilestonewidget"></a>`milestoneWidget` | [`WorkItemWidgetMilestoneInput`](#workitemwidgetmilestoneinput) | Input for milestone widget. |
| <a id="mutationworkitemcreateprojectpath"></a>`projectPath` | [`ID!`](#id) | Full path of the project the work item is associated with. |
| <a id="mutationworkitemcreatetitle"></a>`title` | [`String!`](#string) | Title of the work item. |
| <a id="mutationworkitemcreateworkitemtypeid"></a>`workItemTypeId` | [`WorkItemsTypeID!`](#workitemstypeid) | Global ID of a work item type. |
@@ -5788,7 +5905,7 @@ Input type: `WorkItemCreateInput`
### `Mutation.workItemCreateFromTask`
-Creates a work item from a task in another work item's description. Available only when feature flag `work_items` is enabled.
+Creates a work item from a task in another work item's description.
WARNING:
**Introduced** in 15.1.
@@ -5815,7 +5932,7 @@ Input type: `WorkItemCreateFromTaskInput`
### `Mutation.workItemDelete`
-Deletes a work item. Available only when feature flag `work_items` is enabled.
+Deletes a work item.
WARNING:
**Introduced** in 15.1.
@@ -5840,7 +5957,7 @@ Input type: `WorkItemDeleteInput`
### `Mutation.workItemDeleteTask`
-Deletes a task in a work item's description. Available only when feature flag `work_items` is enabled.
+Deletes a task in a work item's description.
WARNING:
**Introduced** in 15.1.
@@ -5867,7 +5984,7 @@ Input type: `WorkItemDeleteTaskInput`
### `Mutation.workItemUpdate`
-Updates a work item by Global ID. Available only when feature flag `work_items` is enabled.
+Updates a work item by Global ID.
WARNING:
**Introduced** in 15.1.
@@ -5887,6 +6004,7 @@ Input type: `WorkItemUpdateInput`
| <a id="mutationworkitemupdateid"></a>`id` | [`WorkItemID!`](#workitemid) | Global ID of the work item. |
| <a id="mutationworkitemupdateiterationwidget"></a>`iterationWidget` | [`WorkItemWidgetIterationInput`](#workitemwidgetiterationinput) | Input for iteration widget. |
| <a id="mutationworkitemupdatelabelswidget"></a>`labelsWidget` | [`WorkItemWidgetLabelsUpdateInput`](#workitemwidgetlabelsupdateinput) | Input for labels widget. |
+| <a id="mutationworkitemupdatemilestonewidget"></a>`milestoneWidget` | [`WorkItemWidgetMilestoneInput`](#workitemwidgetmilestoneinput) | Input for milestone widget. |
| <a id="mutationworkitemupdatestartandduedatewidget"></a>`startAndDueDateWidget` | [`WorkItemWidgetStartAndDueDateUpdateInput`](#workitemwidgetstartandduedateupdateinput) | Input for start and due date widget. |
| <a id="mutationworkitemupdatestateevent"></a>`stateEvent` | [`WorkItemStateEvent`](#workitemstateevent) | Close or reopen a work item. |
| <a id="mutationworkitemupdatestatuswidget"></a>`statusWidget` | [`StatusInput`](#statusinput) | Input for status widget. |
@@ -5903,7 +6021,7 @@ Input type: `WorkItemUpdateInput`
### `Mutation.workItemUpdateTask`
-Updates a work item's task by Global ID. Available only when feature flag `work_items` is enabled.
+Updates a work item's task by Global ID.
WARNING:
**Introduced** in 15.1.
@@ -6934,6 +7052,29 @@ The edge type for [`ContainerRepositoryTag`](#containerrepositorytag).
| <a id="containerrepositorytagedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
| <a id="containerrepositorytagedgenode"></a>`node` | [`ContainerRepositoryTag`](#containerrepositorytag) | The item at the end of the edge. |
+#### `ContributionAnalyticsContributionConnection`
+
+The connection type for [`ContributionAnalyticsContribution`](#contributionanalyticscontribution).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="contributionanalyticscontributionconnectionedges"></a>`edges` | [`[ContributionAnalyticsContributionEdge]`](#contributionanalyticscontributionedge) | A list of edges. |
+| <a id="contributionanalyticscontributionconnectionnodes"></a>`nodes` | [`[ContributionAnalyticsContribution]`](#contributionanalyticscontribution) | A list of nodes. |
+| <a id="contributionanalyticscontributionconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. |
+
+#### `ContributionAnalyticsContributionEdge`
+
+The edge type for [`ContributionAnalyticsContribution`](#contributionanalyticscontribution).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="contributionanalyticscontributionedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
+| <a id="contributionanalyticscontributionedgenode"></a>`node` | [`ContributionAnalyticsContribution`](#contributionanalyticscontribution) | The item at the end of the edge. |
+
#### `CoverageFuzzingCorpusConnection`
The connection type for [`CoverageFuzzingCorpus`](#coveragefuzzingcorpus).
@@ -7142,6 +7283,29 @@ The edge type for [`DependencyProxyBlob`](#dependencyproxyblob).
| <a id="dependencyproxyblobedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
| <a id="dependencyproxyblobedgenode"></a>`node` | [`DependencyProxyBlob`](#dependencyproxyblob) | The item at the end of the edge. |
+#### `DependencyProxyBlobRegistryConnection`
+
+The connection type for [`DependencyProxyBlobRegistry`](#dependencyproxyblobregistry).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="dependencyproxyblobregistryconnectionedges"></a>`edges` | [`[DependencyProxyBlobRegistryEdge]`](#dependencyproxyblobregistryedge) | A list of edges. |
+| <a id="dependencyproxyblobregistryconnectionnodes"></a>`nodes` | [`[DependencyProxyBlobRegistry]`](#dependencyproxyblobregistry) | A list of nodes. |
+| <a id="dependencyproxyblobregistryconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. |
+
+#### `DependencyProxyBlobRegistryEdge`
+
+The edge type for [`DependencyProxyBlobRegistry`](#dependencyproxyblobregistry).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="dependencyproxyblobregistryedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
+| <a id="dependencyproxyblobregistryedgenode"></a>`node` | [`DependencyProxyBlobRegistry`](#dependencyproxyblobregistry) | The item at the end of the edge. |
+
#### `DependencyProxyManifestConnection`
The connection type for [`DependencyProxyManifest`](#dependencyproxymanifest).
@@ -7513,6 +7677,29 @@ The edge type for [`ExternalAuditEventDestination`](#externalauditeventdestinati
| <a id="externalauditeventdestinationedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
| <a id="externalauditeventdestinationedgenode"></a>`node` | [`ExternalAuditEventDestination`](#externalauditeventdestination) | The item at the end of the edge. |
+#### `ExternalStatusCheckConnection`
+
+The connection type for [`ExternalStatusCheck`](#externalstatuscheck).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="externalstatuscheckconnectionedges"></a>`edges` | [`[ExternalStatusCheckEdge]`](#externalstatuscheckedge) | A list of edges. |
+| <a id="externalstatuscheckconnectionnodes"></a>`nodes` | [`[ExternalStatusCheck]`](#externalstatuscheck) | A list of nodes. |
+| <a id="externalstatuscheckconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. |
+
+#### `ExternalStatusCheckEdge`
+
+The edge type for [`ExternalStatusCheck`](#externalstatuscheck).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="externalstatuscheckedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
+| <a id="externalstatuscheckedgenode"></a>`node` | [`ExternalStatusCheck`](#externalstatuscheck) | The item at the end of the edge. |
+
#### `GroupConnection`
The connection type for [`Group`](#group).
@@ -8488,6 +8675,52 @@ The edge type for [`PipelineSecurityReportFinding`](#pipelinesecurityreportfindi
| <a id="pipelinesecurityreportfindingedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
| <a id="pipelinesecurityreportfindingedgenode"></a>`node` | [`PipelineSecurityReportFinding`](#pipelinesecurityreportfinding) | The item at the end of the edge. |
+#### `ProductAnalyticsDashboardConnection`
+
+The connection type for [`ProductAnalyticsDashboard`](#productanalyticsdashboard).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="productanalyticsdashboardconnectionedges"></a>`edges` | [`[ProductAnalyticsDashboardEdge]`](#productanalyticsdashboardedge) | A list of edges. |
+| <a id="productanalyticsdashboardconnectionnodes"></a>`nodes` | [`[ProductAnalyticsDashboard]`](#productanalyticsdashboard) | A list of nodes. |
+| <a id="productanalyticsdashboardconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. |
+
+#### `ProductAnalyticsDashboardEdge`
+
+The edge type for [`ProductAnalyticsDashboard`](#productanalyticsdashboard).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="productanalyticsdashboardedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
+| <a id="productanalyticsdashboardedgenode"></a>`node` | [`ProductAnalyticsDashboard`](#productanalyticsdashboard) | The item at the end of the edge. |
+
+#### `ProductAnalyticsDashboardWidgetConnection`
+
+The connection type for [`ProductAnalyticsDashboardWidget`](#productanalyticsdashboardwidget).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="productanalyticsdashboardwidgetconnectionedges"></a>`edges` | [`[ProductAnalyticsDashboardWidgetEdge]`](#productanalyticsdashboardwidgetedge) | A list of edges. |
+| <a id="productanalyticsdashboardwidgetconnectionnodes"></a>`nodes` | [`[ProductAnalyticsDashboardWidget]`](#productanalyticsdashboardwidget) | A list of nodes. |
+| <a id="productanalyticsdashboardwidgetconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. |
+
+#### `ProductAnalyticsDashboardWidgetEdge`
+
+The edge type for [`ProductAnalyticsDashboardWidget`](#productanalyticsdashboardwidget).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="productanalyticsdashboardwidgetedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
+| <a id="productanalyticsdashboardwidgetedgenode"></a>`node` | [`ProductAnalyticsDashboardWidget`](#productanalyticsdashboardwidget) | The item at the end of the edge. |
+
#### `ProjectConnection`
The connection type for [`Project`](#project).
@@ -9299,6 +9532,29 @@ The edge type for [`TimeTrackingTimelogCategory`](#timetrackingtimelogcategory).
| <a id="timetrackingtimelogcategoryedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
| <a id="timetrackingtimelogcategoryedgenode"></a>`node` | [`TimeTrackingTimelogCategory`](#timetrackingtimelogcategory) | The item at the end of the edge. |
+#### `TimelineEventTagTypeConnection`
+
+The connection type for [`TimelineEventTagType`](#timelineeventtagtype).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="timelineeventtagtypeconnectionedges"></a>`edges` | [`[TimelineEventTagTypeEdge]`](#timelineeventtagtypeedge) | A list of edges. |
+| <a id="timelineeventtagtypeconnectionnodes"></a>`nodes` | [`[TimelineEventTagType]`](#timelineeventtagtype) | A list of nodes. |
+| <a id="timelineeventtagtypeconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. |
+
+#### `TimelineEventTagTypeEdge`
+
+The edge type for [`TimelineEventTagType`](#timelineeventtagtype).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="timelineeventtagtypeedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
+| <a id="timelineeventtagtypeedgenode"></a>`node` | [`TimelineEventTagType`](#timelineeventtagtype) | The item at the end of the edge. |
+
#### `TimelineEventTypeConnection`
The connection type for [`TimelineEventType`](#timelineeventtype).
@@ -9437,6 +9693,29 @@ The edge type for [`TreeEntry`](#treeentry).
| <a id="treeentryedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
| <a id="treeentryedgenode"></a>`node` | [`TreeEntry`](#treeentry) | The item at the end of the edge. |
+#### `UnprotectAccessLevelConnection`
+
+The connection type for [`UnprotectAccessLevel`](#unprotectaccesslevel).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="unprotectaccesslevelconnectionedges"></a>`edges` | [`[UnprotectAccessLevelEdge]`](#unprotectaccessleveledge) | A list of edges. |
+| <a id="unprotectaccesslevelconnectionnodes"></a>`nodes` | [`[UnprotectAccessLevel]`](#unprotectaccesslevel) | A list of nodes. |
+| <a id="unprotectaccesslevelconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. |
+
+#### `UnprotectAccessLevelEdge`
+
+The edge type for [`UnprotectAccessLevel`](#unprotectaccesslevel).
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="unprotectaccessleveledgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
+| <a id="unprotectaccessleveledgenode"></a>`node` | [`UnprotectAccessLevel`](#unprotectaccesslevel) | The item at the end of the edge. |
+
#### `UploadRegistryConnection`
The connection type for [`UploadRegistry`](#uploadregistry).
@@ -9737,6 +10016,36 @@ Represents the access level of a relationship between a User and object that it
| <a id="accesslevelintegervalue"></a>`integerValue` | [`Int`](#int) | Integer representation of access level. |
| <a id="accesslevelstringvalue"></a>`stringValue` | [`AccessLevelEnum`](#accesslevelenum) | String representation of access level. |
+### `AccessLevelGroup`
+
+Representation of a GitLab group.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="accesslevelgroupavatarurl"></a>`avatarUrl` | [`String`](#string) | Avatar URL of the group. |
+| <a id="accesslevelgroupid"></a>`id` | [`ID!`](#id) | ID of the group. |
+| <a id="accesslevelgroupname"></a>`name` | [`String!`](#string) | Name of the group. |
+| <a id="accesslevelgroupparent"></a>`parent` | [`AccessLevelGroup`](#accesslevelgroup) | Parent group. |
+| <a id="accesslevelgroupweburl"></a>`webUrl` | [`String!`](#string) | Web URL of the group. |
+
+### `AccessLevelUser`
+
+Representation of a GitLab user.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="accessleveluseravatarurl"></a>`avatarUrl` | [`String`](#string) | URL of the user's avatar. |
+| <a id="accessleveluserid"></a>`id` | [`ID!`](#id) | ID of the user. |
+| <a id="accesslevelusername"></a>`name` | [`String!`](#string) | Human-readable name of the user. Returns `****` if the user is a project bot and the requester does not have permission to view the project. |
+| <a id="accessleveluserpublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. |
+| <a id="accessleveluserusername"></a>`username` | [`String!`](#string) | Username of the user. Unique within this instance of GitLab. |
+| <a id="accessleveluserwebpath"></a>`webPath` | [`String!`](#string) | Web path of the user. |
+| <a id="accessleveluserweburl"></a>`webUrl` | [`String!`](#string) | Web URL of the user. |
+
### `AgentConfiguration`
Configuration details for an Agent.
@@ -10218,7 +10527,7 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="boardepicchildreniidstartswith"></a>`iidStartsWith` | [`String`](#string) | Filter epics by IID for autocomplete. |
| <a id="boardepicchildreniids"></a>`iids` | [`[ID!]`](#id) | List of IIDs of epics, e.g., `[1, 2]`. |
| <a id="boardepicchildrenin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. |
-| <a id="boardepicchildrenincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include epics from ancestor groups. |
+| <a id="boardepicchildrenincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include child epics from ancestor groups. |
| <a id="boardepicchildrenincludedescendantgroups"></a>`includeDescendantGroups` | [`Boolean`](#boolean) | Include epics from descendant groups. |
| <a id="boardepicchildrenlabelname"></a>`labelName` | [`[String!]`](#string) | Filter epics by labels. |
| <a id="boardepicchildrenmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Filter epics by milestone title, computed from epic's issues. |
@@ -10333,6 +10642,7 @@ Branch protection details for a branch rule.
| <a id="branchprotectioncodeownerapprovalrequired"></a>`codeOwnerApprovalRequired` | [`Boolean!`](#boolean) | Enforce code owner approvals before allowing a merge. |
| <a id="branchprotectionmergeaccesslevels"></a>`mergeAccessLevels` | [`MergeAccessLevelConnection`](#mergeaccesslevelconnection) | Details about who can merge when this branch is the source branch. (see [Connections](#connections)) |
| <a id="branchprotectionpushaccesslevels"></a>`pushAccessLevels` | [`PushAccessLevelConnection`](#pushaccesslevelconnection) | Details about who can push when this branch is the source branch. (see [Connections](#connections)) |
+| <a id="branchprotectionunprotectaccesslevels"></a>`unprotectAccessLevels` | [`UnprotectAccessLevelConnection`](#unprotectaccesslevelconnection) | Details about who can unprotect this branch. (see [Connections](#connections)) |
### `BranchRule`
@@ -10345,7 +10655,9 @@ List of branch rules for a project, grouped by branch name.
| <a id="branchruleapprovalrules"></a>`approvalRules` | [`ApprovalProjectRuleConnection`](#approvalprojectruleconnection) | Merge request approval rules configured for this branch rule. (see [Connections](#connections)) |
| <a id="branchrulebranchprotection"></a>`branchProtection` | [`BranchProtection!`](#branchprotection) | Branch protections configured for this branch rule. |
| <a id="branchrulecreatedat"></a>`createdAt` | [`Time!`](#time) | Timestamp of when the branch rule was created. |
+| <a id="branchruleexternalstatuschecks"></a>`externalStatusChecks` | [`ExternalStatusCheckConnection`](#externalstatuscheckconnection) | External status checks configured for this branch rule. (see [Connections](#connections)) |
| <a id="branchruleisdefault"></a>`isDefault` | [`Boolean!`](#boolean) | Check if this branch rule protects the project's default branch. |
+| <a id="branchrulematchingbranchescount"></a>`matchingBranchesCount` | [`Int!`](#int) | Number of existing branches that match this branch rule. |
| <a id="branchrulename"></a>`name` | [`String!`](#string) | Branch name, with wildcards, for the branch rules. |
| <a id="branchruleupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp of when the branch rule was last updated. |
@@ -10622,7 +10934,8 @@ CI/CD variables given to a manual job.
| Name | Type | Description |
| ---- | ---- | ----------- |
| <a id="ciminutesprojectmonthlyusageminutes"></a>`minutes` | [`Int`](#int) | Number of CI/CD minutes used by the project in the month. |
-| <a id="ciminutesprojectmonthlyusagename"></a>`name` | [`String`](#string) | Name of the project. |
+| <a id="ciminutesprojectmonthlyusagename"></a>`name` **{warning-solid}** | [`String`](#string) | **Deprecated** in 15.6. Use `project.name`. |
+| <a id="ciminutesprojectmonthlyusageproject"></a>`project` | [`Project`](#project) | Project having the recorded usage. |
| <a id="ciminutesprojectmonthlyusagesharedrunnersduration"></a>`sharedRunnersDuration` | [`Int`](#int) | Total duration (in seconds) of shared runners use by the project for the month. |
### `CiProjectVariable`
@@ -10874,6 +11187,7 @@ Represents a code quality degradation on the pipeline.
| <a id="codequalitydegradationline"></a>`line` | [`Int!`](#int) | Line on which the code quality degradation occurred. |
| <a id="codequalitydegradationpath"></a>`path` | [`String!`](#string) | Relative path to the file containing the code quality degradation. |
| <a id="codequalitydegradationseverity"></a>`severity` | [`CodeQualityDegradationSeverity!`](#codequalitydegradationseverity) | Status of the degradation (BLOCKER, CRITICAL, MAJOR, MINOR, INFO, UNKNOWN). |
+| <a id="codequalitydegradationweburl"></a>`webUrl` | [`String`](#string) | URL to the file along with line number. |
### `Commit`
@@ -10894,6 +11208,7 @@ Represents a code quality degradation on the pipeline.
| <a id="commitmessage"></a>`message` | [`String`](#string) | Raw commit message. |
| <a id="commitsha"></a>`sha` | [`String!`](#string) | SHA1 ID of the commit. |
| <a id="commitshortid"></a>`shortId` | [`String!`](#string) | Short SHA1 ID of the commit. |
+| <a id="commitsignature"></a>`signature` | [`CommitSignature`](#commitsignature) | Signature of the commit. |
| <a id="commitsignaturehtml"></a>`signatureHtml` | [`String`](#string) | Rendered HTML of the commit signature. |
| <a id="committitle"></a>`title` | [`String`](#string) | Title of the commit message. |
| <a id="committitlehtml"></a>`titleHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `title`. |
@@ -10934,6 +11249,7 @@ Represents a ComplianceFramework associated with a Project.
| Name | Type | Description |
| ---- | ---- | ----------- |
| <a id="complianceframeworkcolor"></a>`color` | [`String!`](#string) | Hexadecimal representation of compliance framework's label color. |
+| <a id="complianceframeworkdefault"></a>`default` | [`Boolean`](#boolean) | Default compliance framework for the group. |
| <a id="complianceframeworkdescription"></a>`description` | [`String!`](#string) | Description of the compliance framework. |
| <a id="complianceframeworkid"></a>`id` | [`ID!`](#id) | Compliance framework ID. |
| <a id="complianceframeworkname"></a>`name` | [`String!`](#string) | Name of the compliance framework. |
@@ -11141,6 +11457,24 @@ A tag from a container repository.
| <a id="containerrepositorytagshortrevision"></a>`shortRevision` | [`String`](#string) | Short revision of the tag. |
| <a id="containerrepositorytagtotalsize"></a>`totalSize` | [`BigInt`](#bigint) | Size of the tag. |
+### `ContributionAnalyticsContribution`
+
+Represents the contributions of a user.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="contributionanalyticscontributionissuesclosed"></a>`issuesClosed` | [`Int`](#int) | Number of issues closed by the user. |
+| <a id="contributionanalyticscontributionissuescreated"></a>`issuesCreated` | [`Int`](#int) | Number of issues created by the user. |
+| <a id="contributionanalyticscontributionmergerequestsapproved"></a>`mergeRequestsApproved` | [`Int`](#int) | Number of merge requests approved by the user. |
+| <a id="contributionanalyticscontributionmergerequestsclosed"></a>`mergeRequestsClosed` | [`Int`](#int) | Number of merge requests closed by the user. |
+| <a id="contributionanalyticscontributionmergerequestscreated"></a>`mergeRequestsCreated` | [`Int`](#int) | Number of merge requests created by the user. |
+| <a id="contributionanalyticscontributionmergerequestsmerged"></a>`mergeRequestsMerged` | [`Int`](#int) | Number of merge requests merged by the user. |
+| <a id="contributionanalyticscontributionrepopushed"></a>`repoPushed` | [`Int`](#int) | Number of repository pushes the user made. |
+| <a id="contributionanalyticscontributiontotalevents"></a>`totalEvents` | [`Int`](#int) | Total number of events contributed by the user. |
+| <a id="contributionanalyticscontributionuser"></a>`user` | [`UserCore`](#usercore) | Contributor User object. |
+
### `CoverageFuzzingCorpus`
Corpus for a coverage fuzzing job.
@@ -11315,6 +11649,7 @@ Represents a DAST Site Profile.
| <a id="dastsiteprofiletargettype"></a>`targetType` | [`DastTargetTypeEnum`](#dasttargettypeenum) | Type of target to be scanned. |
| <a id="dastsiteprofiletargeturl"></a>`targetUrl` | [`String`](#string) | URL of the target to be scanned. |
| <a id="dastsiteprofileuserpermissions"></a>`userPermissions` | [`DastSiteProfilePermissions!`](#dastsiteprofilepermissions) | Permissions for the current user on the resource. |
+| <a id="dastsiteprofilevalidationstartedat"></a>`validationStartedAt` | [`Time`](#time) | Site profile validation start time. |
| <a id="dastsiteprofilevalidationstatus"></a>`validationStatus` | [`DastSiteProfileValidationStatusEnum`](#dastsiteprofilevalidationstatusenum) | Current validation status of the site profile. |
### `DastSiteProfileAuth`
@@ -11381,6 +11716,25 @@ Dependency proxy blob.
| <a id="dependencyproxyblobsize"></a>`size` | [`String!`](#string) | Size of the blob file. |
| <a id="dependencyproxyblobupdatedat"></a>`updatedAt` | [`Time!`](#time) | Date of most recent update. |
+### `DependencyProxyBlobRegistry`
+
+Represents the Geo replication and verification state of a dependency_proxy_blob.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="dependencyproxyblobregistrycreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp when the DependencyProxyBlobRegistry was created. |
+| <a id="dependencyproxyblobregistrydependencyproxyblobid"></a>`dependencyProxyBlobId` | [`ID!`](#id) | ID of the Dependency Proxy Blob. |
+| <a id="dependencyproxyblobregistryid"></a>`id` | [`ID!`](#id) | ID of the DependencyProxyBlobRegistry. |
+| <a id="dependencyproxyblobregistrylastsyncfailure"></a>`lastSyncFailure` | [`String`](#string) | Error message during sync of the DependencyProxyBlobRegistry. |
+| <a id="dependencyproxyblobregistrylastsyncedat"></a>`lastSyncedAt` | [`Time`](#time) | Timestamp of the most recent successful sync of the DependencyProxyBlobRegistry. |
+| <a id="dependencyproxyblobregistryretryat"></a>`retryAt` | [`Time`](#time) | Timestamp after which the DependencyProxyBlobRegistry is resynced. |
+| <a id="dependencyproxyblobregistryretrycount"></a>`retryCount` | [`Int`](#int) | Number of consecutive failed sync attempts of the DependencyProxyBlobRegistry. |
+| <a id="dependencyproxyblobregistrystate"></a>`state` | [`RegistryState`](#registrystate) | Sync state of the DependencyProxyBlobRegistry. |
+| <a id="dependencyproxyblobregistryverificationretryat"></a>`verificationRetryAt` | [`Time`](#time) | Timestamp after which the DependencyProxyBlobRegistry is reverified. |
+| <a id="dependencyproxyblobregistryverifiedat"></a>`verifiedAt` | [`Time`](#time) | Timestamp of the most recent successful verification of the DependencyProxyBlobRegistry. |
+
### `DependencyProxyImageTtlGroupPolicy`
Group-level Dependency Proxy TTL policy settings.
@@ -11442,6 +11796,33 @@ The deployment of an environment.
| <a id="deploymenttriggerer"></a>`triggerer` | [`UserCore`](#usercore) | User who executed the deployment. |
| <a id="deploymentupdatedat"></a>`updatedAt` | [`Time`](#time) | When the deployment record was updated. |
+### `DeploymentApproval`
+
+Approval of the deployment.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="deploymentapprovalcomment"></a>`comment` | [`String`](#string) | Additional comment. |
+| <a id="deploymentapprovalcreatedat"></a>`createdAt` | [`Time`](#time) | When the user approved/rejected first time. |
+| <a id="deploymentapprovalstatus"></a>`status` | [`DeploymentsApprovalStatus`](#deploymentsapprovalstatus) | Whether the deployment was approved/rejected. |
+| <a id="deploymentapprovalupdatedat"></a>`updatedAt` | [`Time`](#time) | When the user updated the approval. |
+| <a id="deploymentapprovaluser"></a>`user` | [`UserCore`](#usercore) | User who approved or rejected the deployment. |
+
+### `DeploymentApprovalSummary`
+
+Approval summary of the deployment.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="deploymentapprovalsummaryrules"></a>`rules` | [`[ProtectedEnvironmentApprovalRuleForSummary!]`](#protectedenvironmentapprovalruleforsummary) | Approval Rules for the deployment. |
+| <a id="deploymentapprovalsummarystatus"></a>`status` | [`DeploymentApprovalSummaryStatus`](#deploymentapprovalsummarystatus) | Status of the approvals. |
+| <a id="deploymentapprovalsummarytotalpendingapprovalcount"></a>`totalPendingApprovalCount` | [`Int`](#int) | Total pending approval count. |
+| <a id="deploymentapprovalsummarytotalrequiredapprovals"></a>`totalRequiredApprovals` | [`Int`](#int) | Total number of required approvals. |
+
### `DeploymentDetails`
The details of the deployment.
@@ -11450,6 +11831,7 @@ The details of the deployment.
| Name | Type | Description |
| ---- | ---- | ----------- |
+| <a id="deploymentdetailsapprovalsummary"></a>`approvalSummary` | [`DeploymentApprovalSummary`](#deploymentapprovalsummary) | Approval summary of the deployment. |
| <a id="deploymentdetailscommit"></a>`commit` | [`Commit`](#commit) | Commit details of the deployment. |
| <a id="deploymentdetailscreatedat"></a>`createdAt` | [`Time`](#time) | When the deployment record was created. |
| <a id="deploymentdetailsfinishedat"></a>`finishedAt` | [`Time`](#time) | When the deployment finished. |
@@ -12082,7 +12464,7 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="epicchildreniidstartswith"></a>`iidStartsWith` | [`String`](#string) | Filter epics by IID for autocomplete. |
| <a id="epicchildreniids"></a>`iids` | [`[ID!]`](#id) | List of IIDs of epics, e.g., `[1, 2]`. |
| <a id="epicchildrenin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. |
-| <a id="epicchildrenincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include epics from ancestor groups. |
+| <a id="epicchildrenincludeancestorgroups"></a>`includeAncestorGroups` | [`Boolean`](#boolean) | Include child epics from ancestor groups. |
| <a id="epicchildrenincludedescendantgroups"></a>`includeDescendantGroups` | [`Boolean`](#boolean) | Include epics from descendant groups. |
| <a id="epicchildrenlabelname"></a>`labelName` | [`[String!]`](#string) | Filter epics by labels. |
| <a id="epicchildrenmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Filter epics by milestone title, computed from epic's issues. |
@@ -12204,7 +12586,7 @@ Relationship between an epic and an issue.
| Name | Type | Description |
| ---- | ---- | ----------- |
-| <a id="epicissuealertmanagementalert"></a>`alertManagementAlert` | [`AlertManagementAlert`](#alertmanagementalert) | Alert associated to this issue. |
+| <a id="epicissuealertmanagementalert"></a>`alertManagementAlert` **{warning-solid}** | [`AlertManagementAlert`](#alertmanagementalert) | **Deprecated** in 15.6. Use `alert_management_alerts`. |
| <a id="epicissueassignees"></a>`assignees` | [`UserCoreConnection`](#usercoreconnection) | Assignees of the issue. (see [Connections](#connections)) |
| <a id="epicissueauthor"></a>`author` | [`UserCore!`](#usercore) | User that created the issue. |
| <a id="epicissueblocked"></a>`blocked` | [`Boolean!`](#boolean) | Indicates the issue is blocked. |
@@ -12272,6 +12654,27 @@ Relationship between an epic and an issue.
#### Fields with arguments
+##### `EpicIssue.alertManagementAlerts`
+
+Alert Management alerts associated to this issue.
+
+Returns [`AlertManagementAlertConnection`](#alertmanagementalertconnection).
+
+This field returns a [connection](#connections). It accepts the
+four standard [pagination arguments](#connection-pagination-arguments):
+`before: String`, `after: String`, `first: Int`, `last: Int`.
+
+###### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="epicissuealertmanagementalertsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of a user assigned to the issue. |
+| <a id="epicissuealertmanagementalertsdomain"></a>`domain` | [`AlertManagementDomainFilter!`](#alertmanagementdomainfilter) | Filter query for given domain. |
+| <a id="epicissuealertmanagementalertsiid"></a>`iid` | [`String`](#string) | IID of the alert. For example, "1". |
+| <a id="epicissuealertmanagementalertssearch"></a>`search` | [`String`](#string) | Search query for title, description, service, or monitoring_tool. |
+| <a id="epicissuealertmanagementalertssort"></a>`sort` | [`AlertManagementAlertSort`](#alertmanagementalertsort) | Sort alerts by this criteria. |
+| <a id="epicissuealertmanagementalertsstatuses"></a>`statuses` | [`[AlertManagementStatus!]`](#alertmanagementstatus) | Alerts with the specified statues. For example, `[TRIGGERED]`. |
+
##### `EpicIssue.currentUserTodos`
To-do items for the current user.
@@ -12429,6 +12832,7 @@ Represents an external resource to send audit events to.
| Name | Type | Description |
| ---- | ---- | ----------- |
| <a id="externalauditeventdestinationdestinationurl"></a>`destinationUrl` | [`String!`](#string) | External destination to send audit events to. |
+| <a id="externalauditeventdestinationeventtypefilters"></a>`eventTypeFilters` | [`[String!]!`](#string) | List of event type filters added for streaming. |
| <a id="externalauditeventdestinationgroup"></a>`group` | [`Group!`](#group) | Group the destination belongs to. |
| <a id="externalauditeventdestinationheaders"></a>`headers` | [`AuditEventStreamingHeaderConnection!`](#auditeventstreamingheaderconnection) | List of additional HTTP headers sent with each event. (see [Connections](#connections)) |
| <a id="externalauditeventdestinationid"></a>`id` | [`ID!`](#id) | ID of the destination. |
@@ -12450,6 +12854,18 @@ Represents an external issue.
| <a id="externalissueupdatedat"></a>`updatedAt` | [`Time`](#time) | Timestamp of when the issue was updated. |
| <a id="externalissueweburl"></a>`webUrl` | [`String`](#string) | URL to the issue in the external tracker. |
+### `ExternalStatusCheck`
+
+Describes an external status check.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="externalstatuscheckexternalurl"></a>`externalUrl` | [`String!`](#string) | External URL for the status check. |
+| <a id="externalstatuscheckid"></a>`id` | [`GlobalID!`](#globalid) | ID of the rule. |
+| <a id="externalstatuscheckname"></a>`name` | [`String!`](#string) | Name of the rule. |
+
### `FileUpload`
#### Fields
@@ -12524,6 +12940,28 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="geonodecontainerrepositoryregistriesreplicationstate"></a>`replicationState` | [`ReplicationStateEnum`](#replicationstateenum) | Filters registries by their replication state. |
| <a id="geonodecontainerrepositoryregistriesverificationstate"></a>`verificationState` | [`VerificationStateEnum`](#verificationstateenum) | Filters registries by their verification state. |
+##### `GeoNode.dependencyProxyBlobRegistries`
+
+Find Dependency Proxy Blob registries on this Geo node. Ignored if `geo_dependency_proxy_blob_replication` feature flag is disabled.
+
+WARNING:
+**Introduced** in 15.6.
+This feature is in Alpha. It can be changed or removed at any time.
+
+Returns [`DependencyProxyBlobRegistryConnection`](#dependencyproxyblobregistryconnection).
+
+This field returns a [connection](#connections). It accepts the
+four standard [pagination arguments](#connection-pagination-arguments):
+`before: String`, `after: String`, `first: Int`, `last: Int`.
+
+###### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="geonodedependencyproxyblobregistriesids"></a>`ids` | [`[ID!]`](#id) | Filters registries by their ID. |
+| <a id="geonodedependencyproxyblobregistriesreplicationstate"></a>`replicationState` | [`ReplicationStateEnum`](#replicationstateenum) | Filters registries by their replication state. |
+| <a id="geonodedependencyproxyblobregistriesverificationstate"></a>`verificationState` | [`VerificationStateEnum`](#verificationstateenum) | Filters registries by their verification state. |
+
##### `GeoNode.groupWikiRepositoryRegistries`
Find group wiki repository registries on this Geo node.
@@ -12704,6 +13142,22 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="geonodeuploadregistriesreplicationstate"></a>`replicationState` | [`ReplicationStateEnum`](#replicationstateenum) | Filters registries by their replication state. |
| <a id="geonodeuploadregistriesverificationstate"></a>`verificationState` | [`VerificationStateEnum`](#verificationstateenum) | Filters registries by their verification state. |
+### `GpgSignature`
+
+GPG signature for a signed commit.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="gpgsignaturecommitsha"></a>`commitSha` | [`String`](#string) | SHA of the associated commit. |
+| <a id="gpgsignaturegpgkeyprimarykeyid"></a>`gpgKeyPrimaryKeyid` | [`String`](#string) | ID of the GPG key. |
+| <a id="gpgsignaturegpgkeyuseremail"></a>`gpgKeyUserEmail` | [`String`](#string) | User email associated with the GPG key. |
+| <a id="gpgsignaturegpgkeyusername"></a>`gpgKeyUserName` | [`String`](#string) | User name associated with the GPG key. |
+| <a id="gpgsignatureproject"></a>`project` | [`Project`](#project) | Project of the associated commit. |
+| <a id="gpgsignatureuser"></a>`user` | [`UserCore`](#usercore) | User associated with the key. |
+| <a id="gpgsignatureverificationstatus"></a>`verificationStatus` | [`VerificationStatus`](#verificationstatus) | Indicates verification status of the associated key or certificate. |
+
### `GrafanaIntegration`
#### Fields
@@ -12918,6 +13372,23 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="groupcontainerrepositoriesname"></a>`name` | [`String`](#string) | Filter the container repositories by their name. |
| <a id="groupcontainerrepositoriessort"></a>`sort` | [`ContainerRepositorySort`](#containerrepositorysort) | Sort container repositories by this criteria. |
+##### `Group.contributions`
+
+Provides the aggregated contributions by users within the group and its subgroups.
+
+Returns [`ContributionAnalyticsContributionConnection`](#contributionanalyticscontributionconnection).
+
+This field returns a [connection](#connections). It accepts the
+four standard [pagination arguments](#connection-pagination-arguments):
+`before: String`, `after: String`, `first: Int`, `last: Int`.
+
+###### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="groupcontributionsfrom"></a>`from` | [`ISO8601Date!`](#iso8601date) | Start date of the reporting time range. |
+| <a id="groupcontributionsto"></a>`to` | [`ISO8601Date!`](#iso8601date) | End date of the reporting time range. The end date must be within 31 days after the start date. |
+
##### `Group.descendantGroups`
List of descendant groups of this group.
@@ -13095,6 +13566,7 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="groupissuesmilestonewildcardid"></a>`milestoneWildcardId` | [`MilestoneWildcardId`](#milestonewildcardid) | Filter issues by milestone ID wildcard. |
| <a id="groupissuesmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. Wildcard values "NONE" and "ANY" are supported. |
| <a id="groupissuesnot"></a>`not` | [`NegatedIssueFilterInput`](#negatedissuefilterinput) | Negated arguments. |
+| <a id="groupissuesor"></a>`or` | [`UnionedIssueFilterInput`](#unionedissuefilterinput) | List of arguments with inclusive OR. |
| <a id="groupissuessearch"></a>`search` | [`String`](#string) | Search query for title or description. |
| <a id="groupissuessort"></a>`sort` | [`IssueSort`](#issuesort) | Sort issues by this criteria. |
| <a id="groupissuesstate"></a>`state` | [`IssuableState`](#issuablestate) | Current state of this issue. |
@@ -13370,9 +13842,25 @@ four standard [pagination arguments](#connection-pagination-arguments):
| Name | Type | Description |
| ---- | ---- | ----------- |
-| <a id="groupscanexecutionpoliciesactionscantypes"></a>`actionScanTypes` | [`[SecurityReportTypeEnum!]`](#securityreporttypeenum) | Filters policies by the action scan type. Only these scan types are supported: `dast`, `secret_detection`, `cluster_image_scanning`, `container_scanning`, `sast`. |
+| <a id="groupscanexecutionpoliciesactionscantypes"></a>`actionScanTypes` | [`[SecurityReportTypeEnum!]`](#securityreporttypeenum) | Filters policies by the action scan type. Only these scan types are supported: `dast`, `secret_detection`, `cluster_image_scanning`, `container_scanning`, `sast`, `dependency_scanning`. |
| <a id="groupscanexecutionpoliciesrelationship"></a>`relationship` | [`SecurityPolicyRelationType`](#securitypolicyrelationtype) | Filter policies by the given policy relationship. |
+##### `Group.scanResultPolicies`
+
+Scan Result Policies of the project.
+
+Returns [`ScanResultPolicyConnection`](#scanresultpolicyconnection).
+
+This field returns a [connection](#connections). It accepts the
+four standard [pagination arguments](#connection-pagination-arguments):
+`before: String`, `after: String`, `first: Int`, `last: Int`.
+
+###### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="groupscanresultpoliciesrelationship"></a>`relationship` | [`SecurityPolicyRelationType`](#securitypolicyrelationtype) | Filter policies by the given policy relationship. |
+
##### `Group.timelogs`
Time logged on issues and merge requests in the group and its subgroups.
@@ -13474,7 +13962,7 @@ Returns [`VulnerabilitySeveritiesCount`](#vulnerabilityseveritiescount).
##### `Group.workItemTypes`
-Work item types available to the group. Returns `null` if `work_items` feature flag is disabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice.
+Work item types available to the group.
Returns [`WorkItemTypeConnection`](#workitemtypeconnection).
@@ -13755,7 +14243,7 @@ Describes an issuable resource link for incident issues.
| Name | Type | Description |
| ---- | ---- | ----------- |
-| <a id="issuealertmanagementalert"></a>`alertManagementAlert` | [`AlertManagementAlert`](#alertmanagementalert) | Alert associated to this issue. |
+| <a id="issuealertmanagementalert"></a>`alertManagementAlert` **{warning-solid}** | [`AlertManagementAlert`](#alertmanagementalert) | **Deprecated** in 15.6. Use `alert_management_alerts`. |
| <a id="issueassignees"></a>`assignees` | [`UserCoreConnection`](#usercoreconnection) | Assignees of the issue. (see [Connections](#connections)) |
| <a id="issueauthor"></a>`author` | [`UserCore!`](#usercore) | User that created the issue. |
| <a id="issueblocked"></a>`blocked` | [`Boolean!`](#boolean) | Indicates the issue is blocked. |
@@ -13821,6 +14309,27 @@ Describes an issuable resource link for incident issues.
#### Fields with arguments
+##### `Issue.alertManagementAlerts`
+
+Alert Management alerts associated to this issue.
+
+Returns [`AlertManagementAlertConnection`](#alertmanagementalertconnection).
+
+This field returns a [connection](#connections). It accepts the
+four standard [pagination arguments](#connection-pagination-arguments):
+`before: String`, `after: String`, `first: Int`, `last: Int`.
+
+###### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="issuealertmanagementalertsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of a user assigned to the issue. |
+| <a id="issuealertmanagementalertsdomain"></a>`domain` | [`AlertManagementDomainFilter!`](#alertmanagementdomainfilter) | Filter query for given domain. |
+| <a id="issuealertmanagementalertsiid"></a>`iid` | [`String`](#string) | IID of the alert. For example, "1". |
+| <a id="issuealertmanagementalertssearch"></a>`search` | [`String`](#string) | Search query for title, description, service, or monitoring_tool. |
+| <a id="issuealertmanagementalertssort"></a>`sort` | [`AlertManagementAlertSort`](#alertmanagementalertsort) | Sort alerts by this criteria. |
+| <a id="issuealertmanagementalertsstatuses"></a>`statuses` | [`[AlertManagementStatus!]`](#alertmanagementstatus) | Alerts with the specified statues. For example, `[TRIGGERED]`. |
+
##### `Issue.currentUserTodos`
To-do items for the current user.
@@ -14128,7 +14637,7 @@ Maven metadata.
### `MergeAccessLevel`
-Represents the merge access level of a branch protection.
+Defines which user roles, users, or groups can merge into a protected branch.
#### Fields
@@ -14136,8 +14645,8 @@ Represents the merge access level of a branch protection.
| ---- | ---- | ----------- |
| <a id="mergeaccesslevelaccesslevel"></a>`accessLevel` | [`Int!`](#int) | GitLab::Access level. |
| <a id="mergeaccesslevelaccessleveldescription"></a>`accessLevelDescription` | [`String!`](#string) | Human readable representation for this access level. |
-| <a id="mergeaccesslevelgroup"></a>`group` | [`Group`](#group) | Group associated with this access level. |
-| <a id="mergeaccessleveluser"></a>`user` | [`UserCore`](#usercore) | User associated with this access level. |
+| <a id="mergeaccesslevelgroup"></a>`group` | [`AccessLevelGroup`](#accesslevelgroup) | Group associated with this access level. |
+| <a id="mergeaccessleveluser"></a>`user` | [`AccessLevelUser`](#accessleveluser) | User associated with this access level. |
### `MergeRequest`
@@ -14166,7 +14675,7 @@ Represents the merge access level of a branch protection.
| <a id="mergerequestdefaultsquashcommitmessage"></a>`defaultSquashCommitMessage` | [`String`](#string) | Default squash commit message of the merge request. |
| <a id="mergerequestdescription"></a>`description` | [`String`](#string) | Description of the merge request (Markdown rendered as HTML for caching). |
| <a id="mergerequestdescriptionhtml"></a>`descriptionHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `description`. |
-| <a id="mergerequestdetailedmergestatus"></a>`detailedMergeStatus` **{warning-solid}** | [`DetailedMergeStatus`](#detailedmergestatus) | **Introduced** in 15.3. This feature is in Alpha. It can be changed or removed at any time. Detailed merge status of the merge request. |
+| <a id="mergerequestdetailedmergestatus"></a>`detailedMergeStatus` | [`DetailedMergeStatus`](#detailedmergestatus) | Detailed merge status of the merge request. |
| <a id="mergerequestdiffheadsha"></a>`diffHeadSha` | [`String`](#string) | Diff head SHA of the merge request. |
| <a id="mergerequestdiffrefs"></a>`diffRefs` | [`DiffRefs`](#diffrefs) | References of the base SHA, the head SHA, and the start SHA for this merge request. |
| <a id="mergerequestdiffstatssummary"></a>`diffStatsSummary` | [`DiffStatsSummary`](#diffstatssummary) | Summary of which files were changed in this merge request. |
@@ -15274,6 +15783,7 @@ four standard [pagination arguments](#connection-pagination-arguments):
| Name | Type | Description |
| ---- | ---- | ----------- |
+| <a id="metadataenterprise"></a>`enterprise` | [`Boolean!`](#boolean) | Enterprise edition. |
| <a id="metadatakas"></a>`kas` | [`Kas!`](#kas) | Metadata about KAS. |
| <a id="metadatarevision"></a>`revision` | [`String!`](#string) | Revision. |
| <a id="metadataversion"></a>`version` | [`String!`](#string) | Version. |
@@ -15466,9 +15976,25 @@ four standard [pagination arguments](#connection-pagination-arguments):
| Name | Type | Description |
| ---- | ---- | ----------- |
-| <a id="namespacescanexecutionpoliciesactionscantypes"></a>`actionScanTypes` | [`[SecurityReportTypeEnum!]`](#securityreporttypeenum) | Filters policies by the action scan type. Only these scan types are supported: `dast`, `secret_detection`, `cluster_image_scanning`, `container_scanning`, `sast`. |
+| <a id="namespacescanexecutionpoliciesactionscantypes"></a>`actionScanTypes` | [`[SecurityReportTypeEnum!]`](#securityreporttypeenum) | Filters policies by the action scan type. Only these scan types are supported: `dast`, `secret_detection`, `cluster_image_scanning`, `container_scanning`, `sast`, `dependency_scanning`. |
| <a id="namespacescanexecutionpoliciesrelationship"></a>`relationship` | [`SecurityPolicyRelationType`](#securitypolicyrelationtype) | Filter policies by the given policy relationship. |
+##### `Namespace.scanResultPolicies`
+
+Scan Result Policies of the project.
+
+Returns [`ScanResultPolicyConnection`](#scanresultpolicyconnection).
+
+This field returns a [connection](#connections). It accepts the
+four standard [pagination arguments](#connection-pagination-arguments):
+`before: String`, `after: String`, `first: Int`, `last: Int`.
+
+###### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="namespacescanresultpoliciesrelationship"></a>`relationship` | [`SecurityPolicyRelationType`](#securitypolicyrelationtype) | Filter policies by the given policy relationship. |
+
### `NamespaceBan`
#### Fields
@@ -15612,6 +16138,7 @@ Represents a package with pipelines in the Package Registry.
| Name | Type | Description |
| ---- | ---- | ----------- |
+| <a id="package_links"></a>`_links` | [`PackageLinks!`](#packagelinks) | Map of links to perform actions on the package. |
| <a id="packagecandestroy"></a>`canDestroy` | [`Boolean!`](#boolean) | Whether the user can destroy the package. |
| <a id="packagecreatedat"></a>`createdAt` | [`Time!`](#time) | Date of creation. |
| <a id="packageid"></a>`id` | [`PackagesPackageID!`](#packagespackageid) | ID of the package. |
@@ -15633,6 +16160,7 @@ Represents a package in the Package Registry.
| Name | Type | Description |
| ---- | ---- | ----------- |
+| <a id="packagebase_links"></a>`_links` | [`PackageLinks!`](#packagelinks) | Map of links to perform actions on the package. |
| <a id="packagebasecandestroy"></a>`canDestroy` | [`Boolean!`](#boolean) | Whether the user can destroy the package. |
| <a id="packagebasecreatedat"></a>`createdAt` | [`Time!`](#time) | Date of creation. |
| <a id="packagebaseid"></a>`id` | [`PackagesPackageID!`](#packagespackageid) | ID of the package. |
@@ -15691,6 +16219,7 @@ Represents a package details in the Package Registry.
| Name | Type | Description |
| ---- | ---- | ----------- |
+| <a id="packagedetailstype_links"></a>`_links` | [`PackageLinks!`](#packagelinks) | Map of links to perform actions on the package. |
| <a id="packagedetailstypecandestroy"></a>`canDestroy` | [`Boolean!`](#boolean) | Whether the user can destroy the package. |
| <a id="packagedetailstypecomposerconfigrepositoryurl"></a>`composerConfigRepositoryUrl` | [`String`](#string) | Url of the Composer setup endpoint. |
| <a id="packagedetailstypecomposerurl"></a>`composerUrl` | [`String`](#string) | Url of the Composer endpoint. |
@@ -15809,6 +16338,16 @@ Represents the contents of a Helm Chart.yml file.
| <a id="packagehelmmetadatatypetype"></a>`type` | [`String`](#string) | Type of the chart. |
| <a id="packagehelmmetadatatypeversion"></a>`version` | [`String!`](#string) | Version of the chart. |
+### `PackageLinks`
+
+Represents links to perform actions on the package.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="packagelinkswebpath"></a>`webPath` | [`String`](#string) | Path to the package details page. |
+
### `PackageSettings`
Namespace-level Package Registry settings.
@@ -15978,6 +16517,18 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="pipelinejobssecurityreporttypes"></a>`securityReportTypes` | [`[SecurityReportTypeEnum!]`](#securityreporttypeenum) | Filter jobs by the type of security report they produce. |
| <a id="pipelinejobsstatuses"></a>`statuses` | [`[CiJobStatus!]`](#cijobstatus) | Filter jobs by status. |
+##### `Pipeline.securityReportFinding`
+
+Vulnerability finding reported on the pipeline.
+
+Returns [`PipelineSecurityReportFinding`](#pipelinesecurityreportfinding).
+
+###### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="pipelinesecurityreportfindinguuid"></a>`uuid` | [`String!`](#string) | UUID of the security report finding. |
+
##### `Pipeline.securityReportFindings`
Vulnerability findings reported on the pipeline.
@@ -16133,6 +16684,7 @@ Represents vulnerability finding of a security report on the pipeline.
| <a id="pipelinesecurityreportfindingconfidence"></a>`confidence` **{warning-solid}** | [`String`](#string) | **Deprecated** in 15.4. This field will be removed from the Finding domain model. |
| <a id="pipelinesecurityreportfindingdescription"></a>`description` | [`String`](#string) | Description of the vulnerability finding. |
| <a id="pipelinesecurityreportfindingdescriptionhtml"></a>`descriptionHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `description`. |
+| <a id="pipelinesecurityreportfindingdetails"></a>`details` | [`[VulnerabilityDetail!]!`](#vulnerabilitydetail) | Details of the security finding. |
| <a id="pipelinesecurityreportfindingevidence"></a>`evidence` | [`VulnerabilityEvidence`](#vulnerabilityevidence) | Evidence for the vulnerability. |
| <a id="pipelinesecurityreportfindingfalsepositive"></a>`falsePositive` | [`Boolean`](#boolean) | Indicates whether the vulnerability is a false positive. |
| <a id="pipelinesecurityreportfindingidentifiers"></a>`identifiers` | [`[VulnerabilityIdentifier!]!`](#vulnerabilityidentifier) | Identifiers of the vulnerability finding. |
@@ -16159,6 +16711,29 @@ Represents vulnerability finding of a security report on the pipeline.
| <a id="previewbillableuserchangenewbillableusercount"></a>`newBillableUserCount` | [`Int`](#int) | Total number of billable users after change. |
| <a id="previewbillableuserchangeseatsinsubscription"></a>`seatsInSubscription` | [`Int`](#int) | Number of seats in subscription. |
+### `ProductAnalyticsDashboard`
+
+Represents a product analytics dashboard.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="productanalyticsdashboarddescription"></a>`description` | [`String`](#string) | Description of the dashboard. |
+| <a id="productanalyticsdashboardtitle"></a>`title` | [`String!`](#string) | Title of the dashboard. |
+| <a id="productanalyticsdashboardwidgets"></a>`widgets` | [`ProductAnalyticsDashboardWidgetConnection!`](#productanalyticsdashboardwidgetconnection) | Widgets shown on the dashboard. (see [Connections](#connections)) |
+
+### `ProductAnalyticsDashboardWidget`
+
+Represents a product analytics dashboard widget.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="productanalyticsdashboardwidgetgridattributes"></a>`gridAttributes` | [`JSON`](#json) | Description of the position and size of the widget. |
+| <a id="productanalyticsdashboardwidgettitle"></a>`title` | [`String!`](#string) | Title of the widget. |
+
### `Project`
#### Fields
@@ -16196,10 +16771,12 @@ Represents vulnerability finding of a security report on the pipeline.
| <a id="projecthttpurltorepo"></a>`httpUrlToRepo` | [`String`](#string) | URL to connect to the project via HTTPS. |
| <a id="projectid"></a>`id` | [`ID!`](#id) | ID of the project. |
| <a id="projectimportstatus"></a>`importStatus` | [`String`](#string) | Status of import background job of the project. |
+| <a id="projectincidentmanagementtimelineeventtags"></a>`incidentManagementTimelineEventTags` | [`[TimelineEventTagType!]`](#timelineeventtagtype) | Timeline event tags for the project. |
| <a id="projectissuesenabled"></a>`issuesEnabled` | [`Boolean`](#boolean) | Indicates if Issues are enabled for the current user. |
| <a id="projectjiraimportstatus"></a>`jiraImportStatus` | [`String`](#string) | Status of Jira import background job of the project. |
| <a id="projectjiraimports"></a>`jiraImports` | [`JiraImportConnection`](#jiraimportconnection) | Jira imports into the project. (see [Connections](#connections)) |
| <a id="projectjobsenabled"></a>`jobsEnabled` | [`Boolean`](#boolean) | Indicates if CI/CD pipeline jobs are enabled for the current user. |
+| <a id="projectlanguages"></a>`languages` | [`[RepositoryLanguage!]`](#repositorylanguage) | Programming languages used in the project. |
| <a id="projectlastactivityat"></a>`lastActivityAt` | [`Time`](#time) | Timestamp of the project last activity. |
| <a id="projectlfsenabled"></a>`lfsEnabled` | [`Boolean`](#boolean) | Indicates if the project has Large File Storage (LFS) enabled. |
| <a id="projectmergecommittemplate"></a>`mergeCommitTemplate` | [`String`](#string) | Template used to create merge commit message in merge requests. |
@@ -16226,7 +16803,6 @@ Represents vulnerability finding of a security report on the pipeline.
| <a id="projectrequestaccessenabled"></a>`requestAccessEnabled` | [`Boolean`](#boolean) | Indicates if users can request member access to the project. |
| <a id="projectrequirementstatescount"></a>`requirementStatesCount` | [`RequirementStatesCount`](#requirementstatescount) | Number of requirements for the project by their state. |
| <a id="projectsastciconfiguration"></a>`sastCiConfiguration` | [`SastCiConfiguration`](#sastciconfiguration) | SAST CI configuration for the project. |
-| <a id="projectscanresultpolicies"></a>`scanResultPolicies` | [`ScanResultPolicyConnection`](#scanresultpolicyconnection) | Scan Result Policies of the project. (see [Connections](#connections)) |
| <a id="projectsecuritydashboardpath"></a>`securityDashboardPath` | [`String`](#string) | Path to project's security dashboard. |
| <a id="projectsecurityscanners"></a>`securityScanners` | [`SecurityScanners`](#securityscanners) | Information about security analyzers used in the project. |
| <a id="projectsentryerrors"></a>`sentryErrors` | [`SentryErrorCollection`](#sentryerrorcollection) | Paginated collection of Sentry errors on the project. |
@@ -16692,6 +17268,7 @@ Returns [`Issue`](#issue).
| <a id="projectissuemilestonewildcardid"></a>`milestoneWildcardId` | [`MilestoneWildcardId`](#milestonewildcardid) | Filter issues by milestone ID wildcard. |
| <a id="projectissuemyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. Wildcard values "NONE" and "ANY" are supported. |
| <a id="projectissuenot"></a>`not` | [`NegatedIssueFilterInput`](#negatedissuefilterinput) | Negated arguments. |
+| <a id="projectissueor"></a>`or` | [`UnionedIssueFilterInput`](#unionedissuefilterinput) | List of arguments with inclusive OR. |
| <a id="projectissuereleasetag"></a>`releaseTag` | [`[String!]`](#string) | Release tag associated with the issue's milestone. |
| <a id="projectissuereleasetagwildcardid"></a>`releaseTagWildcardId` | [`ReleaseTagWildcardId`](#releasetagwildcardid) | Filter issues by release tag ID wildcard. |
| <a id="projectissuesearch"></a>`search` | [`String`](#string) | Search query for title or description. |
@@ -16723,20 +17300,27 @@ Returns [`IssueStatusCountsType`](#issuestatuscountstype).
| <a id="projectissuestatuscountscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Issues created before this date. |
| <a id="projectissuestatuscountscrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. |
| <a id="projectissuestatuscountscrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. |
+| <a id="projectissuestatuscountsepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. |
+| <a id="projectissuestatuscountshealthstatusfilter"></a>`healthStatusFilter` | [`HealthStatusFilter`](#healthstatusfilter) | Health status of the issue, "none" and "any" values are supported. |
| <a id="projectissuestatuscountsiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". |
| <a id="projectissuestatuscountsiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. |
| <a id="projectissuestatuscountsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. |
+| <a id="projectissuestatuscountsincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. |
+| <a id="projectissuestatuscountsiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. |
+| <a id="projectissuestatuscountsiterationwildcardid"></a>`iterationWildcardId` | [`IterationWildcardId`](#iterationwildcardid) | Filter by iteration ID wildcard. |
| <a id="projectissuestatuscountslabelname"></a>`labelName` | [`[String]`](#string) | Labels applied to this issue. |
| <a id="projectissuestatuscountsmilestonetitle"></a>`milestoneTitle` | [`[String]`](#string) | Milestone applied to this issue. |
| <a id="projectissuestatuscountsmilestonewildcardid"></a>`milestoneWildcardId` | [`MilestoneWildcardId`](#milestonewildcardid) | Filter issues by milestone ID wildcard. |
| <a id="projectissuestatuscountsmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. Wildcard values "NONE" and "ANY" are supported. |
| <a id="projectissuestatuscountsnot"></a>`not` | [`NegatedIssueFilterInput`](#negatedissuefilterinput) | Negated arguments. |
+| <a id="projectissuestatuscountsor"></a>`or` | [`UnionedIssueFilterInput`](#unionedissuefilterinput) | List of arguments with inclusive OR. |
| <a id="projectissuestatuscountsreleasetag"></a>`releaseTag` | [`[String!]`](#string) | Release tag associated with the issue's milestone. |
| <a id="projectissuestatuscountsreleasetagwildcardid"></a>`releaseTagWildcardId` | [`ReleaseTagWildcardId`](#releasetagwildcardid) | Filter issues by release tag ID wildcard. |
| <a id="projectissuestatuscountssearch"></a>`search` | [`String`](#string) | Search query for title or description. |
| <a id="projectissuestatuscountstypes"></a>`types` | [`[IssueType!]`](#issuetype) | Filter issues by the given issue types. |
| <a id="projectissuestatuscountsupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Issues updated after this date. |
| <a id="projectissuestatuscountsupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Issues updated before this date. |
+| <a id="projectissuestatuscountsweight"></a>`weight` | [`String`](#string) | Weight applied to the issue, "none" and "any" values are supported. |
##### `Project.issues`
@@ -16777,6 +17361,7 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="projectissuesmilestonewildcardid"></a>`milestoneWildcardId` | [`MilestoneWildcardId`](#milestonewildcardid) | Filter issues by milestone ID wildcard. |
| <a id="projectissuesmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. Wildcard values "NONE" and "ANY" are supported. |
| <a id="projectissuesnot"></a>`not` | [`NegatedIssueFilterInput`](#negatedissuefilterinput) | Negated arguments. |
+| <a id="projectissuesor"></a>`or` | [`UnionedIssueFilterInput`](#unionedissuefilterinput) | List of arguments with inclusive OR. |
| <a id="projectissuesreleasetag"></a>`releaseTag` | [`[String!]`](#string) | Release tag associated with the issue's milestone. |
| <a id="projectissuesreleasetagwildcardid"></a>`releaseTagWildcardId` | [`ReleaseTagWildcardId`](#releasetagwildcardid) | Filter issues by release tag ID wildcard. |
| <a id="projectissuessearch"></a>`search` | [`String`](#string) | Search query for title or description. |
@@ -17068,6 +17653,26 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="projectpipelinesupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Pipelines updated before this date. |
| <a id="projectpipelinesusername"></a>`username` | [`String`](#string) | Filter pipelines by the user that triggered the pipeline. |
+##### `Project.productAnalyticsDashboards`
+
+Product Analytics dashboards of the project.
+
+WARNING:
+**Introduced** in 15.6.
+This feature is in Alpha. It can be changed or removed at any time.
+
+Returns [`ProductAnalyticsDashboardConnection`](#productanalyticsdashboardconnection).
+
+This field returns a [connection](#connections). It accepts the
+four standard [pagination arguments](#connection-pagination-arguments):
+`before: String`, `after: String`, `first: Int`, `last: Int`.
+
+###### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="projectproductanalyticsdashboardsslug"></a>`slug` | [`String`](#string) | Find by dashboard slug. |
+
##### `Project.projectMembers`
Members of the project.
@@ -17168,9 +17773,25 @@ four standard [pagination arguments](#connection-pagination-arguments):
| Name | Type | Description |
| ---- | ---- | ----------- |
-| <a id="projectscanexecutionpoliciesactionscantypes"></a>`actionScanTypes` | [`[SecurityReportTypeEnum!]`](#securityreporttypeenum) | Filters policies by the action scan type. Only these scan types are supported: `dast`, `secret_detection`, `cluster_image_scanning`, `container_scanning`, `sast`. |
+| <a id="projectscanexecutionpoliciesactionscantypes"></a>`actionScanTypes` | [`[SecurityReportTypeEnum!]`](#securityreporttypeenum) | Filters policies by the action scan type. Only these scan types are supported: `dast`, `secret_detection`, `cluster_image_scanning`, `container_scanning`, `sast`, `dependency_scanning`. |
| <a id="projectscanexecutionpoliciesrelationship"></a>`relationship` | [`SecurityPolicyRelationType`](#securitypolicyrelationtype) | Filter policies by the given policy relationship. |
+##### `Project.scanResultPolicies`
+
+Scan Result Policies of the project.
+
+Returns [`ScanResultPolicyConnection`](#scanresultpolicyconnection).
+
+This field returns a [connection](#connections). It accepts the
+four standard [pagination arguments](#connection-pagination-arguments):
+`before: String`, `after: String`, `first: Int`, `last: Int`.
+
+###### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="projectscanresultpoliciesrelationship"></a>`relationship` | [`SecurityPolicyRelationType`](#securitypolicyrelationtype) | Filter policies by the given policy relationship. |
+
##### `Project.securityTrainingProviders`
List of security training providers for the project.
@@ -17343,7 +17964,7 @@ Returns [`VulnerabilitySeveritiesCount`](#vulnerabilityseveritiescount).
##### `Project.workItemTypes`
-Work item types available to the project. Returns `null` if `work_items` feature flag is disabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice.
+Work item types available to the project.
Returns [`WorkItemTypeConnection`](#workitemtypeconnection).
@@ -17381,6 +18002,7 @@ four standard [pagination arguments](#connection-pagination-arguments):
| <a id="projectworkitemssearch"></a>`search` | [`String`](#string) | Search query for title or description. |
| <a id="projectworkitemssort"></a>`sort` | [`WorkItemSort`](#workitemsort) | Sort work items by this criteria. |
| <a id="projectworkitemsstate"></a>`state` | [`IssuableState`](#issuablestate) | Current state of this work item. |
+| <a id="projectworkitemsstatuswidget"></a>`statusWidget` | [`StatusFilterInput`](#statusfilterinput) | Input for status widget filter. Ignored if `work_items_mvc_2` is disabled. |
| <a id="projectworkitemstypes"></a>`types` | [`[IssueType!]`](#issuetype) | Filter work items by the given work item types. |
### `ProjectCiCdSetting`
@@ -17558,6 +18180,23 @@ Which group, user or role is allowed to approve deployments to the environment.
| <a id="protectedenvironmentapprovalrulerequiredapprovals"></a>`requiredApprovals` | [`Int`](#int) | Number of required approvals. |
| <a id="protectedenvironmentapprovalruleuser"></a>`user` | [`UserCore`](#usercore) | User details. Present if it's user specific access control. |
+### `ProtectedEnvironmentApprovalRuleForSummary`
+
+Which group, user or role is allowed to approve deployments to the environment.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="protectedenvironmentapprovalruleforsummaryaccesslevel"></a>`accessLevel` | [`AccessLevel`](#accesslevel) | Role details. Present if it's role specific access control. |
+| <a id="protectedenvironmentapprovalruleforsummaryapprovals"></a>`approvals` | [`[DeploymentApproval!]`](#deploymentapproval) | Current approvals of the deployment. |
+| <a id="protectedenvironmentapprovalruleforsummaryapprovedcount"></a>`approvedCount` | [`Int`](#int) | Approved count. |
+| <a id="protectedenvironmentapprovalruleforsummarygroup"></a>`group` | [`Group`](#group) | Group details. Present if it's group specific access control. |
+| <a id="protectedenvironmentapprovalruleforsummarypendingapprovalcount"></a>`pendingApprovalCount` | [`Int`](#int) | Pending approval count. |
+| <a id="protectedenvironmentapprovalruleforsummaryrequiredapprovals"></a>`requiredApprovals` | [`Int`](#int) | Number of required approvals. |
+| <a id="protectedenvironmentapprovalruleforsummarystatus"></a>`status` | [`DeploymentApprovalSummaryStatus`](#deploymentapprovalsummarystatus) | Status of the approval summary. |
+| <a id="protectedenvironmentapprovalruleforsummaryuser"></a>`user` | [`UserCore`](#usercore) | User details. Present if it's user specific access control. |
+
### `ProtectedEnvironmentDeployAccessLevel`
Which group, user or role is allowed to execute deployments to the environment.
@@ -17572,7 +18211,7 @@ Which group, user or role is allowed to execute deployments to the environment.
### `PushAccessLevel`
-Represents the push access level of a branch protection.
+Defines which user roles, users, or groups can push to a protected branch.
#### Fields
@@ -17580,8 +18219,8 @@ Represents the push access level of a branch protection.
| ---- | ---- | ----------- |
| <a id="pushaccesslevelaccesslevel"></a>`accessLevel` | [`Int!`](#int) | GitLab::Access level. |
| <a id="pushaccesslevelaccessleveldescription"></a>`accessLevelDescription` | [`String!`](#string) | Human readable representation for this access level. |
-| <a id="pushaccesslevelgroup"></a>`group` | [`Group`](#group) | Group associated with this access level. |
-| <a id="pushaccessleveluser"></a>`user` | [`UserCore`](#usercore) | User associated with this access level. |
+| <a id="pushaccesslevelgroup"></a>`group` | [`AccessLevelGroup`](#accesslevelgroup) | Group associated with this access level. |
+| <a id="pushaccessleveluser"></a>`user` | [`AccessLevelUser`](#accessleveluser) | User associated with this access level. |
### `PushRules`
@@ -17838,6 +18477,16 @@ Returns [`Tree`](#tree).
| <a id="repositoryblobstoredexternally"></a>`storedExternally` | [`Boolean`](#boolean) | Whether the blob's content is stored externally (for instance, in LFS). |
| <a id="repositoryblobwebpath"></a>`webPath` | [`String`](#string) | Web path of the blob. |
+### `RepositoryLanguage`
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="repositorylanguagecolor"></a>`color` | [`Color`](#color) | Color to visualize the repository language. |
+| <a id="repositorylanguagename"></a>`name` | [`String!`](#string) | Name of the repository language. |
+| <a id="repositorylanguageshare"></a>`share` | [`Float`](#float) | Percentage of the repository's languages. |
+
### `Requirement`
Represents a requirement.
@@ -17937,6 +18586,7 @@ Counts of requirements by their state.
| Name | Type | Description |
| ---- | ---- | ----------- |
+| <a id="runnerpermissionsassignrunner"></a>`assignRunner` | [`Boolean!`](#boolean) | Indicates the user can perform `assign_runner` on this resource. |
| <a id="runnerpermissionsdeleterunner"></a>`deleteRunner` | [`Boolean!`](#boolean) | Indicates the user can perform `delete_runner` on this resource. |
| <a id="runnerpermissionsreadrunner"></a>`readRunner` | [`Boolean!`](#boolean) | Indicates the user can perform `read_runner` on this resource. |
| <a id="runnerpermissionsupdaterunner"></a>`updateRunner` | [`Boolean!`](#boolean) | Indicates the user can perform `update_runner` on this resource. |
@@ -18064,6 +18714,7 @@ Represents the scan result policy.
| <a id="scanresultpolicyenabled"></a>`enabled` | [`Boolean!`](#boolean) | Indicates whether this policy is enabled. |
| <a id="scanresultpolicygroupapprovers"></a>`groupApprovers` | [`[Group!]`](#group) | Approvers of the group type. |
| <a id="scanresultpolicyname"></a>`name` | [`String!`](#string) | Name of the policy. |
+| <a id="scanresultpolicysource"></a>`source` | [`SecurityPolicySource!`](#securitypolicysource) | Source of the policy. Its fields depend on the source type. |
| <a id="scanresultpolicyupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp of when the policy YAML was last updated. |
| <a id="scanresultpolicyuserapprovers"></a>`userApprovers` | [`[UserCore!]`](#usercore) | Approvers of the user type. |
| <a id="scanresultpolicyyaml"></a>`yaml` | [`String!`](#string) | YAML definition of the policy. |
@@ -18695,6 +19346,17 @@ Explains why we could not generate a timebox report.
| <a id="timeboxreporterrorcode"></a>`code` | [`TimeboxReportErrorReason`](#timeboxreporterrorreason) | Machine readable code, categorizing the error. |
| <a id="timeboxreporterrormessage"></a>`message` | [`String`](#string) | Human readable message explaining what happened. |
+### `TimelineEventTagType`
+
+Describes a tag on an incident management timeline event.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="timelineeventtagtypeid"></a>`id` | [`IncidentManagementTimelineEventTagID!`](#incidentmanagementtimelineeventtagid) | ID of the timeline event tag. |
+| <a id="timelineeventtagtypename"></a>`name` | [`String!`](#string) | Name of the timeline event tag. |
+
### `TimelineEventType`
Describes an incident management timeline event.
@@ -18713,6 +19375,7 @@ Describes an incident management timeline event.
| <a id="timelineeventtypenotehtml"></a>`noteHtml` | [`String`](#string) | HTML note of the timeline event. |
| <a id="timelineeventtypeoccurredat"></a>`occurredAt` | [`Time!`](#time) | Timestamp when the event occurred. |
| <a id="timelineeventtypepromotedfromnote"></a>`promotedFromNote` | [`Note`](#note) | Note from which the timeline event was created. |
+| <a id="timelineeventtypetimelineeventtags"></a>`timelineEventTags` | [`TimelineEventTagTypeConnection`](#timelineeventtagtypeconnection) | Tags for the incident timeline event. (see [Connections](#connections)) |
| <a id="timelineeventtypeupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp when the event updated. |
| <a id="timelineeventtypeupdatedbyuser"></a>`updatedByUser` | [`UserCore`](#usercore) | User that updated the timeline event. |
@@ -18801,6 +19464,19 @@ Represents a directory.
| <a id="treeentrywebpath"></a>`webPath` | [`String`](#string) | Web path for the tree entry (directory). |
| <a id="treeentryweburl"></a>`webUrl` | [`String`](#string) | Web URL for the tree entry (directory). |
+### `UnprotectAccessLevel`
+
+Defines which user roles, users, or groups can unprotect a protected branch.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="unprotectaccesslevelaccesslevel"></a>`accessLevel` | [`Int!`](#int) | GitLab::Access level. |
+| <a id="unprotectaccesslevelaccessleveldescription"></a>`accessLevelDescription` | [`String!`](#string) | Human readable representation for this access level. |
+| <a id="unprotectaccesslevelgroup"></a>`group` | [`AccessLevelGroup`](#accesslevelgroup) | Group associated with this access level. |
+| <a id="unprotectaccessleveluser"></a>`user` | [`AccessLevelUser`](#accessleveluser) | User associated with this access level. |
+
### `UploadRegistry`
Represents the Geo replication and verification state of an upload.
@@ -19171,7 +19847,8 @@ Represents a vulnerability.
| <a id="vulnerabilitytitle"></a>`title` | [`String`](#string) | Title of the vulnerability. |
| <a id="vulnerabilityusernotescount"></a>`userNotesCount` | [`Int!`](#int) | Number of user notes attached to the vulnerability. |
| <a id="vulnerabilityuserpermissions"></a>`userPermissions` | [`VulnerabilityPermissions!`](#vulnerabilitypermissions) | Permissions for the current user on the resource. |
-| <a id="vulnerabilityvulnerabilitypath"></a>`vulnerabilityPath` | [`String`](#string) | URL to the vulnerability's details page. |
+| <a id="vulnerabilityvulnerabilitypath"></a>`vulnerabilityPath` | [`String`](#string) | Path to the vulnerability's details page. |
+| <a id="vulnerabilityweburl"></a>`webUrl` | [`String`](#string) | URL to the vulnerability's details page. |
#### Fields with arguments
@@ -19812,6 +20489,17 @@ Represents the labels widget.
| <a id="workitemwidgetlabelslabels"></a>`labels` | [`LabelConnection`](#labelconnection) | Labels assigned to the work item. (see [Connections](#connections)) |
| <a id="workitemwidgetlabelstype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. |
+### `WorkItemWidgetMilestone`
+
+Represents a milestone widget.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="workitemwidgetmilestonemilestone"></a>`milestone` | [`Milestone`](#milestone) | Milestone of the work item. |
+| <a id="workitemwidgetmilestonetype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. |
+
### `WorkItemWidgetStartAndDueDate`
Represents a start and due date widget.
@@ -19846,6 +20534,53 @@ Represents a weight widget.
| <a id="workitemwidgetweighttype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. |
| <a id="workitemwidgetweightweight"></a>`weight` | [`Int`](#int) | Weight of the work item. |
+### `X509Certificate`
+
+Represents an X.509 certificate.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="x509certificatecertificatestatus"></a>`certificateStatus` | [`String!`](#string) | Indicates if the certificate is good or revoked. |
+| <a id="x509certificatecreatedat"></a>`createdAt` | [`Time!`](#time) | Timestamp of when the certificate was saved. |
+| <a id="x509certificateemail"></a>`email` | [`String!`](#string) | Email associated with the cerificate. |
+| <a id="x509certificateid"></a>`id` | [`ID!`](#id) | ID of the certificate. |
+| <a id="x509certificateserialnumber"></a>`serialNumber` | [`String!`](#string) | Serial number of the certificate. |
+| <a id="x509certificatesubject"></a>`subject` | [`String!`](#string) | Subject of the certificate. |
+| <a id="x509certificatesubjectkeyidentifier"></a>`subjectKeyIdentifier` | [`String!`](#string) | Subject key identifier of the certificate. |
+| <a id="x509certificateupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp of when the certificate was last updated. |
+| <a id="x509certificatex509issuer"></a>`x509Issuer` | [`X509Issuer!`](#x509issuer) | Issuer of the certificate. |
+
+### `X509Issuer`
+
+Issuer of an X.509 certificate.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="x509issuercreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of when the issuer was created. |
+| <a id="x509issuercrlurl"></a>`crlUrl` | [`String`](#string) | Certificate revokation list of the issuer. |
+| <a id="x509issuerid"></a>`id` | [`ID`](#id) | ID of the issuer. |
+| <a id="x509issuersubject"></a>`subject` | [`String`](#string) | Subject of the issuer. |
+| <a id="x509issuersubjectkeyidentifier"></a>`subjectKeyIdentifier` | [`String`](#string) | Subject key identifier of the issuer. |
+| <a id="x509issuerupdatedat"></a>`updatedAt` | [`Time`](#time) | Timestamp of when the issuer was last updated. |
+
+### `X509Signature`
+
+X.509 signature for a signed commit.
+
+#### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="x509signaturecommitsha"></a>`commitSha` | [`String`](#string) | SHA of the associated commit. |
+| <a id="x509signatureproject"></a>`project` | [`Project`](#project) | Project of the associated commit. |
+| <a id="x509signatureuser"></a>`user` | [`UserCore`](#usercore) | User associated with the key. |
+| <a id="x509signatureverificationstatus"></a>`verificationStatus` | [`VerificationStatus`](#verificationstatus) | Indicates verification status of the associated key or certificate. |
+| <a id="x509signaturex509certificate"></a>`x509Certificate` | [`X509Certificate`](#x509certificate) | Certificate used for the signature. |
+
## Enumeration types
Also called _Enums_, enumeration types are a special kind of scalar that
@@ -20302,6 +21037,7 @@ Status of a container repository.
| Value | Description |
| ----- | ----------- |
| <a id="containerrepositorystatusdelete_failed"></a>`DELETE_FAILED` | Delete Failed status. |
+| <a id="containerrepositorystatusdelete_ongoing"></a>`DELETE_ONGOING` | Delete Ongoing status. |
| <a id="containerrepositorystatusdelete_scheduled"></a>`DELETE_SCHEDULED` | Delete Scheduled status. |
### `ContainerRepositoryTagSort`
@@ -20431,6 +21167,16 @@ Weight of the data visualization palette.
| <a id="dependencyproxymanifeststatuspending_destruction"></a>`PENDING_DESTRUCTION` | Dependency proxy manifest has a status of pending_destruction. |
| <a id="dependencyproxymanifeststatusprocessing"></a>`PROCESSING` | Dependency proxy manifest has a status of processing. |
+### `DeploymentApprovalSummaryStatus`
+
+Status of the deployment approval summary.
+
+| Value | Description |
+| ----- | ----------- |
+| <a id="deploymentapprovalsummarystatusapproved"></a>`APPROVED` | Summarized deployment approval status that is approved. |
+| <a id="deploymentapprovalsummarystatuspending_approval"></a>`PENDING_APPROVAL` | Summarized deployment approval status that is pending approval. |
+| <a id="deploymentapprovalsummarystatusrejected"></a>`REJECTED` | Summarized deployment approval status that is rejected. |
+
### `DeploymentStatus`
All deployment statuses.
@@ -20457,6 +21203,15 @@ All environment deployment tiers.
| <a id="deploymenttierstaging"></a>`STAGING` | Staging. |
| <a id="deploymenttiertesting"></a>`TESTING` | Testing. |
+### `DeploymentsApprovalStatus`
+
+Status of the deployment approval.
+
+| Value | Description |
+| ----- | ----------- |
+| <a id="deploymentsapprovalstatusapproved"></a>`APPROVED` | A deployment approval that is approved. |
+| <a id="deploymentsapprovalstatusrejected"></a>`REJECTED` | A deployment approval that is rejected. |
+
### `DesignCollectionCopyState`
Copy state of a DesignCollection.
@@ -20746,6 +21501,8 @@ Values for sorting issues.
| <a id="issuesortdue_date_desc"></a>`DUE_DATE_DESC` | Due date by descending order. |
| <a id="issuesortescalation_status_asc"></a>`ESCALATION_STATUS_ASC` | Status from triggered to resolved. |
| <a id="issuesortescalation_status_desc"></a>`ESCALATION_STATUS_DESC` | Status from resolved to triggered. |
+| <a id="issuesorthealth_status_asc"></a>`HEALTH_STATUS_ASC` | Issues with healthy issues first. |
+| <a id="issuesorthealth_status_desc"></a>`HEALTH_STATUS_DESC` | Issues with unhealthy issues first. |
| <a id="issuesortlabel_priority_asc"></a>`LABEL_PRIORITY_ASC` | Label priority by ascending order. |
| <a id="issuesortlabel_priority_desc"></a>`LABEL_PRIORITY_DESC` | Label priority by descending order. |
| <a id="issuesortmilestone_due_asc"></a>`MILESTONE_DUE_ASC` | Milestone due date by ascending order. |
@@ -20800,8 +21557,9 @@ Issue type.
| ----- | ----------- |
| <a id="issuetypeincident"></a>`INCIDENT` | Incident issue type. |
| <a id="issuetypeissue"></a>`ISSUE` | Issue issue type. |
+| <a id="issuetypeobjective"></a>`OBJECTIVE` **{warning-solid}** | **Introduced** in 15.6. This feature is in Alpha. It can be changed or removed at any time. Objective issue type. Available only when feature flag `okrs_mvc` is enabled. |
| <a id="issuetyperequirement"></a>`REQUIREMENT` | Requirement issue type. |
-| <a id="issuetypetask"></a>`TASK` **{warning-solid}** | **Introduced** in 15.2. This feature is in Alpha. It can be changed or removed at any time. Task issue type. Available only when feature flag `work_items` is enabled. |
+| <a id="issuetypetask"></a>`TASK` **{warning-solid}** | **Introduced** in 15.2. This feature is in Alpha. It can be changed or removed at any time. Task issue type. |
| <a id="issuetypetest_case"></a>`TEST_CASE` | Test Case issue type. |
### `IterationSearchableField`
@@ -21612,6 +22370,7 @@ Name of the feature that the callout is for.
| Value | Description |
| ----- | ----------- |
| <a id="usercalloutfeaturenameenumactive_user_count_threshold"></a>`ACTIVE_USER_COUNT_THRESHOLD` | Callout feature name for active_user_count_threshold. |
+| <a id="usercalloutfeaturenameenumartifacts_management_page_feedback_banner"></a>`ARTIFACTS_MANAGEMENT_PAGE_FEEDBACK_BANNER` | Callout feature name for artifacts_management_page_feedback_banner. |
| <a id="usercalloutfeaturenameenumbuy_pipeline_minutes_notification_dot"></a>`BUY_PIPELINE_MINUTES_NOTIFICATION_DOT` | Callout feature name for buy_pipeline_minutes_notification_dot. |
| <a id="usercalloutfeaturenameenumcanary_deployment"></a>`CANARY_DEPLOYMENT` | Callout feature name for canary_deployment. |
| <a id="usercalloutfeaturenameenumci_deprecation_warning_for_types_keyword"></a>`CI_DEPRECATION_WARNING_FOR_TYPES_KEYWORD` | Callout feature name for ci_deprecation_warning_for_types_keyword. |
@@ -21684,6 +22443,20 @@ Possible states of a user.
| <a id="verificationstateenumstarted"></a>`STARTED` | Verification process is in progress. |
| <a id="verificationstateenumsucceeded"></a>`SUCCEEDED` | Verification process finished successfully. |
+### `VerificationStatus`
+
+Verification status of a GPG or X.509 signature for a commit.
+
+| Value | Description |
+| ----- | ----------- |
+| <a id="verificationstatusmultiple_signatures"></a>`MULTIPLE_SIGNATURES` | multiple_signatures verification status. |
+| <a id="verificationstatusother_user"></a>`OTHER_USER` | other_user verification status. |
+| <a id="verificationstatussame_user_different_email"></a>`SAME_USER_DIFFERENT_EMAIL` | same_user_different_email verification status. |
+| <a id="verificationstatusunknown_key"></a>`UNKNOWN_KEY` | unknown_key verification status. |
+| <a id="verificationstatusunverified"></a>`UNVERIFIED` | unverified verification status. |
+| <a id="verificationstatusunverified_key"></a>`UNVERIFIED_KEY` | unverified_key verification status. |
+| <a id="verificationstatusverified"></a>`VERIFIED` | verified verification status. |
+
### `VisibilityLevelsEnum`
| Value | Description |
@@ -21869,6 +22642,7 @@ Type of a work item widget.
| <a id="workitemwidgettypehierarchy"></a>`HIERARCHY` | Hierarchy widget. |
| <a id="workitemwidgettypeiteration"></a>`ITERATION` | Iteration widget. |
| <a id="workitemwidgettypelabels"></a>`LABELS` | Labels widget. |
+| <a id="workitemwidgettypemilestone"></a>`MILESTONE` | Milestone widget. |
| <a id="workitemwidgettypestart_and_due_date"></a>`START_AND_DUE_DATE` | Start And Due Date widget. |
| <a id="workitemwidgettypestatus"></a>`STATUS` | Status widget. |
| <a id="workitemwidgettypeweight"></a>`WEIGHT` | Weight widget. |
@@ -22199,6 +22973,12 @@ A `IncidentManagementTimelineEventID` is a global ID. It is encoded as a string.
An example `IncidentManagementTimelineEventID` is: `"gid://gitlab/IncidentManagement::TimelineEvent/1"`.
+### `IncidentManagementTimelineEventTagID`
+
+A `IncidentManagementTimelineEventTagID` is a global ID. It is encoded as a string.
+
+An example `IncidentManagementTimelineEventTagID` is: `"gid://gitlab/IncidentManagement::TimelineEventTag/1"`.
+
### `Int`
Represents non-fractional signed whole numeric values. Int can represent values between -(2^31) and 2^31 - 1.
@@ -22651,6 +23431,23 @@ Implementations:
| <a id="civariablevalue"></a>`value` | [`String`](#string) | Value of the variable. |
| <a id="civariablevariabletype"></a>`variableType` | [`CiVariableType`](#civariabletype) | Type of the variable. |
+#### `CommitSignature`
+
+Represents signing information for a commit.
+
+Implementations:
+
+- [`GpgSignature`](#gpgsignature)
+- [`X509Signature`](#x509signature)
+
+##### Fields
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="commitsignaturecommitsha"></a>`commitSha` | [`String`](#string) | SHA of the associated commit. |
+| <a id="commitsignatureproject"></a>`project` | [`Project`](#project) | Project of the associated commit. |
+| <a id="commitsignatureverificationstatus"></a>`verificationStatus` | [`VerificationStatus`](#verificationstatus) | Indicates verification status of the associated key or certificate. |
+
#### `CurrentUserTodos`
Implementations:
@@ -23139,6 +23936,7 @@ Implementations:
- [`WorkItemWidgetHierarchy`](#workitemwidgethierarchy)
- [`WorkItemWidgetIteration`](#workitemwidgetiteration)
- [`WorkItemWidgetLabels`](#workitemwidgetlabels)
+- [`WorkItemWidgetMilestone`](#workitemwidgetmilestone)
- [`WorkItemWidgetStartAndDueDate`](#workitemwidgetstartandduedate)
- [`WorkItemWidgetStatus`](#workitemwidgetstatus)
- [`WorkItemWidgetWeight`](#workitemwidgetweight)
@@ -23193,6 +23991,7 @@ Field that are available while modifying the custom mapping attributes for an HT
| <a id="boardissueinputmilestonewildcardid"></a>`milestoneWildcardId` | [`MilestoneWildcardId`](#milestonewildcardid) | Filter by milestone ID wildcard. |
| <a id="boardissueinputmyreactionemoji"></a>`myReactionEmoji` | [`String`](#string) | Filter by reaction emoji applied by the current user. Wildcard values "NONE" and "ANY" are supported. |
| <a id="boardissueinputnot"></a>`not` | [`NegatedBoardIssueInput`](#negatedboardissueinput) | List of negated arguments. |
+| <a id="boardissueinputor"></a>`or` | [`UnionedIssueFilterInput`](#unionedissuefilterinput) | List of arguments with inclusive OR. |
| <a id="boardissueinputreleasetag"></a>`releaseTag` | [`String`](#string) | Filter by release tag. |
| <a id="boardissueinputsearch"></a>`search` | [`String`](#string) | Search query for issue title or description. |
| <a id="boardissueinputtypes"></a>`types` | [`[IssueType!]`](#issuetype) | Filter by the given issue types. |
@@ -23231,6 +24030,7 @@ Attributes for defining a CI/CD variable.
| Name | Type | Description |
| ---- | ---- | ----------- |
| <a id="complianceframeworkinputcolor"></a>`color` | [`String`](#string) | New color representation of the compliance framework in hex format. e.g. #FCA121. |
+| <a id="complianceframeworkinputdefault"></a>`default` | [`Boolean`](#boolean) | Set this compliance framework as the default framework for the group. |
| <a id="complianceframeworkinputdescription"></a>`description` | [`String`](#string) | New description for the compliance framework. |
| <a id="complianceframeworkinputname"></a>`name` | [`String`](#string) | New name for the compliance framework. |
| <a id="complianceframeworkinputpipelineconfigurationfullpath"></a>`pipelineConfigurationFullPath` | [`String`](#string) | Full path of the compliance pipeline configuration stored in a project repository, such as `.gitlab/.compliance-gitlab-ci.yml@compliance/hipaa` **(ULTIMATE)**. |
@@ -23569,6 +24369,14 @@ Represents an action to perform over a snippet file.
| <a id="snippetblobactioninputtypefilepath"></a>`filePath` | [`String!`](#string) | Path of the snippet file. |
| <a id="snippetblobactioninputtypepreviouspath"></a>`previousPath` | [`String`](#string) | Previous path of the snippet file. |
+### `StatusFilterInput`
+
+#### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="statusfilterinputstatus"></a>`status` | [`RequirementStatusFilter!`](#requirementstatusfilter) | Status of the work item. |
+
### `StatusInput`
#### Arguments
@@ -23588,6 +24396,15 @@ A time-frame defined as a closed inclusive range of two dates.
| <a id="timeframeend"></a>`end` | [`Date!`](#date) | End of the range. |
| <a id="timeframestart"></a>`start` | [`Date!`](#date) | Start of the range. |
+### `UnionedIssueFilterInput`
+
+#### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="unionedissuefilterinputassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Filters issues that are assigned to at least one of the given users. |
+| <a id="unionedissuefilterinputauthorusernames"></a>`authorUsernames` | [`[String!]`](#string) | Filters issues that are authored by one of the given users. |
+
### `UpdateDiffImagePositionInput`
#### Arguments
@@ -23664,6 +24481,7 @@ A time-frame defined as a closed inclusive range of two dates.
| <a id="workitemupdatedtaskinputhierarchywidget"></a>`hierarchyWidget` | [`WorkItemWidgetHierarchyUpdateInput`](#workitemwidgethierarchyupdateinput) | Input for hierarchy widget. |
| <a id="workitemupdatedtaskinputid"></a>`id` | [`WorkItemID!`](#workitemid) | Global ID of the work item. |
| <a id="workitemupdatedtaskinputlabelswidget"></a>`labelsWidget` | [`WorkItemWidgetLabelsUpdateInput`](#workitemwidgetlabelsupdateinput) | Input for labels widget. |
+| <a id="workitemupdatedtaskinputmilestonewidget"></a>`milestoneWidget` | [`WorkItemWidgetMilestoneInput`](#workitemwidgetmilestoneinput) | Input for milestone widget. |
| <a id="workitemupdatedtaskinputstartandduedatewidget"></a>`startAndDueDateWidget` | [`WorkItemWidgetStartAndDueDateUpdateInput`](#workitemwidgetstartandduedateupdateinput) | Input for start and due date widget. |
| <a id="workitemupdatedtaskinputstateevent"></a>`stateEvent` | [`WorkItemStateEvent`](#workitemstateevent) | Close or reopen a work item. |
| <a id="workitemupdatedtaskinputtitle"></a>`title` | [`String`](#string) | Title of the work item. |
@@ -23718,6 +24536,14 @@ A time-frame defined as a closed inclusive range of two dates.
| <a id="workitemwidgetlabelsupdateinputaddlabelids"></a>`addLabelIds` | [`[LabelID!]`](#labelid) | Global IDs of labels to be added to the work item. |
| <a id="workitemwidgetlabelsupdateinputremovelabelids"></a>`removeLabelIds` | [`[LabelID!]`](#labelid) | Global IDs of labels to be removed from the work item. |
+### `WorkItemWidgetMilestoneInput`
+
+#### Arguments
+
+| Name | Type | Description |
+| ---- | ---- | ----------- |
+| <a id="workitemwidgetmilestoneinputmilestoneid"></a>`milestoneId` | [`MilestoneID`](#milestoneid) | Milestone to assign to the work item. |
+
### `WorkItemWidgetStartAndDueDateUpdateInput`
#### Arguments
diff --git a/doc/api/group_iterations.md b/doc/api/group_iterations.md
index 92333de701c..988986d8965 100644
--- a/doc/api/group_iterations.md
+++ b/doc/api/group_iterations.md
@@ -20,8 +20,8 @@ Returns a list of group iterations.
GET /groups/:id/iterations
GET /groups/:id/iterations?state=opened
GET /groups/:id/iterations?state=closed
-GET /groups/:id/iterations?title=1.0
GET /groups/:id/iterations?search=version
+GET /groups/:id/iterations?include_ancestors=false
```
| Attribute | Type | Required | Description |
diff --git a/doc/api/group_level_variables.md b/doc/api/group_level_variables.md
index cdda15d9610..6ca4cc1d080 100644
--- a/doc/api/group_level_variables.md
+++ b/doc/api/group_level_variables.md
@@ -32,6 +32,7 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a
"value": "TEST_1",
"protected": false,
"masked": false,
+ "raw": false,
"environment_scope": "*"
},
{
@@ -40,6 +41,7 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a
"value": "TEST_2",
"protected": false,
"masked": false,
+ "raw": false,
"environment_scope": "*"
}
]
@@ -69,6 +71,7 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a
"value": "TEST_1",
"protected": false,
"masked": false,
+ "raw": false,
"environment_scope": "*"
}
```
@@ -89,6 +92,7 @@ POST /groups/:id/variables
| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` |
| `protected` | boolean | no | Whether the variable is protected |
| `masked` | boolean | no | Whether the variable is masked |
+| `raw` | boolean | no | Whether the variable is expandable |
| `environment_scope` **(PREMIUM)** | string | no | The [environment scope](../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable) of a variable |
```shell
@@ -103,6 +107,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
"variable_type": "env_var",
"protected": false,
"masked": false,
+ "raw": false,
"environment_scope": "*"
}
```
@@ -123,6 +128,7 @@ PUT /groups/:id/variables/:key
| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` |
| `protected` | boolean | no | Whether the variable is protected |
| `masked` | boolean | no | Whether the variable is masked |
+| `raw` | boolean | no | Whether the variable is expandable |
| `environment_scope` **(PREMIUM)** | string | no | The [environment scope](../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable) of a variable |
```shell
@@ -137,6 +143,7 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \
"variable_type": "env_var",
"protected": true,
"masked": true,
+ "raw": true,
"environment_scope": "*"
}
```
diff --git a/doc/api/group_protected_environments.md b/doc/api/group_protected_environments.md
index 4cf87cb4305..ecd433bf321 100644
--- a/doc/api/group_protected_environments.md
+++ b/doc/api/group_protected_environments.md
@@ -95,7 +95,7 @@ Example response:
}
```
-## Protect an environment
+## Protect a single environment
Protects a single environment.
@@ -136,7 +136,7 @@ Example response:
}
```
-## Update an environment
+## Update a protected environment
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/351854) in GitLab 15.4.
@@ -304,7 +304,7 @@ Example response:
}
```
-## Unprotect environment
+## Unprotect a single environment
Unprotects the given protected environment.
diff --git a/doc/api/groups.md b/doc/api/groups.md
index c1effc78a4d..cba54648705 100644
--- a/doc/api/groups.md
+++ b/doc/api/groups.md
@@ -1413,7 +1413,7 @@ Parameters:
## Group members
-Please consult the [Group Members](members.md) documentation.
+See the [Group Members](members.md) documentation.
## LDAP Group Links
@@ -1605,7 +1605,7 @@ If successful, returns [`201`](index.md#status-codes) and the following response
Example request:
```shell
-curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/1/saml_group_links"
+curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --header "Content-Type: application/json" --data '{ "saml_group_name": "<your_saml_group_name`>", "access_level": <chosen_access_level> }' --url "https://gitlab.example.com/api/v4/groups/1/saml_group_links"
```
Example response:
diff --git a/doc/api/index.md b/doc/api/index.md
index 7e14137b0fe..cc54731de81 100644
--- a/doc/api/index.md
+++ b/doc/api/index.md
@@ -593,7 +593,7 @@ GET /api/v4/projects/1/repository/tags/my%2Ftag
## Request Payload
API Requests can use parameters sent as [query strings](https://en.wikipedia.org/wiki/Query_string)
-or as a [payload body](https://tools.ietf.org/html/draft-ietf-httpbis-p3-payload-14#section-3.2).
+or as a [payload body](https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-p3-payload-14#section-3.2).
GET requests usually send a query string, while PUT or POST requests usually
send the payload body:
diff --git a/doc/api/instance_level_ci_variables.md b/doc/api/instance_level_ci_variables.md
index 776b1000651..e71cf777b2c 100644
--- a/doc/api/instance_level_ci_variables.md
+++ b/doc/api/instance_level_ci_variables.md
@@ -28,14 +28,16 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a
"variable_type": "env_var",
"value": "TEST_1",
"protected": false,
- "masked": false
+ "masked": false,
+ "raw": false
},
{
"key": "TEST_VARIABLE_2",
"variable_type": "env_var",
"value": "TEST_2",
"protected": false,
- "masked": false
+ "masked": false,
+ "raw": false
}
]
```
@@ -62,7 +64,8 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a
"variable_type": "env_var",
"value": "TEST_1",
"protected": false,
- "masked": false
+ "masked": false,
+ "raw": false
}
```
@@ -83,6 +86,7 @@ POST /admin/ci/variables
| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file`. |
| `protected` | boolean | no | Whether the variable is protected. |
| `masked` | boolean | no | Whether the variable is masked. |
+| `raw` | boolean | no | Whether the variable is expandable. |
```shell
curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
@@ -95,7 +99,8 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
"value": "new value",
"variable_type": "env_var",
"protected": false,
- "masked": false
+ "masked": false,
+ "raw": false
}
```
@@ -114,6 +119,7 @@ PUT /admin/ci/variables/:key
| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file`. |
| `protected` | boolean | no | Whether the variable is protected. |
| `masked` | boolean | no | Whether the variable is masked. |
+| `raw` | boolean | no | Whether the variable is expandable. |
```shell
curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \
@@ -126,7 +132,8 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \
"value": "updated value",
"variable_type": "env_var",
"protected": true,
- "masked": true
+ "masked": true,
+ "raw": true
}
```
diff --git a/doc/api/integrations.md b/doc/api/integrations.md
index 176ed931d38..d64c67e1402 100644
--- a/doc/api/integrations.md
+++ b/doc/api/integrations.md
@@ -262,7 +262,7 @@ GET /projects/:id/integrations/buildkite
## Campfire
Send notifications about push events to Campfire chat rooms.
-[New users can no longer sign up for Campfire](https://basecamp.com/retired/campfire).
+[New users can no longer sign up for Campfire](https://basecamp.com/handbook/05-product-histories#campfire).
### Create/Edit Campfire integration
diff --git a/doc/api/issues.md b/doc/api/issues.md
index a0eb518f23e..dd5a1354a3a 100644
--- a/doc/api/issues.md
+++ b/doc/api/issues.md
@@ -1506,8 +1506,8 @@ Please use `iid` of the `epic` attribute instead.
Clone the issue to given project. If the user has insufficient permissions,
an error message with status code `400` is returned.
-Copies as much data as possible as long as the target project contains equivalent labels, milestones,
-and so on.
+Copies as much data as possible as long as the target project contains equivalent
+criteria such as labels or milestones.
```plaintext
POST /projects/:id/issues/:issue_iid/clone
diff --git a/doc/api/iterations.md b/doc/api/iterations.md
index 5704bcd3418..4997a917a5a 100644
--- a/doc/api/iterations.md
+++ b/doc/api/iterations.md
@@ -22,8 +22,8 @@ Returns a list of project iterations.
GET /projects/:id/iterations
GET /projects/:id/iterations?state=opened
GET /projects/:id/iterations?state=closed
-GET /projects/:id/iterations?title=1.0
GET /projects/:id/iterations?search=version
+GET /projects/:id/iterations?include_ancestors=false
```
| Attribute | Type | Required | Description |
diff --git a/doc/api/job_artifacts.md b/doc/api/job_artifacts.md
index 2a40f7c15ef..3e5f49377cb 100644
--- a/doc/api/job_artifacts.md
+++ b/doc/api/job_artifacts.md
@@ -25,7 +25,7 @@ GET /projects/:id/jobs/:job_id/artifacts
Example request using the `PRIVATE-TOKEN` header:
```shell
-curl --output artifacts.zip --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/42/artifacts"
+curl --location --output artifacts.zip --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/42/artifacts"
```
To use this in a [`script` definition](../ci/yaml/index.md#script) inside
@@ -90,7 +90,7 @@ Parameters
Example request using the `PRIVATE-TOKEN` header:
```shell
-curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/main/download?job=test"
+curl --location --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/main/download?job=test"
```
To use this in a [`script` definition](../ci/yaml/index.md#script) inside
diff --git a/doc/api/lint.md b/doc/api/lint.md
index c1d95f65a86..e50832a9528 100644
--- a/doc/api/lint.md
+++ b/doc/api/lint.md
@@ -210,7 +210,7 @@ Example responses:
Checks if a project's latest (`HEAD` of the project's default branch)
`.gitlab-ci.yml` configuration is valid. This endpoint uses all namespace
-specific data available, including variables, local includes, and so on.
+specific data available, including variables and local includes.
```plaintext
GET /projects/:id/ci/lint
diff --git a/doc/api/members.md b/doc/api/members.md
index 52bc3f6f468..66729abcad8 100644
--- a/doc/api/members.md
+++ b/doc/api/members.md
@@ -313,9 +313,6 @@ Gets a list of group members that count as billable. The list includes members i
This API endpoint works on top-level groups only. It does not work on subgroups.
-NOTE:
-Unlike other API endpoints, billable members is updated once per day at 12:00 UTC.
-
This function takes [pagination](index.md#pagination) parameters `page` and `per_page` to restrict the list of users.
[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/262875) in GitLab 13.7, the `search` and
diff --git a/doc/api/merge_request_approvals.md b/doc/api/merge_request_approvals.md
index e65263030b1..0476035784a 100644
--- a/doc/api/merge_request_approvals.md
+++ b/doc/api/merge_request_approvals.md
@@ -6,6 +6,8 @@ info: "To determine the technical writer assigned to the Stage/Group associated
# Merge request approvals API **(PREMIUM)**
+> Changing approval configuration with the `/approvals` endpoint was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/11132) in GitLab 12.3.
+
Configuration for
[approvals on all merge requests](../user/project/merge_requests/approvals/index.md)
in the project. Must be authenticated for all endpoints.
@@ -57,7 +59,7 @@ Supported attributes:
| Attribute | Type | Required | Description |
| ------------------------------------------------ | ------- | -------- | -- |
| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). |
-| `approvals_before_merge` | integer | **{dotted-circle}** No | How many approvals are required before a merge request can be merged. Deprecated in GitLab 12.0 in favor of Approval Rules API. |
+| `approvals_before_merge` (deprecated) | integer | **{dotted-circle}** No | How many approvals are required before a merge request can be merged. [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/11132) in GitLab 12.3. |
| `disable_overriding_approvers_per_merge_request` | boolean | **{dotted-circle}** No | Allow or prevent overriding approvers per merge request. |
| `merge_requests_author_approval` | boolean | **{dotted-circle}** No | Allow or prevent authors from self approving merge requests; `true` means authors can self approve. |
| `merge_requests_disable_committers_approval` | boolean | **{dotted-circle}** No | Allow or prevent committers from self approving merge requests. |
@@ -582,9 +584,16 @@ Supported attributes:
}
```
-### Change approval configuration
+### Change approval configuration (deprecated)
-> Moved to GitLab Premium in 13.9.
+> - Moved to GitLab Premium in GitLab 13.9.
+> - Endpoint `/approvals` [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/11132) in GitLab 12.3.
+
+WARNING:
+The `/approvals` endpoint was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/11132) in GitLab 12.3
+and is planned for removal in 16.0. To change the approvals required for a merge request,
+use the `/approval_rules` endpoint described in [Create merge request level rule](#create-merge-request-level-rule).
+on this page. This change is a breaking change.
If you are allowed to, you can change `approvals_required` using the following
endpoint:
@@ -598,7 +607,7 @@ Supported attributes:
| Attribute | Type | Required | Description |
|----------------------|-------------------|----------|-------------|
| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding). |
-| `approvals_required` | integer | **{check-circle}** Yes | Approvals required before MR can be merged. Deprecated in GitLab 12.0 in favor of Approval Rules API. |
+| `approvals_required` | integer | **{check-circle}** Yes | Approvals required before MR can be merged. |
| `merge_request_iid` | integer | **{check-circle}** Yes | The IID of the merge request. |
```json
diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md
index 4667e48f233..4b4d36ec23e 100644
--- a/doc/api/merge_requests.md
+++ b/doc/api/merge_requests.md
@@ -160,6 +160,7 @@ Supported attributes:
},
"merge_when_pipeline_succeeds": true,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "not_open",
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
"squash_commit_sha": null,
@@ -360,6 +361,7 @@ Supported attributes:
},
"merge_when_pipeline_succeeds": true,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "not_open",
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
"squash_commit_sha": null,
@@ -547,6 +549,7 @@ Supported attributes:
},
"merge_when_pipeline_succeeds": true,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "not_open",
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
"squash_commit_sha": null,
@@ -616,6 +619,67 @@ Supported attributes:
| `include_rebase_in_progress` | boolean | **{dotted-circle}** No | If `true`, response includes whether a rebase operation is in progress. |
| `render_html` | boolean | **{dotted-circle}** No | If `true`, response includes rendered HTML for title and description. |
+### Response
+
+| Attribute | Type | Description |
+|----------------------------------|------|-------------|
+| `approvals_before_merge` | integer | **(PREMIUM)** Number of approvals required before this merge request can merge. |
+| `assignee` | object | First assignee of the merge request. |
+| `assignees` | array | Assignees of the merge request. |
+| `author` | object | User who created this merge request. |
+| `blocking_discussions_resolved` | boolean | Indicates if all discussions are resolved only if all are required before merge request can be merged. |
+| `changes_count` | string | Number of changes made on the merge request. |
+| `closed_at` | datetime | Timestamp of when the merge request was closed. |
+| `closed_by` | object | User who closed this merge request. |
+| `created_at` | datetime | Timestamp of when the merge request was created. |
+| `description` | string | Description of the merge request. Contains Markdown rendered as HTML for caching. |
+| `detailed_merge_status` | string | Detailed merge status of the merge request. |
+| `diff_refs` | object | References of the base SHA, the head SHA, and the start SHA for this merge request. Corresponds to the latest diff version of the merge request. |
+| `discussion_locked` | boolean | Indicates if comments on the merge request are locked to members only. |
+| `downvotes` | integer | Number of downvotes for the merge request. |
+| `draft` | boolean | Indicates if the merge request is a draft. |
+| `first_contribution` | boolean | Indicates if the merge request is the first contribution of the author. |
+| `first_deployed_to_production_at` | datetime | Timestamp of when the first deployment finished. |
+| `force_remove_source_branch` | boolean | Indicates if the project settings will lead to source branch deletion after merge. |
+| `has_conflicts` | boolean | Indicates if merge request has conflicts and cannot be merged. Dependent on the `merge_status` property. Returns `false` unless `merge_status` is `cannot_be_merged`. |
+| `head_pipeline` | object | Pipeline running on the branch HEAD of the merge request. Contains more complete information than `pipeline` and should be used instead of it. |
+| `id` | integer | ID of the merge request. |
+| `iid` | integer | Internal ID of the merge request. |
+| `labels` | array | Labels of the merge request. |
+| `latest_build_finished_at` | datetime | Timestamp of when the latest build for the merge request finished. |
+| `latest_build_started_at` | datetime | Timestamp of when the latest build for the merge request started. |
+| `merge_commit_sha` | string | SHA of the merge request commit. Returns `null` until merged. |
+| `merge_error` | string | Error message due to a merge error. |
+| `merge_user` | object | The user who merged this merge request, the user who set it to merge when pipeline succeeds, or `null`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/349031) in GitLab 14.7. |
+| `merge_status` | string | Status of the merge request. Can be `unchecked`, `checking`, `can_be_merged`, `cannot_be_merged`, or `cannot_be_merged_recheck`. Affects the `has_conflicts` property. [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/3169#note_1162532204) in GitLab 15.6. Use `detailed_merge_status` instead. |
+| `merge_when_pipeline_succeeds` | boolean | Indicates if the merge has been set to be merged when its pipeline succeeds. |
+| `merged_at` | datetime | Timestamp of when the merge request was merged. |
+| `merged_by` | object | User who merged this merge request or set it to merge when pipeline succeeds. [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/350534) in GitLab 14.7, and scheduled for removal in [API version 5](https://gitlab.com/groups/gitlab-org/-/epics/8115). Use `merge_user` instead. |
+| `milestone` | object | Milestone of the merge request. |
+| `pipeline` | object | Pipeline running on the branch HEAD of the merge request. Consider using `head_pipeline` instead, as it contains more information. |
+| `project_id` | integer | ID of the merge request project. |
+| `reference` | string | Internal reference of the merge request. Returned in shortened format by default. [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20354) in GitLab 12.7, and scheduled for removal in [API version 5](https://gitlab.com/groups/gitlab-org/-/epics/8115). Use `references` instead. |
+| `references` | object | Internal references of the merge request. Includes `short`, `relative`, and `full` references. `references.relative` is relative to the merge request's group or project. When fetched from the merge request's project, `relative` and `short` formats are identical. When requested across groups or projects, `relative` and `full` formats are identical.|
+| `reviewers` | array | Reviewers of the merge request. |
+| `sha` | string | Diff head SHA of the merge request. |
+| `should_remove_source_branch` | boolean | Indicates if the source branch of the merge request will be deleted after merge. |
+| `source_branch` | string | Source branch of the merge request. |
+| `source_project_id` | integer | ID of the merge request source project. |
+| `squash` | boolean | Indicates if squash on merge is enabled. |
+| `squash_commit_sha` | string | SHA of the squash commit. Empty until merged. |
+| `state` | string | State of the merge request. Can be `opened`, `closed`, `merged` or `locked`. |
+| `subscribed` | boolean | Indicates if the currently logged in user is subscribed to this merge request. |
+| `target_branch` | string | Target branch of the merge request. |
+| `target_project_id` | integer | ID of the merge request target project. |
+| `task_completion_status` | object | Completion status of tasks. |
+| `title` | string | Title of the merge request. |
+| `updated_at` | datetime | Timestamp of when the merge request was updated. |
+| `upvotes` | integer | Number of upvotes for the merge request. |
+| `user` | object | Permissions of the user requested for the merge request. |
+| `user_notes_count` | integer | User notes count of the merge request. |
+| `web_url` | string | Web URL of the merge request. |
+| `work_in_progress` | boolean | Deprecated: Use `draft` instead. Indicates if the merge request is a draft. |
+
```json
{
"id": 155016530,
@@ -626,7 +690,7 @@ Supported attributes:
"state": "opened",
"created_at": "2022-05-13T07:26:38.402Z",
"updated_at": "2022-05-14T03:38:31.354Z",
- "merged_by": null, // Deprecated and will be removed in API v5, use `merge_user` instead
+ "merged_by": null, // Deprecated and will be removed in API v5. Use `merge_user` instead.
"merge_user": null,
"merged_at": null,
"closed_by": null,
@@ -655,13 +719,14 @@ Supported attributes:
"milestone": null,
"merge_when_pipeline_succeeds": false,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "can_be_merged",
"sha": "e82eb4a098e32c796079ca3915e07487fc4db24c",
"merge_commit_sha": null,
"squash_commit_sha": null,
"discussion_locked": null,
"should_remove_source_branch": null,
"force_remove_source_branch": true,
- "reference": "!133",
+ "reference": "!133", // Deprecated. Use `references` instead.
"references": {
"short": "!133",
"relative": "!133",
@@ -687,7 +752,7 @@ Supported attributes:
"latest_build_started_at": "2022-05-13T09:46:50.032Z",
"latest_build_finished_at": null,
"first_deployed_to_production_at": null,
- "pipeline": { // Old parameter, use `head_pipeline` instead.
+ "pipeline": { // Use `head_pipeline` instead.
"id": 538317940,
"iid": 1877,
"project_id": 15513260,
@@ -748,44 +813,43 @@ Supported attributes:
"first_contribution": false,
"user": {
"can_merge": true
- }
-}
-```
-
-Users on [GitLab Premium or higher](https://about.gitlab.com/pricing/) also see
-the `approvals_before_merge` parameter:
-
-```json
-{
- "id": 1,
- "title": "test1",
- "approvals_before_merge": null
- ...
+ },
+ "approvals_before_merge": { // Available for GitLab Premium and higher tiers only
+ "id": 1,
+ "title": "test1",
+ "approvals_before_merge": null
+ },
}
```
### Single merge request response notes
-- The `merge_status` field may hold one of the following values:
- - `unchecked`: This merge request has not yet been checked.
- - `checking`: This merge request is currently being checked to see if it can be merged.
- - `can_be_merged`: This merge request can be merged without conflict.
- - `cannot_be_merged`: There are merge conflicts between the source and target branches.
- - `cannot_be_merged_recheck`: Currently unchecked. Before the current changes, there were conflicts.
-- The `diff_refs` in the response correspond to the latest diff version of the merge request.
- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29984) in GitLab 12.8, the mergeability (`merge_status`)
of each merge request is checked asynchronously when a request is made to this endpoint. Poll this API endpoint
to get updated status. This affects the `has_conflicts` property as it is dependent on the `merge_status`. It returns
`false` unless `merge_status` is `cannot_be_merged`.
-- `references.relative` is relative to the group or project that the merge request is being requested. When the merge
- request is fetched from its project, `relative` format would be the same as `short` format, and when requested across
- groups or projects, it is expected to be the same as `full` format.
-- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/349031) in GitLab 14.7,
- field `merge_user` can be either user who merged this merge request,
- user who set it to merge when pipeline succeeds or `null`.
- Field `merged_by` (user who merged this merge request or `null`) has been deprecated.
-- `pipeline` is an old parameter and should not be used. Use `head_pipeline` instead,
- as it is faster and returns more information.
+
+### Merge status
+
+> - The `detailed_merge_status` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101724) in GitLab 15.6.
+> - The `merge_status` field was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/3169#note_1162532204) in GitLab 15.6.
+
+Use `detailed_merge_status` instead of `merge_status` to account for all potential statuses.
+
+- The `detailed_merge_status` field may hold one of the following values:
+ - `blocked_status`: Merge request is blocked by another merge request.
+ - `broken_status`: Can not merge the source into the target branch, potential conflict.
+ - `checking`: currently checking for mergeability.
+ - `ci_must_pass`: Pipeline must succeed before merging.
+ - `ci_still_running`: Pipeline is still running.
+ - `discussions_not_resolved`: Discussions must be resolved before merging.
+ - `draft_status`: Merge request must not be draft before merging.
+ - `external_status_checks`: Status checks must pass.
+ - `mergeable`: branch can be merged.
+ - `not_approved`: Merge request must be approved before merging.
+ - `not_open`: merge request must be open before merging.
+ - `policies_denied`: There are denied policies for the merge request.
+ - `unchecked`: merge status has not been checked.
## Get single MR participants
@@ -994,6 +1058,7 @@ Supported attributes:
},
"merge_when_pipeline_succeeds": true,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "can_be_merged",
"subscribed" : true,
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
@@ -1211,6 +1276,7 @@ If `approvals_before_merge` is not provided, it inherits the value from the targ
},
"merge_when_pipeline_succeeds": true,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "not_open",
"merge_error": null,
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
@@ -1392,6 +1458,7 @@ Must include at least one non-required attribute from above.
},
"merge_when_pipeline_succeeds": true,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "not_open",
"merge_error": null,
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
@@ -1580,6 +1647,7 @@ Supported attributes:
},
"merge_when_pipeline_succeeds": true,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "not_open",
"merge_error": null,
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
@@ -1792,6 +1860,7 @@ Supported attributes:
},
"merge_when_pipeline_succeeds": false,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "not_open",
"merge_error": null,
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
@@ -2124,6 +2193,7 @@ Example response:
},
"merge_when_pipeline_succeeds": true,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "not_open",
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
"squash_commit_sha": null,
@@ -2294,6 +2364,7 @@ Example response:
},
"merge_when_pipeline_succeeds": true,
"merge_status": "can_be_merged",
+ "detailed_merge_status": "not_open",
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
"squash_commit_sha": null,
@@ -2479,6 +2550,7 @@ Example response:
},
"merge_when_pipeline_succeeds": false,
"merge_status": "unchecked",
+ "detailed_merge_status": "not_open",
"subscribed": true,
"sha": "8888888888888888888888888888888888888888",
"merge_commit_sha": null,
diff --git a/doc/api/merge_trains.md b/doc/api/merge_trains.md
index 64ca2bf9b97..111cf5255d6 100644
--- a/doc/api/merge_trains.md
+++ b/doc/api/merge_trains.md
@@ -84,3 +84,73 @@ Example response:
}
]
```
+
+## List merge requests in a merge train
+
+Get all merge requests added to a merge train for the requested target branch.
+
+```plaintext
+GET /projects/:id/merge_trains/:target_branch
+```
+
+Supported attributes:
+
+| Attribute | Type | Required | Description |
+| --------------- | ---------------| -------- | ------------ |
+| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
+| `target_branch` | string | yes | The target branch of the merge train. |
+| `scope` | string | no | Return Merge Trains filtered by the given scope. Available scopes are `active` (to be merged) and `complete` (have been merged). |
+| `sort` | string | no | Return Merge Trains sorted in `asc` or `desc` order. Default is `desc`. |
+
+Example request:
+
+```shell
+curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/597/merge_trains/main"
+```
+
+Example response:
+
+```json
+[
+ {
+ "id": 267,
+ "merge_request": {
+ "id": 273,
+ "iid": 1,
+ "project_id": 597,
+ "title": "My title 9",
+ "description": null,
+ "state": "opened",
+ "created_at": "2022-10-31T19:06:05.725Z",
+ "updated_at": "2022-10-31T19:06:05.725Z",
+ "web_url": "http://localhost/namespace18/project21/-/merge_requests/1"
+ },
+ "user": {
+ "id": 933,
+ "username": "user12",
+ "name": "Sidney Jones31",
+ "state": "active",
+ "avatar_url": "https://www.gravatar.com/avatar/6c8365de387cb3db10ecc7b1880203c4?s=80\u0026d=identicon",
+ "web_url": "http://localhost/user12"
+ },
+ "pipeline": {
+ "id": 273,
+ "iid": 1,
+ "project_id": 598,
+ "sha": "b83d6e391c22777fca1ed3012fce84f633d7fed0",
+ "ref": "main",
+ "status": "pending",
+ "source": "push",
+ "created_at": "2022-10-31T19:06:06.231Z",
+ "updated_at": "2022-10-31T19:06:06.231Z",
+ "web_url": "http://localhost/namespace19/project22/-/pipelines/273"
+ },
+ "created_at": "2022-10-31T19:06:06.237Z",
+ "updated_at":"2022-10-31T19:06:06.237Z",
+ "target_branch":"main",
+ "status":"idle",
+ "merged_at":null,
+ "duration":null
+ }
+]
+```
diff --git a/doc/api/metadata.md b/doc/api/metadata.md
index 3803173b0b8..c3cbae70a54 100644
--- a/doc/api/metadata.md
+++ b/doc/api/metadata.md
@@ -24,6 +24,7 @@ Response body attributes:
| `kas.enabled` | boolean | Indicates whether KAS is enabled. |
| `kas.externalUrl` | string or null | URL used by the agents to communicate with KAS. It's `null` if `kas.enabled` is `false`. |
| `kas.version` | string or null | Version of KAS. It's `null` if `kas.enabled` is `false`. |
+| `enterprise` | boolean | Indicates whether GitLab instance is Enterprise Edition. |
Example request:
@@ -41,6 +42,7 @@ Example response:
"enabled": true,
"externalUrl": "grpc://gitlab.example.com:8150",
"version": "15.0.0"
- }
+ },
+ "enterprise": true
}
```
diff --git a/doc/api/milestones.md b/doc/api/milestones.md
index df09e374395..22746d4ceed 100644
--- a/doc/api/milestones.md
+++ b/doc/api/milestones.md
@@ -88,8 +88,8 @@ Parameters:
| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
| `title` | string | yes | The title of a milestone |
| `description` | string | no | The description of the milestone |
-| `due_date` | string | no | The due date of the milestone |
-| `start_date` | string | no | The start date of the milestone |
+| `due_date` | string | no | The due date of the milestone (`YYYYMMDD`) |
+| `start_date` | string | no | The start date of the milestone (`YYYYMMDD`) |
## Edit milestone
@@ -107,8 +107,8 @@ Parameters:
| `milestone_id` | integer | yes | The ID of the project's milestone |
| `title` | string | no | The title of a milestone |
| `description` | string | no | The description of the milestone |
-| `due_date` | string | no | The due date of the milestone |
-| `start_date` | string | no | The start date of the milestone |
+| `due_date` | string | no | The due date of the milestone (`YYYYMMDD`) |
+| `start_date` | string | no | The start date of the milestone (`YYYYMMDD`) |
| `state_event` | string | no | The state event of the milestone (close or activate) |
## Delete project milestone
diff --git a/doc/api/oauth2.md b/doc/api/oauth2.md
index aca6ee74b15..371e3f9ae47 100644
--- a/doc/api/oauth2.md
+++ b/doc/api/oauth2.md
@@ -63,7 +63,7 @@ For a list of scopes in GitLab, see [the provider documentation](../integration/
### Prevent CSRF attacks
-To [protect redirect-based flows](https://tools.ietf.org/id/draft-ietf-oauth-security-topics-13.html#rec_redirect),
+To [protect redirect-based flows](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-security-topics-13#section-3.1),
the OAuth specification recommends the use of "One-time use CSRF tokens carried in the state
parameter, which are securely bound to the user agent", with each request to the
`/oauth/authorize` endpoint. This can prevent
diff --git a/doc/api/openapi/openapi.yaml b/doc/api/openapi/openapi.yaml
index a3a0485428d..9ee2b8119be 100644
--- a/doc/api/openapi/openapi.yaml
+++ b/doc/api/openapi/openapi.yaml
@@ -43,35 +43,657 @@ components:
paths:
# METADATA
/v4/metadata:
- $ref: 'v4/metadata.yaml'
+ $ref: '#/metadata'
# VERSION
/v4/version:
- $ref: 'v4/version.yaml'
+ $ref: '#/version'
# ACCESS REQUESTS (PROJECTS)
/v4/projects/{id}/access_requests:
- $ref: 'v4/access_requests.yaml#/accessRequestsProjects'
+ $ref: '#/accessRequestsProjects'
/v4/projects/{id}/access_requests/{user_id}/approve:
- $ref: 'v4/access_requests.yaml#/accessRequestsProjectsApprove'
+ $ref: '#/accessRequestsProjectsApprove'
/v4/projects/{id}/access_requests/{user_id}:
- $ref: 'v4/access_requests.yaml#/accessRequestsProjectsDeny'
+ $ref: '#/accessRequestsProjectsDeny'
# ACCESS REQUESTS (GROUPS)
/v4/groups/{id}/access_requests:
- $ref: 'v4/access_requests.yaml#/accessRequestsGroups'
+ $ref: '#/accessRequestsGroups'
/v4/groups/{id}/access_requests/{user_id}/approve:
- $ref: 'v4/access_requests.yaml#/accessRequestsGroupsApprove'
+ $ref: '#/accessRequestsGroupsApprove'
/v4/groups/{id}/access_requests/{user_id}:
- $ref: 'v4/access_requests.yaml#/accessRequestsGroupsDeny'
+ $ref: '#/accessRequestsGroupsDeny'
# ACCESS REQUESTS (PROJECTS)
/v4/projects/{id}/access_tokens:
- $ref: 'v4/access_tokens.yaml#/accessTokens'
+ $ref: '#/accessTokens'
/v4/projects/{id}/access_tokens/{token_id}:
- $ref: 'v4/access_tokens.yaml#/accessTokensRevoke'
+ $ref: '#/accessTokensRevoke'
+
+metadata:
+ get:
+ tags:
+ - metadata
+ summary: 'Retrieve metadata information for this GitLab instance.'
+ operationId: 'getMetadata'
+ responses:
+ '401':
+ description: 'unauthorized operation'
+ '200':
+ description: 'successful operation'
+ content:
+ 'application/json':
+ schema:
+ title: 'MetadataResponse'
+ type: 'object'
+ properties:
+ version:
+ type: 'string'
+ revision:
+ type: 'string'
+ kas:
+ type: 'object'
+ properties:
+ enabled:
+ type: 'boolean'
+ externalUrl:
+ type: 'string'
+ nullable: true
+ version:
+ type: 'string'
+ nullable: true
+ examples:
+ Example:
+ value:
+ version: '15.0-pre'
+ revision: 'c401a659d0c'
+ kas:
+ enabled: true
+ externalUrl: 'grpc://gitlab.example.com:8150'
+ version: '15.0.0'
+
+version:
+ get:
+ tags:
+ - version
+ summary: 'Retrieve version information for this GitLab instance.'
+ operationId: 'getVersion'
+ responses:
+ '401':
+ description: 'unauthorized operation'
+ '200':
+ description: 'successful operation'
+ content:
+ 'application/json':
+ schema:
+ title: 'VersionResponse'
+ type: 'object'
+ properties:
+ version:
+ type: 'string'
+ revision:
+ type: 'string'
+ examples:
+ Example:
+ value:
+ version: '13.3.0-pre'
+ revision: 'f2b05afebb0'
+
+#/v4/projects/{id}/access_requests
+accessRequestsProjects:
+ get:
+ description: Lists access requests for a project
+ summary: List access requests for a project
+ operationId: accessRequestsProjects_get
+ tags:
+ - access_requests
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the project owned by the authenticated user.
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ responses:
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+ content:
+ application/json:
+ schema:
+ title: ProjectAccessResponse
+ type: object
+ properties:
+ id:
+ type: integer
+ usename:
+ type: string
+ name:
+ type: string
+ state:
+ type: string
+ created_at:
+ type: string
+ requested_at:
+ type: string
+ example:
+ - 'id': 1
+ 'username': 'raymond_smith'
+ 'name': 'Raymond Smith'
+ 'state': 'active'
+ 'created_at': '2012-10-22T14:13:35Z'
+ 'requested_at': '2012-10-22T14:13:35Z'
+ - 'id': 2
+ 'username': 'john_doe'
+ 'name': 'John Doe'
+ 'state': 'active'
+ 'created_at': '2012-10-22T14:13:35Z'
+ 'requested_at': '2012-10-22T14:13:35Z'
+ post:
+ description: Requests access for the authenticated user to a project
+ summary: Requests access for the authenticated user to a project
+ operationId: accessRequestsProjects_post
+ tags:
+ - access_requests
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the project owned by the authenticated user.
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ responses:
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+ content:
+ application/json:
+ schema:
+ title: ProjectAccessRequest
+ type: object
+ properties:
+ id:
+ type: integer
+ usename:
+ type: string
+ name:
+ type: string
+ state:
+ type: string
+ created_at:
+ type: string
+ requested_at:
+ type: string
+ example:
+ 'id': 1
+ 'username': 'raymond_smith'
+ 'name': 'Raymond Smith'
+ 'state': 'active'
+ 'created_at': '2012-10-22T14:13:35Z'
+ 'requested_at': '2012-10-22T14:13:35Z'
+
+#/v4/projects/{id}/access_requests/{user_id}/approve
+accessRequestsProjectsApprove:
+ put:
+ description: Approves access for the authenticated user to a project
+ summary: Approves access for the authenticated user to a project
+ operationId: accessRequestsProjectsApprove_put
+ tags:
+ - access_requests
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the project owned by the authenticated user.
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ - name: user_id
+ in: path
+ description: The userID of the access requester
+ required: true
+ schema:
+ type: integer
+ - name: access_level
+ in: query
+ description: A valid project access level. 0 = no access , 10 = guest, 20 = reporter, 30 = developer, 40 = Maintainer. Default is 30.'
+ required: false
+ schema:
+ enum: [0, 10, 20, 30, 40]
+ default: 30
+ type: integer
+ responses:
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+ content:
+ application/json:
+ schema:
+ title: ProjectAccessApprove
+ type: object
+ properties:
+ id:
+ type: integer
+ usename:
+ type: string
+ name:
+ type: string
+ state:
+ type: string
+ created_at:
+ type: string
+ access_level:
+ type: integer
+ example:
+ 'id': 1
+ 'username': 'raymond_smith'
+ 'name': 'Raymond Smith'
+ 'state': 'active'
+ 'created_at': '2012-10-22T14:13:35Z'
+ 'access_level': 20
+
+#/v4/projects/{id}/access_requests/{user_id}
+accessRequestsProjectsDeny:
+ delete:
+ description: Denies a project access request for the given user
+ summary: Denies a project access request for the given user
+ operationId: accessRequestProjectsDeny_delete
+ tags:
+ - access_requests
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the project owned by the authenticated user.
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ - name: user_id
+ in: path
+ description: The user ID of the access requester
+ required: true
+ schema:
+ type: integer
+ responses: # Does anything go here? Markdown doc does not list a response.
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+
+#/v4/groups/{id}/access_requests
+accessRequestsGroups:
+ get:
+ description: List access requests for a group
+ summary: List access requests for a group
+ operationId: accessRequestsGroups_get
+ tags:
+ - access_requests
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the group owned by the authenticated user.
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ responses:
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+ content:
+ application/json:
+ schema:
+ title: GroupAccessResponse
+ type: object
+ properties:
+ id:
+ type: integer
+ usename:
+ type: string
+ name:
+ type: string
+ state:
+ type: string
+ created_at:
+ type: string
+ requested_at:
+ type: string
+ example:
+ - 'id': 1
+ 'username': 'raymond_smith'
+ 'name': 'Raymond Smith'
+ 'state': 'active'
+ 'created_at': '2012-10-22T14:13:35Z'
+ 'requested_at': '2012-10-22T14:13:35Z'
+ - 'id': 2
+ 'username': 'john_doe'
+ 'name': 'John Doe'
+ 'state': 'active'
+ 'created_at': '2012-10-22T14:13:35Z'
+ 'requested_at': '2012-10-22T14:13:35Z'
+ post:
+ description: Requests access for the authenticated user to a group
+ summary: Requests access for the authenticated user to a group
+ operationId: accessRequestsGroups_post
+ tags:
+ - access_requests
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the group owned by the authenticated user.
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ responses:
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+ content:
+ application/json:
+ schema:
+ title: GroupAccessRequest
+ type: object
+ properties:
+ id:
+ type: integer
+ usename:
+ type: string
+ name:
+ type: string
+ state:
+ type: string
+ created_at:
+ type: string
+ requested_at:
+ type: string
+ example:
+ 'id': 1
+ 'username': 'raymond_smith'
+ 'name': 'Raymond Smith'
+ 'state': 'active'
+ 'created_at': '2012-10-22T14:13:35Z'
+ 'requested_at': '2012-10-22T14:13:35Z'
+
+#/v4/groups/{id}/access_requests/{user_id}/approve
+accessRequestsGroupsApprove:
+ put:
+ description: Approves access for the authenticated user to a group
+ summary: Approves access for the authenticated user to a group
+ operationId: accessRequestsGroupsApprove_put
+ tags:
+ - access_requests
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the group owned by the authenticated user.
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ - name: user_id
+ in: path
+ description: The userID of the access requester
+ required: true
+ schema:
+ type: integer
+ - name: access_level
+ in: query
+ description: A valid group access level. 0 = no access , 10 = Guest, 20 = Reporter, 30 = Developer, 40 = Maintainer, 50 = Owner. Default is 30.
+ required: false
+ schema:
+ enum: [0, 10, 20, 30, 40, 50]
+ default: 30
+ type: integer
+ responses:
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+ content:
+ application/json:
+ schema:
+ title: GroupAccessApprove
+ type: object
+ properties:
+ id:
+ type: integer
+ usename:
+ type: string
+ name:
+ type: string
+ state:
+ type: string
+ created_at:
+ type: string
+ access_level:
+ type: integer
+ example:
+ 'id': 1
+ 'username': 'raymond_smith'
+ 'name': 'Raymond Smith'
+ 'state': 'active'
+ 'created_at': '2012-10-22T14:13:35Z'
+ 'access_level': 20
+
+#/v4/groups/{id}/access_requests/{user_id}
+accessRequestsGroupsDeny:
+ delete:
+ description: Denies a group access request for the given user
+ summary: Denies a group access request for the given user
+ operationId: accessRequestsGroupsDeny_delete
+ tags:
+ - access_requests
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the group owned by the authenticated user.
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ - name: user_id
+ in: path
+ description: The userID of the access requester
+ required: true
+ schema:
+ type: integer
+ responses: # Does anything go here? Markdown doc does not list a response.
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+#/v4/projects/{id}/access_tokens
+accessTokens:
+ get:
+ description: Lists access tokens for a project
+ summary: List access tokens for a project
+ operationId: accessTokens_get
+ tags:
+ - access_tokens
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the project
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ responses:
+ '404':
+ description: Not Found
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+ content:
+ application/json:
+ schema:
+ title: AccessTokenList
+ type: object
+ properties:
+ user_id:
+ type: integer
+ scopes:
+ type: array
+ name:
+ type: string
+ expires_at:
+ type: date
+ id:
+ type: integer
+ active:
+ type: boolean
+ created_at:
+ type: date
+ revoked:
+ type: boolean
+ example:
+ 'user_id': 141
+ 'scopes': ['api']
+ 'name': 'token'
+ 'expires_at': '2022-01-31'
+ 'id': 42
+ 'active': true
+ 'created_at': '2021-01-20T14:13:35Z'
+ 'revoked': false
+ post:
+ description: Creates an access token for a project
+ summary: Creates an access token for a project
+ operationId: accessTokens_post
+ tags:
+ - access_tokens
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the project
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ - name: name
+ in: query
+ description: The name of the project access token
+ required: true
+ schema:
+ type: string
+ - name: scopes
+ in: query
+ description: Defines read and write permissions for the token
+ required: true
+ schema:
+ type: array
+ items:
+ type: string
+ enum:
+ [
+ 'api',
+ 'read_api',
+ 'read_registry',
+ 'write_registry',
+ 'read_repository',
+ 'write_repository',
+ ]
+ - name: expires_at
+ in: query
+ description: Date when the token expires. Time of day is Midnight UTC of that date.
+ required: false
+ schema:
+ type: date
+ responses:
+ '404':
+ description: Not Found
+ '401':
+ description: Unauthorized operation
+ '200':
+ description: Successful operation
+ content:
+ application/json:
+ schema:
+ title: AccessTokenList
+ type: object
+ properties:
+ user_id:
+ type: integer
+ scopes:
+ type: array
+ name:
+ type: string
+ expires_at:
+ type: date
+ id:
+ type: integer
+ active:
+ type: boolean
+ created_at:
+ type: date
+ revoked:
+ type: boolean
+ token:
+ type: string
+ example:
+ 'user_id': 166
+ 'scopes': ['api', 'read_repository']
+ 'name': 'test'
+ 'expires_at': '2022-01-31'
+ 'id': 58
+ 'active': true
+ 'created_at': '2021-01-20T14:13:35Z'
+ 'revoked': false
+ 'token': 'D4y...Wzr'
+
+#/v4/projects/{id}/access_tokens/{token_id}
+accessTokensRevoke:
+ delete:
+ description: Revokes an access token
+ summary: Revokes an access token
+ operationId: accessTokens_delete
+ tags:
+ - access_tokens
+ parameters:
+ - name: id
+ in: path
+ description: The ID or URL-encoded path of the project
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ - name: token_id
+ in: path
+ description: The ID of the project access token
+ required: true
+ schema:
+ oneOf:
+ - type: integer
+ - type: string
+ responses:
+ '400':
+ description: Bad Request
+ '404':
+ description: Not Found
+ '204':
+ description: No content if successfully revoked
diff --git a/doc/api/openapi/openapi_v2.yaml b/doc/api/openapi/openapi_v2.yaml
index 59b21ecd048..5a99f6c8793 100644
--- a/doc/api/openapi/openapi_v2.yaml
+++ b/doc/api/openapi/openapi_v2.yaml
@@ -16,9 +16,295 @@ securityDefinitions:
in: query
host: gitlab.com
tags:
+- name: user_counts
+ description: Operations about user_counts
- name: metadata
description: Operations related to metadata of the GitLab instance
+- name: access_requests
+ description: Operations related to access requests
paths:
+ "/api/v4/groups/{id}/access_requests/{user_id}":
+ delete:
+ summary: Denies an access request for the given user.
+ description: This feature was introduced in GitLab 8.11.
+ produces:
+ - application/json
+ parameters:
+ - in: path
+ name: id
+ description: The ID or URL-encoded path of the group owned by the authenticated
+ user
+ type: string
+ required: true
+ - in: path
+ name: user_id
+ description: The user ID of the access requester
+ type: integer
+ format: int32
+ required: true
+ responses:
+ '204':
+ description: Denies an access request for the given user.
+ tags:
+ - access_requests
+ operationId: deleteApiV4GroupsIdAccessRequestsUserId
+ "/api/v4/groups/{id}/access_requests/{user_id}/approve":
+ put:
+ summary: Approves an access request for the given user.
+ description: This feature was introduced in GitLab 8.11.
+ produces:
+ - application/json
+ consumes:
+ - application/json
+ parameters:
+ - in: path
+ name: id
+ description: The ID or URL-encoded path of the group owned by the authenticated
+ user
+ type: string
+ required: true
+ - in: path
+ name: user_id
+ description: The user ID of the access requester
+ type: integer
+ format: int32
+ required: true
+ - in: formData
+ name: access_level
+ description: 'A valid access level (defaults: `30`, the Developer role)'
+ type: integer
+ format: int32
+ default: 30
+ required: false
+ responses:
+ '200':
+ description: successful operation
+ schema:
+ "$ref": "#/definitions/API_Entities_AccessRequester"
+ examples:
+ successfull_response:
+ id: 1
+ username: raymond_smith
+ name: Raymond Smith
+ state: active
+ created_at: '2012-10-22T14:13:35Z'
+ access_level: 20
+ tags:
+ - access_requests
+ operationId: putApiV4GroupsIdAccessRequestsUserIdApprove
+ "/api/v4/groups/{id}/access_requests":
+ post:
+ summary: Requests access for the authenticated user to a group.
+ description: This feature was introduced in GitLab 8.11.
+ produces:
+ - application/json
+ consumes:
+ - application/json
+ parameters:
+ - in: path
+ name: id
+ description: The ID or URL-encoded path of the group owned by the authenticated
+ user
+ type: string
+ required: true
+ responses:
+ '200':
+ description: successful operation
+ schema:
+ "$ref": "#/definitions/API_Entities_AccessRequester"
+ examples:
+ successfull_response:
+ id: 1
+ username: raymond_smith
+ name: Raymond Smith
+ state: active
+ created_at: '2012-10-22T14:13:35Z'
+ access_level: 20
+ tags:
+ - access_requests
+ operationId: postApiV4GroupsIdAccessRequests
+ get:
+ summary: Gets a list of access requests for a group.
+ description: This feature was introduced in GitLab 8.11.
+ produces:
+ - application/json
+ parameters:
+ - in: path
+ name: id
+ description: The ID or URL-encoded path of the group owned by the authenticated
+ user
+ type: string
+ required: true
+ - in: query
+ name: page
+ description: Current page number
+ type: integer
+ format: int32
+ default: 1
+ required: false
+ - in: query
+ name: per_page
+ description: Number of items per page
+ type: integer
+ format: int32
+ default: 20
+ required: false
+ responses:
+ '200':
+ description: Gets a list of access requests for a group.
+ schema:
+ "$ref": "#/definitions/API_Entities_AccessRequester"
+ tags:
+ - access_requests
+ operationId: getApiV4GroupsIdAccessRequests
+ "/api/v4/projects/{id}/access_requests/{user_id}":
+ delete:
+ summary: Denies an access request for the given user.
+ description: This feature was introduced in GitLab 8.11.
+ produces:
+ - application/json
+ parameters:
+ - in: path
+ name: id
+ description: The ID or URL-encoded path of the project owned by the authenticated
+ user
+ type: string
+ required: true
+ - in: path
+ name: user_id
+ description: The user ID of the access requester
+ type: integer
+ format: int32
+ required: true
+ responses:
+ '204':
+ description: Denies an access request for the given user.
+ tags:
+ - access_requests
+ operationId: deleteApiV4ProjectsIdAccessRequestsUserId
+ "/api/v4/projects/{id}/access_requests/{user_id}/approve":
+ put:
+ summary: Approves an access request for the given user.
+ description: This feature was introduced in GitLab 8.11.
+ produces:
+ - application/json
+ consumes:
+ - application/json
+ parameters:
+ - in: path
+ name: id
+ description: The ID or URL-encoded path of the project owned by the authenticated
+ user
+ type: string
+ required: true
+ - in: path
+ name: user_id
+ description: The user ID of the access requester
+ type: integer
+ format: int32
+ required: true
+ - in: formData
+ name: access_level
+ description: 'A valid access level (defaults: `30`, the Developer role)'
+ type: integer
+ format: int32
+ default: 30
+ required: false
+ responses:
+ '200':
+ description: successful operation
+ schema:
+ "$ref": "#/definitions/API_Entities_AccessRequester"
+ examples:
+ successfull_response:
+ id: 1
+ username: raymond_smith
+ name: Raymond Smith
+ state: active
+ created_at: '2012-10-22T14:13:35Z'
+ access_level: 20
+ tags:
+ - access_requests
+ operationId: putApiV4ProjectsIdAccessRequestsUserIdApprove
+ "/api/v4/projects/{id}/access_requests":
+ post:
+ summary: Requests access for the authenticated user to a project.
+ description: This feature was introduced in GitLab 8.11.
+ produces:
+ - application/json
+ consumes:
+ - application/json
+ parameters:
+ - in: path
+ name: id
+ description: The ID or URL-encoded path of the project owned by the authenticated
+ user
+ type: string
+ required: true
+ responses:
+ '200':
+ description: successful operation
+ schema:
+ "$ref": "#/definitions/API_Entities_AccessRequester"
+ examples:
+ successfull_response:
+ id: 1
+ username: raymond_smith
+ name: Raymond Smith
+ state: active
+ created_at: '2012-10-22T14:13:35Z'
+ access_level: 20
+ tags:
+ - access_requests
+ operationId: postApiV4ProjectsIdAccessRequests
+ get:
+ summary: Gets a list of access requests for a project.
+ description: This feature was introduced in GitLab 8.11.
+ produces:
+ - application/json
+ parameters:
+ - in: path
+ name: id
+ description: The ID or URL-encoded path of the project owned by the authenticated
+ user
+ type: string
+ required: true
+ - in: query
+ name: page
+ description: Current page number
+ type: integer
+ format: int32
+ default: 1
+ required: false
+ - in: query
+ name: per_page
+ description: Number of items per page
+ type: integer
+ format: int32
+ default: 20
+ required: false
+ responses:
+ '200':
+ description: Gets a list of access requests for a project.
+ schema:
+ "$ref": "#/definitions/API_Entities_AccessRequester"
+ tags:
+ - access_requests
+ operationId: getApiV4ProjectsIdAccessRequests
+ "/api/v4/user_counts":
+ get:
+ summary: Return the user specific counts
+ description: Assigned open issues, assigned MRs and pending todos count
+ produces:
+ - application/json
+ responses:
+ '200':
+ description: Return the user specific counts
+ schema:
+ "$ref": "#/definitions/API_Entities_UserCounts"
+ tags:
+ - user_counts
+ operationId: getApiV4UserCounts
"/api/v4/metadata":
get:
summary: Retrieve metadata information for this GitLab instance.
@@ -71,6 +357,63 @@ paths:
- metadata
operationId: getApiV4Version
definitions:
+ API_Entities_AccessRequester:
+ type: object
+ properties:
+ id:
+ type: string
+ username:
+ type: string
+ name:
+ type: string
+ state:
+ type: string
+ avatar_url:
+ type: string
+ avatar_path:
+ type: string
+ custom_attributes:
+ "$ref": "#/definitions/API_Entities_CustomAttribute"
+ web_url:
+ type: string
+ is_gitlab_employee:
+ type: string
+ email:
+ type: string
+ requested_at:
+ type: string
+ description: API_Entities_AccessRequester model
+ API_Entities_CustomAttribute:
+ type: object
+ properties:
+ key:
+ type: string
+ value:
+ type: string
+ API_Entities_UserCounts:
+ type: object
+ properties:
+ merge_requests:
+ type: integer
+ format: int32
+ example: 10
+ assigned_issues:
+ type: integer
+ format: int32
+ example: 10
+ assigned_merge_requests:
+ type: integer
+ format: int32
+ example: 10
+ review_requested_merge_requests:
+ type: integer
+ format: int32
+ example: 10
+ todos:
+ type: integer
+ format: int32
+ example: 10
+ description: API_Entities_UserCounts model
API_Entities_Metadata:
type: object
properties:
diff --git a/doc/api/openapi/v4/access_requests.yaml b/doc/api/openapi/v4/access_requests.yaml
deleted file mode 100644
index 157a0973e1e..00000000000
--- a/doc/api/openapi/v4/access_requests.yaml
+++ /dev/null
@@ -1,381 +0,0 @@
-# Markdown documentation: https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/api/access_requests.md
-
-#/v4/projects/{id}/access_requests
-accessRequestsProjects:
- get:
- description: Lists access requests for a project
- summary: List access requests for a project
- operationId: accessRequestsProjects_get
- tags:
- - access_requests
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the project owned by the authenticated user.
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- responses:
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
- content:
- application/json:
- schema:
- title: ProjectAccessResponse
- type: object
- properties:
- id:
- type: integer
- usename:
- type: string
- name:
- type: string
- state:
- type: string
- created_at:
- type: string
- requested_at:
- type: string
- example:
- - "id": 1
- "username": "raymond_smith"
- "name": "Raymond Smith"
- "state": "active"
- "created_at": "2012-10-22T14:13:35Z"
- "requested_at": "2012-10-22T14:13:35Z"
- - "id": 2
- "username": "john_doe"
- "name": "John Doe"
- "state": "active"
- "created_at": "2012-10-22T14:13:35Z"
- "requested_at": "2012-10-22T14:13:35Z"
- post:
- description: Requests access for the authenticated user to a project
- summary: Requests access for the authenticated user to a project
- operationId: accessRequestsProjects_post
- tags:
- - access_requests
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the project owned by the authenticated user.
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- responses:
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
- content:
- application/json:
- schema:
- title: ProjectAccessRequest
- type: object
- properties:
- id:
- type: integer
- usename:
- type: string
- name:
- type: string
- state:
- type: string
- created_at:
- type: string
- requested_at:
- type: string
- example:
- "id": 1
- "username": "raymond_smith"
- "name": "Raymond Smith"
- "state": "active"
- "created_at": "2012-10-22T14:13:35Z"
- "requested_at": "2012-10-22T14:13:35Z"
-
-#/v4/projects/{id}/access_requests/{user_id}/approve
-accessRequestsProjectsApprove:
- put:
- description: Approves access for the authenticated user to a project
- summary: Approves access for the authenticated user to a project
- operationId: accessRequestsProjectsApprove_put
- tags:
- - access_requests
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the project owned by the authenticated user.
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- - name: user_id
- in: path
- description: The userID of the access requester
- required: true
- schema:
- type: integer
- - name: access_level
- in: query
- description: A valid project access level. 0 = no access , 10 = guest, 20 = reporter, 30 = developer, 40 = Maintainer. Default is 30.'
- required: false
- schema:
- enum: [0, 10, 20, 30, 40]
- default: 30
- type: integer
- responses:
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
- content:
- application/json:
- schema:
- title: ProjectAccessApprove
- type: object
- properties:
- id:
- type: integer
- usename:
- type: string
- name:
- type: string
- state:
- type: string
- created_at:
- type: string
- access_level:
- type: integer
- example:
- "id": 1
- "username": "raymond_smith"
- "name": "Raymond Smith"
- "state": "active"
- "created_at": "2012-10-22T14:13:35Z"
- "access_level": 20
-
-#/v4/projects/{id}/access_requests/{user_id}
-accessRequestsProjectsDeny:
- delete:
- description: Denies a project access request for the given user
- summary: Denies a project access request for the given user
- operationId: accessRequestProjectsDeny_delete
- tags:
- - access_requests
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the project owned by the authenticated user.
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- - name: user_id
- in: path
- description: The user ID of the access requester
- required: true
- schema:
- type: integer
- responses: # Does anything go here? Markdown doc does not list a response.
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
-
-#/v4/groups/{id}/access_requests
-accessRequestsGroups:
- get:
- description: List access requests for a group
- summary: List access requests for a group
- operationId: accessRequestsGroups_get
- tags:
- - access_requests
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the group owned by the authenticated user.
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- responses:
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
- content:
- application/json:
- schema:
- title: GroupAccessResponse
- type: object
- properties:
- id:
- type: integer
- usename:
- type: string
- name:
- type: string
- state:
- type: string
- created_at:
- type: string
- requested_at:
- type: string
- example:
- - "id": 1
- "username": "raymond_smith"
- "name": "Raymond Smith"
- "state": "active"
- "created_at": "2012-10-22T14:13:35Z"
- "requested_at": "2012-10-22T14:13:35Z"
- - "id": 2
- "username": "john_doe"
- "name": "John Doe"
- "state": "active"
- "created_at": "2012-10-22T14:13:35Z"
- "requested_at": "2012-10-22T14:13:35Z"
- post:
- description: Requests access for the authenticated user to a group
- summary: Requests access for the authenticated user to a group
- operationId: accessRequestsGroups_post
- tags:
- - access_requests
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the group owned by the authenticated user.
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- responses:
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
- content:
- application/json:
- schema:
- title: GroupAccessRequest
- type: object
- properties:
- id:
- type: integer
- usename:
- type: string
- name:
- type: string
- state:
- type: string
- created_at:
- type: string
- requested_at:
- type: string
- example:
- "id": 1
- "username": "raymond_smith"
- "name": "Raymond Smith"
- "state": "active"
- "created_at": "2012-10-22T14:13:35Z"
- "requested_at": "2012-10-22T14:13:35Z"
-
-#/v4/groups/{id}/access_requests/{user_id}/approve
-accessRequestsGroupsApprove:
- put:
- description: Approves access for the authenticated user to a group
- summary: Approves access for the authenticated user to a group
- operationId: accessRequestsGroupsApprove_put
- tags:
- - access_requests
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the group owned by the authenticated user.
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- - name: user_id
- in: path
- description: The userID of the access requester
- required: true
- schema:
- type: integer
- - name: access_level
- in: query
- description: A valid group access level. 0 = no access , 10 = Guest, 20 = Reporter, 30 = Developer, 40 = Maintainer, 50 = Owner. Default is 30.
- required: false
- schema:
- enum: [0, 10, 20, 30, 40, 50]
- default: 30
- type: integer
- responses:
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
- content:
- application/json:
- schema:
- title: GroupAccessApprove
- type: object
- properties:
- id:
- type: integer
- usename:
- type: string
- name:
- type: string
- state:
- type: string
- created_at:
- type: string
- access_level:
- type: integer
- example:
- "id": 1
- "username": "raymond_smith"
- "name": "Raymond Smith"
- "state": "active"
- "created_at": "2012-10-22T14:13:35Z"
- "access_level": 20
-
-#/v4/groups/{id}/access_requests/{user_id}
-accessRequestsGroupsDeny:
- delete:
- description: Denies a group access request for the given user
- summary: Denies a group access request for the given user
- operationId: accessRequestsGroupsDeny_delete
- tags:
- - access_requests
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the group owned by the authenticated user.
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- - name: user_id
- in: path
- description: The userID of the access requester
- required: true
- schema:
- type: integer
- responses: # Does anything go here? Markdown doc does not list a response.
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
diff --git a/doc/api/openapi/v4/access_tokens.yaml b/doc/api/openapi/v4/access_tokens.yaml
deleted file mode 100644
index 9a1a6960eea..00000000000
--- a/doc/api/openapi/v4/access_tokens.yaml
+++ /dev/null
@@ -1,170 +0,0 @@
-# Markdown documentation: https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/api/resource_access_tokens.md
-
-#/v4/projects/{id}/access_tokens
-accessTokens:
- get:
- description: Lists access tokens for a project
- summary: List access tokens for a project
- operationId: accessTokens_get
- tags:
- - access_tokens
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the project
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- responses:
- '404':
- description: Not Found
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
- content:
- application/json:
- schema:
- title: AccessTokenList
- type: object
- properties:
- user_id:
- type: integer
- scopes:
- type: array
- name:
- type: string
- expires_at:
- type: date
- id:
- type: integer
- active:
- type: boolean
- created_at:
- type: date
- revoked:
- type: boolean
- example:
- "user_id": 141
- "scopes" : ["api"]
- "name": "token"
- "expires_at": "2022-01-31"
- "id": 42
- "active": true
- "created_at": "2021-01-20T14:13:35Z"
- "revoked" : false
- post:
- description: Creates an access token for a project
- summary: Creates an access token for a project
- operationId: accessTokens_post
- tags:
- - access_tokens
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the project
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- - name: name
- in: query
- description: The name of the project access token
- required: true
- schema:
- type: string
- - name: scopes
- in: query
- description: Defines read and write permissions for the token
- required: true
- schema:
- type: array
- items:
- type: string
- enum: ["api", "read_api", "read_registry", "write_registry", "read_repository", "write_repository"]
- - name: expires_at
- in: query
- description: Date when the token expires. Time of day is Midnight UTC of that date.
- required: false
- schema:
- type: date
- responses:
- '404':
- description: Not Found
- '401':
- description: Unauthorized operation
- '200':
- description: Successful operation
- content:
- application/json:
- schema:
- title: AccessTokenList
- type: object
- properties:
- user_id:
- type: integer
- scopes:
- type: array
- name:
- type: string
- expires_at:
- type: date
- id:
- type: integer
- active:
- type: boolean
- created_at:
- type: date
- revoked:
- type: boolean
- token:
- type: string
- example:
- "user_id": 166
- "scopes" : [
- "api",
- "read_repository"
- ]
- "name": "test"
- "expires_at": "2022-01-31"
- "id": 58
- "active": true
- "created_at": "2021-01-20T14:13:35Z"
- "revoked" : false
- "token" : "D4y...Wzr"
-
-#/v4/projects/{id}/access_tokens/{token_id}
-accessTokensRevoke:
- delete:
- description: Revokes an access token
- summary: Revokes an access token
- operationId: accessTokens_delete
- tags:
- - access_tokens
- parameters:
- - name: id
- in: path
- description: The ID or URL-encoded path of the project
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- - name: token_id
- in: path
- description: The ID of the project access token
- required: true
- schema:
- oneOf:
- - type: integer
- - type: string
- responses:
- '400':
- description: Bad Request
- '404':
- description: Not Found
- '204':
- description: No content if successfully revoked
diff --git a/doc/api/openapi/v4/metadata.yaml b/doc/api/openapi/v4/metadata.yaml
deleted file mode 100644
index 6a5ef9f3355..00000000000
--- a/doc/api/openapi/v4/metadata.yaml
+++ /dev/null
@@ -1,43 +0,0 @@
-# Markdown documentation: https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/api/metadata.md
-
-get:
- tags:
- - metadata
- summary: "Retrieve metadata information for this GitLab instance."
- operationId: "getMetadata"
- responses:
- "401":
- description: "unauthorized operation"
- "200":
- description: "successful operation"
- content:
- "application/json":
- schema:
- title: "MetadataResponse"
- type: "object"
- properties:
- version:
- type: "string"
- revision:
- type: "string"
- kas:
- type: "object"
- properties:
- enabled:
- type: "boolean"
- externalUrl:
- type: "string"
- nullable: true
- version:
- type: "string"
- nullable: true
- examples:
- Example:
- value:
- version: "15.0-pre"
- revision: "c401a659d0c"
- kas:
- enabled: true
- externalUrl: "grpc://gitlab.example.com:8150"
- version: "15.0.0"
-
diff --git a/doc/api/openapi/v4/version.yaml b/doc/api/openapi/v4/version.yaml
deleted file mode 100644
index 3a689840f4c..00000000000
--- a/doc/api/openapi/v4/version.yaml
+++ /dev/null
@@ -1,28 +0,0 @@
-# Markdown documentation: https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/api/version.md
-
-get:
- tags:
- - version
- summary: "Retrieve version information for this GitLab instance."
- operationId: "getVersion"
- responses:
- "401":
- description: "unauthorized operation"
- "200":
- description: "successful operation"
- content:
- "application/json":
- schema:
- title: "VersionResponse"
- type: "object"
- properties:
- version:
- type: "string"
- revision:
- type: "string"
- examples:
- Example:
- value:
- version: "13.3.0-pre"
- revision: "f2b05afebb0"
-
diff --git a/doc/api/packages.md b/doc/api/packages.md
index 4fd21a43a92..ffdd9fe7d11 100644
--- a/doc/api/packages.md
+++ b/doc/api/packages.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/packages/composer.md b/doc/api/packages/composer.md
index 913831776d9..e75dbfeb92f 100644
--- a/doc/api/packages/composer.md
+++ b/doc/api/packages/composer.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/packages/conan.md b/doc/api/packages/conan.md
index d0077d18c11..c37296ad664 100644
--- a/doc/api/packages/conan.md
+++ b/doc/api/packages/conan.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -494,7 +494,7 @@ Upload a recipe file to the package registry. You must use an upload URL that th
returned.
```plaintext
-GET packages/conan/v1/files/:package_name/:package_version/:package_username/:package_channel/:recipe_revision/export/:file_name
+PUT packages/conan/v1/files/:package_name/:package_version/:package_username/:package_channel/:recipe_revision/export/:file_name
```
| Attribute | Type | Required | Description |
@@ -509,7 +509,8 @@ GET packages/conan/v1/files/:package_name/:package_version/:package_username/:pa
Provide the file context in the request body:
```shell
-curl --header "Authorization: Bearer <authenticate_token>" \
+curl --request PUT \
+ --user <username>:<personal_access_token> \
--upload-file path/to/conanfile.py \
"https://gitlab.example.com/api/v4/packages/conan/v1/files/my-package/1.0/my-group+my-project/stable/0/export/conanfile.py"
```
@@ -558,7 +559,7 @@ Upload a package file to the package registry. You must use an upload URL that t
returned.
```plaintext
-GET packages/conan/v1/files/:package_name/:package_version/:package_username/:package_channel/:recipe_revision/package/:conan_package_reference/:package_revision/:file_name
+PUT packages/conan/v1/files/:package_name/:package_version/:package_username/:package_channel/:recipe_revision/package/:conan_package_reference/:package_revision/:file_name
```
| Attribute | Type | Required | Description |
@@ -575,9 +576,10 @@ GET packages/conan/v1/files/:package_name/:package_version/:package_username/:pa
Provide the file context in the request body:
```shell
-curl --header "Authorization: Bearer <authenticate_token>" \
+curl --request PUT \
+ --user <username>:<personal_access_token> \
--upload-file path/to/conaninfo.txt \
- "https://gitlab.example.com/api/v4/packages/conan/v1/files/my-package/1.0/my-group+my-project/stable/packages/103f6067a947f366ef91fc1b7da351c588d1827f/0/conaninfo.txt"
+ "https://gitlab.example.com/api/v4/packages/conan/v1/files/my-package/1.0/my-group+my-project/stable/0/package/103f6067a947f366ef91fc1b7da351c588d1827f/0/conaninfo.txt"
```
## Delete a Package (delete a Conan recipe)
diff --git a/doc/api/packages/debian.md b/doc/api/packages/debian.md
index 0e85e554f01..88baf760973 100644
--- a/doc/api/packages/debian.md
+++ b/doc/api/packages/debian.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -61,8 +61,8 @@ PUT projects/:id/packages/debian/:file_name
```shell
curl --request PUT \
+ --user <username>:<personal_access_token> \
--upload-file path/to/mypkg.deb \
- --header "Private-Token: <personal_access_token>" \
"https://gitlab.example.com/api/v4/projects/1/packages/debian/mypkg.deb"
```
diff --git a/doc/api/packages/debian_group_distributions.md b/doc/api/packages/debian_group_distributions.md
index e3fabd92c51..95f3bae2c1d 100644
--- a/doc/api/packages/debian_group_distributions.md
+++ b/doc/api/packages/debian_group_distributions.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/packages/debian_project_distributions.md b/doc/api/packages/debian_project_distributions.md
index f8ad7259866..573f1bffd1a 100644
--- a/doc/api/packages/debian_project_distributions.md
+++ b/doc/api/packages/debian_project_distributions.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/packages/go_proxy.md b/doc/api/packages/go_proxy.md
index 3a98c5acd2f..08a7138a82a 100644
--- a/doc/api/packages/go_proxy.md
+++ b/doc/api/packages/go_proxy.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/packages/helm.md b/doc/api/packages/helm.md
index b9888f2eed7..0e9a72c2389 100644
--- a/doc/api/packages/helm.md
+++ b/doc/api/packages/helm.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/packages/maven.md b/doc/api/packages/maven.md
index 6eb343dc7ab..4086b68b750 100644
--- a/doc/api/packages/maven.md
+++ b/doc/api/packages/maven.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/packages/npm.md b/doc/api/packages/npm.md
index a35fe630075..7e8732d9553 100644
--- a/doc/api/packages/npm.md
+++ b/doc/api/packages/npm.md
@@ -1,7 +1,7 @@
---
stage: Package
-group: Package
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.example/handbook/product/ux/technical-writing/#assignments
+group: Package Registry
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# npm API **(FREE)**
@@ -65,7 +65,7 @@ curl --request PUT
--header "Content-Type: application/json"
--data @./path/to/metadata/file.json
--header "Authorization: Bearer <personal_access_token>" \
- "https://gitlab.example.com/api/v4/projects/1/packages/npm/@myscope/my-pkg"
+ "https://gitlab.example.com/api/v4/projects/1/packages/npm/@myscope%2fmy-pkg"
```
The metadata file content is generated by npm, but looks something like this:
diff --git a/doc/api/packages/nuget.md b/doc/api/packages/nuget.md
index 4c63524291b..ebcb51be447 100644
--- a/doc/api/packages/nuget.md
+++ b/doc/api/packages/nuget.md
@@ -1,7 +1,7 @@
---
stage: Package
-group: Package
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about..example/handbook/product/ux/technical-writing/#assignments
+group: Package Registry
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# NuGet API **(FREE)**
@@ -97,9 +97,9 @@ PUT projects/:id/packages/nuget
```shell
curl --request PUT \
- --upload-file path/to/mynugetpkg.1.3.0.17.nupkg \
+ --form 'package=@path/to/mynugetpkg.1.3.0.17.nupkg' \
--user <username>:<personal_access_token> \
- "https://gitlab.example.com/api/v4/projects/1/packages/nuget"
+ "https://gitlab.example.com/api/v4/projects/1/packages/nuget/"
```
## Upload a symbol package file
@@ -121,7 +121,7 @@ PUT projects/:id/packages/nuget/symbolpackage
```shell
curl --request PUT \
- --upload-file path/to/mynugetpkg.1.3.0.17.snupkg \
+ --form 'package=@path/to/mynugetpkg.1.3.0.17.snupkg' \
--user <username>:<personal_access_token> \
"https://gitlab.example.com/api/v4/projects/1/packages/nuget/symbolpackage"
```
diff --git a/doc/api/packages/pypi.md b/doc/api/packages/pypi.md
index 061de5bb9dd..4e4c060d99b 100644
--- a/doc/api/packages/pypi.md
+++ b/doc/api/packages/pypi.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -264,8 +264,10 @@ PUT projects/:id/packages/pypi
| `requires_python` | string | no | The PyPI required version. |
```shell
-curl --request PUT \
- --upload-file path/to/my.pypi.package-0.0.1.tar.gz \
+curl --request POST \
+ --form 'content=@path/to/my.pypi.package-0.0.1.tar.gz' \
+ --form 'name=my.pypi.package'
+ --form 'version=1.3.7'
--user <username>:<personal_access_token> \
"https://gitlab.example.com/api/v4/projects/1/packages/pypi"
```
diff --git a/doc/api/packages/rubygems.md b/doc/api/packages/rubygems.md
index a87df17ab43..c2d05f24085 100644
--- a/doc/api/packages/rubygems.md
+++ b/doc/api/packages/rubygems.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/packages/terraform-modules.md b/doc/api/packages/terraform-modules.md
index 4c32e3f7cb4..bb5b39b5161 100644
--- a/doc/api/packages/terraform-modules.md
+++ b/doc/api/packages/terraform-modules.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/api/pipeline_triggers.md b/doc/api/pipeline_triggers.md
index 1fc29d2a654..999b223a934 100644
--- a/doc/api/pipeline_triggers.md
+++ b/doc/api/pipeline_triggers.md
@@ -170,7 +170,7 @@ Supported attributes:
| `id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. |
| `ref` | string | **{check-circle}** Yes | The branch or tag to run the pipeline on. |
| `token` | string | **{check-circle}** Yes | The trigger token or CI/CD job token. |
-| `variables` | array | **{dotted-circle}** No | An [array of hashes](index.md#array-of-hashes) containing the variables available in the pipeline, matching the structure `[{ 'key': 'UPLOAD_TO_S3', 'variable_type': 'file', 'value': 'true' }, {'key': 'TEST', 'value': 'test variable'}]`. If `variable_type` is excluded, it defaults to `env_var`. |
+| `variables` | hash | **{dotted-circle}** No | A map of key-valued strings containing the pipeline variables. For example: `{ VAR1: "value1", VAR2: "value2" }`. |
Example request:
diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md
index a44d02982c0..8242f8cff00 100644
--- a/doc/api/pipelines.md
+++ b/doc/api/pipelines.md
@@ -269,6 +269,67 @@ Sample response:
}
```
+## Get the latest pipeline
+
+Get the latest pipeline for a specific ref in a project.
+
+```plaintext
+POST /projects/:id/pipeline/latest
+```
+
+| Attribute | Type | Required | Description |
+|-------------|---------|----------|---------------------|
+| `ref` | string | no | The branch or tag to check for the latest pipeline. Defaults to the default branch when not specified. |
+
+```shell
+curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines/latest"
+```
+
+Example of response
+
+```json
+{
+ "id": 287,
+ "iid": 144,
+ "project_id": 21,
+ "sha": "50f0acb76a40e34a4ff304f7347dcc6587da8a14",
+ "ref": "main",
+ "status": "success",
+ "source": "push",
+ "created_at": "2022-09-21T01:05:07.200Z",
+ "updated_at": "2022-09-21T01:05:50.185Z",
+ "web_url": "http://127.0.0.1:3000/test-group/test-project/-/pipelines/287",
+ "before_sha": "8a24fb3c5877a6d0b611ca41fc86edc174593e2b",
+ "tag": false,
+ "yaml_errors": null,
+ "user": {
+ "id": 1,
+ "username": "root",
+ "name": "Administrator",
+ "state": "active",
+ "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon",
+ "web_url": "http://127.0.0.1:3000/root"
+ },
+ "started_at": "2022-09-21T01:05:14.197Z",
+ "finished_at": "2022-09-21T01:05:50.175Z",
+ "committed_at": null,
+ "duration": 34,
+ "queued_duration": 6,
+ "coverage": null,
+ "detailed_status": {
+ "icon": "status_success",
+ "text": "passed",
+ "label": "passed",
+ "group": "success",
+ "tooltip": "passed",
+ "has_details": false,
+ "details_path": "/test-group/test-project/-/pipelines/287",
+ "illustration": null,
+ "favicon": "/assets/ci_favicons/favicon_status_success-8451333011eee8ce9f2ab25dc487fe24a8758c694827a582f17f42b0a90446a2.png"
+ }
+}
+```
+
## Create a new pipeline
> `iid` in response [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342223) in GitLab 14.6.
diff --git a/doc/api/product_analytics.md b/doc/api/product_analytics.md
index 4653a52732a..e10327bc59b 100644
--- a/doc/api/product_analytics.md
+++ b/doc/api/product_analytics.md
@@ -4,7 +4,7 @@ group: Product Analytics
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Product analytics API
+# Product analytics API **(ULTIMATE)**
> Introduced in GitLab 15.4 [with a flag](../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default.
diff --git a/doc/api/project_level_variables.md b/doc/api/project_level_variables.md
index d900e1da78c..c6b56c53413 100644
--- a/doc/api/project_level_variables.md
+++ b/doc/api/project_level_variables.md
@@ -31,6 +31,7 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a
"value": "TEST_1",
"protected": false,
"masked": true,
+ "raw": false,
"environment_scope": "*"
},
{
@@ -39,6 +40,7 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a
"value": "TEST_2",
"protected": false,
"masked": false,
+ "raw": false,
"environment_scope": "*"
}
]
@@ -70,6 +72,7 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a
"value": "TEST_1",
"protected": false,
"masked": true,
+ "raw": false,
"environment_scope": "*"
}
```
@@ -92,6 +95,7 @@ POST /projects/:id/variables
| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` |
| `protected` | boolean | no | Whether the variable is protected. Default: `false` |
| `masked` | boolean | no | Whether the variable is masked. Default: `false` |
+| `raw` | boolean | no | Whether the variable is expandable. Default: `false` |
| `environment_scope` | string | no | The `environment_scope` of the variable. Default: `*` |
```shell
@@ -106,6 +110,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
"value": "new value",
"protected": false,
"masked": false,
+ "raw": false,
"environment_scope": "*"
}
```
@@ -127,6 +132,7 @@ PUT /projects/:id/variables/:key
| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` |
| `protected` | boolean | no | Whether the variable is protected |
| `masked` | boolean | no | Whether the variable is masked |
+| `raw` | boolean | no | Whether the variable is expandable. Default: `false` |
| `environment_scope` | string | no | The `environment_scope` of the variable |
| `filter` | hash | no | Available filters: `[environment_scope]`. See the [`filter` parameter details](#the-filter-parameter). |
@@ -142,6 +148,7 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \
"value": "updated value",
"protected": true,
"masked": false,
+ "raw": false,
"environment_scope": "*"
}
```
diff --git a/doc/api/projects.md b/doc/api/projects.md
index 470b3721b23..638af168f22 100644
--- a/doc/api/projects.md
+++ b/doc/api/projects.md
@@ -243,6 +243,7 @@ When the user is authenticated and `simple` is not set this returns something li
"suggestion_commit_message": null,
"merge_commit_template": null,
"squash_commit_template": null,
+ "issue_branch_template": "gitlab/%{id}-%{title}",
"auto_devops_enabled": false,
"auto_devops_deploy_strategy": "continuous",
"autoclose_referenced_issues": true,
@@ -421,6 +422,7 @@ GET /users/:user_id/projects
"suggestion_commit_message": null,
"merge_commit_template": null,
"squash_commit_template": null,
+ "issue_branch_template": "gitlab/%{id}-%{title}",
"marked_for_deletion_at": "2020-04-03", // Deprecated and will be removed in API v5 in favor of marked_for_deletion_on
"marked_for_deletion_on": "2020-04-03",
"statistics": {
@@ -544,6 +546,7 @@ GET /users/:user_id/projects
"suggestion_commit_message": null,
"merge_commit_template": null,
"squash_commit_template": null,
+ "issue_branch_template": "gitlab/%{id}-%{title}",
"statistics": {
"commit_count": 12,
"storage_size": 2066080,
@@ -677,6 +680,7 @@ Example response:
"suggestion_commit_message": null,
"merge_commit_template": null,
"squash_commit_template": null,
+ "issue_branch_template": "gitlab/%{id}-%{title}",
"statistics": {
"commit_count": 37,
"storage_size": 1038090,
@@ -793,6 +797,7 @@ Example response:
"suggestion_commit_message": null,
"merge_commit_template": null,
"squash_commit_template": null,
+ "issue_branch_template": "gitlab/%{id}-%{title}",
"statistics": {
"commit_count": 12,
"storage_size": 2066080,
@@ -969,6 +974,7 @@ GET /projects/:id
"enforce_auth_checks_on_uploads": true,
"merge_commit_template": null,
"squash_commit_template": null,
+ "issue_branch_template": "gitlab/%{id}-%{title}",
"marked_for_deletion_at": "2020-04-03", // Deprecated and will be removed in API v5 in favor of marked_for_deletion_on
"marked_for_deletion_on": "2020-04-03",
"compliance_frameworks": [ "sox" ],
@@ -1334,6 +1340,7 @@ POST /projects/user/:user_id
| `shared_runners_enabled` | boolean | **{dotted-circle}** No | Enable shared runners for this project. |
| `snippets_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. |
| `snippets_enabled` | boolean | **{dotted-circle}** No | _(Deprecated)_ Enable snippets for this project. Use `snippets_access_level` instead. |
+| `issue_branch_template` | string | **{dotted-circle}** No | Template used to suggest names for [branches created from issues](../user/project/repository/web_editor.md#create-a-new-branch-from-an-issue). _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21243) in GitLab 15.6.)_ |
| `squash_commit_template` | string | **{dotted-circle}** No | [Template](../user/project/merge_requests/commit_templates.md) used to create squash commit message in merge requests. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345275) in GitLab 14.6.)_ |
| `squash_option` | string | **{dotted-circle}** No | One of `never`, `always`, `default_on`, or `default_off`. |
| `suggestion_commit_message` | string | **{dotted-circle}** No | The commit message used to apply merge request [suggestions](../user/project/merge_requests/reviews/suggestions.md). |
@@ -1385,7 +1392,7 @@ Supported attributes:
| `builds_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. |
| `ci_config_path` | string | **{dotted-circle}** No | The path to CI configuration file. |
| `ci_default_git_depth` | integer | **{dotted-circle}** No | Default number of revisions for [shallow cloning](../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone). |
-| `ci_forward_deployment_enabled` | boolean | **{dotted-circle}** No | When a new deployment job starts, [skip older deployment jobs](../ci/pipelines/settings.md#skip-outdated-deployment-jobs) that are still pending |
+| `ci_forward_deployment_enabled` | boolean | **{dotted-circle}** No | Enable or disable [prevent outdated deployment jobs](../ci/pipelines/settings.md#prevent-outdated-deployment-jobs). |
| `ci_allow_fork_pipelines_to_run_in_parent_project` | boolean | **{dotted-circle}** No | Enable or disable [running pipelines in the parent project for merge requests from forks](../ci/pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project). _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325189) in GitLab 15.3.)_ |
| `ci_separated_caches` | boolean | **{dotted-circle}** No | Set whether or not caches should be [separated](../ci/caching/index.md#cache-key-names) by branch protection status. |
| `container_expiration_policy_attributes` | hash | **{dotted-circle}** No | Update the image cleanup policy for this project. Accepts: `cadence` (string), `keep_n` (integer), `older_than` (string), `name_regex` (string), `name_regex_delete` (string), `name_regex_keep` (string), `enabled` (boolean). |
@@ -1439,6 +1446,7 @@ Supported attributes:
| `shared_runners_enabled` | boolean | **{dotted-circle}** No | Enable shared runners for this project. |
| `snippets_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. |
| `snippets_enabled` | boolean | **{dotted-circle}** No | _(Deprecated)_ Enable snippets for this project. Use `snippets_access_level` instead. |
+| `issue_branch_template` | string | **{dotted-circle}** No | Template used to suggest names for [branches created from issues](../user/project/repository/web_editor.md#create-a-new-branch-from-an-issue). _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21243) in GitLab 15.6.)_ |
| `squash_commit_template` | string | **{dotted-circle}** No | [Template](../user/project/merge_requests/commit_templates.md) used to create squash commit message in merge requests. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345275) in GitLab 14.6.)_ |
| `squash_option` | string | **{dotted-circle}** No | One of `never`, `always`, `default_on`, or `default_off`. |
| `suggestion_commit_message` | string | **{dotted-circle}** No | The commit message used to apply merge request suggestions. |
@@ -2830,6 +2838,42 @@ Read more in the [Project members](members.md) documentation.
Read more in the [Project vulnerabilities](project_vulnerabilities.md) documentation.
+## Get a project's pull mirror details **(PREMIUM)**
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354506) in GitLab 15.5.
+
+Returns the details of the project's pull mirror.
+
+```plaintext
+GET /projects/:id/mirror/pull
+```
+
+Supported attributes:
+
+| Attribute | Type | Required | Description |
+|:----------|:------|:------------|:------------|
+| `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding). |
+
+Example request:
+
+```shell
+curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/:id/mirror/pull"
+```
+
+Example response:
+
+```json
+{
+ "id": 101486,
+ "last_error": null,
+ "last_successful_update_at": "2020-01-06T17:32:02.823Z",
+ "last_update_at": "2020-01-06T17:32:02.823Z",
+ "last_update_started_at": "2020-01-06T17:31:55.864Z",
+ "update_status": "finished",
+ "url": "https://*****:*****@gitlab.com/gitlab-org/security/gitlab.git"
+}
+```
+
## Configure pull mirroring for a project **(PREMIUM)**
> Moved to GitLab Premium in 13.9.
diff --git a/doc/api/protected_branches.md b/doc/api/protected_branches.md
index 620cb0e0bae..96bd5c15f13 100644
--- a/doc/api/protected_branches.md
+++ b/doc/api/protected_branches.md
@@ -43,12 +43,14 @@ Example response:
"name": "master",
"push_access_levels": [
{
+ "id": 1,
"access_level": 40,
"access_level_description": "Maintainers"
}
],
"merge_access_levels": [
{
+ "id": 1,
"access_level": 40,
"access_level_description": "Maintainers"
}
@@ -61,12 +63,14 @@ Example response:
"name": "release/*",
"push_access_levels": [
{
+ "id": 1,
"access_level": 40,
"access_level_description": "Maintainers"
}
],
"merge_access_levels": [
{
+ "id": 1,
"access_level": 40,
"access_level_description": "Maintainers"
}
@@ -90,6 +94,7 @@ Example response:
"name": "master",
"push_access_levels": [
{
+ "id": 1,
"access_level": 40,
"user_id": null,
"group_id": null,
@@ -98,6 +103,7 @@ Example response:
],
"merge_access_levels": [
{
+ "id": 1,
"access_level": null,
"user_id": null,
"group_id": 1234,
@@ -136,12 +142,14 @@ Example response:
"name": "master",
"push_access_levels": [
{
+ "id": 1,
"access_level": 40,
"access_level_description": "Maintainers"
}
],
"merge_access_levels": [
{
+ "id": 1,
"access_level": 40,
"access_level_description": "Maintainers"
}
@@ -162,6 +170,7 @@ Example response:
"name": "master",
"push_access_levels": [
{
+ "id": 1,
"access_level": 40,
"user_id": null,
"group_id": null,
@@ -170,6 +179,7 @@ Example response:
],
"merge_access_levels": [
{
+ "id": 1,
"access_level": null,
"user_id": null,
"group_id": 1234,
@@ -215,18 +225,21 @@ Example response:
"name": "*-stable",
"push_access_levels": [
{
+ "id": 1,
"access_level": 30,
"access_level_description": "Developers + Maintainers"
}
],
"merge_access_levels": [
{
+ "id": 1,
"access_level": 30,
"access_level_description": "Developers + Maintainers"
}
],
"unprotect_access_levels": [
{
+ "id": 1,
"access_level": 40,
"access_level_description": "Maintainers"
}
@@ -247,6 +260,7 @@ Example response:
"name": "*-stable",
"push_access_levels": [
{
+ "id": 1,
"access_level": 30,
"user_id": null,
"group_id": null,
@@ -255,6 +269,7 @@ Example response:
],
"merge_access_levels": [
{
+ "id": 1,
"access_level": 30,
"user_id": null,
"group_id": null,
@@ -263,6 +278,7 @@ Example response:
],
"unprotect_access_levels": [
{
+ "id": 1,
"access_level": 40,
"user_id": null,
"group_id": null,
@@ -291,6 +307,7 @@ Example response:
"name": "*-stable",
"push_access_levels": [
{
+ "id": 1,
"access_level": null,
"user_id": 1,
"group_id": null,
@@ -299,6 +316,7 @@ Example response:
],
"merge_access_levels": [
{
+ "id": 1,
"access_level": 40,
"user_id": null,
"group_id": null,
@@ -307,6 +325,7 @@ Example response:
],
"unprotect_access_levels": [
{
+ "id": 1,
"access_level": 40,
"user_id": null,
"group_id": null,
@@ -348,6 +367,7 @@ Example response:
"name": "master",
"push_access_levels": [
{
+ "id": 1,
"access_level": 30,
"access_level_description": "Developers + Maintainers",
"user_id": null,
@@ -356,12 +376,14 @@ Example response:
],
"merge_access_levels": [
{
+ "id": 1,
"access_level": 30,
"access_level_description": "Developers + Maintainers",
"user_id": null,
"group_id": null
},
{
+ "id": 2,
"access_level": 40,
"access_level_description": "Maintainers",
"user_id": null,
@@ -370,6 +392,7 @@ Example response:
],
"unprotect_access_levels": [
{
+ "id": 1,
"access_level": 40,
"access_level_description": "Maintainers",
"user_id": null,
@@ -398,20 +421,107 @@ curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://git
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
| `name` | string | yes | The name of the branch |
-## Require code owner approvals for a single branch **(PREMIUM)**
+## Update a protected branch
-Update the "code owner approval required" option for the given protected branch.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101903) in GitLab 15.6.
+
+Updates a protected branch.
```plaintext
PATCH /projects/:id/protected_branches/:name
```
```shell
-curl --request PATCH --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/protected_branches/feature-branch?code_owner_approval_required=true"
+curl --request PATCH --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/protected_branches/feature-branch?allow_force_push=true&code_owner_approval_required=true"
```
-| Attribute | Type | Required | Description |
+| Attribute | Type | Required | Description |
| -------------------------------------------- | ---- | -------- | ----------- |
-| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
-| `name` | string | yes | The name of the branch |
-| `code_owner_approval_required` | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/code_owners.md). (defaults: false)|
+| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
+| `name` | string | yes | The name of the branch |
+| `allow_force_push` | boolean | no | When enabled, members who can push to this branch can also force push. |
+| `code_owner_approval_required` **(PREMIUM)** | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/code_owners.md). Defaults to `false`. |
+| `allowed_to_push` **(PREMIUM)** | array | no | Array of push access levels, with each described by a hash. |
+| `allowed_to_merge` **(PREMIUM)** | array | no | Array of merge access levels, with each described by a hash. |
+| `allowed_to_unprotect` **(PREMIUM)** | array | no | Array of unprotect access levels, with each described by a hash. |
+
+Elements in the `allowed_to_push`, `allowed_to_merge` and `allowed_to_unprotect` arrays should be one of `user_id`, `group_id` or
+`access_level`, and take the form `{user_id: integer}`, `{group_id: integer}` or
+`{access_level: integer}`.
+
+To update:
+
+- `user_id`: Ensure the updated user has access to the project. You must also pass the
+ `id` of the `access_level` in the respective hash.
+- `group_id`: Ensure the updated group [has this project shared](../user/project/members/share_project_with_groups.md).
+ You must also pass the `id` of the `access_level` in the respective hash.
+
+To delete:
+
+- You must pass `_destroy` set to `true`. See the following examples.
+
+### Example: create a `push_access_level` record
+
+```shell
+curl --header 'Content-Type: application/json' --request PATCH \
+ --data '{"push_access_levels": [{"group_id": 9899829, access_level: 40}]}' \
+ --header "PRIVATE-TOKEN: <your_access_token>" \
+ "https://gitlab.example.com/api/v4/projects/22034114/protected_branches/master"
+```
+
+Example response:
+
+```json
+{
+ "name": "master",
+ "allowed_to_push": [
+ {
+ "id": 12,
+ "access_level": 40,
+ "access_level_description": "Administrator",
+ "user_id": null,
+ "group_id": 9899829
+ }
+ ]
+}
+```
+
+### Example: update a `push_access_level` record
+
+```shell
+curl --header 'Content-Type: application/json' --request PUT \
+ --data '{"push_access_levels": [{"id": 12, "group_id": 22034120}]' \
+ --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/22034114/protected_branches/master"
+```
+
+```json
+{
+ "name": "master",
+ "deploy_access_levels": [
+ {
+ "id": 12,
+ "access_level": 40,
+ "access_level_description": "Administrator",
+ "user_id": null,
+ "group_id": 22034120
+ }
+ ]
+}
+```
+
+### Example: delete a `push_access_level` record
+
+```shell
+curl --header 'Content-Type: application/json' --request PUT \
+ --data '{"push_access_levels": [{"id": 12, "_destroy": true}]' \
+ --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/22034114/protected_branches/master"
+```
+
+Example response:
+
+```json
+{
+ "name": "master",
+ "push_access_levels": []
+}
+```
diff --git a/doc/api/protected_environments.md b/doc/api/protected_environments.md
index 9ff3c34d2d7..ec0657148da 100644
--- a/doc/api/protected_environments.md
+++ b/doc/api/protected_environments.md
@@ -103,7 +103,7 @@ Example response:
}
```
-## Protect repository environments
+## Protect a single environment
Protects a single environment:
@@ -343,7 +343,7 @@ Example response:
}
```
-## Unprotect environment
+## Unprotect a single environment
Unprotects the given protected environment:
diff --git a/doc/api/protected_tags.md b/doc/api/protected_tags.md
index c8e7117e5a9..88b868a3965 100644
--- a/doc/api/protected_tags.md
+++ b/doc/api/protected_tags.md
@@ -41,6 +41,7 @@ Example response:
"name": "release-1-0",
"create_access_levels": [
{
+ "id":1,
"access_level": 40,
"access_level_description": "Maintainers"
}
@@ -75,6 +76,7 @@ Example response:
"name": "release-1-0",
"create_access_levels": [
{
+ "id": 1,
"access_level": 40,
"access_level_description": "Maintainers"
}
@@ -109,6 +111,7 @@ Example response:
"name": "*-stable",
"create_access_levels": [
{
+ "id": 1,
"access_level": 30,
"access_level_description": "Developers + Maintainers"
}
diff --git a/doc/api/releases/index.md b/doc/api/releases/index.md
index e3b1d9230f6..66d5775f499 100644
--- a/doc/api/releases/index.md
+++ b/doc/api/releases/index.md
@@ -28,7 +28,7 @@ For authentication, the Releases API accepts either:
> [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72448) to allow for `JOB-TOKEN` in GitLab 14.5.
-Paginated list of Releases, sorted by `released_at`.
+Returns a paginated list of releases, sorted by `released_at`.
```plaintext
GET /projects/:id/releases
@@ -235,7 +235,7 @@ Example response:
> [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72448) to allow for `JOB-TOKEN` in GitLab 14.5.
-Get a Release for the given tag.
+Gets a release for the given tag.
```plaintext
GET /projects/:id/releases/:tag_name
@@ -364,9 +364,62 @@ Example response:
}
```
+## Download a release asset
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/358188) in GitLab 15.4.
+
+Download a release asset file by making a request with the following format:
+
+```plaintext
+GET /projects/:id/releases/:tag_name/downloads/:filepath
+```
+
+| Attribute | Type | Required | Description |
+|----------------------------| -------------- | -------- | ----------------------------------------------------------------------------------- |
+| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../index.md#namespaced-path-encoding). |
+| `tag_name` | string | yes | The Git tag the release is associated with. |
+| `filepath` | string | yes | Path to the release asset file as specified when [creating](links.md#create-a-release-link) or [updating](links.md#update-a-release-link) its link. |
+
+Example request:
+
+```shell
+curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/24/releases/v0.1/downloads/bin/asset.exe"
+```
+
+### Get the latest release
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/358188) in GitLab 15.4.
+
+Latest release information is accessible through a permanent API URL.
+
+The format of the URL is:
+
+```plaintext
+GET /projects/:id/releases/permalink/latest
+```
+
+To call any other GET API that requires a release tag, append a suffix to the `permalink/latest` API path.
+
+For example, to get latest [release evidence](#collect-release-evidence) you can use:
+
+```plaintext
+GET /projects/:id/releases/permalink/latest/evidence
+```
+
+Another example is [downloading an asset](#download-a-release-asset) of the latest release, for which you can use:
+
+```plaintext
+GET /projects/:id/releases/permalink/latest/downloads/bin/asset.exe
+```
+
+#### Sorting preferences
+
+By default, GitLab fetches the release using `released_at` time. The use of the query parameter
+`?order_by=released_at` is optional, and support for `?order_by=semver` is tracked [in issue 352945](https://gitlab.com/gitlab-org/gitlab/-/issues/352945).
+
## Create a release
-Create a release. Developer level access to the project is required to create a release.
+Creates a release. Developer level access to the project is required to create a release.
```plaintext
POST /projects/:id/releases
@@ -516,7 +569,7 @@ adding milestones for ancestor groups raises an error.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/199065) in GitLab 12.10.
> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72448) to allow for `JOB-TOKEN` in GitLab 14.5.
-Create Evidence for an existing Release.
+Creates an evidence for an existing release.
```plaintext
POST /projects/:id/releases/:tag_name/evidence
@@ -543,7 +596,7 @@ Example response:
> [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72448) to allow for `JOB-TOKEN` in GitLab 14.5.
-Update a release. Developer level access to the project is required to update a release.
+Updates a release. Developer level access to the project is required to update a release.
```plaintext
PUT /projects/:id/releases/:tag_name
@@ -652,7 +705,7 @@ Example response:
> [Changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72448) to allow for `JOB-TOKEN` in GitLab 14.5.
-Delete a release. Deleting a release doesn't delete the associated tag. Maintainer level access to the project is required to delete a release.
+Deletes a release. Deleting a release doesn't delete the associated tag. Maintainer level access to the project is required to delete a release.
```plaintext
DELETE /projects/:id/releases/:tag_name
diff --git a/doc/api/releases/links.md b/doc/api/releases/links.md
index 050d7cabdf9..732f5ea57ef 100644
--- a/doc/api/releases/links.md
+++ b/doc/api/releases/links.md
@@ -13,9 +13,9 @@ links. For manipulating other Release assets, see [Release API](index.md).
GitLab supports links to `http`, `https`, and `ftp` assets.
-## Get links
+## List links of a release
-Get assets as links from a Release.
+Get assets as links from a release.
```plaintext
GET /projects/:id/releases/:tag_name/assets/links
@@ -53,9 +53,9 @@ Example response:
]
```
-## Get a link
+## Get a release link
-Get an asset as a link from a Release.
+Get an asset as a link from a release.
```plaintext
GET /projects/:id/releases/:tag_name/assets/links/:link_id
@@ -85,9 +85,9 @@ Example response:
}
```
-## Create a link
+## Create a release link
-Create an asset as a link from a Release.
+Creates an asset as a link from a release.
```plaintext
POST /projects/:id/releases/:tag_name/assets/links
@@ -126,9 +126,9 @@ Example response:
}
```
-## Update a link
+## Update a release link
-Update an asset as a link from a Release.
+Updates an asset as a link from a release.
```plaintext
PUT /projects/:id/releases/:tag_name/assets/links/:link_id
@@ -167,9 +167,9 @@ Example response:
}
```
-## Delete a link
+## Delete a release link
-Delete an asset as a link from a Release.
+Deletes an asset as a link from a release.
```plaintext
DELETE /projects/:id/releases/:tag_name/assets/links/:link_id
diff --git a/doc/api/remote_mirrors.md b/doc/api/remote_mirrors.md
index 1013ffb49fb..bd59aa64e45 100644
--- a/doc/api/remote_mirrors.md
+++ b/doc/api/remote_mirrors.md
@@ -8,15 +8,15 @@ type: reference, api
# Project remote mirrors API **(FREE)**
[Push mirrors](../user/project/repository/mirror/push.md)
-defined on a project's repository settings are called "remote mirrors", and the
-state of these mirrors can be queried and modified via the remote mirror API
-outlined below.
+defined on a project's repository settings are called "remote mirrors". You
+can query and modify the state of these mirrors with the remote mirror API.
-## List a project's remote mirrors
+For security reasons, the `url` attribute in the API response is always scrubbed of username
+and password information.
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38121) in GitLab 12.9.
+## List a project's remote mirrors
-Returns an Array of remote mirrors and their statuses:
+Returns an array of remote mirrors and their statuses:
```plaintext
GET /projects/:id/remote_mirrors
@@ -47,10 +47,6 @@ Example response:
]
```
-NOTE:
-For security reasons, the `url` attribute is always scrubbed of username
-and password information.
-
## Get a single project's remote mirror
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82770) in GitLab 14.10.
@@ -84,19 +80,14 @@ Example response:
}
```
-NOTE:
-For security reasons, the `url` attribute is always scrubbed of username
-and password information.
-
## Create a pull mirror
Learn how to [configure a pull mirror](projects.md#configure-pull-mirroring-for-a-project) using the Projects API.
## Create a push mirror
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24189) in GitLab 12.9.
-
-Push mirroring is disabled by default. You can enable it by including the optional parameter `enabled` when creating it:
+Push mirroring is disabled by default. To enable it, include the optional parameter
+`enabled` when you create the mirror:
```plaintext
POST /projects/:id/remote_mirrors
@@ -106,8 +97,8 @@ POST /projects/:id/remote_mirrors
| :---------- | :----- | :--------- | :------------ |
| `url` | String | yes | The target URL to which the repository is mirrored. |
| `enabled` | Boolean | no | Determines if the mirror is enabled. |
-| `only_protected_branches` | Boolean | no | Determines if only protected branches are mirrored. |
| `keep_divergent_refs` | Boolean | no | Determines if divergent refs are skipped. |
+| `only_protected_branches` | Boolean | no | Determines if only protected branches are mirrored. |
Example request:
@@ -135,8 +126,6 @@ Example response:
## Update a remote mirror's attributes
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38121) in GitLab 12.9.
-
Toggle a remote mirror on or off, or change which types of branches are
mirrored:
@@ -148,8 +137,8 @@ PUT /projects/:id/remote_mirrors/:mirror_id
| :---------- | :----- | :--------- | :------------ |
| `mirror_id` | Integer | yes | The remote mirror ID. |
| `enabled` | Boolean | no | Determines if the mirror is enabled. |
-| `only_protected_branches` | Boolean | no | Determines if only protected branches are mirrored. |
| `keep_divergent_refs` | Boolean | no | Determines if divergent refs are skipped. |
+| `only_protected_branches` | Boolean | no | Determines if only protected branches are mirrored. |
Example request:
diff --git a/doc/api/repositories.md b/doc/api/repositories.md
index 751bbd75c7a..428a09f1bbe 100644
--- a/doc/api/repositories.md
+++ b/doc/api/repositories.md
@@ -223,7 +223,7 @@ Example response:
}],
"compare_timeout": false,
"compare_same_ref": false,
- "web_url": "https://gitlab.example.com/thedude/gitlab-foss/-/compare/ae73cb07c9eeaf35924a10f713b364d32b2dd34f...0b4bc9a49b562e85de7cc9e834518ea6828729b9"
+ "web_url": "https://gitlab.example.com/janedoe/gitlab-foss/-/compare/ae73cb07c9eeaf35924a10f713b364d32b2dd34f...0b4bc9a49b562e85de7cc9e834518ea6828729b9"
}
```
diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md
index 9f06467964b..1db614fce36 100644
--- a/doc/api/repository_files.md
+++ b/doc/api/repository_files.md
@@ -18,14 +18,14 @@ in the following table.
| Scope | Description |
| ----- | ----------- |
-| `read_repository` | Allows read-access to the repository files. |
| `api` | Allows read-write access to the repository files. |
+| `read_repository` | Allows read-access to the repository files. |
## Get file from repository
> The `execute_filemode` field in the response was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83499) in GitLab 14.10.
-Allows you to receive information about file in repository like name, size,
+Allows you to receive information about file in repository like name, size, and
content. File content is Base64 encoded. This endpoint can be accessed
without authentication if the repository is publicly accessible.
@@ -37,11 +37,11 @@ GET /projects/:id/repository/files/:file_path
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fmodels%2Fkey%2Erb?ref=master"
```
-| Attribute | Type | Required | Description |
-|-------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------|
-| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
-| `file_path` | string | yes | URL encoded full path to new file. Ex. `lib%2Fclass%2Erb`. |
-| `ref` | string | yes | The name of branch, tag or commit |
+| Attribute | Type | Required | Description |
+|-------------|----------------|----------|-------------|
+| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. |
+| `file_path` | string | yes | URL encoded full path to new file, such as `lib%2Fclass%2Erb`. |
+| `ref` | string | yes | The name of branch, tag or commit. |
Example response:
@@ -62,7 +62,8 @@ Example response:
```
NOTE:
-`blob_id` is the blob SHA, see [repositories - Get a blob from repository](repositories.md#get-a-blob-from-repository)
+`blob_id` is the blob SHA. Refer to [Get a blob from repository](repositories.md#get-a-blob-from-repository)
+in the Repositories API.
In addition to the `GET` method, you can also use `HEAD` to get just file metadata.
@@ -100,14 +101,14 @@ Allows you to receive blame information. Each blame range contains lines and cor
GET /projects/:id/repository/files/:file_path/blame
```
-| Attribute | Type | Required | Description |
-|-----------------|-------------------|----------|--------------------------------------------------------------------------------------------------------------|
-| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
-| `file_path` | string | yes | URL encoded full path to new file. Ex. `lib%2Fclass%2Erb`. |
-| `ref` | string | yes | The name of branch, tag or commit |
-| `range` | hash | no | Blame range |
-| `range[start]` | integer | yes | The first line of the range to blame |
-| `range[end]` | integer | yes | The last line of the range to blame |
+| Attribute | Type | Required | Description |
+|-----------------|-------------------|----------|-------------|
+| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. |
+| `file_path` | string | yes | URL-encoded full path to new file, such as`lib%2Fclass%2Erb`. |
+| `ref` | string | yes | The name of branch, tag or commit. |
+| `range[end]` | integer | yes | The last line of the range to blame. |
+| `range[start]` | integer | yes | The first line of the range to blame. |
+| `range` | hash | no | Blame range. |
```shell
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/13083/repository/files/path%2Fto%2Ffile.rb/blame?ref=master"
@@ -142,7 +143,7 @@ Example response:
```
NOTE:
-`HEAD` method return just file metadata as in [Get file from repository](repository_files.md#get-file-from-repository).
+`HEAD` method returns just file metadata, as in [Get file from repository](repository_files.md#get-file-from-repository).
```shell
curl --head --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/13083/repository/files/path%2Fto%2Ffile.rb/blame?ref=master"
@@ -168,7 +169,8 @@ X-Gitlab-Execute-Filemode: false
### Examples
-To request a blame range, specify `range[start]` and `range[end]` parameters with the start and end line numbers of the file.
+To request a blame range, specify `range[start]` and `range[end]` parameters with
+the starting and ending line numbers of the file.
```shell
curl --head --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/13083/repository/files/path%2Fto%2Ffile.rb/blame?ref=master&range[start]=1&range[end]=2"
@@ -207,24 +209,25 @@ Example response:
GET /projects/:id/repository/files/:file_path/raw
```
-| Attribute | Type | Required | Description |
-|-------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------|
-| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
-| `file_path` | string | yes | URL encoded full path to new file. Ex. `lib%2Fclass%2Erb`. |
-| `ref` | string | yes | The name of branch, tag or commit. Default is the `HEAD` of the project. |
+| Attribute | Type | Required | Description |
+|-------------|----------------|----------|------------|
+| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. |
+| `file_path` | string | yes | URL-encoded full path to new file, such as `lib%2Fclass%2Erb`. |
+| `ref` | string | yes | The name of branch, tag or commit. Default is the `HEAD` of the project. |
```shell
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fmodels%2Fkey%2Erb/raw?ref=master"
```
NOTE:
-Like [Get file from repository](repository_files.md#get-file-from-repository) you can use `HEAD` to get just file metadata.
+Like [Get file from repository](repository_files.md#get-file-from-repository), you can use `HEAD` to get just file metadata.
## Create new file in repository
> The `execute_filemode` parameter was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83499) in GitLab 14.10.
-This allows you to create a single file. For creating multiple files with a single request see the [commits API](commits.md#create-a-commit-with-multiple-files-and-actions).
+Allows you to create a single file. For creating multiple files with a single request,
+refer to the [commits API](commits.md#create-a-commit-with-multiple-files-and-actions).
```plaintext
POST /projects/:id/repository/files/:file_path
@@ -232,16 +235,16 @@ POST /projects/:id/repository/files/:file_path
| Attribute | Type | Required | Description |
| ---------------- | -------------- | -------- | ----------- |
-| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. |
-| `file_path` | string | yes | URL-encoded full path to new file. For example: `lib%2Fclass%2Erb`. |
| `branch` | string | yes | Name of the new branch to create. The commit is added to this branch. |
-| `start_branch` | string | no | Name of the base branch to create the new branch from. |
-| `encoding` | string | no | Change encoding to `base64`. Default is `text`. |
+| `commit_message` | string | yes | The commit message. |
+| `content` | string | yes | The file's content. |
+| `file_path` | string | yes | URL-encoded full path to new file. For example: `lib%2Fclass%2Erb`. |
+| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. |
| `author_email` | string | no | The commit author's email address. |
| `author_name` | string | no | The commit author's name. |
-| `content` | string | yes | The file's content. |
-| `commit_message` | string | yes | The commit message. |
+| `encoding` | string | no | Change encoding to `base64`. Default is `text`. |
| `execute_filemode` | boolean | no | Enables or disables the `execute` flag on the file. Can be `true` or `false`. |
+| `start_branch` | string | no | Name of the base branch to create the new branch from. |
```shell
curl --request POST --header 'PRIVATE-TOKEN: <your_access_token>' \
@@ -264,7 +267,8 @@ Example response:
> The `execute_filemode` parameter was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83499) in GitLab 14.10.
-This allows you to update a single file. For updating multiple files with a single request see the [commits API](commits.md#create-a-commit-with-multiple-files-and-actions).
+Allows you to update a single file. For updating multiple files with a single request,
+refer to the [commits API](commits.md#create-a-commit-with-multiple-files-and-actions).
```plaintext
PUT /projects/:id/repository/files/:file_path
@@ -272,17 +276,17 @@ PUT /projects/:id/repository/files/:file_path
| Attribute | Type | Required | Description |
| ---------------- | -------------- | -------- | ----------- |
-| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
-| `file_path` | string | yes | URL-encoded full path to new file. For example: `lib%2Fclass%2Erb`. |
| `branch` | string | yes | Name of the new branch to create. The commit is added to this branch. |
-| `start_branch` | string | no | Name of the base branch to create the new branch from. |
-| `encoding` | string | no | Change encoding to `base64`. Default is `text`. |
+| `commit_message` | string | yes | The commit message. |
+| `content` | string | yes | The file's content. |
+| `file_path` | string | yes | URL-encoded full path to new file. For example: `lib%2Fclass%2Erb`. |
+| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
| `author_email` | string | no | The commit author's email address. |
| `author_name` | string | no | The commit author's name. |
-| `content` | string | yes | The file's content. |
-| `commit_message` | string | yes | The commit message. |
-| `last_commit_id` | string | no | Last known file commit ID. |
+| `encoding` | string | no | Change encoding to `base64`. Default is `text`. |
| `execute_filemode` | boolean | no | Enables or disables the `execute` flag on the file. Can be `true` or `false`. |
+| `last_commit_id` | string | no | Last known file commit ID. |
+| `start_branch` | string | no | Name of the base branch to create the new branch from. |
```shell
curl --request PUT --header 'PRIVATE-TOKEN: <your_access_token>' \
@@ -301,19 +305,19 @@ Example response:
}
```
-If the commit fails for any reason we return a 400 error with a non-specific
+If the commit fails for any reason we return a `400 Bad Request` error with a non-specific
error message. Possible causes for a failed commit include:
-- the `file_path` contained `/../` (attempted directory traversal);
-- the new file contents were identical to the current file contents. That is, the
- user tried to make an empty commit;
-- the branch was updated by a Git push while the file edit was in progress.
+- The `file_path` contained `/../` (attempted directory traversal).
+- The commit was empty: new file contents were identical to the current file contents.
+- The branch was updated by `git push` while the file edit was in progress.
-GitLab Shell has a boolean return code, preventing GitLab from specifying the error.
+[GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell/) has a boolean return code, preventing GitLab from specifying the error.
## Delete existing file in repository
-This allows you to delete a single file. For deleting multiple files with a single request, see the [commits API](commits.md#create-a-commit-with-multiple-files-and-actions).
+This allows you to delete a single file. For deleting multiple files with a single request,
+refer to the [commits API](commits.md#create-a-commit-with-multiple-files-and-actions).
```plaintext
DELETE /projects/:id/repository/files/:file_path
@@ -321,14 +325,14 @@ DELETE /projects/:id/repository/files/:file_path
| Attribute | Type | Required | Description |
| ---------------- | -------------- | -------- | ----------- |
-| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. |
-| `file_path` | string | yes | URL-encoded full path to new file. For example: `lib%2Fclass%2Erb`. |
| `branch` | string | yes | Name of the new branch to create. The commit is added to this branch. |
-| `start_branch` | string | no | Name of the base branch to create the new branch from. |
+| `commit_message` | string | yes | The commit message. |
+| `file_path` | string | yes | URL-encoded full path to new file. For example: `lib%2Fclass%2Erb`. |
+| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. |
| `author_email` | string | no | The commit author's email address. |
| `author_name` | string | no | The commit author's name. |
-| `commit_message` | string | yes | The commit message. |
| `last_commit_id` | string | no | Last known file commit ID. |
+| `start_branch` | string | no | Name of the base branch to create the new branch from. |
```shell
curl --request DELETE --header 'PRIVATE-TOKEN: <your_access_token>' \
diff --git a/doc/api/runners.md b/doc/api/runners.md
index 657eb4d470c..f690e0cb9c1 100644
--- a/doc/api/runners.md
+++ b/doc/api/runners.md
@@ -15,7 +15,7 @@ There are two tokens to take into account when connecting a runner with GitLab.
| Token | Description |
| ----- | ----------- |
| Registration token | Token used to [register the runner](https://docs.gitlab.com/runner/register/). It can be [obtained through GitLab](../ci/runners/index.md). |
-| Authentication token | Token used to authenticate the runner with the GitLab instance. It is obtained automatically when you [register a runner](https://docs.gitlab.com/runner/register/) or by the Runner API when you manually [register a runner](#register-a-new-runner) or [reset the authentication token](#reset-runners-authentication-token-by-using-the-runner-id). |
+| Authentication token | Token used to authenticate the runner with the GitLab instance. It is obtained automatically when you [register a runner](https://docs.gitlab.com/runner/register/) or by the Runner API when you manually [register a runner](#register-a-new-runner-deprecated) or [reset the authentication token](#reset-runners-authentication-token-by-using-the-runner-id). |
Here's an example of how the two tokens are used in runner registration:
@@ -46,11 +46,11 @@ GET /runners?tag_list=tag1,tag2
| Attribute | Type | Required | Description |
|------------|--------------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `scope` | string | no | Deprecated: Use `type` or `status` instead. The scope of specific runners to show, one of: `active`, `paused`, `online` and `offline`; showing all runners if none provided |
-| `type` | string | no | The type of runners to show, one of: `instance_type`, `group_type`, `project_type` |
-| `status` | string | no | The status of runners to show, one of: `online`, `offline`, `stale`, and `never_contacted`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 |
+| `scope` | string | no | Deprecated: Use `type` or `status` instead. The scope of specific runners to return, one of: `active`, `paused`, `online` and `offline`; showing all runners if none provided |
+| `type` | string | no | The type of runners to return, one of: `instance_type`, `group_type`, `project_type` |
+| `status` | string | no | The status of runners to return, one of: `online`, `offline`, `stale`, and `never_contacted`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 |
| `paused` | boolean | no | Whether to include only runners that are accepting or ignoring new jobs |
-| `tag_list` | string array | no | List of the runner's tags |
+| `tag_list` | string array | no | A list of runner tags |
```shell
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/runners"
@@ -111,11 +111,11 @@ GET /runners/all?tag_list=tag1,tag2
| Attribute | Type | Required | Description |
|------------|--------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `scope` | string | no | Deprecated: Use `type` or `status` instead. The scope of runners to show, one of: `specific`, `shared`, `active`, `paused`, `online` and `offline`; showing all runners if none provided |
-| `type` | string | no | The type of runners to show, one of: `instance_type`, `group_type`, `project_type` |
-| `status` | string | no | The status of runners to show, one of: `online`, `offline`, `stale`, and `never_contacted`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 |
+| `scope` | string | no | Deprecated: Use `type` or `status` instead. The scope of runners to return, one of: `specific`, `shared`, `active`, `paused`, `online` and `offline`; showing all runners if none provided |
+| `type` | string | no | The type of runners to return, one of: `instance_type`, `group_type`, `project_type` |
+| `status` | string | no | The status of runners to return, one of: `online`, `offline`, `stale`, and `never_contacted`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 |
| `paused` | boolean | no | Whether to include only runners that are accepting or ignoring new jobs |
-| `tag_list` | string array | no | List of the runner's tags |
+| `tag_list` | string array | no | A list of runner tags |
```shell
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/runners/all"
@@ -260,17 +260,17 @@ Update details of a runner.
PUT /runners/:id
```
-| Attribute | Type | Required | Description |
-|-------------------|---------|----------|--------------------------------------------------------------------------------------------------|
-| `id` | integer | yes | The ID of a runner |
-| `description` | string | no | The description of a runner |
-| `active` | boolean | no | Deprecated: Use `:paused` instead. Flag indicating whether the runner is allowed to receive jobs |
-| `paused` | boolean | no | Flag indicating whether the runner should ignore new jobs |
-| `tag_list` | array | no | The list of tags for a runner; put array of tags, that should be finally assigned to a runner |
-| `run_untagged` | boolean | no | Flag indicating the runner can execute untagged jobs |
-| `locked` | boolean | no | Flag indicating the runner is locked |
-| `access_level` | string | no | The access_level of the runner; `not_protected` or `ref_protected` |
-| `maximum_timeout` | integer | no | Maximum timeout set when this runner handles the job |
+| Attribute | Type | Required | Description |
+|-------------------|---------|----------|-------------------------------------------------------------------------------------------------|
+| `id` | integer | yes | The ID of a runner |
+| `description` | string | no | The description of the runner |
+| `active` | boolean | no | Deprecated: Use `paused` instead. Flag indicating whether the runner is allowed to receive jobs |
+| `paused` | boolean | no | Specifies whether the runner should ignore new jobs |
+| `tag_list` | array | no | The list of tags for the runner |
+| `run_untagged` | boolean | no | Specifies whether the runner can execute untagged jobs |
+| `locked` | boolean | no | Specifies whether the runner is locked |
+| `access_level` | string | no | The access level of the runner; `not_protected` or `ref_protected` |
+| `maximum_timeout` | integer | no | Maximum timeout that limits the amount of time (in seconds) that runners can run jobs |
```shell
curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/runners/6" \
@@ -370,7 +370,7 @@ GET /runners/:id/jobs
|-----------|---------|----------|---------------------|
| `id` | integer | yes | The ID of a runner |
| `status` | string | no | Status of the job; one of: `running`, `success`, `failed`, `canceled` |
-| `order_by`| string | no | Order jobs by `id`. |
+| `order_by`| string | no | Order jobs by `id` |
| `sort` | string | no | Sort jobs in `asc` or `desc` order (default: `desc`). Specify `order_by` as well, including for `id`. |
```shell
@@ -463,11 +463,11 @@ GET /projects/:id/runners?tag_list=tag1,tag2
| Attribute | Type | Required | Description |
|------------|----------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user |
-| `scope` | string | no | Deprecated: Use `type` or `status` instead. The scope of specific runners to show, one of: `active`, `paused`, `online` and `offline`; showing all runners if none provided |
-| `type` | string | no | The type of runners to show, one of: `instance_type`, `group_type`, `project_type` |
-| `status` | string | no | The status of runners to show, one of: `online`, `offline`, `stale`, and `never_contacted`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 |
+| `scope` | string | no | Deprecated: Use `type` or `status` instead. The scope of specific runners to return, one of: `active`, `paused`, `online` and `offline`; showing all runners if none provided |
+| `type` | string | no | The type of runners to return, one of: `instance_type`, `group_type`, `project_type` |
+| `status` | string | no | The status of runners to return, one of: `online`, `offline`, `stale`, and `never_contacted`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 |
| `paused` | boolean | no | Whether to include only runners that are accepting or ignoring new jobs |
-| `tag_list` | string array | no | List of the runner's tags |
+| `tag_list` | string array | no | A list of runner tags |
```shell
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/9/runners"
@@ -580,10 +580,10 @@ GET /groups/:id/runners?tag_list=tag1,tag2
| Attribute | Type | Required | Description |
|------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `id` | integer | yes | The ID of the group owned by the authenticated user |
-| `type` | string | no | The type of runners to show, one of: `instance_type`, `group_type`, `project_type`. The `project_type` value is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351466) and will be removed in GitLab 15.0 |
-| `status` | string | no | The status of runners to show, one of: `online`, `offline`, `stale`, and `never_contacted`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 |
+| `type` | string | no | The type of runners to return, one of: `instance_type`, `group_type`, `project_type`. The `project_type` value is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351466) and will be removed in GitLab 15.0 |
+| `status` | string | no | The status of runners to return, one of: `online`, `offline`, `stale`, and `never_contacted`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 |
| `paused` | boolean | no | Whether to include only runners that are accepting or ignoring new jobs |
-| `tag_list` | string array | no | List of the runner's tags |
+| `tag_list` | string array | no | A list of runner tags |
```shell
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/9/runners"
@@ -640,7 +640,12 @@ Example response:
]
```
-## Register a new runner
+## Register a new runner (deprecated)
+
+> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/102579) in GitLab 15.6.
+
+WARNING:
+This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/102579) in GitLab 15.6 and is planned for removal in 16.0. This change is a breaking change.
Register a new runner for the instance.
@@ -652,14 +657,14 @@ POST /runners
|--------------------|--------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `token` | string | yes | [Registration token](#registration-and-authentication-tokens) |
| `description` | string | no | Runner's description |
-| `info` | hash | no | Runner's metadata. You can include `name`, `version`, `revision`, `platform`, and `architecture`, but only `version` is displayed in the Admin Area of the UI |
-| `active` | boolean | no | Deprecated: Use `:paused` instead. Whether the runner is allowed to receive jobs |
-| `paused` | boolean | no | Whether the runner should ignore new jobs |
-| `locked` | boolean | no | Whether the runner should be locked for current project |
-| `run_untagged` | boolean | no | Whether the runner should handle untagged jobs |
-| `tag_list` | string array | no | List of runner's tags |
-| `access_level` | string | no | The access_level of the runner; `not_protected` or `ref_protected` |
-| `maximum_timeout` | integer | no | Maximum timeout set when this runner handles the job |
+| `info` | hash | no | Runner's metadata. You can include `name`, `version`, `revision`, `platform`, and `architecture`, but only `version`, `platform`, and `architecture` are displayed in the Admin Area of the UI |
+| `active` | boolean | no | Deprecated: Use `paused` instead. Specifies whether the runner is allowed to receive new jobs |
+| `paused` | boolean | no | Specifies whether the runner should ignore new jobs |
+| `locked` | boolean | no | Specifies whether the runner should be locked for the current project |
+| `run_untagged` | boolean | no | Specifies whether the runner should handle untagged jobs |
+| `tag_list` | string array | no | A list of runner tags |
+| `access_level` | string | no | The access level of the runner; `not_protected` or `ref_protected` |
+| `maximum_timeout` | integer | no | Maximum timeout that limits the amount of time (in seconds) that runners can run jobs |
| `maintainer_note` | string | no | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/350730), see `maintenance_note` |
| `maintenance_note` | string | no | Free-form maintenance notes for the runner (1024 characters) |
@@ -739,9 +744,9 @@ Validates authentication credentials for a registered runner.
POST /runners/verify
```
-| Attribute | Type | Required | Description |
-|-------------|---------|----------|---------------------|
-| `token` | string | yes | Runner's [authentication token](#registration-and-authentication-tokens). |
+| Attribute | Type | Required | Description |
+|-------------|---------|----------|-------------------------------------------------------------------------------|
+| `token` | string | yes | The runner's [authentication token](#registration-and-authentication-tokens). |
```shell
curl --request POST "https://gitlab.example.com/api/v4/runners/verify" \
@@ -759,7 +764,7 @@ Response:
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3.
-Resets the runner registration token for the GitLab instance.
+Reset the runner registration token for the GitLab instance.
```plaintext
POST /runners/reset_registration_token
@@ -774,7 +779,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3.
-Resets the runner registration token for a project.
+Reset the runner registration token for a project.
```plaintext
POST /projects/:id/runners/reset_registration_token
@@ -789,7 +794,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3.
-Resets the runner registration token for a group.
+Reset the runner registration token for a group.
```plaintext
POST /groups/:id/runners/reset_registration_token
@@ -802,7 +807,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
## Reset runner's authentication token by using the runner ID
-Resets the runner's authentication token by using its runner ID.
+Reset the runner's authentication token by using its runner ID.
```plaintext
POST /runners/:id/reset_authentication_token
@@ -828,15 +833,15 @@ Example response:
## Reset runner's authentication token by using the current token
-Resets the runner's authentication token by using the current token's value as an input.
+Reset the runner's authentication token by using the current token's value as an input.
```plaintext
POST /runners/reset_authentication_token
```
-| Attribute | Type | Required | Description |
-|-----------|---------|----------|---------------------------------|
-| `token` | string | yes | The current token of the runner |
+| Attribute | Type | Required | Description |
+|-----------|---------|----------|----------------------------------------|
+| `token` | string | yes | The authentication token of the runner |
```shell
curl --request POST --form "token=<current token>" \
diff --git a/doc/api/scim.md b/doc/api/scim.md
index b1763a44fc4..53897cadf90 100644
--- a/doc/api/scim.md
+++ b/doc/api/scim.md
@@ -25,7 +25,7 @@ Supported attributes:
| Attribute | Type | Required | Description |
|:------------------|:--------|:---------|:----------------------|
-| `id` | integer | Yes | Return SAML identities for the given group ID. |
+| `id` | integer | Yes | Return SCIM identities for the given group ID. |
If successful, returns [`200`](index.md#status-codes) and the following
response attributes:
@@ -49,8 +49,8 @@ Example response:
Example request:
```shell
-curl --location --request GET "https://gdk.test:3443/api/v4/groups/33/scim/identities" \
---header "<PRIVATE-TOKEN>" \
+curl --location --request GET "https://gitlab.example.com/api/v4/groups/33/scim/identities" \
+--header "PRIVATE-TOKEN: <PRIVATE-TOKEN>" \
--form "extern_uid=<ID_TO_BE_UPDATED>" \
```
@@ -77,7 +77,7 @@ Parameters:
Example request:
```shell
-curl --location --request PATCH "https://gdk.test:3443/api/v4/groups/33/scim/sydney_jones" \
---header "<PRIVATE TOKEN>" \
+curl --location --request PATCH "https://gitlab.example.com/api/v4/groups/33/scim/sydney_jones" \
+--header "PRIVATE-TOKEN: <PRIVATE TOKEN>" \
--form "extern_uid=sydney_jones_new" \
```
diff --git a/doc/api/secure_files.md b/doc/api/secure_files.md
index 0c34dd30cdf..c2c21134d6f 100644
--- a/doc/api/secure_files.md
+++ b/doc/api/secure_files.md
@@ -42,14 +42,34 @@ Example response:
"name": "myfile.jks",
"checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac",
"checksum_algorithm": "sha256",
- "created_at": "2022-02-22T22:22:22.222Z"
+ "created_at": "2022-02-22T22:22:22.222Z",
+ "expires_at": null,
+ "metadata": null
},
{
"id": 2,
- "name": "myotherfile.jks",
+ "name": "myfile.cer",
"checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aa2",
"checksum_algorithm": "sha256",
- "created_at": "2022-02-22T22:22:22.222Z"
+ "created_at": "2022-02-22T22:22:22.222Z",
+ "expires_at": "2022-09-21T14:56:00.000Z",
+ "metadata": {
+ "id":"75949910542696343243264405377658443914",
+ "issuer": {
+ "C":"US",
+ "O":"Apple Inc.",
+ "CN":"Apple Worldwide Developer Relations Certification Authority",
+ "OU":"G3"
+ },
+ "subject": {
+ "C":"US",
+ "O":"Organization Name",
+ "CN":"Apple Distribution: Organization Name (ABC123XYZ)",
+ "OU":"ABC123XYZ",
+ "UID":"ABC123XYZ"
+ },
+ "expires_at":"2022-09-21T14:56:00.000Z"
+ }
}
]
```
@@ -83,7 +103,9 @@ Example response:
"name": "myfile.jks",
"checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac",
"checksum_algorithm": "sha256",
- "created_at": "2022-02-22T22:22:22.222Z"
+ "created_at": "2022-02-22T22:22:22.222Z",
+ "expires_at": null,
+ "metadata": null
}
```
@@ -118,7 +140,9 @@ Example response:
"name": "myfile.jks",
"checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac",
"checksum_algorithm": "sha256",
- "created_at": "2022-02-22T22:22:22.222Z"
+ "created_at": "2022-02-22T22:22:22.222Z",
+ "expires_at": null,
+ "metadata": null
}
```
diff --git a/doc/api/settings.md b/doc/api/settings.md
index 3d8d02b8429..78dc81c4f84 100644
--- a/doc/api/settings.md
+++ b/doc/api/settings.md
@@ -109,12 +109,13 @@ Example response:
```
Users on [GitLab Premium or Ultimate](https://about.gitlab.com/pricing/) may also see
-the `file_template_project_id`, `delayed_project_deletion`, `delayed_group_deletion`, `deletion_adjourned_period`, or the `geo_node_allowed_ips` parameters:
+the `group_owners_can_manage_default_branch_protection`, `file_template_project_id`, `delayed_project_deletion`, `delayed_group_deletion`, `deletion_adjourned_period`, or the `geo_node_allowed_ips` parameters:
```json
{
"id" : 1,
"signup_enabled" : true,
+ "group_owners_can_manage_default_branch_protection" : true,
"file_template_project_id": 1,
"geo_node_allowed_ips": "0.0.0.0/0, ::/0",
"delayed_project_deletion": false,
@@ -224,6 +225,7 @@ Example response:
Users on [GitLab Premium or Ultimate](https://about.gitlab.com/pricing/) may also see
these parameters:
+- `group_owners_can_manage_default_branch_protection`
- `file_template_project_id`
- `geo_node_allowed_ips`
- `geo_status_timeout`
@@ -295,6 +297,7 @@ listed in the descriptions of the relevant settings.
| `diff_max_patch_bytes` | integer | no | Maximum [diff patch size](../user/admin_area/diff_limits.md), in bytes. |
| `diff_max_files` | integer | no | Maximum [files in a diff](../user/admin_area/diff_limits.md). |
| `diff_max_lines` | integer | no | Maximum [lines in a diff](../user/admin_area/diff_limits.md). |
+| `disable_admin_oauth_scopes` | boolean | no | Stops administrators from connecting their GitLab accounts to non-trusted OAuth 2.0 applications that have the `api`, `read_api`, `read_repository`, `write_repository`, `read_registry`, `write_registry`, or `sudo` scopes. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375043) in GitLab 15.6. |
| `disable_feed_token` | boolean | no | Disable display of RSS/Atom and calendar feed tokens. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231493) in GitLab 13.7. |
| `disabled_oauth_sign_in_sources` | array of strings | no | Disabled OAuth sign-in sources. |
| `dns_rebinding_protection_enabled` | boolean | no | Enforce DNS rebinding attack protection. |
@@ -352,6 +355,7 @@ listed in the descriptions of the relevant settings.
| `grafana_enabled` | boolean | no | Enable Grafana. |
| `grafana_url` | string | no | Grafana URL. |
| `gravatar_enabled` | boolean | no | Enable Gravatar. |
+| `group_owners_can_manage_default_branch_protection` **(PREMIUM SELF)** | boolean | no | Prevent overrides of default branch protection. |
| `hashed_storage_enabled` | boolean | no | Create new projects using hashed storage paths: Enable immutable, hash-based paths and repository names to store repositories on disk. This prevents repositories from having to be moved or renamed when the Project URL changes and may improve disk I/O performance. (Always enabled in GitLab versions 13.0 and later, configuration is scheduled for removal in 14.0) |
| `help_page_hide_commercial_content` | boolean | no | Hide marketing-related entries from help. |
| `help_page_support_url` | string | no | Alternate support URL for help page and help dropdown list. |
diff --git a/doc/api/status_checks.md b/doc/api/status_checks.md
index a30bfbd4dfb..e6a9c633418 100644
--- a/doc/api/status_checks.md
+++ b/doc/api/status_checks.md
@@ -121,8 +121,8 @@ defined external service. This includes confidential merge requests.
| Attribute | Type | Required | Description |
|------------------------|------------------|----------|------------------------------------------------|
| `id` | integer | yes | ID of a project |
-| `name` | string | yes | Display name of status check |
-| `external_url` | string | yes | URL of status check resource |
+| `name` | string | yes | Display name of external status check |
+| `external_url` | string | yes | URL of external status check resource |
| `protected_branch_ids` | `array<Integer>` | no | IDs of protected branches to scope the rule by |
## Delete external status check
@@ -135,7 +135,7 @@ DELETE /projects/:id/external_status_checks/:check_id
| Attribute | Type | Required | Description |
|------------------------|----------------|----------|-----------------------|
-| `rule_id` | integer | yes | ID of an status check |
+| `check_id` | integer | yes | ID of an external status check |
| `id` | integer | yes | ID of a project |
## Update external status check
@@ -149,8 +149,8 @@ PUT /projects/:id/external_status_checks/:check_id
| Attribute | Type | Required | Description |
|------------------------|------------------|----------|------------------------------------------------|
| `id` | integer | yes | ID of a project |
-| `rule_id` | integer | yes | ID of an external status check |
-| `name` | string | no | Display name of status check |
+| `check_id` | integer | yes | ID of an external status check |
+| `name` | string | no | Display name of external status check |
| `external_url` | string | no | URL of external status check resource |
| `protected_branch_ids` | `array<Integer>` | no | IDs of protected branches to scope the rule by |
diff --git a/doc/api/suggestions.md b/doc/api/suggestions.md
index 9771225ad31..1e1f226481c 100644
--- a/doc/api/suggestions.md
+++ b/doc/api/suggestions.md
@@ -11,7 +11,7 @@ This page describes the API for [suggesting changes](../user/project/merge_reque
Every API call to suggestions must be authenticated.
-## Applying suggestions
+## Applying a suggestion
Applies a suggested patch in a merge request. Users must have
at least the Developer role to perform such action.
@@ -22,7 +22,7 @@ PUT /suggestions/:id/apply
| Attribute | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
-| `id` | integer/string | yes | The ID of a suggestion |
+| `id` | integer | yes | The ID of a suggestion |
| `commit_message` | string | no | A custom commit message to use instead of the default generated message or the project's default message |
```shell
@@ -32,13 +32,53 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab
Example response:
```json
+{
+ "id": 5,
+ "from_line": 10,
+ "to_line": 10,
+ "applicable": true,
+ "applied": false,
+ "from_content": "This is an eaxmple\n",
+ "to_content": "This is an example\n"
+}
+```
+
+## Applying multiple suggestions
+
+```plaintext
+PUT /suggestions/batch_apply
+```
+
+| Attribute | Type | Required | Description |
+| --------- | ---- | -------- | ----------- |
+| `ids` | integer | yes | The IDs of suggestions |
+| `commit_message` | string | no | A custom commit message to use instead of the default generated message or the project's default message |
+
+```shell
+curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" --header 'Content-Type: application/json' --data '{"ids": [5, 6]}' "https://gitlab.example.com/api/v4/suggestions/batch_apply"
+```
+
+Example response:
+
+```json
+[
{
- "id": 36,
+ "id": 5,
"from_line": 10,
"to_line": 10,
- "applicable": false,
- "applied": true,
- "from_content": " \"--talk-name=org.freedesktop.\",\n",
- "to_content": " \"--talk-name=org.free.\",\n \"--talk-name=org.desktop.\",\n"
+ "applicable": true,
+ "applied": false,
+ "from_content": "This is an eaxmple\n",
+ "to_content": "This is an example\n"
+ }
+ {
+ "id": 6,
+ "from_line": 19
+ "to_line": 19,
+ "applicable": true,
+ "applied": false,
+ "from_content": "This is another eaxmple\n",
+ "to_content": "This is another example\n"
}
+ ]
```
diff --git a/doc/api/templates/dockerfiles.md b/doc/api/templates/dockerfiles.md
index 9636393dfe9..31b676558db 100644
--- a/doc/api/templates/dockerfiles.md
+++ b/doc/api/templates/dockerfiles.md
@@ -142,6 +142,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/api/templates/gitignores.md b/doc/api/templates/gitignores.md
index 7c68daa5c48..1569a2bc89d 100644
--- a/doc/api/templates/gitignores.md
+++ b/doc/api/templates/gitignores.md
@@ -146,6 +146,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/api/templates/gitlab_ci_ymls.md b/doc/api/templates/gitlab_ci_ymls.md
index 152f3373ad6..b7048795313 100644
--- a/doc/api/templates/gitlab_ci_ymls.md
+++ b/doc/api/templates/gitlab_ci_ymls.md
@@ -147,6 +147,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/api/templates/licenses.md b/doc/api/templates/licenses.md
index e676da2e999..6abdb3ca3b0 100644
--- a/doc/api/templates/licenses.md
+++ b/doc/api/templates/licenses.md
@@ -166,6 +166,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/api/users.md b/doc/api/users.md
index 4a924f3b5f3..19c25236f55 100644
--- a/doc/api/users.md
+++ b/doc/api/users.md
@@ -431,7 +431,7 @@ see the `group_saml` option and `provisioned_by_group_id` parameter:
}
```
-Administrators can use the `created_by` parameter to see if a user account was created:
+Administrators can use the `created_by` parameter to see if a user account was created:
- [Manually by an administrator](../user/profile/account/create_accounts.md#create-users-in-admin-area).
- As a [project bot user](../user/project/settings/project_access_tokens.md#bot-users-for-projects).
@@ -510,6 +510,8 @@ Parameters:
Modifies an existing user. Only administrators can change attributes of a user.
+The `email` field is the user's primary email address. You can only change this field to an already-added secondary email address for that user. To add more email addresses to the same user, use the [add email function](#add-email).
+
```plaintext
PUT /users/:id
```
@@ -966,6 +968,33 @@ Example response:
Please refer to the [List of user projects](projects.md#list-user-projects).
+## List associations count for user
+
+Get a list of a specified user's count of projects, groups, issues and merge requests.
+
+Administrators can query any user, but non-administrators can only query themselves.
+
+```plaintext
+GET /users/:id/associations_count
+```
+
+Parameters:
+
+| Attribute | Type | Required | Description |
+|-----------|---------|----------|------------------|
+| `id` | integer | yes | ID of a user |
+
+Example response:
+
+```json
+{
+ "groups_count": 2,
+ "projects_count": 3,
+ "issues_count": 8,
+ "merge_requests_count": 5
+}
+```
+
## List SSH keys
Get a list of currently authenticated user's SSH keys.
@@ -1486,6 +1515,7 @@ Parameters:
Deletes email owned by currently authenticated user.
This returns a `204 No Content` status code if the operation was successfully or `404` if the resource was not found.
+This cannot delete a primary email address.
```plaintext
DELETE /user/emails/:email_id
@@ -1499,7 +1529,11 @@ Parameters:
## Delete email for given user **(FREE SELF)**
-Deletes email owned by a specified user. Available only for administrator.
+Prerequisite:
+
+- You must be an administrator of a self-managed GitLab instance.
+
+Deletes an email address owned by a specified user. This cannot delete a primary email address.
```plaintext
DELETE /users/:id/emails/:email_id
@@ -1573,7 +1607,7 @@ Returns:
- `404 User Not Found` if user cannot be found.
- `403 Forbidden` when trying to deactivate a user:
- Blocked by administrator or by LDAP synchronization.
- - That has any activity in past 90 days. These users cannot be deactivated.
+ - That is not [dormant](../user/admin_area/moderate_users.md#automatically-deactivate-dormant-users).
- That is internal.
## Activate user **(FREE SELF)**
diff --git a/doc/api/vulnerability_exports.md b/doc/api/vulnerability_exports.md
index 14966d2e925..8c166c2ba2a 100644
--- a/doc/api/vulnerability_exports.md
+++ b/doc/api/vulnerability_exports.md
@@ -198,18 +198,11 @@ The response is `404 Not Found` if the vulnerability export is not finished yet
Example response:
```csv
-Group Name,Project Name,Tool,Scanner Name,Status,Vulnerability,Details,Additional Info,Severity,CVE,CWE,Other Identifiers
-Gitlab.org,Defend,container_scanning,Trivy,detected,CVE-2017-16997 in glibc,,CVE-2017-16997 in glibc,critical,CVE-2017-16997
-Gitlab.org,Defend,container_scanning,Trivy,detected,CVE-2017-18269 in glibc,,CVE-2017-18269 in glibc,critical,CVE-2017-18269
-Gitlab.org,Defend,container_scanning,Trivy,detected,CVE-2018-1000001 in glibc,,CVE-2018-1000001 in glibc,high,CVE-2018-1000001
-Gitlab.org,Defend,container_scanning,Trivy,detected,CVE-2016-10228 in glibc,,CVE-2016-10228 in glibc,medium,CVE-2016-10228
-Gitlab.org,Defend,container_scanning,Trivy,detected,CVE-2010-4052 in glibc,,CVE-2010-4052 in glibc,low,CVE-2010-4052
-Gitlab.org,Defend,container_scanning,Trivy,detected,CVE-2018-18520 in elfutils,,CVE-2018-18520 in elfutils,low,CVE-2018-18520
-Gitlab.org,Defend,container_scanning,Trivy,detected,CVE-2018-16869 in nettle,,CVE-2018-16869 in nettle,unknown,CVE-2018-16869,CWE-1
-Gitlab.org,Defend,dependency_scanning,Gemnasium,detected,Regular Expression Denial of Service in debug,,Regular Expression Denial of Service in debug,unknown,CVE-2021-1234,CWE-2,"""yarn.lock:debug:gemnasium:37283ed4-0380-40d7-ada7-2d994afcc62a"""
-Gitlab.org,Defend,dependency_scanning,Gemnasium,detected,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,,Authentication bypass via incorrect DOM traversal and canonicalization in saml2-js,unknown,,,"""yarn.lock:saml2-js:gemnasium:9952e574-7b5b-46fa-a270-aeb694198a98"""
-Gitlab.org,Defend,sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,,,"""818bf5dacb291e15d9e6dc3c5ac32178:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:47"""
-Gitlab.org,Defend,sast,Find Security Bugs,detected,Cipher with no integrity,,Cipher with no integrity,medium,,,"""e6449b89335daf53c0db4c0219bc1634:CIPHER_INTEGRITY:src/main/java/com/gitlab/security_products/tests/App.java:29"""
-Gitlab.org,Defend,sast,Find Security Bugs,detected,Predictable pseudorandom number generator,,Predictable pseudorandom number generator,medium,,,"""e8ff1d01f74cd372f78da8f5247d3e73:PREDICTABLE_RANDOM:src/main/java/com/gitlab/security_products/tests/App.java:41"""
-Gitlab.org,Defend,sast,Find Security Bugs,detected,ECB mode is insecure,,ECB mode is insecure,medium,,,"""ea0f905fc76f2739d5f10a1fd1e37a10:ECB_MODE:src/main/java/com/gitlab/security_products/tests/App.java:29"""
+Group Name,Project Name,Tool,Scanner Name,Status,Vulnerability,Details,Additional Info,Severity,CVE,CWE,Other Identifiers,Detected At,Location,Activity,Comments,
+Gitlab.org,Defend,container_scanning,Trivy,resolved,CVE-2019-14697 in musl-utils-1.1.20-r4,"musl libc through 1.1.23 has an x87 floating-point stack adjustment imbalance, related to the math/i386/ directory. In some cases, use of this library could introduce out-of-bounds writes that are not present in an application's source code.",CVE-2019-14697 in musl-utils-1.1.20-r4,critical,CVE-2019-14697,,"",2022-10-07 13:34:41 UTC,"{""image""=>""python:3.4-alpine"", ""dependency""=>{""package""=>{""name""=>""musl-utils""}, ""version""=>""1.1.20-r4""}, ""operating_system""=>""alpine 3.9.2""}",true,"2022-10-07 13:41:08 UTC|root|resolved|changed vulnerability status to resolved",
+Gitlab.org,Defend,container_scanning,Trivy,detected,CVE-2019-19242 in sqlite-libs-3.26.0-r3,"SQLite 3.30.1 mishandles pExpr->y.pTab, as demonstrated by the TK_COLUMN case in sqlite3ExprCodeTarget in expr.c.",CVE-2019-19242 in sqlite-libs-3.26.0-r3,medium,CVE-2019-19242,,"",2022-10-07 13:34:41 UTC,"{""image""=>""python:3.4-alpine"", ""dependency""=>{""package""=>{""name""=>""sqlite-libs""}, ""version""=>""3.26.0-r3""}, ""operating_system""=>""alpine 3.9.2""}",true,"",
+Gitlab.org,Defend,container_scanning,Trivy,detected,CVE-2020-28928 in musl-1.1.20-r4,"In musl libc through 1.2.1, wcsnrtombs mishandles particular combinations of destination buffer size and source character limit, as demonstrated by an invalid write access (buffer overflow).",CVE-2020-28928 in musl-1.1.20-r4,medium,CVE-2020-28928,,"",2022-10-07 13:34:41 UTC,"{""image""=>""python:3.4-alpine"", ""dependency""=>{""package""=>{""name""=>""musl""}, ""version""=>""1.1.20-r4""}, ""operating_system""=>""alpine 3.9.2""}",true,"",
+Gitlab.org,Defend,dependency_scanning,Gemnasium,detected,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection') in rack,Carefully crafted requests can cause shell escape sequences to be written to the terminal via Rack's Lint middleware and CommonLogger middleware. These escape sequences can be leveraged to possibly execute commands in the victim's terminal.,Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection') in rack,unknown,Gemfile.lock:rack:gemnasium:60b5a27f-4e4d-4ab4-8ae7-74b4b212e177,,Gemnasium-60b5a27f-4e4d-4ab4-8ae7-74b4b212e177; GHSA-wq4h-7r42-5hrr,2022-10-14 13:16:00 UTC,"{""file""=>""Gemfile.lock"", ""dependency""=>{""package""=>{""name""=>""rack""}, ""version""=>""2.2.3""}}",false,"",
+Gitlab.org,Defend,dependency_scanning,Gemnasium,detected,Denial of Service Vulnerability in Rack Multipart Parsing in rack,"Carefully crafted multipart POST requests can cause Rack's multipart parser to take much longer than expected, leading to a possible denial of service vulnerability. Impacted code will use Rack's multipart parser to parse multipart posts.",Denial of Service Vulnerability in Rack Multipart Parsing in rack,unknown,Gemfile.lock:rack:gemnasium:20daa17a-47b5-4f79-80c2-cd8f2db9805c,,Gemnasium-20daa17a-47b5-4f79-80c2-cd8f2db9805c; GHSA-hxqx-xwvh-44m2,2022-10-14 13:16:00 UTC,"{""file""=>""Gemfile.lock"", ""dependency""=>{""package""=>{""name""=>""rack""}, ""version""=>""2.2.3""}}",false,"",
+Gitlab.org,Defend,sast,Brakeman,detected,Possible SQL injection,,Possible SQL injection,medium,e52f23a259cd489168b4313317ac94a3f13bffde57b9635171c1a44a9f329e9a,,"""Brakeman Warning Code 0""",2022-10-13 15:16:36 UTC,"{""file""=>""main.rb"", ""class""=>""User"", ""method""=>""index"", ""start_line""=>3}",false,""
```
diff --git a/doc/architecture/blueprints/_template.md b/doc/architecture/blueprints/_template.md
index 7637c3bf5fa..798d51a97ad 100644
--- a/doc/architecture/blueprints/_template.md
+++ b/doc/architecture/blueprints/_template.md
@@ -1,14 +1,11 @@
---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
status: proposed
creation-date: yyyy-mm-dd
authors: [ "@username" ]
coach: "@username"
-owning-section: "~section::<section>"
-participating-sections: []
approvers: [ "@product-manager", "@engineering-manager" ]
+owning-stage: "~devops::<stage>"
+participating-stages: []
---
<!--
diff --git a/doc/architecture/blueprints/ci_data_decay/index.md b/doc/architecture/blueprints/ci_data_decay/index.md
index 221c2364f79..b7c3bdde2f8 100644
--- a/doc/architecture/blueprints/ci_data_decay/index.md
+++ b/doc/architecture/blueprints/ci_data_decay/index.md
@@ -1,8 +1,11 @@
---
-stage: none
-group: unassigned
-comments: false
-description: 'CI/CD data time decay'
+status: ready
+creation-date: "2021-09-10"
+authors: [ "@grzesiek" ]
+coach: "@kamil"
+approvers: [ "@jporter", "@cheryl.li" ]
+owning-stage: "~devops::verify"
+participating-stages: []
---
# CI/CD data time decay
@@ -23,18 +26,6 @@ the data storage for pipeline builds remains almost the same since 2012. In
ia separate database. Now we want to improve the architecture of GitLab CI/CD
product to enable further scaling.
-_Disclaimer: The following contains information related to upcoming products,
-features, and functionality._
-
-_It is important to note that the information presented is for informational
-purposes only. Please do not rely on this information for purchasing or
-planning purposes._
-
-_As with all projects, the items mentioned in this document and linked pages are
-subject to change or delay. The development, release and timing of any
-products, features, or functionality remain at the sole discretion of GitLab
-Inc._
-
## Goals
**Implement a new architecture of CI/CD data storage to enable scaling.**
@@ -67,7 +58,7 @@ When a build gets archived it will not be possible to retry it, but we still do
keep all the processing metadata in the database, and it consumes resources
that are scarce in the primary database.
-In order to improve performance and make it easier to scale CI/CD data storage
+To improve performance and make it easier to scale CI/CD data storage
we might want to follow these three tracks described below.
![pipeline data time decay](pipeline_data_time_decay.png)
@@ -210,7 +201,7 @@ We accept the possible necessity of building a separate API endpoint /
endpoints needed to access pipeline data through the API.
In the new API users might need to provide a time range in which the data has
-been created to search through their pipelines / builds. In order to make it
+been created to search through their pipelines / builds. To make it
efficient it might be necessary to restrict access to querying data residing in
more than two partitions at once. We can do that by supporting time ranges
spanning the duration that equals to the builds archival policy.
@@ -246,35 +237,4 @@ In progress.
- 2022-04-30: Additional [benchmarking started](https://gitlab.com/gitlab-org/gitlab/-/issues/361019) to evaluate impact.
- 2022-06-31: [Pipeline partitioning design](pipeline_partitioning.md) document [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87683) merged.
- 2022-09-01: Engineering effort started to implement partitioning.
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Grzegorz Bizon |
-| Engineering Leader | Cheryl Li |
-| Product Manager | Jackie Porter |
-| Architecture Evolution Coach | Kamil Trzciński |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Leadership | Cheryl Li |
-| Product | Jackie Porter |
-| Engineering | Grzegorz Bizon |
-
-Domain experts:
-
-| Area | Who
-|------------------------------|------------------------|
-| Verify / Pipeline execution | Fabio Pitino |
-| Verify / Pipeline execution | Marius Bobin |
-| Verify / Pipeline insights | Maxime Orefice |
-| PostgreSQL Database | Andreas Brandl |
-
-<!-- vale gitlab.Spelling = YES -->
+- 2022-11-01: The fastest growing CI table partitioned: `ci_builds_metadata`.
diff --git a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md
index 5f907ecdaa4..d61412ae1ed 100644
--- a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md
+++ b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md
@@ -7,18 +7,6 @@ description: 'Pipeline data partitioning design'
# Pipeline data partitioning design
-_Disclaimer: The following contains information related to upcoming products,
-features, and functionality._
-
-_It is important to note that the information presented is for informational
-purposes only. Please do not rely on this information for purchasing or
-planning purposes._
-
-_As with all projects, the items mentioned in this document and linked pages
-are subject to change or delay. The development, release and timing of any
-products, features, or functionality remain at the sole discretion of GitLab
-Inc._
-
## What problem are we trying to solve?
We want to partition the CI/CD dataset, because some of the database tables are
@@ -267,9 +255,27 @@ new routing tables. Depending on the chosen
[partitioning strategy](#how-do-we-want-to-partition-cicd-data) for a given
table, it is possible to have many logical partitions per one physical partition.
+### Attaching first partition and acquiring locks
+
+We learned when [partitioning](https://gitlab.com/gitlab-org/gitlab/-/issues/378644)
+the first table that `PostgreSQL` requires an `AccessExclusiveLock` on the table and
+all of the other tables that it references through foreign keys. This can cause a deadlock
+if the migration tries to acquire the locks in a different order from the application
+business logic.
+
+To solve this problem, we introduced a **priority locking strategy** to avoid
+further deadlock errors. This allows us to define the locking order and
+then try keep retrying aggressively until we acquire the locks or run out of retries.
+This process can take up to 40 minutes.
+
+With this strategy, we successfully acquired a lock on `ci_builds` table after 15 retries
+during a low traffic period([after `00:00 UTC`](https://dashboards.gitlab.net/d/web-main/web-overview?orgId=1&viewPanel=537181794&from=now-2d&to=now)).
+
+See an example of this strategy in our [partition tooling](../../../development/database/table_partitioning.md#step-6---create-parent-table-and-attach-existing-table-as-the-initial-partition)).
+
## Storing partitions metadata in the database
-In order to build an efficient mechanism that will be responsible for creating
+To build an efficient mechanism that will be responsible for creating
new partitions, and to implement time decay we want to introduce a partitioning
metadata table, called `ci_partitions`. In that table we would store metadata
about all the logical partitions, with many pipelines per partition. We may
@@ -366,6 +372,20 @@ scope block takes an argument). Preloading instance dependent scopes is not
supported.
```
+### Query analyzers
+
+We implemented 2 query analyzers to detect queries that need to be fixed so that everything
+keeps working with partitioned tables:
+
+- One analyzer to detect queries not going through a routing table.
+- One analyzer to detect queries that use routing tables without specifying the `partition_id` in the `WHERE` clauses.
+
+We started by enabling our first analyzer in `test` environment to detect existing broken
+queries. It is also enabled on `production` environment, but for a small subset of the traffic (`0.1%`)
+because of scalability concerns.
+
+The second analyzer will be enabled in a future iteration.
+
### Primary key
Primary key must include the partitioning key column to partition the table.
@@ -652,7 +672,7 @@ application-wide outage.
1. Make it possible to create partitions in an automatic way.
1. Deliver the new architecture to self-managed instances.
-The diagram below visualizes this plan on Gantt chart. Please note that dates
+The diagram below visualizes this plan on Gantt chart. The dates
on the chart below are just estimates to visualize the plan better, these are
not deadlines and can change at any time.
diff --git a/doc/architecture/blueprints/ci_pipeline_components/index.md b/doc/architecture/blueprints/ci_pipeline_components/index.md
index 115f6909d2d..a3c72227f3e 100644
--- a/doc/architecture/blueprints/ci_pipeline_components/index.md
+++ b/doc/architecture/blueprints/ci_pipeline_components/index.md
@@ -1,12 +1,13 @@
---
-stage: Stage
-group: Pipeline Authoring
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: 'Create a catalog of shareable pipeline constructs'
+status: proposed
+creation-date: "2022-09-14"
+authors: [ "@fabio", "@grzesiek" ]
+coach: "@kamil"
+approvers: [ "@dov" ]
+owning-stage: "~devops::verify"
+participating-stages: []
---
-
# CI/CD pipeline components catalog
## Summary
@@ -115,21 +116,22 @@ while encapsulating and isolating implementation details.
Components allow a pipeline to be assembled by using abstractions instead of having all the details defined in one place.
When using a component in a pipeline, a user shouldn't need to know the implementation details of the component and should
-only rely on the provided interface. The interface will have a version / revision, so that users understand which revision they are interfacing with.
+only rely on the provided interface.
A pipeline component defines its type which indicates in which context of the pipeline configuration the component can be used.
For example, a component of type X can only be used according to the type X use-case.
-For best experience with any systems made of components it's fundamental that components are single purpose,
-isolated, reusable and resolvable.
+For best experience with any systems made of components it's fundamental that components:
- **Single purpose**: a component must focus on a single goal and the scope be as small as possible.
-- **Isolation**: when a component is used in a pipeline, its implementation details should not leak outside the
+- **Isolated**: when a component is used in a pipeline, its implementation details should not leak outside the
component itself and into the main pipeline.
-- **Reusability:** a component is designed to be used in different pipelines.
+- **Reusable**: a component is designed to be used in different pipelines.
Depending on the assumptions it's built on a component can be more or less generic.
Generic components are more reusable but may require more customization.
-- **Resolvable:** When a component depends on another component, this dependency must be explicit and trackable.
+- **Versioned**: when using a component we must specify the version we are interested in.
+ The version identifies the exact interface and behavior of the component.
+- **Resolvable**: when a component depends on another component, this dependency must be explicit and trackable.
## Proposal
@@ -186,35 +188,3 @@ Some limits we could consider adding:
- Allow self-managed administrators to populate their self-managed catalog by importing/updating
components from GitLab.com or from repository exports.
- Iterate on feedback.
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Fabio Pitino |
-| Engineering Leader | ? |
-| Product Manager | Dov Hershkovitch |
-| Architecture Evolution Coach | Kamil Trzciński |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Leadership | ? |
-| Product | Dov Hershkovitch |
-| Engineering | ? |
-| UX | Nadia Sotnikova |
-
-Domain experts:
-
-| Area | Who
-|------------------------------|------------------------|
-| Verify / Pipeline authoring | Avielle Wolfe |
-| Verify / Pipeline authoring | Furkan Ayhan |
-| Verify / Pipeline execution | Fabio Pitino |
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/cloud_native_build_logs/index.md b/doc/architecture/blueprints/cloud_native_build_logs/index.md
index b77d7998fc8..20cfb46abc4 100644
--- a/doc/architecture/blueprints/cloud_native_build_logs/index.md
+++ b/doc/architecture/blueprints/cloud_native_build_logs/index.md
@@ -1,9 +1,11 @@
---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: 'Next iteration of build logs architecture at GitLab'
+status: implemented
+creation-date: "2020-08-26"
+authors: [ "@grzesiek" ]
+coach: "@kamil"
+approvers: [ "@thaoyeager", "@darbyfrey" ]
+owning-stage: "~devops::release"
+participating-stages: []
---
# Cloud Native Build Logs
@@ -31,7 +33,7 @@ a job is complete, the trace file contents are sent to the object store.
New architecture writes data to Redis instead of writing build logs into a
file.
-In order to make this performant and resilient enough, we implemented a chunked
+To make this performant and resilient enough, we implemented a chunked
I/O mechanism - we store data in Redis in chunks, and migrate them to an object
store once we reach a desired chunk size.
@@ -121,27 +123,3 @@ Enabling this feature on GitLab.com is a subtask of
This change has been implemented and enabled on GitLab.com.
We are working on [an epic to make this feature more resilient and observable](https://gitlab.com/groups/gitlab-org/-/epics/4860).
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Grzegorz Bizon |
-| Architecture Evolution Coach | Gerardo Lopez-Fernandez |
-| Engineering Leader | Darby Frey |
-| Domain Expert | Kamil Trzciński |
-| Domain Expert | Sean McGivern |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Product | Thao Yeager |
-| Leadership | Darby Frey |
-| Engineering | Grzegorz Bizon |
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/cloud_native_gitlab_pages/index.md b/doc/architecture/blueprints/cloud_native_gitlab_pages/index.md
index 127badabb71..b6f3a59dc0b 100644
--- a/doc/architecture/blueprints/cloud_native_gitlab_pages/index.md
+++ b/doc/architecture/blueprints/cloud_native_gitlab_pages/index.md
@@ -1,9 +1,11 @@
---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: 'Making GitLab Pages a Cloud Native application - architecture blueprint.'
+status: implemented
+creation-date: "2019-05-16"
+authors: [ "@grzesiek" ]
+coach: "@kamil"
+approvers: [ "@ogolowinski", "@dcroft", "@vshushlin" ]
+owning-stage: "~devops::release"
+participating-stages: []
---
# GitLab Pages New Architecture
@@ -100,38 +102,3 @@ too.
[GitLab Pages Architecture](https://gitlab.com/groups/gitlab-org/-/epics/1316)
epic with detailed roadmap is also available.
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Grzegorz Bizon |
-| Architecture Evolution Coach | Kamil Trzciński |
-| Engineering Leader | Daniel Croft |
-| Domain Expert | Grzegorz Bizon |
-| Domain Expert | Vladimir Shushlin |
-| Domain Expert | Jaime Martinez |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Product | Orit Golowinski |
-| Leadership | Daniel Croft |
-| Engineering | Vladimir Shushlin |
-
-Domain Experts:
-
-| Role | Who
-|------------------------------|------------------------|
-| Domain Expert | Kamil Trzciński |
-| Domain Expert | Grzegorz Bizon |
-| Domain Expert | Vladimir Shushlin |
-| Domain Expert | Jaime Martinez |
-| Domain Expert | Krasimir Angelov |
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md b/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md
index 4111e2ef056..53f38fa85fd 100644
--- a/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md
+++ b/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md
@@ -1,16 +1,18 @@
---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: 'Making a GitLab codebase composable - allowing to run parts of the application'
+status: proposed
+creation-date: "2021-05-19"
+authors: [ "@kamil", "@mkaeppler" ]
+coach: "@glopezfernandez"
+approvers: []
+owning-stage: "~devops::non_devops"
+participating-stages: []
---
+# Composable GitLab codebase - using Rails Engines
+
NOTE:
Due to our focus on improving the overall availability of GitLab.com and reducing tech debt, we do not have capacity to act on this blueprint. We will re-evaluate in Q1-FY23.
-# Composable GitLab codebase - using Rails Engines
-
The one of the major risks of a single codebase is an infinite growth of the whole
application. The more code being added results in not only ever increasing resource requirements
for running the application, but increased application coupling and explosion of the complexity.
@@ -585,33 +587,3 @@ to be created to ensure that we do not have explosion of engines.
- [Use nested structure to organize CI classes](https://gitlab.com/gitlab-org/gitlab/-/issues/209745)
- [WIP: Make it simple to build and use "Decoupled Services"](https://gitlab.com/gitlab-org/gitlab/-/issues/31121)
- [Rails takes awhile to boot, let's see if we can improve this](https://gitlab.com/gitlab-org/gitlab/-/issues/213992)
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Kamil Trzciński |
-| Architecture Evolution Coach | ? |
-| Engineering Leader | ? |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Product | ? |
-| Leadership | Craig Gomes |
-| Engineering | ? |
-
-Domain Experts:
-
-| Role | Who
-|------------------------------|------------------------|
-| Domain Expert | Nikola Milojevic |
-| Domain Expert | ? |
-| Domain Expert | ? |
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/consolidating_groups_and_projects/index.md b/doc/architecture/blueprints/consolidating_groups_and_projects/index.md
index 433c23bf188..0818d9b973d 100644
--- a/doc/architecture/blueprints/consolidating_groups_and_projects/index.md
+++ b/doc/architecture/blueprints/consolidating_groups_and_projects/index.md
@@ -1,9 +1,11 @@
---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: Consolidating groups and projects
+status: proposed
+creation-date: "2021-02-07"
+authors: [ "@alexpooley", "@ifarkas" ]
+coach: "@grzesiek"
+approvers: [ "@m_gill", "@mushakov" ]
+owning-stage: "~devops::plan"
+participating-stages: []
---
# Consolidating Groups and Projects
@@ -143,34 +145,6 @@ The initial iteration will provide a framework to house features under `Namespac
- Start small: What are the product changes that need to be made to assist with the migration?
- Move fast: Prioritise these solution ideas, document in issues, and create a roadmap for implementation.
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------------------|
-| Author | Alex Pooley, Imre Farkas |
-| Architecture Evolution Coach | Dmitriy Zaporozhets, Grzegorz Bizon |
-| Engineering Leader | Michelle Gill |
-| Domain Expert | Jan Provaznik |
-
-<!-- vale gitlab.Spelling = YES -->
-
-DRIs:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|------------------------|
-| Product | Melissa Ushakov |
-| Leadership | Michelle Gill |
-| Engineering | Imre Farkas |
-| Design | Nick Post |
-
-<!-- vale gitlab.Spelling = YES -->
-
## Related topics
- [Workspace developer documentation](../../../development/workspace/index.md)
diff --git a/doc/architecture/blueprints/container_registry_metadata_database/index.md b/doc/architecture/blueprints/container_registry_metadata_database/index.md
index 58d59fe5737..63e27286756 100644
--- a/doc/architecture/blueprints/container_registry_metadata_database/index.md
+++ b/doc/architecture/blueprints/container_registry_metadata_database/index.md
@@ -1,9 +1,11 @@
---
-stage: Package
-group: Package
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: 'Container Registry metadata database'
+status: implemented
+creation-date: "2020-09-29"
+authors: [ "@jdrpereira" ]
+coach: "@glopezfernandez"
+approvers: [ "@trizzi", "@hswimelar" ]
+owning-stage: "~devops::package"
+participating-stages: []
---
# Container Registry Metadata Database
@@ -344,24 +346,3 @@ A more detailed list of all tasks, as well as periodic progress updates can be f
- [Gradual migration proposal for the GitLab.com container registry](https://gitlab.com/gitlab-org/container-registry/-/issues/191)
- [Create a self-serve registry deployment](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/316)
- [Database cluster for container registry](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/11154)
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | João Pereira |
-| Architecture Evolution Coach | Gerardo Lopez-Fernandez |
-| Engineering Leader | |
-| Domain Expert | Hayley Swimelar |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Product | Tim Rizzi |
-| Leadership | |
-| Engineering | João Pereira |
diff --git a/doc/architecture/blueprints/database/scalability/patterns/read_mostly.md b/doc/architecture/blueprints/database/scalability/patterns/read_mostly.md
index 6cf8e17edeb..ec236c9bfe3 100644
--- a/doc/architecture/blueprints/database/scalability/patterns/read_mostly.md
+++ b/doc/architecture/blueprints/database/scalability/patterns/read_mostly.md
@@ -8,7 +8,7 @@ description: 'Learn how to scale operating on read-mostly data at scale'
# Read-mostly data
-[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326037) in GitLab 14.0.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326037) in GitLab 14.0.
This document describes the *read-mostly* pattern introduced in the
[Database Scalability Working Group](https://about.gitlab.com/company/team/structure/working-groups/database-scalability/#read-mostly-data).
diff --git a/doc/architecture/blueprints/database/scalability/patterns/time_decay.md b/doc/architecture/blueprints/database/scalability/patterns/time_decay.md
index ff5f7c25ea1..93f5dffd3f5 100644
--- a/doc/architecture/blueprints/database/scalability/patterns/time_decay.md
+++ b/doc/architecture/blueprints/database/scalability/patterns/time_decay.md
@@ -8,7 +8,7 @@ description: 'Learn how to operate on large time-decay data'
# Time-decay data
-[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326035) in GitLab 14.0.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326035) in GitLab 14.0.
This document describes the *time-decay pattern* introduced in the
[Database Scalability Working Group](https://about.gitlab.com/company/team/structure/working-groups/database-scalability/#time-decay-data).
diff --git a/doc/architecture/blueprints/database_scaling/size-limits.md b/doc/architecture/blueprints/database_scaling/size-limits.md
index 0bb1ae9efb4..e530bd6eff0 100644
--- a/doc/architecture/blueprints/database_scaling/size-limits.md
+++ b/doc/architecture/blueprints/database_scaling/size-limits.md
@@ -117,7 +117,7 @@ limit 30;
NOTE:
In PostgreSQL context, a **physical table** is either a regular table or a partition of a partitioned table.
-In order to maintain and improve operational stability and lessen development burden, we target a **table size less than 100 GB for a physical table on GitLab.com** (including its indexes). This has numerous benefits:
+To maintain and improve operational stability and lessen development burden, we target a **table size less than 100 GB for a physical table on GitLab.com** (including its indexes). This has numerous benefits:
1. Improved query performance and more stable query plans
1. Significantly reduce vacuum run times and increase frequency of vacuum runs to maintain a healthy state - reducing overhead on the database primary
diff --git a/doc/architecture/blueprints/database_testing/index.md b/doc/architecture/blueprints/database_testing/index.md
index 3f8041ea416..fe6dcf1723d 100644
--- a/doc/architecture/blueprints/database_testing/index.md
+++ b/doc/architecture/blueprints/database_testing/index.md
@@ -1,9 +1,11 @@
---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: 'Database Testing'
+status: accepted
+creation-date: "2021-02-08"
+authors: [ "@abrandl" ]
+coach: "@glopezfernandez"
+approvers: [ "@fabian", "@craig-gomes" ]
+owning-stage: "~devops::data_stores"
+participating-stages: []
---
# Database Testing
@@ -122,26 +124,3 @@ An alternative approach we have discussed and abandoned is to "scrub" and anonym
- Annotating data as "sensitive" is error prone, with the wrong anonymization approach used for a data type or one sensitive attribute accidentally not marked as such possibly leading to a data breach.
- Scrubbing not only removes sensitive data, but it also changes data distribution, which greatly affects performance of migrations and queries.
- Scrubbing heavily changes the database contents, potentially updating a lot of data, which leads to different data storage details (think MVC bloat), affecting performance of migrations and queries.
-
-## Who
-
-<!-- vale gitlab.Spelling = NO -->
-
-This effort is owned and driven by the [GitLab Database Team](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/database/) with support from the [GitLab.com Reliability Datastores](https://about.gitlab.com/handbook/engineering/infrastructure/team/reliability/) team.
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Andreas Brandl |
-| Architecture Evolution Coach | Gerardo Lopez-Fernandez |
-| Engineering Leader | Craig Gomes |
-| Domain Expert | Yannis Roussos |
-| Domain Expert | Pat Bair |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Product | Fabian Zimmer |
-| Engineering | Andreas Brandl |
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/feature_flags_development/index.md b/doc/architecture/blueprints/feature_flags_development/index.md
index 866be9d8a70..730daf56f0d 100644
--- a/doc/architecture/blueprints/feature_flags_development/index.md
+++ b/doc/architecture/blueprints/feature_flags_development/index.md
@@ -1,9 +1,11 @@
---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: 'Internal usage of Feature Flags for GitLab development'
+status: accepted
+creation-date: "2020-06-10"
+authors: [ "@kamil" ]
+coach: "@glopezfernandez"
+approvers: [ "@kencjohnston", "@craig-gomes" ]
+owning-stage: "~devops::non_devops"
+participating-stages: []
---
# Architectural discussion of feature flags
@@ -118,26 +120,3 @@ These are reason why these changes are needed:
This work is being done as part of dedicated epic:
[Improve internal usage of Feature Flags](https://gitlab.com/groups/gitlab-org/-/epics/3551).
This epic describes a meta reasons for making these changes.
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Kamil Trzciński |
-| Architecture Evolution Coach | Gerardo Lopez-Fernandez |
-| Engineering Leader | Kamil Trzciński |
-| Domain Expert | Shinya Maeda |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Product | Kenny Johnston |
-| Leadership | Craig Gomes |
-| Engineering | Kamil Trzciński |
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/gitlab_to_kubernetes_communication/index.md b/doc/architecture/blueprints/gitlab_to_kubernetes_communication/index.md
index 19fd995bead..6ac67dd0f18 100644
--- a/doc/architecture/blueprints/gitlab_to_kubernetes_communication/index.md
+++ b/doc/architecture/blueprints/gitlab_to_kubernetes_communication/index.md
@@ -1,9 +1,11 @@
---
-stage: Configure
-group: Configure
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: 'GitLab to Kubernetes communication'
+status: implemented
+creation-date: "2020-12-03"
+authors: [ "@ash2k" ]
+coach: "@andrewn"
+approvers: [ "@nicholasklick", "@nagyv-gitlab" ]
+owning-stage: "~devops::configure"
+participating-stages: []
---
# GitLab to Kubernetes communication **(FREE)**
@@ -137,28 +139,3 @@ flowchart LR
### Iterations
Iterations are tracked in [the dedicated epic](https://gitlab.com/groups/gitlab-org/-/epics/4591).
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Mikhail Mazurskiy |
-| Architecture Evolution Coach | Andrew Newdigate |
-| Engineering Leader | Nicholas Klick |
-| Domain Expert | Thong Kuah |
-| Domain Expert | Graeme Gillies |
-| Security Expert | Vitor Meireles De Sousa |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Product Lead | Viktor Nagy |
-| Engineering Leader | Nicholas Klick |
-| Domain Expert | Mikhail Mazurskiy |
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/graphql_api/index.md b/doc/architecture/blueprints/graphql_api/index.md
index 1ee322c412b..4b446a78541 100644
--- a/doc/architecture/blueprints/graphql_api/index.md
+++ b/doc/architecture/blueprints/graphql_api/index.md
@@ -1,8 +1,11 @@
---
-stage: none
-group: unassigned
-comments: false
-description: 'GraphQL API architecture foundation'
+status: accepted
+creation-date: "2021-01-07"
+authors: [ "@grzesiek" ]
+coach: "@kamil"
+approvers: [ "@dsatcher", "@deuley" ]
+owning-stage: "~devops::manage"
+participating-stages: []
---
# GraphQL API
@@ -155,43 +158,3 @@ state synchronization mechanisms and hooking into existing ones.
1. [Build a scalable state synchronization for GraphQL](https://gitlab.com/groups/gitlab-org/-/epics/5319)
1. [Add support for direct uploads for GraphQL](https://gitlab.com/gitlab-org/gitlab/-/issues/280819)
1. [Review GraphQL design choices related to security](https://gitlab.com/gitlab-org/security/gitlab/-/issues/339)
-
-## Status
-
-Current status: in progress.
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Grzegorz Bizon |
-| Architecture Evolution Coach | Kamil Trzciński |
-| Engineering Leader | Darva Satcher |
-| Product Manager | Patrick Deuley |
-| Domain Expert / GraphQL | Charlie Ablett |
-| Domain Expert / GraphQL | Alex Kalderimis |
-| Domain Expert / GraphQL | Natalia Tepluhina |
-| Domain Expert / Scalability | Bob Van Landuyt |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Leadership | Darva Satcher |
-| Product | Patrick Deuley |
-| Engineering | Paul Slaughter |
-
-Domain Experts:
-
-| Area | Who
-|------------------------------|------------------------|
-| Domain Expert / GraphQL | Charlie Ablett |
-| Domain Expert / GraphQL | Alex Kalderimis |
-| Domain Expert / GraphQL | Natalia Tepluhina |
-| Domain Expert / Scalability | Bob Van Landuyt |
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/image_resizing/index.md b/doc/architecture/blueprints/image_resizing/index.md
index dd7ce27f459..948378d8834 100644
--- a/doc/architecture/blueprints/image_resizing/index.md
+++ b/doc/architecture/blueprints/image_resizing/index.md
@@ -1,9 +1,11 @@
---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
-description: 'Image Resizing'
+status: implemented
+creation-date: "2020-10-21"
+authors: [ "@craig-gomes" ]
+coach: "@kamil"
+approvers: [ "@timzallmann", "@joshlambert" ]
+owning-stage: "~devops::non_devops"
+participating-stages: []
---
# Image resizing for avatars and content images
diff --git a/doc/architecture/blueprints/object_storage/index.md b/doc/architecture/blueprints/object_storage/index.md
index 7a4ecd0e5a8..61dc37d7706 100644
--- a/doc/architecture/blueprints/object_storage/index.md
+++ b/doc/architecture/blueprints/object_storage/index.md
@@ -1,8 +1,11 @@
---
-stage: none
-group: unassigned
-comments: false
-description: 'Object storage: direct_upload consolidation - architecture blueprint.'
+status: ready
+creation-date: "2021-11-18"
+authors: [ "@nolith" ]
+coach: "@glopezfernandez"
+approvers: [ "@marin" ]
+owning-stage: "~devops::data_stores"
+participating-stages: []
---
# Object storage: `direct_upload` consolidation
@@ -197,24 +200,3 @@ require one bucket.
- [Speed up the monolith, building a smart reverse proxy in Go](https://archive.fosdem.org/2020/schedule/event/speedupmonolith/): a presentation explaining a bit of workhorse history and the challenge we faced in releasing the first cloud-native installation.
- [Object Storage improvements epic](https://gitlab.com/groups/gitlab-org/-/epics/483).
- We are moving to GraphQL API, but [we do not support direct upload](https://gitlab.com/gitlab-org/gitlab/-/issues/280819).
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who |
-|--------------------------------|-------------------------|
-| Author | Alessio Caiazza |
-| Architecture Evolution Coach | Gerardo Lopez-Fernandez |
-| Engineering Leader | Marin Jankovski |
-| Domain Expert / Object storage | Stan Hu |
-| Domain Expert / Security | Joern Schneeweisz |
-
-DRIs:
-
-The DRI for this blueprint is the
-[Object Storage Working Group](https://about.gitlab.com/company/team/structure/working-groups/object-storage/).
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/pods/images/iteration0-organizations-introduction.png b/doc/architecture/blueprints/pods/images/iteration0-organizations-introduction.png
new file mode 100644
index 00000000000..5725b0fa71f
--- /dev/null
+++ b/doc/architecture/blueprints/pods/images/iteration0-organizations-introduction.png
Binary files differ
diff --git a/doc/architecture/blueprints/pods/images/pods-and-fulfillment.png b/doc/architecture/blueprints/pods/images/pods-and-fulfillment.png
new file mode 100644
index 00000000000..aab8556a5d3
--- /dev/null
+++ b/doc/architecture/blueprints/pods/images/pods-and-fulfillment.png
Binary files differ
diff --git a/doc/architecture/blueprints/pods/images/term-cluster.png b/doc/architecture/blueprints/pods/images/term-cluster.png
new file mode 100644
index 00000000000..87e4d631551
--- /dev/null
+++ b/doc/architecture/blueprints/pods/images/term-cluster.png
Binary files differ
diff --git a/doc/architecture/blueprints/pods/images/term-organization.png b/doc/architecture/blueprints/pods/images/term-organization.png
new file mode 100644
index 00000000000..4c82c62b8f4
--- /dev/null
+++ b/doc/architecture/blueprints/pods/images/term-organization.png
Binary files differ
diff --git a/doc/architecture/blueprints/pods/term-pod.png b/doc/architecture/blueprints/pods/images/term-pod.png
index d8f79df2f29..d8f79df2f29 100644
--- a/doc/architecture/blueprints/pods/term-pod.png
+++ b/doc/architecture/blueprints/pods/images/term-pod.png
Binary files differ
diff --git a/doc/architecture/blueprints/pods/term-top-level-namespace.png b/doc/architecture/blueprints/pods/images/term-top-level-namespace.png
index c1cd317d878..c1cd317d878 100644
--- a/doc/architecture/blueprints/pods/term-top-level-namespace.png
+++ b/doc/architecture/blueprints/pods/images/term-top-level-namespace.png
Binary files differ
diff --git a/doc/architecture/blueprints/pods/index.md b/doc/architecture/blueprints/pods/index.md
index 01d56c483ea..3ba319d169b 100644
--- a/doc/architecture/blueprints/pods/index.md
+++ b/doc/architecture/blueprints/pods/index.md
@@ -1,15 +1,15 @@
---
-stage: enablement
-group: pods
-comments: false
-description: 'Pods'
+status: accepted
+creation-date: "2022-09-07"
+authors: [ "@fzimmer", "@DylanGriffith" ]
+coach: "@kamil"
+approvers: [ "@fzimmer" ]
+owning-stage: "~devops::enablement"
+participating-stages: []
---
# Pods
-DISCLAIMER:
-This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
-
This document is a work-in-progress and represents a very early state of the Pods design. Significant aspects are not documented, though we expect to add them in the future.
## Summary
@@ -24,7 +24,7 @@ We use the following terms to describe components and properties of the Pods arc
A Pod is a set of infrastructure components that contains multiple top-level namespaces that belong to different organizations. The components include both datastores (PostgreSQL, Redis etc.) and stateless services (web etc.). The infrastructure components provided within a Pod are shared among organizations and their top-level namespaces but not shared with other Pods. This isolation of infrastructure components means that Pods are independent from each other.
-![Term Pod](term-pod.png)
+![Term Pod](images/term-pod.png)
#### Pod properties
@@ -42,7 +42,7 @@ Discouraged synonyms: GitLab instance, cluster, shard
A cluster is a collection of Pods.
-![Term Cluster](term-cluster.png)
+![Term Cluster](images/term-cluster.png)
#### Cluster properties
@@ -66,7 +66,7 @@ Organizations work under the following assumptions:
1. Users understand that the majority of pages they view are only scoped to a single organization at a time.
1. Organizations are located on a single pod.
-![Term Organization](term-organization.png)
+![Term Organization](images/term-organization.png)
#### Organization properties
@@ -94,7 +94,7 @@ Top-level namespaces may [be replaced by workspaces](https://gitlab.com/gitlab-o
Discouraged synonyms: Root-level namespace
-![Term Top-level Namespace](term-top-level-namespace.png)
+![Term Top-level Namespace](images/term-top-level-namespace.png)
#### Top-level namespace properties
@@ -111,8 +111,8 @@ Users are available globally and not restricted to a single Pod. Users can be me
- Users can create multiple top-level namespaces
- Users can be a member of multiple top-level namespaces
- Users can be a member of multiple organizations
-- Users can administrate organizations
-- User activity is aggregated within an organization
+- Users can administer organizations
+- User activity is aggregated in an organization
- Every user has one personal namespace
## Goals
@@ -160,6 +160,59 @@ A number of technical issues need to be resolved to implement Pods (in no partic
1. How are Pods provisioned?
1. How can Pods implement disaster recovery capabilities?
+## Cross-section impact
+
+Pods is a fundamental architecture change that impacts other sections and stages. This section summarizes and links to other groups that may be impacted and highlights potential conflicts that need to be resolved. The Pods group is not responsible for achieving the goals of other groups but we want to ensure that dependencies are resolved.
+
+### Summary
+
+Based on discussions with other groups the net impact of introducing Pods and a new entity called organizations is mostly neutral. It may slow down development in some areas. We did not discover major blockers for other teams.
+
+1. We need to resolve naming conflicts (proposal is TBD)
+1. Pods requires introducing Organizations. Organizations are a new entity **above** top-level groups. Because this is a new entity, it may impact the ability to consolidate settings for Group Workspace and influence their decision on [how to approach introducing a workspace](https://gitlab.com/gitlab-org/gitlab/-/issues/376285#approach-2-workspace-is-built-on-top-of-top-level-groups)
+1. Organizations may make it slightly easier for Fulfillment to realize their billing plans.
+
+### Impact on Group Manage Workspace
+
+We synced with the Workspace PM and Designer ([recording](https://youtu.be/b5Opn9cFWFk)) and discussed the similarities and differences between the Pods and Workspace proposal ([presentation](https://docs.google.com/presentation/d/1FsUi22Up15b_tu6p2m-yLML3hCZ3rgrZrmzJAxUsNmU/edit?usp=sharing)).
+
+#### Goals of Group Manage Workspace
+
+As defined in the [workspace documentation](../../../user/workspace/index.md):
+
+1. Create an entity to manage everything you do as a GitLab administrator, including:
+ 1. Defining and applying settings to all of your groups, subgroups, and projects.
+ 1. Aggregating data from all your groups, subgroups, and projects.
+1. Reach feature parity between SaaS and self-managed installations, with all Admin Area settings moving to groups (?). Hardware controls remain on the instance level.
+
+The [workspace roadmap outlines](https://gitlab.com/gitlab-org/gitlab/-/issues/368237#high-level-goals) the current goals in detail.
+
+#### Potential conflicts with Pods
+
+- Workspace and Organization are different terms for the same entity. Both define a new entity as the primary organizational object for groups and projects. This is mainly a semantic difference and **we need to decide on a name** following [user research to decide if workspace](https://gitlab.com/gitlab-org/ux-research/-/issues/2147). This is also driven by the fact that the Remote Development team is looking at better names and [are considering the term Workspace as well](https://gitlab.com/gitlab-com/Product/-/issues/4812).
+- We will only introduce one entity
+- Group workspace highlighted the need to further validate the key assumption that users only care about what happens within their organization.
+
+### Impact on Fulfillment
+
+We synced with Fulfillment ([recording](https://youtu.be/FkQF3uF7vTY)) to discuss how Pods would impact them. Fulfillment is supportive of an entity above top-level namespaces. Their perspective is outline in [!5639](https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/5639/diffs).
+
+#### Goals of Fulfillment
+
+- Fulfillment has a longstanding plan to move billing from the top-level namespace to a level above. This would mean that a license applies for an organization and all its top-level namespaces.
+- Fulfillment uses Zuora for billing and would like to have a 1-to-1 relationship between an organization and their Zuora entity called BillingAccount. They want to move away from tying a license to a single user.
+- If a customer needs multiple organizations, the corresponding BillingAccounts can be rolled up into a consolidated billing account (similar to [AWS consolidated billing](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/consolidated-billing.html))
+- Ideally, a self-managed instance has a single Organization by default, which should be enough for most customers.
+- Fulfillment prefers only one additional entity.
+
+A rough representation of this is:
+
+![Pods and Fulfillment](images/pods-and-fulfillment.png)
+
+#### Potential conflicts with Pods
+
+- There are no known conflicts between Fulfillment's plans and Pods
+
## Iteration plan
We can't ship the entire Pods architecture in one go - it is too large. Instead, we are adopting an iteration plan that provides value along the way.
@@ -189,7 +242,7 @@ Organizations solve the following problems:
1. Self-managed instances would set a default organization.
1. Organizations can control user-profiles in a central way. This could be achieved by having an organization specific user-profile. Such a profile makes it possible for the organization administrators to control the user role in a company, enforce user emails, or show a graphical indicator of a user being part of the organization. An example would be a "GitLab Employee stamp" on comments.
-![Move to Organizations](iteration0-organizations-introduction.png)
+![Move to Organizations](images/iteration0-organizations-introduction.png)
#### Why would customers opt-in to Organizations?
@@ -251,28 +304,31 @@ Based on user research, we may want to change certain features to work across or
- Specific features allow for cross-organization interactions, for example forking, search.
-### Links
+## Technical Proposals
+
+The Pods architecture do have long lasting implications to data processing, location, scalability and the GitLab architecture.
+This section links all different technical proposals that are being evaluated.
+
+- [Stateless Router That Uses a Cache to Pick Pod and Is Redirected When Wrong Pod Is Reached](proposal-stateless-router-with-buffering-requests.md)
+
+- [Stateless Router That Uses a Cache to Pick Pod and pre-flight `/api/v4/pods/learn`](proposal-stateless-router-with-routes-learning.md)
+
+## Impacted features
+
+The Pods architecture will impact many features requiring some of them to be rewritten, or changed significantly.
+This is the list of known affected features with the proposed solutions.
+
+- [Pods: Git Access](pods-feature-git-access.md)
+- [Pods: Data Migration](pods-feature-data-migration.md)
+- [Pods: Database Sequences](pods-feature-database-sequences.md)
+- [Pods: GraphQL](pods-feature-graphql.md)
+- [Pods: Organizations](pods-feature-organizations.md)
+- [Pods: Router Endpoints Classification](pods-feature-router-endpoints-classification.md)
+
+## Links
- [Internal Pods presentation](https://docs.google.com/presentation/d/1x1uIiN8FR9fhL7pzFh9juHOVcSxEY7d2_q4uiKKGD44/edit#slide=id.ge7acbdc97a_0_155)
- [Pods Epic](https://gitlab.com/groups/gitlab-org/-/epics/7582)
- [Database Group investigation](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/database/doc/root-namespace-sharding.html)
- [Shopify Pods architecture](https://shopify.engineering/a-pods-architecture-to-allow-shopify-to-scale)
- [Opstrace architecture](https://gitlab.com/gitlab-org/opstrace/opstrace/-/blob/main/docs/architecture/overview.md)
-
-### Who
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Fabian Zimmer |
-| Architecture Evolution Coach | Kamil Trzciński |
-| Engineering Leader | TBD |
-| Product Manager | Fabian Zimmer |
-| Domain Expert / Database | TBD |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Leadership | TBD |
-| Product | Fabian Zimmer |
-| Engineering | Thong Kuah |
diff --git a/doc/architecture/blueprints/pods/iteration0-organizations-introduction.png b/doc/architecture/blueprints/pods/iteration0-organizations-introduction.png
deleted file mode 100644
index 5f5cad7b169..00000000000
--- a/doc/architecture/blueprints/pods/iteration0-organizations-introduction.png
+++ /dev/null
Binary files differ
diff --git a/doc/architecture/blueprints/pods/pods-feature-data-migration.md b/doc/architecture/blueprints/pods/pods-feature-data-migration.md
new file mode 100644
index 00000000000..fad6bca45fa
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-data-migration.md
@@ -0,0 +1,82 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Data migration'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: Data migration
+
+It is essential for Pods architecture to provide a way to migrate data out of big Pods
+into smaller ones. This describes various approaches to provide this type of split.
+
+## 1. Definition
+
+## 2. Data flow
+
+## 3. Proposal
+
+### 3.1. Split large Pods
+
+A single Pod can only be divided into many Pods. This is based on principle
+that it is easier to create exact clone of an existing Pod in many replicas
+out of which some will be made authoritative once migrated. Keeping those
+replicas up-to date with Pod 0 is also much easier due to pre-existing
+replication solutions that can replicate the whole systems: Geo, PostgreSQL
+physical replication, etc.
+
+1. All data of an organization needs to not be divided across many Pods.
+1. Split should be doable online.
+1. New Pods cannot contain pre-existing data.
+1. N Pods contain exact replica of Pod 0.
+1. The data of Pod 0 is live replicated to as many Pods it needs to be split.
+1. Once consensus is achieved between Pod 0 and N-Pods the organizations to be migrated away
+ are marked as read-only cluster-wide.
+1. The `routes` is updated on for all organizations to be split to indicate an authorative
+ Pod holding the most recent data, like `gitlab-org` on `pod-100`.
+1. The data for `gitlab-org` on Pod 0, and on other non-authoritative N-Pods are dormant
+ and will be removed in the future.
+1. All accesses to `gitlab-org` on a given Pod are validated about `pod_id` of `routes`
+ to ensure that given Pod is authoritative to handle the data.
+
+### 3.2. Migrate organization from an existing Pod
+
+This is different to split, as we intend to perform logical and selective replication
+of data belonging to a single organization.
+
+Today this type of selective replication is only implemented by Gitaly where we can migrate
+Git repository from a single Gitaly node to another with minimal downtime.
+
+In this model we would require identifying all resources belonging to a given organization:
+database rows, object storage files, Git repositories, etc. and selectively copy them over
+to another (likely) existing Pod importing data into it. Ideally ensuring that we can
+perform logical replication live of all changed data, but change similarly to split
+which Pod is authoritative for this organization.
+
+1. It is hard to identify all resources belonging to organization.
+1. It requires either downtime for organization or a robust system to identify
+ live changes made.
+1. It likely will require a full database structure analysis (more robust than project import/export)
+ to perform selective PostgreSQL logical replication.
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-database-sequences.md b/doc/architecture/blueprints/pods/pods-feature-database-sequences.md
new file mode 100644
index 00000000000..0a8bb4d250e
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-database-sequences.md
@@ -0,0 +1,94 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Database Sequences'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: Database Sequences
+
+GitLab today ensures that every database row create has unique ID, allowing
+to access Merge Request, CI Job or Project by a known global ID.
+
+Pods will use many distinct and not connected databases, each of them having
+a separate IDs for most of entities.
+
+It might be desirable to retain globally unique IDs for all database rows
+to allow migrating resources between Pods in the future.
+
+## 1. Definition
+
+## 2. Data flow
+
+## 3. Proposal
+
+This are some preliminary ideas how we can retain unique IDs across the system.
+
+### 3.1. UUID
+
+Instead of using incremental sequences use UUID (128 bit) that is stored in database.
+
+- This might break existing IDs and requires adding UUID column for all existing tables.
+- This makes all indexes larger as it requires storing 128 bit instead of 32/64 bit in index.
+
+### 3.2. Use Pod index encoded in ID
+
+Since significant number of tables already use 64 bit ID numbers we could use MSB to encode
+Pod ID effectively enabling
+
+- This might limit amount of Pods that can be enabled in system, as we might decide to only
+ allocate 1024 possible Pod numbers.
+- This might make IDs to be migratable between Pods, since even if entity from Pod 1 is migrated to Pod 100
+ this ID would still be unique.
+- If resources are migrated the ID itself will not be enough to decode Pod number and we would need
+ lookup table.
+- This requires updating all IDs to 32 bits.
+
+### 3.3. Allocate sequence ranges from central place
+
+Each Pod might receive its own range of the sequences as they are consumed from a centrally managed place.
+Once Pod consumes all IDs assigned for a given table it would be replenished and a next range would be allocated.
+Ranges would be tracked to provide a faster lookup table if a random access pattern is required.
+
+- This might make IDs to be migratable between Pods, since even if entity from Pod 1 is migrated to Pod 100
+ this ID would still be unique.
+- If resources are migrated the ID itself will not be enough to decode Pod number and we would need
+ much more robust lookup table as we could be breaking previously assigned sequence ranges.
+- This does not require updating all IDs to 64 bits.
+- This adds some performance penalty to all `INSERT` statements in Postgres or at least from Rails as we need to check for the sequence number and potentially wait for our range to be refreshed from the ID server
+- The available range will need to be stored and incremented in a centralized place so that concurrent transactions cannot possibly get the same value.
+
+### 3.4. Define only some tables to require unique IDs
+
+Maybe this is acceptable only for some tables to have a globally unique IDs. It could be projects, groups
+and other top-level entities. All other tables like `merge_requests` would only offer Pod-local ID,
+but when referenced outside it would rather use IID (an ID that is monotonic in context of a given resource, like project).
+
+- This makes the ID 10000 for `merge_requests` be present on all Pods, which might be sometimes confusing
+ as for uniqueness of the resource.
+- This might make random access by ID (if ever needed) be impossible without using composite key, like: `project_id+merge_request_id`.
+- This would require us to implement a transformation/generation of new ID if we need to migrate records to another pod. This can lead to very difficult migration processes when these IDs are also used as foreign keys for other records being migrated.
+- If IDs need to change when moving between pods this means that any links to records by ID would no longer work even if those links included the `project_id`.
+- If we plan to allow these ids to not be unique and change the unique constraint to be based on a composite key then we'd need to update all foreign key references to be based on the composite key
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-git-access.md b/doc/architecture/blueprints/pods/pods-feature-git-access.md
new file mode 100644
index 00000000000..ae996281d46
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-git-access.md
@@ -0,0 +1,163 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Git Access'
+---
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: Git Access
+
+This document describes impact of Pods architecture on all Git access (over HTTPS and SSH)
+patterns providing explanantion of how potentially those features should be changed
+to work well with Pods.
+
+## 1. Definition
+
+Git access is done through out the application. It can be an operation performed by the system
+(read Git repository) or by user (create a new file via Web IDE, `git clone` or `git push` via command line).
+
+The Pods architecture defines that all Git repositories will be local to the Pod,
+so no repository could be shared with another Pod.
+
+The Pods architecture will require that any Git operation done can only be handled by a Pod holding
+the data. It means that any operation either via Web interface, API, or GraphQL needs to be routed
+to the correct Pod. It means that any `git clone` or `git push` operation can only be performed
+in a context of a Pod.
+
+## 2. Data flow
+
+The are various operations performed today by the GitLab on a Git repository. This describes
+the data flow how they behave today to better represent the impact.
+
+It appears that Git access does require changes only to a few endpoints that are scoped to project.
+There appear to be different types of repositories:
+
+- Project: assigned to Group
+- Wiki: additional repository assigned to Project
+- Design: similar to Wiki, additional repository assigned to Project
+- Snippet: creates a virtual project to hold repository, likely tied to the User
+
+### 2.1. Git clone over HTTPS
+
+Execution of: `git clone` over HTTPS
+
+```mermaid
+sequenceDiagram
+ User ->> Workhorse: GET /gitlab-org/gitlab.git/info/refs?service=git-upload-pack
+ Workhorse ->> Rails: GET /gitlab-org/gitlab.git/info/refs?service=git-upload-pack
+ Rails ->> Workhorse: 200 OK
+ Workhorse ->> Gitaly: RPC InfoRefsUploadPack
+ Gitaly ->> User: Response
+ User ->> Workhorse: POST /gitlab-org/gitlab.git/git-upload-pack
+ Workhorse ->> Gitaly: RPC PostUploadPackWithSidechannel
+ Gitaly ->> User: Response
+```
+
+### 2.2. Git clone over SSH
+
+Execution of: `git clone` over SSH
+
+```mermaid
+sequenceDiagram
+ User ->> Git SSHD: ssh git@gitlab.com
+ Git SSHD ->> Rails: GET /api/v4/internal/authorized_keys
+ Rails ->> Git SSHD: 200 OK (list of accepted SSH keys)
+ Git SSHD ->> User: Accept SSH
+ User ->> Git SSHD: git clone over SSH
+ Git SSHD ->> Rails: POST /api/v4/internal/allowed?project=/gitlab-org/gitlab.git&service=git-upload-pack
+ Rails ->> Git SSHD: 200 OK
+ Git SSHD ->> Gitaly: RPC SSHUploadPackWithSidechannel
+ Gitaly ->> User: Response
+```
+
+### 2.3. Git push over HTTPS
+
+Execution of: `git push` over HTTPS
+
+```mermaid
+sequenceDiagram
+ User ->> Workhorse: GET /gitlab-org/gitlab.git/info/refs?service=git-receive-pack
+ Workhorse ->> Rails: GET /gitlab-org/gitlab.git/info/refs?service=git-receive-pack
+ Rails ->> Workhorse: 200 OK
+ Workhorse ->> Gitaly: RPC PostReceivePack
+ Gitaly ->> Rails: POST /api/v4/internal/allowed?gl_repository=project-111&service=git-receive-pack
+ Gitaly ->> Rails: POST /api/v4/internal/pre_receive?gl_repository=project-111
+ Gitaly ->> Rails: POST /api/v4/internal/post_receive?gl_repository=project-111
+ Gitaly ->> User: Response
+```
+
+### 2.4. Git push over SSHD
+
+Execution of: `git clone` over SSH
+
+```mermaid
+sequenceDiagram
+ User ->> Git SSHD: ssh git@gitlab.com
+ Git SSHD ->> Rails: GET /api/v4/internal/authorized_keys
+ Rails ->> Git SSHD: 200 OK (list of accepted SSH keys)
+ Git SSHD ->> User: Accept SSH
+ User ->> Git SSHD: git clone over SSH
+ Git SSHD ->> Rails: POST /api/v4/internal/allowed?project=/gitlab-org/gitlab.git&service=git-receive-pack
+ Rails ->> Git SSHD: 200 OK
+ Git SSHD ->> Gitaly: RPC ReceivePack
+ Gitaly ->> Rails: POST /api/v4/internal/allowed?gl_repository=project-111
+ Gitaly ->> Rails: POST /api/v4/internal/pre_receive?gl_repository=project-111
+ Gitaly ->> Rails: POST /api/v4/internal/post_receive?gl_repository=project-111
+ Gitaly ->> User: Response
+```
+
+### 2.5. Create commit via Web
+
+Execution of `Add CHANGELOG` to repository:
+
+```mermaid
+sequenceDiagram
+ Web ->> Puma: POST /gitlab-org/gitlab/-/create/main
+ Puma ->> Gitaly: RPC TreeEntry
+ Gitaly ->> Rails: POST /api/v4/internal/allowed?gl_repository=project-111
+ Gitaly ->> Rails: POST /api/v4/internal/pre_receive?gl_repository=project-111
+ Gitaly ->> Rails: POST /api/v4/internal/post_receive?gl_repository=project-111
+ Gitaly ->> Puma: Response
+ Puma ->> Web: See CHANGELOG
+```
+
+## 3. Proposal
+
+The Pods stateless router proposal requires that any ambigious path (that is not routable)
+will be made to be routable. It means that at least the following paths will have to be updated
+do introduce a routable entity (project, group, or organization).
+
+Change:
+
+- `/api/v4/internal/allowed` => `/api/v4/internal/projects/<gl_repository>/allowed`
+- `/api/v4/internal/pre_receive` => `/api/v4/internal/projects/<gl_repository>/pre_receive`
+- `/api/v4/internal/post_receive` => `/api/v4/internal/projects/<gl_repository>/post_receive`
+- `/api/v4/internal/lfs_authenticate` => `/api/v4/internal/projects/<gl_repository>/lfs_authenticate`
+
+Where:
+
+- `gl_repository` can be `project-1111` (`Gitlab::GlRepository`)
+- `gl_repository` in some cases might be a full path to repository as executed by GitLab Shell (`/gitlab-org/gitlab.git`)
+
+## 4. Evaluation
+
+Supporting Git repositories if a Pod can access only its own repositories does not appear to be complex.
+
+The one major complication is supporting snippets, but this likely falls in the same category as for the approach
+to support user's personal namespaces.
+
+## 4.1. Pros
+
+1. The API used for supporting HTTPS/SSH and Hooks are well defined and can easily be made routable.
+
+## 4.2. Cons
+
+1. The sharing of repositories objects is limited to the given Pod and Gitaly node.
+1. The across-Pods forks are likely impossible to be supported (discover: how this work today across different Gitaly node).
diff --git a/doc/architecture/blueprints/pods/pods-feature-graphql.md b/doc/architecture/blueprints/pods/pods-feature-graphql.md
new file mode 100644
index 00000000000..5f8a39c0b3f
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-graphql.md
@@ -0,0 +1,94 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: GraphQL'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: GraphQL
+
+GitLab exensively uses GraphQL to perform efficient data query operations.
+GraphQL due to it's nature is not directly routable. The way how GitLab uses
+it calls the `/api/graphql` endpoint, and only query or mutation of body request
+might define where the data can be accessed.
+
+## 1. Definition
+
+## 2. Data flow
+
+## 3. Proposal
+
+There are at least two main ways to implement GraphQL in Pods architecture.
+
+### 3.1. GraphQL routable by endpoint
+
+Change `/api/graphql` to `/api/organization/<organization>/graphql`.
+
+- This breaks all existing usages of `/api/graphql` endpoint
+ since the API URI is changed.
+
+### 3.2. GraphQL routable by body
+
+As part of router parse GraphQL body to find a routable entity, like `project`.
+
+- This still makes the GraphQL query be executed only in context of a given Pod
+ and not allowing the data to be merged.
+
+```json
+# Good example
+{
+ project(fullPath:"gitlab-org/gitlab") {
+ id
+ description
+ }
+}
+
+# Bad example, since Merge Request is not routable
+{
+ mergeRequest(id: 1111) {
+ iid
+ description
+ }
+}
+```
+
+### 3.3. Merging GraphQL Proxy
+
+Implement as part of router GraphQL Proxy which can parse body
+and merge results from many Pods.
+
+- This might make pagination hard to achieve, or we might assume that
+ we execute many queries of which results are merged across all Pods.
+
+```json
+{
+ project(fullPath:"gitlab-org/gitlab"){
+ id, description
+ }
+ group(fullPath:"gitlab-com") {
+ id, description
+ }
+}
+```
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-organizations.md b/doc/architecture/blueprints/pods/pods-feature-organizations.md
new file mode 100644
index 00000000000..a0a87458767
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-organizations.md
@@ -0,0 +1,58 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Organizations'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: Organizations
+
+One of the major designs of Pods architecture is strong isolation between Groups.
+Organizations as described by this blueprint provides a way to have plausible UX
+for joining together many Groups that are isolated from the rest of systems.
+
+## 1. Definition
+
+Pods do require that all groups and projects of a single organization can
+only be stored on a single Pod since a Pod can only access data that it holds locally
+and has very limited capabilities to read information from other Pods.
+
+Pods with Organizations do require strong isolation between organizations.
+
+It will have significant implications on various user-facing features,
+like Todos, dropdowns allowing to select projects, references to other issues
+or projects, or any other social functions present at GitLab. Today those functions
+were able to reference anything in the whole system. With the introduction of
+organizations such will be forbidden.
+
+This problem definition aims to answer effort and implications required to add
+strong isolation between organizations to the system. Including features affected
+and their data processing flow. The purpose is to ensure that our solution when
+implemented consistently avoids data leakage between organizations residing on
+a single Pod.
+
+## 2. Data flow
+
+## 3. Proposal
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-router-endpoints-classification.md b/doc/architecture/blueprints/pods/pods-feature-router-endpoints-classification.md
new file mode 100644
index 00000000000..c672342fff9
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-router-endpoints-classification.md
@@ -0,0 +1,46 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Router Endpoints Classification'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: Router Endpoints Classification
+
+Classification of all endpoints is essential to properly route request
+hitting load balancer of a GitLab installation to a Pod that can serve it.
+
+Each Pod should be able to decode each request and classify for which Pod
+it belongs to.
+
+GitLab currently implements houndreds of endpoints. This document tries
+to describe various techniques that can be implemented to allow the Rails
+to provide this information efficiently.
+
+## 1. Definition
+
+## 2. Data flow
+
+## 3. Proposal
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-template.md b/doc/architecture/blueprints/pods/pods-feature-template.md
new file mode 100644
index 00000000000..dfae21b5406
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-template.md
@@ -0,0 +1,29 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Problem A'
+---
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: A
+
+> TL;DR
+
+## 1. Definition
+
+## 2. Data flow
+
+## 3. Proposal
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/proposal-stateless-router-with-buffering-requests.md b/doc/architecture/blueprints/pods/proposal-stateless-router-with-buffering-requests.md
new file mode 100644
index 00000000000..21aa72273fe
--- /dev/null
+++ b/doc/architecture/blueprints/pods/proposal-stateless-router-with-buffering-requests.md
@@ -0,0 +1,648 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods Stateless Router Proposal'
+---
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Proposal: Stateless Router
+
+We will decompose `gitlab_users`, `gitlab_routes` and `gitlab_admin` related
+tables so that they can be shared between all pods and allow any pod to
+authenticate a user and route requests to the correct pod. Pods may receive
+requests for the resources they don't own, but they know how to redirect back
+to the correct pod.
+
+The router is stateless and does not read from the `routes` database which
+means that all interactions with the database still happen from the Rails
+monolith. This architecture also supports regions by allowing for low traffic
+databases to be replicated across regions.
+
+Users are not directly exposed to the concept of Pods but instead they see
+different data dependent on their currently chosen "organization".
+[Organizations](index.md#organizations) will be a new model introduced to enforce isolation in the
+application and allow us to decide which request route to which pod, since an
+organization can only be on a single pod.
+
+## Differences
+
+The main difference between this proposal and the one [with learning routes](proposal-stateless-router-with-routes-learning.md)
+is that this proposal always sends requests to any of the Pods. If the requests cannot be processed,
+the requests will be bounced back with relevant headers. This requires that request to be buffered.
+It allows that request decoding can be either via URI or Body of request by Rails.
+This means that each request might be sent more than once and be processed more than once as result.
+
+The [with learning routes proposal](proposal-stateless-router-with-routes-learning.md) requires that
+routable information is always encoded in URI, and the router sends a pre-flight request.
+
+## Summary in diagrams
+
+This shows how a user request routes via DNS to the nearest router and the router chooses a pod to send the request to.
+
+```mermaid
+graph TD;
+ user((User));
+ dns[DNS];
+ router_us(Router);
+ router_eu(Router);
+ pod_us0{Pod US0};
+ pod_us1{Pod US1};
+ pod_eu0{Pod EU0};
+ pod_eu1{Pod EU1};
+ user-->dns;
+ dns-->router_us;
+ dns-->router_eu;
+ subgraph Europe
+ router_eu-->pod_eu0;
+ router_eu-->pod_eu1;
+ end
+ subgraph United States
+ router_us-->pod_us0;
+ router_us-->pod_us1;
+ end
+```
+
+<details><summary>More detail</summary>
+
+This shows that the router can actually send requests to any pod. The user will
+get the closest router to them geographically.
+
+```mermaid
+graph TD;
+ user((User));
+ dns[DNS];
+ router_us(Router);
+ router_eu(Router);
+ pod_us0{Pod US0};
+ pod_us1{Pod US1};
+ pod_eu0{Pod EU0};
+ pod_eu1{Pod EU1};
+ user-->dns;
+ dns-->router_us;
+ dns-->router_eu;
+ subgraph Europe
+ router_eu-->pod_eu0;
+ router_eu-->pod_eu1;
+ end
+ subgraph United States
+ router_us-->pod_us0;
+ router_us-->pod_us1;
+ end
+ router_eu-.->pod_us0;
+ router_eu-.->pod_us1;
+ router_us-.->pod_eu0;
+ router_us-.->pod_eu1;
+```
+
+</details>
+
+<details><summary>Even more detail</summary>
+
+This shows the databases. `gitlab_users` and `gitlab_routes` exist only in the
+US region but are replicated to other regions. Replication does not have an
+arrow because it's too hard to read the diagram.
+
+```mermaid
+graph TD;
+ user((User));
+ dns[DNS];
+ router_us(Router);
+ router_eu(Router);
+ pod_us0{Pod US0};
+ pod_us1{Pod US1};
+ pod_eu0{Pod EU0};
+ pod_eu1{Pod EU1};
+ db_gitlab_users[(gitlab_users Primary)];
+ db_gitlab_routes[(gitlab_routes Primary)];
+ db_gitlab_users_replica[(gitlab_users Replica)];
+ db_gitlab_routes_replica[(gitlab_routes Replica)];
+ db_pod_us0[(gitlab_main/gitlab_ci Pod US0)];
+ db_pod_us1[(gitlab_main/gitlab_ci Pod US1)];
+ db_pod_eu0[(gitlab_main/gitlab_ci Pod EU0)];
+ db_pod_eu1[(gitlab_main/gitlab_ci Pod EU1)];
+ user-->dns;
+ dns-->router_us;
+ dns-->router_eu;
+ subgraph Europe
+ router_eu-->pod_eu0;
+ router_eu-->pod_eu1;
+ pod_eu0-->db_pod_eu0;
+ pod_eu0-->db_gitlab_users_replica;
+ pod_eu0-->db_gitlab_routes_replica;
+ pod_eu1-->db_gitlab_users_replica;
+ pod_eu1-->db_gitlab_routes_replica;
+ pod_eu1-->db_pod_eu1;
+ end
+ subgraph United States
+ router_us-->pod_us0;
+ router_us-->pod_us1;
+ pod_us0-->db_pod_us0;
+ pod_us0-->db_gitlab_users;
+ pod_us0-->db_gitlab_routes;
+ pod_us1-->db_gitlab_users;
+ pod_us1-->db_gitlab_routes;
+ pod_us1-->db_pod_us1;
+ end
+ router_eu-.->pod_us0;
+ router_eu-.->pod_us1;
+ router_us-.->pod_eu0;
+ router_us-.->pod_eu1;
+```
+
+</details>
+
+## Summary of changes
+
+1. Tables related to User data (including profile settings, authentication credentials, personal access tokens) are decomposed into a `gitlab_users` schema
+1. The `routes` table is decomposed into `gitlab_routes` schema
+1. The `application_settings` (and probably a few other instance level tables) are decomposed into `gitlab_admin` schema
+1. A new column `routes.pod_id` is added to `routes` table
+1. A new Router service exists to choose which pod to route a request to.
+1. A new concept will be introduced in GitLab called an organization and a user can select a "default organization" and this will be a user level setting. The default organization is used to redirect users away from ambiguous routes like `/dashboard` to organization scoped routes like `/organizations/my-organization/-/dashboard`. Legacy users will have a special default organization that allows them to keep using global resources on `Pod US0`. All existing namespaces will initially move to this public organization.
+1. If a pod receives a request for a `routes.pod_id` that it does not own it returns a `302` with `X-Gitlab-Pod-Redirect` header so that the router can send the request to the correct pod. The correct pod can also set a header `X-Gitlab-Pod-Cache` which contains information about how this request should be cached to remember the pod. For example if the request was `/gitlab-org/gitlab` then the header would encode `/gitlab-org/* => Pod US0` (ie. any requests starting with `/gitlab-org/` can always be routed to `Pod US0`
+1. When the pod does not know (from the cache) which pod to send a request to it just picks a random pod within it's region
+1. Writes to `gitlab_users` and `gitlab_routes` are sent to a primary PostgreSQL server in our `US` region but reads can come from replicas in the same region. This will add latency for these writes but we expect they are infrequent relative to the rest of GitLab.
+
+## Detailed explanation of default organization in the first iteration
+
+All users will get a new column `users.default_organization` which they can
+control in user settings. We will introduce a concept of the
+`GitLab.com Public` organization. This will be set as the default organization for all existing
+users. This organization will allow the user to see data from all namespaces in
+`Pod US0` (ie. our original GitLab.com instance). This behavior can be invisible to
+existing users such that they don't even get told when they are viewing a
+global page like `/dashboard` that it's even scoped to an organization.
+
+Any new users with a default organization other than `GitLab.com Public` will have
+a distinct user experience and will be fully aware that every page they load is
+only ever scoped to a single organization. These users can never
+load any global pages like `/dashboard` and will end up being redirected to
+`/organizations/<DEFAULT_ORGANIZATION>/-/dashboard`. This may also be the case
+for legacy APIs and such users may only ever be able to use APIs scoped to a
+organization.
+
+## Detailed explanation of Admin Area settings
+
+We believe that maintaining and synchronizing Admin Area settings will be
+frustrating and painful so to avoid this we will decompose and share all Admin Area
+settings in the `gitlab_admin` schema. This should be safe (similar to other
+shared schemas) because these receive very little write traffic.
+
+In cases where different pods need different settings (eg. the
+Elasticsearch URL), we will either decide to use a templated
+format in the relevant `application_settings` row which allows it to be dynamic
+per pod. Alternatively if that proves difficult we'll introduce a new table
+called `per_pod_application_settings` and this will have 1 row per pod to allow
+setting different settings per pod. It will still be part of the `gitlab_admin`
+schema and shared which will allow us to centrally manage it and simplify
+keeping settings in sync for all pods.
+
+## Pros
+
+1. Router is stateless and can live in many regions. We use Anycast DNS to resolve to nearest region for the user.
+1. Pods can receive requests for namespaces in the wrong pod and the user
+ still gets the right response as well as caching at the router that
+ ensures the next request is sent to the correct pod so the next request
+ will go to the correct pod
+1. The majority of the code still lives in `gitlab` rails codebase. The Router doesn't actually need to understand how GitLab URLs are composed.
+1. Since the responsibility to read and write `gitlab_users`,
+ `gitlab_routes` and `gitlab_admin` still lives in Rails it means minimal
+ changes will be needed to the Rails application compared to extracting
+ services that need to isolate the domain models and build new interfaces.
+1. Compared to a separate routing service this allows the Rails application
+ to encode more complex rules around how to map URLs to the correct pod
+ and may work for some existing API endpoints.
+1. All the new infrastructure (just a router) is optional and a single-pod
+ self-managed installation does not even need to run the Router and there are
+ no other new services.
+
+## Cons
+
+1. `gitlab_users`, `gitlab_routes` and `gitlab_admin` databases may need to be
+ replicated across regions and writes need to go across regions. We need to
+ do an analysis on write TPS for the relevant tables to determine if this is
+ feasible.
+1. Sharing access to the database from many different Pods means that they are
+ all coupled at the Postgres schema level and this means changes to the
+ database schema need to be done carefully in sync with the deployment of all
+ Pods. This limits us to ensure that Pods are kept in closely similar
+ versions compared to an architecture with shared services that have an API
+ we control.
+1. Although most data is stored in the right region there can be requests
+ proxied from another region which may be an issue for certain types
+ of compliance.
+1. Data in `gitlab_users` and `gitlab_routes` databases must be replicated in
+ all regions which may be an issue for certain types of compliance.
+1. The router cache may need to be very large if we get a wide variety of URLs
+ (ie. long tail). In such a case we may need to implement a 2nd level of
+ caching in user cookies so their frequently accessed pages always go to the
+ right pod the first time.
+1. Having shared database access for `gitlab_users` and `gitlab_routes`
+ from multiple pods is an unusual architecture decision compared to
+ extracting services that are called from multiple pods.
+1. It is very likely we won't be able to find cacheable elements of a
+ GraphQL URL and often existing GraphQL endpoints are heavily dependent on
+ ids that won't be in the `routes` table so pods won't necessarily know
+ what pod has the data. As such we'll probably have to update our GraphQL
+ calls to include an organization context in the path like
+ `/api/organizations/<organization>/graphql`.
+1. This architecture implies that implemented endpoints can only access data
+ that are readily accessible on a given Pod, but are unlikely
+ to aggregate information from many Pods.
+1. All unknown routes are sent to the latest deployment which we assume to be `Pod US0`.
+ This is required as newly added endpoints will be only decodable by latest pod.
+ This Pod could later redirect to correct one that can serve the given request.
+ Since request processing might be heavy some Pods might receive significant amount
+ of traffic due to that.
+
+## Example database configuration
+
+Handling shared `gitlab_users`, `gitlab_routes` and `gitlab_admin` databases, while having dedicated `gitlab_main` and `gitlab_ci` databases should already be handled by the way we use `config/database.yml`. We should also, already be able to handle the dedicated EU replicas while having a single US primary for `gitlab_users` and `gitlab_routes`. Below is a snippet of part of the database configuration for the Pod architecture described above.
+
+<details><summary>Pod US0</summary>
+
+```yaml
+# config/database.yml
+production:
+ main:
+ host: postgres-main.pod-us0.primary.consul
+ load_balancing:
+ discovery: postgres-main.pod-us0.replicas.consul
+ ci:
+ host: postgres-ci.pod-us0.primary.consul
+ load_balancing:
+ discovery: postgres-ci.pod-us0.replicas.consul
+ users:
+ host: postgres-users-primary.consul
+ load_balancing:
+ discovery: postgres-users-replicas.us.consul
+ routes:
+ host: postgres-routes-primary.consul
+ load_balancing:
+ discovery: postgres-routes-replicas.us.consul
+ admin:
+ host: postgres-admin-primary.consul
+ load_balancing:
+ discovery: postgres-admin-replicas.us.consul
+```
+
+</details>
+
+<details><summary>Pod EU0</summary>
+
+```yaml
+# config/database.yml
+production:
+ main:
+ host: postgres-main.pod-eu0.primary.consul
+ load_balancing:
+ discovery: postgres-main.pod-eu0.replicas.consul
+ ci:
+ host: postgres-ci.pod-eu0.primary.consul
+ load_balancing:
+ discovery: postgres-ci.pod-eu0.replicas.consul
+ users:
+ host: postgres-users-primary.consul
+ load_balancing:
+ discovery: postgres-users-replicas.eu.consul
+ routes:
+ host: postgres-routes-primary.consul
+ load_balancing:
+ discovery: postgres-routes-replicas.eu.consul
+ admin:
+ host: postgres-admin-primary.consul
+ load_balancing:
+ discovery: postgres-admin-replicas.eu.consul
+```
+
+</details>
+
+## Request flows
+
+1. `gitlab-org` is a top level namespace and lives in `Pod US0` in the `GitLab.com Public` organization
+1. `my-company` is a top level namespace and lives in `Pod EU0` in the `my-organization` organization
+
+### Experience for paying user that is part of `my-organization`
+
+Such a user will have a default organization set to `/my-organization` and will be
+unable to load any global routes outside of this organization. They may load other
+projects/namespaces but their MR/Todo/Issue counts at the top of the page will
+not be correctly populated in the first iteration. The user will be aware of
+this limitation.
+
+#### Navigates to `/my-company/my-project` while logged in
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. They request `/my-company/my-project` without the router cache, so the router chooses randomly `Pod EU1`
+1. `Pod EU1` does not have `/my-company`, but it knows that it lives in `Pod EU0` so it redirects the router to `Pod EU0`
+1. `Pod EU0` returns the correct response as well as setting the cache headers for the router `/my-company/* => Pod EU0`
+1. The router now caches and remembers any request paths matching `/my-company/*` should go to `Pod EU0`
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ participant pod_eu1 as Pod EU1
+ user->>router_eu: GET /my-company/my-project
+ router_eu->>pod_eu1: GET /my-company/my-project
+ pod_eu1->>router_eu: 302 /my-company/my-project X-Gitlab-Pod-Redirect={pod:Pod EU0}
+ router_eu->>pod_eu0: GET /my-company/my-project
+ pod_eu0->>user: <h1>My Project... X-Gitlab-Pod-Cache={path_prefix:/my-company/}
+```
+
+#### Navigates to `/my-company/my-project` while not logged in
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. The router does not have `/my-company/*` cached yet so it chooses randomly `Pod EU1`
+1. `Pod EU1` redirects them through a login flow
+1. Stil they request `/my-company/my-project` without the router cache, so the router chooses a random pod `Pod EU1`
+1. `Pod EU1` does not have `/my-company`, but it knows that it lives in `Pod EU0` so it redirects the router to `Pod EU0`
+1. `Pod EU0` returns the correct response as well as setting the cache headers for the router `/my-company/* => Pod EU0`
+1. The router now caches and remembers any request paths matching `/my-company/*` should go to `Pod EU0`
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ participant pod_eu1 as Pod EU1
+ user->>router_eu: GET /my-company/my-project
+ router_eu->>pod_eu1: GET /my-company/my-project
+ pod_eu1->>user: 302 /users/sign_in?redirect=/my-company/my-project
+ user->>router_eu: GET /users/sign_in?redirect=/my-company/my-project
+ router_eu->>pod_eu1: GET /users/sign_in?redirect=/my-company/my-project
+ pod_eu1->>user: <h1>Sign in...
+ user->>router_eu: POST /users/sign_in?redirect=/my-company/my-project
+ router_eu->>pod_eu1: POST /users/sign_in?redirect=/my-company/my-project
+ pod_eu1->>user: 302 /my-company/my-project
+ user->>router_eu: GET /my-company/my-project
+ router_eu->>pod_eu1: GET /my-company/my-project
+ pod_eu1->>router_eu: 302 /my-company/my-project X-Gitlab-Pod-Redirect={pod:Pod EU0}
+ router_eu->>pod_eu0: GET /my-company/my-project
+ pod_eu0->>user: <h1>My Project... X-Gitlab-Pod-Cache={path_prefix:/my-company/}
+```
+
+#### Navigates to `/my-company/my-other-project` after last step
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. The router cache now has `/my-company/* => Pod EU0`, so the router chooses `Pod EU0`
+1. `Pod EU0` returns the correct response as well as the cache header again
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ participant pod_eu1 as Pod EU1
+ user->>router_eu: GET /my-company/my-project
+ router_eu->>pod_eu0: GET /my-company/my-project
+ pod_eu0->>user: <h1>My Project... X-Gitlab-Pod-Cache={path_prefix:/my-company/}
+```
+
+#### Navigates to `/gitlab-org/gitlab` after last step
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. The router has no cached value for this URL so randomly chooses `Pod EU0`
+1. `Pod EU0` redirects the router to `Pod US0`
+1. `Pod US0` returns the correct response as well as the cache header again
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ participant pod_us0 as Pod US0
+ user->>router_eu: GET /gitlab-org/gitlab
+ router_eu->>pod_eu0: GET /gitlab-org/gitlab
+ pod_eu0->>router_eu: 302 /gitlab-org/gitlab X-Gitlab-Pod-Redirect={pod:Pod US0}
+ router_eu->>pod_us0: GET /gitlab-org/gitlab
+ pod_us0->>user: <h1>GitLab.org... X-Gitlab-Pod-Cache={path_prefix:/gitlab-org/}
+```
+
+In this case the user is not on their "default organization" so their TODO
+counter will not include their normal todos. We may choose to highlight this in
+the UI somewhere. A future iteration may be able to fetch that for them from
+their default organization.
+
+#### Navigates to `/`
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. Router does not have a cache for `/` route (specifically rails never tells it to cache this route)
+1. The Router choose `Pod EU0` randomly
+1. The Rails application knows the users default organization is `/my-organization`, so
+ it redirects the user to `/organizations/my-organization/-/dashboard`
+1. The Router has a cached value for `/organizations/my-organization/*` so it then sends the
+ request to `POD EU0`
+1. `Pod EU0` serves up a new page `/organizations/my-organization/-/dashboard` which is the same
+ dashboard view we have today but scoped to an organization clearly in the UI
+1. The user is (optionally) presented with a message saying that data on this page is only
+ from their default organization and that they can change their default
+ organization if it's not right.
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ user->>router_eu: GET /
+ router_eu->>pod_eu0: GET /
+ pod_eu0->>user: 302 /organizations/my-organization/-/dashboard
+ user->>router: GET /organizations/my-organization/-/dashboard
+ router->>pod_eu0: GET /organizations/my-organization/-/dashboard
+ pod_eu0->>user: <h1>My Company Dashboard... X-Gitlab-Pod-Cache={path_prefix:/organizations/my-organization/}
+```
+
+#### Navigates to `/dashboard`
+
+As above, they will end up on `/organizations/my-organization/-/dashboard` as
+the rails application will already redirect `/` to the dashboard page.
+
+### Navigates to `/not-my-company/not-my-project` while logged in (but they don't have access since this project/group is private)
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. The router knows that `/not-my-company` lives in `Pod US1` so sends the request to this
+1. The user does not have access so `Pod US1` returns 404
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_us1 as Pod US1
+ user->>router_eu: GET /not-my-company/not-my-project
+ router_eu->>pod_us1: GET /not-my-company/not-my-project
+ pod_us1->>user: 404
+```
+
+#### Creates a new top level namespace
+
+The user will be asked which organization they want the namespace to belong to.
+If they select `my-organization` then it will end up on the same pod as all
+other namespaces in `my-organization`. If they select nothing we default to
+`GitLab.com Public` and it is clear to the user that this is isolated from
+their existing organization such that they won't be able to see data from both
+on a single page.
+
+### Experience for GitLab team member that is part of `/gitlab-org`
+
+Such a user is considered a legacy user and has their default organization set to
+`GitLab.com Public`. This is a "meta" organization that does not really exist but
+the Rails application knows to interpret this organization to mean that they are
+allowed to use legacy global functionality like `/dashboard` to see data across
+namespaces located on `Pod US0`. The rails backend also knows that the default pod to render any ambiguous
+routes like `/dashboard` is `Pod US0`. Lastly the user will be allowed to
+navigate to organizations on another pod like `/my-organization` but when they do the
+user will see a message indicating that some data may be missing (eg. the
+MRs/Issues/Todos) counts.
+
+#### Navigates to `/gitlab-org/gitlab` while not logged in
+
+1. User is in the US so DNS resolves to the US router
+1. The router knows that `/gitlab-org` lives in `Pod US0` so sends the request
+ to this pod
+1. `Pod US0` serves up the response
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_us as Router US
+ participant pod_us0 as Pod US0
+ user->>router_us: GET /gitlab-org/gitlab
+ router_us->>pod_us0: GET /gitlab-org/gitlab
+ pod_us0->>user: <h1>GitLab.org... X-Gitlab-Pod-Cache={path_prefix:/gitlab-org/}
+```
+
+#### Navigates to `/`
+
+1. User is in US so DNS resolves to the router in US
+1. Router does not have a cache for `/` route (specifically rails never tells it to cache this route)
+1. The Router chooses `Pod US1` randomly
+1. The Rails application knows the users default organization is `GitLab.com Public`, so
+ it redirects the user to `/dashboards` (only legacy users can see
+ `/dashboard` global view)
+1. Router does not have a cache for `/dashboard` route (specifically rails never tells it to cache this route)
+1. The Router chooses `Pod US1` randomly
+1. The Rails application knows the users default organization is `GitLab.com Public`, so
+ it allows the user to load `/dashboards` (only legacy users can see
+ `/dashboard` global view) and redirects to router the legacy pod which is `Pod US0`
+1. `Pod US0` serves up the global view dashboard page `/dashboard` which is the same
+ dashboard view we have today
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_us as Router US
+ participant pod_us0 as Pod US0
+ participant pod_us1 as Pod US1
+ user->>router_us: GET /
+ router_us->>pod_us1: GET /
+ pod_us1->>user: 302 /dashboard
+ user->>router_us: GET /dashboard
+ router_us->>pod_us1: GET /dashboard
+ pod_us1->>router_us: 302 /dashboard X-Gitlab-Pod-Redirect={pod:Pod US0}
+ router_us->>pod_us0: GET /dashboard
+ pod_us0->>user: <h1>Dashboard...
+```
+
+#### Navigates to `/my-company/my-other-project` while logged in (but they don't have access since this project is private)
+
+They get a 404.
+
+### Experience for non-logged in users
+
+Flow is similar to logged in users except global routes like `/dashboard` will
+redirect to the login page as there is no default organization to choose from.
+
+### A new customers signs up
+
+They will be asked if they are already part of an organization or if they'd
+like to create one. If they choose neither they end up no the default
+`GitLab.com Public` organization.
+
+### An organization is moved from 1 pod to another
+
+TODO
+
+### GraphQL/API requests which don't include the namespace in the URL
+
+TODO
+
+### The autocomplete suggestion functionality in the search bar which remembers recent issues/MRs
+
+TODO
+
+### Global search
+
+TODO
+
+## Administrator
+
+### Loads `/admin` page
+
+1. Router picks a random pod `Pod US0`
+1. Pod US0 redirects user to `/admin/pods/podus0`
+1. Pod US0 renders an Admin Area page and also returns a cache header to cache `/admin/podss/podus0/* => Pod US0`. The Admin Area page contains a dropdown list showing other pods they could select and it changes the query parameter.
+
+Admin Area settings in Postgres are all shared across all pods to avoid
+divergence but we still make it clear in the URL and UI which pod is serving
+the Admin Area page as there is dynamic data being generated from these pages and
+the operator may want to view a specific pod.
+
+## More Technical Problems To Solve
+
+### Replicating User Sessions Between All Pods
+
+Today user sessions live in Redis but each pod will have their own Redis instance. We already use a dedicated Redis instance for sessions so we could consider sharing this with all pods like we do with `gitlab_users` PostgreSQL database. But an important consideration will be latency as we would still want to mostly fetch sessions from the same region.
+
+An alternative might be that user sessions get moved to a JWT payload that encodes all the session data but this has downsides. For example, it is difficult to expire a user session, when their password changes or for other reasons, if the session lives in a JWT controlled by the user.
+
+### How do we migrate between Pods
+
+Migrating data between pods will need to factor all data stores:
+
+1. PostgreSQL
+1. Redis Shared State
+1. Gitaly
+1. Elasticsearch
+
+### Is it still possible to leak the existence of private groups via a timing attack?
+
+If you have router in EU, and you know that EU router by default redirects
+to EU located Pods, you know their latency (lets assume 10ms). Now, if your
+request is bounced back and redirected to US which has different latency
+(lets assume that roundtrip will be around 60ms) you can deduce that 404 was
+returned by US Pod and know that your 404 is in fact 403.
+
+We may defer this until we actually implement a pod in a different region. Such timing attacks are already theoretically possible with the way we do permission checks today but the timing difference is probably too small to be able to detect.
+
+One technique to mitigate this risk might be to have the router add a random
+delay to any request that returns 404 from a pod.
+
+## Should runners be shared across all pods?
+
+We have 2 options and we should decide which is easier:
+
+1. Decompose runner registration and queuing tables and share them across all
+ pods. This may have implications for scalability, and we'd need to consider
+ if this would include group/project runners as this may have scalability
+ concerns as these are high traffic tables that would need to be shared.
+1. Runners are registered per-pod and, we probably have a separate fleet of
+ runners for every pod or just register the same runners to many pods which
+ may have implications for queueing
+
+## How do we guarantee unique ids across all pods for things that cannot conflict?
+
+This project assumes at least namespaces and projects have unique ids across
+all pods as many requests need to be routed based on their ID. Since those
+tables are across different databases then guaranteeing a unique ID will
+require a new solution. There are likely other tables where unique IDs are
+necessary and depending on how we resolve routing for GraphQL and other APIs
+and other design goals it may be determined that we want the primary key to be
+unique for all tables.
diff --git a/doc/architecture/blueprints/pods/proposal-stateless-router-with-routes-learning.md b/doc/architecture/blueprints/pods/proposal-stateless-router-with-routes-learning.md
new file mode 100644
index 00000000000..e7520f3d6a8
--- /dev/null
+++ b/doc/architecture/blueprints/pods/proposal-stateless-router-with-routes-learning.md
@@ -0,0 +1,672 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods Stateless Router Proposal'
+---
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Proposal: Stateless Router
+
+We will decompose `gitlab_users`, `gitlab_routes` and `gitlab_admin` related
+tables so that they can be shared between all pods and allow any pod to
+authenticate a user and route requests to the correct pod. Pods may receive
+requests for the resources they don't own, but they know how to redirect back
+to the correct pod.
+
+The router is stateless and does not read from the `routes` database which
+means that all interactions with the database still happen from the Rails
+monolith. This architecture also supports regions by allowing for low traffic
+databases to be replicated across regions.
+
+Users are not directly exposed to the concept of Pods but instead they see
+different data dependent on their currently chosen "organization".
+[Organizations](index.md#organizations) will be a new model introduced to enforce isolation in the
+application and allow us to decide which request route to which pod, since an
+organization can only be on a single pod.
+
+## Differences
+
+The main difference between this proposal and one [with buffering requests](proposal-stateless-router-with-buffering-requests.md)
+is that this proposal uses a pre-flight API request (`/api/v4/pods/learn`) to redirect the request body to the correct Pod.
+This means that each request is sent exactly once to be processed, but the URI is used to decode which Pod it should be directed.
+
+## Summary in diagrams
+
+This shows how a user request routes via DNS to the nearest router and the router chooses a pod to send the request to.
+
+```mermaid
+graph TD;
+ user((User));
+ dns[DNS];
+ router_us(Router);
+ router_eu(Router);
+ pod_us0{Pod US0};
+ pod_us1{Pod US1};
+ pod_eu0{Pod EU0};
+ pod_eu1{Pod EU1};
+ user-->dns;
+ dns-->router_us;
+ dns-->router_eu;
+ subgraph Europe
+ router_eu-->pod_eu0;
+ router_eu-->pod_eu1;
+ end
+ subgraph United States
+ router_us-->pod_us0;
+ router_us-->pod_us1;
+ end
+```
+
+### More detail
+
+This shows that the router can actually send requests to any pod. The user will
+get the closest router to them geographically.
+
+```mermaid
+graph TD;
+ user((User));
+ dns[DNS];
+ router_us(Router);
+ router_eu(Router);
+ pod_us0{Pod US0};
+ pod_us1{Pod US1};
+ pod_eu0{Pod EU0};
+ pod_eu1{Pod EU1};
+ user-->dns;
+ dns-->router_us;
+ dns-->router_eu;
+ subgraph Europe
+ router_eu-->pod_eu0;
+ router_eu-->pod_eu1;
+ end
+ subgraph United States
+ router_us-->pod_us0;
+ router_us-->pod_us1;
+ end
+ router_eu-.->pod_us0;
+ router_eu-.->pod_us1;
+ router_us-.->pod_eu0;
+ router_us-.->pod_eu1;
+```
+
+### Even more detail
+
+This shows the databases. `gitlab_users` and `gitlab_routes` exist only in the
+US region but are replicated to other regions. Replication does not have an
+arrow because it's too hard to read the diagram.
+
+```mermaid
+graph TD;
+ user((User));
+ dns[DNS];
+ router_us(Router);
+ router_eu(Router);
+ pod_us0{Pod US0};
+ pod_us1{Pod US1};
+ pod_eu0{Pod EU0};
+ pod_eu1{Pod EU1};
+ db_gitlab_users[(gitlab_users Primary)];
+ db_gitlab_routes[(gitlab_routes Primary)];
+ db_gitlab_users_replica[(gitlab_users Replica)];
+ db_gitlab_routes_replica[(gitlab_routes Replica)];
+ db_pod_us0[(gitlab_main/gitlab_ci Pod US0)];
+ db_pod_us1[(gitlab_main/gitlab_ci Pod US1)];
+ db_pod_eu0[(gitlab_main/gitlab_ci Pod EU0)];
+ db_pod_eu1[(gitlab_main/gitlab_ci Pod EU1)];
+ user-->dns;
+ dns-->router_us;
+ dns-->router_eu;
+ subgraph Europe
+ router_eu-->pod_eu0;
+ router_eu-->pod_eu1;
+ pod_eu0-->db_pod_eu0;
+ pod_eu0-->db_gitlab_users_replica;
+ pod_eu0-->db_gitlab_routes_replica;
+ pod_eu1-->db_gitlab_users_replica;
+ pod_eu1-->db_gitlab_routes_replica;
+ pod_eu1-->db_pod_eu1;
+ end
+ subgraph United States
+ router_us-->pod_us0;
+ router_us-->pod_us1;
+ pod_us0-->db_pod_us0;
+ pod_us0-->db_gitlab_users;
+ pod_us0-->db_gitlab_routes;
+ pod_us1-->db_gitlab_users;
+ pod_us1-->db_gitlab_routes;
+ pod_us1-->db_pod_us1;
+ end
+ router_eu-.->pod_us0;
+ router_eu-.->pod_us1;
+ router_us-.->pod_eu0;
+ router_us-.->pod_eu1;
+```
+
+## Summary of changes
+
+1. Tables related to User data (including profile settings, authentication credentials, personal access tokens) are decomposed into a `gitlab_users` schema
+1. The `routes` table is decomposed into `gitlab_routes` schema
+1. The `application_settings` (and probably a few other instance level tables) are decomposed into `gitlab_admin` schema
+1. A new column `routes.pod_id` is added to `routes` table
+1. A new Router service exists to choose which pod to route a request to.
+1. If a router receives a new request it will send `/api/v4/pods/learn?method=GET&path_info=/group-org/project` to learn which Pod can process it
+1. A new concept will be introduced in GitLab called an organization
+1. We require all existing endpoints to be routable by URI, or be fixed to a specific Pod for processing. This requires changing ambiguous endpoints like `/dashboard` to be scoped like `/organizations/my-organization/-/dashboard`
+1. Endpoints like `/admin` would be routed always to the specific Pod, like `pod_0`
+1. Each Pod can respond to `/api/v4/pods/learn` and classify each endpoint
+1. Writes to `gitlab_users` and `gitlab_routes` are sent to a primary PostgreSQL server in our `US` region but reads can come from replicas in the same region. This will add latency for these writes but we expect they are infrequent relative to the rest of GitLab.
+
+## Pre-flight request learning
+
+While processing a request the URI will be decoded and a pre-flight request
+will be sent for each non-cached endpoint.
+
+When asking for the endpoint GitLab Rails will return information about
+the routable path. GitLab Rails will decode `path_info` and match it to
+an existing endpoint and find a routable entity (like project). The router will
+treat this as short-lived cache information.
+
+1. Prefix match: `/api/v4/pods/learn?method=GET&path_info=/gitlab-org/gitlab-test/-/issues`
+
+ ```json
+ {
+ "path": "/gitlab-org/gitlab-test",
+ "pod": "pod_0",
+ "source": "routable"
+ }
+ ```
+
+1. Some endpoints might require an exact match: `/api/v4/pods/learn?method=GET&path_info=/-/profile`
+
+ ```json
+ {
+ "path": "/-/profile",
+ "pod": "pod_0",
+ "source": "fixed",
+ "exact": true
+ }
+ ```
+
+## Detailed explanation of default organization in the first iteration
+
+All users will get a new column `users.default_organization` which they can
+control in user settings. We will introduce a concept of the
+`GitLab.com Public` organization. This will be set as the default organization for all existing
+users. This organization will allow the user to see data from all namespaces in
+`Pod US0` (ie. our original GitLab.com instance). This behavior can be invisible to
+existing users such that they don't even get told when they are viewing a
+global page like `/dashboard` that it's even scoped to an organization.
+
+Any new users with a default organization other than `GitLab.com Public` will have
+a distinct user experience and will be fully aware that every page they load is
+only ever scoped to a single organization. These users can never
+load any global pages like `/dashboard` and will end up being redirected to
+`/organizations/<DEFAULT_ORGANIZATION>/-/dashboard`. This may also be the case
+for legacy APIs and such users may only ever be able to use APIs scoped to a
+organization.
+
+## Detailed explanation of Admin Area settings
+
+We believe that maintaining and synchronizing Admin Area settings will be
+frustrating and painful so to avoid this we will decompose and share all Admin Area
+settings in the `gitlab_admin` schema. This should be safe (similar to other
+shared schemas) because these receive very little write traffic.
+
+In cases where different pods need different settings (eg. the
+Elasticsearch URL), we will either decide to use a templated
+format in the relevant `application_settings` row which allows it to be dynamic
+per pod. Alternatively if that proves difficult we'll introduce a new table
+called `per_pod_application_settings` and this will have 1 row per pod to allow
+setting different settings per pod. It will still be part of the `gitlab_admin`
+schema and shared which will allow us to centrally manage it and simplify
+keeping settings in sync for all pods.
+
+## Pros
+
+1. Router is stateless and can live in many regions. We use Anycast DNS to resolve to nearest region for the user.
+1. Pods can receive requests for namespaces in the wrong pod and the user
+ still gets the right response as well as caching at the router that
+ ensures the next request is sent to the correct pod so the next request
+ will go to the correct pod
+1. The majority of the code still lives in `gitlab` rails codebase. The Router doesn't actually need to understand how GitLab URLs are composed.
+1. Since the responsibility to read and write `gitlab_users`,
+ `gitlab_routes` and `gitlab_admin` still lives in Rails it means minimal
+ changes will be needed to the Rails application compared to extracting
+ services that need to isolate the domain models and build new interfaces.
+1. Compared to a separate routing service this allows the Rails application
+ to encode more complex rules around how to map URLs to the correct pod
+ and may work for some existing API endpoints.
+1. All the new infrastructure (just a router) is optional and a single-pod
+ self-managed installation does not even need to run the Router and there are
+ no other new services.
+
+## Cons
+
+1. `gitlab_users`, `gitlab_routes` and `gitlab_admin` databases may need to be
+ replicated across regions and writes need to go across regions. We need to
+ do an analysis on write TPS for the relevant tables to determine if this is
+ feasible.
+1. Sharing access to the database from many different Pods means that they are
+ all coupled at the Postgres schema level and this means changes to the
+ database schema need to be done carefully in sync with the deployment of all
+ Pods. This limits us to ensure that Pods are kept in closely similar
+ versions compared to an architecture with shared services that have an API
+ we control.
+1. Although most data is stored in the right region there can be requests
+ proxied from another region which may be an issue for certain types
+ of compliance.
+1. Data in `gitlab_users` and `gitlab_routes` databases must be replicated in
+ all regions which may be an issue for certain types of compliance.
+1. The router cache may need to be very large if we get a wide variety of URLs
+ (ie. long tail). In such a case we may need to implement a 2nd level of
+ caching in user cookies so their frequently accessed pages always go to the
+ right pod the first time.
+1. Having shared database access for `gitlab_users` and `gitlab_routes`
+ from multiple pods is an unusual architecture decision compared to
+ extracting services that are called from multiple pods.
+1. It is very likely we won't be able to find cacheable elements of a
+ GraphQL URL and often existing GraphQL endpoints are heavily dependent on
+ ids that won't be in the `routes` table so pods won't necessarily know
+ what pod has the data. As such we'll probably have to update our GraphQL
+ calls to include an organization context in the path like
+ `/api/organizations/<organization>/graphql`.
+1. This architecture implies that implemented endpoints can only access data
+ that are readily accessible on a given Pod, but are unlikely
+ to aggregate information from many Pods.
+1. All unknown routes are sent to the latest deployment which we assume to be `Pod US0`.
+ This is required as newly added endpoints will be only decodable by latest pod.
+ Likely this is not a problem for the `/pods/learn` is it is lightweight
+ to process and this should not cause a performance impact.
+
+## Example database configuration
+
+Handling shared `gitlab_users`, `gitlab_routes` and `gitlab_admin` databases, while having dedicated `gitlab_main` and `gitlab_ci` databases should already be handled by the way we use `config/database.yml`. We should also, already be able to handle the dedicated EU replicas while having a single US primary for `gitlab_users` and `gitlab_routes`. Below is a snippet of part of the database configuration for the Pod architecture described above.
+
+**Pod US0**:
+
+```yaml
+# config/database.yml
+production:
+ main:
+ host: postgres-main.pod-us0.primary.consul
+ load_balancing:
+ discovery: postgres-main.pod-us0.replicas.consul
+ ci:
+ host: postgres-ci.pod-us0.primary.consul
+ load_balancing:
+ discovery: postgres-ci.pod-us0.replicas.consul
+ users:
+ host: postgres-users-primary.consul
+ load_balancing:
+ discovery: postgres-users-replicas.us.consul
+ routes:
+ host: postgres-routes-primary.consul
+ load_balancing:
+ discovery: postgres-routes-replicas.us.consul
+ admin:
+ host: postgres-admin-primary.consul
+ load_balancing:
+ discovery: postgres-admin-replicas.us.consul
+```
+
+**Pod EU0**:
+
+```yaml
+# config/database.yml
+production:
+ main:
+ host: postgres-main.pod-eu0.primary.consul
+ load_balancing:
+ discovery: postgres-main.pod-eu0.replicas.consul
+ ci:
+ host: postgres-ci.pod-eu0.primary.consul
+ load_balancing:
+ discovery: postgres-ci.pod-eu0.replicas.consul
+ users:
+ host: postgres-users-primary.consul
+ load_balancing:
+ discovery: postgres-users-replicas.eu.consul
+ routes:
+ host: postgres-routes-primary.consul
+ load_balancing:
+ discovery: postgres-routes-replicas.eu.consul
+ admin:
+ host: postgres-admin-primary.consul
+ load_balancing:
+ discovery: postgres-admin-replicas.eu.consul
+```
+
+## Request flows
+
+1. `gitlab-org` is a top level namespace and lives in `Pod US0` in the `GitLab.com Public` organization
+1. `my-company` is a top level namespace and lives in `Pod EU0` in the `my-organization` organization
+
+### Experience for paying user that is part of `my-organization`
+
+Such a user will have a default organization set to `/my-organization` and will be
+unable to load any global routes outside of this organization. They may load other
+projects/namespaces but their MR/Todo/Issue counts at the top of the page will
+not be correctly populated in the first iteration. The user will be aware of
+this limitation.
+
+#### Navigates to `/my-company/my-project` while logged in
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. They request `/my-company/my-project` without the router cache, so the router chooses randomly `Pod EU1`
+1. The `/pods/learn` is sent to `Pod EU1`, which responds that resource lives on `Pod EU0`
+1. `Pod EU0` returns the correct response
+1. The router now caches and remembers any request paths matching `/my-company/*` should go to `Pod EU0`
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ participant pod_eu1 as Pod EU1
+ user->>router_eu: GET /my-company/my-project
+ router_eu->>pod_eu1: /api/v4/pods/learn?method=GET&path_info=/my-company/my-project
+ pod_eu1->>router_eu: {path: "/my-company", pod: "pod_eu0", source: "routable"}
+ router_eu->>pod_eu0: GET /my-company/my-project
+ pod_eu0->>user: <h1>My Project...
+```
+
+#### Navigates to `/my-company/my-project` while not logged in
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. The router does not have `/my-company/*` cached yet so it chooses randomly `Pod EU1`
+1. The `/pods/learn` is sent to `Pod EU1`, which responds that resource lives on `Pod EU0`
+1. `Pod EU0` redirects them through a login flow
+1. User requests `/users/sign_in`, uses random Pod to run `/pods/learn`
+1. The `Pod EU1` responds with `pod_0` as a fixed route
+1. User after login requests `/my-company/my-project` which is cached and stored in `Pod EU0`
+1. `Pod EU0` returns the correct response
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ participant pod_eu1 as Pod EU1
+ user->>router_eu: GET /my-company/my-project
+ router_eu->>pod_eu1: /api/v4/pods/learn?method=GET&path_info=/my-company/my-project
+ pod_eu1->>router_eu: {path: "/my-company", pod: "pod_eu0", source: "routable"}
+ router_eu->>pod_eu0: GET /my-company/my-project
+ pod_eu0->>user: 302 /users/sign_in?redirect=/my-company/my-project
+ user->>router_eu: GET /users/sign_in?redirect=/my-company/my-project
+ router_eu->>pod_eu1: /api/v4/pods/learn?method=GET&path_info=/users/sign_in
+ pod_eu1->>router_eu: {path: "/users", pod: "pod_eu0", source: "fixed"}
+ router_eu->>pod_eu0: GET /users/sign_in?redirect=/my-company/my-project
+ pod_eu0-->>user: <h1>Sign in...
+ user->>router_eu: POST /users/sign_in?redirect=/my-company/my-project
+ router_eu->>pod_eu0: POST /users/sign_in?redirect=/my-company/my-project
+ pod_eu0->>user: 302 /my-company/my-project
+ user->>router_eu: GET /my-company/my-project
+ router_eu->>pod_eu0: GET /my-company/my-project
+ router_eu->>pod_eu0: GET /my-company/my-project
+ pod_eu0->>user: <h1>My Project...
+```
+
+#### Navigates to `/my-company/my-other-project` after last step
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. The router cache now has `/my-company/* => Pod EU0`, so the router chooses `Pod EU0`
+1. `Pod EU0` returns the correct response as well as the cache header again
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ participant pod_eu1 as Pod EU1
+ user->>router_eu: GET /my-company/my-project
+ router_eu->>pod_eu0: GET /my-company/my-project
+ pod_eu0->>user: <h1>My Project...
+```
+
+#### Navigates to `/gitlab-org/gitlab` after last step
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. The router has no cached value for this URL so randomly chooses `Pod EU0`
+1. `Pod EU0` redirects the router to `Pod US0`
+1. `Pod US0` returns the correct response as well as the cache header again
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ participant pod_us0 as Pod US0
+ user->>router_eu: GET /gitlab-org/gitlab
+ router_eu->>pod_eu0: /api/v4/pods/learn?method=GET&path_info=/gitlab-org/gitlab
+ pod_eu0->>router_eu: {path: "/gitlab-org", pod: "pod_us0", source: "routable"}
+ router_eu->>pod_us0: GET /gitlab-org/gitlab
+ pod_us0->>user: <h1>GitLab.org...
+```
+
+In this case the user is not on their "default organization" so their TODO
+counter will not include their normal todos. We may choose to highlight this in
+the UI somewhere. A future iteration may be able to fetch that for them from
+their default organization.
+
+#### Navigates to `/`
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. Router does not have a cache for `/` route (specifically rails never tells it to cache this route)
+1. The Router choose `Pod EU0` randomly
+1. The Rails application knows the users default organization is `/my-organization`, so
+ it redirects the user to `/organizations/my-organization/-/dashboard`
+1. The Router has a cached value for `/organizations/my-organization/*` so it then sends the
+ request to `POD EU0`
+1. `Pod EU0` serves up a new page `/organizations/my-organization/-/dashboard` which is the same
+ dashboard view we have today but scoped to an organization clearly in the UI
+1. The user is (optionally) presented with a message saying that data on this page is only
+ from their default organization and that they can change their default
+ organization if it's not right.
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_eu0 as Pod EU0
+ user->>router_eu: GET /
+ router_eu->>pod_eu0: GET /
+ pod_eu0->>user: 302 /organizations/my-organization/-/dashboard
+ user->>router: GET /organizations/my-organization/-/dashboard
+ router->>pod_eu0: GET /organizations/my-organization/-/dashboard
+ pod_eu0->>user: <h1>My Company Dashboard... X-Gitlab-Pod-Cache={path_prefix:/organizations/my-organization/}
+```
+
+#### Navigates to `/dashboard`
+
+As above, they will end up on `/organizations/my-organization/-/dashboard` as
+the rails application will already redirect `/` to the dashboard page.
+
+### Navigates to `/not-my-company/not-my-project` while logged in (but they don't have access since this project/group is private)
+
+1. User is in Europe so DNS resolves to the router in Europe
+1. The router knows that `/not-my-company` lives in `Pod US1` so sends the request to this
+1. The user does not have access so `Pod US1` returns 404
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_eu as Router EU
+ participant pod_us1 as Pod US1
+ user->>router_eu: GET /not-my-company/not-my-project
+ router_eu->>pod_us1: GET /not-my-company/not-my-project
+ pod_us1->>user: 404
+```
+
+#### Creates a new top level namespace
+
+The user will be asked which organization they want the namespace to belong to.
+If they select `my-organization` then it will end up on the same pod as all
+other namespaces in `my-organization`. If they select nothing we default to
+`GitLab.com Public` and it is clear to the user that this is isolated from
+their existing organization such that they won't be able to see data from both
+on a single page.
+
+### Experience for GitLab team member that is part of `/gitlab-org`
+
+Such a user is considered a legacy user and has their default organization set to
+`GitLab.com Public`. This is a "meta" organization that does not really exist but
+the Rails application knows to interpret this organization to mean that they are
+allowed to use legacy global functionality like `/dashboard` to see data across
+namespaces located on `Pod US0`. The rails backend also knows that the default pod to render any ambiguous
+routes like `/dashboard` is `Pod US0`. Lastly the user will be allowed to
+navigate to organizations on another pod like `/my-organization` but when they do the
+user will see a message indicating that some data may be missing (eg. the
+MRs/Issues/Todos) counts.
+
+#### Navigates to `/gitlab-org/gitlab` while not logged in
+
+1. User is in the US so DNS resolves to the US router
+1. The router knows that `/gitlab-org` lives in `Pod US0` so sends the request
+ to this pod
+1. `Pod US0` serves up the response
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_us as Router US
+ participant pod_us0 as Pod US0
+ user->>router_us: GET /gitlab-org/gitlab
+ router_us->>pod_us0: GET /gitlab-org/gitlab
+ pod_us0->>user: <h1>GitLab.org...
+```
+
+#### Navigates to `/`
+
+1. User is in US so DNS resolves to the router in US
+1. Router does not have a cache for `/` route (specifically rails never tells it to cache this route)
+1. The Router chooses `Pod US1` randomly
+1. The Rails application knows the users default organization is `GitLab.com Public`, so
+ it redirects the user to `/dashboards` (only legacy users can see
+ `/dashboard` global view)
+1. Router does not have a cache for `/dashboard` route (specifically rails never tells it to cache this route)
+1. The Router chooses `Pod US1` randomly
+1. The Rails application knows the users default organization is `GitLab.com Public`, so
+ it allows the user to load `/dashboards` (only legacy users can see
+ `/dashboard` global view) and redirects to router the legacy pod which is `Pod US0`
+1. `Pod US0` serves up the global view dashboard page `/dashboard` which is the same
+ dashboard view we have today
+
+```mermaid
+sequenceDiagram
+ participant user as User
+ participant router_us as Router US
+ participant pod_us0 as Pod US0
+ participant pod_us1 as Pod US1
+ user->>router_us: GET /
+ router_us->>pod_us1: GET /
+ pod_us1->>user: 302 /dashboard
+ user->>router_us: GET /dashboard
+ router_us->>pod_us1: /api/v4/pods/learn?method=GET&path_info=/dashboard
+ pod_us1->>router_us: {path: "/dashboard", pod: "pod_us0", source: "routable"}
+ router_us->>pod_us0: GET /dashboard
+ pod_us0->>user: <h1>Dashboard...
+```
+
+#### Navigates to `/my-company/my-other-project` while logged in (but they don't have access since this project is private)
+
+They get a 404.
+
+### Experience for non-logged in users
+
+Flow is similar to logged in users except global routes like `/dashboard` will
+redirect to the login page as there is no default organization to choose from.
+
+### A new customers signs up
+
+They will be asked if they are already part of an organization or if they'd
+like to create one. If they choose neither they end up no the default
+`GitLab.com Public` organization.
+
+### An organization is moved from 1 pod to another
+
+TODO
+
+### GraphQL/API requests which don't include the namespace in the URL
+
+TODO
+
+### The autocomplete suggestion functionality in the search bar which remembers recent issues/MRs
+
+TODO
+
+### Global search
+
+TODO
+
+## Administrator
+
+### Loads `/admin` page
+
+1. The `/admin` is locked to `Pod US0`
+1. Some endpoints of `/admin`, like Projects in Admin are scoped to a Pod
+ and users needs to choose the correct one in a dropdown, which results in endpoint
+ like `/admin/pods/pod_0/projects`.
+
+Admin Area settings in Postgres are all shared across all pods to avoid
+divergence but we still make it clear in the URL and UI which pod is serving
+the Admin Area page as there is dynamic data being generated from these pages and
+the operator may want to view a specific pod.
+
+## More Technical Problems To Solve
+
+### Replicating User Sessions Between All Pods
+
+Today user sessions live in Redis but each pod will have their own Redis instance. We already use a dedicated Redis instance for sessions so we could consider sharing this with all pods like we do with `gitlab_users` PostgreSQL database. But an important consideration will be latency as we would still want to mostly fetch sessions from the same region.
+
+An alternative might be that user sessions get moved to a JWT payload that encodes all the session data but this has downsides. For example, it is difficult to expire a user session, when their password changes or for other reasons, if the session lives in a JWT controlled by the user.
+
+### How do we migrate between Pods
+
+Migrating data between pods will need to factor all data stores:
+
+1. PostgreSQL
+1. Redis Shared State
+1. Gitaly
+1. Elasticsearch
+
+### Is it still possible to leak the existence of private groups via a timing attack?
+
+If you have router in EU, and you know that EU router by default redirects
+to EU located Pods, you know their latency (lets assume 10ms). Now, if your
+request is bounced back and redirected to US which has different latency
+(lets assume that roundtrip will be around 60ms) you can deduce that 404 was
+returned by US Pod and know that your 404 is in fact 403.
+
+We may defer this until we actually implement a pod in a different region. Such timing attacks are already theoretically possible with the way we do permission checks today but the timing difference is probably too small to be able to detect.
+
+One technique to mitigate this risk might be to have the router add a random
+delay to any request that returns 404 from a pod.
+
+## Should runners be shared across all pods?
+
+We have 2 options and we should decide which is easier:
+
+1. Decompose runner registration and queuing tables and share them across all
+ pods. This may have implications for scalability, and we'd need to consider
+ if this would include group/project runners as this may have scalability
+ concerns as these are high traffic tables that would need to be shared.
+1. Runners are registered per-pod and, we probably have a separate fleet of
+ runners for every pod or just register the same runners to many pods which
+ may have implications for queueing
+
+## How do we guarantee unique ids across all pods for things that cannot conflict?
+
+This project assumes at least namespaces and projects have unique ids across
+all pods as many requests need to be routed based on their ID. Since those
+tables are across different databases then guaranteeing a unique ID will
+require a new solution. There are likely other tables where unique IDs are
+necessary and depending on how we resolve routing for GraphQL and other APIs
+and other design goals it may be determined that we want the primary key to be
+unique for all tables.
diff --git a/doc/architecture/blueprints/pods/term-cluster.png b/doc/architecture/blueprints/pods/term-cluster.png
deleted file mode 100644
index f52e31b52ad..00000000000
--- a/doc/architecture/blueprints/pods/term-cluster.png
+++ /dev/null
Binary files differ
diff --git a/doc/architecture/blueprints/pods/term-organization.png b/doc/architecture/blueprints/pods/term-organization.png
deleted file mode 100644
index f605adb124d..00000000000
--- a/doc/architecture/blueprints/pods/term-organization.png
+++ /dev/null
Binary files differ
diff --git a/doc/architecture/blueprints/rate_limiting/index.md b/doc/architecture/blueprints/rate_limiting/index.md
index 2ed66f22b53..ffe0712d69b 100644
--- a/doc/architecture/blueprints/rate_limiting/index.md
+++ b/doc/architecture/blueprints/rate_limiting/index.md
@@ -1,8 +1,11 @@
---
-stage: none
-group: unassigned
-comments: false
-description: 'Next Rate Limiting Architecture'
+status: ready
+creation-date: "2022-09-08"
+authors: [ "@grzesiek", "@marshall007", "@fabiopitino", "@hswimelar" ]
+coach: "@andrewn"
+approvers: [ "@sgoldstein" ]
+owning-stage:
+participating-stages: []
---
# Next Rate Limiting Architecture
@@ -35,18 +38,6 @@ stack.
This blueprint has been written to consolidate our limits and to describe the
vision of our next rate limiting and policies enforcement architecture.
-_Disclaimer: The following contains information related to upcoming products,
-features, and functionality._
-
-_It is important to note that the information presented is for informational
-purposes only. Please do not rely on this information for purchasing or
-planning purposes._
-
-_As with all projects, the items mentioned in this document and linked pages are
-subject to change or delay. The development, release and timing of any
-products, features, or functionality remain at the sole discretion of GitLab
-Inc._
-
## Goals
**Implement a next architecture for rate limiting and policies definition.**
@@ -361,6 +352,31 @@ hierarchy. Choosing a proper solution will require a thoughtful research.
1. Maintain consistent features and behavior across SaaS and self-managed codebase.
1. Be mindful about a cognitive load added by the hierarchical limits, aim to reduce it.
+## Phases and iterations
+
+**Phase 1**: Compile examples of current most important application limits — Owning Team
+ a. Owning Team (in collaboration with Stage Groups) compiles a list of the
+ most important application limits used in Rails today.
+**Phase 2**: Implement Rate Limiting Framework in Rails - Owning Team
+ a. Triangulate rate limiting abstractions based on the data gathered in Phase 1
+ b. Develop YAML model for limits.
+ c. Build Rails SDK.
+ d. Create examples showcasing usage of the new rate limits SDK.
+**Phase 3**: Team Fanout of Rails SDK - Stage Groups
+ a. Individual stage groups begin using the SDK built in Phase 2 for new limit and policies.
+ b. Stage groups begin replacing historical adhoc limit implementations with the SDK.
+ c. Provides means to monitor and observe the progress of the replacement effort. Ideally this is broken down to the `feature_category` level to drive group-level buy-in -- Owning Team.
+**Phase 4**: Enable Satellite Services to Use the Rate Limiting Framework - Owning Team
+ a. Determine if the goals of Phase 4 are best met by either
+ 1. Extracting the Rails rate limiting service into a decoupled service OR
+ 2. Implementing a separate Go library which uses the same backend (eg, Redis) for rate limiting.
+**Phase 5**: SDK for Satellite Services - Owning Team
+ a. Build Golang SDK.
+ c. Create examples showcasing usage of the new rate limits SDK.
+**Phase 6**: Team Fanout for Satellite Services - Stage Groups
+ a. Individual stage groups being using the SDK built in Phase 5 for new limit and policies.
+ b. Stage groups begin replacing historical adhoc limit implementations with the SDK.
+
## Status
Request For Comments.
@@ -373,39 +389,3 @@ Request For Comments.
- 2022-07-06: A fourth, [consolidated proposal](https://gitlab.com/gitlab-org/gitlab/-/issues/364524#note_1017640650), has been submitted.
- 2022-07-12: Started working on the design document following [Architecture Evolution Workflow](https://about.gitlab.com/handbook/engineering/architecture/workflow/).
- 2022-09-08: The initial version of the blueprint has been merged.
-
-## Who
-
-Proposal:
-
-<!-- vale gitlab.Spelling = NO -->
-
-| Role | Who
-|------------------------------|-------------------------|
-| Author | Grzegorz Bizon |
-| Author | Fabio Pitino |
-| Author | Marshall Cottrell |
-| Author | Hayley Swimelar |
-| Engineering Leader | Sam Goldstein |
-| Product Manager | |
-| Architecture Evolution Coach | Andrew Newdigate |
-| Recommender | |
-| Recommender | |
-| Recommender | |
-| Recommender | |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Leadership | |
-| Product | |
-| Engineering | |
-
-Domain experts:
-
-| Area | Who
-|------------------------------|------------------------|
-| | |
-
-<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/runner_scaling/index.md b/doc/architecture/blueprints/runner_scaling/index.md
index 415884449ed..24c6820f94a 100644
--- a/doc/architecture/blueprints/runner_scaling/index.md
+++ b/doc/architecture/blueprints/runner_scaling/index.md
@@ -1,8 +1,11 @@
---
-stage: none
-group: unassigned
-comments: false
-description: 'Next Runner Auto-scaling Architecture'
+status: accepted
+creation-date: "2022-01-19"
+authors: [ "@grzesiek", "@tmaczukin", "@josephburnett" ]
+coach: "@kamil"
+approvers: [ "@DarrenEastman" ]
+owning-stage: "~devops::verify"
+participating-stages: []
---
# Next Runner Auto-scaling Architecture
@@ -50,18 +53,6 @@ build on top of it to improve efficiency, reliability and availability.
We call this new mechanism the "next GitLab Runner Scaling architecture".
-_Disclaimer The following contain information related to upcoming products,
-features, and functionality._
-
-_It is important to note that the information presented is for informational
-purposes only. Please do not rely on this information for purchasing or
-planning purposes._
-
-_As with all projects, the items mentioned in this document and linked pages are
-subject to change or delay. The development, release and timing of any
-products, features, or functionality remain at the sole discretion of GitLab
-Inc._
-
## Continuing building on Docker Machine
At this moment one of our core products - GitLab Runner - and one of its most
@@ -210,7 +201,7 @@ easier to understand how it performs.
## Details
-How the abstraction for the custom provider will look exactly is something that
+How the abstraction will look exactly is something that
we will need to prototype, PoC and decide in a data-informed way. There are a
few proposals that we should describe in detail, develop requirements for, PoC
and score. We will choose the solution that seems to support our goals the
@@ -257,6 +248,10 @@ them each separately.
to the Runner system. These details are highly dependent on the VM
architecture and operating system as well as Executor type.
+See also Glossary below.
+
+#### Current state
+
The current architecture has several points of coupling between concerns.
Coupling reduces opportunities for abstraction (e.g. community supported
plugins) and increases complexity, making the code harder to understand,
@@ -391,7 +386,7 @@ for by the plugin.
Rationale: [Description of the Custom Executor Provider proposal](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28848#note_823321515)
-### Fleeting VM provider
+### Taskscaler provider
We can introduce a more simple version of the `Machine` abstraction in the
form of a "Fleeting" interface. Fleeting provides a low-level interface to
@@ -412,6 +407,22 @@ component so it can be used by multiple Runner Executors (not just `docker+autos
Rationale: [Description of the InstanceGroup / Fleeting proposal](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28848#note_823430883)
POC: [Merge request](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3315)
+## Glossary
+
+- **[GitLab Runner](../../../development/documentation/styleguide/word_list.md#gitlab-runner)** - the software application that you can choose to install and manage, whose source code is hosted at `gitlab.com/gitlab-org/gitlab-runner`.
+- **[runners](../../../development/documentation/styleguide/word_list.md#runner-runners)** - the runner is the agent that's responsible for running GitLab CI/CD jobs in an environment and reporting the results to a GitLab instance. It /1/ retrieves jobs from GitLab, /2/ configures a local or remote build environment, and /3/ executes jobs within the provisioned environment, passing along log data and status updates to GitLab.
+- **runner manager** - the runner process is often referred to as the `Runner Manager` as it manages multiple runners, which are the `[[runners]]` workers defined in the runners `config.toml` file.
+- **executor** - a concrete environment which can be prepared and used to run a job. A new executor is created for each job.
+- **executor provider** - an implementation capable of providing executors on demand. Executor providers are registered on import and initialized once when a runner starts up.
+- **custom executor** - works as an interface between GitLab Runner and a set of binaries or shell scripts with environment variable inputs that enable executing CI jobs in any host computing environment. New custom executors can be added to the system without making any changes to the GitLab Runner codebase.
+- **custom executor provider** - a new abstraction, proposed under the custom provider heading in the plugin boundary proposal section above, which allows new executor providers to be created without modifying the GitLab Runner codebase. The protocol could be similar to custom executors or done over gRPC. This abstraction places all the mechanics of producing executors within the plugin, delegating autoscaling and lifecycle management concerns to each implementation.
+- **taskscaler** - a new library, proposed under the taskscaler provider heading in the plugin boundary proposal section above, which is parameterized with a concrete executor provider and a fleeting provider. Taskscaler is responsible for the autoscaling concern and can be used to autoscale any executor provider using any VM shape. Taskscaler is also responsible for the runner-specific aspect of VM lifecycle and keeps track of how many jobs are using a give VM and how many times a VM has been used.
+- **fleeting** - a new library proposed along with taskscaler which provides abstractions for cloud provider VMs.
+- **fleeting instance group** - the abstraction that fleeting uses to represent a pool of like VMs. This would represent a GCP IGM or an AWS ASG (without the autoscaling). Instance groups can be increased, decreased or can provide connection details for a specific VM.
+- **fleeting plugin** - a concrete implementation of a fleeting instance group representing a specific IGM or ASG (when initialized). There will be N of these, one for each provider, each in its own project. We will own and maintain the core ones but some will be community supported. A new fleeting plugin can be created without making any changes to the runner, taskscaler or fleeting code bases. This makes it analogous to the custom executor provider in terms of self-service and decoupling, but along a different line of concerns.
+- **fleeting plugin Google Compute** - the fleeting plugin which creates GCP instances. This lives in a separate project from the fleeting and taskscaler.
+- **fleeting plugin AWS** - the fleeting plugin which creates AWS instances. This lives in a separate project from the fleeting and taskscaler.
+
## Status
Status: RFC.
diff --git a/doc/architecture/blueprints/runner_tokens/index.md b/doc/architecture/blueprints/runner_tokens/index.md
new file mode 100644
index 00000000000..3f8a27e503d
--- /dev/null
+++ b/doc/architecture/blueprints/runner_tokens/index.md
@@ -0,0 +1,227 @@
+---
+stage: Verify
+group: Runner
+comments: false
+description: 'Next Runner Token Architecture'
+---
+
+# Next GitLab Runner Token Architecture
+
+## Summary
+
+GitLab Runner is a core component of GitLab CI/CD that runs
+CI/CD jobs in a reliable and concurrent environment. Ever since the beginnings
+of the service as a Ruby program, runners are registered in a GitLab instance with
+a registration token - a randomly generated string of text. The registration token is unique for its given scope
+(instance, group, or project). The registration token proves that the party that registers the runner has
+administrator access to the instance, group, or project to which the runner is registered.
+
+This approach has worked well in the initial years, but some major known issues started to
+become apparent as the target audience grew:
+
+| Problem | Symptoms |
+|---------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Single token per scope | - The registration token is shared by multiple runners: <br/>- Single tokens lower the value of auditing and make traceability almost impossible; <br/>- Copied in many places for [self-registration of runners](https://docs.gitlab.com/runner/install/kubernetes.html#required-configuration); <br/>- Reports of users storing tokens in unsecured locations; <br/>- Makes rotation of tokens costly. <br/>- In the case of a security event affecting the whole instance, rotating tokens requires users to update a table of projects/namespaces, which takes a significant amount of time. |
+| No provision for automatic expiration | Requires manual intervention to change token. Addressed in [#30942](https://gitlab.com/gitlab-org/gitlab/-/issues/30942). |
+| No permissions model | Used to register a runner for protected branches, and for any tags. In this case, the registration token has permission to do everything. Effectively, someone taking a possession of registration token could steal secrets or source code. |
+| No traceability | Given that the token is not created by a user, and is accessible to all administrators, there is no possibility to know the source of a leaked token. |
+| No historical records | When reset, the previous value of the registration token is not stored so there is no historical data to enable deeper auditing and inspection. |
+| Token stored in project/namespace model | Inadvertent disclosure of token is possible. |
+| Too many registered runners | It is too straightforward to register a new runner using a well-known registration token. |
+
+In light of these issues, it is important that we redesign the way in which we connect runners to the GitLab instance so that we can guarantee traceability, security, and performance.
+
+We call this new mechanism the "next GitLab Runner Token architecture".
+
+## Proposal
+
+The proposal addresses the issues of a _single token per scope_ and _token storage_
+by eliminating the need for a registration token. Runner creation happens
+in the GitLab Runners settings page for the given scope, in the context of the logged-in user
+, which provides traceability. The page provides instructions to configure the newly-created
+runner in supported environments.
+
+The runner configuration will be generated through a new `deploy` command, which will leverage
+the `/runners/verify` REST endpoint to ensure the validity of the authentication token.
+The remaining concerns become non-issues due to the elimination of the registration token.
+
+The configuration can be applied across many machines by reusing the same instructions.
+A unique system identifier will be generated automatically if a value is missing from
+the runner entry in the `config.toml` file. This allows differentiating systems sharing the same
+runner token (for example, in auto-scaling scenarios), and is crucial for the proper functioning of our
+long-polling mechanism when the same authentication token is shared across two or more runner managers.
+
+Given that the creation of runners involves user interaction, it should be possible
+to eventually lower the per-plan limit of CI runners that can be registered per scope.
+
+### Auto-scaling scenarios (for example Helm chart)
+
+In the existing model, a new runner is created whenever a new worker is required. This
+has led to many situations where runners are left behind and become stale.
+
+In the proposed model, a `ci_runners` table entry describes a configuration,
+which the runner could reuse across multiple machines. This allows differentiating the context in
+which the runner is being used. In situations where we must differentiate between runners
+that reuse the same configuration, we can use the unique system identifier to track all
+unique "runners" that are executed in context of a single `ci_runners` model. This unique
+system identifier would be present in the Runner's `config.toml` configuration file and
+initially set when generating the new `[[runners]]` configuration by means of the `deploy` command.
+Legacy files that miss values for unique system identifiers will get rewritten automatically with new values.
+
+### Runner identification in CI jobs
+
+For users to identify the machine where the job was executed, the unique identifier will need to be visible in CI job contexts.
+As a first iteration, GitLab Runner will include the unique system identifier in the build logs,
+wherever it publishes the short token SHA.
+
+Given that the runner will potentially be reused with different unique system identifiers,
+we can store the unique system ID. This ensures the unique system ID maps to a GitLab Runner's `config.toml` entry with
+the runner token. The `ci_runner_machines` would hold information about each unique runner machine,
+with information when runner last connected, and what type of runner it was. The relevant fields
+will be moved from the `ci_runners`.
+The `ci_builds_runner_session` (or `ci_builds` or `ci_builds_metadata`) will reference
+`ci_runner_machines`.
+We might consider a more efficient way to store `contacted_at` than updating the existing record.
+
+```sql
+CREATE TABLE ci_builds_runner_session (
+ ...
+ runner_machine_id bigint NOT NULL
+);
+
+CREATE TABLE ci_runner_machines (
+ id integer NOT NULL,
+ machine_id character varying UNIQUE NOT NULL,
+ contacted_at timestamp without time zone,
+ version character varying,
+ revision character varying,
+ platform character varying,
+ architecture character varying,
+ ip_address character varying,
+ executor_type smallint,
+);
+```
+
+## Advantages
+
+- Easier for users to wrap their minds around the concept: instead of two types of tokens,
+ there is a single type of token - the per-runner authentication token. Having two types of tokens
+ frequently results in misunderstandings when discussing issues;
+- Runners can always be traced back to the user who created it, using the audit log;
+- The claims of a CI runner are known at creation time, and cannot be changed from the runner
+ (for example, changing the `access_level`/`protected` flag). Authenticated users
+ may however still edit these settings through the GitLab UI.
+
+## Details
+
+In the proposed approach, we create a distinct way to configure runners that is usable
+alongside the current registration token method during a transition period. The idea is
+to avoid having the Runner make API calls that allow it to leverage a single "god-like"
+token to register new runners.
+
+The new workflow looks as follows:
+
+ 1. The user opens the Runners settings page;
+ 1. The user fills in the details regarding the new desired runner, namely description,
+ tags, protected, locked, etc.;
+ 1. The user clicks `Create`. That results in the following:
+
+ 1. Creates a new runner in the `ci_runners` table (and corresponding authentication token);
+ 1. Presents the user with instructions on how to configure this new runner on a machine,
+ with possibilities for different supported deployment scenarios (e.g. shell, `docker-compose`, Helm chart, etc.)
+ This information contains a token which will only be available to the user once, and the UI
+ will make it clear to the user that the value will not be shown again, as registering the same runner multiple times
+ is discouraged (though not impossible).
+
+ 1. The user copies and pastes the instructions for the intended deployment scenario (a `deploy` command), leading to the following actions:
+
+ 1. Upon executing the new `gitlab-runner deploy` command in the instructions, `gitlab-runner` will perform
+ a call to the `POST /runners/verify` with the given runner token;
+ 1. If the `POST /runners/verify` GitLab endpoint validates the token, the `config.toml` file will be populated with the configuration.
+
+ The `gitlab-runner deploy` will also accept executor-specific arguments
+ currently present in the `register` command.
+
+As part of the transition period, we will provide admins and top-level group owners with a instance/group-level setting to disable
+the legacy registration token functionality and enforce using only the new workflow.
+Any attempt by a `gitlab-runner register` command to hit the `POST /runners` endpoint to register a new runner
+will result in a `HTTP 410 - Gone` status code. The instance setting is inherited by the groups
+, which means that if the legacy registration method is disabled at the instance method, the descendant groups/projects will also mandatorily
+prevent the legacy registration method.
+
+The registration token workflow is to be deprecated (with a deprecation notice printed by the `gitlab-runner register` command)
+and removed at a future major release after the concept is proven stable and customers have migrated to the new workflow.
+
+### Handling of legacy runners
+
+Legacy versions of GitLab Runner will not send the unique system identifier in its requests, and we
+will not change logic in Workhorse to handle unique system IDs. This can be improved upon in the
+future once the legacy registration system is removed, and runners have been upgraded to newer
+versions.
+
+Not using the unique system ID means that all connected runners with the same token will be
+notified, instead of just the runner matching the exact system identifier. While not ideal, this is
+not an issue per-se.
+
+### Helm chart
+
+The `runnerRegistrationToken` entry in the [`values.yaml` file](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/a70bc29a903b79d5675bb0c45d981adf8b7a8659/values.yaml#L52)
+will be retired. The `runnerRegistrationToken` entry will be replaced by the existing `runnerToken` value, which will be passed
+to the new `gitlab-runner deploy` command in [`configmap.yaml`](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/a70bc29a903b79d5675bb0c45d981adf8b7a8659/templates/configmap.yaml#L116).
+
+### Runner creation through API
+
+Automated runner creation may be allowed, although always through authenticated API calls -
+using PAT tokens for example - such that every runner is associated with an owner.
+
+## Implementation plan
+
+| Component | Milestone | Changes |
+|------------------|-----------|---------|
+| GitLab Rails app | `15.x` (latest at `15.6`) | Deprecate `POST /api/v4/runners` endpoint for `16.0`. This hinges on a [proposal](https://gitlab.com/gitlab-org/gitlab/-/issues/373774) to allow deprecating REST API endpoints for security reasons. |
+| GitLab Runner | `15.x` (latest at `15.8`) | Add deprecation notice for `register` command for `16.0`. |
+| GitLab Runner | `15.x` | Ensure all runner entries in `config.toml` have unique system identifier values assigned. Log new system ID values with `INFO` level as they get created. |
+| GitLab Runner | `15.x` | Start additionally logging unique system ID anywhere we log the runner short SHA. |
+| GitLab Rails app | `15.x` | Create database migrations to add settings from `application_settings` and `namaspace_settings` tables. |
+| GitLab Runner | `15.x` | Start sending `unique_id` value in `POST /jobs/request` request and other follow-up requests that require identifying the unique system. |
+| GitLab Runner | `15.x` | Implement new user-authenticated API (REST and GraphQL) to create a new runner. |
+| GitLab Rails app | `15.x` | Implement UI to create new runner. |
+| GitLab Runner | `16.0` | Remove `register` command and support for `POST /runners` endpoint. |
+| GitLab Rails app | `16.0` | Remove legacy UI showing registration with a registration token. |
+| GitLab Rails app | `16.0` | Create database migrations to remove settings from `application_settings` and `namaspace_settings` tables. |
+| GitLab Rails app | `16.0` | Make [`POST /api/v4/runners` endpoint](../../../api/runners.md#register-a-new-runner-deprecated) permanently return `410 Gone`. A future v5 version of the API would return `404 Not Found`. |
+| GitLab Rails app | `16.0` | Start refusing job requests that don't include a unique ID. |
+
+## Status
+
+Status: RFC.
+
+## Who
+
+Proposal:
+
+<!-- vale gitlab.Spelling = NO -->
+
+| Role | Who
+|------------------------------|--------------------------------------------------|
+| Authors | Kamil Trzciński, Tomasz Maczukin, Pedro Pombeiro |
+| Architecture Evolution Coach | Kamil Trzciński |
+| Engineering Leader | Elliot Rushton, Cheryl Li |
+| Product Manager | Darren Eastman, Jackie Porter |
+| Domain Expert / Runner | Tomasz Maczukin |
+
+DRIs:
+
+| Role | Who |
+|------------------------------|---------------------------------|
+| Leadership | Elliot Rushton |
+| Product | Darren Eastman |
+| Engineering | Tomasz Maczukin, Pedro Pombeiro |
+
+Domain experts:
+
+| Area | Who |
+|------------------------------|-----------------|
+| Domain Expert / Runner | Tomasz Maczukin |
+
+<!-- vale gitlab.Spelling = YES -->
diff --git a/doc/architecture/blueprints/work_items/index.md b/doc/architecture/blueprints/work_items/index.md
index 42864e7112e..75a9d8d76ad 100644
--- a/doc/architecture/blueprints/work_items/index.md
+++ b/doc/architecture/blueprints/work_items/index.md
@@ -1,15 +1,15 @@
---
-stage: Plan
-group: Project Management
-comments: false
-description: 'Work Items'
+status: accepted
+creation-date: "2022-09-28"
+authors: [ "@ntepluhina" ]
+coach: "@kamil"
+approvers: [ "@gweaver" ]
+owning-stage: "~devops::plan"
+participating-stages: []
---
# Work Items
-DISCLAIMER:
-This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
-
This document is a work-in-progress. Some aspects are not documented, though we expect to add them in the future.
## Summary
@@ -55,14 +55,19 @@ All Work Item types share the same pool of predefined widgets and are customized
### Work Item widget types (updating)
-- assignees
-- description
-- hierarchy
-- iteration
-- labels
-- start and due date
-- verification status
-- weight
+| widget type | feature flag |
+|---|---|---|
+| assignees | |
+| description | |
+| hierarchy | |
+| [iteration](https://gitlab.com/gitlab-org/gitlab/-/issues/367456) | work_items_mvc_2 |
+| [milestone](https://gitlab.com/gitlab-org/gitlab/-/issues/367463) | work_items_mvc_2 |
+| labels | |
+| start and due date | |
+| status\* | |
+| weight | |
+
+\* status is not currently a widget, but a part of the root work item, similar to title
### Work Item view
@@ -72,6 +77,16 @@ The new frontend view that renders Work Items of any type using global Work Item
Task is a special Work Item type. Tasks can be added to issues as child items and can be displayed in the modal on the issue view.
+### Feature flags
+
+Since this is a large project with numerous moving parts, feature flags are being used to track promotions of available widgets. The table below shows the different feature flags that are being used, and the audience that they are available to.
+
+| feature flag name | audience |
+|---|---|
+| `work_items` | defaulted to on |
+| `work_items_mvc` | `gitlab-org`, `gitlab-com` |
+| `work_items_mvc_2` | `gitlab-org/plan-stage` |
+
## Motivation
Work Items main goal is to enhance the planning toolset to become the most popular collaboration tool for knowledge workers in any industry.
@@ -107,24 +122,3 @@ Work Item architecture is designed with making all the features for all the type
- [Tasks roadmap](https://gitlab.com/groups/gitlab-org/-/epics/7103?_gl=1*zqatx*_ga*NzUyOTc3NTc1LjE2NjEzNDcwMDQ.*_ga_ENFH3X7M5Y*MTY2MjU0MDQ0MC43LjEuMTY2MjU0MDc2MC4wLjAuMA..)
- [Work Item "Vision" Prototype](https://gitlab.com/gitlab-org/gitlab/-/issues/368607)
- [Work Item Discussions](https://gitlab.com/groups/gitlab-org/-/epics/7060)
-
-### Who
-
-| Role | Who
-|------------------------------|-----------------------------|
-| Author | Natalia Tepluhina |
-| Architecture Evolution Coach | Kamil Trzciński |
-| Engineering Leader | TBD |
-| Product Manager | Gabe Weaver |
-| Domain Expert / Frontend | Natalia Tepluhina |
-| Domain Expert / Backend | Heinrich Lee Yu |
-| Domain Expert / Backend | Jan Provaznik |
-| Domain Expert / Backend | Mario Celi |
-
-DRIs:
-
-| Role | Who
-|------------------------------|------------------------|
-| Leadership | TBD |
-| Product | Gabe Weaver |
-| Engineering | TBD |
diff --git a/doc/architecture/index.md b/doc/architecture/index.md
index 643f5766b0a..689ff2afaa0 100644
--- a/doc/architecture/index.md
+++ b/doc/architecture/index.md
@@ -1,9 +1,7 @@
---
-stage: none
-group: unassigned
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+feedback: false
comments: false
-description: 'Architecture Practice at GitLab'
+toc: false
---
# Architecture at GitLab
diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md
index 3354c94aff2..f0efb5fc884 100644
--- a/doc/ci/chatops/index.md
+++ b/doc/ci/chatops/index.md
@@ -121,6 +121,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md
index 6256f11f1ea..a0665e1c054 100644
--- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md
+++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md
@@ -25,11 +25,11 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:
- You can generate and use a [Bitbucket App Password](https://support.atlassian.com/bitbucket-cloud/docs/app-passwords/) for the password field.
GitLab imports the repository and enables [Pull Mirroring](../../user/project/repository/mirror/pull.md).
- You can check that mirroring is working in the project by going to **Settings > Repository > Mirroring repositories**.
+ You can check that mirroring is working in the project in **Settings > Repository > Mirroring repositories**.
1. In GitLab, create a
[Personal Access Token](../../user/profile/personal_access_tokens.md)
- with `api` scope. This is used to authenticate requests from the web
+ with `api` scope. The token is used to authenticate requests from the web
hook that is created in Bitbucket to notify GitLab of new commits.
1. In Bitbucket, from **Settings > Webhooks**, create a new web hook to notify
@@ -58,18 +58,14 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:
1. In GitLab, from **Settings > CI/CD > Variables**, add variables to allow
communication with Bitbucket via the Bitbucket API:
- `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above.
+ - `BITBUCKET_ACCESS_TOKEN`: The Bitbucket app password created above. This variable should be [masked](../variables/index.md#mask-a-cicd-variable).
+ - `BITBUCKET_USERNAME`: The username of the Bitbucket account.
+ - `BITBUCKET_NAMESPACE`: Set this variable if your GitLab and Bitbucket namespaces differ.
+ - `BITBUCKET_REPOSITORY`: Set this variable if your GitLab and Bitbucket project names differ.
- `BITBUCKET_USERNAME`: the username of the Bitbucket account.
-
- `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ.
-
- `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ.
-
-1. In Bitbucket, add a script to push the pipeline status to Bitbucket.
-
- NOTE:
- The changes must be made in Bitbucket as any changes in the GitLab repository are overwritten by Bitbucket when GitLab next mirrors the repository.
+1. In Bitbucket, add a script that pushes the pipeline status to Bitbucket. The script
+ is created in Bitbucket, but the mirroring process copies it to the GitLab mirror. The GitLab
+ CI/CD pipeline runs the script, and pushes the status back to Bitbucket.
Create a file `build_status` and insert the script below and run
`chmod +x build_status` in your terminal to make the script executable.
@@ -125,7 +121,8 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:
```
1. In Bitbucket, create a `.gitlab-ci.yml` file to use the script to push
- pipeline success and failures to Bitbucket.
+ pipeline success and failures to Bitbucket. Similar to the script added above,
+ this file is copied to the GitLab repo as part of the mirroring process.
```yaml
stages:
@@ -168,6 +165,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md
index 18cc430c225..9933fafcb69 100644
--- a/doc/ci/ci_cd_for_external_repos/github_integration.md
+++ b/doc/ci/ci_cd_for_external_repos/github_integration.md
@@ -98,6 +98,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md
index 83774f63566..bd9276275a7 100644
--- a/doc/ci/cloud_deployment/index.md
+++ b/doc/ci/cloud_deployment/index.md
@@ -76,7 +76,7 @@ Prerequisites:
- [Authenticate AWS with GitLab](#authenticate-gitlab-with-aws).
- Create a cluster on Amazon ECS.
-- Create related components, like an ECS service, a database on Amazon RDS, and so on.
+- Create related components, like an ECS service or a database on Amazon RDS.
- Create an ECS task definition, where the value for the `containerDefinitions[].name` attribute is
the same as the `Container name` defined in your targeted ECS service. The task definition can be:
- An existing task definition in ECS.
diff --git a/doc/ci/cloud_services/azure/index.md b/doc/ci/cloud_services/azure/index.md
index 944f95c03e2..b2f78648be9 100644
--- a/doc/ci/cloud_services/azure/index.md
+++ b/doc/ci/cloud_services/azure/index.md
@@ -16,7 +16,7 @@ Prerequisites:
- Access to an existing Azure Subscription with `Owner` access level.
- Access to the corresponding Azure Active Directory Tenant with at least the `Application Developer` access level.
-- A local installation of the [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli).
+- A local installation of the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).
Alternatively, you can follow all the steps below with the [Azure Cloud Shell](https://shell.azure.com/).
- A GitLab project.
@@ -27,11 +27,11 @@ To complete this tutorial:
1. [Grant permissions for the service principal](#grant-permissions-for-the-service-principal).
1. [Retrieve a temporary credential](#retrieve-a-temporary-credential).
-For more information, review Azure's documentation on [Workload identity federation](https://learn.microsoft.com/azure/active-directory/develop/workload-identity-federation).
+For more information, review Azure's documentation on [Workload identity federation](https://learn.microsoft.com/en-us/azure/active-directory/develop/workload-identity-federation).
## Create Azure AD application and service principal
-To create an [Azure AD application](https://learn.microsoft.com/cli/azure/ad/app?view=azure-cli-latest#az-ad-app-create)
+To create an [Azure AD application](https://learn.microsoft.com/en-us/cli/azure/ad/app?view=azure-cli-latest#az-ad-app-create)
and service principal:
1. In the Azure CLI, create the AD application:
@@ -43,13 +43,13 @@ and service principal:
Save the `appId` (Application client ID) output, as you need it later
to configure your GitLab CI/CD pipeline.
-1. Create a corresponding [Service Principal](https://learn.microsoft.com/cli/azure/ad/sp?view=azure-cli-latest#az-ad-sp-create):
+1. Create a corresponding [Service Principal](https://learn.microsoft.com/en-us/cli/azure/ad/sp?view=azure-cli-latest#az-ad-sp-create):
```shell
az ad sp create --id $appId --query appId -otsv
```
-Instead of the Azure CLI, you can [use the Azure Portal to create these resources](https://learn.microsoft.com/azure/active-directory/develop/howto-create-service-principal-portal).
+Instead of the Azure CLI, you can [use the Azure Portal to create these resources](https://learn.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal).
## Create Azure AD federated identity credentials
@@ -88,7 +88,7 @@ identity credentials from the Azure Portal:
## Grant permissions for the service principal
-After you create the credentials, use [`role assignment`](https://learn.microsoft.com/cli/azure/role/assignment?view=azure-cli-latest#az-role-assignment-create)
+After you create the credentials, use [`role assignment`](https://learn.microsoft.com/en-us/cli/azure/role/assignment?view=azure-cli-latest#az-role-assignment-create)
to grant permissions to the above service principal to access to Azure resources:
```shell
@@ -97,13 +97,13 @@ az role assignment create --assignee $appId --role Reader --scope /subscriptions
You can find your subscription ID in:
-- The [Azure Portal](https://learn.microsoft.com/azure/azure-portal/get-subscription-tenant-id#find-your-azure-subscription).
-- The [Azure CLI](https://learn.microsoft.com/cli/azure/manage-azure-subscriptions-azure-cli#get-the-active-subscription).
+- The [Azure Portal](https://learn.microsoft.com/en-us/azure/azure-portal/get-subscription-tenant-id#find-your-azure-subscription).
+- The [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/manage-azure-subscriptions-azure-cli#get-the-active-subscription).
## Retrieve a temporary credential
After you configure the Azure AD application and federated identity credentials,
-the CI/CD job can retrieve a temporary credential by using the [Azure CLI](https://learn.microsoft.com/cli/azure/reference-index?view=azure-cli-latest#az-login):
+the CI/CD job can retrieve a temporary credential by using the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/reference-index?view=azure-cli-latest#az-login):
```yaml
default:
@@ -123,7 +123,7 @@ The CI/CD variables are:
- `AZURE_CLIENT_ID`: The [application client ID you saved earlier](#create-azure-ad-application-and-service-principal).
- `AZURE_TENANT_ID`: Your Azure Active Directory. You can
- [find it by using the Azure CLI or Azure Portal](https://learn.microsoft.com/azure/active-directory/fundamentals/active-directory-how-to-find-tenant).
+ [find it by using the Azure CLI or Azure Portal](https://learn.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-how-to-find-tenant).
- `CI_JOB_JWT_V2`: The JSON web token is a [predefined CI/CD variable](../../variables/predefined_variables.md).
## Troubleshooting
diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md
index ded91edeff5..39f45471021 100644
--- a/doc/ci/directed_acyclic_graph/index.md
+++ b/doc/ci/directed_acyclic_graph/index.md
@@ -91,7 +91,7 @@ To see the needs visualization, select **Needs** when viewing a pipeline that us
![Needs visualization example](img/dag_graph_example_v13_1.png)
-Clicking a node highlights all the job paths it depends on.
+Selecting a node highlights all the job paths it depends on.
![Needs visualization with path highlight](img/dag_graph_example_clicked_v13_1.png)
diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md
index 06197d71690..ea62133cb7f 100644
--- a/doc/ci/docker/using_docker_build.md
+++ b/doc/ci/docker/using_docker_build.md
@@ -47,7 +47,7 @@ the Docker commands, but needs permission to do so.
```
1. On the server where GitLab Runner is installed, install Docker Engine.
- View a list of [supported platforms](https://docs.docker.com/engine/installation/).
+ View a list of [supported platforms](https://docs.docker.com/engine/install/).
1. Add the `gitlab-runner` user to the `docker` group:
@@ -829,7 +829,7 @@ environment = ["DOCKER_DRIVER=overlay2"]
If you're running multiple runners, you have to modify all configuration files.
Read more about the [runner configuration](https://docs.gitlab.com/runner/configuration/)
-and [using the OverlayFS storage driver](https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/).
+and [using the OverlayFS storage driver](https://docs.docker.com/storage/storagedriver/overlayfs-driver/).
## Docker alternatives
diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md
index 70680a44ed2..0ba510acdbc 100644
--- a/doc/ci/docker/using_docker_images.md
+++ b/doc/ci/docker/using_docker_images.md
@@ -64,7 +64,7 @@ For example, you can set the [Docker pull policy](https://docs.gitlab.com/runner
to use local images.
For more information about images and Docker Hub, see
-the [Docker Fundamentals](https://docs.docker.com/engine/understanding-docker/) documentation.
+the [Docker overview](https://docs.docker.com/get-started/overview/).
## Define `image` in the `.gitlab-ci.yml` file
diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md
index adb9b5a87d5..e75f902c153 100644
--- a/doc/ci/enable_or_disable_ci.md
+++ b/doc/ci/enable_or_disable_ci.md
@@ -54,6 +54,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md
index 5f5ec027827..d7fa31b583b 100644
--- a/doc/ci/environments/deployment_approvals.md
+++ b/doc/ci/environments/deployment_approvals.md
@@ -64,10 +64,9 @@ co-exist and multiple approval rules takes the precedence over the unified appro
There are two ways to configure approvals for a protected environment:
-1. Using the [UI](protected_environments.md#protecting-environments)
- 1. Set the **Required approvals** field to 1 or more.
-1. Using the [REST API](../../api/protected_environments.md#protect-repository-environments)
- 2. Set the `required_approval_count` field to 1 or more.
+- Using the [UI](protected_environments.md#protecting-environments), set the **Required approvals** field to 1 or more.
+- Using the [REST API](../../api/protected_environments.md#protect-a-single-environment),
+ set the `required_approval_count` field to 1 or more.
After this is configured, all jobs deploying to this environment automatically go into a blocked state and wait for approvals before running. Ensure that the number of required approvals is less than the number of users allowed to deploy.
@@ -89,9 +88,9 @@ Maintainer role.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) in GitLab 14.10 with a flag named `deployment_approval_rules`. Disabled by default.
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) in GitLab 15.0. [Feature flag `deployment_approval_rules`](https://gitlab.com/gitlab-org/gitlab/-/issues/345678) removed.
-1. Using the [REST API](../../api/group_protected_environments.md#protect-an-environment).
- 1. `deploy_access_levels` represents which entity can execute the deployment job.
- 1. `approval_rules` represents which entity can approve the deployment job.
+- Using the [REST API](../../api/group_protected_environments.md#protect-a-single-environment).
+ - `deploy_access_levels` represents which entity can execute the deployment job.
+ - `approval_rules` represents which entity can approve the deployment job.
After this is configured, all jobs deploying to this environment automatically go into a blocked state and wait for approvals before running. Ensure that the number of required approvals is less than the number of users allowed to deploy.
@@ -138,10 +137,6 @@ To approve or reject a deployment to a protected environment using the UI:
1. Optional. Add a comment which describes your reason for approving or rejecting the deployment.
1. Select **Approve** or **Reject**.
-NOTE:
-This feature might not work as expected when [Multiple approval rules](#multiple-approval-rules) is configured.
-See the [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/355708) for planned improvement.
-
### Approve or reject a deployment using the API
Prerequisites:
@@ -191,6 +186,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md
index 665aeb23d28..1e4eb54c559 100644
--- a/doc/ci/environments/deployment_safety.md
+++ b/doc/ci/environments/deployment_safety.md
@@ -13,16 +13,20 @@ that help maintain deployment security and stability.
You can:
+- Set appropriate roles to your project. See [Project members permissions](../../user/permissions.md#project-members-permissions)
+ for the different user roles GitLab supports and the permissions of each.
- [Restrict write-access to a critical environment](#restrict-write-access-to-a-critical-environment)
- [Prevent deployments during deploy freeze windows](#prevent-deployments-during-deploy-freeze-windows)
-- [Set appropriate roles to your project](#setting-appropriate-roles-to-your-project)
- [Protect production secrets](#protect-production-secrets)
- [Separate project for deployments](#separate-project-for-deployments)
If you are using a continuous deployment workflow and want to ensure that concurrent deployments to the same environment do not happen, you should enable the following options:
- [Ensure only one deployment job runs at a time](#ensure-only-one-deployment-job-runs-at-a-time)
-- [Skip outdated deployment jobs](#skip-outdated-deployment-jobs)
+- [Prevent outdated deployment jobs](#prevent-outdated-deployment-jobs)
+
+<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
+For an overview, see [How to secure your CD pipelines/workflow](https://www.youtube.com/watch?v=Mq3C1KveDc0).
## Restrict write access to a critical environment
@@ -63,7 +67,10 @@ The improved pipeline flow **after** using the resource group:
For more information, see [Resource Group documentation](../resource_groups/index.md).
-## Skip outdated deployment jobs
+## Prevent outdated deployment jobs
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25276) in GitLab 12.9.
+> - In GitLab 15.5, the behavior was [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/363328) to prevent outdated job runs.
The effective execution order of pipeline jobs can vary from run to run, which
could cause undesired behavior. For example, a [deployment job](../jobs/index.md#deployment-jobs)
@@ -71,22 +78,46 @@ in a newer pipeline could finish before a deployment job in an older pipeline.
This creates a race condition where the older deployment finishes later,
overwriting the "newer" deployment.
-You can ensure that older deployment jobs are cancelled automatically when a newer deployment
-job is started by enabling the [Skip outdated deployment jobs](../pipelines/settings.md#skip-outdated-deployment-jobs) feature.
+You can prevent older deployment jobs from running when a newer deployment
+job is started by enabling the [Prevent outdated deployment jobs](../pipelines/settings.md#prevent-outdated-deployment-jobs) feature.
+
+When an older deployment job starts, it fails and is labeled:
+
+- `failed outdated deployment job` in the pipeline view.
+- `The deployment job is older than the latest deployment, and therefore failed.`
+ when viewing the completed job.
+
+When an older deployment job is manual, the play button is disabled with a message
+`This deployment job does not run automatically and must be started manually, but it's older than the latest deployment, and therefore can't run.`.
+
+Job age is determined by the job start time, not the commit time, so a newer commit
+can be prevented in some circumstances.
+
+### How to rollback to an outdated deployment
-Example of a problematic pipeline flow **before** enabling Skip outdated deployment jobs:
+> In GitLab 15.6, [rollback via job retry was introduced back](https://gitlab.com/gitlab-org/gitlab/-/issues/378359).
+
+In some cases, you need to rollback to an outdated deployment.
+This feature explicitly allows rollback via [Environment Rollback](index.md#environment-rollback),
+so that you can quickly rollback in an urgent case.
+
+Alternatively, you can run a new pipeline with a previous commit. It contains newer deployment jobs than the latest deployment.
+
+### Example
+
+Example of a problematic pipeline flow **before** enabling Prevent outdated deployment jobs:
1. Pipeline-A is created on the default branch.
1. Later, Pipeline-B is created on the default branch (with a newer commit SHA).
1. The `deploy` job in Pipeline-B finishes first, and deploys the newer code.
1. The `deploy` job in Pipeline-A finished later, and deploys the older code, **overwriting** the newer (latest) deployment.
-The improved pipeline flow **after** enabling Skip outdated deployment jobs:
+The improved pipeline flow **after** enabling Prevent outdated deployment jobs:
1. Pipeline-A is created on the default branch.
1. Later, Pipeline-B is created on the default branch (with a newer SHA).
1. The `deploy` job in Pipeline-B finishes first, and deploys the newer code.
-1. The `deploy` job in Pipeline-A was automatically cancelled, so that it doesn't overwrite the deployment from the newer pipeline.
+1. The `deploy` job in Pipeline-A fails, so that it doesn't overwrite the deployment from the newer pipeline.
## Prevent deployments during deploy freeze windows
@@ -95,19 +126,6 @@ vacation period when most employees are out, you can set up a [Deploy Freeze](..
During a deploy freeze period, no deployment can be executed. This is helpful to
ensure that deployments do not happen unexpectedly.
-## Setting appropriate roles to your project
-
-GitLab supports several different roles that can be assigned to your project members. See
-[Project members permissions](../../user/permissions.md#project-members-permissions)
-for an explanation of these roles and the permissions of each.
-
-<div class="video-fallback">
- See the video: <a href="https://www.youtube.com/watch?v=Mq3C1KveDc0">How to secure your CD pipelines</a>.
-</div>
-<figure class="video-container">
- <iframe src="https://www.youtube.com/embed/Mq3C1KveDc0" frameborder="0" allowfullscreen="true"> </iframe>
-</figure>
-
## Protect production secrets
Production secrets are needed to deploy successfully. For example, when deploying to the cloud,
@@ -153,7 +171,7 @@ Before promoting a deployment to a production environment, cross-verifying it wi
### Pipelines jobs fail with `The deployment job is older than the previously succeeded deployment job...`
-This is caused by the [Skip outdated deployment jobs](../pipelines/settings.md#skip-outdated-deployment-jobs) feature.
+This is caused by the [Prevent outdated deployment jobs](../pipelines/settings.md#prevent-outdated-deployment-jobs) feature.
If you have multiple jobs for the same environment (including non-deployment jobs), you might encounter this problem, for example:
```yaml
@@ -166,5 +184,5 @@ build:service-b:
name: production
```
-The [Skip outdated deployment jobs](../pipelines/settings.md#skip-outdated-deployment-jobs) might
+The [Prevent outdated deployment jobs](../pipelines/settings.md#prevent-outdated-deployment-jobs) might
not work well with this configuration, and must be disabled.
diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md
index f662c156f55..479b783202d 100644
--- a/doc/ci/environments/environments_dashboard.md
+++ b/doc/ci/environments/environments_dashboard.md
@@ -29,7 +29,7 @@ The Environments dashboard displays a paginated list of projects that includes
up to three environments per project.
The listed environments for each project are unique, such as
-"production", "staging", and so on. Review apps and other grouped
+"production" and "staging". Review apps and other grouped
environments are not displayed.
## Adding a project to the dashboard
diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md
index af431a9812e..10cda68c4b5 100644
--- a/doc/ci/environments/incremental_rollouts.md
+++ b/doc/ci/environments/incremental_rollouts.md
@@ -76,7 +76,7 @@ available, demonstrating manually triggered incremental rollouts.
## Timed Rollouts
Timed rollouts behave in the same way as manual rollouts, except that each job is defined with a
-delay in minutes before it deploys. Clicking the job reveals the countdown.
+delay in minutes before it deploys. Selecting the job reveals the countdown.
![Timed rollout](img/timed_rollout_v12_7.png)
diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md
index 63641a9b7c3..c4672b9dc7e 100644
--- a/doc/ci/environments/index.md
+++ b/doc/ci/environments/index.md
@@ -178,7 +178,7 @@ deploy_prod:
The `when: manual` action:
- Exposes a play button for the job in the GitLab UI, with the text **Can be manually deployed to &lt;environment&gt;**.
-- Means the `deploy_prod` job is only triggered when the play button is clicked.
+- Means the `deploy_prod` job is only triggered when the play button is selected.
You can find the play button in the pipelines, environments, deployments, and jobs views.
@@ -383,6 +383,11 @@ To retry or rollback a deployment:
- To retry a deployment, select **Re-deploy to environment**.
- To roll back to a deployment, next to a previously successful deployment, select **Rollback environment**.
+NOTE:
+If you have [prevented outdated deployment jobs](deployment_safety.md#prevent-outdated-deployment-jobs) in your project,
+the rollback buttons might be hidden or disabled.
+In this case, see [how to rollback to an outdated deployment](deployment_safety.md#how-to-rollback-to-an-outdated-deployment).
+
### Environment URL
> - [Fixed](https://gitlab.com/gitlab-org/gitlab/-/issues/337417) to persist arbitrary URLs in GitLab 15.2 [with a flag](../../administration/feature_flags.md) named `soft_validation_on_external_url`. Disabled by default.
@@ -721,7 +726,7 @@ build:
```
This gives you access to environment-scoped variables, and can be used to protect builds from unauthorized access. Also,
-it's effective to avoid the [skip outdated deployment jobs](deployment_safety.md#skip-outdated-deployment-jobs) feature.
+it's effective to avoid the [prevent outdated deployment jobs](deployment_safety.md#prevent-outdated-deployment-jobs) feature.
### Group similar environments
@@ -1063,8 +1068,8 @@ To fix this, use one of the following solutions:
- Remove `environment` keyword from the deployment job. GitLab has already been
ignoring the invalid keyword, therefore your deployment pipelines stay intact
even after the keyword removal.
-- Ensure the variable exists in the pipeline. Please note that there is
- [a limitation on supported variables](../variables/where_variables_can_be_used.md#gitlab-ciyml-file).
+- Ensure the variable exists in the pipeline. Review the
+ [limitation on supported variables](../variables/where_variables_can_be_used.md#gitlab-ciyml-file).
#### If you get this error on Review Apps
@@ -1107,6 +1112,6 @@ to execute the following command on Rails console:
Project.find_by_full_path(<your-project-full-path>).deployments.where(archived: true).each(&:create_ref)
```
-Please note that GitLab could drop this support in the future for the performance concern.
+GitLab might drop this support in the future for the performance concern.
You can open an issue in [GitLab Issue Tracker](https://gitlab.com/gitlab-org/gitlab/-/issues/new)
to discuss the behavior of this feature.
diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md
index 0f943679c07..5c120da32a0 100644
--- a/doc/ci/environments/protected_environments.md
+++ b/doc/ci/environments/protected_environments.md
@@ -26,7 +26,7 @@ Maintainer role.
Prerequisites:
-- When granting the **Allowed to deploy** permission to a group or subgroup, the user configuring the protected environment must be a **direct member** of the group or subgroup to be added. Otherwise, the group or subgroup will not show up in the dropdown. For more information see [issue #345140](https://gitlab.com/gitlab-org/gitlab/-/issues/345140).
+- When granting the **Allowed to deploy** permission to a group or subgroup, the user configuring the protected environment must be a **direct member** of the group or subgroup to be added. Otherwise, the group or subgroup will not show up in the dropdown list. For more information see [issue #345140](https://gitlab.com/gitlab-org/gitlab/-/issues/345140).
To protect an environment:
@@ -147,7 +147,7 @@ Maintainers can:
- Update existing protected environments at any time by changing the access in the
**Allowed to Deploy** dropdown list.
-- Unprotect a protected environment by clicking the **Unprotect** button for that environment.
+- Unprotect a protected environment by selecting the **Unprotect** button for that environment.
After an environment is unprotected, all access entries are deleted and must
be re-entered if the environment is re-protected.
diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md
index 59e377dbb09..7208caaccae 100644
--- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md
+++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md
@@ -29,7 +29,7 @@ You must replace the `vault.example.com` URL below with the URL of your Vault se
## How it works
-Each job has JSON Web Token (JWT) provided as CI/CD variable named `CI_JOB_JWT`. This JWT can be used to authenticate with Vault using the [JWT Auth](https://www.vaultproject.io/docs/auth/jwt#jwt-authentication) method.
+Each job has JSON Web Token (JWT) provided as CI/CD variable named `CI_JOB_JWT`. This JWT can be used to authenticate with Vault using the [JWT Auth](https://developer.hashicorp.com/vault/docs/auth/jwt#jwt-authentication) method.
The following fields are included in the JWT:
@@ -90,7 +90,7 @@ The JWT is encoded by using RS256 and signed with a dedicated private key. The e
You can use this JWT and your instance's JWKS endpoint (`https://gitlab.example.com/-/jwks`) to authenticate with a Vault server that is configured to allow the JWT Authentication method for authentication.
-When configuring roles in Vault, you can use [bound_claims](https://www.vaultproject.io/docs/auth/jwt#bound-claims) to match against the JWT's claims and restrict which secrets each CI job has access to.
+When configuring roles in Vault, you can use [bound_claims](https://developer.hashicorp.com/vault/docs/auth/jwt#bound-claims) to match against the JWT's claims and restrict which secrets each CI job has access to.
To communicate with Vault, you can use either its CLI client or perform API requests (using `curl` or another client).
@@ -109,7 +109,7 @@ $ vault kv get -field=password secret/myproject/production/db
real-pa$$w0rd
```
-To configure your Vault server, start by enabling the [JWT Auth](https://www.vaultproject.io/docs/auth/jwt) method:
+To configure your Vault server, start by enabling the [JWT Auth](https://developer.hashicorp.com/vault/docs/auth/jwt) method:
```shell
$ vault auth enable jwt
@@ -180,17 +180,17 @@ $ vault write auth/jwt/role/myproject-production - <<EOF
EOF
```
-This example uses [bound_claims](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims) to specify that only a JWT with matching values for the specified claims is allowed to authenticate.
+This example uses [bound_claims](https://developer.hashicorp.com/vault/api-docs/auth/jwt#bound_claims) to specify that only a JWT with matching values for the specified claims is allowed to authenticate.
Combined with [protected branches](../../../user/project/protected_branches.md), you can restrict who is able to authenticate and read the secrets.
-[`token_explicit_max_ttl`](https://www.vaultproject.io/api-docs/auth/jwt#token_explicit_max_ttl) specifies that the token issued by Vault, upon successful authentication, has a hard lifetime limit of 60 seconds.
+[`token_explicit_max_ttl`](https://developer.hashicorp.com/vault/api-docs/auth/jwt#token_explicit_max_ttl) specifies that the token issued by Vault, upon successful authentication, has a hard lifetime limit of 60 seconds.
-[`user_claim`](https://www.vaultproject.io/api-docs/auth/jwt#user_claim) specifies the name for the Identity alias created by Vault upon a successful login.
+[`user_claim`](https://developer.hashicorp.com/vault/api-docs/auth/jwt#user_claim) specifies the name for the Identity alias created by Vault upon a successful login.
-[`bound_claims_type`](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values are interpreted as globs, with `*` matching any number of characters.
+[`bound_claims_type`](https://developer.hashicorp.com/vault/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values are interpreted as globs, with `*` matching any number of characters.
-The claim fields listed in [the table above](#how-it-works) can also be accessed for [Vault's policy path templating](https://learn.hashicorp.com/tutorials/vault/policy-templating?in=vault/policies) purposes by using the accessor name of the JWT auth within Vault. The [mount accessor name](https://learn.hashicorp.com/tutorials/vault/identity#step-1-create-an-entity-with-alias) (`ACCESSOR_NAME` in the example below) can be retrieved by running `vault auth list`.
+The claim fields listed in [the table above](#how-it-works) can also be accessed for [Vault's policy path templating](https://developer.hashicorp.com/vault/tutorials/policies/policy-templating?in=vault%2Fpolicies) purposes by using the accessor name of the JWT auth within Vault. The [mount accessor name](https://developer.hashicorp.com/vault/tutorials/auth-methods/identity#step-1-create-an-entity-with-alias) (`ACCESSOR_NAME` in the example below) can be retrieved by running `vault auth list`.
Policy template example making use of a named metadata field named `project_path`:
@@ -200,7 +200,7 @@ path "secret/data/{{identity.entity.aliases.ACCESSOR_NAME.metadata.project_path}
}
```
-Role example to support the templated policy above, mapping the claim field `project_path` as a metadata field through use of [`claim_mappings`](https://www.vaultproject.io/api-docs/auth/jwt#claim_mappings) configuration:
+Role example to support the templated policy above, mapping the claim field `project_path` as a metadata field through use of [`claim_mappings`](https://developer.hashicorp.com/vault/api-docs/auth/jwt#claim_mappings) configuration:
```plaintext
{
@@ -212,7 +212,7 @@ Role example to support the templated policy above, mapping the claim field `pro
}
```
-For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api-docs/auth/jwt#create-role).
+For the full list of options, see Vault's [Create Role documentation](https://developer.hashicorp.com/vault/api-docs/auth/jwt#create-role).
WARNING:
Always restrict your roles to project or namespace by using one of the provided claims (for example, `project_id` or `namespace_id`). Otherwise any JWT generated by this instance may be allowed to authenticate using this role.
@@ -225,9 +225,9 @@ $ vault write auth/jwt/config \
bound_issuer="gitlab.example.com"
```
-[bound_issuer](https://www.vaultproject.io/api-docs/auth/jwt#bound_issuer) specifies that only a JWT with the issuer (that is, the `iss` claim) set to `gitlab.example.com` can use this method to authenticate, and that the JWKS endpoint (`https://gitlab.example.com/-/jwks`) should be used to validate the token.
+[bound_issuer](https://developer.hashicorp.com/vault/api-docs/auth/jwt#bound_issuer) specifies that only a JWT with the issuer (that is, the `iss` claim) set to `gitlab.example.com` can use this method to authenticate, and that the JWKS endpoint (`https://gitlab.example.com/-/jwks`) should be used to validate the token.
-For the full list of available configuration options, see Vault's [API documentation](https://www.vaultproject.io/api-docs/auth/jwt#configure).
+For the full list of available configuration options, see Vault's [API documentation](https://developer.hashicorp.com/vault/api-docs/auth/jwt#configure).
The following job, when run for the default branch, is able to read secrets under `secret/myproject/staging/`, but not the secrets under `secret/myproject/production/`:
@@ -242,7 +242,7 @@ read_secrets:
# Vault's address can be provided here or as CI/CD variable
- export VAULT_ADDR=http://vault.example.com:8200
# Authenticate and get token. Token expiry time and other properties can be configured
- # when configuring JWT Auth - https://www.vaultproject.io/api-docs/auth/jwt#parameters-1
+ # when configuring JWT Auth - https://developer.hashicorp.com/vault/api-docs/auth/jwt#parameters-1
- export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=myproject-staging jwt=$CI_JOB_JWT)"
# Now use the VAULT_TOKEN to read the secret and store it in an environment variable
- export PASSWORD="$(vault kv get -field=password secret/myproject/staging/db)"
@@ -271,7 +271,7 @@ read_secrets:
# Vault's address can be provided here or as CI/CD variable
- export VAULT_ADDR=http://vault.example.com:8200
# Authenticate and get token. Token expiry time and other properties can be configured
- # when configuring JWT Auth - https://www.vaultproject.io/api-docs/auth/jwt#parameters-1
+ # when configuring JWT Auth - https://developer.hashicorp.com/vault/api-docs/auth/jwt#parameters-1
- export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=myproject-production jwt=$CI_JOB_JWT)"
# Now use the VAULT_TOKEN to read the secret and store it in environment variable
- export PASSWORD="$(vault kv get -field=password secret/myproject/production/db)"
@@ -286,11 +286,11 @@ read_secrets:
You can control `CI_JOB_JWT` access to Vault secrets by using Vault protections
and GitLab features. For example, restrict the token by:
-- Using Vault [bound_claims](https://www.vaultproject.io/docs/auth/jwt#bound-claims)
+- Using Vault [bound_claims](https://developer.hashicorp.com/vault/docs/auth/jwt#bound-claims)
for specific groups using `group_claim`.
- Hard coding values for Vault bound claims based on the `user_login` and `user_email`
of specific users.
-- Setting Vault time limits for TTL of the token as specified in [`token_explicit_max_ttl`](https://www.vaultproject.io/api-docs/auth/jwt#token_explicit_max_ttl),
+- Setting Vault time limits for TTL of the token as specified in [`token_explicit_max_ttl`](https://developer.hashicorp.com/vault/api-docs/auth/jwt#token_explicit_max_ttl),
where the token expires after authentication.
- Scoping the JWT to [GitLab protected branches](../../../user/project/protected_branches.md)
that are restricted to a subset of project users.
diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md
index f14372757a9..a603207aef7 100644
--- a/doc/ci/examples/deployment/composer-npm-deploy.md
+++ b/doc/ci/examples/deployment/composer-npm-deploy.md
@@ -47,7 +47,7 @@ All these operations put all files into a `build` folder, which is ready to be d
## How to transfer files to a live server
-You have multiple options: rsync, SCP, SFTP, and so on. For now, use SCP.
+You have multiple options such as rsync, SCP, or SFTP. For now, use SCP.
To make this work, you must add a GitLab CI/CD Variable (accessible on `gitlab.example/your-project-name/variables`). Name this variable `STAGING_PRIVATE_KEY` and set it to the **private** SSH key of your server.
diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md
index e9f126b0409..7b5690e2d3d 100644
--- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md
+++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md
@@ -88,7 +88,7 @@ hit our 404 page. We can then use [`browser.getUrl`](http://v4.webdriver.io/api/
to verify that the current page is indeed at the location we specified. To interact with the page,
we can pass CSS selectors to
[`browser.element`](http://v4.webdriver.io/api/protocol/element.html) to get access to elements on the
-page and to interact with them - for example, to click on the link back to the home page.
+page and to interact with them - for example, to select the link back to the home page.
The simple test shown above
can already give us a lot of confidence if it passes: we know our deployment has succeeded, that the
diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md
index 80e476f2a87..28016216dbb 100644
--- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md
+++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md
@@ -444,7 +444,7 @@ On your GitLab project repository navigate to the **Registry** tab.
You may need to enable the Container Registry for your project to see this tab. You'll find it under your project's **Settings > General > Visibility, project features, permissions**.
To start using Container Registry on our machine, we first need to sign in to the GitLab registry using our GitLab username and password.
-Make sure you have [Docker](https://docs.docker.com/engine/installation/) installed on our machine,
+Make sure you have [Docker](https://docs.docker.com/engine/install/) installed on our machine,
then run the following commands:
```shell
@@ -628,11 +628,11 @@ To do that, commit and push `.gitlab-ci.yml` to the `main` branch. It will trigg
Here we see our **Test** and **Deploy** stages.
The **Test** stage has the `unit_test` build running.
-click on it to see the runner's output.
+Select it to see the runner's output.
![pipeline page](img/pipeline_page.png)
-After our code passed through the pipeline successfully, we can deploy to our production server by clicking the **play** button on the right side.
+After our code passed through the pipeline successfully, we can deploy to our production server by selecting the **play** button on the right side.
![pipelines page deploy button](img/pipelines_page_deploy_button.png)
@@ -644,7 +644,7 @@ If something doesn't work as expected, you can roll back to the latest working v
![environment page](img/environment_page.png)
-By clicking on the external link icon specified on the right side, GitLab opens the production website.
+By selecting the external link icon specified on the right side, GitLab opens the production website.
Our deployment successfully was done and we can see the application is live.
![Laravel welcome page](img/laravel_welcome_page.png)
diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md
index 88e63a7f36f..8f0321517ab 100644
--- a/doc/ci/examples/semantic-release.md
+++ b/doc/ci/examples/semantic-release.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md
index ee9d15894fe..68cbff21fae 100644
--- a/doc/ci/git_submodules.md
+++ b/doc/ci/git_submodules.md
@@ -60,6 +60,13 @@ To make submodules work correctly in CI/CD jobs:
GIT_SUBMODULE_STRATEGY: recursive
```
+1. Use `GIT_SUBMODULE_DEPTH` to configure the cloning depth of submodules independently of the [`GIT_DEPTH`](runners/configure_runners.md#shallow-cloning) variable:
+
+ ```yaml
+ variables:
+ GIT_SUBMODULE_DEPTH: 1
+ ```
+
1. You can filter or exclude specific submodules to control which submodules will be synced using
[`GIT_SUBMODULE_PATHS`](runners/configure_runners.md#git-submodule-paths).
@@ -76,6 +83,14 @@ To make submodules work correctly in CI/CD jobs:
GIT_SUBMODULE_STRATEGY: recursive
GIT_SUBMODULE_UPDATE_FLAGS: --jobs 4
```
+
+1. You can set the [GIT_SUBMODULE_PATHS](runners/configure_runners.md#sync-or-exclude-specific-submodules-from-ci-jobs) to explicitly ignore submodules during cloning:
+
+ ```yaml
+ variables:
+ GIT_SUBMODULE_STRATEGY: recursive
+ GIT_SUBMODULE_PATHS: ':(exclude)submodule'
+ ```
If you use the [`CI_JOB_TOKEN`](jobs/ci_job_token.md) to clone a submodule in a
pipeline job, the user executing the job must be assigned to a role that has
diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md
index a1df55cfc38..44c081b9d7f 100644
--- a/doc/ci/interactive_web_terminal/index.md
+++ b/doc/ci/interactive_web_terminal/index.md
@@ -65,7 +65,7 @@ for the current job.
![Example of job running with terminal available](img/interactive_web_terminal_running_job.png)
-When clicked, a new tab opens to the terminal page where you can access
+When selected, a new tab opens to the terminal page where you can access
the terminal and type commands like a normal shell.
![terminal of the job](img/interactive_web_terminal_page.png)
diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md
index 7282ebb0909..1e7b389c84a 100644
--- a/doc/ci/jobs/ci_job_token.md
+++ b/doc/ci/jobs/ci_job_token.md
@@ -26,8 +26,8 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints
- [Terraform plan](../../user/infrastructure/index.md).
The token has the same permissions to access the API as the user that caused the
-job to run. A user can cause a job to run by pushing a commit, triggering a manual job,
-being the owner of a scheduled pipeline, and so on. Therefore, this user must be assigned to
+job to run. A user can cause a job to run by taking action like pushing a commit,
+triggering a manual job, or being the owner of a scheduled pipeline. Therefore, this user must be assigned to
[a role that has the required privileges](../../user/permissions.md#gitlab-cicd-permissions).
The token is valid only while the pipeline job runs. After the job finishes, you can't
diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md
index d1a4f1e35bf..4eb8952dd73 100644
--- a/doc/ci/jobs/index.md
+++ b/doc/ci/jobs/index.md
@@ -37,7 +37,7 @@ independently from each other.
When you access a pipeline, you can see the related jobs for that pipeline.
-Clicking an individual job shows you its job log, and allows you to:
+Selecting an individual job shows you its job log, and allows you to:
- Cancel the job.
- Retry the job.
@@ -389,5 +389,5 @@ deploy me:
The behavior of deployment jobs can be controlled with
[deployment safety](../environments/deployment_safety.md) settings like
-[skipping outdated deployment jobs](../environments/deployment_safety.md#skip-outdated-deployment-jobs)
+[preventing outdated deployment jobs](../environments/deployment_safety.md#prevent-outdated-deployment-jobs)
and [ensuring only one deployment job runs at a time](../environments/deployment_safety.md#ensure-only-one-deployment-job-runs-at-a-time).
diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md
index 39ab0998291..d26c698af89 100644
--- a/doc/ci/jobs/job_control.md
+++ b/doc/ci/jobs/job_control.md
@@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
When a new pipeline starts, GitLab checks the pipeline configuration to determine
which jobs should run in that pipeline. You can configure jobs to run depending on
-the status of variables, the pipeline type, and so on.
+factors like the status of variables, or the pipeline type.
To configure a job to be included or excluded from certain pipelines, you can use:
@@ -107,6 +107,33 @@ job:
- make build
```
+#### Skip job if the branch is empty
+
+Use [`rules:changes:compare_to`](../yaml/index.md#ruleschangescompare_to) to avoid
+running a job when the branch is empty, which saves CI/CD resources. Compare the
+branch to the default branch, and if the branch:
+
+- Doesn't have changed files, the job doesn't run.
+- Has changed files, the job runs.
+
+For example, in a project with `main` as the default branch:
+
+```yaml
+job:
+ script:
+ - echo "This job only runs for branches that are not empty"
+ rules:
+ - if: $CI_COMMIT_BRANCH
+ changes:
+ compare_to: refs/heads/main
+ paths:
+ - '*'
+```
+
+The rule for this job compares all files and paths (`*`) in the current branch against
+the default branch `main`. The rule matches and the job runs only when there are
+changes to the files in the branch.
+
### Complex rules
You can use all `rules` keywords, like `if`, `changes`, and `exists`, in the same
@@ -1061,7 +1088,7 @@ docker_build:
When the `DOCKERFILES_DIR` variable is expanded in the `changes:` section, the full
path becomes `path/to/files//*`. The double slashes might cause unexpected behavior
-depending on the keyword used, shell and OS of the runner, and so on.
+depending on factors like the keyword used, or the shell and OS of the runner.
### `You are not allowed to download code from this project.` error message
diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md
index 5354e406e34..034c8ba5ad6 100644
--- a/doc/ci/migration/circleci.md
+++ b/doc/ci/migration/circleci.md
@@ -268,7 +268,7 @@ test_async:
## Contexts and variables
-CircleCI provides [Contexts](https://circleci.com/docs/contexts) to securely pass environment variables across project pipelines. In GitLab, a [Group](../../user/group/index.md) can be created to assemble related projects together. At the group level, [CI/CD variables](../variables/index.md#add-a-cicd-variable-to-a-group) can be stored outside the individual projects, and securely passed into pipelines across multiple projects.
+CircleCI provides [Contexts](https://circleci.com/docs/contexts/) to securely pass environment variables across project pipelines. In GitLab, a [Group](../../user/group/index.md) can be created to assemble related projects together. At the group level, [CI/CD variables](../variables/index.md#add-a-cicd-variable-to-a-group) can be stored outside the individual projects, and securely passed into pipelines across multiple projects.
## Orbs
diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md
index 300de610aa7..4ba59e14811 100644
--- a/doc/ci/migration/jenkins.md
+++ b/doc/ci/migration/jenkins.md
@@ -49,8 +49,8 @@ things we have found that help this:
your users understand why the effort is worth it. The value is clear when
the work is done, but people need to be aware while it's in progress too.
- Sponsorship and alignment from the relevant leadership team helps with the point above.
-- Spending time educating your users on what's different, sharing this document with them,
- and so on helps ensure you are successful.
+- Spending time educating your users on what's different and sharing this document
+ with them helps ensure you are successful.
- Finding ways to sequence or delay parts of the migration can help a lot, but you
don't want to leave things in a non-migrated (or partially-migrated) state for too
long. To gain all the benefits of GitLab, moving your existing Jenkins setup over
@@ -67,7 +67,7 @@ of transition, by letting you delay the migration of less urgent pipelines for a
If you are interested in helping GitLab test the wrapper, join our [public testing issue](https://gitlab.com/gitlab-org/gitlab/-/issues/215675) for instructions and to provide your feedback.
NOTE:
-If you have a paid GitLab subscription, note that the JenkinsFile Wrapper is not packaged as part of GitLab, and falls outside of the scope of support. For more information, see the [Statement of Support](https://about.gitlab.com/support/statement-of-support.html).
+If you have a paid GitLab subscription, note that the JenkinsFile Wrapper is not packaged as part of GitLab, and falls outside of the scope of support. For more information, see the [Statement of Support](https://about.gitlab.com/support/statement-of-support/).
## Important product differences
@@ -197,7 +197,7 @@ can leverage. You can see the complete list of packaging features in the
## Integrated features
-Where you may have used plugins to get things like code quality, unit tests, security scanning, and so on working in Jenkins,
+Where you may have used plugins to get things like code quality, unit tests, and security scanning working in Jenkins,
GitLab takes advantage of our connected ecosystem to automatically pull these kinds of results into
your merge requests, pipeline details pages, and other locations. You may find that you actually don't
need to configure anything to have these appear.
diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md
index a5484fcdf5a..772a06980af 100644
--- a/doc/ci/pipelines/cicd_minutes.md
+++ b/doc/ci/pipelines/cicd_minutes.md
@@ -128,9 +128,7 @@ These additional CI/CD minutes:
- Are carried over to the next month, if any remain at the end of the month.
- Are valid for 12 months from date of purchase or until all minutes are consumed, whichever comes first. Expiry of minutes is not currently enforced.
-If you use more CI/CD minutes than your monthly quota, when you purchase more,
-those CI/CD minutes are deducted from your quota. For example, with a GitLab SaaS
-Premium license:
+For example, with a GitLab SaaS Premium license:
- You have `10,000` monthly minutes.
- You purchase an additional `5,000` minutes.
@@ -232,7 +230,7 @@ For this reduced cost factor:
- The merge request source project must be a fork of a GitLab-maintained project,
such as [`gitlab-com/www-gitlab-com`](https://gitlab.com/gitlab-com/www-gitlab-com),
- [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab), and so on.
+ or [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab).
- The merge request target project must be the fork's parent project.
- The pipeline must be a merge request, merged results, or merge train pipeline.
@@ -271,9 +269,9 @@ the next month.
For example:
-- On **1st April**, you purchase `5,000` CI/CD minutes.
-- During April, you use only `3,000` of the `5,000` minutes.
-- On **1st May**, the remaining `2,000` minutes roll over and are added to your monthly quota.
+- On **1st April**, you purchase `5,000` additional CI/CD minutes.
+- During April, you use only `3,000` of the `5,000` additional minutes.
+- On **1st May**, the unused minute roll over, so you have `2,000` additional minutes available for May.
Additional CI/CD minutes are a one-time purchase and do not renew or refresh each month.
diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md
index 0b1963e1874..c8fcd06da07 100644
--- a/doc/ci/pipelines/downstream_pipelines.md
+++ b/doc/ci/pipelines/downstream_pipelines.md
@@ -20,8 +20,10 @@ but there are [key differences](pipeline_architectures.md).
## Parent-child pipelines
-A parent pipeline is one that triggers a downstream pipeline in the same project.
-The downstream pipeline is called a child pipeline. Child pipelines:
+A parent pipeline is a pipeline that triggers a downstream pipeline in the same project.
+The downstream pipeline is called a child pipeline.
+
+Child pipelines:
- Run under the same project, ref, and commit SHA as the parent pipeline.
- Do not directly affect the overall status of the ref the pipeline runs against. For example,
@@ -30,18 +32,18 @@ The downstream pipeline is called a child pipeline. Child pipelines:
pipeline is triggered with [`strategy:depend`](../yaml/index.md#triggerstrategy).
- Are automatically canceled if the pipeline is configured with [`interruptible`](../yaml/index.md#interruptible)
when a new pipeline is created for the same ref.
-- Are not displayed in the pipeline index page. You can only view child pipelines on
- their parent pipeline's page.
+- Are not displayed in the project's pipeline list. You can only view child pipelines on
+ their parent pipeline's details page.
### Nested child pipelines
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29651) in GitLab 13.4.
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243747) in GitLab 13.5.
-Parent and child pipelines were introduced with a maximum depth of one level of child
-pipelines, which was later increased to two. A parent pipeline can trigger many child
-pipelines, and these child pipelines can trigger their own child pipelines. It's not
-possible to trigger another level of child pipelines.
+Parent and child pipelines have a maximum depth of two levels of child pipelines.
+
+A parent pipeline can trigger many child pipelines, and these child pipelines can trigger
+their own child pipelines. You cannot trigger another level of child pipelines.
<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
For an overview, see [Nested Dynamic Pipelines](https://youtu.be/C5j3ju9je2M).
@@ -52,11 +54,6 @@ A pipeline in one project can trigger downstream pipelines in another project,
called multi-project pipelines. The user triggering the upstream pipeline must be able to
start pipelines in the downstream project, otherwise [the downstream pipeline fails to start](#trigger-job-fails-and-does-not-create-multi-project-pipeline).
-For example, you might deploy your web application from three different GitLab projects.
-With multi-project pipelines you can trigger a pipeline in each project, where each
-has its own build, test, and deploy process. You can visualize the connected pipelines
-in one place, including all cross-project interdependencies.
-
Multi-project pipelines:
- Are triggered from another project's pipeline, but the upstream (triggering) pipeline does
@@ -68,8 +65,7 @@ Multi-project pipelines:
- Are not automatically canceled in the downstream project when using [`interruptible`](../yaml/index.md#interruptible)
if a new pipeline runs for the same ref in the upstream pipeline. They can be
automatically canceled if a new pipeline is triggered for the same ref on the downstream project.
-- Multi-project pipelines are standalone pipelines because they are normal pipelines
- that happened to be triggered by an external project. They are all visible on the pipeline index page.
+- Are visible in the downstream project's pipeline list.
- Are independent, so there are no nesting limits.
Learn more in the "Cross-project Pipeline Triggering and Visualization" demo at
@@ -87,48 +83,49 @@ always displays:
Use the [`trigger`](../yaml/index.md#trigger) keyword in your `.gitlab-ci.yml` file
to create a job that triggers a downstream pipeline. This job is called a trigger job.
-After the trigger job starts, the initial status of the job is `pending` while GitLab
-attempts to create the downstream pipeline. If the downstream pipeline is created,
-GitLab marks the job as passed, otherwise the job failed. Alternatively,
-you can [set the trigger job to show the downstream pipeline's status](#mirror-the-status-of-a-downstream-pipeline-in-the-trigger-job)
-instead.
-
For example:
::Tabs
-:::TabTitle Multi-project pipeline
+:::TabTitle Parent-child pipeline
```yaml
trigger_job:
trigger:
- project: project-group/my-downstream-project
+ include:
+ - local: path/to/child-pipeline.yml
```
-:::TabTitle Parent-child pipeline
+:::TabTitle Multi-project pipeline
```yaml
trigger_job:
trigger:
- include:
- - local: path/to/child-pipeline.yml
+ project: project-group/my-downstream-project
```
::EndTabs
+After the trigger job starts, the initial status of the job is `pending` while GitLab
+attempts to create the downstream pipeline. The trigger job shows `passed` if the
+downstream pipeline is created successfully, otherwise it shows `failed`. Alternatively,
+you can [set the trigger job to show the downstream pipeline's status](#mirror-the-status-of-a-downstream-pipeline-in-the-trigger-job)
+instead.
+
### Use `rules` to control downstream pipeline jobs
-You can use CI/CD variables or the [`rules`](../yaml/index.md#rulesif) keyword to
-[control job behavior](../jobs/job_control.md) for downstream pipelines.
+Use CI/CD variables or the [`rules`](../yaml/index.md#rulesif) keyword to
+[control job behavior](../jobs/job_control.md) in downstream pipelines.
-When a downstream pipeline is triggered with the [`trigger`](../yaml/index.md#trigger) keyword,
+When you trigger a downstream pipeline with the [`trigger`](../yaml/index.md#trigger) keyword,
the value of the [`$CI_PIPELINE_SOURCE` predefined variable](../variables/predefined_variables.md)
for all jobs is:
- `pipeline` for multi-project pipelines.
-- `parent` for parent-child pipelines.
+- `parent_pipeline` for parent-child pipelines.
-For example, with a multi-project pipeline:
+For example, to control jobs in multi-project pipelines in a project that also runs
+merge request pipelines:
```yaml
job1:
@@ -148,38 +145,12 @@ job3:
script: echo "This job runs in both multi-project and merge request pipelines"
```
-### Specify a branch for multi-project pipelines
-
-You can specify a branch name for a multi-project pipeline to use. GitLab uses
-the commit on the head of the branch to create the downstream pipeline:
-
-```yaml
-rspec:
- stage: test
- script: bundle exec rspec
-
-staging:
- stage: deploy
- trigger:
- project: my/deployment
- branch: stable-11-2
-```
-
-Use:
-
-- The `project` keyword to specify the full path to a downstream project.
- In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/367660), variable expansion is
- supported.
-- The `branch` keyword to specify the name of a branch in the project specified by `project`.
- In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/10126), variable expansion is
- supported.
-
### Use a child pipeline configuration file in a different project
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/205157) in GitLab 13.5.
-You can use [`include:file`](../yaml/index.md#includefile) to trigger child pipelines
-with a configuration file in a different project:
+You can use [`include:project`](../yaml/index.md#includeproject) in a trigger job
+to trigger child pipelines with a configuration file in a different project:
```yaml
microservice_a:
@@ -208,8 +179,6 @@ microservice_a:
### Dynamic child pipelines
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35632) in GitLab 12.9.
-
You can trigger a child pipeline from a YAML file generated in a job, instead of a
static file saved in your project. This technique can be very powerful for generating pipelines
targeting content that changed or to build a matrix of targets and architectures.
@@ -231,43 +200,42 @@ To trigger a child pipeline from a dynamically generated configuration file:
1. Generate the configuration file in a job and save it as an [artifact](../yaml/index.md#artifactspaths):
- ```yaml
- generate-config:
- stage: build
- script: generate-ci-config > generated-config.yml
- artifacts:
- paths:
- - generated-config.yml
- ```
+ ```yaml
+ generate-config:
+ stage: build
+ script: generate-ci-config > generated-config.yml
+ artifacts:
+ paths:
+ - generated-config.yml
+ ```
1. Configure the trigger job to run after the job that generated the configuration file,
and set `include: artifact` to the generated artifact:
- ```yaml
- child-pipeline:
- stage: test
- trigger:
- include:
- - artifact: generated-config.yml
- job: generate-config
- ```
+ ```yaml
+ child-pipeline:
+ stage: test
+ trigger:
+ include:
+ - artifact: generated-config.yml
+ job: generate-config
+ ```
-In this example, `generated-config.yml` is extracted from the artifacts and used as the configuration
-for triggering the child pipeline.
+In this example, GitLab retrieves `generated-config.yml` and triggers a child pipeline
+with the CI/CD configuration in that file.
The artifact path is parsed by GitLab, not the runner, so the path must match the
syntax for the OS running GitLab. If GitLab is running on Linux but using a Windows
runner for testing, the path separator for the trigger job is `/`. Other CI/CD
-configuration for jobs that use the Windows runner, like scripts, use `\`.
+configuration for jobs that use the Windows runner, like scripts, use <code>&#92;</code>.
### Run child pipelines with merge request pipelines
To trigger a child pipeline as a [merge request pipeline](merge_request_pipelines.md):
-1. Set the trigger job to run on merge requests:
+1. Set the trigger job to run on merge requests in the parent pipeline's configuration file:
```yaml
- # parent .gitlab-ci.yml
microservice_a:
trigger:
include: path/to/microservice_a.yml
@@ -275,45 +243,50 @@ To trigger a child pipeline as a [merge request pipeline](merge_request_pipeline
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
```
-1. Configure the child pipeline jobs to run in merge request pipelines:
+1. Configure the child pipeline jobs to run in merge request pipelines with [`rules`](../yaml/index.md#rules)
+ or [`workflow:rules`](../yaml/index.md#workflowrules). For example, with `rules`
+ in a child pipeline's configuration file:
- - With [`workflow:rules`](../yaml/index.md#workflowrules):
+ ```yaml
+ job1:
+ script: ...
+ rules:
+ - if: $CI_PIPELINE_SOURCE == "merge_request_event"
- ```yaml
- # child path/to/microservice_a.yml
- workflow:
- rules:
- - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+ job2:
+ script: ...
+ rules:
+ - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+ ```
- job1:
- script: ...
+### Specify a branch for multi-project pipelines
- job2:
- script: ...
- ```
+You can specify the branch to use when triggering a multi-project pipeline. GitLab uses
+the commit on the head of the branch to create the downstream pipeline. For example:
- - By configuring [rules](../yaml/index.md#rules) for each job:
+```yaml
+staging:
+ stage: deploy
+ trigger:
+ project: my/deployment
+ branch: stable-11-2
+```
- ```yaml
- # child path/to/microservice_a.yml
- job1:
- script: ...
- rules:
- - if: $CI_PIPELINE_SOURCE == "merge_request_event"
+Use:
- job2:
- script: ...
- rules:
- - if: $CI_PIPELINE_SOURCE == "merge_request_event"
- ```
+- The `project` keyword to specify the full path to the downstream project.
+ In [GitLab 15.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/367660),
+ you can use [variable expansion](../variables/where_variables_can_be_used.md#gitlab-ciyml-file).
+- The `branch` keyword to specify the name of a branch or [tag](../../topics/git/tags.md)
+ in the project specified by `project`. You can use variable expansion.
## Trigger a multi-project pipeline by using the API
You can use the [CI/CD job token (`CI_JOB_TOKEN`)](../jobs/ci_job_token.md) with the
[pipeline trigger API endpoint](../../api/pipeline_triggers.md#trigger-a-pipeline-with-a-token)
-to trigger multi-project pipelines from a CI/CD job. GitLab recognizes the source of the job token
-and marks the pipelines as related. In the pipeline graph, the relationships are displayed
-as inbound and outbound connections for upstream and downstream pipeline dependencies.
+to trigger multi-project pipelines from inside a CI/CD job. GitLab sets pipelines triggered
+with a job token as downstream pipelines of the pipeline that contains the job that
+made the API call.
For example:
@@ -357,27 +330,27 @@ To cancel a downstream pipeline that is still running, select **Cancel** (**{can
### Mirror the status of a downstream pipeline in the trigger job
-You can mirror the pipeline status from the triggered pipeline to the source trigger job
+You can mirror the status of the downstream pipeline in the trigger job
by using [`strategy: depend`](../yaml/index.md#triggerstrategy):
::Tabs
-:::TabTitle Multi-project pipeline
+:::TabTitle Parent-child pipeline
```yaml
trigger_job:
trigger:
- project: my/project
+ include:
+ - local: path/to/child-pipeline.yml
strategy: depend
```
-:::TabTitle Parent-child pipeline
+:::TabTitle Multi-project pipeline
```yaml
trigger_job:
trigger:
- include:
- - local: path/to/child-pipeline.yml
+ project: my/project
strategy: depend
```
@@ -385,7 +358,7 @@ trigger_job:
### View multi-project pipelines in pipeline graphs **(PREMIUM)**
-When you trigger a multi-project pipeline, the downstream pipeline displays
+After you trigger a multi-project pipeline, the downstream pipeline displays
to the right of the [pipeline graph](index.md#visualize-pipelines).
![Multi-project pipeline graph](img/multi_project_pipeline_graph_v14_3.png)
@@ -395,12 +368,13 @@ displays to the right of the mini graph.
![Multi-project pipeline mini graph](img/pipeline_mini_graph_v15_0.png)
-## Pass artifacts to a downstream pipeline
+## Fetch artifacts from an upstream pipeline
-You can pass artifacts to a downstream pipeline by using [`needs:project`](../yaml/index.md#needsproject).
+Use [`needs:project`](../yaml/index.md#needsproject) to fetch artifacts from an
+upstream pipeline:
-1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword.
-1. Trigger the downstream pipeline with a trigger job:
+1. In the upstream pipeline, save the artifacts in a job with the [`artifacts`](../yaml/index.md#artifacts)
+ keyword, then trigger the downstream pipeline with a trigger job:
```yaml
build_artifacts:
@@ -416,9 +390,7 @@ You can pass artifacts to a downstream pipeline by using [`needs:project`](../ya
trigger: my/downstream_project
```
-1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline
- by using `needs:project`. Set `job` to the job in the upstream pipeline to fetch artifacts from,
- `ref` to the branch, and `artifacts: true`.
+1. Use `needs:project` in a job in the downstream pipeline to fetch the artifacts.
```yaml
test:
@@ -432,22 +404,27 @@ You can pass artifacts to a downstream pipeline by using [`needs:project`](../ya
artifacts: true
```
-### Pass artifacts from a Merge Request pipeline
+ Set:
-When you use `needs:project` to [pass artifacts to a downstream pipeline](#pass-artifacts-to-a-downstream-pipeline),
+ - `job` to the job in the upstream pipeline that created the artifacts.
+ - `ref` to the branch.
+ - `artifacts` to `true`.
+
+### Fetch artifacts from an upstream merge request pipeline
+
+When you use `needs:project` to [pass artifacts to a downstream pipeline](#fetch-artifacts-from-an-upstream-pipeline),
the `ref` value is usually a branch name, like `main` or `development`.
-For merge request pipelines, the `ref` value is in the form of `refs/merge-requests/<id>/head`,
+For [merge request pipelines](merge_request_pipelines.md), the `ref` value is in the form of `refs/merge-requests/<id>/head`,
where `id` is the merge request ID. You can retrieve this ref with the [`CI_MERGE_REQUEST_REF_PATH`](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines)
CI/CD variable. Do not use a branch name as the `ref` with merge request pipelines,
because the downstream pipeline attempts to fetch artifacts from the latest branch pipeline.
To fetch the artifacts from the upstream `merge request` pipeline instead of the `branch` pipeline,
-pass this variable to the downstream pipeline using variable inheritance:
+pass `CI_MERGE_REQUEST_REF_PATH` to the downstream pipeline using [variable inheritance](#pass-yaml-defined-cicd-variables):
1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword.
-1. In the job that triggers the downstream pipeline, pass the `$CI_MERGE_REQUEST_REF_PATH` variable by using
- [variable inheritance](#pass-yaml-defined-cicd-variables):
+1. In the job that triggers the downstream pipeline, pass the `$CI_MERGE_REQUEST_REF_PATH` variable:
```yaml
build_artifacts:
@@ -467,8 +444,7 @@ pass this variable to the downstream pipeline using variable inheritance:
```
1. In a job in the downstream pipeline, fetch the artifacts from the upstream pipeline
- by using `needs:project`. Set the `ref` to the `UPSTREAM_REF` variable, and `job`
- to the job in the upstream pipeline to fetch artifacts from:
+ by using `needs:project` and the passed variable as the `ref`:
```yaml
test:
@@ -482,86 +458,137 @@ pass this variable to the downstream pipeline using variable inheritance:
artifacts: true
```
-This method works for fetching artifacts from a regular merge request parent pipeline,
-but fetching artifacts from [merge results](merged_results_pipelines.md) pipelines is not supported.
+You can use this method to fetch artifacts from upstream merge request pipeline,
+but not from [merge results pipelines](merged_results_pipelines.md).
## Pass CI/CD variables to a downstream pipeline
-You can pass CI/CD variables to a downstream pipeline with a few different methods,
-based on where the variable is created or defined.
+You can pass [CI/CD variables](../variables/index.md) to a downstream pipeline with
+a few different methods, based on where the variable is created or defined.
### Pass YAML-defined CI/CD variables
-You can use the `variables` keyword to pass CI/CD variables to a downstream pipeline,
-just like you would for any other job.
+You can use the `variables` keyword to pass CI/CD variables to a downstream pipeline.
-For example, in a [multi-project pipeline](#multi-project-pipelines):
+For example:
+
+::Tabs
+
+:::TabTitle Parent-child pipeline
```yaml
-rspec:
- stage: test
- script: bundle exec rspec
+variables:
+ VERSION: "1.0.0"
staging:
variables:
ENVIRONMENT: staging
stage: deploy
- trigger: my/deployment
+ trigger:
+ include:
+ - local: path/to/child-pipeline.yml
```
-The `ENVIRONMENT` variable is passed to every job defined in a downstream
-pipeline. It is available as a variable when GitLab Runner picks a job.
-
-In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline
-that is created when the `trigger-downstream` job is queued. This behavior is because `trigger-downstream`
-job inherits variables declared in [global `variables`](../yaml/index.md#variables) blocks,
-and then GitLab passes these variables to the downstream pipeline.
+:::TabTitle Multi-project pipeline
```yaml
variables:
- MY_VARIABLE: my-value
+ VERSION: "1.0.0"
-trigger-downstream:
+staging:
variables:
- ENVIRONMENT: something
- trigger: my/project
+ ENVIRONMENT: staging
+ stage: deploy
+ trigger: my-group/my-deployment-project
```
+::EndTabs
+
+The `ENVIRONMENT` variable is available in every job defined in the downstream pipeline.
+
+The `VERSION` global variable is also available in the downstream pipeline, because
+all jobs in a pipeline, including trigger jobs, inherit [global `variables`](../yaml/index.md#variables).
+
#### Prevent global variables from being passed
-You can stop global variables from reaching the downstream pipeline by using the [`inherit:variables` keyword](../yaml/index.md#inheritvariables).
-For example, in a [multi-project pipeline](#multi-project-pipelines):
+You can stop global CI/CD variables from reaching the downstream pipeline with
+[`inherit:variables:false`](../yaml/index.md#inheritvariables).
+
+For example:
+
+::Tabs
+
+:::TabTitle Parent-child pipeline
```yaml
variables:
- MY_GLOBAL_VAR: value
+ GLOBAL_VAR: value
-trigger-downstream:
+trigger-job:
inherit:
variables: false
variables:
- MY_LOCAL_VAR: value
- trigger: my/project
+ JOB_VAR: value
+ trigger:
+ include:
+ - local: path/to/child-pipeline.yml
```
-In this example, the `MY_GLOBAL_VAR` variable is not available in the triggered pipeline.
+:::TabTitle Multi-project pipeline
+
+```yaml
+variables:
+ GLOBAL_VAR: value
+
+trigger-job:
+ inherit:
+ variables: false
+ variables:
+ JOB_VAR: value
+ trigger: my-group/my-project
+```
+
+::EndTabs
+
+The `GLOBAL_VAR` variable is not available in the triggered pipeline, but `JOB_VAR`
+is available.
### Pass a predefined variable
-You might want to pass some information about the upstream pipeline using predefined variables.
-To do that, you can use interpolation to pass any variable. For example,
-in a [multi-project pipeline](#multi-project-pipelines):
+To pass information about the upstream pipeline using [predefined CI/CD variables](../variables/predefined_variables.md).
+use interpolation. Save the predefined variable as a new job variable in the trigger
+job, which is passed to the downstream pipeline. For example:
+
+::Tabs
+
+:::TabTitle Parent-child pipeline
+
+```yaml
+trigger-job:
+ variables:
+ PARENT_BRANCH: $CI_COMMIT_REF_NAME
+ trigger:
+ include:
+ - local: path/to/child-pipeline.yml
+```
+
+:::TabTitle Multi-project pipeline
```yaml
-downstream-job:
+trigger-job:
variables:
UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME
- trigger: my/project
+ trigger: my-group/my-project
```
-In this scenario, the `UPSTREAM_BRANCH` variable with the value of the upstream pipeline's
-`$CI_COMMIT_REF_NAME` is passed to `downstream-job`. It is available in the
-context of all downstream builds.
+::EndTabs
+
+The `UPSTREAM_BRANCH` variable, which contains the value of the upstream pipeline's `$CI_COMMIT_REF_NAME`
+predefined CI/CD variable, is available in the downstream pipeline.
+
+Do not use this method to pass [masked variables](../variables/index.md#mask-a-cicd-variable)
+to a multi-project pipeline. The CI/CD masking configuration is not passed to the
+downstream pipeline and the variable could be unmasked in job logs in the downstream project.
You cannot use this method to forward [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables)
to a downstream pipeline, as they are not available in trigger jobs.
@@ -623,3 +650,10 @@ With multi-project pipelines, the trigger job fails and does not create the down
- The downstream pipeline targets a protected branch and the user does not have permission
to run pipelines against the protected branch. See [pipeline security for protected branches](index.md#pipeline-security-on-protected-branches)
for more information.
+
+### `Ref is ambiguous`
+
+You cannot trigger a multi-project pipeline with a tag when a branch exists with the same
+name. The downstream pipeline fails to create with the error: `downstream pipeline can not be created, Ref is ambiguous`.
+
+Only trigger multi-project pipelines with tag names that do not match branch names.
diff --git a/doc/ci/pipelines/img/merge_train_cancel_v12_0.png b/doc/ci/pipelines/img/merge_train_cancel_v12_0.png
deleted file mode 100644
index d7720ac1143..00000000000
--- a/doc/ci/pipelines/img/merge_train_cancel_v12_0.png
+++ /dev/null
Binary files differ
diff --git a/doc/ci/pipelines/img/merge_train_failure.png b/doc/ci/pipelines/img/merge_train_failure.png
deleted file mode 100644
index 8a795fff432..00000000000
--- a/doc/ci/pipelines/img/merge_train_failure.png
+++ /dev/null
Binary files differ
diff --git a/doc/ci/pipelines/img/merge_train_immediate_merge_v12_6.png b/doc/ci/pipelines/img/merge_train_immediate_merge_v12_6.png
deleted file mode 100644
index 7b903716a3d..00000000000
--- a/doc/ci/pipelines/img/merge_train_immediate_merge_v12_6.png
+++ /dev/null
Binary files differ
diff --git a/doc/ci/pipelines/img/merge_train_position_v12_0.png b/doc/ci/pipelines/img/merge_train_position_v12_0.png
deleted file mode 100644
index ec4b157d428..00000000000
--- a/doc/ci/pipelines/img/merge_train_position_v12_0.png
+++ /dev/null
Binary files differ
diff --git a/doc/ci/pipelines/img/merge_train_start_v12_0.png b/doc/ci/pipelines/img/merge_train_start_v12_0.png
deleted file mode 100644
index a4d0c8cf0e6..00000000000
--- a/doc/ci/pipelines/img/merge_train_start_v12_0.png
+++ /dev/null
Binary files differ
diff --git a/doc/ci/pipelines/img/merge_train_start_when_pipeline_succeeds_v12_0.png b/doc/ci/pipelines/img/merge_train_start_when_pipeline_succeeds_v12_0.png
deleted file mode 100644
index 45762b8e85e..00000000000
--- a/doc/ci/pipelines/img/merge_train_start_when_pipeline_succeeds_v12_0.png
+++ /dev/null
Binary files differ
diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md
index ed0583e0872..f1ca8afa62c 100644
--- a/doc/ci/pipelines/index.md
+++ b/doc/ci/pipelines/index.md
@@ -80,7 +80,7 @@ You can also configure specific aspects of your pipelines through the GitLab UI.
### Ref specs for runners
When a runner picks a pipeline job, GitLab provides that job's metadata. This includes the [Git refspecs](https://git-scm.com/book/en/v2/Git-Internals-The-Refspec),
-which indicate which ref (branch, tag, and so on) and commit (SHA1) are checked out from your
+which indicate which ref (such as branch or tag) and commit (SHA1) are checked out from your
project repository.
This table lists the refspecs injected for each pipeline type:
@@ -158,7 +158,7 @@ The pipeline now executes the jobs as configured.
You can use the [`description` and `value`](../yaml/index.md#variablesdescription)
keywords to define [pipeline-level (global) variables](../variables/index.md#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file)
that are prefilled when running a pipeline manually. Use the description to explain
-what the variable is used for, what the acceptable values are, and so on.
+information such as what the variable is used for, and what the acceptable values are.
Job-level variables cannot be pre-filled.
@@ -260,7 +260,7 @@ pipelines.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24851) in GitLab 12.7.
Users with the Owner role for a project can delete a pipeline
-by clicking on the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details**
+by selecting the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details**
page, then selecting **Delete**.
![Pipeline Delete](img/pipeline-delete.png)
@@ -301,6 +301,9 @@ preserving deployment keys and other credentials from being unintentionally
accessed. To ensure that jobs intended to be executed on protected
runners do not use regular runners, they must be tagged accordingly.
+Review the [deployment safety](../environments/deployment_safety.md)
+page for additional security recommendations for securing your pipelines.
+
## Trigger a pipeline when an upstream project is rebuilt **(PREMIUM)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9045) in GitLab 12.8.
diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md
index 33a9069240b..cc26ede5f6e 100644
--- a/doc/ci/pipelines/job_artifacts.md
+++ b/doc/ci/pipelines/job_artifacts.md
@@ -309,61 +309,50 @@ for [parent and child pipelines](downstream_pipelines.md#parent-child-pipelines)
order from parent to child. For example, if both parent and child pipelines have a
job with the same name, the job artifact from the parent pipeline is returned.
-## Access the latest job artifacts by URL
+## Access the latest job artifacts
-You can download job artifacts from the latest successful pipeline by using a URL.
+You can download job artifacts from the latest successful pipeline by using [the job artifacts API](../../api/job_artifacts.md).
+You cannot download [artifact reports](../yaml/artifacts_reports.md) with the job artifacts API,
+unless the report is added as a regular artifact with `artifacts:paths`.
-To download the whole artifacts archive:
+### Download the whole artifacts archive for a specific job
-```plaintext
-https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/download?job=<job_name>
-```
+You can download the artifacts archive for a specific job with [the job artifacts API](../../api/job_artifacts.md#download-the-artifacts-archive).
-To download a single file from the artifacts:
+For example, to download the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com:
```plaintext
-https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/raw/<path_to_file>?job=<job_name>
+https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/download?job=build
```
-For example, to download the latest artifacts of the job named `coverage` in
-the `main` branch of the `gitlab` project in the `gitlab-org`
-namespace:
-
-```plaintext
-https://gitlab.com/gitlab-org/gitlab/-/jobs/artifacts/main/download?job=coverage
-```
+Replace `<project-id>` with a valid project ID, found at the top of the project details page.
-To download the file `review/index.html` from the same artifacts:
+### Download a single file from the artifacts
-```plaintext
-https://gitlab.com/gitlab-org/gitlab/-/jobs/artifacts/main/raw/review/index.html?job=coverage
-```
+You can download a specific file from the artifacts archive for a specific job with [the job artifacts API](../../api/job_artifacts.md#download-a-single-artifact-file-by-job-id).
-To browse the latest job artifacts:
+For example, to download the file `review/index.html` from the latest job named `build` in the `main` branch of the `gitlab` project in the `gitlab-org` namespace:
```plaintext
-https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/browse?job=<job_name>
+https://gitlab.com/api/v4/projects/27456355/jobs/artifacts/main/raw/review/index.html?job=build
```
-For example:
+### Browse job artifacts
-```plaintext
-https://gitlab.com/gitlab-org/gitlab/-/jobs/artifacts/main/browse?job=coverage
-```
-
-To download specific files, including HTML files that
-are shown in [GitLab Pages](../../administration/pages/index.md):
+To browse the job artifacts of the latest successful pipeline for a specific job you can use the following URL:
```plaintext
-https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/file/<path>?job=<job_name>
+https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/browse?job=<job_name>
```
-For example, when a job `coverage` creates the artifact `htmlcov/index.html`:
+For example, to browse the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com:
```plaintext
-https://gitlab.com/gitlab-org/gitlab/-/jobs/artifacts/main/file/htmlcov/index.html?job=coverage
+https://gitlab.com/<full-project-path>/-/jobs/artifacts/main/browse?job=build
```
+Replace `<full-project-path>` with a valid project path, you can find it in the URL for your project.
+
## When job artifacts are deleted
See the [`expire_in`](../yaml/index.md#artifactsexpire_in) documentation for information on when
@@ -396,7 +385,24 @@ a project, you can disable this behavior to save space:
You can disable this behavior for all projects on a self-managed instance in the
[instance's CI/CD settings](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines).
-## Troubleshooting job artifacts
+## Troubleshooting
+
+### Job does not retrieve certain artifacts
+
+By default, jobs fetch all artifacts from previous stages, but jobs using `dependencies`
+or `needs` do not fetch artifacts from all jobs by default.
+
+If you use these keywords, artifacts are fetched from only a subset of jobs. Review
+the keyword reference for information on how to fetch artifacts with these keywords:
+
+- [`dependencies`](../yaml/index.md#dependencies)
+- [`needs`](../yaml/index.md#needs)
+- [`needs:artifacts`](../yaml/index.md#needsartifacts)
+
+### Job artifacts using too much disk space
+
+There are a number of potential causes for this.
+[Read more in the job artifacts administration documentation](../../administration/job_artifacts.md#job-artifacts-using-too-much-disk-space).
### Error message `No files to upload`
diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md
index c501d2a7904..370d81dacc4 100644
--- a/doc/ci/pipelines/merge_trains.md
+++ b/doc/ci/pipelines/merge_trains.md
@@ -6,253 +6,245 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Merge trains **(PREMIUM)**
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9186) in GitLab 12.0.
-> - [Squash and merge](../../user/project/merge_requests/squash_and_merge.md) support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13001) in GitLab 12.6.
+Use merge trains to queue merge requests and verify their changes work together before
+they are merged to the target branch.
-For more information about why you might want to use merge trains, read [How starting merge trains improve efficiency for DevOps](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/).
+In projects with frequent merges to the default branch, changes in different merge requests
+might conflict with each other. [Merged results pipelines](merged_results_pipelines.md)
+ensure the changes work with the content in the default branch, but not content
+that others are merging at the same time.
-When [merged results pipelines](merged_results_pipelines.md) are
-enabled, the pipeline jobs run as if the changes from your source branch have already
-been merged into the target branch.
+Merge trains do not work with [Semi-linear history merge requests](../../user/project/merge_requests/methods/index.md#merge-commit-with-semi-linear-history)
+or [fast-forward merge requests](../../user/project/merge_requests/methods/index.md#fast-forward-merge).
-However, the target branch may be changing rapidly. When you're ready to merge,
-if you haven't run the pipeline in a while, the target branch may have already changed.
-Merging now could introduce breaking changes.
+For more information about:
-*Merge trains* can prevent this from happening. A merge train is a queued list of merge
-requests, each waiting to be merged into the target branch.
+- How merge trains work, review the [merge train workflow](#merge-train-workflow).
+- Why you might want to use merge trains, read [How starting merge trains improve efficiency for DevOps](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/).
-Many merge requests can be added to the train. Each merge request runs its own merged results pipeline,
-which includes the changes from all of the other merge requests in *front* of it on the train.
-All the pipelines run in parallel, to save time. The author of the internal merged result commit is always the user that initiated the merge.
+## Merge train workflow
-If the pipeline for a merge request fails, the breaking changes are not merged, and the target
-branch is unaffected. The merge request is removed from the train, and all pipelines behind it restart.
+A merge train starts when there are no merge requests waiting to merge and you
+select [**Start merge train**](#start-a-merge-train). GitLab starts a merge train pipeline
+that verifies that the changes can merge into the default branch. This first pipeline
+is the same as a [merged results pipeline](merged_results_pipelines.md), which runs on
+the changes of the source and target branches combined together. The author of the
+internal merged result commit is the user that initiated the merge.
-If the pipeline for the merge request at the front of the train completes successfully,
-the changes are merged into the target branch, and the other pipelines continue to
-run.
+To queue a second merge request to merge immediately after the first pipeline completes, select
+[**Add to merge train**](#add-a-merge-request-to-a-merge-train) and add it to the train.
+This second merge train pipeline runs on the changes of _both_ merge requests combined with the
+target branch. Similarly, if you add a third merge request, that pipeline runs on the changes
+of all three merge requests merged with the target branch. The pipelines all run in parallel.
-To add a merge request to a merge train, you need [permissions](../../user/permissions.md) to merge or push to the
-target branch.
+Each merge request merges into the target branch only after:
-Each merge train can run a maximum of **twenty** pipelines in parallel.
-If more than twenty merge requests are added to the merge train, the merge requests
-are queued until a slot in the merge train is free. There is no limit to the
-number of merge requests that can be queued.
+- The merge request's pipeline completes successfully.
+- All other merge requests queued before it are merged.
-## Merge train example
+If a merge train pipeline fails, the merge request is not merged. GitLab
+removes that merge request from the merge train, and starts new pipelines for all
+the merge requests that were queued after it.
-Three merge requests (`A`, `B` and `C`) are added to a merge train in order, which
+For example:
+
+Three merge requests (`A`, `B`, and `C`) are added to a merge train in order, which
creates three merged results pipelines that run in parallel:
1. The first pipeline runs on the changes from `A` combined with the target branch.
1. The second pipeline runs on the changes from `A` and `B` combined with the target branch.
1. The third pipeline runs on the changes from `A`, `B`, and `C` combined with the target branch.
-If the pipeline for `B` fails, it is removed from the train. The pipeline for
-`C` restarts with the `A` and `C` changes, but without the `B` changes.
+If the pipeline for `B` fails:
+
+- The first pipeline (`A`) continues to run.
+- `B` is removed from the train.
+- The pipeline for `C` [is cancelled](#automatic-pipeline-cancellation), and a new pipeline
+ starts for the changes from `A` and `C` combined with the target branch (without the `B` changes).
If `A` then completes successfully, it merges into the target branch, and `C` continues
-to run. If more merge requests are added to the train, they now include the `A`
-changes that are included in the target branch, and the `C` changes that are from
-the merge request already in the train.
+to run. Any new merge requests added to the train include the `A` changes now in
+the target branch, and the `C` changes from the merge train.
<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
-Watch this video for a demonstration on
-[how parallel execution of merge trains can prevent commits from breaking the default branch](https://www.youtube.com/watch?v=D4qCqXgZkHQ).
+Watch this video for a demonstration on [how parallel execution of merge trains can prevent commits from breaking the default branch](https://www.youtube.com/watch?v=D4qCqXgZkHQ).
-## Prerequisites
+### Automatic pipeline cancellation
-To enable merge trains:
+GitLab CI/CD detects redundant pipelines, and cancels them to conserve resources.
-- You must have the Maintainer role.
-- You must be using [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) 11.9 or later.
-- Your repository must be a GitLab repository, not an
- [external repository](../ci_cd_for_external_repos/index.md).
+Redundant merge train pipelines happen when:
-Merge trains do not work with [Semi-linear history merge requests](../../user/project/merge_requests/methods/index.md#merge-commit-with-semi-linear-history)
-or [fast-forward merge requests](../../user/project/merge_requests/methods/index.md#fast-forward-merge).
+- The pipeline fails for one of the merge requests in the merge train.
+- You [skip the merge train and merge immediately](#skip-the-merge-train-and-merge-immediately).
+- You [remove a merge request from a merge train](#remove-a-merge-request-from-a-merge-train).
+
+In these cases, GitLab must create new merge train pipelines for some or all of the
+merge requests on the train. The old pipelines were comparing against the previous
+combined changes in the merge train, which are no longer valid, so these old pipelines
+are cancelled.
## Enable merge trains
-To enable merge trains for your project:
+Prerequisites:
+
+- You must have the Maintainer role.
+- Your repository must be a GitLab repository, not an [external repository](../ci_cd_for_external_repos/index.md).
+- Your pipeline must be [configured to use merge request pipelines](merge_request_pipelines.md#prerequisites).
+ Otherwise your merge requests may become stuck in an unresolved state or your pipelines
+ might be dropped.
+
+To enable merge trains:
-1. If you are on a self-managed GitLab instance, ensure the [feature flag](#merge-trains-feature-flag) is set correctly.
-1. [Configure your CI/CD configuration file](merge_request_pipelines.md#prerequisites)
- so that the pipeline or individual jobs run for merge requests.
1. On the top bar, select **Main menu > Projects** and find your project.
1. On the left sidebar, select **Settings > Merge requests**.
1. In the **Merge method** section, verify that **Merge commit** is selected.
-1. In the **Merge options** section, select **Enable merged results pipelines** (if not already selected) and **Enable merge trains**.
+1. In the **Merge options** section:
+ - In GitLab 13.6 and later, select **Enable merged results pipelines** and **Enable merge trains**.
+ - In GitLab 13.5 and earlier, select **Enable merge trains and pipelines for merged results**.
+ Additionally, [a feature flag](#disable-merge-trains-in-gitlab-135-and-earlier)
+ must be set correctly.
1. Select **Save changes**.
-In GitLab 13.5 and earlier, there is only one checkbox, named
-**Enable merge trains and pipelines for merged results**.
+## Start a merge train
-WARNING:
-If you select the checkbox but don't configure your CI/CD to use
-merge request pipelines, your merge requests may become stuck in an
-unresolved state or your pipelines may be dropped.
+Prerequisites:
-## Start a merge train
+- You must have [permissions](../../user/permissions.md) to merge or push to the target branch.
To start a merge train:
1. Visit a merge request.
-1. Select **Start merge train**.
+1. Select:
+ - When no pipeline is running, **Start merge train**.
+ - When a pipeline is running, **Start merge train when pipeline succeeds**.
-![Start merge train](img/merge_train_start_v12_0.png)
+The merge request's merge train status displays under the pipeline widget with a
+message similar to `A new merge train has started and this merge request is the first of the queue.`
Other merge requests can now be added to the train.
## Add a merge request to a merge train
+Prerequisites:
+
+- You must have [permissions](../../user/permissions.md) to merge or push to the target branch.
+
To add a merge request to a merge train:
1. Visit a merge request.
-1. Select **Add to merge train**.
+1. Select:
+ - When no pipeline is running, **Add to merge train**.
+ - When a pipeline is running, **Add to merge train when pipeline succeeds**.
-If pipelines are already running for the merge request, you cannot add the merge request
-to the train. Instead, you can schedule to add the merge request to a merge train **when the latest
-pipeline succeeds**.
+The merge request's merge train status displays under the pipeline widget with a
+message similar to `Added to the merge train. There are 2 merge requests waiting to be merged.`
-![Add to merge train when pipeline succeeds](img/merge_train_start_when_pipeline_succeeds_v12_0.png)
+Each merge train can run a maximum of twenty pipelines in parallel. If you add more than
+twenty merge requests to the merge train, the extra merge requests are queued, waiting
+for pipelines to complete. There is no limit to the number of queued merge requests
+waiting to join the merge train.
## Remove a merge request from a merge train
-1. Visit a merge request.
-1. Select **Remove from merge train**.
-
-![Cancel merge train](img/merge_train_cancel_v12_0.png)
+To remove a merge request from a merge train, select **Remove from merge train**.
+You can add the merge request to a merge train again later.
-If you want to add the merge request to a merge train again later, you can.
+When you remove a merge request from a merge train:
-## View a merge request's current position on the merge train
+- All pipelines for merge requests queued after the removed merge request restart.
+- Redundant pipelines [are cancelled](#automatic-pipeline-cancellation).
-After a merge request has been added to the merge train, the merge request's
-current position is displayed under the pipeline widget:
+## Skip the merge train and merge immediately
-![Merge train position indicator](img/merge_train_position_v12_0.png)
+If you have a high-priority merge request, like a critical patch that must
+be merged urgently, select **Merge Immediately**.
-## Immediately merge a merge request with a merge train
+When you merge a merge request immediately:
-If you have a high-priority merge request (for example, a critical patch) that must
-be merged urgently, you can bypass the merge train by using the **Merge Immediately** option.
-This is the fastest option to get the change merged into the target branch.
-
-![Merge Immediately](img/merge_train_immediate_merge_v12_6.png)
+- The current merge train is recreated.
+- All pipelines restart.
+- Redundant pipelines [are cancelled](#automatic-pipeline-cancellation).
WARNING:
-Each time you merge a merge request immediately, the current merge train is recreated,
-all pipelines restart, and [redundant pipelines are cancelled](#automatic-pipeline-cancellation).
+Merging immediately can use a lot of CI/CD resources. Use this option
+only in critical situations.
-### Automatic pipeline cancellation
+## Disable merge trains in GitLab 13.5 and earlier **(PREMIUM SELF)**
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12996) in GitLab 12.3.
+In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/244831),
+you can [enable or disable merge trains in the project settings](#enable-merge-trains).
-GitLab CI/CD can detect the presence of redundant pipelines, and cancels them
-to conserve CI resources.
+In GitLab 13.5 and earlier, merge trains are automatically enabled when
+[merged results pipelines](merged_results_pipelines.md) are enabled.
+To use merged results pipelines but not merge trains, enable the `disable_merge_trains`
+[feature flag](../../user/feature_flags.md).
-When a user merges a merge request immediately in an ongoing merge
-train, the train is reconstructed, because it recreates the expected
-post-merge commit and pipeline. In this case, the merge train may already
-have pipelines running against the previous expected post-merge commit.
-These pipelines are considered redundant and are automatically
-canceled.
+[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md)
+can enable the feature flag to disable merge trains:
-## Troubleshooting
+```ruby
+Feature.enable(:disable_merge_trains)
+```
-### Merge request dropped from the merge train immediately
+After you enable this feature flag, GitLab cancels existing merge trains and removes
+the **Start/Add to merge train** option from merge requests.
-If a merge request is not mergeable (for example, it's a draft merge request or it has a merge
-conflict), the merge train drops your merge request automatically.
+To disable the feature flag, which enables merge trains again:
-In these cases, the reason for dropping the merge request is in the **system notes**.
+```ruby
+Feature.disable(:disable_merge_trains)
+```
-To check the reason:
+## Troubleshooting
-1. Open the merge request that was dropped from the merge train.
-1. Select the **Discussion** tab.
-1. Find a system note that includes either:
- - **... removed this merge request from the merge train because ...**
- - **... aborted this merge request from the merge train because ...**
+### Merge request dropped from the merge train
-The reason is given in the text after the **because ...** phrase.
+If a merge request becomes unmergeable while a merge train pipeline is running,
+the merge train drops your merge request automatically. For example, this could be caused by:
-![Merge train failure](img/merge_train_failure.png)
+- Changing the merge request to a [draft](../../user/project/merge_requests/drafts.md).
+- A merge conflict.
+- A new conversation thread that is unresolved, when [all threads must be resolved](../../user/discussions/index.md#prevent-merge-unless-all-threads-are-resolved)
+ is enabled.
-### Merge When Pipeline Succeeds cannot be chosen
+You can find reason the merge request was dropped from the merge train in the system
+notes. Check the **Activity** section in the **Overview** tab for a message similar to:
+`User removed this merge request from the merge train because ...`
-[Merge When Pipeline Succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md)
-is currently unavailable when merge trains are enabled.
+### Cannot use merge when pipeline succeeds
-See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/12267)
+You cannot use [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md)
+when merge trains are enabled. See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/12267)
for more information.
-### Merge train pipeline cannot be retried
+### Cannot retry merge train pipeline cannot
When a merge train pipeline fails, the merge request is dropped from the train and the pipeline can't be retried.
Merge train pipelines run on the merged result of the changes in the merge request and
-the changes from other merge requests already on the train. If the merge request is dropped from the train,
+changes from other merge requests already on the train. If the merge request is dropped from the train,
the merged result is out of date and the pipeline can't be retried.
-Instead, you should [add the merge request to the train](#add-a-merge-request-to-a-merge-train)
-again, which triggers a new pipeline.
-
-If a job only fails intermittently, you can try using the [`retry`](../yaml/index.md#retry)
-keyword in the `.gitlab-ci.yml` file to have the job retried before the pipeline completes.
-If it succeeds after a retry, the merge request is not removed from the merge train.
+You can:
-### Unable to add to merge train with message "The pipeline for this merge request failed."
+- [Add the merge request to the train](#add-a-merge-request-to-a-merge-train) again,
+ which triggers a new pipeline.
+- Add the [`retry`](../yaml/index.md#retry) keyword to the job if it fails intermittently.
+ If it succeeds after a retry, the merge request is not removed from the merge train.
-Sometimes the **Start/Add to merge train** button is not available and the merge request says,
-"The pipeline for this merge request failed. Please retry the job or push a new commit to fix the failure."
+### Unable to add to the merge train
-This issue occurs when [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge)
-is enabled in **Settings > General > Merge requests**. This option requires that you
-run a new successful pipeline before you can re-add a merge request to a merge train.
+When [**Pipelines must succeed**](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#require-a-successful-pipeline-for-merge)
+is enabled, but the latest pipeline failed:
-Merge trains ensure that each pipeline has succeeded before a merge happens, so
-you can:
+- The **Start/Add to merge train** option is not available.
+- The merge request displays `The pipeline for this merge request failed. Please retry the job or push a new commit to fix the failure.`
-- Clear the **Pipelines must succeed** checkbox.
-- Select the **Enable merged results pipelines** and **Enable merge trains** checkboxes.
+Before you can re-add a merge request to a merge train, you can try to:
- In GitLab 13.5 and earlier, there is only one checkbox, named
- **Enable merge trains and pipelines for merged results**.
-
-If you want to keep the **Pipelines must succeed** option selected along with merge
-trains, create a new merged results pipeline when this error occurs:
-
-1. On the **Pipelines** tab, select **Run pipeline**.
-1. Select **Start/Add to merge train when pipeline succeeds**.
+- Retry the failed job. If it passes, and no other jobs failed, the pipeline is marked as successful.
+- Rerun the whole pipeline. On the **Pipelines** tab, select **Run pipeline**.
+- Push a new commit that fixes the issue, which also triggers a new pipeline.
See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/35135)
for more information.
-
-### Merge trains feature flag **(PREMIUM SELF)**
-
-In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/244831),
-you can [enable or disable merge trains in the project settings](#enable-merge-trains).
-
-In GitLab 13.5 and earlier, merge trains are automatically enabled when
-[merged results pipelines](merged_results_pipelines.md) are enabled.
-To use merged results pipelines without using merge trains, you can enable a
-[feature flag](../../user/feature_flags.md) that blocks the merge trains feature.
-
-[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md)
-can enable the feature flag to disable merge trains:
-
-```ruby
-Feature.enable(:disable_merge_trains)
-```
-
-After you enable this feature flag, all existing merge trains are cancelled and
-the **Start/Add to merge train** button no longer appears in merge requests.
-
-To disable the feature flag, and enable merge trains again:
-
-```ruby
-Feature.disable(:disable_merge_trains)
-```
diff --git a/doc/ci/pipelines/pipeline_architectures.md b/doc/ci/pipelines/pipeline_architectures.md
index e36eb24055f..1e4d32fc331 100644
--- a/doc/ci/pipelines/pipeline_architectures.md
+++ b/doc/ci/pipelines/pipeline_architectures.md
@@ -23,13 +23,18 @@ own advantages. These methods can be mixed and matched if needed:
- [Multi-project pipelines](downstream_pipelines.md#multi-project-pipelines): Good for larger products that require cross-project interdependencies,
like those with a [microservices architecture](https://about.gitlab.com/blog/2016/08/16/trends-in-version-control-land-microservices/).
+ For example, you might deploy your web application from three different GitLab projects.
+ With multi-project pipelines you can trigger a pipeline in each project, where each
+ has its own build, test, and deploy process. You can visualize the connected pipelines
+ in one place, including all cross-project interdependencies.
+
<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
For an overview, see the [Multi-project pipelines demo](https://www.youtube.com/watch?v=g_PIwBM1J84).
## Basic Pipelines
This is the simplest pipeline in GitLab. It runs everything in the build stage concurrently,
-and once all of those finish, it runs everything in the test stage the same way, and so on.
+and once all of those finish, it runs everything in the test and subsequent stages the same way.
It's not the most efficient, and if you have lots of steps it can grow quite complex, but it's
easier to maintain:
diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md
index aa2f074693a..276a03cb480 100644
--- a/doc/ci/pipelines/pipeline_efficiency.md
+++ b/doc/ci/pipelines/pipeline_efficiency.md
@@ -225,7 +225,7 @@ has more information about building efficient Docker images.
Methods to reduce Docker image size:
- Use a small base image, for example `debian-slim`.
-- Do not install convenience tools like vim, curl, and so on, if they aren't strictly needed.
+- Do not install convenience tools such as vim or curl if they aren't strictly needed.
- Create a dedicated development image.
- Disable man pages and docs installed by packages to save space.
- Reduce the `RUN` layers and combine software installation steps.
diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md
index 0eeb0eada87..02728d5e1e0 100644
--- a/doc/ci/pipelines/schedules.md
+++ b/doc/ci/pipelines/schedules.md
@@ -95,6 +95,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md
index d7155004359..a7f3cfb59d3 100644
--- a/doc/ci/pipelines/settings.md
+++ b/doc/ci/pipelines/settings.md
@@ -82,9 +82,10 @@ You can set pending or running pipelines to cancel automatically when a new pipe
Use the [`interruptible`](../yaml/index.md#interruptible) keyword to indicate if a
running job can be cancelled before it completes.
-## Skip outdated deployment jobs
+## Prevent outdated deployment jobs
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25276) in GitLab 12.9.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25276) in GitLab 12.9.
+> - In GitLab 15.5, the behavior was [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/363328) to prevent outdated job runs.
Your project may have multiple concurrent deployment jobs that are
scheduled to run in the same time frame.
@@ -97,29 +98,10 @@ To avoid this scenario:
1. On the top bar, select **Main menu > Projects** and find your project.
1. On the left sidebar, select **Settings > CI/CD**.
1. Expand **General pipelines**.
-1. Select the **Skip outdated deployment jobs** checkbox.
+1. Select the **Prevent outdated deployment jobs** checkbox.
1. Select **Save changes**.
-When a new deployment starts, older deployment jobs are skipped. Skipped jobs are labeled:
-
-- `forward deployment failure` in the pipeline view.
-- `The deployment job is older than the previously succeeded deployment job, and therefore cannot be run`
- when viewing the completed job.
-
-Job age is determined by the job start time, not the commit time, so a newer commit
-can be skipped in some circumstances.
-
-For more information, see [Deployment safety](../environments/deployment_safety.md).
-
-## Retry outdated jobs
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211339) in GitLab 13.6.
-
-A deployment job can fail because a newer one has run. If you retry the failed deployment job, the
-environment could be overwritten with older source code. If you select **Retry**, a modal warns you
-about this and asks for confirmation.
-
-For more information, see [Deployment safety](../environments/deployment_safety.md).
+For more information, see [Deployment safety](../environments/deployment_safety.md#prevent-outdated-deployment-jobs).
## Specify a custom CI/CD configuration file
@@ -246,53 +228,6 @@ averaged.
To add test coverage results to a merge request using the project's `.gitlab-ci.yml` file, provide a regular expression
using the [`coverage`](../yaml/index.md#coverage) keyword.
-<!-- start_remove The following content will be removed on remove_date: '2023-08-22' -->
-
-### Add test coverage results using project settings (removed)
-
-> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 14.8. Replaced by [`coverage` keyword](../yaml/index.md#coverage).
-> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0.
-
-This feature is in its end-of-life process. It was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633)
-in GitLab 14.8. The feature is [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0.
-
-To migrate from a project setting to the `coverage` keyword, add the [former project setting](#locate-former-project-setting)
-to a CI/CD job. For example:
-
-- A Go test coverage project setting: `coverage: \d+.\d+% of statements`.
-- A CI/CD job with `coverage` keyword setting:
-
- ```yaml
- unit-test:
- stage: test
- coverage: '/coverage: \d+.\d+% of statements/'
- script:
- - go test -cover
- ```
-
-The `.gitlab-ci.yml` job [`coverage`](../yaml/index.md#coverage) keyword must be:
-
-- A regular expression starts and ends with the `/` character.
-- Defined as single-quoted string.
-
-You can verify correct syntax using the [pipeline editor](../pipeline_editor/index.md).
-
-#### Locate former project setting
-
-To migrate from the project coverage setting to the `coverage` keyword, use the
-regular expression displayed in the settings. Available in GitLab 14.10 and earlier:
-
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Settings > CI/CD**.
-1. Expand **General pipelines**.
-
-The regular expression you need is in the **Test coverage parsing** field.
-
-If you need to retrieve the project coverage setting from many projects, you can
-[use the API to programmatically retrieve the setting](https://gitlab.com/gitlab-org/gitlab/-/issues/17633#note_945941397).
-
-<!-- end_remove -->
-
### Test coverage examples
Use this regex for commonly used test tools.
@@ -302,6 +237,7 @@ Use this regex for commonly used test tools.
- Simplecov (Ruby). Example: `/\(\d+.\d+\%\) covered/`.
- pytest-cov (Python). Example: `/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/`.
- Scoverage (Scala). Example: `/Statement coverage[A-Za-z\.*]\s*:\s*([^%]+)/`.
+- `pest --coverage --colors=never` (PHP). Example: `/^\s*Cov:\s*\d+\.\d+?%$/`.
- `phpunit --coverage-text --colors=never` (PHP). Example: `/^\s*Lines:\s*\d+.\d+\%/`.
- gcovr (C/C++). Example: `/^TOTAL.*\s+(\d+\%)$/`.
- `tap --coverage-report=text-summary` (NodeJS). Example: `/^Statements\s*:\s*([^%]+)/`.
@@ -437,12 +373,12 @@ https://gitlab.example.com/<namespace>/<project>/badges/<branch>/coverage.svg
```
To get the coverage report from a specific job, add
-the `job=coverage_job_name` parameter to the URL. For example, the following
-Markdown code embeds the test coverage report badge of the `coverage` job
-in your `README.md`:
+the `job=coverage_job_name` parameter to the URL. For example, you can use code
+similar to the following to add the test coverage report badge of the `coverage` job
+to a Markdown file:
```markdown
-![coverage](https://gitlab.com/gitlab-org/gitlab/badges/main/coverage.svg?job=coverage)
+![coverage](https://gitlab.example.com/<namespace>/<project>/badges/<branch>/coverage.svg?job=coverage)
```
#### Test coverage report badge colors and limits
@@ -527,6 +463,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md
index ad41e4fa88a..8d71f4569e5 100644
--- a/doc/ci/quick_start/index.md
+++ b/doc/ci/quick_start/index.md
@@ -1,50 +1,41 @@
---
stage: Verify
-group: Pipeline Execution
+group: Pipeline Authoring
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
type: reference
---
-# Get started with GitLab CI/CD **(FREE)**
+# Tutorial: Create and run your first GitLab CI/CD pipeline **(FREE)**
-Use this document to get started with [GitLab CI/CD](../index.md).
+This tutorial shows you how to configure and run your first CI/CD pipeline in GitLab.
+
+## Prerequisites
Before you start, make sure you have:
- A project in GitLab that you would like to use CI/CD for.
- The Maintainer or Owner role for the project.
-If you are migrating from another CI/CD tool, view this documentation:
-
-- [Migrate from CircleCI](../migration/circleci.md).
-- [Migrate from Jenkins](../migration/jenkins.md).
+If you don't have a project, you can create a public project for free on <https://gitlab.com>.
-> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i>&nbsp;Watch [First time GitLab & CI/CD](https://www.youtube.com/watch?v=kTNfi5z6Uvk&t=553s). This includes a quick introduction to GitLab, the first steps with CI/CD, building a Go project, running tests, using the CI/CD pipeline editor, detecting secrets and security vulnerabilities and offers more exercises for asynchronous practice.
-> - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i>&nbsp;Watch [Intro to GitLab CI](https://www.youtube.com/watch?v=l5705U8s_nQ&t=358s). This workshop uses the Web IDE to quickly get going with building source code using CI/CD, and run unit tests.
+## Steps
-## CI/CD process overview
-
-To use GitLab CI/CD:
+To create and run your first pipeline:
1. [Ensure you have runners available](#ensure-you-have-runners-available) to run your jobs.
- GitLab SaaS provides runners, so if you're using GitLab.com, you can skip this step.
- If you don't have a runner, [install GitLab Runner](https://docs.gitlab.com/runner/install/)
- and [register a runner](https://docs.gitlab.com/runner/register/) for your instance, project, or group.
+ If you're using GitLab.com, you can skip this step. GitLab.com provides shared runners for you.
+
1. [Create a `.gitlab-ci.yml` file](#create-a-gitlab-ciyml-file)
- at the root of your repository. This file is where you define your CI/CD jobs.
+ at the root of your repository. This file is where you define the CI/CD jobs.
When you commit the file to your repository, the runner runs your jobs.
The job results [are displayed in a pipeline](#view-the-status-of-your-pipeline-and-jobs).
-### Ensure you have runners available
+## Ensure you have runners available
In GitLab, runners are agents that run your CI/CD jobs.
-You might already have runners available for your project, including
-[shared runners](../runners/runners_scope.md), which are
-available to all projects in your GitLab instance.
-
To view available runners:
- Go to **Settings > CI/CD** and expand **Runners**.
@@ -52,34 +43,32 @@ To view available runners:
As long as you have at least one runner that's active, with a green circle next to it,
you have a runner available to process your jobs.
-If no runners are listed on the **Runners** page in the UI, you or an administrator
-must [install GitLab Runner](https://docs.gitlab.com/runner/install/) and
-[register](https://docs.gitlab.com/runner/register/) at least one runner.
+### If you don't have a runner
+
+If you don't have a runner:
+
+1. [Install GitLab Runner](https://docs.gitlab.com/runner/install/) on your local machine.
+1. [Register the runner](https://docs.gitlab.com/runner/register/) for your project.
+ Choose the `shell` executor.
-If you are testing CI/CD, you can install GitLab Runner and register runners on your local machine.
-When your CI/CD jobs run, they run on your local machine.
+When your CI/CD jobs run, in a later step, they will run on your local machine.
-### Create a `.gitlab-ci.yml` file
+## Create a `.gitlab-ci.yml` file
-The `.gitlab-ci.yml` file is a [YAML](https://en.wikipedia.org/wiki/YAML) file where
-you configure specific instructions for GitLab CI/CD.
+Now create a `.gitlab-ci.yml` file. It is a [YAML](https://en.wikipedia.org/wiki/YAML) file where
+you specify instructions for GitLab CI/CD.
In this file, you define:
- The structure and order of jobs that the runner should execute.
- The decisions the runner should make when specific conditions are encountered.
-For example, you might want to run a suite of tests when you commit to
-any branch except the default branch. When you commit to the default branch, you want
-to run the same suite, but also publish your application.
-
-All of this is defined in the `.gitlab-ci.yml` file.
-
To create a `.gitlab-ci.yml` file:
-1. On the left sidebar, select **Project information > Details**.
-1. Above the file list, select the branch you want to commit to,
- select the plus icon, then select **New file**:
+1. On the left sidebar, select **Repository > Files**.
+1. Above the file list, select the branch you want to commit to.
+ If you're not sure, leave `master` or `main`.
+ Then select the plus icon (**{plus}**) and **New file**:
![New file](img/new_file_v13_6.png)
@@ -112,46 +101,50 @@ To create a `.gitlab-ci.yml` file:
environment: production
```
- `$GITLAB_USER_LOGIN` and `$CI_COMMIT_BRANCH` are
- [predefined variables](../variables/predefined_variables.md)
- that populate when the job runs.
+ This example shows four jobs: `build-job`, `test-job1`, `test-job2`, and `deploy-prod`.
+ The comments listed in the `echo` commands are displayed in the UI when you view the jobs.
+ The values for the [predefined variables](../variables/predefined_variables.md)
+ `$GITLAB_USER_LOGIN` and `$CI_COMMIT_BRANCH` are populated when the jobs run.
1. Select **Commit changes**.
-The pipeline starts when the commit is committed.
+The pipeline starts and runs the jobs you defined in the `.gitlab-ci.yml` file.
+
+## View the status of your pipeline and jobs
+
+Now take a look at your pipeline and the jobs within.
+
+1. Go to **CI/CD > Pipelines**. A pipeline with three stages should be displayed:
+
+ ![Three stages](img/three_stages_v13_6.png)
-#### `.gitlab-ci.yml` tips
+1. View a visual representation of your pipeline by selecting the pipeline ID:
-- After you create your first `.gitlab-ci.yml` file, use the [pipeline editor](../pipeline_editor/index.md)
- for all future edits to the file. With the pipeline editor, you can:
- - Edit the pipeline configuration with automatic syntax highlighting and validation.
- - View the [CI/CD configuration visualization](../pipeline_editor/index.md#visualize-ci-configuration),
- a graphical representation of your `.gitlab-ci.yml` file.
-- If you want the runner to [use a Docker container to run the jobs](../docker/using_docker_images.md),
- edit the `.gitlab-ci.yml` file
- to include an image name:
+ ![Pipeline graph](img/pipeline_graph_v13_6.png)
- ```yaml
- default:
- image: ruby:2.7.5
- ```
+1. View details of a job by selecting the job name. For example, `deploy-prod`:
- This command tells the runner to use a Ruby image from Docker Hub
- and to run the jobs in a container that's generated from the image.
+ ![Job details](img/job_details_v13_6.png)
- This process is different than
- [building an application as a Docker container](../docker/using_docker_build.md).
- Your application does not need to be built as a Docker container to
- run CI/CD jobs in Docker containers.
+You have successfully created your first CI/CD pipeline in GitLab. Congratulations!
-- Each job contains scripts and stages:
+Now you can get started customizing your `.gitlab-ci.yml` and defining more advanced jobs.
+
+## `.gitlab-ci.yml` tips
+
+Here are some tips to get started working with the `.gitlab-ci.yml` file.
+
+For the complete `.gitlab-ci.yml` syntax, see [the full `.gitlab-ci.yml` keyword reference](../yaml/index.md).
+
+- Use the [pipeline editor](../pipeline_editor/index.md) to edit your `.gitlab-ci.yml` file.
+- Each job contains a script section and belongs to a stage:
- The [`default`](../yaml/index.md#default) keyword is for
custom defaults, for example with [`before_script`](../yaml/index.md#before_script)
and [`after_script`](../yaml/index.md#after_script).
- [`stage`](../yaml/index.md#stage) describes the sequential execution of jobs.
Jobs in a single stage run in parallel as long as there are available runners.
- - Use [Directed Acyclic Graphs (DAG)](../directed_acyclic_graph/index.md) keywords
- to run jobs out of stage order.
+ - Use the [`needs` keyword](../yaml/index.md#needs) to run jobs out of stage order.
+ This creates a [Directed Acyclic Graph (DAG)](../directed_acyclic_graph/index.md).
- You can set additional configuration to customize how your jobs and stages perform:
- Use the [`rules`](../yaml/index.md#rules) keyword to specify when to run or skip jobs.
The `only` and `except` legacy keywords are still supported, but can't be used
@@ -159,26 +152,10 @@ The pipeline starts when the commit is committed.
- Keep information across jobs and stages persistent in a pipeline with [`cache`](../yaml/index.md#cache)
and [`artifacts`](../yaml/index.md#artifacts). These keywords are ways to store
dependencies and job output, even when using ephemeral runners for each job.
-- For the complete `.gitlab-ci.yml` syntax, see [the full `.gitlab-ci.yml` reference topic](../yaml/index.md).
-
-### View the status of your pipeline and jobs
-
-When you committed your changes, a pipeline started.
-
-To view your pipeline:
-
-- Go to **CI/CD > Pipelines**.
-
- A pipeline with three stages should be displayed:
-
- ![Three stages](img/three_stages_v13_6.png)
-
-- To view a visual representation of your pipeline, select the pipeline ID.
-
- ![Pipeline graph](img/pipeline_graph_v13_6.png)
-
-- To view details of a job, select the job name, for example, `deploy-prod`.
- ![Job details](img/job_details_v13_6.png)
+## Related topics
-If the job status is `stuck`, check to ensure a runner is properly configured for the project.
+- [Follow this guide to migrate from CircleCI](../migration/circleci.md).
+- [Follow this guide to migrate from Jenkins](../migration/jenkins.md).
+- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i>&nbsp;Watch [First time GitLab & CI/CD](https://www.youtube.com/watch?v=kTNfi5z6Uvk&t=553s). This includes a quick introduction to GitLab, the first steps with CI/CD, building a Go project, running tests, using the CI/CD pipeline editor, detecting secrets and security vulnerabilities and offers more exercises for asynchronous practice.
+- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i>&nbsp;Watch [Intro to GitLab CI](https://www.youtube.com/watch?v=l5705U8s_nQ&t=358s). This workshop uses the Web IDE to quickly get going with building source code using CI/CD, and run unit tests.
diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md
index 2803ddba828..b46008abb07 100644
--- a/doc/ci/resource_groups/index.md
+++ b/doc/ci/resource_groups/index.md
@@ -89,7 +89,7 @@ The following modes are supported:
that are sorted by pipeline ID in descending order.
This mode is efficient when you want to ensure that the jobs are executed from the newest pipeline and
- cancel all of the old deploy jobs with the [skip outdated deployment jobs](../environments/deployment_safety.md#skip-outdated-deployment-jobs) feature.
+ prevent all of the old deploy jobs with the [prevent outdated deployment jobs](../environments/deployment_safety.md#prevent-outdated-deployment-jobs) feature.
This is the most efficient option in terms of the pipeline efficiency, but you must ensure that each deployment job is idempotent.
### Change the process mode
@@ -171,7 +171,6 @@ deploy:
include: deploy.gitlab-ci.yml
strategy: depend
resource_group: AWS-production
- environment: production
```
```yaml
diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md
index 2f23ebbfd9f..7a0f43f9ec5 100644
--- a/doc/ci/review_apps/index.md
+++ b/doc/ci/review_apps/index.md
@@ -4,96 +4,96 @@ group: Pipeline Insights
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Review Apps **(FREE)**
+# Review apps **(FREE)**
-Review Apps is a collaboration tool that assists with providing an environment to showcase product changes.
+Review apps are a collaboration tool that provide an environment to showcase product changes.
NOTE:
If you have a Kubernetes cluster, you can automate this feature in your applications
by using [Auto DevOps](../../topics/autodevops/index.md).
-Review Apps:
+Review apps:
- Provide an automatic live preview of changes made in a feature branch by spinning up a dynamic environment for your merge requests.
- Allow designers and product managers to see your changes without needing to check out your branch and run your changes in a sandbox environment.
- Are fully integrated with the [GitLab DevOps LifeCycle](../../index.md#the-entire-devops-lifecycle).
- Allow you to deploy your changes wherever you want.
-![Review Apps Workflow](img/continuous-delivery-review-apps.svg)
+![review apps workflow](img/continuous-delivery-review-apps.svg)
In the previous example:
-- A Review App is built every time a commit is pushed to `topic branch`.
+- A review app is built every time a commit is pushed to `topic branch`.
- The reviewer fails two reviews before passing the third review.
- After the review passes, `topic branch` is merged into the default branch, where it's deployed to staging.
- After its approval in staging, the changes that were merged into the default branch are deployed to production.
-## How Review Apps work
+## How review apps work
-A Review App is a mapping of a branch with an [environment](../environments/index.md).
-Access to the Review App is made available as a link on the [merge request](../../user/project/merge_requests/index.md) relevant to the branch.
+A review app is a mapping of a branch with an [environment](../environments/index.md).
+Access to the review app is made available as a link on the [merge request](../../user/project/merge_requests/index.md) relevant to the branch.
The following is an example of a merge request with an environment set dynamically.
-![Review App in merge request](img/review_apps_preview_in_mr.png)
+![review app in merge request](img/review_apps_preview_in_mr.png)
In this example, a branch was:
- Successfully built.
- Deployed under a dynamic environment that can be reached by selecting **View app**.
-After adding Review Apps to your workflow, you follow the branched Git flow. That is:
+After adding review apps to your workflow, you follow the branched Git flow. That is:
-1. Push a branch and let the runner deploy the Review App based on the `script` definition of the dynamic environment job.
+1. Push a branch and let the runner deploy the review app based on the `script` definition of the dynamic environment job.
1. Wait for the runner to build and deploy your web application.
1. To view the changes live, select the link in the merge request related to the branch.
-## Configuring Review Apps
+## Configuring review apps
-Review Apps are built on [dynamic environments](../environments/index.md#create-a-dynamic-environment), which allow you to dynamically create a new environment for each branch.
+Review apps are built on [dynamic environments](../environments/index.md#create-a-dynamic-environment), which allow you to dynamically create a new environment for each branch.
-The process of configuring Review Apps is as follows:
+The process of configuring review apps is as follows:
-1. Set up the infrastructure to host and deploy the Review Apps (check the [examples](#review-apps-examples) below).
+1. Set up the infrastructure to host and deploy the review apps (check the [examples](#review-apps-examples) below).
1. [Install](https://docs.gitlab.com/runner/install/) and [configure](https://docs.gitlab.com/runner/commands/) a runner to do deployment.
1. Set up a job in `.gitlab-ci.yml` that uses the [predefined CI/CD variable](../variables/index.md) `${CI_COMMIT_REF_SLUG}`
to create dynamic environments and restrict it to run only on branches.
- Alternatively, you can get a YAML template for this job by [enabling review apps](#enable-review-apps-button) for your project.
-1. Optionally, set a job that [manually stops](../environments/index.md#stop-an-environment) the Review Apps.
+ Alternatively, you can get a YAML template for this job by [enabling review apps](#enable-review-app-button) for your project.
+1. Optionally, set a job that [manually stops](../environments/index.md#stop-an-environment) the review apps.
-### Enable Review Apps button
+### Enable review app button
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118844) in GitLab 12.8.
-When configuring Review Apps for a project, you add a new job to the `.gitlab-ci.yml` file,
+When configuring review apps for a project, you add a new job to the `.gitlab-ci.yml` file,
as mentioned above. To facilitate this, and if you are using Kubernetes, you can select
-**Enable Review Apps** and GitLab prompts you with a template code block that
+**Enable review app** and GitLab prompts you with a template code block that
you can copy and paste into `.gitlab-ci.yml` as a starting point.
Prerequisite:
- You need at least the Developer role for the project.
-To use the Review Apps template:
+To use the review apps template:
-1. On the top bar, select **Main menu > Projects** and find the project you want to create a Review App job for.
+1. On the top bar, select **Main menu > Projects** and find the project you want to create a review app job for.
1. On the left sidebar, select **Deployments > Environments**.
-1. Select **Enable Review Apps**.
+1. Select **Enable review app**.
1. Copy the provided code snippet and paste it into your
`.gitlab-ci.yml` file:
- ![Enable Review Apps modal](img/enable_review_app_v12_8.png)
+ ![enable review apps modal](img/enable_review_app_v12_8.png)
You can edit this template as needed.
-## Review Apps auto-stop
+## Review apps auto-stop
-See how to [configure Review Apps environments to expire and auto-stop](../environments/index.md#stop-an-environment-after-a-certain-time-period)
+See how to [configure review apps environments to expire and auto-stop](../environments/index.md#stop-an-environment-after-a-certain-time-period)
after a given period of time.
-## Review Apps examples
+## Review apps examples
-The following are example projects that demonstrate Review App configuration:
+The following are example projects that demonstrate review app configuration:
| Project | Configuration file |
|-------------------------------------------------------------------------|--------------------|
@@ -104,17 +104,17 @@ The following are example projects that demonstrate Review App configuration:
| [`https://about.gitlab.com/`](https://gitlab.com/gitlab-com/www-gitlab-com/) | [`.gitlab-ci.yml`](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/6ffcdc3cb9af2abed490cbe5b7417df3e83cd76c/.gitlab-ci.yml#L332) |
| [GitLab Insights](https://gitlab.com/gitlab-org/gitlab-insights/) | [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab-insights/-/blob/9e63f44ac2a5a4defc965d0d61d411a768e20546/.gitlab-ci.yml#L234) |
-Other examples of Review Apps:
+Other examples of review apps:
- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
[Cloud Native Development with GitLab](https://www.youtube.com/watch?v=jfIyQEwrocw).
-- [Review Apps for Android](https://about.gitlab.com/blog/2020/05/06/how-to-create-review-apps-for-android-with-gitlab-fastlane-and-appetize-dot-io/).
+- [Review apps for Android](https://about.gitlab.com/blog/2020/05/06/how-to-create-review-apps-for-android-with-gitlab-fastlane-and-appetize-dot-io/).
## Route Maps
Route Maps allows you to go directly from source files
to public pages on the [environment](../environments/index.md) defined for
-Review Apps.
+review apps.
Once set up, the review app link in the merge request
widget can take you directly to the pages changed, making it easier
@@ -198,7 +198,7 @@ After you have the route mapping set up, it takes effect in the following locati
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10761) in GitLab 12.0.
> - [Moved](https://about.gitlab.com/blog/2021/01/26/new-gitlab-product-subscription-model/) to GitLab Premium in 13.9.
> - It's [deployed behind a feature flag](../../user/feature_flags.md), `anonymous_visual_review_feedback`, disabled by default.
-> - It's enabled on GitLab.com.
+> - It's disabled on GitLab.com.
FLAG:
On self-managed GitLab, by default this feature is not available. To make it available,
@@ -209,7 +209,7 @@ With Visual Reviews, members of any team (Product, Design, Quality, and so on) c
### Using Visual Reviews
After Visual Reviews has been [configured](#configure-review-apps-for-visual-reviews) for the
-Review App, the Visual Reviews feedback form is overlaid on the right side of every page.
+review app, the Visual Reviews feedback form is overlaid on the right side of every page.
![Visual review feedback form](img/toolbar_feedback_form_v13_5.png)
@@ -227,9 +227,9 @@ To use the feedback form to make a comment in the merge request:
<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
To see Visual reviews in action, see the [Visual Reviews Walk through](https://youtu.be/1_tvWTlPfM4).
-### Configure Review Apps for Visual Reviews
+### Configure review apps for Visual Reviews
-The feedback form is served through a script you add to pages in your Review App.
+The feedback form is served through a script you add to pages in your review app.
It should be added to the `<head>` of your application and
consists of some project and merge request specific values. Here's how it
looks for a project with code hosted in a project on GitLab.com:
diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md
index 19e0c1e3c81..c675f7204ec 100644
--- a/doc/ci/runners/configure_runners.md
+++ b/doc/ci/runners/configure_runners.md
@@ -97,6 +97,13 @@ Whenever a project is forked, it copies the settings of the jobs that relate
to it. This means that if you have shared runners set up for a project and
someone forks that project, the shared runners serve jobs of this project.
+Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/364303), you might encounter the message `An error occurred while forking the project. Please try again.` if the runner settings of the project you are forking does not match the new project namespace.
+
+To work around this issue, you should make sure that the shared runner settings are consistent in the forked project and the new namespace.
+
+- If shared runners are **enabled** on the forked project, then this should also be **enabled** on the new namespace.
+- If shared runners are **disabled** on the forked project, then this should also be **disabled** on the new namespace.
+
### Attack vectors in runners
Mentioned briefly earlier, but the following things of runners can be exploited.
@@ -299,12 +306,14 @@ globally or for individual jobs:
- [`GIT_STRATEGY`](#git-strategy)
- [`GIT_SUBMODULE_STRATEGY`](#git-submodule-strategy)
+- [`GIT_SUBMODULE_PATHS`](#sync-or-exclude-specific-submodules-from-ci-jobs)
- [`GIT_CHECKOUT`](#git-checkout)
- [`GIT_CLEAN_FLAGS`](#git-clean-flags)
- [`GIT_FETCH_EXTRA_FLAGS`](#git-fetch-extra-flags)
- [`GIT_SUBMODULE_PATHS`](#git-submodule-paths)
- [`GIT_SUBMODULE_UPDATE_FLAGS`](#git-submodule-update-flags)
- [`GIT_DEPTH`](#shallow-cloning) (shallow cloning)
+- [`GIT_SUBMODULE_DEPTH`](#git-submodule-depth)
- [`GIT_CLONE_PATH`](#custom-build-directories) (custom build directories)
- [`TRANSFER_METER_FREQUENCY`](#artifact-and-cache-settings) (artifact/cache meter update frequency)
- [`ARTIFACT_COMPRESSION_LEVEL`](#artifact-and-cache-settings) (artifact archiver compression level)
@@ -556,6 +565,34 @@ You should be aware of the implications for the security, stability, and reprodu
your builds when using the `--remote` flag. In most cases, it is better to explicitly track
submodule commits as designed, and update them using an auto-remediation/dependency bot.
+### Sync or exclude specific submodules from CI jobs
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26495) in GitLab Runner 14.0.
+
+Some projects have a large number of submodules, and not all of them need to be
+synced or updated in all CI jobs. Use the `GIT_SUBMODULE_PATHS` variable to control this behavior.
+The path syntax is the same as [`git submodule`](https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt-ltpathgt82308203):
+
+- To sync and update specific paths:
+
+ ```yaml
+ variables:
+ GIT_SUBMODULE_PATHS: 'submoduleA'
+ ```
+
+- To exclude specific paths:
+
+ ```yaml
+ variables:
+ GIT_SUBMODULE_PATHS: ':(exclude)submoduleA'
+ ```
+
+WARNING:
+Git ignores nested and multiple submodule paths. To ignore a nested submodule, exclude
+the parent submodule and then manually clone it in the job's scripts. For example,
+ `git clone <repo> --recurse-submodules=':(exclude)nested-submodule'`. Make sure
+to wrap the string in single quotes so the YAML can be parsed successfully.
+
### Shallow cloning
> Introduced in GitLab 8.9 as an experimental feature.
@@ -590,6 +627,24 @@ variables:
You can set it globally or per-job in the [`variables`](../yaml/index.md#variables) section.
+### Git submodule depth
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3651) in GitLab Runner 15.5.
+
+Use the `GIT_SUBMODULE_DEPTH` variable to specify the depth of fetching and cloning submodules
+when [`GIT_SUBMODULE_STRATEGY`](#git-submodule-strategy) is set to either `normal` or `recursive`.
+You can set it globally or for a specific job in the [`variables`](../yaml/index.md#variables) section.
+
+When you set the `GIT_SUBMODULE_DEPTH` variable, it overwrites the [`GIT_DEPTH`](#shallow-cloning) setting
+for the submodules only.
+
+To fetch or clone only the last 3 commits:
+
+```yaml
+variables:
+ GIT_SUBMODULE_DEPTH: 3
+```
+
### Custom build directories
> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2211) in GitLab Runner 11.10.
@@ -752,7 +807,7 @@ variables:
NOTE:
Zip archives are the only supported artifact type. Follow [the issue for details](https://gitlab.com/gitlab-org/gitlab/-/issues/367203).
-GitLab Runner can generate and produce attestation metadata for all build artifacts. To enable this feature, you must set the `RUNNER_GENERATE_ARTIFACTS_METADATA` environment variable to `true`. This variable can either be set globally or it can be set for individual jobs. The metadata is in rendered in a plain text `.json` file that's stored with the artifact. The file name is as follows: `{JOB_ID}-artifacts-metadata.json`.
+GitLab Runner can generate and produce attestation metadata for all build artifacts. To enable this feature, you must set the `RUNNER_GENERATE_ARTIFACTS_METADATA` environment variable to `true`. This variable can either be set globally or it can be set for individual jobs. The metadata is in rendered in a plain text `.json` file that's stored with the artifact. The file name is as follows: `{ARTIFACT_NAME}-metadata.json` where `ARTIFACT_NAME` is what was defined as the [name for the artifact](../pipelines/job_artifacts.md#use-cicd-variables-to-define-the-artifacts-name) in the CI file. The file name, however, defaults to `artifacts-metadata.json` if no name was given to the build artifacts.
### Attestation format
diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md
index 405310fb8ba..41cde709143 100644
--- a/doc/ci/runners/index.md
+++ b/doc/ci/runners/index.md
@@ -1,6 +1,6 @@
---
stage: Verify
-group: Runner
+group: Runner SaaS
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
type: reference
---
diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md
index 4be4ed33feb..6354a519810 100644
--- a/doc/ci/runners/runners_scope.md
+++ b/doc/ci/runners/runners_scope.md
@@ -192,6 +192,25 @@ You must have the Owner role for the group.
From this page, you can edit, pause, and remove runners from the group, its subgroups, and projects.
+#### Delete multiple group runners
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361721/) in GitLab 15.6
+
+Prerequisites:
+
+- You must have either:
+ - Owner role for the group.
+ - Access to delete any runners in the group.
+
+To delete multiple runners in a single action in the group list:
+
+1. On the top bar, select **Main menu > Groups** and find your group.
+1. On the left sidebar, select **CI/CD > Runners**.
+1. To delete multiple runners, you can either:
+ - Select the checkbox next to the runner.
+ - Select the checkbox at the top of the runner list to select all runners in the list.
+1. To delete the runners, select **Delete selected**.
+
#### Filter group runners to show only inherited
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/337838/) in GitLab 15.5.
diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md
index d1c63ab7291..df7d5570953 100644
--- a/doc/ci/runners/saas/linux_saas_runner.md
+++ b/doc/ci/runners/saas/linux_saas_runner.md
@@ -116,7 +116,7 @@ The `CI_PRE_CLONE_SCRIPT` variable does not work on GitLab SaaS Windows or macOS
This example was used in the `gitlab-org/gitlab` project until November 2021.
The project no longer uses this optimization because the
[pack-objects cache](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache)
-lets Gitaly serve the full CI/CD fetch traffic. See [Git fetch caching](../../../development/pipelines.md#git-fetch-caching).
+lets Gitaly serve the full CI/CD fetch traffic. See [Git fetch caching](../../../development/pipelines/performance.md#git-fetch-caching).
The `CI_PRE_CLONE_SCRIPT` was defined as a project CI/CD variable:
diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md
index 50c24349712..cd40aac25bc 100644
--- a/doc/ci/runners/saas/macos_saas_runner.md
+++ b/doc/ci/runners/saas/macos_saas_runner.md
@@ -14,7 +14,7 @@ Use these runners to build, test, and deploy apps for the Apple ecosystem (macOS
of all the capabilities of the GitLab single DevOps platform and not have to manage or operate a
build environment.
-Jobs handled by macOS shared runners on GitLab.com **time out after 2 hours**, regardless of the timeout configured in a project.
+Jobs handled by macOS shared runners on GitLab.com **time out after 3 hours**, regardless of the timeout configured in a project.
## Access request process
diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md
index 62350905bd4..a5082af89bc 100644
--- a/doc/ci/secrets/index.md
+++ b/doc/ci/secrets/index.md
@@ -20,11 +20,11 @@ required by a job. Read [GitLab CI/CD pipeline configuration reference](../yaml/
for more information about the syntax.
GitLab has selected [Vault by HashiCorp](https://www.vaultproject.io) as the
-first supported provider, and [KV-V2](https://www.vaultproject.io/docs/secrets/kv/kv-v2)
+first supported provider, and [KV-V2](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2)
as the first supported secrets engine.
GitLab authenticates using Vault's
-[JSON Web Token (JWT) authentication method](https://www.vaultproject.io/docs/auth/jwt#jwt-authentication), using
+[JSON Web Token (JWT) authentication method](https://developer.hashicorp.com/vault/docs/auth/jwt#jwt-authentication), using
the [JSON Web Token](https://gitlab.com/gitlab-org/gitlab/-/issues/207125) (`CI_JOB_JWT`)
introduced in GitLab 12.10.
@@ -88,10 +88,10 @@ To configure your Vault server:
- `VAULT_SERVER_URL` - The URL of your Vault server, such as `https://vault.example.com:8200`.
Required.
- `VAULT_AUTH_ROLE` - Optional. The role to use when attempting to authenticate.
- If no role is specified, Vault uses the [default role](https://www.vaultproject.io/api-docs/auth/jwt#default_role)
+ If no role is specified, Vault uses the [default role](https://developer.hashicorp.com/vault/api-docs/auth/jwt#default_role)
specified when the authentication method was configured.
- `VAULT_AUTH_PATH` - Optional. The path where the authentication method is mounted, default is `jwt`.
- - `VAULT_NAMESPACE` - Optional. The [Vault Enterprise namespace](https://www.vaultproject.io/docs/enterprise/namespaces) to use for reading secrets and authentication.
+ - `VAULT_NAMESPACE` - Optional. The [Vault Enterprise namespace](https://developer.hashicorp.com/vault/docs/enterprise/namespaces) to use for reading secrets and authentication.
If no namespace is specified, Vault uses the `root` ("`/`") namespace.
The setting is ignored by Vault Open Source.
@@ -142,7 +142,7 @@ When a CI job attempts to authenticate, it specifies a role. You can use roles t
different policies together. If authentication is successful, these policies are
attached to the resulting Vault token.
-[Bound claims](https://www.vaultproject.io/docs/auth/jwt#bound-claims) are predefined
+[Bound claims](https://developer.hashicorp.com/vault/docs/auth/jwt#bound-claims) are predefined
values that are matched to the JWT's claims. With bounded claims, you can restrict access
to specific GitLab users, specific projects, or even jobs running for specific Git
references. You can have as many bounded claims you need, but they must *all* match
@@ -183,7 +183,7 @@ For a full list of `CI_JOB_JWT` claims, read the
You can also specify some attributes for the resulting Vault tokens, such as time-to-live,
IP address range, and number of uses. The full list of options is available in
-[Vault's documentation on creating roles](https://www.vaultproject.io/api-docs/auth/jwt#create-role)
+[Vault's documentation on creating roles](https://developer.hashicorp.com/vault/api-docs/auth/jwt#create-role)
for the JSON web token method.
## Using a self-signed Vault server
diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md
index 0f82f2301c7..830b9406c6e 100644
--- a/doc/ci/services/index.md
+++ b/doc/ci/services/index.md
@@ -37,7 +37,7 @@ the CI container itself.
## How services are linked to the job
To better understand how container linking works, read
-[Linking containers together](https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/).
+[Linking containers together](https://docs.docker.com/network/links/).
If you add `mysql` as service to your application, the image is
used to create a container that's linked to the job container.
@@ -392,6 +392,46 @@ time.
1. Check the exit status of build script.
1. Remove the build container and all created service containers.
+## Capturing service container logs
+
+Logs generated by applications running in service containers can be captured for subsequent examination and debugging.
+You might want to look at service container's logs when the service container has started successfully, but is not
+behaving es expected, leading to job failures. The logs can indicate missing or incorrect configuration of the service
+within the container.
+
+`CI_DEBUG_SERVICES` should only by enabled when service containers are being actively debugged as there are both storage
+and performance consequences to capturing service container logs.
+
+To enable service logging, add the `CI_DEBUG_SERVICES` variable to the project's
+`.gitlab-ci.yml` file:
+
+```yaml
+variables:
+ CI_DEBUG_SERVICES: "true"
+```
+
+Accepted values are:
+
+- Enabled: `TRUE`, `true`, `True`
+- Disabled: `FALSE`, `false`, `False`
+
+Any other values will result in an error message and effectively disable the feature.
+
+When enabled, logs for _all_ service containers will be captured and streamed into the jobs trace log concurrently with
+other logs. Logs from each container will be prefixed with the container's aliases, and displayed in a different color.
+
+NOTE:
+You may want to adjust the logging level in the service container for which you want to capture logs since the default
+logging level may not provide sufficient details to diagnose job failures.
+
+WARNING:
+Enabling `CI_DEBUG_SERVICES` _may_ result in masked variables being revealed. When `CI_DEBUG_SERVICES` is enabled,
+service container logs and the CI job's logs are streamed to the job's trace log _concurrently_, which makes it possible
+for a service container log to be inserted _inside_ a job's masked log. This would thwart the variable masking mechanism
+and result in the masked variable being revealed.
+
+See [Mask a CI/CD Variable](../variables/index.md#mask-a-cicd-variable)
+
## Debug a job locally
The following commands are run without root privileges. You should be
diff --git a/doc/ci/testing/code_quality.md b/doc/ci/testing/code_quality.md
index 7345c7ca5eb..d1ed28b79c0 100644
--- a/doc/ci/testing/code_quality.md
+++ b/doc/ci/testing/code_quality.md
@@ -339,6 +339,20 @@ This example is specific to GitLab Code Quality. For more general
instructions on how to configure DinD with a registry mirror, see the
relevant [documentation](../docker/using_docker_build.md#enable-registry-mirror-for-dockerdind-service).
+#### List of images to be stored in the private container registry
+
+The following images are needed for the [default `.codeclimate.yml`](https://gitlab.com/gitlab-org/ci-cd/codequality/-/blob/master/codeclimate_defaults/.codeclimate.yml.template):
+
+- `codeclimate/codeclimate-structure:latest`
+- `codeclimate/codeclimate-csslint:latest`
+- `codeclimate/codeclimate-coffeelint:latest`
+- `codeclimate/codeclimate-duplication:latest`
+- `codeclimate/codeclimate-eslint:latest`
+- `codeclimate/codeclimate-fixme:latest`
+- `codeclimate/codeclimate-rubocop:rubocop-0-92`
+
+If you are using a custom `.codeclimate.yml` configuration file, you must add the specified plugins in your private container registry.
+
#### Configure Code Quality to use the Dependency Proxy
Prerequisite:
diff --git a/doc/ci/testing/load_performance_testing.md b/doc/ci/testing/load_performance_testing.md
index 6e1b440f252..ba7bf14fb59 100644
--- a/doc/ci/testing/load_performance_testing.md
+++ b/doc/ci/testing/load_performance_testing.md
@@ -138,7 +138,7 @@ For example, you can override the duration of the test with a CLI option:
```
GitLab only displays the key performance metrics in the MR widget if k6's results are saved
-via [summary export](https://k6.io/docs/results-visualization/json#summary-export)
+via [summary export](https://k6.io/docs/results-output/real-time/json/#summary-export)
as a [Load Performance report artifact](../yaml/artifacts_reports.md#artifactsreportsload_performance).
The latest Load Performance artifact available is always used, using the
summary values from the test.
diff --git a/doc/ci/testing/test_coverage_visualization.md b/doc/ci/testing/test_coverage_visualization.md
index ee6b47e69a5..9bff8dbf780 100644
--- a/doc/ci/testing/test_coverage_visualization.md
+++ b/doc/ci/testing/test_coverage_visualization.md
@@ -360,7 +360,7 @@ The following [`.gitlab-ci.yml`](../yaml/index.md) example for Go uses:
- [`gocover-cobertura`](https://github.com/boumenot/gocover-cobertura) to convert Go's coverage profile into the Cobertura XML format.
This example assumes that [Go modules](https://go.dev/ref/mod)
-are being used. Please note that the `-covermode count` option does not work with the `-race` flag.
+are being used. The `-covermode count` option does not work with the `-race` flag.
If you want to generate code coverage while also using the `-race` flag, you must switch to
`-covermode atomic` which is slower than `-covermode count`. See [this blog post](https://go.dev/blog/cover)
for more details.
diff --git a/doc/ci/testing/unit_test_report_examples.md b/doc/ci/testing/unit_test_report_examples.md
index c14e4eedd7c..5d4cfa88d88 100644
--- a/doc/ci/testing/unit_test_report_examples.md
+++ b/doc/ci/testing/unit_test_report_examples.md
@@ -18,7 +18,10 @@ Use the following job in `.gitlab-ci.yml`. This includes the `artifacts:paths` k
```yaml
## Use https://github.com/sj26/rspec_junit_formatter to generate a JUnit report format XML file with rspec
ruby:
+ image: ruby:3.0.4
stage: test
+ before_script:
+ - apt-get update -y && apt-get install -y bundler
script:
- bundle install
- bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml
diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md
index cafa64c4832..a667836fee4 100644
--- a/doc/ci/triggers/index.md
+++ b/doc/ci/triggers/index.md
@@ -193,3 +193,11 @@ A response of `{"message":"404 Not Found"}` when triggering a pipeline might be
by using a [personal access token](../../user/profile/personal_access_tokens.md)
instead of a trigger token. [Create a new trigger token](#create-a-trigger-token)
and use it instead of the personal access token.
+
+### `The requested URL returned error: 400` when triggering a pipeline
+
+If you attempt to trigger a pipeline by using a `ref` that is a branch name that
+doesn't exist, GitLab returns `The requested URL returned error: 400`.
+
+For example, you might accidentally use `main` for the branch name in a project that
+uses a different branch name for its default branch.
diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md
index 8f78469af18..87ebff74600 100644
--- a/doc/ci/troubleshooting.md
+++ b/doc/ci/troubleshooting.md
@@ -182,6 +182,14 @@ a branch pipeline instead.
It's also possible that your [`workflow: rules`](yaml/index.md#workflow) configuration
blocked the pipeline, or allowed the wrong pipeline type.
+### Pipeline with many jobs fails to start
+
+A Pipeline that has more jobs than the instance's defined [CI/CD limits](../user/admin_area/settings/continuous_integration.md#set-cicd-limits)
+fails to start.
+
+To reduce the number of jobs in your pipeline, you can split your `.gitlab-ci.yml`
+configuration using [parent-child pipelines](../ci/pipelines/pipeline_architectures.md#parent-child-pipelines).
+
### A job runs unexpectedly
A common reason a job is added to a pipeline unexpectedly is because the `changes`
@@ -285,14 +293,14 @@ has failed or been canceled.
If a merge request pipeline or merged result pipeline was canceled or failed, you can:
-- Re-run the entire pipeline by clicking **Run pipeline** in the pipeline tab in the merge request.
+- Re-run the entire pipeline by selecting **Run pipeline** in the pipeline tab in the merge request.
- [Retry only the jobs that failed](pipelines/index.md#view-pipelines). If you re-run the entire pipeline, this is not necessary.
- Push a new commit to fix the failure.
If the merge train pipeline has failed, you can:
- Check the failure and determine if you can use the [`/merge` quick action](../user/project/quick_actions.md) to immediately add the merge request to the train again.
-- Re-run the entire pipeline by clicking **Run pipeline** in the pipeline tab in the merge request, then add the merge request to the train again.
+- Re-run the entire pipeline by selecting **Run pipeline** in the pipeline tab in the merge request, then add the merge request to the train again.
- Push a commit to fix the failure, then add the merge request to the train again.
If the merge train pipeline was canceled before the merge request was merged, without a failure, you can:
@@ -400,6 +408,77 @@ This flag reduces system resource usage on the `jobs/request` endpoint.
When enabled, jobs created in the last hour can run in projects which are out of quota.
Earlier jobs are already canceled by a periodic background worker (`StuckCiJobsWorker`).
+## CI/CD troubleshooting rails console commands
+
+The following commands are run in the [rails console](../administration/operations/rails_console.md#starting-a-rails-console-session).
+
+WARNING:
+Any command that changes data directly could be damaging if not run correctly, or under the right conditions.
+We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+
+### Cancel stuck pending pipelines
+
+```ruby
+project = Project.find_by_full_path('<project_path>')
+Ci::Pipeline.where(project_id: project.id).where(status: 'pending').count
+Ci::Pipeline.where(project_id: project.id).where(status: 'pending').each {|p| p.cancel if p.stuck?}
+Ci::Pipeline.where(project_id: project.id).where(status: 'pending').count
+```
+
+### Try merge request integration
+
+```ruby
+project = Project.find_by_full_path('<project_path>')
+mr = project.merge_requests.find_by(iid: <merge_request_iid>)
+mr.project.try(:ci_integration)
+```
+
+### Validate the `.gitlab-ci.yml` file
+
+```ruby
+project = Project.find_by_full_path('<project_path>')
+content = p.repository.gitlab_ci_yml_for(project.repository.root_ref_sha)
+Gitlab::Ci::Lint.new(project: project, current_user: User.first).validate(content)
+```
+
+### Disable AutoDevOps on Existing Projects
+
+```ruby
+Project.all.each do |p|
+ p.auto_devops_attributes={"enabled"=>"0"}
+ p.save
+end
+```
+
+### Obtain runners registration token
+
+```ruby
+Gitlab::CurrentSettings.current_application_settings.runners_registration_token
+```
+
+### Seed runners registration token
+
+```ruby
+appSetting = Gitlab::CurrentSettings.current_application_settings
+appSetting.set_runners_registration_token('<new-runners-registration-token>')
+appSetting.save!
+```
+
+### Run pipeline schedules manually
+
+You can run pipeline schedules manually through the Rails console to reveal any errors that are usually not visible.
+
+```ruby
+# schedule_id can be obtained from Edit Pipeline Schedule page
+schedule = Ci::PipelineSchedule.find_by(id: <schedule_id>)
+
+# Select the user that you want to run the schedule for
+user = User.find_by_username('<username>')
+
+# Run the schedule
+ps = Ci::CreatePipelineService.new(schedule.project, user, ref: schedule.ref).execute!(:schedule, ignore_skip_ci: true, save_on_errors: false, schedule: schedule)
+```
+
## How to get help
If you are unable to resolve pipeline issues, you can get help from:
diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md
index 7ad42aaf96b..97707c603bd 100644
--- a/doc/ci/variables/index.md
+++ b/doc/ci/variables/index.md
@@ -106,7 +106,7 @@ job1:
Variables saved in the `.gitlab-ci.yml` file should store only non-sensitive project
configuration, like a `RAILS_ENV` or `DATABASE_URL` variable. These variables are
-visible in the repository. Store sensitive variables containing secrets, keys, and so on
+visible in the repository. Store sensitive variables containing values like secrets or keys
in project settings.
Variables saved in the `.gitlab-ci.yml` file are also available in [service containers](../docker/using_docker_images.md).
@@ -161,7 +161,7 @@ in the project settings, not in the `.gitlab-ci.yml` file.
To add or update variables in the project settings:
1. Go to your project's **Settings > CI/CD** and expand the **Variables** section.
-1. Select the **Add Variable** button and fill in the details:
+1. Select **Add variable** and fill in the details:
- **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`.
- **Value**: No limitations.
@@ -203,7 +203,7 @@ Use group variables to store secrets like passwords, SSH keys, and credentials,
To add a group variable:
1. In the group, go to **Settings > CI/CD**.
-1. Select the **Add Variable** button and fill in the details:
+1. Select **Add variable** and fill in the details:
- **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`.
- **Value**: No limitations.
@@ -241,7 +241,7 @@ To add an instance variable:
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > CI/CD** and expand the **Variables** section.
-1. Select the **Add variable** button, and fill in the details:
+1. Select **Add variable** and fill in the details:
- **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`.
- **Value**: In [GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/220028),
@@ -255,8 +255,6 @@ To add an instance variable:
### CI/CD variable types
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/46806) in GitLab 11.11.
-
All predefined CI/CD variables and variables defined in the `.gitlab-ci.yml` file
are `Variable` type. Project, group and instance CI/CD variables can be `Variable`
or `File` type.
@@ -333,7 +331,7 @@ job1:
### Mask a CI/CD variable
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/13784) in GitLab 11.10
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330650) in GitLab 13.12, the `~` character can be used in masked variables.
You can mask a project, group, or instance CI/CD variable so the value of the variable
does not display in job logs.
@@ -352,20 +350,24 @@ The value of the variable must:
- Be a single line.
- Be 8 characters or longer, consisting only of:
- Characters from the Base64 alphabet (RFC4648).
- - The `@` and `:` characters (In [GitLab 12.2 and later](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63043)).
- - The `.` character (In [GitLab 12.10 and later](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29022)).
- - The `~` character (In [GitLab 13.12 and later](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61517)).
+ - The `@`, `:`, `.`, or `~` characters.
- Not match the name of an existing predefined or custom CI/CD variable.
-NOTE:
-Masking a CI/CD variable is not a guaranteed way to prevent malicious users from accessing
-variable values. To make variables more secure, you can [use external secrets](../secrets/index.md).
-
WARNING:
-Due to a technical limitation, masked variables that are more than 4 KiB in length are not recommended. Printing such
-a large value to the trace log has the potential to be [revealed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28128).
-When using GitLab Runner 14.2, only the tail of the variable, characters beyond 4KiB in length, have the potential to
-be revealed.
+Masking a CI/CD variable is not a guaranteed way to prevent malicious users from
+accessing variable values. The masking feature is "best-effort" and there to
+help when a variable is accidentally revealed. To make variables more secure,
+consider using [external secrets](../secrets/index.md) and [file type variables](#cicd-variable-types)
+to prevent commands such as `env`/`printenv` from printing secret variables.
+
+Runner versions implement masking in different ways, some with technical
+limitations. Below is a table of such limitations.
+
+| Version from | Version to | Limitations |
+| ------------ | ---------- | ------ |
+| v11.9.0 | v14.1.0 | Masking of large secrets (greater than 4 KiB) could potentially be [revealed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28128). No sensitive URL parameter masking. |
+| v14.2.0 | v15.3.0 | The tail of a large secret (greater than 4 KiB) could potentially be [revealed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28128). No sensitive URL parameter masking. |
+| v15.7.0 | | Potential for secrets to be revealed when `CI_DEBUG_SERVICES` is enabled. For details, read about [service container logging](../services/index.md#capturing-service-container-logs). |
### Protected CI/CD variables
@@ -506,7 +508,7 @@ job_name:
# - 'dir env:' # Use this for PowerShell
```
-Example job log output:
+Example job log output (truncated):
```shell
export CI_JOB_ID="50"
@@ -528,31 +530,6 @@ export CI_PROJECT_ID="34"
export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-foss"
export CI_PROJECT_NAME="gitlab-foss"
export CI_PROJECT_TITLE="GitLab FOSS"
-export CI_PROJECT_DESCRIPTION="GitLab FOSS is a read-only mirror of GitLab, with all proprietary code removed."
-export CI_PROJECT_NAMESPACE="gitlab-org"
-export CI_PROJECT_ROOT_NAMESPACE="gitlab-org"
-export CI_PROJECT_PATH="gitlab-org/gitlab-foss"
-export CI_PROJECT_URL="https://example.com/gitlab-org/gitlab-foss"
-export CI_REGISTRY="registry.example.com"
-export CI_REGISTRY_IMAGE="registry.example.com/gitlab-org/gitlab-foss"
-export CI_REGISTRY_USER="gitlab-ci-token"
-export CI_REGISTRY_PASSWORD="[masked]"
-export CI_RUNNER_ID="10"
-export CI_RUNNER_DESCRIPTION="my runner"
-export CI_RUNNER_TAGS="docker, linux"
-export CI_SERVER="yes"
-export CI_SERVER_URL="https://example.com"
-export CI_SERVER_HOST="example.com"
-export CI_SERVER_PORT="443"
-export CI_SERVER_PROTOCOL="https"
-export CI_SERVER_NAME="GitLab"
-export CI_SERVER_REVISION="70606bf"
-export CI_SERVER_VERSION="8.9.0"
-export CI_SERVER_VERSION_MAJOR="8"
-export CI_SERVER_VERSION_MINOR="9"
-export CI_SERVER_VERSION_PATCH="0"
-export GITLAB_USER_EMAIL="user@example.com"
-export GITLAB_USER_ID="42"
...
```
@@ -776,8 +753,6 @@ explains if the integration has any deployment variables available.
## Auto DevOps environment variables
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/49056) in GitLab 11.7.
-
You can configure [Auto DevOps](../../topics/autodevops/index.md) to pass CI/CD variables
to a running application.
@@ -817,7 +792,7 @@ job_name:
Example output (truncated):
-```shell
+```plaintext
...
export CI_SERVER_TLS_CA_FILE="/builds/gitlab-examples/ci-debug-trace.tmp/CI_SERVER_TLS_CA_FILE"
if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then
@@ -885,48 +860,6 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then
++ CI_PROJECT_ID=17893
++ export CI_PROJECT_NAME=ci-debug-trace
++ CI_PROJECT_NAME=ci-debug-trace
-++ export CI_PROJECT_TITLE='GitLab FOSS'
-++ CI_PROJECT_TITLE='GitLab FOSS'
-++ export CI_PROJECT_DESCRIPTION='GitLab FOSS is a read-only mirror of GitLab, with all proprietary code removed.'
-++ CI_PROJECT_DESCRIPTION='GitLab FOSS is a read-only mirror of GitLab, with all proprietary code removed.'
-++ export CI_PROJECT_PATH=gitlab-examples/ci-debug-trace
-++ CI_PROJECT_PATH=gitlab-examples/ci-debug-trace
-++ export CI_PROJECT_PATH_SLUG=gitlab-examples-ci-debug-trace
-++ CI_PROJECT_PATH_SLUG=gitlab-examples-ci-debug-trace
-++ export CI_PROJECT_NAMESPACE=gitlab-examples
-++ CI_PROJECT_NAMESPACE=gitlab-examples
-++ export CI_PROJECT_ROOT_NAMESPACE=gitlab-examples
-++ CI_PROJECT_ROOT_NAMESPACE=gitlab-examples
-++ export CI_PROJECT_URL=https://gitlab.com/gitlab-examples/ci-debug-trace
-++ CI_PROJECT_URL=https://gitlab.com/gitlab-examples/ci-debug-trace
-++ export CI_PROJECT_VISIBILITY=public
-++ CI_PROJECT_VISIBILITY=public
-++ export CI_PROJECT_REPOSITORY_LANGUAGES=
-++ CI_PROJECT_REPOSITORY_LANGUAGES=
-++ export CI_PROJECT_CLASSIFICATION_LABEL=
-++ CI_PROJECT_CLASSIFICATION_LABEL=
-++ export CI_DEFAULT_BRANCH=main
-++ CI_DEFAULT_BRANCH=main
-++ export CI_REGISTRY=registry.gitlab.com
-++ CI_REGISTRY=registry.gitlab.com
-++ export CI_API_V4_URL=https://gitlab.com/api/v4
-++ CI_API_V4_URL=https://gitlab.com/api/v4
-++ export CI_PIPELINE_IID=123
-++ CI_PIPELINE_IID=123
-++ export CI_PIPELINE_SOURCE=web
-++ CI_PIPELINE_SOURCE=web
-++ export CI_CONFIG_PATH=.gitlab-ci.yml
-++ CI_CONFIG_PATH=.gitlab-ci.yml
-++ export CI_COMMIT_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045
-++ CI_COMMIT_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045
-++ export CI_COMMIT_SHORT_SHA=dd648b2e
-++ CI_COMMIT_SHORT_SHA=dd648b2e
-++ export CI_COMMIT_BEFORE_SHA=0000000000000000000000000000000000000000
-++ CI_COMMIT_BEFORE_SHA=0000000000000000000000000000000000000000
-++ export CI_COMMIT_REF_NAME=main
-++ CI_COMMIT_REF_NAME=main
-++ export CI_COMMIT_REF_SLUG=main
-++ CI_COMMIT_REF_SLUG=main
...
```
diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md
index 606847ee756..852110a1415 100644
--- a/doc/ci/variables/predefined_variables.md
+++ b/doc/ci/variables/predefined_variables.md
@@ -46,6 +46,7 @@ as it can cause the pipeline to behave unexpectedly.
| `CI_CONCURRENT_PROJECT_ID` | all | 11.10 | The unique ID of build execution in a single executor and project. |
| `CI_CONFIG_PATH` | 9.4 | 0.5 | The path to the CI/CD configuration file. Defaults to `.gitlab-ci.yml`. Read-only inside a running pipeline. |
| `CI_DEBUG_TRACE` | all | 1.7 | `true` if [debug logging (tracing)](index.md#debug-logging) is enabled. |
+| `CI_DEBUG_SERVICES` | 15.7 | 15.7 | `true` if [service container logging](../services/index.md#capturing-service-container-logs) is enabled. |
| `CI_DEFAULT_BRANCH` | 12.4 | all | The name of the project's default branch. |
| `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` | 13.7 | all | The top-level group image prefix for pulling images through the Dependency Proxy. |
| `CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX` | 14.3 | all | The direct group image prefix for pulling images through the Dependency Proxy. |
@@ -89,7 +90,6 @@ as it can cause the pipeline to behave unexpectedly.
| `CI_PIPELINE_TRIGGERED` | all | all | `true` if the job was [triggered](../triggers/index.md). |
| `CI_PIPELINE_URL` | 11.1 | 0.5 | The URL for the pipeline details. |
| `CI_PIPELINE_CREATED_AT` | 13.10 | all | The UTC datetime when the pipeline was created, in [ISO 8601](https://www.rfc-editor.org/rfc/rfc3339#appendix-A) format. |
-| `CI_PROJECT_CONFIG_PATH` | 13.8 to 13.12 | all | [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/322807) in GitLab 14.0. Use `CI_CONFIG_PATH`. |
| `CI_PROJECT_DIR` | all | all | The full path the repository is cloned to, and where the job runs from. If the GitLab Runner `builds_dir` parameter is set, this variable is set relative to the value of `builds_dir`. For more information, see the [Advanced GitLab Runner configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section). |
| `CI_PROJECT_ID` | all | all | The ID of the current project. This ID is unique across all projects on the GitLab instance. |
| `CI_PROJECT_NAME` | 8.10 | 0.5 | The name of the directory for the project. For example if the project URL is `gitlab.example.com/group-name/project-1`, `CI_PROJECT_NAME` is `project-1`. |
diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md
index 7e5451658f2..c8436fc044d 100644
--- a/doc/ci/variables/where_variables_can_be_used.md
+++ b/doc/ci/variables/where_variables_can_be_used.md
@@ -36,13 +36,15 @@ There are two places defined variables can be used. On the:
| [`include`](../yaml/index.md#include) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>See [Use variables with include](../yaml/includes.md#use-variables-with-include) for more information on supported variables. |
| [`only:variables`](../yaml/index.md#onlyvariables--exceptvariables) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). |
| [`resource_group`](../yaml/index.md#resource_group) | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/>- `CI_ENVIRONMENT_URL`<br/>- [Persisted variables](#persisted-variables). |
+| [`rules:exists`](../yaml/index.md#rulesexists) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. |
| [`rules:if`](../yaml/index.md#rulesif) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). |
| [`script`](../yaml/index.md#script) | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment). |
| [`services:name`](../yaml/index.md#services) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). |
| [`services`](../yaml/index.md#services) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). |
| [`tags`](../yaml/index.md#tags) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35742) in GitLab 14.1. |
-| [`trigger` and `trigger:project`](../yaml/index.md#trigger) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367660) in GitLab 15.3. |
+| [`trigger` and `trigger:project`](../yaml/index.md#trigger) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. Variable expansion for `trigger:project` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367660) in GitLab 15.3. |
| [`variables`](../yaml/index.md#variables) | yes | GitLab/Runner | The variable expansion is first made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab, and then any unrecognized or unavailable variables are expanded by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). |
+| [`workflow:name`](../yaml/index.md#workflowname) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables available in `workflow`:<br/>- Project/Group variables.<br/>- Global `variables` and `workflow:rules:variables` (when matching the rule).<br/>- Variables inherited from parent pipelines.<br/>- Variables from triggers.<br/>- Variables from pipeline schedules.<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml`, variables defined in jobs, or [Persisted variables](#persisted-variables). |
### `config.toml` file
diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md
index 67cbe989f74..9f5e4e919b0 100644
--- a/doc/ci/yaml/artifacts_reports.md
+++ b/doc/ci/yaml/artifacts_reports.md
@@ -178,7 +178,7 @@ report uploads to GitLab as an artifact.
GitLab can display the results of one or more reports in:
-- The merge request [security widget](../../user/application_security/dast/index.md#view-details-of-a-vulnerability-detected-by-dast).
+- The merge request security widget.
- The pipeline [**Security** tab](../../user/application_security/vulnerability_report/pipeline.md#view-vulnerabilities-in-a-pipeline).
- The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md).
- The [security dashboard](../../user/application_security/security_dashboard/index.md).
diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md
index e06abe1dc69..3392304775a 100644
--- a/doc/ci/yaml/index.md
+++ b/doc/ci/yaml/index.md
@@ -146,7 +146,7 @@ the time limit to resolve all files is 30 seconds.
**Possible inputs**: The `include` subkeys:
- [`include:local`](#includelocal)
-- [`include:file`](#includefile)
+- [`include:project`](#includeproject)
- [`include:remote`](#includeremote)
- [`include:template`](#includetemplate)
@@ -203,58 +203,52 @@ include: '.gitlab-ci-production.yml'
- All [nested includes](includes.md#use-nested-includes) are executed in the scope of the same project,
so you can use local, project, remote, or template includes.
-#### `include:file`
+#### `include:project`
> Including multiple files from the same project [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26793) in GitLab 13.6. [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/271560) in GitLab 13.8.
To include files from another private project on the same GitLab instance,
-use `include:file`. You can use `include:file` in combination with `include:project` only.
+use `include:project` and `include:file`.
**Keyword type**: Global keyword.
**Possible inputs**:
-A full path, relative to the root directory (`/`):
+- `include:project`: The full GitLab project path.
+- `include:file` A full file path, or array of file paths, relative to the root directory (`/`).
+ The YAML files must have the `.yml` or `.yaml` extension.
+- `include:ref`: Optional. The ref to retrieve the file from. Defaults to the `HEAD` of the project
+ when not specified.
-- The YAML file must have the extension `.yml` or `.yaml`.
-- You can use [certain CI/CD variables](includes.md#use-variables-with-include).
+You can use [certain CI/CD variables](includes.md#use-variables-with-include).
-**Example of `include:file`**:
+**Example of `include:project`**:
```yaml
include:
- project: 'my-group/my-project'
file: '/templates/.gitlab-ci-template.yml'
+ - project: 'my-group/my-subgroup/my-project-2'
+ file:
+ - '/templates/.builds.yml'
+ - '/templates/.tests.yml'
```
-You can also specify a `ref`. If you do not specify a value, the ref defaults to the `HEAD` of the project:
+You can also specify a `ref`:
```yaml
include:
- project: 'my-group/my-project'
- ref: main
+ ref: main # Git branch
file: '/templates/.gitlab-ci-template.yml'
-
- project: 'my-group/my-project'
- ref: v1.0.0 # Git Tag
+ ref: v1.0.0 # Git Tag
file: '/templates/.gitlab-ci-template.yml'
-
- project: 'my-group/my-project'
ref: 787123b47f14b552955ca2786bc9542ae66fee5b # Git SHA
file: '/templates/.gitlab-ci-template.yml'
```
-You can include multiple files from the same project:
-
-```yaml
-include:
- - project: 'my-group/my-project'
- ref: main
- file:
- - '/templates/.builds.yml'
- - '/templates/.tests.yml'
-```
-
**Additional details**:
- All [nested includes](includes.md#use-nested-includes) are executed in the scope of the target project.
@@ -379,7 +373,7 @@ start. Jobs in the current stage are not stopped and continue to run.
- If a job does not specify a [`stage`](#stage), the job is assigned the `test` stage.
- If a stage is defined but no jobs use it, the stage is not visible in the pipeline,
- which can help [compliance pipeline configurations](../../user/group/manage.md#configure-a-compliance-pipeline):
+ which can help [compliance pipeline configurations](../../user/group/compliance_frameworks.md#configure-a-compliance-pipeline):
- Stages can be defined in the compliance configuration but remain hidden if not used.
- The defined stages become visible when developers use them in job definitions.
@@ -414,12 +408,33 @@ All pipelines are assigned the defined name. Any leading or trailing spaces in t
**Possible inputs**:
- A string.
+- [CI/CD variables](../variables/where_variables_can_be_used.md#gitlab-ciyml-file).
+- A combination of both.
-**Example of `workflow:name`**:
+**Examples of `workflow:name`**:
+
+A simple pipeline name with a predefined variable:
```yaml
workflow:
- name: 'Pipeline name'
+ name: 'Pipeline for branch: $CI_COMMIT_BRANCH'
+```
+
+A configuration with different pipeline names depending on the pipeline conditions:
+
+```yaml
+variables:
+ PIPELINE_NAME: 'Default pipeline name'
+
+workflow:
+ name: '$PIPELINE_NAME'
+ rules:
+ - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
+ variables:
+ PIPELINE_NAME: 'MR pipeline: $CI_COMMIT_BRANCH'
+ - if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-in-ruby3/'
+ variables:
+ PIPELINE_NAME: 'Ruby 3 pipeline'
```
#### `workflow:rules`
@@ -982,7 +997,7 @@ rspec:
- Combining reports in parent pipelines using [artifacts from child pipelines](#needspipelinejob) is
not supported. Track progress on adding support in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/215725).
-- To be able to browse the report output files, include the [`artifacts:paths`](#artifactspaths) keyword. Please note that this will upload and store the artifact twice.
+- To be able to browse the report output files, include the [`artifacts:paths`](#artifactspaths) keyword. This will upload and store the artifact twice.
- The test reports are collected regardless of the job results (success or failure).
You can use [`artifacts:expire_in`](#artifactsexpire_in) to set up an expiration
date for artifacts reports.
@@ -1134,7 +1149,7 @@ that use the same cache key use the same cache, including in different pipelines
If not set, the default key is `default`. All jobs with the `cache` keyword but
no `cache:key` share the `default` cache.
-Must be used with `cache: path`, or nothing is cached.
+Must be used with `cache: paths`, or nothing is cached.
**Keyword type**: Job keyword. You can use it only as part of a job or in the
[`default` section](#default).
@@ -1304,7 +1319,7 @@ rspec:
Use `cache:when` to define when to save the cache, based on the status of the job.
-Must be used with `cache: path`, or nothing is cached.
+Must be used with `cache: paths`, or nothing is cached.
**Keyword type**: Job keyword. You can use it only as part of a job or in the
[`default` section](#default).
@@ -1344,7 +1359,7 @@ Use the `pull` policy when you have many jobs executing in parallel that use the
This policy speeds up job execution and reduces load on the cache server. You can
use a job with the `push` policy to build the cache.
-Must be used with `cache: path`, or nothing is cached.
+Must be used with `cache: paths`, or nothing is cached.
**Keyword type**: Job keyword. You can use it only as part of a job or in the
[`default` section](#default).
@@ -1464,8 +1479,8 @@ to select a specific site profile and scanner profile.
**Related topics**:
-- [Site profile](../../user/application_security/dast/index.md#site-profile).
-- [Scanner profile](../../user/application_security/dast/index.md#scanner-profile).
+- [Site profile](../../user/application_security/dast/proxy-based.md#site-profile).
+- [Scanner profile](../../user/application_security/dast/proxy-based.md#scanner-profile).
### `dependencies`
@@ -1749,6 +1764,11 @@ deploy:
deployment_tier: production
```
+**Additional details**:
+
+- Enviroments created from this job definition are assigned a [tier](../environments/index.md#deployment-tier-of-environments) based on this value.
+- Existing environments don't have their tier updated if this value is added later. Existing enviroments must have their tier updated via the [Environments API](../../api/environments.md#update-an-existing-environment).
+
**Related topics**:
- [Deployment tier of environments](../environments/index.md#deployment-tier-of-environments).
@@ -2809,6 +2829,13 @@ deploystacks: [vultr, data]
deploystacks: [vultr, processing]
```
+**Additional details**:
+
+- `parallel:matrix` jobs add the variable values to the job names to differentiate
+ the jobs from each other, but [large values can cause names to exceed limits](https://gitlab.com/gitlab-org/gitlab/-/issues/362262):
+ - Job names must be [255 characters or fewer](../jobs/index.md#job-name-limitations).
+ - When using [`needs`](#needs), job names must be 128 characters or fewer.
+
**Related topics**:
- [Run a one-dimensional matrix of parallel jobs](../jobs/job_control.md#run-a-one-dimensional-matrix-of-parallel-jobs).
@@ -3229,7 +3256,8 @@ Use `rules:if` clauses to specify when to add a job to a pipeline:
- If no `if` statements are true, do not add the job to the pipeline.
`if` clauses are evaluated based on the values of [predefined CI/CD variables](../variables/predefined_variables.md)
-or [custom CI/CD variables](../variables/index.md#custom-cicd-variables).
+or [custom CI/CD variables](../variables/index.md#custom-cicd-variables), with
+[some exceptions](../variables/where_variables_can_be_used.md#gitlab-ciyml-file).
**Keyword type**: Job-specific and pipeline-specific. You can use it as part of a job
to configure the job behavior, or with [`workflow`](#workflow) to configure the pipeline behavior.
@@ -3403,7 +3431,8 @@ relative to `refs/heads/branch1` and the pipeline source is a merge request even
#### `rules:exists`
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24021) in GitLab 12.4.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24021) in GitLab 12.4.
+> - CI/CD variable support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/283881) in GitLab 15.6.
Use `exists` to run a job when certain files exist in the repository.
@@ -3411,8 +3440,7 @@ Use `exists` to run a job when certain files exist in the repository.
**Possible inputs**:
-- An array of file paths. Paths are relative to the project directory (`$CI_PROJECT_DIR`)
- and can't directly link outside it. File paths can use glob patterns.
+- An array of file paths. Paths are relative to the project directory (`$CI_PROJECT_DIR`) and can't directly link outside it. File paths can use glob patterns and [CI/CD variables](../variables/where_variables_can_be_used.md#gitlab-ciyml-file).
**Example of `rules:exists`**:
@@ -3573,7 +3601,7 @@ Use `secrets:vault` to specify secrets provided by a [HashiCorp Vault](https://w
**Example of `secrets:vault`**:
-To specify all details explicitly and use the [KV-V2](https://www.vaultproject.io/docs/secrets/kv/kv-v2) secrets engine:
+To specify all details explicitly and use the [KV-V2](https://developer.hashicorp.com/vault/docs/secrets/kv/kv-v2) secrets engine:
```yaml
job:
@@ -3938,14 +3966,15 @@ Use `trigger` to declare that a job is a "trigger job" which starts a
Trigger jobs can use only a limited set of GitLab CI/CD configuration keywords.
The keywords available for use in trigger jobs are:
-- [`trigger`](#trigger).
-- [`stage`](#stage).
- [`allow_failure`](#allow_failure).
-- [`rules`](#rules).
-- [`only` and `except`](#only--except).
-- [`when`](#when) (only with a value of `on_success`, `on_failure`, or `always`).
- [`extends`](#extends).
- [`needs`](#needs), but not [`needs:project`](#needsproject).
+- [`only` and `except`](#only--except).
+- [`rules`](#rules).
+- [`stage`](#stage).
+- [`trigger`](#trigger).
+- [`variables`](#variables).
+- [`when`](#when) (only with a value of `on_success`, `on_failure`, or `always`).
**Keyword type**: Job keyword. You can use it only as part of a job.
@@ -3969,6 +3998,8 @@ trigger-multi-project-pipeline:
- In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201938), you
can use [`when:manual`](#when) in the same job as `trigger`. In GitLab 13.4 and
earlier, using them together causes the error `jobs:#{job-name} when should be on_success, on_failure or always`.
+- You cannot [manually specify CI/CD variables](../jobs/index.md#specifying-variables-when-running-manual-jobs)
+ before running a manual trigger job.
- [Manual pipeline variables](../variables/index.md#override-a-defined-cicd-variable)
and [scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule)
are not passed to downstream pipelines by default. Use [trigger:forward](#triggerforward)
@@ -4079,6 +4110,8 @@ successfully complete before starting.
shows **pending** (**{status_pending}**) if the downstream pipeline status is
**waiting for manual action** (**{status_manual}**) due to manual jobs. By default,
jobs in later stages do not start until the trigger job completes.
+- If the downstream pipeline has a failed job, but the job uses [`allow_failure: true`](#allow_failure),
+ the downstream pipeline is considered successful and the trigger job shows **success**.
#### `trigger:forward`
@@ -4222,6 +4255,38 @@ variables:
- A global variable defined with `value` but no `description` behaves the same as
[`variables`](#variables).
+#### `variables:expand`
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353991) in GitLab 15.6 [with a flag](../../administration/feature_flags.md) named `ci_raw_variables_in_yaml_config`. Disabled by default.
+
+Use the `expand` keyword to configure a variable to be expandable or not.
+
+**Keyword type**: Global and job keyword. You can use it at the global level, and also at the job level.
+
+**Possible inputs**:
+
+- `true` (default): The variable is expandable.
+- `false`: The variable is not expandable.
+
+**Example of `variables:expand`**:
+
+```yaml
+variables:
+ VAR1: value1
+ VAR2: value2 $VAR1
+ VAR3:
+ value: value3 $VAR1
+ expand: false
+```
+
+- The result of `VAR2` is `value2 value1`.
+- The result of `VAR3` is `value3 $VAR1`.
+
+**Additional details**:
+
+- The `expand` keyword can only be used with the global and job-level `variables` keywords.
+ You can't use it with [`rules:variables`](#rulesvariables) or [`workflow:rules:variables`](#workflowrulesvariables).
+
### `when`
Use `when` to configure the conditions for when jobs run. If not defined in a job,
diff --git a/doc/cloud_seed/index.md b/doc/cloud_seed/index.md
index bf51da88cb4..04b560f7f87 100644
--- a/doc/cloud_seed/index.md
+++ b/doc/cloud_seed/index.md
@@ -13,6 +13,9 @@ Cloud Seed is an open-source program led
by [GitLab Incubation Engineering](https://about.gitlab.com/handbook/engineering/incubation/) in collaboration with
[Google Cloud](https://cloud.google.com/).
+Cloud Seed combines Heroku-like ease-of-use with hyper-cloud flexibility. We do this by using OAuth 2 to provision
+services on a hyper-cloud based on a foundation of Terraform and infrastructure-as-code to enable day 2 operations.
+
## Purpose
We believe that it should be **trivial** to deploy web applications (and other workloads) from GitLab to major cloud
diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md
index d3053a1ca5f..c19341a1404 100644
--- a/doc/development/api_graphql_styleguide.md
+++ b/doc/development/api_graphql_styleguide.md
@@ -1294,15 +1294,25 @@ class MyThingResolver < BaseResolver
end
```
-The final thing that is needed is that every field that uses this resolver needs
-to advertise the need for lookahead:
+The `LooksAhead` concern also provides basic support for preloading associations based on nested GraphQL field
+definitions. The [WorkItemsResolver](https://gitlab.com/gitlab-org/gitlab/-/blob/e824a7e39e08a83fb162db6851de147cf0bfe14a/app/graphql/resolvers/work_items_resolver.rb#L46)
+is a good example for this. `nested_preloads` is another method you can define to return a hash, but unlike the
+`preloads` method, the value for each hash key is another hash and not the list of associations to preload. So in
+the previous example, you could override `nested_preloads` like this:
```ruby
- # in ParentType
- field :my_things, MyThingType.connection_type, null: true,
- extras: [:lookahead], # Necessary
- resolver: MyThingResolver,
- description: 'My things.'
+class MyThingResolver < BaseResolver
+ # ...
+
+ def nested_preloads
+ {
+ root_field: {
+ nested_field1: :association_to_preload,
+ nested_field2: [:association1, :association2]
+ }
+ }
+ end
+end
```
For an example of real world use, please
@@ -1733,15 +1743,18 @@ there are no problems we need to inform the user of.
#### Failure (relevant to the user)
-An error that affects the **user** occurred. We refer to these as _mutation errors_. In
-this case there is typically no `thing` to return:
+An error that affects the **user** occurred. We refer to these as _mutation errors_.
+
+In a _create_ mutation there is typically no `thing` to return.
+
+In an _update_ mutation we return the current true state of `thing`. Developers may need to call `#reset` on the `thing` instance to ensure this happens.
```javascript
{
data: {
doTheThing: {
errors: ["you cannot touch the thing"],
- thing: null
+ thing: { .. }
}
}
}
@@ -2165,32 +2178,44 @@ end
```ruby
NameError: uninitialized constant Resolvers::GroupIssuesResolver
+
+ or
+
+ GraphQL::Pagination::Connections::ImplementationMissingError
```
+ though you might see something different.
+
To fix this, we must create a new file that encapsulates the connection type,
and then reference it using double quotes. This gives a delayed resolution,
and the proper connection type. For example:
+ [app/graphql/resolvers/base_issues_resolver.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/resolvers/base_issues_resolver.rb)
+ originally contained the line
+
```ruby
- module Types
- # rubocop: disable Graphql/AuthorizeTypes
- class IssueConnectionType < CountableConnectionType
- end
- end
+ type Types::IssueType.connection_type, null: true
+ ```
+
+ Running the specs locally for this file caused the
+ `NameError: uninitialized constant Resolvers::GroupIssuesResolver` error.
+
+ The fix was to create a new file, [app/graphql/types/issue_connection.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/types/issue_connection.rb) with the
+ line:
- Types::IssueConnectionType.prepend_mod_with('Types::IssueConnectionType')
+ ```ruby
+ Types::IssueConnection = Types::IssueType.connection_type
```
- in [types/issue_connection_type.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/types/issue_connection_type.rb)
- defines a new `Types::IssueConnectionType`, and is then referenced in
- [app/graphql/resolvers/base_issues_resolver.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/resolvers/base_issues_resolver.rb)
+ and in [app/graphql/resolvers/base_issues_resolver.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/resolvers/base_issues_resolver.rb)
+ we use the line
```ruby
type "Types::IssueConnection", null: true
```
Only use this style if you are having spec failures. This is not intended to be a new
- pattern that we use. This issue may disappear after we've upgraded to `2.x`.
+ pattern that we use. This issue should disappear after we've upgraded to `2.x`.
- There can be instances where a spec fails because the class is not loaded correctly.
It relates to the
diff --git a/doc/development/application_limits.md b/doc/development/application_limits.md
index edf159a116a..b64d25ccf64 100644
--- a/doc/development/application_limits.md
+++ b/doc/development/application_limits.md
@@ -38,7 +38,7 @@ It's recommended to create two separate migration script files.
desired limit using `create_or_update_plan_limit` migration helper, such as:
```ruby
- class InsertProjectHooksPlanLimits < Gitlab::Database::Migration[1.0]
+ class InsertProjectHooksPlanLimits < Gitlab::Database::Migration[2.0]
def up
create_or_update_plan_limit('project_hooks', 'default', 0)
create_or_update_plan_limit('project_hooks', 'free', 10)
diff --git a/doc/development/audit_event_guide/index.md b/doc/development/audit_event_guide/index.md
index 5c8938aa46a..dfa6d56b3b5 100644
--- a/doc/development/audit_event_guide/index.md
+++ b/doc/development/audit_event_guide/index.md
@@ -21,9 +21,9 @@ While any events could trigger an Audit Event, not all events should. In general
- Not attributable to one specific user.
- Not of specific interest to an administrator or owner persona.
- Are tracking information for product feature adoption.
-- Are covered in the direction page's discussion on [what is not planned](https://about.gitlab.com/direction/manage/compliance/audit-events/#what-is-not-planned-right-now).
+- Are covered in the direction page's discussion on [what is not planned](https://about.gitlab.com/direction/govern/compliance/audit-events/#what-is-not-planned-right-now).
-If you have any questions, please reach out to `@gitlab-org/manage/compliance` to see if an Audit Event, or some other approach, may be best for your event.
+If you have any questions, please reach out to `@gitlab-org/govern/compliance` to see if an Audit Event, or some other approach, may be best for your event.
## Audit Event Schemas
@@ -120,7 +120,7 @@ end
Because every audit event is persisted to the database, consider the amount of data we expect to generate, and the rate of generation, for new
audit events. For new audit events that will produce a lot of data in the database, consider adding a
[streaming-only audit event](#event-streaming) instead. If you have questions about this, feel free to ping
-`@gitlab-org/manage/compliance/backend` in an issue or merge request.
+`@gitlab-org/govern/compliance/backend` in an issue or merge request.
## Audit Event instrumentation flows
@@ -205,8 +205,10 @@ All new audit events must have a type definition stored in `config/audit_events/
To add a new audit event type:
-1. Create a new file in `config/audit_events/types/` with the filename matching the name of the event type. For example, a definition for the event type triggered when a
- user is added to a project might be stored in `config/audit_events/types/project_add_user.yml`.
+1. Create the YAML definition. You can either:
+ - Use the `bin/audit-event-type` CLI to create the YAML definition automatically.
+ - Perform manual steps to create a new file in `config/audit_events/types/` with the filename matching the name of the event type. For example,
+ a definition for the event type triggered when a user is added to a project might be stored in `config/audit_events/types/project_add_user.yml`.
1. Add contents to the file that conform to the [schema](#schema) defined in `config/audit_events/types/type_schema.json`.
1. Ensure that all calls to `Gitlab::Audit::Auditor` use the `name` defined in your file.
diff --git a/doc/development/backend/ruby_style_guide.md b/doc/development/backend/ruby_style_guide.md
index 9b5a68e4292..1a1c0db49f7 100644
--- a/doc/development/backend/ruby_style_guide.md
+++ b/doc/development/backend/ruby_style_guide.md
@@ -7,31 +7,32 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Ruby style guide
-This is a GitLab-specific style guide for Ruby code.
+This is a GitLab-specific style guide for Ruby code. Everything documented in this page can be [reopened for discussion](https://about.gitlab.com/handbook/values/#disagree-commit-and-disagree).
-Generally, if a style is not covered by [existing RuboCop rules or style guides](../contributing/style_guides.md#ruby-rails-rspec), it shouldn't be a blocker.
-Before adding a new cop to enforce a given style, make sure to discuss it with your team.
-When the style is approved by a backend EM or by a BE staff eng, add a new section to this page to
-document the new rule. For every new guideline, add it in a new section and link the discussion from the section's
-[version history note](../documentation/versions.md#add-a-version-history-item)
-to provide context and serve as a reference.
+We use [RuboCop](../rubocop_development_guide.md) to enforce Ruby style guide rules.
-See also [guidelines for reusing abstractions](../reusing_abstractions.md).
+Where a RuboCop rule is absent, refer to the following style guides as general guidelines to write idiomatic Ruby:
-Everything listed here can be [reopened for discussion](https://about.gitlab.com/handbook/values/#disagree-commit-and-disagree).
+- [Ruby Style Guide](https://github.com/rubocop/ruby-style-guide).
+- [Rails Style Guide](https://github.com/rubocop/rails-style-guide).
+- [RSpec Style Guide](https://github.com/rubocop/rspec-style-guide).
-## String literals quoting
+Generally, if a style is not covered by existing RuboCop rules or the above style guides, it shouldn't be a blocker.
-Due to the sheer amount of work to rectify, we do not care whether string
-literals are single, or double quoted.
+Some styles we have decided [no one should not have a strong opinion on](#styles-we-have-no-opinion-on).
-Previous discussions include:
+See also:
-- <https://gitlab.com/gitlab-org/gitlab-foss/-/issues/44234>
-- <https://gitlab.com/gitlab-org/gitlab-foss/-/issues/36076>
-- <https://gitlab.com/gitlab-org/gitlab/-/issues/198046>
+- [Guidelines for reusing abstractions](../reusing_abstractions.md).
+- [Test-specific style guides and best practices](../testing_guide/index.md).
+
+## Styles we have no rule for
-## Instance variable access using `attr_reader`
+These styles are not backed by a RuboCop rule.
+
+For every style added to this section, link the discussion from the section's [version history note](../documentation/versions.md#add-a-version-history-item) to provide context and serve as a reference.
+
+### Instance variable access using `attr_reader`
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52351) in GitLab 14.1.
@@ -84,11 +85,11 @@ Public attributes should only be used if they are accessed outside of the class.
There is not a strong opinion on what strategy is used when attributes are only
accessed internally, as long as there is consistency in related code.
-## Newlines style guide
+### Newlines style guide
-This style guide recommends best practices for newlines in Ruby code.
+In addition to the RuboCops `Layout/EmptyLinesAroundMethodBody` and `Cop/LineBreakAroundConditionalBlock` that enforce some newline styles, we have the following guidelines that are not backed by RuboCop.
-### Rule: separate code with newlines only to group together related logic
+#### Rule: separate code with newlines only to group together related logic
```ruby
# bad
@@ -111,9 +112,7 @@ def method
end
```
-### Rule: separate code and block with newlines
-
-#### Newline before block
+#### Rule: newline before block
```ruby
# bad
@@ -136,35 +135,11 @@ def method
end
```
-### Rule: Newline after block
-
-```ruby
-# bad
-def method
- if issue.save
- issue.send_email
- end
- render json: issue
-end
-```
-
-```ruby
-# good
-def method
- if issue.save
- issue.send_email
- end
-
- render json: issue
-end
-```
-
-#### Exception: no need for newline when code block starts or ends right inside another code block
+##### Exception: no need for a newline when code block starts or ends right inside another code block
```ruby
# bad
def method
-
if issue
if issue.valid?
@@ -172,7 +147,6 @@ def method
end
end
-
end
```
@@ -186,3 +160,18 @@ def method
end
end
```
+
+## Styles we have no opinion on
+
+If a RuboCop rule is proposed and we choose not to add it, we should document that decision in this guide so it is more discoverable and link the relevant discussion as a reference.
+
+### Quoting string literals
+
+Due to the sheer amount of work to rectify, we do not care whether string
+literals are single or double-quoted.
+
+Previous discussions include:
+
+- <https://gitlab.com/gitlab-org/gitlab-foss/-/issues/44234>
+- <https://gitlab.com/gitlab-org/gitlab-foss/-/issues/36076>
+- <https://gitlab.com/gitlab-org/gitlab/-/issues/198046>
diff --git a/doc/development/cached_queries.md b/doc/development/cached_queries.md
index fbb857106be..e4625a50d79 100644
--- a/doc/development/cached_queries.md
+++ b/doc/development/cached_queries.md
@@ -87,7 +87,7 @@ and the number of executed cached queries:
![Performance Bar Database Queries](img/performance_bar_members_page.png)
-The page included 55 cached queries. Clicking the number displays a modal window
+The page included 55 cached queries. Selecting the number displays a modal window
with more details about queries. Cached queries are marked with the `cached` label
below the query. You can see multiple duplicate cached queries in this modal window:
diff --git a/doc/development/cascading_settings.md b/doc/development/cascading_settings.md
index 22f146c3f5a..1a0f0ec5b5f 100644
--- a/doc/development/cascading_settings.md
+++ b/doc/development/cascading_settings.md
@@ -38,7 +38,7 @@ Settings are not cascading by default. To define a cascading setting, take the f
`application_settings`.
```ruby
- class AddDelayedProjectRemovalCascadingSetting < Gitlab::Database::Migration[1.0]
+ class AddDelayedProjectRemovalCascadingSetting < Gitlab::Database::Migration[2.0]
include Gitlab::Database::MigrationHelpers::CascadingNamespaceSettings
enable_lock_retries!
diff --git a/doc/development/changelog.md b/doc/development/changelog.md
index 7dc3ae0a80b..27bcffe7560 100644
--- a/doc/development/changelog.md
+++ b/doc/development/changelog.md
@@ -41,6 +41,10 @@ vendor field to be `gitlab` to avoid cve matching old versions.
Changelog: changed
```
+If your merge request has multiple commits,
+[make sure to add the `Changelog` entry to the first commit](changelog.md#how-to-generate-a-changelog-entry).
+This ensures that the correct entry is generated when commits are squashed.
+
### Overriding the associated merge request
GitLab automatically links the merge request to the commit when generating the
diff --git a/doc/development/chatops_on_gitlabcom.md b/doc/development/chatops_on_gitlabcom.md
index 16dc17dd229..c9cb2591e4e 100644
--- a/doc/development/chatops_on_gitlabcom.md
+++ b/doc/development/chatops_on_gitlabcom.md
@@ -23,7 +23,7 @@ To request access to ChatOps on GitLab.com:
with one of the following methods (Okta is not supported):
- The same username you use on GitLab.com. You may have to choose a different username later.
- - Clicking the **Sign in with Google** button to sign in with your GitLab.com email address.
+ - Selecting the **Sign in with Google** button to sign in with your GitLab.com email address.
1. Confirm that your username in [Internal GitLab for Operations](https://ops.gitlab.net/)
is the same as your username in [GitLab.com](https://gitlab.com/). If the usernames
@@ -31,7 +31,7 @@ To request access to ChatOps on GitLab.com:
1. Comment in your onboarding issue, and tag your onboarding buddy and your manager.
Request they add you to the `ops` ChatOps project by running this command
- in the `#chat-ops-test` Slack channel, replacing `<username>` with your username:
+ in the `#chat-ops-test` Slack channel, replacing `<username>` with your GitLab.com username:
`/chatops run member add <username> gitlab-com/chatops --ops`
```plaintext
diff --git a/doc/development/cicd/index.md b/doc/development/cicd/index.md
index cd31038ddd1..73ece709b8d 100644
--- a/doc/development/cicd/index.md
+++ b/doc/development/cicd/index.md
@@ -33,7 +33,7 @@ On the left side we have the events that can trigger a pipeline based on various
- A `git push` is the most common event that triggers a pipeline.
- The [Web API](../../api/pipelines.md#create-a-new-pipeline).
-- A user clicking the "Run pipeline" button in the UI.
+- A user selecting the "Run pipeline" button in the UI.
- When a [merge request is created or updated](../../ci/pipelines/merge_request_pipelines.md).
- When an MR is added to a [Merge Train](../../ci/pipelines/merge_trains.md#merge-trains).
- A [scheduled pipeline](../../ci/pipelines/schedules.md).
diff --git a/doc/development/cicd/templates.md b/doc/development/cicd/templates.md
index 85ac58e749d..6bc6c57e809 100644
--- a/doc/development/cicd/templates.md
+++ b/doc/development/cicd/templates.md
@@ -418,7 +418,7 @@ is updated in a major version GitLab release.
Every CI/CD template must also have metrics defined to track their use. The CI/CD template monthly usage report
can be found in [Sisense (GitLab team members only)](https://app.periscopedata.com/app/gitlab/785953/Pipeline-Authoring-Dashboard?widget=13440051&udv=0).
-Double click a template to see the graph for that single template.
+Select a template to see the graph for that single template.
To add a metric definition for a new template:
diff --git a/doc/development/code_review.md b/doc/development/code_review.md
index 5b745f06d22..90f33319365 100644
--- a/doc/development/code_review.md
+++ b/doc/development/code_review.md
@@ -28,6 +28,13 @@ The reviewer can:
- Give you a second opinion on the chosen solution and implementation.
- Help look for bugs, logic problems, or uncovered edge cases.
+If the merge request is trivial (for example, fixing a typo or a tiny refactor that doesn't change the behavior or any data),
+you can skip the reviewer step and directly ask a [maintainer](https://about.gitlab.com/handbook/engineering/workflow/code-review/#maintainer).
+Otherwise, a merge request should always be first reviewed by a reviewer in each
+[category (e.g. backend, database)](#approval-guidelines)
+the MR touches, as maintainers may not have the relevant domain knowledge, and
+also to spread the workload.
+
For assistance with security scans or comments, include the Application Security Team (`@gitlab-com/gl-security/appsec`).
The reviewers use the [reviewer functionality](../user/project/merge_requests/getting_started.md#reviewer) in the sidebar.
@@ -56,8 +63,8 @@ We make the following assumption with regards to automatically being considered
- Team members working in a specific stage/group (for example, create: source code) are considered domain experts for that area of the app they work on.
- Team members working on a specific feature (for example, search) are considered domain experts for that feature.
-We default to assigning reviews to team members with domain expertise.
-When a suitable [domain expert](#domain-experts) isn't available, you can choose any team member to review the MR, or follow the [Reviewer roulette](#reviewer-roulette) recommendation.
+We default to assigning reviews to team members with domain expertise for code reviews. For UX reviews we default to the recommended designer from the Reviewer roulette.
+When a suitable [domain expert](#domain-experts) isn't available, you can choose any team member to review the MR, or follow the [Reviewer roulette](#reviewer-roulette) recommendation (see above for UX reviews).
To find a domain expert:
@@ -77,7 +84,7 @@ Reviewer roulette is an internal tool for use on GitLab.com, and not available f
The [Danger bot](dangerbot.md) randomly picks a reviewer and a maintainer for
each area of the codebase that your merge request seems to touch. It makes
**recommendations** for developer reviewers and you should override it if you think someone else is a better
-fit. User-facing changes are required to have a UX review, too. Default to the recommended UX reviewer suggested.
+fit. User-facing changes are also required to have a UX review, even if it's behind a feature flag. Default to the recommended UX reviewer suggested.
It picks reviewers and maintainers from the list at the
[engineering projects](https://about.gitlab.com/handbook/engineering/projects/)
@@ -147,7 +154,7 @@ with [domain expertise](#domain-experts).
| `~workhorse` changes | [Workhorse maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_workhorse). |
| `~frontend` changes (*1*) | [Frontend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_maintainers_frontend). |
| `~UX` user-facing changes (*3*) | [Product Designer](https://about.gitlab.com/handbook/engineering/projects/#gitlab_reviewers_UX). Refer to the [design and user interface guidelines](contributing/design.md) for details. |
-| Adding a new JavaScript library (*1*) | - [Frontend foundations member](https://about.gitlab.com/direction/ecosystem/foundations/) if the library significantly increases the [bundle size](https://gitlab.com/gitlab-org/frontend/playground/webpack-memory-metrics/-/blob/master/doc/report.md).<br/>- A [legal department member](https://about.gitlab.com/handbook/legal/) if the license used by the new library hasn't been approved for use in GitLab.<br/><br/>More information about license compatibility can be found in our [GitLab Licensing and Compatibility documentation](licensing.md). |
+| Adding a new JavaScript library (*1*) | - [Frontend foundations member](https://about.gitlab.com/direction/manage/foundations/) if the library significantly increases the [bundle size](https://gitlab.com/gitlab-org/frontend/playground/webpack-memory-metrics/-/blob/master/doc/report.md).<br/>- A [legal department member](https://about.gitlab.com/handbook/legal/) if the license used by the new library hasn't been approved for use in GitLab.<br/><br/>More information about license compatibility can be found in our [GitLab Licensing and Compatibility documentation](licensing.md). |
| A new dependency or a file system change | - [Distribution team member](https://about.gitlab.com/company/team/). See how to work with the [Distribution team](https://about.gitlab.com/handbook/engineering/development/enablement/systems/distribution/#how-to-work-with-distribution) for more details.<br/>- For Rubygems, request an [AppSec review](gemfile.md#request-an-appsec-review). |
| `~documentation` or `~UI text` changes | [Technical writer](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments) based on assignments in the appropriate [DevOps stage group](https://about.gitlab.com/handbook/product/categories/#devops-stages). |
| Changes to development guidelines | Follow the [review process](development_processes.md#development-guidelines-review) and get the approvals accordingly. |
@@ -212,8 +219,8 @@ See the [test engineering process](https://about.gitlab.com/handbook/engineering
##### Security
-1. I have confirmed that if this MR contains changes to processing or storing of credentials or tokens, authorization, and authentication methods, or other items described in [the security review guidelines](https://about.gitlab.com/handbook/engineering/security/#when-to-request-a-security-review), I have added the `~security` label and I have `@`-mentioned `@gitlab-com/gl-security/appsec`.
-1. I have reviewed the documentation regarding [internal application security reviews](https://about.gitlab.com/handbook/engineering/security/#internal-application-security-reviews) for **when** and **how** to request a security review and requested a security review if this is warranted for this change.
+1. I have confirmed that if this MR contains changes to processing or storing of credentials or tokens, authorization, and authentication methods, or other items described in [the security review guidelines](https://about.gitlab.com/handbook/security/#when-to-request-a-security-review), I have added the `~security` label and I have `@`-mentioned `@gitlab-com/gl-security/appsec`.
+1. I have reviewed the documentation regarding [internal application security reviews](https://about.gitlab.com/handbook/security/#internal-application-security-reviews) for **when** and **how** to request a security review and requested a security review if this is warranted for this change.
##### Deployment
@@ -508,7 +515,7 @@ people who add commits to an MR are not authorized to approve the merge request,
so they must seek a maintainer who has not contributed to the MR to approve the MR before it can be merged.
This policy is in place to satisfy the CHG-04 control of the GitLab
-[Change Management Controls](https://about.gitlab.com/handbook/engineering/security/security-assurance/security-compliance/guidance/change-management.html).
+[Change Management Controls](https://about.gitlab.com/handbook/security/security-assurance/security-compliance/guidance/change-management.html).
To implement this policy in `gitlab-org/gitlab`, we have enabled the following
settings to ensure MRs get an approval from a top-level CODEOWNERS maintainer:
@@ -518,6 +525,9 @@ settings to ensure MRs get an approval from a top-level CODEOWNERS maintainer:
- [Prevent editing approval rules in merge requests](../user/project/merge_requests/approvals/settings.md#prevent-editing-approval-rules-in-merge-requests).
- [Remove all approvals when commits are added to the source branch](../user/project/merge_requests/approvals/settings.md#remove-all-approvals-when-commits-are-added-to-the-source-branch)
+To update the code owners in the `CODEOWNERS` file for `gitlab-org/gitlab`, follow
+the process explained in the [code owners approvals handbook section](https://about.gitlab.com/handbook/engineering/workflow/code-review/#code-owner-approvals).
+
There are scenarios such as rebasing locally or applying suggestions that are considered
the same as adding a commit and could reset existing approvals. Approvals are not removed
when rebasing from the UI or with the [`/rebase` quick action](../user/project/quick_actions.md).
diff --git a/doc/development/contributing/design.md b/doc/development/contributing/design.md
index 9e54b92337a..e6b6b56cf73 100644
--- a/doc/development/contributing/design.md
+++ b/doc/development/contributing/design.md
@@ -24,7 +24,7 @@ screenshots (or videos) of your changes in the description, as explained in our
[MR workflow](merge_request_workflow.md). These screenshots/videos are very helpful
for all reviewers and can speed up the review process, especially if the changes
are small.
-- Attach the ~UX label to any merge request that impacts the user experience. This will enable Product Designers to [review](https://about.gitlab.com/handbook/engineering/ux/product-designer/mr-reviews/#stage-group-mrs/) any user facing changes.
+- Attach the ~UX label to any merge request that impacts the user experience. This will enable Product Designers to [review](https://about.gitlab.com/handbook/product/ux/product-designer/mr-reviews/#stage-group-mrs/) any user facing changes.
- Assign the Product Designer suggested by Reviewer Roulette as the reviewer of your merge request. The reviewer does not have to be the domain expert unless this is a community contribution.
## Checklist
@@ -64,7 +64,7 @@ Check visual design properties using your browser's _elements inspector_ ([Chrom
guidelines.
- _Optionally_ consider [dark mode](../../user/profile/preferences.md#dark-mode). [^1]
- [^1]: You're not required to design for [dark mode](../../user/profile/preferences.md#dark-mode) while the feature is in [alpha](../../policy/alpha-beta-support.md#alpha-features). The [UX Foundations team](https://about.gitlab.com/direction/ecosystem/foundations/) plans to improve the dark mode in the future. Until we integrate [Pajamas](https://design.gitlab.com/) components into the product and the underlying design strategy is in place to support dark mode, we cannot guarantee that we won't introduce bugs and debt to this mode. At your discretion, evaluate the need to create dark mode patches.
+ [^1]: You're not required to design for [dark mode](../../user/profile/preferences.md#dark-mode) while the feature is in [alpha](../../policy/alpha-beta-support.md#alpha-features). The [UX Foundations team](https://about.gitlab.com/direction/manage/foundations/) plans to improve the dark mode in the future. Until we integrate [Pajamas](https://design.gitlab.com/) components into the product and the underlying design strategy is in place to support dark mode, we cannot guarantee that we won't introduce bugs and debt to this mode. At your discretion, evaluate the need to create dark mode patches.
### States
diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md
index cbccd832d78..8c0d18f877b 100644
--- a/doc/development/contributing/index.md
+++ b/doc/development/contributing/index.md
@@ -100,7 +100,7 @@ If you have any questions or need help, visit [Getting Help](https://about.gitla
communicate with the GitLab community. GitLab prefers [asynchronous communication](https://about.gitlab.com/handbook/communication/#internal-communication) over real-time communication.
We do encourage you to connect and hang out with us. GitLab has a Gitter room dedicated for [contributors](https://gitter.im/gitlab/contributors), which is bridged with our
-internal Slack. We actively monitor this channel. There is also a community-run [Discord server](https://discord.gg/S4cwz9sR8u) where you can
+internal Slack. We actively monitor this channel. There is also a community-run [Discord server](http://discord.gg/gitlab) where you can
find other contributors in the `#contributors` channel.
Thanks for your contribution!
diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md
index b40df01cbe9..f2c06e289c9 100644
--- a/doc/development/contributing/merge_request_workflow.md
+++ b/doc/development/contributing/merge_request_workflow.md
@@ -38,6 +38,10 @@ and see the [Development section](../../index.md) for the required guidelines.
If you find an issue, please submit a merge request with a fix or improvement,
if you can, and include tests.
+NOTE:
+Consider placing your code behind a feature flag if you think it might affect production availability.
+Not sure? Read [When to use feature flags](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/#when-to-use-feature-flags).
+
If the change is non-trivial, we encourage you to
start a discussion with [a product manager or a member of the team](https://about.gitlab.com/handbook/product/categories/).
You can do
diff --git a/doc/development/contributing/style_guides.md b/doc/development/contributing/style_guides.md
index 9d04e1590d0..fe1aa8449c2 100644
--- a/doc/development/contributing/style_guides.md
+++ b/doc/development/contributing/style_guides.md
@@ -108,117 +108,6 @@ pre-push:
For more information, check out [Lefthook documentation Skipping commands section](https://github.com/evilmartians/lefthook/blob/master/docs/full_guide.md#skipping-commands).
-## Ruby, Rails, RSpec
-
-Our codebase style is defined and enforced by [RuboCop](https://github.com/rubocop-hq/rubocop).
-
-You can check for any offenses locally with `bundle exec rubocop --parallel`.
-On the CI, this is automatically checked by the `static-analysis` jobs.
-
-In addition, you can [integrate RuboCop](../developing_with_solargraph.md) into
-supported IDEs using the [Solargraph](https://github.com/castwide/solargraph) gem.
-
-For RuboCop rules that we have not taken a decision on yet, we follow the
-[Ruby Style Guide](https://github.com/rubocop-hq/ruby-style-guide),
-[Rails Style Guide](https://github.com/rubocop-hq/rails-style-guide), and
-[RSpec Style Guide](https://github.com/rubocop-hq/rspec-style-guide) as general
-guidelines to write idiomatic Ruby/Rails/RSpec, but reviewers/maintainers should
-be tolerant and not too pedantic about style.
-
-Similarly, some RuboCop rules are currently disabled, and for those,
-reviewers/maintainers must not ask authors to use one style or the other, as both
-are accepted. This isn't an ideal situation since this leaves space for
-[bike-shedding](https://en.wiktionary.org/wiki/bikeshedding), and ideally we
-should enable all RuboCop rules to avoid style-related
-discussions/nitpicking/back-and-forth in reviews. There are some styles that
-commonly come up in reviews that are not enforced, the
-[GitLab Ruby style guide](../backend/ruby_style_guide.md) includes a non-exhaustive
-list of these topics.
-
-Additionally, we have a dedicated
-[newlines style guide](../newlines_styleguide.md), as well as dedicated
-[test-specific style guides and best practices](../testing_guide/index.md).
-
-### Creating new RuboCop cops
-
-Typically it is better for the linting rules to be enforced programmatically as it
-reduces the aforementioned [bike-shedding](https://en.wiktionary.org/wiki/bikeshedding).
-
-To that end, we encourage creation of new RuboCop rules in the codebase.
-
-We maintain Cops across several Ruby code bases, and not all of them are
-specific to the GitLab application.
-When creating a new cop that could be applied to multiple applications, we encourage you
-to add it to our [GitLab Styles](https://gitlab.com/gitlab-org/gitlab-styles) gem.
-If the Cop targets rules that only apply to the main GitLab application,
-it should be added to [GitLab](https://gitlab.com/gitlab-org/gitlab) instead.
-
-### Cop grace period
-
-A cop is in a "grace period" if it is enabled and has `Details: grace period` defined in its TODO YAML configuration.
-
-On the default branch, all of the offenses from cops in the ["grace period"](../rake_tasks.md#run-rubocop-in-graceful-mode) will not fail the RuboCop CI job. The job will notify Slack in the `#f_rubocop` channel when offenses have been silenced in the scheduled pipeline. However, on merge request pipelines, the RuboCop job will fail.
-
-A grace period can safely be lifted as soon as there are no warnings for 2 weeks in the `#f_rubocop` channel on Slack.
-
-### Enabling a new cop
-
-1. Enable the new cop in `.rubocop.yml` (if not already done via [`gitlab-styles`](https://gitlab.com/gitlab-org/ruby/gems/gitlab-styles)).
-1. [Generate TODOs for the new cop](../rake_tasks.md#generate-initial-rubocop-todo-list).
-1. [Set the new cop to "grace period"](#cop-grace-period).
-1. Create an issue to fix TODOs and encourage Community contributions (via ~"good for new contributors" and/or ~"Seeking community contributions"). [See some examples](https://gitlab.com/gitlab-org/gitlab/-/issues/?sort=created_date&state=opened&label_name%5B%5D=good%20for%20new%20contributors&label_name%5B%5D=static%20code%20analysis&first_page_size=20).
-1. Create an issue to remove "grace period" after 2 weeks silence in `#f_rubocop` Slack channel. ([See an example](https://gitlab.com/gitlab-org/gitlab/-/issues/374903).)
-
-### Silenced offenses
-
-When offenses are silenced for cops in ["grace period"](#cop-grace-period),
-the `#f_rubocop` Slack channel receives a notification message every two hours.
-
-To fix this issue:
-
-1. Find cops with silenced offenses in the linked CI job.
-1. [Generate TODOs](../rake_tasks.md#generate-initial-rubocop-todo-list) for these cops.
-
-#### RuboCop node pattern
-
-When creating [node patterns](https://docs.rubocop.org/rubocop-ast/node_pattern.html) to match
-Ruby's AST, you can use [`scripts/rubocop-parse`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/rubocop-parse)
-to display the AST of a Ruby expression, to help you create the matcher.
-See also [!97024](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97024).
-
-### Resolving RuboCop exceptions
-
-When the number of RuboCop exceptions exceed the default [`exclude-limit` of 15](https://docs.rubocop.org/rubocop/1.2/usage/basic_usage.html#command-line-flags),
-we may want to resolve exceptions over multiple commits. To minimize confusion,
-we should track our progress through the exception list.
-
-The preferred way to [generate the initial list or a list for specific RuboCop rules](../rake_tasks.md#generate-initial-rubocop-todo-list)
-is to run the Rake task `rubocop:todo:generate`:
-
-```shell
-# Initial list
-bundle exec rake rubocop:todo:generate
-
-# List for specific RuboCop rules
-bundle exec rake 'rubocop:todo:generate[Gitlab/NamespacedClass,Lint/Syntax]'
-```
-
-This Rake task creates or updates the exception list in `.rubocop_todo/`. For
-example, the configuration for the RuboCop rule `Gitlab/NamespacedClass` is
-located in `.rubocop_todo/gitlab/namespaced_class.yml`.
-
-Make sure to commit any changes in `.rubocop_todo/` after running the Rake task.
-
-### Reveal existing RuboCop exceptions
-
-To reveal existing RuboCop exceptions in the code that have been excluded via `.rubocop_todo.yml` and
-`.rubocop_todo/**/*.yml`, set the environment variable `REVEAL_RUBOCOP_TODO` to `1`.
-
-This allows you to reveal existing RuboCop exceptions during your daily work cycle and fix them along the way.
-
-NOTE:
-Define permanent `Exclude`s in `.rubocop.yml` instead of `.rubocop_todo/**/*.yml`.
-
## Database migrations
See the dedicated [Database Migrations Style Guide](../migration_style_guide.md).
@@ -231,6 +120,10 @@ See the dedicated [JS Style Guide](../fe_guide/style/javascript.md).
See the dedicated [SCSS Style Guide](../fe_guide/style/scss.md).
+## Ruby
+
+See the dedicated [Ruby Style Guide](../backend/ruby_style_guide.md).
+
## Go
See the dedicated [Go standards and style guidelines](../go_guide/index.md).
diff --git a/doc/development/database/adding_database_indexes.md b/doc/development/database/adding_database_indexes.md
index 040c6780316..d4cd807ef22 100644
--- a/doc/development/database/adding_database_indexes.md
+++ b/doc/development/database/adding_database_indexes.md
@@ -215,6 +215,42 @@ def down
end
```
+## Indexes for partitioned tables
+
+Indexes [cannot be created](https://www.postgresql.org/docs/15/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE-MAINTENANCE)
+**concurrently** on a partitioned table. You must use `CONCURRENTLY` to avoid service disruption in a hot system.
+
+To create an index on a partitioned table, use `add_concurrent_partitioned_index`, provided by the database team.
+
+Under the hood, `add_concurrent_partitioned_index`:
+
+1. Creates indexes on each partition using `CONCURRENTLY`.
+1. Creates an index on the parent table.
+
+A Rails migration example:
+
+```ruby
+# in db/post_migrate/
+
+class AddIndexToPartitionedTable < Gitlab::Database::Migration[2.0]
+ include Gitlab::Database::PartitioningMigrationHelpers
+
+ disable_ddl_transaction!
+
+ TABLE_NAME = :table_name
+ COLUMN_NAMES = [:partition_id, :id]
+ INDEX_NAME = :index_name
+
+ def up
+ add_concurrent_partitioned_index(TABLE_NAME, COLUMN_NAMES, name: INDEX_NAME)
+ end
+
+ def down
+ remove_concurrent_partitioned_index_by_name(TABLE_NAME, INDEX_NAME)
+ end
+end
+```
+
## Create indexes asynchronously
For very large tables, index creation can be a challenge to manage.
diff --git a/doc/development/database/background_migrations.md b/doc/development/database/background_migrations.md
index 8e6f29b9eb8..fe62bbc6b14 100644
--- a/doc/development/database/background_migrations.md
+++ b/doc/development/database/background_migrations.md
@@ -236,7 +236,7 @@ Next we need a post-deployment migration that schedules the migration for
existing data.
```ruby
-class ScheduleExtractIntegrationsUrl < Gitlab::Database::Migration[1.0]
+class ScheduleExtractIntegrationsUrl < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
MIGRATION = 'ExtractIntegrationsUrl'
@@ -263,7 +263,7 @@ jobs and manually run on any un-migrated rows. Such a migration would look like
this:
```ruby
-class ConsumeRemainingExtractIntegrationsUrlJobs < Gitlab::Database::Migration[1.0]
+class ConsumeRemainingExtractIntegrationsUrlJobs < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
def up
diff --git a/doc/development/database/batched_background_migrations.md b/doc/development/database/batched_background_migrations.md
index a48a9c42e27..ca11e9c8dd3 100644
--- a/doc/development/database/batched_background_migrations.md
+++ b/doc/development/database/batched_background_migrations.md
@@ -219,6 +219,7 @@ In this example, `copy_from` returns `name`, and `copy_to` returns `name_convert
```ruby
class CopyColumnUsingBackgroundMigrationJob < BatchedMigrationJob
job_arguments :copy_from, :copy_to
+ operation_name :update_all
def perform
from_column = connection.quote_column_name(copy_from)
@@ -226,7 +227,7 @@ class CopyColumnUsingBackgroundMigrationJob < BatchedMigrationJob
assignment_clause = "#{to_column} = #{from_column}"
- each_sub_batch(operation_name: :update_all) do |relation|
+ each_sub_batch do |relation|
relation.update_all(assignment_clause)
end
end
@@ -267,9 +268,10 @@ In the second (filtered) example, we know exactly 100 will be updated with each
```ruby
class BackfillNamespaceType < BatchedMigrationJob
scope_to ->(relation) { relation.where(type: nil) }
+ operation_name :update_all
def perform
- each_sub_batch(operation_name: :update_all) do |sub_batch|
+ each_sub_batch do |sub_batch|
sub_batch.update_all(type: 'User')
end
end
@@ -327,9 +329,10 @@ background migration.
# self.table_name = 'routes'
# end
+ operation_name :update_all
+
def perform
each_sub_batch(
- operation_name: :update_all,
batching_scope: -> (relation) { relation.where("source_type <> 'UnusedType'") }
) do |sub_batch|
sub_batch.update_all('namespace_id = source_id')
@@ -483,8 +486,10 @@ module Gitlab
class BatchProjectsWithIssues < Gitlab::BackgroundMigration::BatchedMigrationJob
include Gitlab::Database::DynamicModelHelpers
+ operation_name :backfill_issues
+
def perform
- distinct_each_batch(operation_name: :backfill_issues) do |batch|
+ distinct_each_batch do |batch|
project_ids = batch.pluck(batch_column)
# do something with the distinct project_ids
end
diff --git a/doc/development/database/database_debugging.md b/doc/development/database/database_debugging.md
index 4dc6a3bdcfa..0d6e9955a19 100644
--- a/doc/development/database/database_debugging.md
+++ b/doc/development/database/database_debugging.md
@@ -49,9 +49,11 @@ bundle exec rake db:reset RAILS_ENV=test
- `bundle exec rake db:migrate RAILS_ENV=development`: Execute any pending migrations that you may have picked up from a MR
- `bundle exec rake db:migrate:status RAILS_ENV=development`: Check if all migrations are `up` or `down`
-- `bundle exec rake db:migrate:down VERSION=20170926203418 RAILS_ENV=development`: Tear down a migration
-- `bundle exec rake db:migrate:up VERSION=20170926203418 RAILS_ENV=development`: Set up a migration
-- `bundle exec rake db:migrate:redo VERSION=20170926203418 RAILS_ENV=development`: Re-run a specific migration
+- `bundle exec rake db:migrate:down:main VERSION=20170926203418 RAILS_ENV=development`: Tear down a migration
+- `bundle exec rake db:migrate:up:main VERSION=20170926203418 RAILS_ENV=development`: Set up a migration
+- `bundle exec rake db:migrate:redo:main VERSION=20170926203418 RAILS_ENV=development`: Re-run a specific migration
+
+Replace `main` in the above commands to execute agains the `ci` database instead of `main`.
## Manually access the database
diff --git a/doc/development/database/database_migration_pipeline.md b/doc/development/database/database_migration_pipeline.md
index 148dc1e94a0..06e16b4c7f1 100644
--- a/doc/development/database/database_migration_pipeline.md
+++ b/doc/development/database/database_migration_pipeline.md
@@ -22,34 +22,54 @@ For security reasons, access to the pipeline is restricted to database maintaine
When the pipeline starts, a bot notifies you with a comment in the merge request.
When it finishes, the comment gets updated with the test results.
-There are three sections which are described below.
+
+The comment contains testing information for both the `main` and `ci` databases.
+Each database tested has four sections which are described below.
## Summary
The first section of the comment contains a summary of the test results, including:
-| Result | Description |
-|-------------------|---------------------------------------------------------------------------------------------------------------------|
-| Warnings | Highlights critical issues such as exceptions or long-running queries. |
-| Migrations | The time each migration took to complete, whether it was successful, and the increment in the size of the database. |
-| Runtime histogram | Expand this section to see a histogram of query runtimes across all migrations. |
+- **Warnings** - Highlights critical issues such as exceptions or long-running queries.
+- **Migrations** - The time each migration took to complete, whether it was successful,
+ and the increment in the size of the database.
+- **Runtime histogram** - Expand this section to see a histogram of query runtimes across all migrations.
## Migration details
The next section of the comment contains detailed information for each migration, including:
-| Result | Description |
-|-------------------|-------------------------------------------------------------------------------------------------------------------------|
-| Details | The type of migration, total duration, and database size change. |
-| Queries | Every query executed during the migration, along with the number of calls, timings, and the number of the changed rows. |
-| Runtime histogram | Indicates the distribution of query times for the migration. |
+- **Details** - The type of migration, total duration, and database size change.
+- **Queries** - Every query executed during the migration, along with the number of
+ calls, timings, and the number of the changed rows.
+- **Runtime histogram** - Indicates the distribution of query times for the migration.
+
+## Background migration details
+
+The next section of the comment contains detailed information about each batched background migration, including:
+
+- **Sampling information** - The number of batches sampled during this test run.
+ Sampled batches are chosen uniformly across the table's ID range. Sampling runs
+ for 30 minutes, split evenly across each background migration to test.
+- **Aggregated query information** - Aggregate data about each query executed across
+ all the sampled batches, along with the number of calls, timings, and the number of changed rows.
+- **Batch runtime histogram** - A histogram of timings for each sampled batch
+ from the background migration.
+- **Query runtime histogram** - A histogram of timings for all queries executed
+ in any batch of this background migration.
## Clone details and artifacts
Some additional information is included at the bottom of the comment:
-| Result | Description |
-|----------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| Migrations pending on GitLab.com | A summary of migrations not deployed yet to GitLab.com. This information is useful when testing a migration that was merged but not deployed yet. |
-| Clone details | A link to the `Postgres.ai` thin clone created for this testing pipeline, along with information about its expiry. This can be used to further explore the results of running the migration. Only accessible by database maintainers or with an access request. |
-| Artifacts | A link to the pipeline's artifacts. Full query logs for each migration (ending in `.log`) are available there and only accessible by database maintainers or with an access request. |
+- **Migrations pending on GitLab.com** - A summary of migrations not deployed yet
+ to GitLab.com. This information is useful when testing a migration that was merged
+ but not deployed yet.
+- **Clone details** - A link to the `Postgres.ai` thin clone created for this
+ testing pipeline, along with information about its expiry. This can be used to
+ further explore the results of running the migration. Only accessible by
+ database maintainers or with an access request.
+- **Artifacts** - A link to the pipeline's artifacts. Full query logs for each
+ migration (ending in `.log`) are available there, and only accessible by
+ database maintainers or with an access request. Details of the specific
+ batched background migration batches sampled are also available.
diff --git a/doc/development/database/loose_foreign_keys.md b/doc/development/database/loose_foreign_keys.md
index abf66368548..962cd2602bc 100644
--- a/doc/development/database/loose_foreign_keys.md
+++ b/doc/development/database/loose_foreign_keys.md
@@ -192,7 +192,7 @@ trigger needs to be configured only once. If the model already has at least one
`loose_foreign_key` definition, then this step can be skipped:
```ruby
-class TrackProjectRecordChanges < Gitlab::Database::Migration[1.0]
+class TrackProjectRecordChanges < Gitlab::Database::Migration[2.0]
include Gitlab::Database::MigrationHelpers::LooseForeignKeyHelpers
enable_lock_retries!
@@ -227,7 +227,7 @@ trigger. If the foreign key is deleted earlier, there is a good chance of
introducing data inconsistency which needs manual cleanup:
```ruby
-class RemoveProjectsCiPipelineFk < Gitlab::Database::Migration[1.0]
+class RemoveProjectsCiPipelineFk < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
def up
@@ -258,7 +258,7 @@ records in the database.
Migration for removing the trigger:
```ruby
-class UnTrackProjectRecordChanges < Gitlab::Database::Migration[1.0]
+class UnTrackProjectRecordChanges < Gitlab::Database::Migration[2.0]
include Gitlab::Database::MigrationHelpers::LooseForeignKeyHelpers
enable_lock_retries!
@@ -278,7 +278,7 @@ table however, there is still a chance for having leftover pending records in th
must be removed with an inline data migration.
```ruby
-class RemoveLeftoverProjectDeletions < Gitlab::Database::Migration[1.0]
+class RemoveLeftoverProjectDeletions < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
def up
diff --git a/doc/development/database/not_null_constraints.md b/doc/development/database/not_null_constraints.md
index dccaff2df00..53ab9a83d60 100644
--- a/doc/development/database/not_null_constraints.md
+++ b/doc/development/database/not_null_constraints.md
@@ -25,7 +25,7 @@ For example, consider a migration that creates a table with two `NOT NULL` colum
`db/migrate/20200401000001_create_db_guides.rb`:
```ruby
-class CreateDbGuides < Gitlab::Database::Migration[1.0]
+class CreateDbGuides < Gitlab::Database::Migration[2.0]
def change
create_table :db_guides do |t|
t.bigint :stars, default: 0, null: false
@@ -44,7 +44,7 @@ For example, consider a migration that adds a new `NOT NULL` column `active` to
`db/migrate/20200501000001_add_active_to_db_guides.rb`:
```ruby
-class AddExtendedTitleToSprints < Gitlab::Database::Migration[1.0]
+class AddExtendedTitleToSprints < Gitlab::Database::Migration[2.0]
def change
add_column :db_guides, :active, :boolean, default: true, null: false
end
@@ -116,7 +116,7 @@ with `validate: false` in a post-deployment migration,
`db/post_migrate/20200501000001_add_not_null_constraint_to_epics_description.rb`:
```ruby
-class AddNotNullConstraintToEpicsDescription < Gitlab::Database::Migration[1.0]
+class AddNotNullConstraintToEpicsDescription < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
def up
@@ -147,7 +147,7 @@ so we add a post-deployment migration for the 13.0 milestone (current),
`db/post_migrate/20200501000002_cleanup_epics_with_null_description.rb`:
```ruby
-class CleanupEpicsWithNullDescription < Gitlab::Database::Migration[1.0]
+class CleanupEpicsWithNullDescription < Gitlab::Database::Migration[2.0]
# With BATCH_SIZE=1000 and epics.count=29500 on GitLab.com
# - 30 iterations will be run
# - each requires on average ~150ms
@@ -185,7 +185,7 @@ migration helper in a final post-deployment migration,
`db/post_migrate/20200601000001_validate_not_null_constraint_on_epics_description.rb`:
```ruby
-class ValidateNotNullConstraintOnEpicsDescription < Gitlab::Database::Migration[1.0]
+class ValidateNotNullConstraintOnEpicsDescription < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
def up
diff --git a/doc/development/database/query_recorder.md b/doc/development/database/query_recorder.md
index 3fc38c10d68..f1540e7e2ae 100644
--- a/doc/development/database/query_recorder.md
+++ b/doc/development/database/query_recorder.md
@@ -39,12 +39,17 @@ end
As an example you might create 5 issues in between counts, which would cause the query count to increase by 5 if an N+1 problem exists.
-In some cases the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible.
+In some cases, the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible.
If this test fails, and the control was passed as a `QueryRecorder`, then the
failure message indicates where the extra queries are by matching queries on
the longest common prefix, grouping similar queries together.
+In some cases, N+1 specs have been written to include three requests: first one to
+warm the cache, second one to establish a control, third one to validate that
+ther are no N+1 queries. Rather than make an extra request to warm the cache, prefer two requests
+(control and test) and configure your test to ignore [cached queries](#cached-queries) in N+1 specs.
+
## Cached queries
By default, QueryRecorder ignores [cached queries](../merge_request_performance_guidelines.md#cached-queries) in the count. However, it may be better to count
@@ -62,7 +67,7 @@ end
## Use request specs instead of controller specs
-Use a [request spec](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/spec/requests) when writing a N+1 test on the controller level.
+Use a [request spec](https://gitlab.com/gitlab-org/gitlab/-/tree/master/spec/requests) when writing a N+1 test on the controller level.
Controller specs should not be used to write N+1 tests as the controller is only initialized once per example.
This could lead to false successes where subsequent "requests" could have queries reduced (for example, because of memoization).
diff --git a/doc/development/database/single_table_inheritance.md b/doc/development/database/single_table_inheritance.md
index ad0101e1594..32de1fdea35 100644
--- a/doc/development/database/single_table_inheritance.md
+++ b/doc/development/database/single_table_inheritance.md
@@ -47,7 +47,7 @@ This ensures that the migration loads the columns for the migration in isolation
and the helper disables STI by default.
```ruby
-class EnqueueSomeBackgroundMigration < Gitlab::Database::Migration[1.0]
+class EnqueueSomeBackgroundMigration < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
def up
diff --git a/doc/development/database/table_partitioning.md b/doc/development/database/table_partitioning.md
index bf12329473d..ac715b871da 100644
--- a/doc/development/database/table_partitioning.md
+++ b/doc/development/database/table_partitioning.md
@@ -173,7 +173,7 @@ An example migration of partitioning the `audit_events` table by its
`created_at` column would look like:
```ruby
-class PartitionAuditEvents < Gitlab::Database::Migration[1.0]
+class PartitionAuditEvents < Gitlab::Database::Migration[2.0]
include Gitlab::Database::PartitioningMigrationHelpers
def up
@@ -200,7 +200,7 @@ into the partitioned copy.
Continuing the above example, the migration would look like:
```ruby
-class BackfillPartitionAuditEvents < Gitlab::Database::Migration[1.0]
+class BackfillPartitionAuditEvents < Gitlab::Database::Migration[2.0]
include Gitlab::Database::PartitioningMigrationHelpers
def up
@@ -233,7 +233,7 @@ failed jobs.
Once again, continuing the example, this migration would look like:
```ruby
-class CleanupPartitionedAuditEventsBackfill < Gitlab::Database::Migration[1.0]
+class CleanupPartitionedAuditEventsBackfill < Gitlab::Database::Migration[2.0]
include Gitlab::Database::PartitioningMigrationHelpers
def up
@@ -447,6 +447,7 @@ class ConvertTableToListPartitioning < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
TABLE_NAME = :table_name
+ TABLE_FK = :table_references_by_fk
PARENT_TABLE_NAME = :p_table_name
FIRST_PARTITION = 100
PARTITION_COLUMN = :partition_id
@@ -457,6 +458,7 @@ class ConvertTableToListPartitioning < Gitlab::Database::Migration[2.0]
partitioning_column: PARTITION_COLUMN,
parent_table_name: PARENT_TABLE_NAME,
initial_partitioning_value: FIRST_PARTITION
+ lock_tables: [TABLE_FK, TABLE_NAME]
)
end
@@ -470,3 +472,14 @@ class ConvertTableToListPartitioning < Gitlab::Database::Migration[2.0]
end
end
```
+
+NOTE:
+Do not forget to set the sequence name explicitly in your model because it will
+be owned by the routing table and `ActiveRecord` can't determine it. This can
+be cleaned up after the `table_name` is changed to the routing table.
+
+```ruby
+class Model < ApplicationRecord
+ self.sequence_name = 'model_id_seq'
+end
+```
diff --git a/doc/development/database_review.md b/doc/development/database_review.md
index 58776c5330c..7fbc48af91c 100644
--- a/doc/development/database_review.md
+++ b/doc/development/database_review.md
@@ -43,15 +43,14 @@ If your merge request description does not include these items, the review is re
#### Migrations
-If new migrations are introduced, in the MR **you are required to provide**:
-
-- The output of both migrating (`db:migrate`) and rolling back (`db:rollback`) for all migrations.
+If new migrations are introduced, database reviewers must review the output of both migrating (`db:migrate`)
+and rolling back (`db:rollback`) for all migrations.
We have automated tooling for
[GitLab](https://gitlab.com/gitlab-org/gitlab) (provided by the
-[`db:check-migrations`](database/dbcheck-migrations-job.md) pipeline job) that provides this output for migrations on
-~database merge requests. You do not need to provide this information manually
-if the bot can do it for you. The bot also checks that migrations are correctly
+[`db:check-migrations`](database/dbcheck-migrations-job.md) pipeline job) that provides this output in the CI job logs.
+It is not required for the author to provide this output in the merge request description,
+but doing so may be helpful for reviewers. The bot also checks that migrations are correctly
reversible.
#### Queries
@@ -176,7 +175,7 @@ Include in the MR description:
- The query plan for each raw SQL query included in the merge request along with the link to the query plan following each raw SQL snippet.
- Provide a public link to the plan from either:
- [postgres.ai](https://postgres.ai/): Follow the link in `#database-lab` and generate a shareable, public link
- by clicking **Share** in the upper right corner.
+ by selecting **Share** in the upper right corner.
- [explain.depesz.com](https://explain.depesz.com) or [explain.dalibo.com](https://explain.dalibo.com): Paste both the plan and the query used in the form.
- When providing query plans, make sure it hits enough data:
- You can use a GitLab production replica to test your queries on a large scale,
diff --git a/doc/development/deprecation_guidelines/index.md b/doc/development/deprecation_guidelines/index.md
index a940cd9404c..be4a3369dcb 100644
--- a/doc/development/deprecation_guidelines/index.md
+++ b/doc/development/deprecation_guidelines/index.md
@@ -83,7 +83,7 @@ For configuration removals, see the [Omnibus deprecation policy](../../administr
For versioning and upgrade details, see our [Release and Maintenance policy](../../policy/maintenance.md).
-## Update the deprecations and removals documentation
+## Update the deprecations and removals documentation pages
The [deprecations](../../update/deprecations.md) and [removals](../../update/removals.md)
documentation is generated from the YAML files located in
@@ -131,3 +131,7 @@ Related Handbook pages:
- <https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes>
- <https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-deprecations-and-removals-docs>
+
+## Update the related documentation
+
+When features are deprecated and removed, [update the related documentation](../documentation/versions.md#deprecations-and-removals).
diff --git a/doc/development/development_processes.md b/doc/development/development_processes.md
index 10818b749ab..e1df3b55d06 100644
--- a/doc/development/development_processes.md
+++ b/doc/development/development_processes.md
@@ -80,7 +80,7 @@ In these cases, use the following workflow:
- [Backend](https://about.gitlab.com/handbook/engineering/)
- [Database](https://about.gitlab.com/handbook/engineering/development/database/)
- [User Experience (UX)](https://about.gitlab.com/handbook/product/ux/)
- - [Security](https://about.gitlab.com/handbook/engineering/security/)
+ - [Security](https://about.gitlab.com/handbook/security/)
- [Quality](https://about.gitlab.com/handbook/engineering/quality/)
- [Engineering Productivity](https://about.gitlab.com/handbook/engineering/quality/engineering-productivity/)
- [Infrastructure](https://about.gitlab.com/handbook/engineering/infrastructure/)
diff --git a/doc/development/distributed_tracing.md b/doc/development/distributed_tracing.md
index 0ca10bc93dd..9e62f019fbf 100644
--- a/doc/development/distributed_tracing.md
+++ b/doc/development/distributed_tracing.md
@@ -91,7 +91,7 @@ Once the performance bar is enabled, select **Trace** in the performance bar to
the Jaeger UI.
The Jaeger search UI returns a query for the `Correlation-ID` of the current request. Normally,
-this search should return a single trace result. Clicking this result shows the detail of the
+this search should return a single trace result. Selecting this result shows the detail of the
trace in a hierarchical time-line.
![Jaeger Search UI](img/distributed_tracing_jaeger_ui.png)
diff --git a/doc/development/documentation/drawers.md b/doc/development/documentation/drawers.md
new file mode 100644
index 00000000000..7ede1a0b652
--- /dev/null
+++ b/doc/development/documentation/drawers.md
@@ -0,0 +1,59 @@
+---
+stage: none
+group: Documentation Guidelines
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Create content for drawers
+
+In the GitLab UI, you can display help content in
+[a drawer component](https://design.gitlab.com/components/drawer/).
+The component for Markdown is
+[in the storybook](https://gitlab-org.gitlab.io/gitlab/storybook/?path=/story/vue-shared-markdown-drawer--default).
+
+The component points to a Markdown file. Any time you update the Markdown
+file, the contents of the drawer are updated.
+
+Drawer content is displayed in drawers only, and not on `docs.gitlab.com`.
+The content is rendered in GitLab Flavored Markdown.
+
+To create this content:
+
+1. In the [GitLab](https://gitlab.com/gitlab-org/gitlab) repository,
+ go to the `/doc/drawers` folder.
+1. Create a Markdown file. Use a descriptive filename.
+ Do not create subfolders.
+1. Add the standard page metadata. Also, include:
+
+ ```markdown
+ type: drawer
+ ```
+
+1. Author the content.
+1. If the page includes content that is also on a page on `docs.gitlab.com`,
+ on the page's metadata, include a path to the other file. For example:
+
+ ```markdown
+ source: /doc/user/search/global_search/advanced_search_syntax.md
+ ```
+
+1. Work with the developer to view the content in the drawer and
+ verify that the content appears correctly.
+
+## Drawer content guidelines
+
+- The headings in the file are used as headings in the drawer.
+ The `H1` heading is the drawer title.
+- Do not include any characters other than plain text in the `H1`.
+- The drawer component is narrow and not resizable.
+ - If you include tables, the content within should be brief.
+ - While no technical limitation exists on the number of characters
+ you can use, you should preview the drawer content to
+ ensure it renders well.
+- To link from the drawer to other content, use absolute URLs.
+- Do not include:
+ - Tier badges
+ - Version history text
+ - Alert boxes
+ - Images
+ - SVG icons
diff --git a/doc/development/documentation/feature_flags.md b/doc/development/documentation/feature_flags.md
index dae62fea603..0fab693fdee 100644
--- a/doc/development/documentation/feature_flags.md
+++ b/doc/development/documentation/feature_flags.md
@@ -57,6 +57,13 @@ FLAG:
<This feature is not ready for production use.>
```
+A `FLAG` note renders on the GitLab documentation site as:
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `example_flag`.
+On GitLab.com, this feature is not available.
+This feature is not ready for production use.
+
### Self-managed GitLab availability information
| If the feature is... | Use this text |
diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md
index c52b3b5adae..d52db71b633 100644
--- a/doc/development/documentation/index.md
+++ b/doc/development/documentation/index.md
@@ -145,68 +145,41 @@ pages. You can safely remove the `type` metadata parameter and its values.
### Batch updates for TW metadata
-NOTE:
-This task is an MVC, and requires significant manual preparation of the output.
-While the task can be time consuming, it is still faster than doing the work
-entirely manually.
+The [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS)
+file contains a list of files and the associated technical writers.
-It's important to keep the [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS)
-file in the `gitlab` project up to date with the current Technical Writing team assignments.
-This information is used in merge requests that contain documentation:
+When a merge request contains documentation, the information in the `CODEOWNERS` file determines:
-- To populate the eligible approvers section.
-- By GitLab Bot to ping reviewers for community contributions.
+- The list of users in the **Approvers** section.
+- The technical writer that the GitLab Bot pings for community contributions.
-GitLab cannot automatically associate the stage and group metadata in our documentation
-pages with the technical writer assigned to that group, so we use a Rake task to
-generate entries for the `CODEOWNERS` file. Declaring code owners for pages reduces
-the number of times GitLab Bot pings the entire Technical Writing team.
+You can use a Rake task to update the `CODEOWNERS` file.
-The `tw:codeowners` Rake task, located in [`lib/tasks/gitlab/tw/codeowners.rake`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/tasks/gitlab/tw/codeowners.rake),
-contains an array of groups and their assigned technical writer. This task:
+#### Update the `CODEOWNERS` file
-- Outputs a line for each doc with metadata that matches a group in `lib/tasks/gitlab/tw/codeowners.rake`.
- Files not matching a group are skipped.
-- Adds the full path to the page, and the assigned technical writer.
+To update the `CODEOWNERS` file:
-To prepare an update to the `CODEOWNERS` file:
+1. Open a merge request to update
+ [the Rake task](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/tasks/gitlab/tw/codeowners.rake)
+ with the latest [TW team assignments](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments).
+1. Assign the merge request to a backend maintainer for review and merge.
+1. After the MR is merged, go to the root of the `gitlab` repository.
+1. Run the Rake task and save the output in a file:
-1. Update `lib/tasks/gitlab/tw/codeowners.rake` with the latest [TW team assignments](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments).
- Make this update in a standalone merge request, as it runs a long pipeline and
- requires backend maintainer review. Make sure this is merged before you update
- `CODEOWNERS` in another merge request.
-1. Run the task from the root directory of the `gitlab` repository, and save the output in a file:
-
- ```ruby
+ ```shell
bundle exec rake tw:codeowners > ~/Desktop/updates.md
```
-1. Open the file you just created (`~/Desktop/updates.md` in this example), and prepare the output:
- - Find and replace `./` with `/`.
- - Sort the lines in alphabetical (ascending) order. If you use VS Code, you can
- select everything, press <kbd>F1</kbd>, type `sort`, and select **Sort lines (ascending, case insensitive**.
-1. Create a new branch for your `CODEOWNERS` updates.
-1. Replace the documentation-related lines in the `^[Documentation Pages]` section
- with the output you prepared.
+1. Open the file (for example, `~/Desktop/updates.md`) and copy everything
+ except the errors at the bottom of the file.
+1. Open the [`CODEOWNERS`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/CODEOWNERS)
+ file and paste the lines into the `^[Documentation Pages]` section.
WARNING:
The documentation section is not the last section of the `CODEOWNERS` file. Don't
delete data that isn't ours!
-1. Create a commit with the raw changes.
-1. From the command line, run `git diff master`.
-1. In the diff, look for directory-level assignments to manually restore to the
- `CODEOWNERS` file. If all files in a single directory are assigned to the same
- technical writer, we simplify these entries. Remove all the lines for the individual
- files, and leave a single entry for the directory, for example: `/doc/directory/ @tech.writer`.
-1. In the diff, look for changes that don't match your expectations:
- - New pages, or newly moved pages, show up as added lines.
- - Deleted pages, and pages that are now redirects, show up as deleted lines.
- - If you see an unusual number of changes to pages that all seem related,
- check the metadata for the pages. A group might have been renamed and the Rake task
- must be updated to match.
-1. Create another commit with your manual changes, and create a second merge request
- with your changes to the `CODEOWNERS` file. Assign it to a technical writing manager for review.
+1. Create a merge request and assign it to a technical writing manager for review.
## Move, rename, or delete a page
diff --git a/doc/development/documentation/restful_api_styleguide.md b/doc/development/documentation/restful_api_styleguide.md
index dc84f3a08dd..21c8c8543ab 100644
--- a/doc/development/documentation/restful_api_styleguide.md
+++ b/doc/development/documentation/restful_api_styleguide.md
@@ -129,7 +129,7 @@ To deprecate an attribute:
```
To widely announce a deprecation, or if it's a breaking change,
-[update the deprecations and removals documentation](../deprecation_guidelines/index.md#update-the-deprecations-and-removals-documentation).
+[update the deprecations and removals documentation pages](../deprecation_guidelines/index.md#update-the-deprecations-and-removals-documentation-pages).
## Method description
@@ -289,7 +289,7 @@ contains spaces in its title. Observe how spaces are escaped using the `%20`
ASCII code.
```shell
-curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20Dude"
+curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20GitLab"
```
Use `%2F` for slashes (`/`).
diff --git a/doc/development/documentation/review_apps.md b/doc/development/documentation/review_apps.md
index cb04f0909c1..3cf77fda22b 100644
--- a/doc/development/documentation/review_apps.md
+++ b/doc/development/documentation/review_apps.md
@@ -47,12 +47,11 @@ If you want to know the in-depth details, here's what's really happening:
1. The preview URL is shown both at the job output and in the merge request
widget. You also get the link to the remote pipeline.
1. In the `gitlab-org/gitlab-docs` project, the pipeline is created and it
- [skips the test jobs](https://gitlab.com/gitlab-org/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55)
+ [skips most test jobs](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/d41ca9323f762132780d2d072f845d28817a5383/.gitlab/ci/rules.gitlab-ci.yml#L101-103)
to lower the build time.
-1. Once the docs site is built, the HTML files are uploaded as artifacts.
-1. A specific runner tied only to the docs project, runs the Review App job
- that downloads the artifacts and uses `rsync` to transfer the files over
- to a location where NGINX serves them.
+1. After the docs site is built, the HTML files are uploaded as artifacts to
+ a GCP bucket (see [issue `gitlab-com/gl-infra/reliability#11021`](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/11021)
+ for the implementation details).
The following GitLab features are used among others:
@@ -60,42 +59,26 @@ The following GitLab features are used among others:
- [Multi project pipelines](../../ci/pipelines/downstream_pipelines.md#multi-project-pipelines)
- [Review Apps](../../ci/review_apps/index.md)
- [Artifacts](../../ci/yaml/index.md#artifacts)
-- [Specific runner](../../ci/runners/runners_scope.md#prevent-a-specific-runner-from-being-enabled-for-other-projects)
- [Merge request pipelines](../../ci/pipelines/merge_request_pipelines.md)
## Troubleshooting review apps
-### Review app returns a 404 error
+### `NoSuchKey The specified key does not exist`
-If the review app URL returns a 404 error, either the site is not
-yet deployed, or something went wrong with the remote pipeline. You can:
+If you see the following message in a review app, either the site is not
+yet deployed, or something went wrong with the downstream pipeline in `gitlab-docs`.
-- Wait a few minutes and it should appear online.
-- Check the manual job's log and verify the URL. If the URL is different, try the
- one from the job log.
-- Check the status of the remote pipeline from the link in the merge request's job output.
- If the pipeline failed or got stuck, GitLab team members can ask for help in the `#docs`
- chat channel. Contributors can ping a technical writer in the merge request.
-
-### Not enough disk space
-
-Sometimes the review app server is full and there is no more disk space. Each review
-app takes about 570MB of disk space.
+```plaintext
+NoSuchKeyThe specified key does not exist.No such object: <URL>
+```
-A cron job to remove review apps older than 20 days runs hourly,
-but the disk space still occasionally fills up. To manually free up more space,
-a GitLab technical writing team member can:
+In that case, you can:
-1. Navigate to the [`gitlab-docs` schedules page](https://gitlab.com/gitlab-org/gitlab-docs/-/pipeline_schedules).
-1. Select the play button for the `Remove old review apps from review app server`
- schedule. By default, this cleans up review apps older than 14 days.
-1. Navigate to the [pipelines page](https://gitlab.com/gitlab-org/gitlab-docs/-/pipelines)
- and start the manual job called `clean-pages`.
-
-If the job says no review apps were found in that period, edit the `CLEAN_REVIEW_APPS_DAYS`
-variable in the schedule, and repeat the process above. Gradually decrease the variable
-until the free disk space reaches an acceptable amount (for example, 3GB).
-Remember to set it to 14 again when you're done.
-
-There's an issue to [migrate from the DigitalOcean server to GCP buckets](https://gitlab.com/gitlab-org/gitlab-docs/-/issues/735)),
-which should solve the disk space problem.
+- Wait a few minutes and the review app should appear online.
+- Check the `review-docs-deploy` job's log and verify the URL. If the URL shown in the merge
+ request UI is different than the job log, try the one from the job log.
+- Check the status of the remote pipeline from the link in the merge request's job output.
+ If the pipeline failed or got stuck, GitLab team members can ask for help in the `#docs`
+ internal Slack channel. Contributors can ping a
+ [technical writer](https://about.gitlab.com/handbook/product/ux/technical-writing/#designated-technical-writers)
+ in the merge request.
diff --git a/doc/development/documentation/site_architecture/deployment_process.md b/doc/development/documentation/site_architecture/deployment_process.md
index b5c3a59b0eb..18cc27adaaa 100644
--- a/doc/development/documentation/site_architecture/deployment_process.md
+++ b/doc/development/documentation/site_architecture/deployment_process.md
@@ -6,6 +6,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Documentation deployments
+## Deployment environments
+
+The [GitLab documentation site](https://docs.gitlab.com/) is a static site hosted by [GitLab Pages](../../../user/project/pages/index.md). The deployment is done by the [Pages deploy job](#pages-deploy-job).
+
+The website hosts documentation only for the [currently supported](../../../policy/maintenance.md) GitLab versions. Documentation for older versions is built and uploaded as Docker images to be downloaded from [GitLab Docs archives](https://docs.gitlab.com/archives/).
+
+## Parts of release process
+
The documentation [release process](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/doc/releases.md)
involves:
@@ -24,10 +32,8 @@ For general information on using Docker with CI/CD pipelines, see [Docker integr
## Stable branches
-Stable branches for documentation include the relevant stable branches of all the projects required to publish the entire
-documentation suite. For example, the stable version of documentation for version `14.4` includes:
+Pipelines for stable branches in the documentation project pull the relevant stable branches of included projects. For example, the documentation for stable version `14.4` is built from the [`14.4`](https://gitlab.com/gitlab-org/gitlab-docs/-/tree/14.4) branch of the `gitlab-docs` project, which then includes:
-- The [`14.4`](https://gitlab.com/gitlab-org/gitlab-docs/-/tree/14.4) branch of the `gitlab-docs` project.
- The [`14-4-stable-ee`](https://gitlab.com/gitlab-org/gitlab/-/tree/14-4-stable-ee) branch of the `gitlab` project.
- The [`14-4-stable`](https://gitlab.com/gitlab-org/gitlab-runner/-/tree/14-4-stable) branch of the `gitlab-runner` project.
- The [`14-4-stable`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/tree/14-4-stable) branch of the `omnibus-gitlab` project.
@@ -117,7 +123,7 @@ graph TD
G--"Latest `gitlab-docs:latest` image<br>pushed up"-->H
```
-## Documentation Pages deployment
+## Pages deploy job
[GitLab Docs](https://docs.gitlab.com) is a [Pages site](../../../user/project/pages/index.md) and documentation updates
for it must be deployed to become available.
@@ -176,7 +182,7 @@ Dockerfiles to build and deploy <https://docs.gitlab.com>. It is heavily inspire
Although build images are built automatically via GitLab CI/CD, you can build and tag all tooling images locally:
-1. Make sure you have [Docker installed](https://docs.docker.com/install/).
+1. Make sure you have [Docker installed](https://docs.docker.com/get-docker/).
1. Make sure you're in the `dockerfiles/` directory of the `gitlab-docs` repository.
1. Build the images:
diff --git a/doc/development/documentation/site_architecture/global_nav.md b/doc/development/documentation/site_architecture/global_nav.md
index 37d10b16fcd..d95ca720119 100644
--- a/doc/development/documentation/site_architecture/global_nav.md
+++ b/doc/development/documentation/site_architecture/global_nav.md
@@ -234,7 +234,7 @@ below the doc link:
external_url: true
```
-All nav links are clickable. If the higher-level link does not have a link
+All nav links are selectable. If the higher-level link does not have a link
of its own, it must link to its first sub-item link, mimicking the navigation in GitLab.
This must be avoided so that we don't have duplicated links nor two `.active` links
at the same time.
diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md
index d629bc0b87e..ef934186981 100644
--- a/doc/development/documentation/styleguide/index.md
+++ b/doc/development/documentation/styleguide/index.md
@@ -18,11 +18,22 @@ In addition to this page, the following resources can help you craft and contrib
- [Recommended word list](word_list.md)
- [Doc style and consistency testing](../testing.md)
- [Guidelines for UI error messages](https://design.gitlab.com/content/error-messages/)
+- [Documentation global navigation](../site_architecture/global_nav.md)
- [GitLab Handbook style guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines)
- [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/welcome/)
- [Google Developer Documentation Style Guide](https://developers.google.com/style)
- [Recent updates to this guide](https://gitlab.com/dashboard/merge_requests?scope=all&state=merged&label_name[]=tw-style&not[label_name][]=docs%3A%3Afix)
+## The GitLab voice
+
+The GitLab brand guidelines define the
+[voice used by the larger organization](https://design.gitlab.com/brand/overview/#tone-of-voice).
+
+Building on that guidance, the voice in the GitLab documentation strives to be concise,
+direct, and precise. The goal is to provide information that's easy to search and scan.
+
+The voice in the documentation should be conversational but brief, friendly but succinct.
+
## Documentation is the single source of truth (SSOT)
The GitLab documentation is the SSOT for all
@@ -125,6 +136,23 @@ Maintaining a knowledge base separate from the documentation would
be against the documentation-first methodology, because the content would overlap with
the documentation.
+## Writing for localization
+
+The GitLab documentation is not localized, but we follow guidelines that
+help benefit translation. For example, we:
+
+- Write in [active voice](word_list.md#active-voice).
+- Write in [present tense](word_list.md#future-tense).
+- Avoid words that can be translated incorrectly, like:
+ - [since and because](word_list.md#since)
+ - [once and after](word_list.md#once)
+ - [it](word_list.md#it)
+- Avoid [ing](word_list.md#-ing-words) words.
+
+[The GitLab voice](#the-gitlab-voice) dictates that we write clearly and directly,
+and with translation in mind. [The word list](word_list.md) and our Vale rules
+also aid in consistency, which is important for localization.
+
## Markdown
All GitLab documentation is written using [Markdown](https://en.wikipedia.org/wiki/Markdown).
@@ -143,26 +171,17 @@ Hard-coded HTML is valid, although it's discouraged from being used. HTML is per
- Special styling is required.
- Reviewed and approved by a technical writer.
-### Headings in Markdown
+### Heading levels in Markdown
Each documentation page begins with a level 1 heading (`#`). This becomes the `h1` element when
the page is rendered to HTML. There can be only **one** level 1 heading per page.
- For each subsection, increment the heading level. In other words, increment the number of `#` characters
- in front of the heading.
-- Avoid headings greater than `H5` (`#####`). If you need more than five heading levels, move the topics to a new page instead.
- Headings greater than `H5` do not display in the right sidebar navigation.
+ in front of the topic title.
+- Avoid heading levels greater than `H5` (`#####`). If you need more than five heading levels, move the topics to a new page instead.
+ Heading levels greater than `H5` do not display in the right sidebar navigation.
- Do not skip a level. For example: `##` > `####`.
-- Leave one blank line before and after the heading.
-
-When you change heading text, the anchor link changes. To avoid broken links:
-
-- Do not use step numbers in headings.
-- When possible, do not use words that might change in the future.
-
-Also, do not use links as part of heading text.
-
-See also [heading guidelines for specific topic types](../structure.md).
+- Leave one blank line before and after the topic title.
### Backticks in Markdown
@@ -221,9 +240,9 @@ GitLab documentation should be clear and easy to understand.
As a company, we tend toward lowercase.
-#### Headings
+#### Topic titles
-Use sentence case. For example:
+Use sentence case for topic titles. For example:
- `# Use variables to configure pipelines`
- `## Use the To-Do List`
@@ -329,7 +348,7 @@ Some contractions, however, should be avoided:
### Acronyms
If you use an acronym, spell it out on first use on a page. You do not need to spell it out more than once on a page.
-When possible, try to avoid acronyms in headings.
+When possible, try to avoid acronyms in topic titles.
### Numbers
@@ -374,22 +393,31 @@ You can use italics when you are introducing a term for the first time. Otherwis
### Punctuation
-Follow these guidelines for punctuation:
+Follow these guidelines for punctuation.
<!-- vale gitlab.Repetition = NO -->
- End full sentences with a period.
-- Use one space between sentences.
-- Do not use semicolons. Use two sentences instead.
-- Do not use double spaces. (Tested in [`SentenceSpacing.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SentenceSpacing.yml).)
-- Do not use non-breaking spaces. Use standard spaces instead. (Tested in [`lint-doc.sh`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/lint-doc.sh).)
-- Do not use tabs for indentation. Use spaces instead. You can configure your code editor to output spaces instead of tabs when pressing the tab key.
- Use serial (Oxford) commas before the final **and** or **or** in a list of three or more items. (Tested in [`OxfordComma.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/OxfordComma.yml).)
-- Avoid dashes. Use separate sentences, or commas, instead.
-- Do not use typographer's ("curly") quotes. Use straight quotes instead. (Tested in [`NonStandardQuotes.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/NonStandardQuotes.yml).)
<!-- vale gitlab.Repetition = YES -->
+When spacing content:
+
+- Use one space between sentences. (Use of more than one space is tested in [`SentenceSpacing.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SentenceSpacing.yml).)
+- Do not use non-breaking spaces. Use standard spaces instead. (Tested in [`lint-doc.sh`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/lint-doc.sh).)
+- Do not use tabs for indentation. Use spaces instead. You can configure your code editor to output spaces instead of tabs when pressing the <kbd>Tab</kbd> key.
+
+<!-- vale gitlab.NonStandardQuotes = NO -->
+
+Do not use these punctuation characters:
+
+- `;` (semicolon): Use two sentences instead.
+- `–` (en dash) or `—` (em dash): Use separate sentences, or commas, instead.
+- `“` `”` `‘` `’`: Double or single typographer's ("curly") quotation marks. Use straight quotes instead. (Tested in [`NonStandardQuotes.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/NonStandardQuotes.yml).)
+
+<!-- vale gitlab.NonStandardQuotes = YES -->
+
### Placeholder text
You might want to provide a command or configuration that
@@ -677,22 +705,27 @@ use a URL like `https://docs.gitlab.com/charts/`.
### Anchor links
-Each heading has an anchor link. For example, a topic with the title
+Each topic title has an anchor link. For example, a topic with the title
`## This is an example` has the anchor `#this-is-an-example`.
-The first topic on a page (the `h1`) has an anchor link,
+The first topic title on a page (the `h1`) has an anchor link,
but do not use it. Link to the page instead.
-If a heading has a [product tier badge](#product-tier-badges),
-do not include it in the anchor link. For example, for the heading
+If a topic title has a [product tier badge](#product-tier-badges),
+do not include it in the anchor link. For example, for the topic
`## This is an example **(FREE)**`, use the anchor `#this-is-an-example`.
With Kramdown, you can add a custom ID to an HTML element, but these IDs
don't work in `/help`, so you should not use them.
+When you change topic title text, the anchor link changes. To avoid broken links:
+
+- Do not use step numbers in topic titles.
+- When possible, do not use words that might change in the future.
+
#### Changing links and titles
-When you change a heading, the anchor link changes. To ensure you update
+When you change a topic title, the anchor link changes. To ensure you update
any related links, search these directories:
- `doc/*`
@@ -823,7 +856,7 @@ To open either project or group settings:
```markdown
1. On the top bar, select **Main menu**, and:
- - For a project, select ***Projects** and find your project.
+ - For a project, select **Projects** and find your project.
- For a group, select **Groups** and find your group.
1. On the left sidebar, select **Settings > CI/CD**.
1. Expand **General pipelines**.
@@ -1195,7 +1228,7 @@ Instead of adding a note:
- Re-write the sentence as part of a paragraph.
- Put the information into its own paragraph.
-- Put the content under a new subheading.
+- Put the content under a new topic title.
If you must use a note, use this format:
@@ -1311,6 +1344,7 @@ It renders on the GitLab documentation site as:
> - Second item in the list
## Tabs
+<!-- markdownlint-disable tabs-blank-lines -->
On the docs site, you can format text so it's displayed as tabs.
@@ -1329,6 +1363,7 @@ Here's some other content in tab two.
::EndTabs
```
+<!-- markdownlint-enable tabs-blank-lines -->
This code renders on the GitLab documentation site as:
@@ -1378,25 +1413,25 @@ When names change, it is more complicated to search or grep text that has line b
### Product tier badges
-Tier badges are displayed as orange text next to a heading. These badges link to the GitLab
+Tier badges are displayed as orange text next to a topic title. These badges link to the GitLab
pricing page. For example:
![Tier badge](img/tier_badge.png)
You must assign a tier badge:
-- To all H1 topic headings, except the pages under `doc/development/*`.
-- To topic headings that don't apply to the same tier as the H1.
+- To all H1 topic titles, except the pages under `doc/development/*`.
+- To topic titles that don't apply to the same tier as the H1.
-To add a tier badge to a heading, add the relevant tier badge
-after the heading text. For example:
+To add a tier badge to a topic title, add the relevant tier badge
+after the title text. For example:
```markdown
-# Heading title **(FREE)**
+# Topic title **(FREE)**
```
Do not add tier badges inline with other text, except for [API attributes](../restful_api_styleguide.md).
-The single source of truth for a feature should be the heading where the
+The single source of truth for a feature should be the topic where the
functionality is described.
#### Available product tier badges
diff --git a/doc/development/documentation/styleguide/word_list.md b/doc/development/documentation/styleguide/word_list.md
index 65ad8dea688..d28972a644b 100644
--- a/doc/development/documentation/styleguide/word_list.md
+++ b/doc/development/documentation/styleguide/word_list.md
@@ -35,7 +35,7 @@ Don't use backticks.
## 2FA, two-factor authentication
-Spell out **two-factor authentication** in sentence case for the first use and in section headings, and **2FA**
+Spell out **two-factor authentication** in sentence case for the first use and in topic titles, and **2FA**
thereafter. If the first word in a sentence, do not capitalize `factor` or `authentication`. For example:
- Two-factor authentication (2FA) helps secure your account. Set up 2FA when you first log in.
@@ -64,6 +64,23 @@ When you create a user, you choose an access level: **Regular**, **Auditor**, or
Capitalize these words when you refer to the UI. Otherwise use lowercase.
+## active voice
+
+Use active voice instead of passive.
+
+Use:
+
+- The contributor writes the documentation.
+
+Instead of:
+
+- The documentation is written by contributors.
+
+NOTE:
+If you can add the phrase "by zombies" to the phrase,
+the construction is passive. For example, `The button is selected by zombies`
+is passive. `Zombies select the button` is active.
+
## administrator
Use **administrator access** instead of **admin** when talking about a user's access level.
@@ -460,6 +477,10 @@ Do not use **foo** in product documentation. You can use it in our API and contr
When possible, use present tense instead of future tense. For example, use **after you execute this command, GitLab displays the result** instead of **after you execute this command, GitLab will display the result**. ([Vale](../testing.md#vale) rule: [`FutureTense.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/FutureTense.yml))
+## GB, gigabytes
+
+For **GB** and **MB**, follow the [Microsoft guidance](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/term-collections/bits-bytes-terms).
+
## Geo
Use title case for **Geo**.
@@ -570,6 +591,15 @@ Do not use Latin abbreviations. Use **that is** instead. ([Vale](../testing.md#v
Do not use **in order to**. Use **to** instead. ([Vale](../testing.md#vale) rule: [`Wordy.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/Wordy.yml))
+## -ing words
+
+Remove **-ing** words whenever possible. They can be difficult to translate,
+and more precise terms are usually available. For example:
+
+- Instead of **The files using storage are deleted**, use **The files that use storage are deleted**.
+- Instead of **Delete files using the Edit button**, use **Delete files by using the Edit button**.
+- Instead of **Replicating your server is required**, use **You must replicate your server**.
+
## issue
Use lowercase for **issue**.
@@ -582,6 +612,21 @@ Use lowercase for **issue board**.
Use lowercase for **issue weights**.
+## it
+
+When you use the word **it**, ensure the word it refers to is obvious.
+If it's not obvious, repeat the word rather than using **it**.
+
+Use:
+
+- The field returns a connection. The field accepts four arguments.
+
+Instead of:
+
+- The field returns a connection. It accepts four arguments.
+
+See also [this, these, that, those](#this-these-that-those).
+
## job
Do not use **build** to be synonymous with **job**. A job is defined in the `.gitlab-ci.yml` file and runs as part of a pipeline.
@@ -692,6 +737,10 @@ Do not use `master`. Use `main` when you need a sample [default branch name](#de
**Might** means something has the probability of occurring. **May** gives permission to do something. Consider **can** instead of **may**.
+## MB, megabytes
+
+For **MB** and **GB**, follow the [Microsoft guidance](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/term-collections/bits-bytes-terms).
+
## me, myself, mine
Do not use first-person singular. Use **you**, **we**, or **us** instead. ([Vale](../testing.md#vale) rule: [`FirstPerson.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/FirstPerson.yml))
@@ -720,24 +769,18 @@ Do not use **navigate**. Use **go** instead. For example:
## need to, should
-Try to avoid **needs to**, because it's wordy. Avoid **should** when you can be more specific. If something is required, use **must**.
+Try to avoid **needs to**, because it's wordy. If something is recommended, use **should** instead. If something is required, use **must**.
Use:
-- You must set the variable.
-- Set the variable.
+- You should set the variable. (recommended)
+- You must set the variable. (required)
+- Set the variable. (required)
Instead of:
- You need to set the variable.
-
-**Should** is acceptable for recommended actions or items, or in cases where an event may not
-happen. For example:
-
-- Although you can configure the installation manually, you should use the express configuration to
- avoid complications.
-- You should see a success message in the console. Contact support if an error message appears
- instead.
+- We recommend that you set the variable.
## note that
@@ -811,7 +854,7 @@ When writing about the Owner role:
- Instead of: if you are an owner
Do not use bold.
-
+
Do not use **Owner permissions**. A user who is assigned the Owner role has a set of associated permissions.
An Owner is the highest role a user can have.
@@ -893,6 +936,10 @@ Instead of:
- Select **Create user** or **Save changes** if you created a new user or
edited an existing one respectively.
+## review app
+
+Use lowercase for **review app**.
+
## roles
Do not use **roles** and [**permissions**](#permissions) interchangeably. Each user is assigned a role. Each role includes a set of permissions.
@@ -981,7 +1028,11 @@ Use **setup** as a noun, and **set up** as a verb. For example:
## sign in
-Use **sign in** instead of **sign on** or **log on** or **log in**. If the user interface has different words, use those.
+Use **sign in** or **sign in to**.
+
+Do not use **sign on** or **sign into**, or **log on**, **log in**, or **log into**.
+
+If the user interface has different words, use those.
You can use **single sign-on**.
@@ -1108,6 +1159,14 @@ For example:
See also [**enter**](#enter).
+## units of measurement
+
+Use a space between the number and the unit of measurement. For example, **128 GB**.
+([Vale](../testing.md#vale) rule: [`Units.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/Units.yml))
+
+For other guidance, follow
+[the Microsoft style guidelines](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/term-collections/bits-bytes-terms).
+
## update
Use **update** for installing a newer **patch** version of the software only. For example:
@@ -1173,7 +1232,26 @@ Instead of:
- We created a feature for you to add widgets.
-One exception: You can use **we recommend** instead of **it is recommended** or **GitLab recommends**. ([Vale](../testing.md#vale) rule: [`SubstitutionSuggestions.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SubstitutionSuggestions.yml))
+([Vale](../testing.md#vale) rule: [`SubstitutionSuggestions.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SubstitutionSuggestions.yml))
+
+## while
+
+Use **while** to refer only to something occurring in time. For example,
+**Leave the window open while the process runs.**
+
+Do not use **while** for comparison. For example, use:
+
+- Job 1 can run quickly. However, job 2 is more precise.
+
+Instead of:
+
+- While job 1 can run quickly, job 2 is more precise.
+
+For details, see the [Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/w/while).
+
+## whilst
+
+Do not use **whilst**. Use [while](#while) instead. **While** is more succinct and easier for non-native English speakers to understand.
## whitelist
@@ -1203,5 +1281,19 @@ Instead of:
- Users can configure a pipeline.
+## you can
+
+When possible, start sentences with an active verb instead of **you can**.
+For example:
+
+- Use code review analytics to view merge request data.
+- Create a board to organize your team tasks.
+- Configure variables to restrict pushes to a repository.
+
+Use **you can** for optional actions. For example:
+
+- Use code review analytics to view metrics per merge request. You can also use the API.
+- Enter the name and value pairs. You can add up to 20 pairs per streaming destination.
+
<!-- vale on -->
<!-- markdownlint-enable -->
diff --git a/doc/development/documentation/testing.md b/doc/development/documentation/testing.md
index c801bb9f877..8b8f281d7c1 100644
--- a/doc/development/documentation/testing.md
+++ b/doc/development/documentation/testing.md
@@ -86,7 +86,7 @@ job, which runs two types of link checks. In both cases, links with destinations
that begin with `http` or `https` are considered external links, and skipped:
- `bundle exec nanoc check internal_links`: Tests links to internal pages.
-- `bundle exec nanoc check internal_anchors`: Tests links to subheadings (anchors) on internal pages.
+- `bundle exec nanoc check internal_anchors`: Tests links to topic title anchors on internal pages.
Failures from these tests are displayed at the end of the test results in the **Issues found!** area.
For example, failures in the `internal_anchors` test follow this format:
@@ -104,7 +104,7 @@ For example, failures in the `internal_anchors` test follow this format:
- **Destination**: The full path to the file not found by the test. To find the
file in the `gitlab` repository, replace `/tmp/gitlab-docs/public/ee` with `doc`, and `.html` with `.md`.
- **Link**: The actual link the script attempted to find.
-- **Anchor**: If present, the subheading (anchor) the script attempted to find.
+- **Anchor**: If present, the topic title anchor the script attempted to find.
Check for multiple instances of the same broken link on each page reporting an error.
Even if a specific broken link appears multiple times on a page, the test reports it only once.
diff --git a/doc/development/documentation/topic_types/concept.md b/doc/development/documentation/topic_types/concept.md
index dfd003b642d..7be6bef4fad 100644
--- a/doc/development/documentation/topic_types/concept.md
+++ b/doc/development/documentation/topic_types/concept.md
@@ -32,13 +32,13 @@ Remember, if you start to describe about another concept, stop yourself.
Each concept should be about one concept only.
```
-## Concept headings
+## Concept topic titles
-For the heading text, use a noun. For example, `Widgets` or `GDK dependency management`.
+For the title text, use a noun. For example, `Widgets` or `GDK dependency management`.
If a noun is ambiguous, you can add a gerund. For example, `Documenting versions` instead of `Versions`.
-Avoid these heading titles:
+Avoid these topic titles:
- `Overview` or `Introduction`. Instead, use a more specific
noun or phrase that someone would search for.
diff --git a/doc/development/documentation/topic_types/index.md b/doc/development/documentation/topic_types/index.md
index 8403fd26517..8e8c474ce3c 100644
--- a/doc/development/documentation/topic_types/index.md
+++ b/doc/development/documentation/topic_types/index.md
@@ -20,11 +20,10 @@ The acronym refers to the first letter of each topic type.
In general, each page in the GitLab documentation contains multiple topics.
Each topic on a page should be recognizable as a specific topic type.
-## Other topic types
+In addition to the four primary topic types, we also have a page type for
+[Tutorials](tutorial.md) and [Get started](#get-started).
-In addition to the four primary topic types, we have a few other types.
-
-### Related topics
+## Related topics
If inline links are not sufficient, you can create a topic called **Related topics**
and include an unordered list of related topics. This topic should be above the Troubleshooting section.
@@ -36,57 +35,7 @@ and include an unordered list of related topics. This topic should be above the
- [Trigger a pipeline manually](link-to-topic).
```
-### Tutorials
-
-A tutorial is page that contains an end-to-end walkthrough of a complex workflow or scenario.
-In general, you might consider using a tutorial when:
-
-- The workflow requires a number of sequential steps where each step consists
- of sub-steps.
-- The steps cover a variety of GitLab features or third-party tools.
-
-Tutorials are learning aids that complement our core documentation.
-They do not introduce new features.
-Always use the primary [topic types](#documentation-topic-types-ctrt) to document new features.
-
-Tutorials should be in this format:
-
-```markdown
-# Title (starts with "Tutorial:" followed by an active verb, like "Tutorial: Create a website")
-
-A paragraph that explains what the tutorial does, and the expected outcome.
-
-To create a website:
-
-1. [Do the first task](#do-the-first-task)
-1. [Do the second task](#do-the-second-task)
-
-Prerequisites (optional):
-
-- Thing 1
-- Thing 2
-- Thing 3
-
-## Do the first task
-
-To do step 1:
-
-1. First step.
-1. Another step.
-1. Another step.
-
-## Do the second task
-
-Before you begin, make sure you have [done the first task](#do-the-first-task).
-
-To do step 2:
-
-1. First step.
-1. Another step.
-1. Another step.
-```
-
-### Get started
+## Get started
A get started page is a set of steps to help a user get set up
quickly to use a single GitLab feature or tool.
@@ -110,21 +59,21 @@ consider using subsections for each distinct task.
In the left nav, use `Get started` as the text. On the page itself, spell out
the full name. For example, `Get started with application security`.
-### Topics and resources
+## Topics and resources
Some pages are solely a list of links to other documentation.
We do not encourage this page type. Lists of links can get out-of-date quickly
and offer little value to users, who prefer to search to find information.
-## Heading text guidelines
+## Topic text guidelines
-In general, for heading text:
+In general, for topic text:
- Be clear and direct. Make every word count.
- Use articles and prepositions.
- Follow [capitalization](../styleguide/index.md#capitalization) guidelines.
-- Do not repeat text from earlier headings. For example, if the page is about merge requests,
+- Do not repeat text from earlier topic titles. For example, if the page is about merge requests,
instead of `Troubleshooting merge requests`, use only `Troubleshooting`.
-See also [guidelines for headings in Markdown](../styleguide/index.md#headings-in-markdown).
+See also [guidelines for heading levels in Markdown](../styleguide/index.md#heading-levels-in-markdown).
diff --git a/doc/development/documentation/topic_types/reference.md b/doc/development/documentation/topic_types/reference.md
index e7ee8b20925..8bb89f4c210 100644
--- a/doc/development/documentation/topic_types/reference.md
+++ b/doc/development/documentation/topic_types/reference.md
@@ -19,11 +19,11 @@ Introductory sentence.
| **Name** | Descriptive sentence about the setting. |
```
-## Reference headings
+## Reference topic titles
-Reference headings are usually nouns.
+Reference topic titles are usually nouns.
-Avoid these heading titles:
+Avoid these topic titles:
- `Important notes`. Instead, incorporate this information
closer to where it belongs. For example, this information might be a prerequisite
diff --git a/doc/development/documentation/topic_types/task.md b/doc/development/documentation/topic_types/task.md
index 60508fbf6ee..78d670a16d6 100644
--- a/doc/development/documentation/topic_types/task.md
+++ b/doc/development/documentation/topic_types/task.md
@@ -52,14 +52,23 @@ To create an issue:
The issue is created. You can view it by going to **Issues > List**.
```
-## Task headings
+## Task topic titles
-For the heading text, use the structure `active verb` + `noun`.
+For the title text, use the structure `active verb` + `noun`.
For example, `Create an issue`.
If you have several tasks on a page that share prerequisites, you can use the title
`Prerequisites` and link to it.
+## Task introductions
+
+To start the task topic, use the structure `active verb` + `noun`, and
+provide context about the action.
+For example, `Create an issue when you want to track bugs or future work`.
+
+To start the task steps, use a succinct action followed by a colon.
+For example, `To create an issue:`
+
## Related topics
- [View the format for writing task steps](../styleguide/index.md#navigation).
diff --git a/doc/development/documentation/topic_types/troubleshooting.md b/doc/development/documentation/topic_types/troubleshooting.md
index e2136de2e06..70475eb0ccf 100644
--- a/doc/development/documentation/topic_types/troubleshooting.md
+++ b/doc/development/documentation/topic_types/troubleshooting.md
@@ -6,10 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Troubleshooting topic type
-Troubleshooting topics should be the last topics on a page.
+Troubleshooting topics should be the final topics on a page.
If a page has more than five troubleshooting topics, put the content on a separate page that has troubleshooting information exclusively. Name the page `Troubleshooting <feature>`
-and in the left nav, use the word `Troubleshoot` only.
+and in the left nav, use the word `Troubleshooting` only.
Troubleshooting can be one of three types.
@@ -46,11 +46,22 @@ The workaround is...
If multiple causes or workarounds exist, consider putting them into a table format.
If you use the exact error message, surround it in backticks so it's styled as code.
-## Troubleshooting headings
+## Troubleshooting topic titles
-For the heading of a **Troubleshooting reference** topic:
+For the title of a **Troubleshooting reference** topic:
- Consider including at least a partial error message.
- Use fewer than 70 characters.
+- Do not use links in the title.
If you do not put the full error in the title, include it in the body text.
+
+## Rails console write functions
+
+If the troubleshooting suggestion includes a function that changes data on the GitLab instance,
+add the following warning:
+
+```markdown
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+```
diff --git a/doc/development/documentation/topic_types/tutorial.md b/doc/development/documentation/topic_types/tutorial.md
new file mode 100644
index 00000000000..1b1426a0465
--- /dev/null
+++ b/doc/development/documentation/topic_types/tutorial.md
@@ -0,0 +1,102 @@
+---
+stage: none
+group: Style Guide
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Tutorial page type
+
+A tutorial is page that contains an end-to-end walkthrough of a complex workflow or scenario.
+In general, you might consider using a tutorial when:
+
+- The workflow requires a number of sequential steps where each step consists
+ of sub-steps.
+- The steps cover a variety of GitLab features or third-party tools.
+
+Tutorials are learning aids that complement our core documentation.
+They do not introduce new features.
+Always use the primary [topic types](index.md) to document new features.
+
+## Tutorial format
+
+Tutorials should be in this format:
+
+```markdown
+# Title (starts with "Tutorial:" followed by an active verb, like "Tutorial: Create a website")
+
+A paragraph that explains what the tutorial does, and the expected outcome.
+
+To create a website:
+
+1. [Do the first task](#do-the-first-task)
+1. [Do the second task](#do-the-second-task)
+
+Prerequisites (optional):
+
+- Thing 1
+- Thing 2
+- Thing 3
+
+## Do the first task
+
+To do step 1:
+
+1. First step.
+1. Another step.
+1. Another step.
+
+## Do the second task
+
+Before you begin, make sure you have [done the first task](#do-the-first-task).
+
+To do step 2:
+
+1. First step.
+1. Another step.
+1. Another step.
+```
+
+An example of a tutorial that follows this format is
+[Tutorial: Make your first Git commit](../../../tutorials/make_your_first_git_commit.md).
+
+## Tutorial page title
+
+Start the page title with `Tutorial:` followed by an active verb, like `Tutorial: Create a website`.
+
+In the left nav, use the full page title. Do not abbreviate it.
+Put the text in quotes so the pipeline will pass. For example,
+`"Tutorial: Make your first Git commit"`.
+
+On [the **Learn GitLab with tutorials** page](../../../tutorials/index.md),
+do not use `Tutorial` in the title.
+
+## Screenshots
+
+You can include screenshots in a tutorial to illustrate important steps in the process.
+In the core product documentation, you should [use screenshots sparingly](../styleguide/index.md#images).
+However, in tutorials, screenshots can help users understand where they are in a complex process.
+
+Try to balance the number of screenshots in the tutorial so they don't disrupt
+the narrative flow. For example, do not put one large screenshot in the middle of the tutorial.
+Instead, put multiple, smaller screenshots throughout.
+
+## Tutorial voice
+
+Use a friendlier tone than you would for other topic types. For example,
+you can:
+
+- Add encouraging or congratulatory phrases after tasks.
+- Use future tense from time to time, especially when you're introducing
+ steps. For example, `Next, you will associate your issues with your epics`.
+- Be more conversational. For example, `This task might take a while to complete`.
+
+## Metadata
+
+On pages that are tutorials, add the most appropriate `stage:` and `group:` metadata at the top of the file.
+If the majority of the content does not align with a single group, specify `none` for the stage
+and `Tutorials` for the group:
+
+```plaintext
+stage: none
+group: Tutorials
+```
diff --git a/doc/development/documentation/versions.md b/doc/development/documentation/versions.md
index 12381b84c2c..030bdec0361 100644
--- a/doc/development/documentation/versions.md
+++ b/doc/development/documentation/versions.md
@@ -31,7 +31,7 @@ You do not need to add version information on the pages in the `/development` di
### Add a **Version history** item
-If all content in a topic is related, add a version history item after the topic heading.
+If all content in a topic is related, add a version history item after the topic title.
For example:
```markdown
diff --git a/doc/development/documentation/workflow.md b/doc/development/documentation/workflow.md
index 85d3d5e9cfc..9d8d25607c8 100644
--- a/doc/development/documentation/workflow.md
+++ b/doc/development/documentation/workflow.md
@@ -153,7 +153,7 @@ Ensure the following if skipping an initial Technical Writer review:
- [Product badges](styleguide/index.md#product-tier-badges) are applied.
- The GitLab [version](versions.md) that
introduced the feature is included.
-- Changes to headings don't affect in-app hyperlinks.
+- Changes to topic titles don't affect in-app hyperlinks.
- Specific [user permissions](../../user/permissions.md) are documented.
- New documents are linked from higher-level indexes, for discoverability.
- The style guide is followed:
diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md
index 2516196d2e0..14df73b8779 100644
--- a/doc/development/ee_features.md
+++ b/doc/development/ee_features.md
@@ -120,24 +120,32 @@ To do so:
### Simulate a SaaS instance
-If you're developing locally and need your instance to act like the SaaS version of the product,
-you can simulate SaaS by exporting an environment variable:
+If you're developing locally and need your instance to simulate the SaaS (GitLab.com)
+version of the product:
-```shell
-export GITLAB_SIMULATE_SAAS=1
-```
+1. Export this environment variable:
+
+ ```shell
+ export GITLAB_SIMULATE_SAAS=1
+ ```
-There are many ways to pass an environment variable to your local GitLab instance.
-For example, you can create a `env.runit` file in the root of your GDK with the above snippet.
+ There are many ways to pass an environment variable to your local GitLab instance.
+ For example, you can create an `env.runit` file in the root of your GDK with the above snippet.
-#### Allow use of licensed EE feature
+1. Enable **Allow use of licensed EE features** to make licensed EE features available to projects
+ only if the project namespace's plan includes the feature.
-To enable plans per namespace turn on the `Allow use of licensed EE features` option from the settings page.
-This will make licensed EE features available to projects only if the project namespace's plan includes the feature
-or if the project is public. To enable it:
+ 1. Visit **Admin > Settings > General**.
+ 1. Expand **Account and limit**.
+ 1. Select the **Allow use of licensed EE features** checkbox.
+ 1. Click **Save changes**.
-1. If you are developing locally, follow the steps in [Simulate a SaaS instance](#simulate-a-saas-instance) to make the option available.
-1. Visit Admin > Settings > General > "Account and limit" and enable "Allow use of licensed EE features".
+1. Ensure that the group for which you want to test the EE feature, is actually using an EE plan:
+ 1. On the top bar, select **Main menu > Admin**.
+ 1. On the left sidebar, select **Overview > Groups**.
+ 1. Identify the group you want to modify, and select **Edit**.
+ 1. Scroll to **Permissions and group features**. For **Plan**, select `Ultimate`.
+ 1. Select **Save changes**.
### Run CI pipelines in a FOSS context
@@ -147,7 +155,7 @@ FOSS context as well.
To run pipelines in both contexts, add the `~"pipeline:run-as-if-foss"` label to the merge request.
-See the [As-if-FOSS jobs](pipelines.md#as-if-foss-jobs) pipelines documentation for more information.
+See the [As-if-FOSS jobs](pipelines/index.md#as-if-foss-jobs) pipelines documentation for more information.
## Separation of EE code in the backend
@@ -692,7 +700,7 @@ module EE
prepended do
params do
- requires :id, type: String, desc: 'The ID of a project'
+ requires :id, types: [String, Integer], desc: 'The ID or URL-encoded path of the project'
end
resource :projects, requirements: ::API::API::NAMESPACE_OR_PROJECT_REQUIREMENTS do
# ...
diff --git a/doc/development/event_store.md b/doc/development/event_store.md
index b9200d3be25..10dc0b1a7a9 100644
--- a/doc/development/event_store.md
+++ b/doc/development/event_store.md
@@ -351,6 +351,12 @@ RSpec.describe MergeRequests::UpdateHeadPipelineWorker do
let(:event) { pipeline_created_event }
end
+ # This shared example ensures that an published event is ignored. This might be useful for
+ # conditional dispatch testing.
+ it_behaves_like 'ignores the published event' do
+ let(:event) { pipeline_created_event }
+ end
+
it 'does something' do
# This helper directly executes `perform` ensuring that `handle_event` is called correctly.
consume_event(subscriber: described_class, event: pipeline_created_event)
diff --git a/doc/development/fe_guide/graphql.md b/doc/development/fe_guide/graphql.md
index 9ed3e551ff2..28ea84301a6 100644
--- a/doc/development/fe_guide/graphql.md
+++ b/doc/development/fe_guide/graphql.md
@@ -907,7 +907,7 @@ For example, we have a query like this:
query searchGroupsWhereUserCanTransfer {
currentUser {
id
- groups {
+ groups(after: 'somecursor') {
nodes {
id
fullName
@@ -920,9 +920,7 @@ query searchGroupsWhereUserCanTransfer {
}
```
-Here, the `groups` field doesn't have a good candidate for `keyArgs`: both
-`nodes` and `pageInfo` will be updated when we're fetching a second page.
-Setting `keyArgs` to `false` makes the update work as intended:
+Here, the `groups` field doesn't have a good candidate for `keyArgs`: we don't want to account for `after` argument because it will change on requesting subsequent pages. Setting `keyArgs` to `false` makes the update work as intended:
```javascript
typePolicies: {
diff --git a/doc/development/fe_guide/merge_request_widget_extensions.md b/doc/development/fe_guide/merge_request_widget_extensions.md
index e5e813bf921..61d3e79a080 100644
--- a/doc/development/fe_guide/merge_request_widget_extensions.md
+++ b/doc/development/fe_guide/merge_request_widget_extensions.md
@@ -85,7 +85,7 @@ special formatting is required. When the extension receives this data,
it is set to `collapsedData`. You can access `collapsedData` in any computed property or
method.
-When the user clicks **Expand**, the `fetchFullData` method is called. This method
+When the user selects **Expand**, the `fetchFullData` method is called. This method
also gets called with the props as an argument. This method **must** also return
the full data. However, this data must be correctly formatted to match the format
mentioned in the data structure section.
diff --git a/doc/development/fe_guide/registry_architecture.md b/doc/development/fe_guide/registry_architecture.md
index a87d38634d5..d86f8416db6 100644
--- a/doc/development/fe_guide/registry_architecture.md
+++ b/doc/development/fe_guide/registry_architecture.md
@@ -71,7 +71,7 @@ main pieces of the desired UI and UX of a registry page. The most important comp
secondary content, right primary and secondary content, right action, and details slots.
- `metadata-item`: represents one piece of metadata, with an icon or a link. Used primarily in the
title area.
-- `persisted-dropdown-selection`: represents a dropdown menu that stores the user selection in the
+- `persisted-dropdown-selection`: represents a menu that stores the user selection in the
`localStorage`.
- `registry-search`: implements `gl-filtered-search` with a sorting section on the right.
- `title-area`: implements the top title area of the registry. Includes: a main title, an avatar, a
diff --git a/doc/development/fe_guide/style/scss.md b/doc/development/fe_guide/style/scss.md
index aacf07fda92..98f74813231 100644
--- a/doc/development/fe_guide/style/scss.md
+++ b/doc/development/fe_guide/style/scss.md
@@ -176,7 +176,7 @@ To check if any warnings are produced by your changes, run `yarn lint:stylelint`
catch any warnings.
If the Rake task is throwing warnings you don't understand, SCSS Lint's
-documentation includes [a full list of their rules](https://stylelint.io/user-guide/rules/list/).
+documentation includes [a full list of their rules](https://stylelint.io/user-guide/rules/).
### Fixing issues
diff --git a/doc/development/fe_guide/view_component.md b/doc/development/fe_guide/view_component.md
index 662d1ad32fc..b61c23cadef 100644
--- a/doc/development/fe_guide/view_component.md
+++ b/doc/development/fe_guide/view_component.md
@@ -15,7 +15,7 @@ watch this [introduction video](https://youtu.be/akRhUbvtnmo).
## Browse components with Lookbook
-We have a [Lookbook](https://github.com/allmarkedup/lookbook) in [http://gdk.test:3000/rails/lookbook](http://gdk.test:3000/rails/lookbook) (only available in development mode) to browse and interact with ViewComponent previews.
+We have a [Lookbook](https://github.com/allmarkedup/lookbook) in `http://gdk.test:3000/rails/lookbook` (only available in development mode) to browse and interact with ViewComponent previews.
## Pajamas components
@@ -24,12 +24,12 @@ available as a ViewComponent in `app/components/pajamas`.
NOTE:
We are still in the process of creating these components, so not every Pajamas component is available as ViewComponent.
-Reach out to the [Foundations team](https://about.gitlab.com/handbook/engineering/development/dev/ecosystem/foundations/)
+Reach out to the [Foundations team](https://about.gitlab.com/handbook/engineering/development/dev/manage/foundations/)
if the component you are looking for is not yet available.
### Available components
-Consider this list a best effort. The full list can be found in [`app/components/pajamas`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/app/components/pajamas). Also see [our Lookbook](http://gdk.test:3000/rails/lookbook) for a more interactive way to browse our components.
+Consider this list a best effort. The full list can be found in [`app/components/pajamas`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/app/components/pajamas). Also see our Lookbook (`http://gdk.test:3000/rails/lookbook`) for a more interactive way to browse our components.
#### Alert
@@ -155,7 +155,7 @@ For the full list of options, see its
#### Checkbox tag
-The `Pajamas::CheckboxTagComponent` follows the [Pajamas Checkbox](https://design.gitlab.com/components/checkbox) specification.
+The `Pajamas::CheckboxTagComponent` follows the [Pajamas Checkbox](https://design.gitlab.com/components/checkbox/) specification.
The `name` argument and `label` slot are required.
@@ -176,7 +176,7 @@ For the full list of options, see its
#### Checkbox
-The `Pajamas::CheckboxComponent` follows the [Pajamas Checkbox](https://design.gitlab.com/components/checkbox) specification.
+The `Pajamas::CheckboxComponent` follows the [Pajamas Checkbox](https://design.gitlab.com/components/checkbox/) specification.
NOTE:
`Pajamas::CheckboxComponent` is used internally by the [GitLab UI form builder](haml.md#use-the-gitlab-ui-form-builder) and requires an instance of [ActionView::Helpers::FormBuilder](https://api.rubyonrails.org/v6.1.0/classes/ActionView/Helpers/FormBuilder.html) to be passed as the `form` argument.
diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md
index 00d9588d087..779010b8aa1 100644
--- a/doc/development/fe_guide/vue.md
+++ b/doc/development/fe_guide/vue.md
@@ -65,6 +65,9 @@ To do that, you can use the `data` attributes in the HTML element and query them
You should only do this while initializing the application, because the mounted element is replaced
with a Vue-generated DOM.
+The `data` attributes are [only able to accept String values](https://developer.mozilla.org/en-US/docs/Learn/HTML/Howto/Use_data_attributes#javascript_access),
+so you will need to cast or convert other variable types to String.
+
The advantage of providing data from the DOM to the Vue instance through `props` or
`provide` in the `render` function, instead of querying the DOM inside the main Vue
component, is that you avoid creating a fixture or an HTML element in the unit test.
diff --git a/doc/development/fe_guide/vue3_migration.md b/doc/development/fe_guide/vue3_migration.md
index c3ce42a80a5..aae7674d190 100644
--- a/doc/development/fe_guide/vue3_migration.md
+++ b/doc/development/fe_guide/vue3_migration.md
@@ -159,3 +159,232 @@ export default {
[In Vue 3](https://v3-migration.vuejs.org/breaking-changes/props-default-this.html),
the props default value factory is passed the raw props as an argument, and can
also access injections.
+
+## Handling libraries that do not work with `@vue/compat`
+
+**Problem**
+
+Some libraries rely on Vue.js 2 internals. They might not work with `@vue/compat`, so we need a strategy to use an updated version with Vue.js 3 while maintaining compatibility with the current codebase.
+
+**Goals**
+
+- We should add as few changes as possible to existing code to support new libraries. Instead, we should **add*- new code, which will act as **facade**, making the new version compatible with the old one
+- Switching between new and old versions should be hidden inside tooling (webpack / jest) and should not be exposed to the code
+- All facades specific to migration should live in the same directory to simplify future migration steps
+
+### Step-by-step migration
+
+In the step-by-step guide, we will be migrating [VueApollo Demo](https://gitlab.com/gitlab-org/frontend/vue3-migration-vue-apollo/-/tree/main/src/vue3compat) project. It will allow us to focus on migration specifics while avoiding nuances of complex tooling setup in the GitLab project. The project intentionally uses the same tooling as GitLab:
+
+- webpack
+- yarn
+- Vue.js + VueApollo
+
+#### Initial state
+
+Right after cloning, you could run [VueApollo Demo](https://gitlab.com/gitlab-org/frontend/vue3-migration-vue-apollo/-/tree/main/src/vue3compat) with Vue.js 2 using `yarn serve` or with Vue.js 3 (compat build) using `yarn serve:vue3`. However latter immediately crashes:
+
+```javascript
+Uncaught TypeError: Cannot read properties of undefined (reading 'loading')
+```
+
+VueApollo v3 (used for Vue.js 2) fails to initialize in Vue.js compat
+
+NOTE:
+While stubbing `Vue.version` will solve VueApollo-related issues in the demo project, it will still lose reactivity on specific scenarios, so an upgrade is still needed
+
+#### Step 1. Perform upgrade according to library docs
+
+According to [VueApollo v4 installation guide](https://v4.apollo.vuejs.org/guide/installation.html), we need to install `@vue/apollo-option` (this package provides VueApollo support for Options API) and make changes to our application:
+
+```diff
+--- a/src/index.js
++++ b/src/index.js
+@@ -1,19 +1,17 @@
+-import Vue from "vue";
+-import VueApollo from "vue-apollo";
++import { createApp, h } from "vue";
++import { createApolloProvider } from "@vue/apollo-option";
+
+ import Demo from "./components/Demo.vue";
+ import createDefaultClient from "./lib/graphql";
+
+-Vue.use(VueApollo);
+-
+-const apolloProvider = new VueApollo({
++const apolloProvider = createApolloProvider({
+ defaultClient: createDefaultClient(),
+ });
+
+-new Vue({
+- el: "#app",
+- apolloProvider,
+- render(h) {
++const app = createApp({
++ render() {
+ return h(Demo);
+ },
+ });
++app.use(apolloProvider);
++app.mount("#app");
+```
+
+You can view these changes in [01-upgrade-vue-apollo](https://gitlab.com/gitlab-org/frontend/vue3-migration-vue-apollo/-/compare/main...01-upgrade-vue-apollo) branch of demo project
+
+#### Step 2. Addressing differences in augmenting applications in Vue.js 2 and 3
+
+In Vue.js 2 tooling like `VueApollo` is initialized in a "lazy" fashion:
+
+```javascript
+// We are registering VueApollo "handler" to handle some data LATER
+Vue.use(VueApollo)
+// ...
+// apolloProvider is provided at app instantiation,
+// previously registered VueApollo will handle that
+new Vue({ /- ... */, apolloProvider })
+```
+
+In Vue.js 3 both steps were merged in one - we are immediately registering the handler and passing configuration:
+
+```javascript
+app.use(apolloProvider)
+```
+
+In order to backport this behavior, we need the following knowledge:
+
+- We can access extra options provided to Vue instance via `$options`, so extra `apolloProvider` will be visible as `this.$options.apolloProvider`
+- We can access the current `app` (in Vue.js 3 meaning) on the Vue instance via `this.$.appContext.app`
+
+NOTE:
+We're relying on non-public Vue.js 3 API in this case. However, since `@vue/compat` builds are expected to be available only for 3.2.x branch, we have reduced risks that this API will be changed
+
+With this knowledge, we can move the initialization of our tooling as early as possible in Vue2 - in the `beforeCreate()` lifecycle hook:
+
+```diff
+--- a/src/index.js
++++ b/src/index.js
+@@ -1,4 +1,4 @@
+-import { createApp, h } from "vue";
++import Vue from "vue";
+ import { createApolloProvider } from "@vue/apollo-option";
+
+ import Demo from "./components/Demo.vue";
+@@ -8,10 +8,13 @@ const apolloProvider = createApolloProvider({
+ defaultClient: createDefaultClient(),
+ });
+
+-const app = createApp({
+- render() {
++new Vue({
++ el: "#app",
++ apolloProvider,
++ render(h) {
+ return h(Demo);
+ },
++ beforeCreate() {
++ this.$.appContext.app.use(this.$options.apolloProvider);
++ },
+ });
+-app.use(apolloProvider);
+-app.mount("#app");
+```
+
+You can view these changes in [02-bring-back-new-vue](https://gitlab.com/gitlab-org/frontend/vue3-migration-vue-apollo/-/compare/01-upgrade-vue-apollo...02-bring-back-new-vue) branch of demo project
+
+#### Step 3. Recreating `VueApollo` class
+
+Vue.js 3 libraries (and Vue.js itself) have a preference for using factories like `createApp` instead of classes (previously `new Vue`)
+
+`VueApollo` class served two purposes:
+
+- constructor for creating `apolloProvider`
+- installation of apollo-related logic in components
+
+We can utilize `Vue.use(VueApollo)` code, which existed in our codebase, to hide there our mixin and avoid modification of our app code:
+
+```diff
+--- a/src/index.js
++++ b/src/index.js
+@@ -4,7 +4,26 @@ import { createApolloProvider } from "@vue/apollo-option";
+ import Demo from "./components/Demo.vue";
+ import createDefaultClient from "./lib/graphql";
+
+-const apolloProvider = createApolloProvider({
++class VueApollo {
++ constructor(...args) {
++ return createApolloProvider(...args);
++ }
++
++ // called by Vue.use
++ static install() {
++ Vue.mixin({
++ beforeCreate() {
++ if (this.$options.apolloProvider) {
++ this.$.appContext.app.use(this.$options.apolloProvider);
++ }
++ },
++ });
++ }
++}
++
++Vue.use(VueApollo);
++
++const apolloProvider = new VueApollo({
+ defaultClient: createDefaultClient(),
+ });
+
+@@ -14,7 +33,4 @@ new Vue({
+ render(h) {
+ return h(Demo);
+ },
+- beforeCreate() {
+- this.$.appContext.app.use(this.$options.apolloProvider);
+- },
+ });
+```
+
+You can view these changes in [03-recreate-vue-apollo](https://gitlab.com/gitlab-org/frontend/vue3-migration-vue-apollo/-/compare/02-bring-back-new-vue...03-recreate-vue-apollo) branch of demo project
+
+#### Step 4. Moving `VueApollo` class to a separate file and setting up an alias
+
+Now, we have almost the same code (excluding import) as in Vue.js 2 version.
+We will move our facade to the separate file and set up `webpack` conditionally execute it if `vue-apollo` is imported when using Vue.js 3:
+
+```diff
+--- a/src/index.js
++++ b/src/index.js
+@@ -1,5 +1,5 @@
+ import Vue from "vue";
+-import { createApolloProvider } from "@vue/apollo-option";
++import VueApollo from "vue-apollo";
+
+ import Demo from "./components/Demo.vue";
+ import createDefaultClient from "./lib/graphql";
+diff --git a/webpack.config.js b/webpack.config.js
+index 6160d3f..b8b955f 100644
+--- a/webpack.config.js
++++ b/webpack.config.js
+@@ -12,6 +12,7 @@ if (USE_VUE3) {
+
+ VUE3_ALIASES = {
+ vue: "@vue/compat",
++ "vue-apollo": path.resolve("src/vue3compat/vue-apollo"),
+ };
+ }
+```
+
+(moving `VueApollo` class from `index.js` to `vue3compat/vue-apollo.js` as default export is omitted for clarity)
+
+You can view these changes in [04-add-webpack-alias](https://gitlab.com/gitlab-org/frontend/vue3-migration-vue-apollo/-/compare/03-recreate-vue-apollo...04-add-webpack-alias) branch of demo project
+
+#### Step 5. Observe the results
+
+At this point, you should be able again to run **both*- Vue.js 2 version with `yarn serve` and Vue.js 3 one with `yarn serve:vue3`
+[Final MR](https://gitlab.com/gitlab-org/frontend/vue3-migration-vue-apollo/-/merge_requests/1/diffs) with all changes from previous steps displays no changes to `index.js` (application code), which was our goal
+
+### Applying this approach in the GitLab project
+
+In [commit adding VueApollo v4 support](https://gitlab.com/gitlab-org/gitlab/-/commit/e0af7e6479695a28a4fe85a88f90815aa3ce2814) we can see additional nuances not covered by step-by-step guide:
+
+- We might need to add additional imports to our facades (our code in GitLab uses `ApolloMutation` component)
+- We need to update aliases not only for webpack but also for jest so our tests could also consume our facade
diff --git a/doc/development/fe_guide/vuex.md b/doc/development/fe_guide/vuex.md
index 5047f1b7f89..19bbfa314ea 100644
--- a/doc/development/fe_guide/vuex.md
+++ b/doc/development/fe_guide/vuex.md
@@ -22,7 +22,7 @@ official [Vuex documentation](https://vuex.vuejs.org).
Vuex is composed of State, Getters, Mutations, Actions, and Modules.
-When a user clicks on an action, we need to `dispatch` it. This action `commits` a mutation that changes the state. The action itself does not update the state; only a mutation should update the state.
+When a user selects an action, we need to `dispatch` it. This action `commits` a mutation that changes the state. The action itself does not update the state; only a mutation should update the state.
## File structure
diff --git a/doc/development/feature_development.md b/doc/development/feature_development.md
index 760ba033633..76447124177 100644
--- a/doc/development/feature_development.md
+++ b/doc/development/feature_development.md
@@ -197,6 +197,7 @@ The following integration guides are internal. Some integrations require access
- [Preventing transient bugs](transient/prevention-patterns.md)
- [GitLab Application SLIs](application_slis/index.md)
- [Spam protection and CAPTCHA development guide](spam_protection_and_captcha/index.md)
+- [RuboCop development guide](rubocop_development_guide.md)
## Other GitLab Development Kit (GDK) guides
diff --git a/doc/development/feature_flags/controls.md b/doc/development/feature_flags/controls.md
index e6c3c20d50b..e804a888e98 100644
--- a/doc/development/feature_flags/controls.md
+++ b/doc/development/feature_flags/controls.md
@@ -16,7 +16,7 @@ To turn on/off features behind feature flags in any of the
GitLab-provided environments, like staging and production, you need to
have access to the [ChatOps](../chatops_on_gitlabcom.md) bot. The ChatOps bot
is currently running on the ops instance, which is different from
-[GitLab.com](https://gitlab.com) or [`dev.gitlab.org`](https://dev.gitlab.org).
+[GitLab.com](https://gitlab.com) or `dev.gitlab.org`.
Follow the ChatOps document to [request access](../chatops_on_gitlabcom.md#requesting-access).
@@ -55,8 +55,8 @@ change feature flags or you do not have access.
### Enabling a feature for pre-production testing
As a first step in a feature rollout, you should enable the feature on
-[`staging.gitlab.com`](https://staging.gitlab.com)
-and [`dev.gitlab.org`](https://dev.gitlab.org).
+`staging.gitlab.com`
+and `dev.gitlab.org`.
These two environments have different scopes.
`dev.gitlab.org` is a production CE environment that has internal GitLab Inc.
diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md
index f1cde4ae1ea..500afa8ba1d 100644
--- a/doc/development/feature_flags/index.md
+++ b/doc/development/feature_flags/index.md
@@ -435,7 +435,7 @@ For example, the following feature flags are enabled for a certain percentage of
If a project A has `:feature-set-1` enabled, there is no guarantee that project A also has `:feature-set-2` enabled.
-For more detail, see [This is how percentages work in Flipper](https://www.hackwithpassion.com/this-is-how-percentages-work-in-flipper).
+For more detail, see [This is how percentages work in Flipper](https://www.hackwithpassion.com/this-is-how-percentages-work-in-flipper/).
#### Use actors for verifying in production
diff --git a/doc/development/fips_compliance.md b/doc/development/fips_compliance.md
index 475c1a49000..c6208d45c77 100644
--- a/doc/development/fips_compliance.md
+++ b/doc/development/fips_compliance.md
@@ -67,7 +67,7 @@ listed here that also do not work properly in FIPS mode:
- [Static Application Security Testing (SAST)](../user/application_security/sast/index.md)
supports a reduced set of [analyzers](../user/application_security/sast/index.md#fips-enabled-images)
when operating in FIPS-compliant mode.
-- Advanced Search is currently not included in FIPS mode. It must not be enabled to be FIPS-compliant.
+- Advanced Search is currently not included in FIPS mode. It must not be enabled to be FIPS-compliant.
- [Gravatar or Libravatar-based profile images](../administration/libravatar.md) are not FIPS-compliant.
Additionally, these package repositories are disabled in FIPS mode:
@@ -203,7 +203,7 @@ This [GitHub pull request](https://github.com/awslabs/amazon-eks-ami/pull/898) m
it possible to create an Amazon Linux 2 EKS AMI with FIPS enabled for Kubernetes v1.21.
To build an image:
-1. [Install Packer](https://learn.hashicorp.com/tutorials/packer/get-started-install-cli).
+1. [Install Packer](https://developer.hashicorp.com/packer/tutorials/docker-get-started/get-started-install-cli).
1. Run the following:
```shell
diff --git a/doc/development/gemfile.md b/doc/development/gemfile.md
index 7d3531afb49..3c7dc19da8e 100644
--- a/doc/development/gemfile.md
+++ b/doc/development/gemfile.md
@@ -66,7 +66,7 @@ This means that new dependencies should, at a minimum, meet the following criter
When adding a new gem to our `Gemfile` or even changing versions in
`Gemfile.lock` it is strongly recommended that you
-[request a Security review](https://about.gitlab.com/handbook/engineering/security/#how-to-request-a-security-review).
+[request a Security review](https://about.gitlab.com/handbook/security/#how-to-request-a-security-review).
New gems add an extra security risk for GitLab, and it is important to
evaluate this risk before we ship this to production. Technically, just adding
a new gem and pushing to a branch in our main `gitlab` project is a security
diff --git a/doc/development/github_importer.md b/doc/development/github_importer.md
index 9ba375439f4..edf78d5f83d 100644
--- a/doc/development/github_importer.md
+++ b/doc/development/github_importer.md
@@ -78,14 +78,21 @@ individually to import this information. A
`Gitlab::GithubImport::ImportPullRequestMergedByWorker` job is scheduled for each fetched pull
request.
-### 6. Stage::ImportPullRequestsReviewsWorker
+### 6. Stage::ImportPullRequestsReviewRequestsWorker
-This worker imports the pull requests' reviews. For each pull request, this worker:
+This worker imports assigned reviewers of pull requests. For each pull request, this worker:
+
+- Fetches all assigned review requests.
+- Schedules a `Gitlab::GithubImport::PullRequests::ImportReviewRequestWorker` job for each fetched review request.
+
+### 7. Stage::ImportPullRequestsReviewsWorker
+
+This worker imports reviews of pull requests. For each pull request, this worker:
- Fetches all the pages of reviews.
- Schedules a `Gitlab::GithubImport::ImportPullRequestReviewWorker` job for each fetched review.
-### 7. Stage::ImportIssuesAndDiffNotesWorker
+### 8. Stage::ImportIssuesAndDiffNotesWorker
This worker imports all issues and pull request comments. For every issue, we
schedule a job for the `Gitlab::GithubImport::ImportIssueWorker` worker. For
@@ -101,7 +108,7 @@ label links in the same worker removes the need for performing a separate crawl
through the API data, reducing the number of API calls necessary to import a
project.
-### 8. Stage::ImportIssueEventsWorker
+### 9. Stage::ImportIssueEventsWorker
This worker imports all issues and pull request events. For every event, we
schedule a job for the `Gitlab::GithubImport::ImportIssueEventWorker` worker.
@@ -117,7 +124,7 @@ Therefore, both issues and pull requests have a common API for most related thin
NOTE:
This stage is optional and can consume significant extra import time (controlled by `Gitlab::GithubImport::Settings`).
-### 9. Stage::ImportNotesWorker
+### 10. Stage::ImportNotesWorker
This worker imports regular comments for both issues and pull requests. For
every comment, we schedule a job for the
@@ -128,7 +135,7 @@ returns comments for both issues and pull requests. This means we have to wait
for all issues and pull requests to be imported before we can import regular
comments.
-### 10. Stage::ImportAttachmentsWorker
+### 11. Stage::ImportAttachmentsWorker
This worker imports note attachments that are linked inside Markdown.
For each entity with Markdown text in the project, we schedule a job of:
@@ -147,7 +154,7 @@ Each job:
NOTE:
It's an optional stage that could consume significant extra import time (controlled by `Gitlab::GithubImport::Settings`).
-### 11. Stage::ImportProtectedBranchesWorker
+### 12. Stage::ImportProtectedBranchesWorker
This worker imports protected branch rules.
For every rule that exists on GitHub, we schedule a job of
@@ -156,7 +163,7 @@ For every rule that exists on GitHub, we schedule a job of
Each job compares the branch protection rules from GitHub and GitLab and applies
the strictest of the rules to the branches in GitLab.
-### 12. Stage::FinishImportWorker
+### 13. Stage::FinishImportWorker
This worker completes the import process by performing some housekeeping
(such as flushing any caches) and by marking the import as completed.
diff --git a/doc/development/gitlab_flavored_markdown/specification_guide/index.md b/doc/development/gitlab_flavored_markdown/specification_guide/index.md
index 95d06907aa6..17afebcf6ee 100644
--- a/doc/development/gitlab_flavored_markdown/specification_guide/index.md
+++ b/doc/development/gitlab_flavored_markdown/specification_guide/index.md
@@ -34,7 +34,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
against the official specification.
- They support [snapshot testing](#markdown-snapshot-testing) of GitLab
internal GLFM processing logic. This is accomplished by automatically
- generating YAML ["example snapshot files"](#example-snapshot-files)
+ generating YAML ["example snapshot files"](#output-example-snapshot-files)
which are used as fixtures to drive automated testing within the GitLab app.
- There are [various scripts and logic](#scripts)
which are used to accomplish the above goals.
@@ -104,13 +104,13 @@ serve as input to automated conformance tests. It is
Here are the HTML-rendered versions of the specifications:
-- [GitLab Flavored Markdown (GLFM) specification](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output/spec.html), which extends the:
+- [GitLab Flavored Markdown (GLFM) specification](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_spec/spec.html), which extends the:
- [GitHub Flavored Markdown (GFM) specification](https://github.github.com/gfm/) (rendered from the [source `spec.txt` for GFM specification](https://github.com/github/cmark-gfm/blob/master/test/spec.txt)), which extends the:
- [CommonMark specification](https://spec.commonmark.org/0.30/) (rendered from the [source `spec.txt` for CommonMark specification](https://github.com/commonmark/commonmark-spec/blob/master/spec.txt))
NOTE:
The creation of the
-[HTML-rendered version of the GitLab Flavored Markdown (GLFM) specification](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output/spec.html)
+[HTML-rendered version of the GitLab Flavored Markdown (GLFM) specification](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_spec/spec.html)
file is still pending.
However, GLFM has more complex parsing, rendering, and testing requirements than
@@ -177,7 +177,7 @@ In this context, it should not be confused with other similar or related meaning
_example_, such as
[RSpec examples](https://relishapp.com/rspec/rspec-core/docs/example-groups/basic-structure-describe-it).
-See the section on the [`glfm_official_specification_examples.md`](#glfm_official_specification_examplesmd) file
+See the section on the [`glfm_official_specification.md`](#glfm_official_specificationmd) file
for more details on the backtick-delimited Markdown+HTML example syntax.
### Parsers and renderers
@@ -341,7 +341,7 @@ The GitLab [Markdown API](../../../api/markdown.md) generates HTML
for a given Markdown string using this method.
The Markdown specified in the [Markdown examples](#markdown-examples) is used to automatically generate HTML in
-[`glfm_specification/example_snapshots/html.yml`](#glfm_specificationexample_snapshotshtmlyml) via
+[`glfm_specification/output_example_snapshots/html.yml`](#htmlyml) via
[`update-example-snapshots.rb`](#update-example-snapshotsrb-script). These examples are
used when running [Markdown snapshot testing](#markdown-snapshot-testing).
@@ -353,7 +353,7 @@ in the ProseMirror WYSIWYG editor.
Just like static HTML,
the Markdown specified in the [Markdown examples](#markdown-examples) is used to automatically generate HTML in
-[`glfm_specification/example_snapshots/html.yml`](#glfm_specificationexample_snapshotshtmlyml) via
+[`glfm_specification/output_example_snapshots/html.yml`](#htmlyml) via
[`update-example-snapshots.rb`](#update-example-snapshotsrb-script). These examples are
used when running [Markdown snapshot testing](#markdown-snapshot-testing).
@@ -387,10 +387,10 @@ Here are more details on the sources of canonical HTML examples:
version of [`spec.txt`](#spectxt).
1. For the examples which are part of the GLFM [_official specification_](#official-specifications),
the canonical HTML is manually maintained and curated via the examples contained in the
- [`glfm_official_specification_examples.md`](#glfm_official_specification_examplesmd) [input specification file](#input-specification-files).
+ [`glfm_official_specification.md`](#glfm_official_specificationmd) [input specification file](#input-specification-files).
1. For the examples which are part of the GLFM [_internal extensions_](#internal-extensions),
the canonical HTML **is never specified**, and **must be left empty in all examples** contained in
- the [`glfm_internal_extension_examples.md`](#glfm_internal_extension_examplesmd) [input specification file](#input-specification-files).
+ the [`glfm_internal_extensions.md`](#glfm_internal_extensionsmd) [input specification file](#input-specification-files).
### Canonicalization of HTML
@@ -401,7 +401,7 @@ or HTML elements, to support specific appearance and behavioral requirements.
Neither the backend nor the frontend rendering logic can directly render the clean, basic HTML
which is necessary to perform comparison to the [canonical HTML](#canonical-html)
when running [Markdown conformance testing](#markdown-conformance-testing)
-for the [GLFM official specification examples](#glfm_official_specification_examplesmd).
+for the [GLFM official specification examples](#glfm_official_specificationmd).
Nor should they be able to, because:
@@ -443,7 +443,7 @@ Fixture-based normalization should be used whenever possible, because it is simp
understand than regex-based normalization.
The [Markdown snapshot testing](#markdown-snapshot-testing) uses RSpec to generate the
-[example snapshot files](#example-snapshot-files). RSpec enables you to:
+[example snapshot files](#output-example-snapshot-files). RSpec enables you to:
- Use the same powerful fixture support and helpers as all the rest of the GitLab RSpec suite.
- Use fixtures to control the state of the database when the example snapshots are generated.
@@ -575,10 +575,10 @@ The documentation on the implementation is split into three sections:
1. [Scripts](#scripts).
1. [Specification files](#specification-files).
-1. [Example snapshot files](#example-snapshot-files):
+1. [Example snapshot files](#output-example-snapshot-files):
These YAML files are used as input data
or fixtures to drive the various tests, and are located under
- `glfm_specification/example_snapshots`. All example snapshot files are automatically
+ `glfm_specification/output_example_snapshots`. All example snapshot files are automatically
generated based on the specification files and the implementation of the parsers and renderers.
However, they can also be directly edited if necessary, such as to
test-drive an incomplete implementation.
@@ -620,8 +620,8 @@ end
subgraph input:<br/>input specification files
C[ghfm_spec_v_0.29.md] --> A
D[glfm_intro.md] --> A
- E[glfm_official_specification_examples.md] --> A
- F[glfm_internal_extension_examples.md] --> A
+ E[glfm_official_specification.md] --> A
+ F[glfm_internal_extensions.md] --> A
end
subgraph output:<br/>GLFM specification files
A --> G[spec.txt]
@@ -646,7 +646,7 @@ script, which expects canonical HTML, against the GitLab renderer implementation
`scripts/glfm/run-spec-tests.sh` is a convenience shell script which runs
conformance specs via the CommonMark standard `spec_tests.py` script,
-which uses the `glfm_specification/output/spec.txt` file and `scripts/glfm/canonicalize-html.rb`
+which uses the `glfm_specification/output_spec/spec.txt` file and `scripts/glfm/canonicalize-html.rb`
helper script to test the GLFM renderer implementations' support for rendering Markdown
specification examples to canonical HTML.
@@ -672,9 +672,9 @@ end
#### `update-example-snapshots.rb` script
The `scripts/glfm/update-example-snapshots.rb` script uses the GLFM
-`glfm_specification/output/spec.txt` specification file and the
+`glfm_specification/output_spec/spec.txt` specification file and the
`glfm_specification/input/gitlab_flavored_markdown/glfm_example_status.yml`
-file to create and update the [example snapshot](#example-snapshot-files)
+file to create and update the [example snapshot](#output-example-snapshot-files)
YAML files:
```mermaid
@@ -746,7 +746,7 @@ runs the [`update-specification.rb`](#update-specificationrb-script).
It fails with an exception and non-zero return code if running these scripts
results in any diffs to the generated and committed
[output specification files](#output-specification-files) or
-[example snapshot files](#example-snapshot-files).
+[example snapshot files](#output-example-snapshot-files).
This script is run via the `glfm-verify` CI job to ensure that all changes to the
[input specification files](#input-specification-files)
@@ -766,12 +766,16 @@ subcategories based on their usage and purpose:
[`ghfm_spec_v_0.29.md`](#github-flavored-markdown-specification) specification.
- `gitlab_flavored_markdown`: Contains all `glfm_*` files.
- `*.md` [input specification files](#input-specification-files),
- which represent the GLFM specification itself.
+ which represent the source of truth for the GLFM specification and all associated examples.
- `*.yml` [input specification configuration files](#input-specification-configuration-files),
which control various aspects of the automated GLFM scripts and processes.
- - `output`: Contains [output specification files](#output-specification-files),
- which are automatically generated from the
+ - `output_spec`: Contains the `spec.txt` and `spec.html` [output specification files](#output-specification-files),
+ which represent the [GLFM official specification](#official-specifications), and are automatically generated from the
input files by running the [`update-specification.rb`](#update-specificationrb-script) script.
+ - `output_example_snapshots`: Contains [output example snapshot files](#output-example-snapshot-files).
+ which are used to drive [snapshot testing](#markdown-snapshot-testing), and are automatically generated from the
+ input files by running the [`update-specification.rb`](#update-specificationrb-script)
+ and [`scripts/glfm/update-example-snapshots.rb`](#update-example-snapshotsrb-script) scripts.
#### Input specification files
@@ -790,31 +794,22 @@ is a copy of the official latest [GFM `spec.txt`](https://github.com/github/cmar
- When it is downloaded, the version number is added to the filename.
- The extension is changed from `*.txt` to `*.md` so that it can be handled better by Markdown editors.
-NOTE:
-For extra clarity, this file uses the `ghfm` acronym in its name instead of `gfm`, as
-explained in the [Acronyms section](#acronyms-glfm-ghfm-gfm-commonmark).
-
-##### `glfm_intro.md`
-
-[`glfm_specification/input/gitlab_flavored_markdown/glfm_intro.md`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/input/gitlab_flavored_markdown/glfm_intro.md)
-is the GitLab-specific version of the prose in the introduction section of the GLFM specification.
+It currently contains additional **Introduction** and **Appendix** prose-only header sections which do not
+contain any examples.
-- It is manually updated.
-- The `update-specification.rb` script inserts it into the generated GLFM `spec.txt` to replace
- the GitHub-specific GFM version of the introductory section.
-
-##### `glfm_canonical_examples.txt`
+All header sections which contain examples are expected to be contained within a contiguous file section
+which is delimited by:
-The `glfm_canonical_examples.txt` file is deprecated and no longer exists. It has been replaced by two files:
+1. The beginning of the second H1 header (the first one after the **Introduction** section)
+1. An `<!-- END TESTS -->` HTML comment line.
-- [`glfm_official_specification_examples.md`](#glfm_official_specification_examplesmd)
- which contains the [GLFM official specification](#official-specifications) examples.
-- [`glfm_internal_extension_examples.md`](#glfm_internal_extension_examplesmd)
- which contains the [GLFM internal extension](#internal-extensions) examples.
+NOTE:
+For extra clarity, this file uses the `ghfm` acronym in its name instead of `gfm`, as
+explained in the [Acronyms section](#acronyms-glfm-ghfm-gfm-commonmark).
-##### `glfm_official_specification_examples.md`
+##### `glfm_official_specification.md`
-[`glfm_specification/input/gitlab_flavored_markdown/glfm_official_specification_examples.md`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/input/gitlab_flavored_markdown/glfm_official_specification_examples.md)
+[`glfm_specification/input/gitlab_flavored_markdown/glfm_official_specification.md`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/input/gitlab_flavored_markdown/glfm_official_specification.md)
consists of the manually updated Markdown+HTML examples for the
[GLFM official specification](#official-specifications), and their associated documentation and descriptions.
@@ -832,7 +827,12 @@ consists of the manually updated Markdown+HTML examples for the
- `H3` header sections must be nested within `H2` header sections. They cannot be
nested directly within `H1` header sections.
-`glfm_specification/input/gitlab_flavored_markdown/glfm_official_specification_examples.md` sample entries:
+It _may_ contain additional prose-only header sections which do not contain any examples.
+
+All header sections which contain examples _must_ be contained within a contiguous file section which
+is delimited by `<!-- BEGIN TESTS -->` and `<!-- END TESTS -->` HTML comment lines.
+
+`glfm_specification/input/gitlab_flavored_markdown/glfm_official_specification.md` sample entries:
<!-- markdownlint-disable MD048 -->
@@ -864,13 +864,13 @@ bold
<!-- markdownlint-enable MD048 -->
-##### `glfm_internal_extension_examples.md`
+##### `glfm_internal_extensions.md`
-[`glfm_specification/input/gitlab_flavored_markdown/glfm_internal_extension_examples.md`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/input/gitlab_flavored_markdown/glfm_internal_extension_examples.md)
+[`glfm_specification/input/gitlab_flavored_markdown/glfm_internal_extensions.md`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/input/gitlab_flavored_markdown/glfm_internal_extensions.md)
consists of the manually updated Markdown examples for the
[GLFM internal extensions](#internal-extensions), and their associated documentation and descriptions.
-Its general format is identical to [`glfm_official_specification_examples.md`](#glfm_official_specification_examplesmd),
+Its general format is identical to [`glfm_official_specification.md`](#glfm_official_specificationmd),
consisting of `H1`, `H2`, or `H3` sections containing [Markdown examples](#markdown-examples) in the
[standard backtick-delimited `spec.txt` format](#various-markdown-specifications).
@@ -878,7 +878,12 @@ However, as described in the [canonical HTML section](#canonical-html), only the
example is specified, and the HTML portion is left empty, because internal extension examples are
never used for [Markdown conformance testing](#markdown-conformance-testing).
-`glfm_specification/input/gitlab_flavored_markdown/glfm_official_specification_examples.md` sample entries:
+It _may_ contain additional prose-only header sections which do not contain any examples.
+
+All header sections which contain examples _must_ be contained within a contiguous file section which
+is delimited by `<!-- BEGIN TESTS -->` and `<!-- END TESTS -->` HTML comment lines.
+
+`glfm_specification/input/gitlab_flavored_markdown/glfm_official_specification.md` sample entries:
NOTE:
All lines in this example are prefixed with a `|` character. This prefix helps avoid false
@@ -908,7 +913,7 @@ See the main [specification files](#specification-files) section for more contex
All of the manually curated example names in the configuration files must correspond to
an existing [Markdown example](#markdown-examples) name found in
-[`example_snapshots/examples_index.yml`](#glfm_specificationexample_snapshotsexamples_indexyml),
+[`output_example_snapshots/examples_index.yml`](#examples_indexyml),
which is automatically generated based on the [input specification files](#input-specification-files).
If there is an invalid reference to an example name that does not exist, the
@@ -943,16 +948,16 @@ controls the behavior of the [scripts](#scripts) and [tests](#types-of-markdown-
The following optional entries are supported for each example. They all default to `false`:
- `skip_update_example_snapshots`: When true, skips any addition or update of any this example's entries
- in the [`glfm_specification/example_snapshots/html.yml`](#glfm_specificationexample_snapshotshtmlyml) file
- or the [`glfm_specification/example_snapshots/prosemirror_json.yml`](#glfm_specificationexample_snapshotsprosemirror_jsonyml) file.
+ in the [`glfm_specification/output_example_snapshots/html.yml`](#htmlyml) file
+ or the [`glfm_specification/output_example_snapshots/prosemirror_json.yml`](#prosemirror_jsonyml) file.
If this value is truthy, then no other `skip_update_example_snapshot_*` entries can be truthy,
and an error is raised if any of them are.
- `skip_update_example_snapshot_html_static`: When true, skips addition or update of this example's [static HTML](#static-html)
- entry in the [`glfm_specification/example_snapshots/html.yml`](#glfm_specificationexample_snapshotshtmlyml) file.
+ entry in the [`glfm_specification/output_example_snapshots/html.yml`](#htmlyml) file.
- `skip_update_example_snapshot_html_wysiwyg`: When true, skips addition or update of this example's [WYSIWYG HTML](#wysiwyg-html)
- entry in the [`glfm_specification/example_snapshots/html.yml`](#glfm_specificationexample_snapshotshtmlyml) file.
+ entry in the [`glfm_specification/output_example_snapshots/html.yml`](#htmlyml) file.
- `skip_update_example_snapshot_prosemirror_json`: When true, skips addition or update of this example's
- entry in the [`glfm_specification/example_snapshots/prosemirror_json.yml`](#glfm_specificationexample_snapshotsprosemirror_jsonyml) file.
+ entry in the [`glfm_specification/output_example_snapshots/prosemirror_json.yml`](#prosemirror_jsonyml) file.
- `skip_running_conformance_static_tests`: When true, skips running the [Markdown conformance tests](#markdown-conformance-testing)
of the [static HTML](#static-html) for this example.
- `skip_running_conformance_wysiwyg_tests`: When true, skips running the [Markdown conformance tests](#markdown-conformance-testing)
@@ -962,7 +967,7 @@ The following optional entries are supported for each example. They all default
- `skip_running_snapshot_wysiwyg_html_tests`: When true, skips running the [Markdown snapshot tests](#markdown-snapshot-testing)
of the [WYSIWYG HTML](#wysiwyg-html) for this example.
- `skip_running_snapshot_prosemirror_json_tests`: When true, skips running the [Markdown snapshot tests](#markdown-snapshot-testing)
- of the [ProseMirror JSON](#glfm_specificationexample_snapshotsprosemirror_jsonyml) for this example.
+ of the [ProseMirror JSON](#prosemirror_jsonyml) for this example.
`glfm_specification/input/gitlab_flavored_markdown/glfm_example_status.yml` sample entry:
@@ -1033,9 +1038,9 @@ or [environment-variable-based normalization](#environment-variable-based-normal
snapshot:
07_01_00_href: *07_01_00_href
07_01_00_id: *07_01_00_id
- wysiwyg:
- 07_01_00_href: *07_01_00_href
- 07_01_00_id: *07_01_00_id
+ wysiwyg:
+ 07_01_00_href: *07_01_00_href
+ 07_01_00_id: *07_01_00_id
prosemirror_json:
07_01_00_href: *07_01_00_href
07_01_00_id: *07_01_00_id
@@ -1093,36 +1098,66 @@ move or copy a hosted version of the rendered HTML `spec.html` version to anothe
##### spec.txt
-[`glfm_specification/output/spec.txt`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output/spec.txt)
+[`glfm_specification/output_spec/spec.txt`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_spec/spec.txt)
is a Markdown specification file, in the standard format
with prose and Markdown + canonical HTML examples.
-It also serves as input for other scripts such as `update-example-snapshots.rb`
-and `run-spec-tests.sh`.
+It also serves as input for other scripts such as
+`run-spec-tests.sh`.
It is generated or updated by the `update-specification.rb` script, using the
[input specification files](#input-specification-files) as input.
See the [`update-specification.rb` script section](#update-specificationrb-script)
for a diagram and more description on this process.
+NOTE:
+Even though `spec.txt` is a Markdown file, it is named with a `*.txt` extension
+for consistency with the GFM and CommonMark specifications. All other GLFM
+Markdown files are named with a `*.md` extension for compatibility with
+various editors to enable Markdown formatting and syntax highlighting.
+
##### spec.html
-[`glfm_specification/output/spec.html`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output/spec.html)
+[`glfm_specification/output_spec/spec.html`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_spec/spec.html)
is an HTML file, rendered based on `spec.txt`. It is
-also generated (or updated) by the `update-specification.rb` script at the same time as
+generated (or updated) by the `update-specification.rb` script at the same time as
`spec.txt`.
It corresponds to the HTML-rendered versions of the
"GitHub Flavored Markdown" (<abbr title="GitHub Flavored Markdown">GFM</abbr>)
[specification](https://github.github.com/gfm/)
-and the [CommonMark specification](https://spec.commonmark.org/0.30/).
+and the [CommonMark specification](https://spec.commonmark.org/0.30/), but only
+contains GitLab Flavored Markdown (GLFM) examples.
-### Example snapshot files
+NOTE:
+
+The formatting of this HTML is currently not identical to the GFM and CommonMark
+HTML-rendered specification. It is only the raw output of running `spec.txt` through
+the GitLab Markdown renderer. Properly formatting the HTML will require
+duplicating or reusing the Lua script and template from the CommonMark project:
+[CommonMark Makefile](https://github.com/commonmark/commonmark-spec/blob/master/Makefile#L11)
+
+#### Output example snapshot files
+
+The `output_example_snapshots` directory contains files which are generated by the
+`update-specification.rb` and `update-example-snapshots.rb` scripts based off of the files in the
+`glfm_specification/input` directory.
+
+The `output-specification.rb` script generates
+`output_snapshot_examples/snapshot_spec.md` and `output_snapshot_examples/snapshot_spec.html`.
+These files are Markdown specification files containing examples generated based on input files,
+similar to the `output_spec/spec.txt` and `output_spec/spec.html`, with the following differences:
-The `example_snapshots` directory contains files which are generated by the
-`update-example-snapshots.rb` script based off of the files in the
-`glfm_specification/input` directory. They are used as fixtures to drive the
-various Markdown snapshot tests.
+1. They contain a superset of _all_ examples from
+ the Commonmark, GitHub Flavored Markdown, and GitLab Flavored Markdown specifications, whereas
+ `spec.*` only contains the GLFM specification. This is to provide a single place to refer to
+ all examples when working with [snapshot testing](#markdown-snapshot-testing).
+1. They contain _only_ header sections which contain examples. They do not contain any prose-only
+ sections which do not contain examples.
+
+The `update-example-snapshots.rb` script generates the various
+`output_snapshot_examples/*.yml` files, which
+are used as fixtures to drive the [snapshot testing](#markdown-snapshot-testing).
After the entire GLFM implementation is complete for both backend (Ruby) and
frontend (JavaScript), all of these YAML files can be automatically generated.
@@ -1131,9 +1166,47 @@ key in `glfm_specification/input/gitlab_flavored_markdown/glfm_example_status.ym
can be used to disable automatic generation of some examples. They can instead
be manually edited as necessary to help drive the implementations.
-#### `glfm_specification/example_snapshots/examples_index.yml`
+##### `snapshot_spec.md`
+
+[`glfm_specification/output_example_snapshots/snapshot_spec.md`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_example_snapshots/snapshot_spec.md)
+is a Markdown file, containing standard Markdown + canonical HTML examples like [`spec.txt`](#spectxt).
+
+It is generated or updated by the `update-specification.rb` script, using the
+[input specification files](#input-specification-files) as input.
+See the [`update-specification.rb` script section](#update-specificationrb-script)
+for a diagram and more description on this process. It also serves as input for other
+scripts such as `update-example-snapshots.rb`.
+
+It is similar to [`spec.txt`](#spectxt), with the following differences:
+
+1. [`spec.txt`](#spectxt) contains only examples for GitLab Flavored Markdown, but
+ `snapshot_spec.md` also contains the full superset of examples from the
+ "GitHub Flavored Markdown" (<abbr title="GitHub Flavored Markdown">GFM</abbr>)[specification](https://github.github.com/gfm/)
+ and the [CommonMark specification](https://spec.commonmark.org/0.30/) specifications.
+1. [`spec.txt`](#spectxt) represents the full GLFM specification, including additional header sections
+ containing only explanatory prose and no examples, but `snapshot_spec.md` consists of only
+ header sections which contain examples. This is because its purpose is to serve as input for
+ the other [`output example snapshot files`](#output-example-snapshot-files) - it is not intended
+ to serve as an actual [specification file](#output-specification-files)
+ like [`spec.txt`](#spectxt) or [`spec.html`](#spechtml).
+
+##### `snapshot_spec.html`
+
+[`glfm_specification/output_snapshot_examples/snapshot_spec.html`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_snapshot_examples/snapshot_spec.html)
+is an HTML file, rendered based on `snapshot_spec.md`. It is
+generated (or updated) by the `update-specification.rb` script at the same time as
+`snapshot_spec.md`.
+
+NOTE:
+The formatting of this HTML is currently not identical to the GFM and CommonMark
+HTML-rendered specification. It is only the raw output of running `snapshot_spec.md` through
+the GitLab Markdown renderer. Properly formatting the HTML will require
+duplicating or reusing the Lua script and template from the CommonMark project:
+[CommonMark Makefile](https://github.com/commonmark/commonmark-spec/blob/master/Makefile#L11)
+
+##### `examples_index.yml`
-[`glfm_specification/example_snapshots/examples_index.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/example_snapshots/examples_index.yml)
+[`glfm_specification/output_example_snapshots/examples_index.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_example_snapshots/examples_index.yml)
is the main list of all
CommonMark, GFM, and GLFM example names, each with a uniquely identifying name.
@@ -1142,9 +1215,9 @@ CommonMark, GFM, and GLFM example names, each with a uniquely identifying name.
- For CommonMark and GFM examples,
these sections originally came from the GFM `spec.txt`.
- For GLFM examples, it is generated from
- [`glfm_official_specification_examples.md`](#glfm_official_specification_examplesmd) and [`glfm_internal_extension_examples.md`](#glfm_internal_extension_examplesmd).
+ [`glfm_official_specification.md`](#glfm_official_specificationmd) and [`glfm_internal_extensions.md`](#glfm_internal_extensionsmd).
- It also contains extra metadata about each example, such as:
- 1. `spec_txt_example_position` - The position of the example in the generated GLFM `spec.txt` file.
+ 1. `spec_example_position` - The position of the example in the generated GLFM `spec.txt` file.
- This value is the index order of each individual Markdown + HTML5 example in the file. It is _not_
the line number in the file.
- This value can be used to locate the example in the rendered `spec.html` file, because the standard
@@ -1159,49 +1232,49 @@ CommonMark, GFM, and GLFM example names, each with a uniquely identifying name.
examples where multiple examples exist for the same Section 7 subsection are
added to the end of the sub-section.
-`glfm_specification/example_snapshots/examples_index.yml` sample entries:
+`examples_index.yml` sample entries:
```yaml
02_01_00_preliminaries_characters_and_lines_1:
- spec_txt_example_position: 1
+ spec_example_position: 1
source_specification: commonmark
03_01_00_blocks_and_inlines_precedence_1:
- spec_txt_example_position: 12
+ spec_example_position: 12
source_specification: commonmark
05_03_00_container_blocks_task_list_items_1:
- spec_txt_example_position: 279
+ spec_example_position: 279
source_specification: github
06_04_00_inlines_emphasis_and_strong_emphasis_1:
- spec_txt_example_position: 360
+ spec_example_position: 360
source_specification: github
07_01_00_audio_link_1:
- spec_txt_example_position: 301
+ spec_example_position: 301
source_specification: gitlab
```
-#### `glfm_specification/example_snapshots/markdown.yml`
+##### `markdown.yml`
-[`glfm_specification/example_snapshots/markdown.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/example_snapshots/markdown.yml) contains the original Markdown
-for each entry in `glfm_specification/example_snapshots/examples_index.yml`
+[`glfm_specification/output_example_snapshots/markdown.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_example_snapshots/markdown.yml) contains the original Markdown
+for each entry in `glfm_specification/output_example_snapshots/examples_index.yml`:
- For CommonMark and GFM Markdown,
it is generated (or updated) from the standard GFM
`spec.txt` using the `update-example-snapshots.rb` script.
- For GLFM, it is generated (or updated) from the
- [`glfm_official_specification_examples.md`](#glfm_official_specification_examplesmd) and [`glfm_internal_extension_examples.md`](#glfm_internal_extension_examplesmd)
+ [`glfm_official_specification.md`](#glfm_official_specificationmd) and [`glfm_internal_extensions.md`](#glfm_internal_extensionsmd)
input specification files.
-`glfm_specification/example_snapshots/markdown.yml` sample entry:
+`glfm_specification/output_example_snapshots/markdown.yml` sample entry:
```yaml
06_04_00_inlines_emphasis_and_strong_emphasis_1: |
*foo bar*
```
-#### `glfm_specification/example_snapshots/html.yml`
+##### `html.yml`
-[`glfm_specification/example_snapshots/html.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/example_snapshots/html.yml)
-contains the HTML for each entry in `glfm_specification/example_snapshots/examples_index.yml`
+[`glfm_specification/output_example_snapshots/html.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_example_snapshots/html.yml)
+contains the HTML for each entry in `glfm_specification/output_example_snapshots/examples_index.yml`:
Three types of entries exist, with different HTML for each:
@@ -1209,16 +1282,16 @@ Three types of entries exist, with different HTML for each:
- The ["Canonical"](#canonicalization-of-html) HTML.
- For CommonMark and GFM examples, the HTML comes from the examples in `spec.txt`.
- For [GLFM official specification](#official-specifications) examples, it is generated/updated from
- [`glfm_official_specification_examples.md`](#glfm_official_specification_examplesmd).
+ [`glfm_official_specification.md`](#glfm_official_specificationmd).
- **Static**
- This is the static (backend (Ruby)-generated) HTML for each entry in
- `glfm_specification/example_snapshots/examples_index.yml`.
+ `glfm_specification/output_example_snapshots/examples_index.yml`.
- It is generated/updated from backend [Markdown API](../../../api/markdown.md)
(or the underlying internal classes) via the `update-example-snapshots.rb` script,
but can be manually updated for static examples with incomplete implementations.
- **WYSIWYG**
- The WYSIWYG (frontend, JavaScript-generated) HTML for each entry in
- `glfm_specification/example_snapshots/examples_index.yml`.
+ `glfm_specification/output_example_snapshots/examples_index.yml`.
- It is generated (or updated) from the frontend Content Editor implementation via the
`update-example-snapshots.rb` script. It can be manually updated for WYSIWYG
examples with incomplete implementations.
@@ -1226,7 +1299,7 @@ Three types of entries exist, with different HTML for each:
Any exceptions or failures which occur when generating HTML are replaced with an
`Error - check implementation` value.
-`glfm_specification/example_snapshots/html.yml` sample entry:
+`glfm_specification/output_example_snapshots/html.yml` sample entry:
```yaml
06_04_00_inlines_emphasis_and_strong_emphasis_1:
@@ -1242,16 +1315,16 @@ NOTE:
The actual `static` or `WYSIWYG` entries may differ from the example `html.yml`,
depending on how the implementations evolve.
-#### `glfm_specification/example_snapshots/prosemirror_json.yml`
+##### `prosemirror_json.yml`
-[`glfm_specification/example_snapshots/prosemirror_json.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/example_snapshots/prosemirror_json.yml)
-contains the ProseMirror JSON for each entry in `glfm_specification/example_snapshots/examples_index.yml`
+[`glfm_specification/output_example_snapshots/prosemirror_json.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/glfm_specification/output_example_snapshots/prosemirror_json.yml)
+contains the ProseMirror JSON for each entry in `glfm_specification/output_example_snapshots/examples_index.yml`
- It is generated (or updated) from the frontend code via the `update-example-snapshots.rb`
script, but can be manually updated for examples with incomplete implementations.
- Any exceptions or failures when generating are replaced with a `Error - check implementation` value.
-`glfm_specification/example_snapshots/prosemirror_json.yml` sample entry:
+`glfm_specification/output_example_snapshots/prosemirror_json.yml` sample entry:
```yaml
06_04_00_inlines_emphasis_and_strong_emphasis_1: |-
@@ -1291,13 +1364,13 @@ This section describes how the scripts can be used to manage the GLFM specificat
1. If you are working on an in-progress feature or bug, make any necessary manual updates to the [input specification files](#input-specification-files). This may include:
1. Updating the canonical Markdown or HTML examples in
- [`glfm_official_specification_examples.md`](#glfm_official_specification_examplesmd) or [`glfm_internal_extension_examples.md`](#glfm_internal_extension_examplesmd).
+ [`glfm_official_specification.md`](#glfm_official_specificationmd) or [`glfm_internal_extensions.md`](#glfm_internal_extensionsmd).
1. Updating `glfm_specification/input/gitlab_flavored_markdown/glfm_example_status.yml` to reflect the current status of the examples or tests.
1. Run [`update-specification.rb`](#update-specificationrb-script) to update the `spec.txt` to reflect any changes which were made to the [input specification files](#input-specification-files).
1. Visually inspect and confirm any resulting changes to the [output specification files](#output-specification-files).
-1. Run [`update-example-snapshots.rb`](#update-example-snapshotsrb-script) to update the [example snapshot files](#example-snapshot-files).
-1. Visually inspect and confirm any resulting changes to the [example snapshot files](#example-snapshot-files).
+1. Run [`update-example-snapshots.rb`](#update-example-snapshotsrb-script) to update the [example snapshot files](#output-example-snapshot-files).
+1. Visually inspect and confirm any resulting changes to the [example snapshot files](#output-example-snapshot-files).
1. Run [`run-snapshot-tests.sh`](#run-snapshot-testssh-script) as a convenience script to run all relevant frontend (RSpec) and backend (Jest) tests which use the example snapshots.
1. Any frontend or backend snapshot test may also be run individually.
1. All frontend and backend tests are also run as part of the continuous integration suite, as they normally are.
-1. Commit any changes to the [input specification files](#input-specification-files), [output specification files](#output-specification-files), or [example snapshot files](#example-snapshot-files).
+1. Commit any changes to the [input specification files](#input-specification-files), [output specification files](#output-specification-files), or [example snapshot files](#output-example-snapshot-files).
diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md
index 197c7616d82..b561ebc4285 100644
--- a/doc/development/go_guide/index.md
+++ b/doc/development/go_guide/index.md
@@ -408,7 +408,7 @@ should be used in functions that can block and passed as the first parameter.
Every project should have a `Dockerfile` at the root of their repository, to
build and run the project. Since Go program are static binaries, they should
not require any external dependency, and shells in the final image are useless.
-We encourage [Multistage builds](https://docs.docker.com/develop/develop-images/multistage-build/):
+We encourage [Multistage builds](https://docs.docker.com/build/building/multi-stage/):
- They let the user build the project with the right Go version and
dependencies.
diff --git a/doc/development/graphql_guide/pagination.md b/doc/development/graphql_guide/pagination.md
index 7b9d2158c60..daa39875119 100644
--- a/doc/development/graphql_guide/pagination.md
+++ b/doc/development/graphql_guide/pagination.md
@@ -20,7 +20,7 @@ See the [general pagination guidelines section](../database/pagination_guideline
This is the traditional, page-by-page pagination, that is most common,
and used across much of GitLab. You can recognize it by
-a list of page numbers near the bottom of a page, which, when clicked,
+a list of page numbers near the bottom of a page, which, when selected,
take you to that page of results.
For example, when you select **Page 100**, we send `100` to the
@@ -248,7 +248,7 @@ incorrect sort order.
There are times when the [complexity of sorting](#limitations-of-query-complexity)
is more than our keyset pagination can handle.
-For example, in [`IssuesResolver`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/resolvers/issues_resolver.rb),
+For example, in [`ProjectIssuesResolver`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/resolvers/project_issues_resolver.rb),
when sorting by `priority_asc`, we can't use keyset pagination as the ordering is much
too complex. For more information, read [`issuable.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/concerns/issuable.rb).
diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md
index 158eb19764b..91e2efcb2a3 100644
--- a/doc/development/i18n/externalization.md
+++ b/doc/development/i18n/externalization.md
@@ -463,12 +463,17 @@ use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. Make sure to
The `n_` and `n__` methods should only be used to fetch pluralized translations of the same
string, not to control the logic of showing different strings for different
-quantities. Some languages have different quantities of target plural forms.
+quantities. For similar strings, pluralize the entire sentence to provide the most context
+when translating. Some languages have different quantities of target plural forms.
For example, Chinese (simplified) has only one target plural form in our
translation tool. This means the translator has to choose to translate only one
of the strings, and the translation doesn't behave as intended in the other case.
-For example, use this:
+Below are some examples:
+
+Example 1: For different strings
+
+Use this:
```ruby
if selected_projects.one?
@@ -485,6 +490,27 @@ Instead of this:
format(n_("%{project_name}", "%d projects selected", count), project_name: 'GitLab')
```
+Example 2: For similar strings
+
+Use this:
+
+```ruby
+n__('Last day', 'Last %d days', days.length)
+```
+
+Instead of this:
+
+```ruby
+# incorrect usage example
+const pluralize = n__('day', 'days', days.length)
+
+if (days.length === 1 ) {
+ return sprintf(s__('Last %{pluralize}', pluralize)
+}
+
+return sprintf(s__('Last %{dayNumber} %{pluralize}'), { dayNumber: days.length, pluralize })
+```
+
### Namespaces
A namespace is a way to group translations that belong together. They provide context to our
diff --git a/doc/development/i18n/merging_translations.md b/doc/development/i18n/merging_translations.md
index 2a086c796c9..f98e5119916 100644
--- a/doc/development/i18n/merging_translations.md
+++ b/doc/development/i18n/merging_translations.md
@@ -36,7 +36,7 @@ it should be disapproved in Crowdin. Our strings must use [variables](externaliz
for HTML instead.
It might be useful to pause the integration on the Crowdin side for a
-moment so translations don't keep coming. You can do this by clicking
+moment so translations don't keep coming. You can do this by selecting
**Pause sync** on the [Crowdin integration settings page](https://translate.gitlab.com/project/gitlab-ee/settings#integration).
## Merging translations
diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md
index b5d57f9912b..8a4ad73efe3 100644
--- a/doc/development/i18n/proofreader.md
+++ b/doc/development/i18n/proofreader.md
@@ -58,6 +58,7 @@ are very appreciative of the work done by translators and proofreaders!
- French
- Davy Defaud - [GitLab](https://gitlab.com/DevDef), [Crowdin](https://crowdin.com/profile/DevDef)
- Germain Gorisse - [GitLab](https://gitlab.com/ggorisse), [Crowdin](https://crowdin.com/profile/germaingorisse)
+ - Xavier Delatour - [GitLab](https://gitlab.com/xdelatour), [Crowdin](https://crowdin.com/profile/xdelatour)
- Galician
- Antón Méixome - [Crowdin](https://crowdin.com/profile/meixome)
- Pedro Garcia - [GitLab](https://gitlab.com/pedgarrod), [Crowdin](https://crowdin.com/profile/breaking_pitt)
diff --git a/doc/development/import_export.md b/doc/development/import_export.md
index f864dd3b678..9aab7f38dbb 100644
--- a/doc/development/import_export.md
+++ b/doc/development/import_export.md
@@ -6,116 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Import/Export development documentation
-Troubleshooting and general development guidelines and tips for the [Import/Export feature](../user/project/settings/import_export.md).
+General development guidelines and tips for the [Import/Export feature](../user/project/settings/import_export.md).
<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> This document is originally based on the [Import/Export 201 presentation available on YouTube](https://www.youtube.com/watch?v=V3i1OfExotE).
-## Troubleshooting commands
-
-Finds information about the status of the import and further logs using the JID:
-
-```ruby
-# Rails console
-Project.find_by_full_path('group/project').import_state.slice(:jid, :status, :last_error)
-> {"jid"=>"414dec93f941a593ea1a6894", "status"=>"finished", "last_error"=>nil}
-```
-
-```shell
-# Logs
-grep JID /var/log/gitlab/sidekiq/current
-grep "Import/Export error" /var/log/gitlab/sidekiq/current
-grep "Import/Export backtrace" /var/log/gitlab/sidekiq/current
-tail /var/log/gitlab/gitlab-rails/importer.log
-```
-
-## Troubleshooting performance issues
-
-Read through the current performance problems using the Import/Export below.
-
-### OOM errors
-
-Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](../administration/operations/sidekiq_memory_killer.md):
-
-```shell
-SIDEKIQ_MEMORY_KILLER_MAX_RSS = 2000000
-SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS = 3000000
-SIDEKIQ_MEMORY_KILLER_GRACE_TIME = 900
-```
-
-An import status `started`, and the following Sidekiq logs signal a memory issue:
-
-```shell
-WARN: Work still in progress <struct with JID>
-```
-
-### Timeouts
-
-Timeout errors occur due to the `Gitlab::Import::StuckProjectImportJobsWorker` marking the process as failed:
-
-```ruby
-module Gitlab
- module Import
- class StuckProjectImportJobsWorker
- include Gitlab::Import::StuckImportJob
- # ...
- end
- end
-end
-
-module Gitlab
- module Import
- module StuckImportJob
- # ...
- IMPORT_JOBS_EXPIRATION = 15.hours.to_i
- # ...
- def perform
- stuck_imports_without_jid_count = mark_imports_without_jid_as_failed!
- stuck_imports_with_jid_count = mark_imports_with_jid_as_failed!
-
- track_metrics(stuck_imports_with_jid_count, stuck_imports_without_jid_count)
- end
- # ...
- end
- end
-end
-```
-
-```shell
-Marked stuck import jobs as failed. JIDs: xyz
-```
-
-```plaintext
- +-----------+ +-----------------------------------+
- |Export Job |--->| Calls ActiveRecord `as_json` and |
- +-----------+ | `to_json` on all project models |
- +-----------------------------------+
-
- +-----------+ +-----------------------------------+
- |Import Job |--->| Loads all JSON in memory, then |
- +-----------+ | inserts into the DB in batches |
- +-----------------------------------+
-```
-
-### Problems and solutions
-
-| Problem | Possible solutions |
-| -------- | -------- |
-| [Slow JSON](https://gitlab.com/gitlab-org/gitlab/-/issues/25251) loading/dumping models from the database | [split the worker](https://gitlab.com/gitlab-org/gitlab/-/issues/25252) |
-| | Batch export
-| | Optimize SQL
-| | Move away from `ActiveRecord` callbacks (difficult)
-| High memory usage (see also some [analysis](https://gitlab.com/gitlab-org/gitlab/-/issues/18857) | DB Commit sweet spot that uses less memory |
-| | [Netflix Fast JSON API](https://github.com/Netflix/fast_jsonapi) may help |
-| | Batch reading/writing to disk and any SQL
-
-### Temporary solutions
-
-While the performance problems are not tackled, there is a process to workaround
-importing big projects, using a foreground import:
-
-[Foreground import](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/5384) of big projects for customers.
-(Using the import template in the [infrastructure tracker](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues))
-
## Security
The Import/Export feature is constantly updated (adding new things to export), however
diff --git a/doc/development/integrations/index.md b/doc/development/integrations/index.md
index a0a81775391..2639da818c6 100644
--- a/doc/development/integrations/index.md
+++ b/doc/development/integrations/index.md
@@ -10,9 +10,9 @@ description: "GitLab's development guidelines for Integrations"
This page provides development guidelines for implementing [GitLab integrations](../../user/project/integrations/index.md),
which are part of our [main Rails project](https://gitlab.com/gitlab-org/gitlab).
-Also see our [direction page](https://about.gitlab.com/direction/ecosystem/integrations/) for an overview of our strategy around integrations.
+Also see our [direction page](https://about.gitlab.com/direction/manage/integrations/) for an overview of our strategy around integrations.
-This guide is a work in progress. You're welcome to ping `@gitlab-org/ecosystem-stage/integrations`
+This guide is a work in progress. You're welcome to ping `@gitlab-org/manage/integrations`
if you need clarification or spot any outdated information.
## Add a new integration
diff --git a/doc/development/integrations/jira_connect.md b/doc/development/integrations/jira_connect.md
index fb0d239db46..d4215662db4 100644
--- a/doc/development/integrations/jira_connect.md
+++ b/doc/development/integrations/jira_connect.md
@@ -109,4 +109,4 @@ The following steps describe setting up an environment to test the GitLab OAuth
1. Go to **Admin > Settings > General**.
1. Scroll down and expand the GitLab for Jira App section.
1. Go to [gitpod.io/variables](https://gitpod.io/variables).
-1. Paste the Application ID into the **Jira Connect Application ID** field and click **Save changes**
+1. Paste the Application ID into the **Jira Connect Application ID** field and select **Save changes**.
diff --git a/doc/development/integrations/secure.md b/doc/development/integrations/secure.md
index f40caf29cfa..787b46133ad 100644
--- a/doc/development/integrations/secure.md
+++ b/doc/development/integrations/secure.md
@@ -157,7 +157,7 @@ If the scanner requires a fully functional Linux environment,
it is recommended to use a [Debian](https://www.debian.org/intro/about) "slim" distribution or [Alpine Linux](https://www.alpinelinux.org/).
If possible, it is recommended to build the image from scratch, using the `FROM scratch` instruction,
and to compile the scanner with all the libraries it needs.
-[Multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/)
+[Multi-stage builds](https://docs.docker.com/build/building/multi-stage/)
might also help with keeping the image small.
To keep an image size small, consider using [dive](https://github.com/wagoodman/dive#dive) to analyze layers in a Docker image to
@@ -314,7 +314,7 @@ This documentation gives an overview of the report JSON format,
as well as recommendations and examples to help integrators set its fields.
The format is extensively described in the documentation of
[SAST](../../user/application_security/sast/index.md#reports-json-format),
-[DAST](../../user/application_security/dast/index.md#reports),
+[DAST](../../user/application_security/dast/proxy-based.md#reports),
[Dependency Scanning](../../user/application_security/dependency_scanning/index.md#reports-json-format),
and [Container Scanning](../../user/application_security/container_scanning/index.md#reports-json-format)
diff --git a/doc/development/integrations/secure_partner_integration.md b/doc/development/integrations/secure_partner_integration.md
index ba0654971d3..bcbc02d4827 100644
--- a/doc/development/integrations/secure_partner_integration.md
+++ b/doc/development/integrations/secure_partner_integration.md
@@ -77,7 +77,7 @@ and complete an integration with the Secure stage.
1. Get a test account to begin developing your integration. You can
request a [GitLab.com Subscription Sandbox](https://about.gitlab.com/partners/technology-partners/integrate/#gitlabcom-subscription-sandbox-request)
or an [EE Developer License](https://about.gitlab.com/partners/technology-partners/integrate/#requesting-ultimate-dev-license-for-rd).
-1. Provide a [pipeline job](../../development/pipelines.md)
+1. Provide a [pipeline job](../../development/pipelines)
template that users could integrate into their own GitLab pipelines.
1. Create a report artifact with your pipeline jobs.
1. Ensure your pipeline jobs create a report artifact that GitLab can process
diff --git a/doc/development/internal_api/index.md b/doc/development/internal_api/index.md
index 9b77a20a0d6..68d9b88bc05 100644
--- a/doc/development/internal_api/index.md
+++ b/doc/development/internal_api/index.md
@@ -496,8 +496,6 @@ metric counters.
| Attribute | Type | Required | Description |
|:---------------------------------------------------------------------------|:--------------|:---------|:-----------------------------------------------------------------------------------------------------------------|
-| `gitops_sync_count` (DEPRECATED) | integer | no | The number to increase the `gitops_sync` counter by |
-| `k8s_api_proxy_request_count` (DEPRECATED) | integer | no | The number to increase the `k8s_api_proxy_request` counter by |
| `counters` | hash | no | The number to increase the `k8s_api_proxy_request` counter by |
| `counters["k8s_api_proxy_request"]` | integer | no | The number to increase the `k8s_api_proxy_request` counter by |
| `counters["gitops_sync"]` | integer | no | The number to increase the `gitops_sync` counter by |
@@ -512,7 +510,7 @@ Example Request:
```shell
curl --request POST --header "Gitlab-Kas-Api-Request: <JWT token>" --header "Content-Type: application/json" \
- --data '{"gitops_sync_count":1}' "http://localhost:3000/api/v4/internal/kubernetes/usage_metrics"
+ --data '{"counters": {"gitops_sync":1}}' "http://localhost:3000/api/v4/internal/kubernetes/usage_metrics"
```
### Create Starboard vulnerability
diff --git a/doc/development/kubernetes.md b/doc/development/kubernetes.md
index e7764326db3..332a3c4ab09 100644
--- a/doc/development/kubernetes.md
+++ b/doc/development/kubernetes.md
@@ -155,15 +155,9 @@ Mitigation strategies include:
## Debugging Kubernetes integrations
Logs related to the Kubernetes integration can be found in
-[`kubernetes.log`](../administration/logs/index.md#kuberneteslog). On a local
+[`kubernetes.log`](../administration/logs/index.md#kuberneteslog-deprecated). On a local
GDK install, these logs are present in `log/kubernetes.log`.
-Some services such as
-[`Clusters::Applications::InstallService`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/clusters/applications/install_service.rb#L18)
-rescues `StandardError` which can make it harder to debug issues in an
-development environment. The current workaround is to temporarily
-comment out the `rescue` in your local development source.
-
You can also follow the installation logs to debug issues related to
installation. Once the installation/upgrade is underway, wait for the
pod to be created. Then run the following to obtain the pods logs as
diff --git a/doc/development/licensed_feature_availability.md b/doc/development/licensed_feature_availability.md
deleted file mode 100644
index b007df8f1da..00000000000
--- a/doc/development/licensed_feature_availability.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: 'ee_features.md'
-remove_date: '2022-10-08'
----
-
-This document was moved to [another location](ee_features.md).
-
-<!-- This redirect file can be deleted after <2022-10-08>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/development/merge_request_concepts/widget_extensions.md b/doc/development/merge_request_concepts/widget_extensions.md
deleted file mode 100644
index 097e9155f2b..00000000000
--- a/doc/development/merge_request_concepts/widget_extensions.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../fe_guide/merge_request_widget_extensions.md'
-remove_date: '2022-10-27'
----
-
-This document was moved to [another location](../fe_guide/merge_request_widget_extensions.md).
-
-<!-- This redirect file can be deleted after <2022-10-27>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md
index f59c1fd8368..5764c876e4d 100644
--- a/doc/development/migration_style_guide.md
+++ b/doc/development/migration_style_guide.md
@@ -52,9 +52,18 @@ work it needs to perform and how long it takes to complete:
of release manager through the [post-deploy migration pipeline](https://gitlab.com/gitlab-org/release/docs/-/blob/master/general/post_deploy_migration/readme.md#how-to-determine-if-a-post-deploy-migration-has-been-executed-on-gitlabcom).
These migrations can be used for schema changes that aren't critical for the application to operate, or data migrations that take at most a few minutes.
Common examples for schema changes that should run post-deploy include:
+
- Clean-ups, like removing unused columns.
- Adding non-critical indices on high-traffic tables.
- Adding non-critical indices that take a long time to create.
+
+ These migrations should not be used for schema changes that are critical for the application to operate. Making such
+ schema changes in a post-deployment migration have caused issues in the past, for example [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/378582).
+ Changes that should always be a regular schema migration and not be executed in a post-deployment migration include:
+
+ - Creating a new table, example: `create_table`.
+ - Adding a new column to an existing table, example: `add_column`.
+
1. [**Batched background migrations.**](database/batched_background_migrations.md) These aren't regular Rails migrations, but application code that is
executed via Sidekiq jobs, although a post-deployment migration is used to schedule them. Use them only for data migrations that
exceed the timing guidelines for post-deploy migrations. Batched background migrations should _not_ change the schema.
@@ -897,9 +906,14 @@ end
Table **has records** but **no foreign keys**:
-- First release: Remove the application code related to the table, such as models,
-controllers and services.
-- Second release: Use the `drop_table` method in your migration.
+- Remove the application code related to the table, such as models,
+ controllers and services.
+- In a post-deployment migration, use `drop_table`.
+
+This can all be in a single migration if you're sure the code is not used.
+If you want to reduce risk slightly, consider putting the migrations into a
+second merge request after the application changes are merged. This approach
+provides an opportunity to roll back.
```ruby
def up
@@ -913,12 +927,16 @@ end
Table **has foreign keys**:
-- First release: Remove the application code related to the table, such as models,
-controllers, and services.
-- Second release: Remove the foreign keys using the `with_lock_retries`
-helper method. Use `drop_table` in another migration file.
+- Remove the application code related to the table, such as models,
+ controllers, and services.
+- In a post-deployment migration, remove the foreign keys using the
+ `with_lock_retries` helper method. In another subsequent post-deployment
+ migration, use `drop_table`.
-**Migrations for the second release:**
+This can all be in a single migration if you're sure the code is not used.
+If you want to reduce risk slightly, consider putting the migrations into a
+second merge request after the application changes are merged. This approach
+provides an opportunity to roll back.
Removing the foreign key on the `projects` table:
diff --git a/doc/development/multi_version_compatibility.md b/doc/development/multi_version_compatibility.md
index 808554d6027..8e2103b2f3e 100644
--- a/doc/development/multi_version_compatibility.md
+++ b/doc/development/multi_version_compatibility.md
@@ -258,7 +258,7 @@ For more information, see [the relevant issue](https://gitlab.com/gitlab-org/git
We bumped the Markdown cache version and found a bug when a user edited a description or comment which was generated from a different Markdown
cache version. The cached HTML wasn't generated properly after saving. In most cases, this wouldn't have happened because users would have
-viewed the Markdown before clicking **Edit** and that would mean the Markdown cache is refreshed. But because we run mixed versions, this is
+viewed the Markdown before selecting **Edit** and that would mean the Markdown cache is refreshed. But because we run mixed versions, this is
more likely to happen. Another user on a different version could view the same page and refresh the cache to the other version behind the scenes.
For more information, see [the relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/208255).
diff --git a/doc/development/packages/cleanup_policies.md b/doc/development/packages/cleanup_policies.md
new file mode 100644
index 00000000000..aa60cde450b
--- /dev/null
+++ b/doc/development/packages/cleanup_policies.md
@@ -0,0 +1,122 @@
+---
+stage: Package
+group: Container Registry
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Cleanup policies
+
+Cleanup policies are recurrent background processes that automatically remove
+objects according to some parameters set by users.
+
+## Container Registry
+
+Cleanup policies for the container registry work on all the container repositories
+hosted in a single project. All tags that match the cleanup parameters are removed.
+
+### Parameters
+
+The [ContainerExpirationPolicy](https://gitlab.com/gitlab-org/gitlab/-/blob/37a76cbfb54a9a3f0dba3c3748eaaac82fb8bf4b/app/models/container_expiration_policy.rb)
+holds all parameters for the container registry cleanup policies.
+
+The parameters are split into two groups:
+
+- The parameters that define tags to keep:
+ - `keep_n`. Keep the `n` most recent tags.
+ - `name_regex_keep`. Keep tags matching this regular expression.
+- The parameters that define tags to destroy:
+ - `older_than`. Destroy tags older than this timestamp.
+ - `name_regex`. Destroy tags matching this regular expression.
+
+The remaining parameters impact when the policy is executed:
+
+- `enabled`. Defines if the policy is enabled or not.
+- `cadence`. Defines the execution cadence of the policy.
+- `next_run_at`. Defines when the next execution should happen.
+
+### Execution
+
+Due to the large number of policies we need to process on GitLab.com, the execution
+follows this design.
+
+- Policy executions are limited in time.
+- Policy executions are either complete or partial.
+- The background jobs will consider the next job to be executed based on two
+priorities:
+ - Policy with a `next_run_at` in the past.
+ - Partially executed policies.
+
+To track the cleanup policy status on a container repository,
+we have an `expiration_policy_cleanup_status` on the `ContainerRepository`
+model.
+
+Background jobs for this execution are organized on:
+
+- A cron background job that runs every hour.
+- A set of background jobs that will loop on container repositories that need
+a policy execution.
+
+#### The cron background job
+
+The [cron background job](https://gitlab.com/gitlab-org/gitlab/-/blob/36454d77a8de76a25896efd7c051d6796985f579/app/workers/container_expiration_policy_worker.rb)
+is quite simple.
+Its main tasks are:
+
+1. Check if there are any container repositories in need of a cleanup. If any,
+enqueue as many limited capacity jobs as necessary, up to a limit.
+1. Compute metrics for cleanup policies and log them.
+
+#### The limited capacity job
+
+This [job](https://gitlab.com/gitlab-org/gitlab/-/blob/36454d77a8de76a25896efd7c051d6796985f579/app/workers/container_expiration_policies/cleanup_container_repository_worker.rb)
+is based on the [limited capacity concern](../sidekiq/limited_capacity_worker.md).
+
+This job will run in parallel up to [a specific capacity](settings.md#container-registry).
+
+The primary responsibility of this job is to select the next container
+repository that requires cleaning and call the related service on it.
+
+This is where the two priorities are evaluated in order. If a container repository
+is found, the cleanup service is called on it.
+
+To ensure that only one cleaning is executed on a given container repository
+at any time, we use a database lock along with the
+`expiration_policy_cleanup_status` column.
+
+This job will re-enqueue itself until no more container repositories require cleanup.
+
+#### Services
+
+Here is the services call that will happen from the limited capacity job:
+
+```mermaid
+flowchart TD
+ job[Limited capacity job] --> cleanup([ContainerExpirationPolicies::CleanupService])
+ cleanup --> cleanup_tags([Projects::ContainerRepository::CleanupTagsService])
+ cleanup_tags --> delete_tags([Projects::ContainerRepository::DeleteTagsService])
+```
+
+- [`ContainerExpirationPolicies::CleanupService`](https://gitlab.com/gitlab-org/gitlab/-/blob/6546ffc6fe4e9b447a1b7f050edddb8926fe4a3d/app/services/container_expiration_policies/cleanup_service.rb).
+This service mainly deals with container repository `expiration_policy_cleanup_status`
+updates and will call the cleanup tags service.
+- [`Projects::ContainerRepository::CleanupTagsService`](https://gitlab.com/gitlab-org/gitlab/-/blob/f23d70b7d638c38d71af102cfd32a3f6751596f9/app/services/projects/container_repository/cleanup_tags_service.rb).
+This service receives the policy parameters and builds the list of tags to
+destroy on the container registry.
+- [`Projects::ContainerRepository::DeleteTagsService`](https://gitlab.com/gitlab-org/gitlab/-/blob/f23d70b7d638c38d71af102cfd32a3f6751596f9/app/services/projects/container_repository/delete_tags_service.rb).
+This service receives a list of tags and loops on that list. For each tag,
+the service will call the container registry API endpoint to destroy the target tag.
+
+The cleanup tags service uses a very specific [execution order](../../user/packages/container_registry/reduce_container_registry_storage.md#how-the-cleanup-policy-works)
+to build the list of tags to destroy.
+
+Lastly, the cleanup tags service and delete tags service work using facades.
+The actual implementation depends on the type of container registry connected.
+If the GitLab container registry is connected, several improvements are available
+and used during cleanup policies execution, such as [better use of the container registry API](https://gitlab.com/groups/gitlab-org/-/epics/8379).
+
+### Historic reference links
+
+- [First iteration](https://gitlab.com/gitlab-org/gitlab/-/issues/15398)
+- [Throttling policy executions](https://gitlab.com/gitlab-org/gitlab/-/issues/208193)
+- [Adding caching](https://gitlab.com/gitlab-org/gitlab/-/issues/339129)
+- [Further improvements](https://gitlab.com/groups/gitlab-org/-/epics/8379)
diff --git a/doc/development/packages/debian_repository.md b/doc/development/packages/debian_repository.md
index 71f6c916652..26a33c548d8 100644
--- a/doc/development/packages/debian_repository.md
+++ b/doc/development/packages/debian_repository.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/development/packages/dependency_proxy.md b/doc/development/packages/dependency_proxy.md
index f3e323fb9fa..d5cc219cba0 100644
--- a/doc/development/packages/dependency_proxy.md
+++ b/doc/development/packages/dependency_proxy.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -130,7 +130,7 @@ graph TD
A[Receive manifest request] --> | We have the manifest cached.| B{Docker manifest HEAD request}
A --> | We do not have manifest cached.| C{Docker manifest GET request}
B --> | Digest matches the one in the DB | D[Fetch manifest from cache]
- B --> | Network failure, cannot reach DockerHub | D[Fetch manifest from cache]
+ B --> | HEAD request error, network failure, cannot reach DockerHub | D[Fetch manifest from cache]
B --> | Digest does not match the one in DB | C
C --> E[Save manifest to cache, save digest to database]
D --> F
diff --git a/doc/development/packages/index.md b/doc/development/packages/index.md
index 41d6e1b84fd..e6ec7e9654a 100644
--- a/doc/development/packages/index.md
+++ b/doc/development/packages/index.md
@@ -1,15 +1,18 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Package Registry Development
+# Package and container registry documentation
-Development and Architectural documentation for the package registry
+The documentation for package and container registry development is split into two groups.
+
+## Package registry development
+
+Development and architectural documentation for the package registry:
- [Debian repository structure](debian_repository.md)
-- [Dependency proxy structure](dependency_proxy.md)
- [Developing a new format](new_format_development.md)
- [Settings](settings.md)
- [Structure / Schema](structure.md)
@@ -25,3 +28,12 @@ Development and Architectural documentation for the package registry
- [NuGet](../../api/packages/nuget.md)
- [PyPI](../../api/packages/pypi.md)
- [Ruby Gems](../../api/packages/rubygems.md)
+
+## Container registry development
+
+Development and architectural documentation for the container registry
+
+- [Dependency proxy structure](dependency_proxy.md)
+- [Settings](settings.md)
+- [Structure / Schema](structure.md)
+- [Cleanup policies](cleanup_policies.md)
diff --git a/doc/development/packages/new_format_development.md b/doc/development/packages/new_format_development.md
index e9decea3094..f730d4f9476 100644
--- a/doc/development/packages/new_format_development.md
+++ b/doc/development/packages/new_format_development.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/development/packages/settings.md b/doc/development/packages/settings.md
index d591f708954..33caa064ab3 100644
--- a/doc/development/packages/settings.md
+++ b/doc/development/packages/settings.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/development/packages/structure.md b/doc/development/packages/structure.md
index e5426f8e2b8..281ec2e6c39 100644
--- a/doc/development/packages/structure.md
+++ b/doc/development/packages/structure.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/development/performance.md b/doc/development/performance.md
index 4f22d9ceb03..3881fad0528 100644
--- a/doc/development/performance.md
+++ b/doc/development/performance.md
@@ -21,8 +21,8 @@ consistent performance of GitLab. Refer to the [Index](#performance-documentatio
- [Query performance guidelines](database/query_performance.md)
- [Pagination performance guidelines](../development/database/pagination_performance_guidelines.md)
- [Keyset pagination performance](../development/database/keyset_pagination.md#performance)
- - [Troubleshooting import/export performance issues](../development/import_export.md#troubleshooting-performance-issues)
- - [Pipelines performance in the `gitlab` project](../development/pipelines.md#performance)
+ - [Troubleshooting import/export performance issues](../user/project/settings/import_export_troubleshooting.md#troubleshooting-performance-issues)
+ - [Pipelines performance in the `gitlab` project](pipelines/performance.md)
- Frontend:
- [Performance guidelines and monitoring](../development/fe_guide/performance.md)
- [Browser performance testing guidelines](../ci/testing/browser_performance_testing.md)
diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md
index 01e42fda2c9..a4e06e98d14 100644
--- a/doc/development/pipelines.md
+++ b/doc/development/pipelines.md
@@ -1,871 +1,11 @@
---
-stage: none
-group: Engineering Productivity
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: 'pipelines/index.md'
+remove_date: '2023-01-20'
---
-# Pipelines for the GitLab project
+This document was moved to [another location](pipelines/index.md).
-Pipelines for [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) (as well as the `dev` instance's) is configured in the usual
-[`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml)
-which itself includes files under
-[`.gitlab/ci/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/.gitlab/ci)
-for easier maintenance.
-
-We're striving to [dogfood](https://about.gitlab.com/handbook/engineering/development/principles/#dogfooding)
-GitLab [CI/CD features and best-practices](../ci/yaml/index.md)
-as much as possible.
-
-## Minimal test jobs before a merge request is approved
-
-**To reduce the pipeline cost and shorten the job duration, before a merge request is approved, the pipeline will run a minimal set of RSpec & Jest tests that are related to the merge request changes.**
-
-After a merge request has been approved, the pipeline would contain the full RSpec & Jest tests. This will ensure that all tests
-have been run before a merge request is merged.
-
-### Overview of the GitLab project test dependency
-
-To understand how the minimal test jobs are executed, we need to understand the dependency between
-GitLab code (frontend and backend) and the respective tests (Jest and RSpec).
-This dependency can be visualized in the following diagram:
-
-```mermaid
-flowchart LR
- subgraph frontend
- fe["Frontend code"]--tested with-->jest
- end
- subgraph backend
- be["Backend code"]--tested with-->rspec
- end
-
- be--generates-->fixtures["frontend fixtures"]
- fixtures--used in-->jest
-```
-
-In summary:
-
-- RSpec tests are dependent on the backend code.
-- Jest tests are dependent on both frontend and backend code, the latter through the frontend fixtures.
-
-### RSpec minimal jobs
-
-#### Determining related RSpec test files in a merge request
-
-To identify the minimal set of tests needed, we use the [`test_file_finder` gem](https://gitlab.com/gitlab-org/ci-cd/test_file_finder), with two strategies:
-
-- dynamic mapping from test coverage tracing (generated via the [`Crystalball` gem](https://github.com/toptal/crystalball))
- ([see where it's used](https://gitlab.com/gitlab-org/gitlab/-/blob/47d507c93779675d73a05002e2ec9c3c467cd698/tooling/bin/find_tests#L15))
-- static mapping maintained in the [`tests.yml` file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/tests.yml) for special cases that cannot
- be mapped via coverage tracing ([see where it's used](https://gitlab.com/gitlab-org/gitlab/-/blob/47d507c93779675d73a05002e2ec9c3c467cd698/tooling/bin/find_tests#L12))
-
-The test mappings contain a map of each source files to a list of test files which is dependent of the source file.
-
-In the `detect-tests` job, we use this mapping to identify the minimal tests needed for the current merge request.
-
-Later on in [the `rspec fail-fast` job](#fail-fast-job-in-merge-request-pipelines), we run the minimal tests needed for the current merge request.
-
-#### Exceptional cases
-
-In addition, there are a few circumstances where we would always run the full RSpec tests:
-
-- when the `pipeline:run-all-rspec` label is set on the merge request
-- when the `pipeline:run-full-rspec` label is set on the merge request, this label is assigned by triage automation when the merge request is approved by any reviewer
-- when the merge request is created by an automation (for example, Gitaly update or MR targeting a stable branch)
-- when the merge request is created in a security mirror
-- when any CI configuration file is changed (for example, `.gitlab-ci.yml` or `.gitlab/ci/**/*`)
-
-### Jest minimal jobs
-
-#### Determining related Jest test files in a merge request
-
-To identify the minimal set of tests needed, we pass a list of all the changed files into `jest` using the [`--findRelatedTests`](https://jestjs.io/docs/cli#--findrelatedtests-spaceseparatedlistofsourcefiles) option.
-In this mode, `jest` would resolve all the dependencies of related to the changed files, which include test files that have these files in the dependency chain.
-
-#### Exceptional cases
-
-In addition, there are a few circumstances where we would always run the full Jest tests:
-
-- when the `pipeline:run-all-jest` label is set on the merge request
-- when the merge request is created by an automation (for example, Gitaly update or MR targeting a stable branch)
-- when the merge request is created in a security mirror
-- when any CI configuration file is changed (for example, `.gitlab-ci.yml` or `.gitlab/ci/**/*`)
-- when any frontend "core" file is changed (for example, `package.json`, `yarn.lock`, `babel.config.js`, `jest.config.*.js`, `config/helpers/**/*.js`)
-- when any vendored JavaScript file is changed (for example, `vendor/assets/javascripts/**/*`)
-- when any backend file is changed ([see the patterns list for details](https://gitlab.com/gitlab-org/gitlab/-/blob/3616946936c1adbd9e754c1bd06f86ba670796d8/.gitlab/ci/rules.gitlab-ci.yml#L205-216))
-
-### Fork pipelines
-
-We run only the minimal RSpec & Jest jobs for fork pipelines, unless the `pipeline:run-all-rspec`
-label is set on the MR. The goal is to reduce the CI/CD minutes consumed by fork pipelines.
-
-See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1170).
-
-## Fail-fast job in merge request pipelines
-
-To provide faster feedback when a merge request breaks existing tests, we are experimenting with a
-fail-fast mechanism.
-
-An `rspec fail-fast` job is added in parallel to all other `rspec` jobs in a merge
-request pipeline. This job runs the tests that are directly related to the changes
-in the merge request.
-
-If any of these tests fail, the `rspec fail-fast` job fails, triggering a
-`fail-pipeline-early` job to run. The `fail-pipeline-early` job:
-
-- Cancels the currently running pipeline and all in-progress jobs.
-- Sets pipeline to have status `failed`.
-
-For example:
-
-```mermaid
-graph LR
- subgraph "prepare stage";
- A["detect-tests"]
- end
-
- subgraph "test stage";
- B["jest"];
- C["rspec migration"];
- D["rspec unit"];
- E["rspec integration"];
- F["rspec system"];
- G["rspec fail-fast"];
- end
-
- subgraph "post-test stage";
- Z["fail-pipeline-early"];
- end
-
- A --"artifact: list of test files"--> G
- G --"on failure"--> Z
-```
-
-The `rspec fail-fast` is a no-op if there are more than 10 test files related to the
-merge request. This prevents `rspec fail-fast` duration from exceeding the average
-`rspec` job duration and defeating its purpose.
-
-This number can be overridden by setting a CI/CD variable named `RSPEC_FAIL_FAST_TEST_FILE_COUNT_THRESHOLD`.
-
-## Faster feedback when reverting merge requests
-
-When you need to revert a merge request, to get accelerated feedback, you can add the `~pipeline:revert` label to your merge request.
-
-When this label is assigned, the following steps of the CI/CD pipeline are skipped:
-
-- The `e2e:package-and-test` job.
-- The `rspec:undercoverage` job.
-- The entire [Review Apps process](testing_guide/review_apps.md).
-
-Apply the label to the merge request, and run a new pipeline for the MR.
-
-## Test jobs
-
-We have dedicated jobs for each [testing level](testing_guide/testing_levels.md) and each job runs depending on the
-changes made in your merge request.
-If you want to force all the RSpec jobs to run regardless of your changes, you can add the `pipeline:run-all-rspec` label to the merge request.
-
-WARNING:
-Forcing all jobs on docs only related MRs would not have the prerequisite jobs and would lead to errors
-
-### Test suite parallelization
-
-Our current RSpec tests parallelization setup is as follows:
-
-1. The `retrieve-tests-metadata` job in the `prepare` stage ensures we have a
- `knapsack/report-master.json` file:
- - The `knapsack/report-master.json` file is fetched from the latest `main` pipeline which runs `update-tests-metadata`
- (for now it's the 2-hourly `maintenance` scheduled master pipeline), if it's not here we initialize the file with `{}`.
-1. Each `[rspec|rspec-ee] [migration|unit|integration|system|geo] n m` job are run with
- `knapsack rspec` and should have an evenly distributed share of tests:
- - It works because the jobs have access to the `knapsack/report-master.json`
- since the "artifacts from all previous stages are passed by default".
- - the jobs set their own report path to
- `"knapsack/${TEST_TOOL}_${TEST_LEVEL}_${DATABASE}_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json"`.
- - if knapsack is doing its job, test files that are run should be listed under
- `Report specs`, not under `Leftover specs`.
-1. The `update-tests-metadata` job (which only runs on scheduled pipelines for
- [the canonical project](https://gitlab.com/gitlab-org/gitlab) takes all the
- `knapsack/rspec*.json` files and merge them all together into a single
- `knapsack/report-master.json` file that is saved as artifact.
-
-After that, the next pipeline uses the up-to-date `knapsack/report-master.json` file.
-
-### Flaky tests
-
-#### Automatic skipping of flaky tests
-
-Tests that are [known to be flaky](testing_guide/flaky_tests.md#automatic-retries-and-flaky-tests-detection) are
-skipped unless the `$SKIP_FLAKY_TESTS_AUTOMATICALLY` variable is set to `false` or if the `~"pipeline:run-flaky-tests"`
-label is set on the MR.
-
-See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1069).
-
-#### Automatic retry of failing tests in a separate process
-
-Unless `$RETRY_FAILED_TESTS_IN_NEW_PROCESS` variable is set to `false` (`true` by default), RSpec tests that failed are automatically retried once in a separate
-RSpec process. The goal is to get rid of most side-effects from previous tests that may lead to a subsequent test failure.
-
-We keep track of retried tests in the `$RETRIED_TESTS_REPORT_FILE` file saved as artifact by the `rspec:flaky-tests-report` job.
-
-See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1148).
-
-### Single database testing
-
-By default, all tests run with [multiple databases](database/multiple_databases.md).
-
-We also run tests with a single database in nightly scheduled pipelines, and in merge requests that touch database-related files.
-
-If you want to force tests to run with a single database, you can add the `pipeline:run-single-db` label to the merge request.
-
-### Monitoring
-
-The GitLab test suite is [monitored](performance.md#rspec-profiling) for the `main` branch, and any branch
-that includes `rspec-profile` in their name.
-
-### Logging
-
-- Rails logging to `log/test.log` is disabled by default in CI
- [for performance reasons](https://jtway.co/speed-up-your-rails-test-suite-by-6-in-1-line-13fedb869ec4).
- To override this setting, provide the
- `RAILS_ENABLE_TEST_LOG` environment variable.
-
-## Review app jobs
-
-Consult the [Review Apps](testing_guide/review_apps.md) dedicated page for more information.
-
-If you want to force a Review App to be deployed regardless of your changes, you can add the `pipeline:run-review-app` label to the merge request.
-
-## As-if-FOSS jobs
-
-The `* as-if-foss` jobs run the GitLab test suite "as if FOSS", meaning as if the jobs would run in the context
-of `gitlab-org/gitlab-foss`. These jobs are only created in the following cases:
-
-- when the `pipeline:run-as-if-foss` label is set on the merge request
-- when the merge request is created in the `gitlab-org/security/gitlab` project
-- when any CI configuration file is changed (for example, `.gitlab-ci.yml` or `.gitlab/ci/**/*`)
-
-The `* as-if-foss` jobs are run in addition to the regular EE-context jobs. They have the `FOSS_ONLY='1'` variable
-set and get the `ee/` folder removed before the tests start running.
-
-The intent is to ensure that a change doesn't introduce a failure after `gitlab-org/gitlab` is synced to `gitlab-org/gitlab-foss`.
-
-## As-if-JH jobs
-
-NOTE:
-This is disabled for now.
-
-The `* as-if-jh` jobs run the GitLab test suite "as if JiHu", meaning as if the jobs would run in the context
-of [GitLab JH](jh_features_review.md). These jobs are only created in the following cases:
-
-- when the `pipeline:run-as-if-jh` label is set on the merge request
-- when the `pipeline:run-all-rspec` label is set on the merge request
-- when any code or backstage file is changed
-- when any startup CSS file is changed
-
-The `* as-if-jh` jobs are run in addition to the regular EE-context jobs. The `jh/` folder is added before the tests start running.
-
-The intent is to ensure that a change doesn't introduce a failure after `gitlab-org/gitlab` is synced to [GitLab JH](https://jihulab.com/gitlab-cn/gitlab).
-
-### When to consider applying `pipeline:run-as-if-jh` label
-
-NOTE:
-This is disabled for now.
-
-If a Ruby file is renamed and there's a corresponding [`prepend_mod` line](jh_features_review.md#jh-features-based-on-ce-or-ee-features),
-it's likely that GitLab JH is relying on it and requires a corresponding
-change to rename the module or class it's prepending.
-
-### Corresponding JH branch
-
-NOTE:
-This is disabled for now.
-
-You can create a corresponding JH branch on [GitLab JH](https://jihulab.com/gitlab-cn/gitlab) by
-appending `-jh` to the branch name. If a corresponding JH branch is found,
-`* as-if-jh` jobs grab the `jh` folder from the respective branch,
-rather than from the default branch `main-jh`.
-
-NOTE:
-For now, CI will try to fetch the branch on the [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab), so it might take some time for the new JH branch to propagate to the mirror.
-
-## Ruby 3.0 jobs
-
-You can add the `pipeline:run-in-ruby3` label to the merge request to switch
-the Ruby version used for running the whole test suite to 3.0. When you do
-this, the test suite will no longer run in Ruby 2.7 (default), and an
-additional job `verify-ruby-2.7` will also run and always fail to remind us to
-remove the label and run in Ruby 2.7 before merging the merge request.
-
-This should let us:
-
-- Test changes for Ruby 3.0
-- Make sure it will not break anything when it's merged into the default branch
-
-## `undercover` RSpec test
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74859) in GitLab 14.6.
-
-The `rspec:undercoverage` job runs [`undercover`](https://rubygems.org/gems/undercover)
-to detect, and fail if any changes introduced in the merge request has zero coverage.
-
-The `rspec:undercoverage` job obtains coverage data from the `rspec:coverage`
-job.
-
-In the event of an emergency, or false positive from this job, add the
-`pipeline:skip-undercoverage` label to the merge request to allow this job to
-fail.
-
-### Troubleshooting `rspec:undercoverage` failures
-
-The `rspec:undercoverage` job has [known bugs](https://gitlab.com/groups/gitlab-org/-/epics/8254)
-that can cause false positive failures. You can test coverage locally to determine if it's
-safe to apply `~"pipeline:skip-undercoverage"`. For example, using `<spec>` as the name of the
-test causing the failure:
-
-1. Run `SIMPLECOV=1 bundle exec rspec <spec>`.
-1. Run `scripts/undercoverage`.
-
-If these commands return `undercover: ✅ No coverage is missing in latest changes` then you can apply `~"pipeline:skip-undercoverage"` to bypass pipeline failures.
-
-## Ruby versions testing
-
-Our test suite runs against Ruby 2 in merge requests and default branch pipelines.
-
-We also run our test suite against Ruby 3 on another 2-hourly scheduled pipelines, as GitLab.com will soon run on Ruby 3.
-
-## PostgreSQL versions testing
-
-Our test suite runs against PG12 as GitLab.com runs on PG12 and
-[Omnibus defaults to PG12 for new installs and upgrades](../administration/package_information/postgresql_versions.md).
-
-We do run our test suite against PG11 and PG13 on nightly scheduled pipelines.
-
-We also run our test suite against PG11 upon specific database library changes in MRs and `main` pipelines (with the `rspec db-library-code pg11` job).
-
-### Current versions testing
-
-| Where? | PostgreSQL version | Ruby version |
-|------------------------------------------------------------------------------------------------|-------------------------------------------------|--------------|
-| Merge requests | 12 (default version), 11 for DB library changes | 2.7 (default version) |
-| `master` branch commits | 12 (default version), 11 for DB library changes | 2.7 (default version) |
-| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour) | 12 (default version), 11 for DB library changes | 2.7 (default version) |
-| `maintenance` scheduled pipelines for the `ruby3` branch (every odd-numbered hour), see below. | 12 (default version), 11 for DB library changes | 3.0 (coded in the branch) |
-| `nightly` scheduled pipelines for the `master` branch | 12 (default version), 11, 13 | 2.7 (default version) |
-
-The pipeline configuration for the scheduled pipeline testing Ruby 3 is
-stored in the `ruby3-sync` branch. The pipeline updates the `ruby3` branch
-with latest `master`, and then it triggers a regular branch pipeline for
-`ruby3`. Any changes in `ruby3` are only for running the pipeline. It should
-never be merged back to `master`. Any other Ruby 3 changes should go into
-`master` directly, which should be compatible with Ruby 2.7.
-
-Previously, `ruby3-sync` was using a project token stored in `RUBY3_SYNC_TOKEN`
-(now backed up in `RUBY3_SYNC_TOKEN_NOT_USED`), however due to various
-permissions issues, we ended up using an access token from `gitlab-bot` so now
-`RUBY3_SYNC_TOKEN` is actually an access token from `gitlab-bot`.
-
-### Long-term plan
-
-We follow the [PostgreSQL versions shipped with Omnibus GitLab](../administration/package_information/postgresql_versions.md):
-
-| PostgreSQL version | 14.1 (July 2021) | 14.2 (August 2021) | 14.3 (September 2021) | 14.4 (October 2021) | 14.5 (November 2021) | 14.6 (December 2021) |
-| -------------------| ---------------------- | ---------------------- | ---------------------- | ---------------------- | ---------------------- | ---------------------- |
-| PG12 | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` |
-| PG11 | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` |
-| PG13 | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` |
-
-## Redis versions testing
-
-Our test suite runs against Redis 6 as GitLab.com runs on Redis 6 and
-[Omnibus defaults to Redis 6 for new installs and upgrades](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/config/software/redis.rb).
-
-We do run our test suite against Redis 5 on `nightly` scheduled pipelines, specifically when running backward-compatible and forward-compatible PostgreSQL jobs.
-
-### Current versions testing
-
-| Where? | Redis version |
-| ------ | ------------------ |
-| MRs | 6 |
-| `default branch` (non-scheduled pipelines) | 6 |
-| `nightly` scheduled pipelines | 5 |
-
-## Pipelines types for merge requests
-
-In general, pipelines for an MR fall into one of the following types (from shorter to longer), depending on the changes made in the MR:
-
-- [Documentation pipeline](#documentation-pipeline): For MRs that touch documentation.
-- [Backend pipeline](#backend-pipeline): For MRs that touch backend code.
-- [Frontend pipeline](#frontend-pipeline): For MRs that touch frontend code.
-- [End-to-end pipeline](#end-to-end-pipeline): For MRs that touch code in the `qa/` folder.
-
-A "pipeline type" is an abstract term that mostly describes the "critical path" (for example, the chain of jobs for which the sum
-of individual duration equals the pipeline's duration).
-We use these "pipeline types" in [metrics dashboards](https://app.periscopedata.com/app/gitlab/858266/GitLab-Pipeline-Durations) to detect what types and jobs need to be optimized first.
-
-An MR that touches multiple areas would be associated with the longest type applicable. For instance, an MR that touches backend
-and frontend would fall into the "Frontend" pipeline type since this type takes longer to finish than the "Backend" pipeline type.
-
-We use the [`rules:`](../ci/yaml/index.md#rules) and [`needs:`](../ci/yaml/index.md#needs) keywords extensively
-to determine the jobs that need to be run in a pipeline. Note that an MR that includes multiple types of changes would
-have a pipelines that include jobs from multiple types (for example, a combination of docs-only and code-only pipelines).
-
-Following are graphs of the critical paths for each pipeline type. Jobs that aren't part of the critical path are omitted.
-
-### Documentation pipeline
-
-[Reference pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipelines/432049110).
-
-```mermaid
-graph LR
- classDef criticalPath fill:#f66;
-
- 1-3["docs-lint links (5 minutes)"];
- class 1-3 criticalPath;
- click 1-3 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356757&udv=0"
-```
-
-### Backend pipeline
-
-[Reference pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipelines/433316063).
-
-```mermaid
-graph RL;
- classDef criticalPath fill:#f66;
-
- 1-3["compile-test-assets (6 minutes)"];
- class 1-3 criticalPath;
- click 1-3 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914317&udv=0"
- 1-6["setup-test-env (4 minutes)"];
- click 1-6 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914315&udv=0"
- 1-14["retrieve-tests-metadata"];
- click 1-14 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356697&udv=0"
- 1-15["detect-tests"];
- click 1-15 "https://app.periscopedata.com/app/gitlab/652085/EP---Jobs-Durations?widget=10113603&udv=1005715"
-
- 2_5-1["rspec & db jobs (24 minutes)"];
- class 2_5-1 criticalPath;
- click 2_5-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations"
- 2_5-1 --> 1-3 & 1-6 & 1-14 & 1-15;
-
- 3_2-1["rspec:coverage (5.35 minutes)"];
- class 3_2-1 criticalPath;
- click 3_2-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=7248745&udv=0"
- 3_2-1 -.->|"(don't use needs<br/>because of limitations)"| 2_5-1;
-
- 4_3-1["rspec:undercoverage (3.5 minutes)"];
- class 4_3-1 criticalPath;
- click 4_3-1 "https://app.periscopedata.com/app/gitlab/652085/EP---Jobs-Durations?widget=13446492&udv=1005715"
- 4_3-1 --> 3_2-1;
-
-```
-
-### Frontend pipeline
-
-[Reference pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipelines/431913287).
-
-```mermaid
-graph RL;
- classDef criticalPath fill:#f66;
-
- 1-2["build-qa-image (2 minutes)"];
- click 1-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914325&udv=0"
- 1-5["compile-production-assets (16 minutes)"];
- class 1-5 criticalPath;
- click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0"
-
- 2_3-1["build-assets-image (1.3 minutes)"];
- class 2_3-1 criticalPath;
- 2_3-1 --> 1-5
-
- 2_6-1["start-review-app-pipeline (49 minutes)"];
- class 2_6-1 criticalPath;
- click 2_6-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations"
- 2_6-1 --> 2_3-1 & 1-2;
-```
-
-### End-to-end pipeline
-
-[Reference pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipelines/431918463).
-
-```mermaid
-graph RL;
- classDef criticalPath fill:#f66;
-
- 1-2["build-qa-image (2 minutes)"];
- click 1-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914325&udv=0"
- 1-5["compile-production-assets (16 minutes)"];
- class 1-5 criticalPath;
- click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0"
- 1-15["detect-tests"];
- click 1-15 "https://app.periscopedata.com/app/gitlab/652085/EP---Jobs-Durations?widget=10113603&udv=1005715"
-
- 2_3-1["build-assets-image (1.3 minutes)"];
- class 2_3-1 criticalPath;
- 2_3-1 --> 1-5
-
- 2_4-1["e2e:package-and-test (102 minutes)"];
- class 2_4-1 criticalPath;
- click 2_4-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914305&udv=0"
- 2_4-1 --> 1-2 & 2_3-1 & 1-15;
-```
-
-## CI configuration internals
-
-### Workflow rules
-
-Pipelines for the GitLab project are created using the [`workflow:rules` keyword](../ci/yaml/index.md#workflow)
-feature of the GitLab CI/CD.
-
-Pipelines are always created for the following scenarios:
-
-- `main` branch, including on schedules, pushes, merges, and so on.
-- Merge requests.
-- Tags.
-- Stable, `auto-deploy`, and security branches.
-
-Pipeline creation is also affected by the following CI/CD variables:
-
-- If `$FORCE_GITLAB_CI` is set, pipelines are created.
-- If `$GITLAB_INTERNAL` is not set, pipelines are not created.
-
-No pipeline is created in any other cases (for example, when pushing a branch with no
-MR for it).
-
-The source of truth for these workflow rules is defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml).
-
-### Default image
-
-The default image is defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml).
-
-<!-- vale gitlab.Spelling = NO -->
-
-It includes Ruby, Go, Git, Git LFS, Chrome, Node, Yarn, PostgreSQL, and Graphics Magick.
-
-<!-- vale gitlab.Spelling = YES -->
-
-The images used in our pipelines are configured in the
-[`gitlab-org/gitlab-build-images`](https://gitlab.com/gitlab-org/gitlab-build-images)
-project, which is push-mirrored to [`gitlab/gitlab-build-images`](https://dev.gitlab.org/gitlab/gitlab-build-images)
-for redundancy.
-
-The current version of the build images can be found in the
-["Used by GitLab section"](https://gitlab.com/gitlab-org/gitlab-build-images/blob/master/.gitlab-ci.yml).
-
-### Default variables
-
-In addition to the [predefined CI/CD variables](../ci/variables/predefined_variables.md),
-each pipeline includes default variables defined in
-[`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml).
-
-### Stages
-
-The current stages are:
-
-- `sync`: This stage is used to synchronize changes from [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) to
- [`gitlab-org/gitlab-foss`](https://gitlab.com/gitlab-org/gitlab-foss).
-- `prepare`: This stage includes jobs that prepare artifacts that are needed by
- jobs in subsequent stages.
-- `build-images`: This stage includes jobs that prepare Docker images
- that are needed by jobs in subsequent stages or downstream pipelines.
-- `fixtures`: This stage includes jobs that prepare fixtures needed by frontend tests.
-- `lint`: This stage includes linting and static analysis jobs.
-- `test`: This stage includes most of the tests, and DB/migration jobs.
-- `post-test`: This stage includes jobs that build reports or gather data from
- the `test` stage's jobs (for example, coverage, Knapsack metadata, and so on).
-- `review`: This stage includes jobs that build the CNG images, deploy them, and
- run end-to-end tests against Review Apps (see [Review Apps](testing_guide/review_apps.md) for details).
- It also includes Docs Review App jobs.
-- `qa`: This stage includes jobs that perform QA tasks against the Review App
- that is deployed in stage `review`.
-- `post-qa`: This stage includes jobs that build reports or gather data from
- the `qa` stage's jobs (for example, Review App performance report).
-- `pages`: This stage includes a job that deploys the various reports as
- GitLab Pages (for example, [`coverage-ruby`](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/),
- and `webpack-report` (found at `https://gitlab-org.gitlab.io/gitlab/webpack-report/`, but there is
- [an issue with the deployment](https://gitlab.com/gitlab-org/gitlab/-/issues/233458)).
-- `notify`: This stage includes jobs that notify various failures to Slack.
-
-### Dependency Proxy
-
-Some of the jobs are using images from Docker Hub, where we also use
-`${GITLAB_DEPENDENCY_PROXY_ADDRESS}` as a prefix to the image path, so that we pull
-images from our [Dependency Proxy](../user/packages/dependency_proxy/index.md).
-By default, this variable is set from the value of `${GITLAB_DEPENDENCY_PROXY}`.
-
-`${GITLAB_DEPENDENCY_PROXY}` is a group CI/CD variable defined in
-[`gitlab-org`](https://gitlab.com/gitlab-org) as
-`${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/`. This means when we use an image
-defined as:
-
-```yaml
-image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}alpine:edge
-```
-
-Projects in the `gitlab-org` group pull from the Dependency Proxy, while
-forks that reside on any other personal namespaces or groups fall back to
-Docker Hub unless `${GITLAB_DEPENDENCY_PROXY}` is also defined there.
-
-#### Work around for when a pipeline is started by a Project access token user
-
-When a pipeline is started by a Project access token user (e.g. the `release-tools approver bot` user which
-automatically updates the Gitaly version used in the main project),
-[the Dependency proxy isn't accessible](https://gitlab.com/gitlab-org/gitlab/-/issues/332411#note_1130388163)
-and the job fails at the `Preparing the "docker+machine" executor` step.
-To work around that, we have a special workflow rule, that overrides the
-`${GITLAB_DEPENDENCY_PROXY_ADDRESS}` variable so that Depdendency proxy isn't used in that case:
-
-```yaml
-- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $GITLAB_USER_LOGIN =~ /project_\d+_bot\d*/'
- variables:
- GITLAB_DEPENDENCY_PROXY_ADDRESS: ""
-```
-
-NOTE:
-We don't directly override the `${GITLAB_DEPENDENCY_PROXY}` variable because group-level
-variables have higher precedence over `.gitlab-ci.yml` variables.
-
-### Common job definitions
-
-Most of the jobs [extend from a few CI definitions](../ci/yaml/index.md#extends)
-defined in [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml)
-that are scoped to a single [configuration keyword](../ci/yaml/index.md#job-keywords).
-
-| Job definitions | Description |
-|------------------|-------------|
-| `.default-retry` | Allows a job to [retry](../ci/yaml/index.md#retry) upon `unknown_failure`, `api_failure`, `runner_system_failure`, `job_execution_timeout`, or `stuck_or_timeout_failure`. |
-| `.default-before_script` | Allows a job to use a default `before_script` definition suitable for Ruby/Rails tasks that may need a database running (for example, tests). |
-| `.setup-test-env-cache` | Allows a job to use a default `cache` definition suitable for setting up test environment for subsequent Ruby/Rails tasks. |
-| `.rails-cache` | Allows a job to use a default `cache` definition suitable for Ruby/Rails tasks. |
-| `.static-analysis-cache` | Allows a job to use a default `cache` definition suitable for static analysis tasks. |
-| `.coverage-cache` | Allows a job to use a default `cache` definition suitable for coverage tasks. |
-| `.qa-cache` | Allows a job to use a default `cache` definition suitable for QA tasks. |
-| `.yarn-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that do a `yarn install`. |
-| `.assets-compile-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that compile assets. |
-| `.use-pg11` | Allows a job to run the `postgres` 11 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
-| `.use-pg11-ee` | Same as `.use-pg11` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). |
-| `.use-pg12` | Allows a job to use the `postgres` 12 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
-| `.use-pg12-ee` | Same as `.use-pg12` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). |
-| `.use-pg13` | Allows a job to use the `postgres` 13 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
-| `.use-pg13-ee` | Same as `.use-pg13` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). |
-| `.use-kaniko` | Allows a job to use the `kaniko` tool to build Docker images. |
-| `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` CI/CD variable. |
-| `.use-docker-in-docker` | Allows a job to use Docker in Docker. |
-
-### `rules`, `if:` conditions and `changes:` patterns
-
-We're using the [`rules` keyword](../ci/yaml/index.md#rules) extensively.
-
-All `rules` definitions are defined in
-[`rules.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml),
-then included in individual jobs via [`extends`](../ci/yaml/index.md#extends).
-
-The `rules` definitions are composed of `if:` conditions and `changes:` patterns,
-which are also defined in
-[`rules.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml)
-and included in `rules` definitions via [YAML anchors](../ci/yaml/yaml_optimization.md#anchors)
-
-#### `if:` conditions
-
-<!-- vale gitlab.Substitutions = NO -->
-
-| `if:` conditions | Description | Notes |
-|------------------|-------------|-------|
-| `if-not-canonical-namespace` | Matches if the project isn't in the canonical (`gitlab-org/`) or security (`gitlab-org/security`) namespace. | Use to create a job for forks (by using `when: on_success|manual`), or **not** create a job for forks (by using `when: never`). |
-| `if-not-ee` | Matches if the project isn't EE (that is, project name isn't `gitlab` or `gitlab-ee`). | Use to create a job only in the FOSS project (by using `when: on_success|manual`), or **not** create a job if the project is EE (by using `when: never`). |
-| `if-not-foss` | Matches if the project isn't FOSS (that is, project name isn't `gitlab-foss`, `gitlab-ce`, or `gitlabhq`). | Use to create a job only in the EE project (by using `when: on_success|manual`), or **not** create a job if the project is FOSS (by using `when: never`). |
-| `if-default-refs` | Matches if the pipeline is for `master`, `main`, `/^[\d-]+-stable(-ee)?$/` (stable branches), `/^\d+-\d+-auto-deploy-\d+$/` (auto-deploy branches), `/^security\//` (security branches), merge requests, and tags. | Note that jobs aren't created for branches with this default configuration. |
-| `if-master-refs` | Matches if the current branch is `master` or `main`. | |
-| `if-master-push` | Matches if the current branch is `master` or `main` and pipeline source is `push`. | |
-| `if-master-schedule-maintenance` | Matches if the current branch is `master` or `main` and pipeline runs on a 2-hourly schedule. | |
-| `if-master-schedule-nightly` | Matches if the current branch is `master` or `main` and pipeline runs on a nightly schedule. | |
-| `if-auto-deploy-branches` | Matches if the current branch is an auto-deploy one. | |
-| `if-master-or-tag` | Matches if the pipeline is for the `master` or `main` branch or for a tag. | |
-| `if-merge-request` | Matches if the pipeline is for a merge request. | |
-| `if-merge-request-title-as-if-foss` | Matches if the pipeline is for a merge request and the MR has label ~"pipeline:run-as-if-foss" | |
-| `if-merge-request-title-update-caches` | Matches if the pipeline is for a merge request and the MR has label ~"pipeline:update-cache". | |
-| `if-merge-request-title-run-all-rspec` | Matches if the pipeline is for a merge request and the MR has label ~"pipeline:run-all-rspec". | |
-| `if-security-merge-request` | Matches if the pipeline is for a security merge request. | |
-| `if-security-schedule` | Matches if the pipeline is for a security scheduled pipeline. | |
-| `if-nightly-master-schedule` | Matches if the pipeline is for a `master` scheduled pipeline with `$NIGHTLY` set. | |
-| `if-dot-com-gitlab-org-schedule` | Limits jobs creation to scheduled pipelines for the `gitlab-org` group on GitLab.com. | |
-| `if-dot-com-gitlab-org-master` | Limits jobs creation to the `master` or `main` branch for the `gitlab-org` group on GitLab.com. | |
-| `if-dot-com-gitlab-org-merge-request` | Limits jobs creation to merge requests for the `gitlab-org` group on GitLab.com. | |
-| `if-dot-com-gitlab-org-and-security-tag` | Limits job creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | |
-| `if-dot-com-gitlab-org-and-security-merge-request` | Limit jobs creation to merge requests for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | |
-| `if-dot-com-gitlab-org-and-security-tag` | Limit jobs creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | |
-| `if-dot-com-ee-schedule` | Limits jobs to scheduled pipelines for the `gitlab-org/gitlab` project on GitLab.com. | |
-| `if-security-pipeline-merge-result` | Matches if the pipeline is for a security merge request triggered by `@gitlab-release-tools-bot`. | |
-
-<!-- vale gitlab.Substitutions = YES -->
-
-#### `changes:` patterns
-
-| `changes:` patterns | Description |
-|------------------------------|--------------------------------------------------------------------------|
-| `ci-patterns` | Only create job for CI configuration-related changes. |
-| `ci-build-images-patterns` | Only create job for CI configuration-related changes related to the `build-images` stage. |
-| `ci-review-patterns` | Only create job for CI configuration-related changes related to the `review` stage. |
-| `ci-qa-patterns` | Only create job for CI configuration-related changes related to the `qa` stage. |
-| `yaml-lint-patterns` | Only create job for YAML-related changes. |
-| `docs-patterns` | Only create job for docs-related changes. |
-| `frontend-dependency-patterns` | Only create job when frontend dependencies are updated (that is, `package.json`, and `yarn.lock`). changes. |
-| `frontend-patterns` | Only create job for frontend-related changes. |
-| `backend-patterns` | Only create job for backend-related changes. |
-| `db-patterns` | Only create job for DB-related changes. |
-| `backstage-patterns` | Only create job for backstage-related changes (that is, Danger, fixtures, RuboCop, specs). |
-| `code-patterns` | Only create job for code-related changes. |
-| `qa-patterns` | Only create job for QA-related changes. |
-| `code-backstage-patterns` | Combination of `code-patterns` and `backstage-patterns`. |
-| `code-qa-patterns` | Combination of `code-patterns` and `qa-patterns`. |
-| `code-backstage-qa-patterns` | Combination of `code-patterns`, `backstage-patterns`, and `qa-patterns`. |
-| `static-analysis-patterns` | Only create jobs for Static Analytics configuration-related changes. |
-
-## Performance
-
-### Interruptible pipelines
-
-By default, all jobs are [interruptible](../ci/yaml/index.md#interruptible), except the
-`dont-interrupt-me` job which runs automatically on `main`, and is `manual`
-otherwise.
-
-If you want a running pipeline to finish even if you push new commits to a merge
-request, be sure to start the `dont-interrupt-me` job before pushing.
-
-### Git fetch caching
-
-Because GitLab.com uses the [pack-objects cache](../administration/gitaly/configure_gitaly.md#pack-objects-cache),
-concurrent Git fetches of the same pipeline ref are deduplicated on
-the Gitaly server (always) and served from cache (when available).
-
-This works well for the following reasons:
-
-- The pack-objects cache is enabled on all Gitaly servers on GitLab.com.
-- The CI/CD [Git strategy setting](../ci/pipelines/settings.md#choose-the-default-git-strategy) for `gitlab-org/gitlab` is **Git clone**,
- causing all jobs to fetch the same data, which maximizes the cache hit ratio.
-- We use [shallow clone](../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone) to avoid downloading the full Git
- history for every job.
-
-### Caching strategy
-
-1. All jobs must only pull caches by default.
-1. All jobs must be able to pass with an empty cache. In other words, caches are only there to speed up jobs.
-1. We currently have several different cache definitions defined in
- [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml),
- with fixed keys:
- - `.setup-test-env-cache`
- - `.ruby-cache`
- - `.rails-cache`
- - `.static-analysis-cache`
- - `.rubocop-cache`
- - `.coverage-cache`
- - `.ruby-node-cache`
- - `.qa-cache`
- - `.yarn-cache`
- - `.assets-compile-cache` (the key includes `${NODE_ENV}` so it's actually two different caches).
-1. These cache definitions are composed of [multiple atomic caches](../ci/caching/index.md#use-multiple-caches).
-1. Only the following jobs, running in 2-hourly `maintenance` scheduled pipelines, are pushing (that is, updating) to the caches:
- - `update-setup-test-env-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rails.gitlab-ci.yml).
- - `update-gitaly-binaries-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rails.gitlab-ci.yml).
- - `update-rubocop-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rails.gitlab-ci.yml).
- - `update-qa-cache`, defined in [`.gitlab/ci/qa.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/qa.gitlab-ci.yml).
- - `update-assets-compile-production-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml).
- - `update-assets-compile-test-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml).
- - `update-yarn-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml).
- - `update-storybook-yarn-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml).
-1. These jobs can also be forced to run in merge requests with the `pipeline:update-cache` label (this can be useful to warm the caches in a MR that updates the cache keys).
-
-### Artifacts strategy
-
-We limit the artifacts that are saved and retrieved by jobs to the minimum to reduce the upload/download time and costs, as well as the artifacts storage.
-
-### Components caching
-
-Some external components (GitLab Workhorse and frontend assets) of GitLab need to be built from source as a preliminary step for running tests.
-
-#### `cache-workhorse`
-
-In [this MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79766), and then
-[this MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96297),
-we introduced a new `cache-workhorse` job that:
-
-- runs automatically for all GitLab.com `gitlab-org/gitlab` scheduled pipelines
-- runs automatically for any `master` commit that touches the `workhorse/` folder
-- is manual for GitLab.com's `gitlab-org`'s MRs that touches caching-related files
-
-This job tries to download a generic package that contains GitLab Workhorse binaries needed in the GitLab test suite (under `tmp/tests/gitlab-workhorse`).
-
-- If the package URL returns a 404:
- 1. It runs `scripts/setup-test-env`, so that the GitLab Workhorse binaries are built.
- 1. It then creates an archive which contains the binaries and upload it [as a generic package](https://gitlab.com/gitlab-org/gitlab/-/packages/).
-- Otherwise, if the package already exists, it exits the job successfully.
-
-We also changed the `setup-test-env` job to:
-
-1. First download the GitLab Workhorse generic package build and uploaded by `cache-workhorse`.
-1. If the package is retrieved successfully, its content is placed in the right folder (for example, `tmp/tests/gitlab-workhorse`), preventing the building of the binaries when `scripts/setup-test-env` is run later on.
-1. If the package URL returns a 404, the behavior doesn't change compared to the current one: the GitLab Workhorse binaries are built as part of `scripts/setup-test-env`.
-
-NOTE:
-The version of the package is the workhorse tree SHA (for example, `git rev-parse HEAD:workhorse`).
-
-#### `cache-assets`
-
-In [this MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96297),
-we introduced three new `cache-assets:test`, `cache-assets:test as-if-foss`,
-and `cache-assets:production` jobs that:
-
-- never run unless `$CACHE_ASSETS_AS_PACKAGE == "true"`
-- runs automatically for all GitLab.com `gitlab-org/gitlab` scheduled pipelines
-- runs automatically for any `master` commit that touches the assets-related folders
-- is manual for GitLab.com's `gitlab-org`'s MRs that touches caching-related files
-
-This job tries to download a generic package that contains GitLab compiled assets
-needed in the GitLab test suite (under `app/assets/javascripts/locale/**/app.js`,
-and `public/assets`).
-
-- If the package URL returns a 404:
- 1. It runs `bin/rake gitlab:assets:compile`, so that the GitLab assets are compiled.
- 1. It then creates an archive which contains the assets and uploads it [as a generic package](https://gitlab.com/gitlab-org/gitlab/-/packages/).
- The package version is set to the assets folders' hash sum.
-- Otherwise, if the package already exists, it exits the job successfully.
-
-#### `compile-*-assets`
-
-We also changed the `compile-test-assets`, `compile-test-assets as-if-foss`,
-and `compile-production-assets` jobs to:
-
-1. First download the "native" cache assets, which contain:
- - The [compiled assets](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/global.gitlab-ci.yml#L86-87).
- - A [`cached-assets-hash.txt` file](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/global.gitlab-ci.yml#L85)
- containing the `SHA256` hexdigest of all the source files on which the assets depend on.
- This list of files is a pessimistic list and the assets might not depend on
- some of these files. At worst we compile the assets more often, which is better than
- using outdated assets.
-
- The file is [created after assets are compiled](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/frontend.gitlab-ci.yml#L83).
-1. We then we compute the `SHA256` hexdigest of all the source files the assets depend on, **for the current checked out branch**. We [store the hexdigest in the `GITLAB_ASSETS_HASH` variable](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/frontend.gitlab-ci.yml#L27).
-1. If `$CACHE_ASSETS_AS_PACKAGE == "true"`, we download the generic package built and uploaded by [`cache-assets:*`](#cache-assets).
- - If the cache is up-to-date for the checked out branch, we download the native cache
- **and** the cache package. We could optimize that by not downloading
- the genetic package but the native cache is actually very often outdated because it's
- rebuilt only every 2 hours.
-1. We [run the `assets_compile_script` function](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/frontend.gitlab-ci.yml#L35),
- which [itself runs](https://gitlab.com/gitlab-org/gitlab/-/blob/c023191ef412e868ae957f3341208a41ca678403/scripts/utils.sh#L76)
- the [`assets:compile` Rake task](https://gitlab.com/gitlab-org/gitlab/-/blob/c023191ef412e868ae957f3341208a41ca678403/lib/tasks/gitlab/assets.rake#L80-109).
-
- This task is responsible for deciding if assets need to be compiled or not.
- It [compares the `HEAD` `SHA256` hexdigest from `$GITLAB_ASSETS_HASH` with the `master` hexdigest from `cached-assets-hash.txt`](https://gitlab.com/gitlab-org/gitlab/-/blob/c023191ef412e868ae957f3341208a41ca678403/lib/tasks/gitlab/assets.rake#L86).
-1. If the hashes are the same, we don't compile anything. If they're different, we compile the assets.
-
-### Pre-clone step
-
-NOTE:
-We no longer use this optimization for `gitlab-org/gitlab` because the [pack-objects cache](../administration/gitaly/configure_gitaly.md#pack-objects-cache)
-allows Gitaly to serve the full CI/CD fetch traffic now. See [Git fetch caching](#git-fetch-caching).
-
-The pre-clone step works by using the `CI_PRE_CLONE_SCRIPT` variable
-[defined by GitLab.com shared runners](../ci/runners/saas/linux_saas_runner.md#pre-clone-script).
-
----
-
-[Return to Development documentation](index.md)
+<!-- This redirect file can be deleted after <2023-01-20>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/development/pipelines/index.md b/doc/development/pipelines/index.md
new file mode 100644
index 00000000000..01bb813e794
--- /dev/null
+++ b/doc/development/pipelines/index.md
@@ -0,0 +1,631 @@
+---
+stage: none
+group: Engineering Productivity
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Pipelines for the GitLab project
+
+Pipelines for [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) (as well as the `dev` instance's) is configured in the usual
+[`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml)
+which itself includes files under
+[`.gitlab/ci/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/.gitlab/ci)
+for easier maintenance.
+
+We're striving to [dogfood](https://about.gitlab.com/handbook/engineering/development/principles/#dogfooding)
+GitLab [CI/CD features and best-practices](../../ci/yaml/index.md)
+as much as possible.
+
+## Minimal test jobs before a merge request is approved
+
+**To reduce the pipeline cost and shorten the job duration, before a merge request is approved, the pipeline will run a minimal set of RSpec & Jest tests that are related to the merge request changes.**
+
+After a merge request has been approved, the pipeline would contain the full RSpec & Jest tests. This will ensure that all tests
+have been run before a merge request is merged.
+
+### Overview of the GitLab project test dependency
+
+To understand how the minimal test jobs are executed, we need to understand the dependency between
+GitLab code (frontend and backend) and the respective tests (Jest and RSpec).
+This dependency can be visualized in the following diagram:
+
+```mermaid
+flowchart LR
+ subgraph frontend
+ fe["Frontend code"]--tested with-->jest
+ end
+ subgraph backend
+ be["Backend code"]--tested with-->rspec
+ end
+
+ be--generates-->fixtures["frontend fixtures"]
+ fixtures--used in-->jest
+```
+
+In summary:
+
+- RSpec tests are dependent on the backend code.
+- Jest tests are dependent on both frontend and backend code, the latter through the frontend fixtures.
+
+### RSpec minimal jobs
+
+#### Determining related RSpec test files in a merge request
+
+To identify the minimal set of tests needed, we use the [`test_file_finder` gem](https://gitlab.com/gitlab-org/ci-cd/test_file_finder), with two strategies:
+
+- dynamic mapping from test coverage tracing (generated via the [`Crystalball` gem](https://github.com/toptal/crystalball))
+ ([see where it's used](https://gitlab.com/gitlab-org/gitlab/-/blob/47d507c93779675d73a05002e2ec9c3c467cd698/tooling/bin/find_tests#L15))
+- static mapping maintained in the [`tests.yml` file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/tests.yml) for special cases that cannot
+ be mapped via coverage tracing ([see where it's used](https://gitlab.com/gitlab-org/gitlab/-/blob/47d507c93779675d73a05002e2ec9c3c467cd698/tooling/bin/find_tests#L12))
+
+The test mappings contain a map of each source files to a list of test files which is dependent of the source file.
+
+In the `detect-tests` job, we use this mapping to identify the minimal tests needed for the current merge request.
+
+Later on in [the `rspec fail-fast` job](#fail-fast-job-in-merge-request-pipelines), we run the minimal tests needed for the current merge request.
+
+#### Exceptional cases
+
+In addition, there are a few circumstances where we would always run the full RSpec tests:
+
+- when the `pipeline:run-all-rspec` label is set on the merge request
+- when the `pipeline:run-full-rspec` label is set on the merge request, this label is assigned by triage automation when the merge request is approved by any reviewer
+- when the merge request is created by an automation (for example, Gitaly update or MR targeting a stable branch)
+- when the merge request is created in a security mirror
+- when any CI configuration file is changed (for example, `.gitlab-ci.yml` or `.gitlab/ci/**/*`)
+
+### Jest minimal jobs
+
+#### Determining related Jest test files in a merge request
+
+To identify the minimal set of tests needed, we pass a list of all the changed files into `jest` using the [`--findRelatedTests`](https://jestjs.io/docs/cli#--findrelatedtests-spaceseparatedlistofsourcefiles) option.
+In this mode, `jest` would resolve all the dependencies of related to the changed files, which include test files that have these files in the dependency chain.
+
+#### Exceptional cases
+
+In addition, there are a few circumstances where we would always run the full Jest tests:
+
+- when the `pipeline:run-all-jest` label is set on the merge request
+- when the merge request is created by an automation (for example, Gitaly update or MR targeting a stable branch)
+- when the merge request is created in a security mirror
+- when any CI configuration file is changed (for example, `.gitlab-ci.yml` or `.gitlab/ci/**/*`)
+- when any frontend "core" file is changed (for example, `package.json`, `yarn.lock`, `babel.config.js`, `jest.config.*.js`, `config/helpers/**/*.js`)
+- when any vendored JavaScript file is changed (for example, `vendor/assets/javascripts/**/*`)
+- when any backend file is changed ([see the patterns list for details](https://gitlab.com/gitlab-org/gitlab/-/blob/3616946936c1adbd9e754c1bd06f86ba670796d8/.gitlab/ci/rules.gitlab-ci.yml#L205-216))
+
+### Fork pipelines
+
+We run only the minimal RSpec & Jest jobs for fork pipelines, unless the `pipeline:run-all-rspec`
+label is set on the MR. The goal is to reduce the CI/CD minutes consumed by fork pipelines.
+
+See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1170).
+
+## Fail-fast job in merge request pipelines
+
+To provide faster feedback when a merge request breaks existing tests, we are experimenting with a
+fail-fast mechanism.
+
+An `rspec fail-fast` job is added in parallel to all other `rspec` jobs in a merge
+request pipeline. This job runs the tests that are directly related to the changes
+in the merge request.
+
+If any of these tests fail, the `rspec fail-fast` job fails, triggering a
+`fail-pipeline-early` job to run. The `fail-pipeline-early` job:
+
+- Cancels the currently running pipeline and all in-progress jobs.
+- Sets pipeline to have status `failed`.
+
+For example:
+
+```mermaid
+graph LR
+ subgraph "prepare stage";
+ A["detect-tests"]
+ end
+
+ subgraph "test stage";
+ B["jest"];
+ C["rspec migration"];
+ D["rspec unit"];
+ E["rspec integration"];
+ F["rspec system"];
+ G["rspec fail-fast"];
+ end
+
+ subgraph "post-test stage";
+ Z["fail-pipeline-early"];
+ end
+
+ A --"artifact: list of test files"--> G
+ G --"on failure"--> Z
+```
+
+The `rspec fail-fast` is a no-op if there are more than 10 test files related to the
+merge request. This prevents `rspec fail-fast` duration from exceeding the average
+`rspec` job duration and defeating its purpose.
+
+This number can be overridden by setting a CI/CD variable named `RSPEC_FAIL_FAST_TEST_FILE_COUNT_THRESHOLD`.
+
+## Faster feedback for merge requests that fix a broken `master`
+
+When you need to [fix a broken `master`](https://about.gitlab.com/handbook/engineering/workflow/#resolution-of-broken-master), you can add the `pipeline:expedite-master-fixing` label to expedite the pipelines that run on the merge request.
+
+When this label is assigned, the following steps of the CI/CD pipeline are skipped:
+
+- The `e2e:package-and-test` job.
+- The `rspec:undercoverage` job.
+- The entire [Review Apps process](../testing_guide/review_apps.md).
+
+Apply the label to the merge request, and run a new pipeline for the MR.
+
+Note that the merge request also needs to have the `master:broken` or `master:foss-broken` label set.
+
+## Test jobs
+
+We have dedicated jobs for each [testing level](../testing_guide/testing_levels.md) and each job runs depending on the
+changes made in your merge request.
+If you want to force all the RSpec jobs to run regardless of your changes, you can add the `pipeline:run-all-rspec` label to the merge request.
+
+WARNING:
+Forcing all jobs on docs only related MRs would not have the prerequisite jobs and would lead to errors
+
+### Test suite parallelization
+
+Our current RSpec tests parallelization setup is as follows:
+
+1. The `retrieve-tests-metadata` job in the `prepare` stage ensures we have a
+ `knapsack/report-master.json` file:
+ - The `knapsack/report-master.json` file is fetched from the latest `main` pipeline which runs `update-tests-metadata`
+ (for now it's the 2-hourly `maintenance` scheduled master pipeline), if it's not here we initialize the file with `{}`.
+1. Each `[rspec|rspec-ee] [migration|unit|integration|system|geo] n m` job are run with
+ `knapsack rspec` and should have an evenly distributed share of tests:
+ - It works because the jobs have access to the `knapsack/report-master.json`
+ since the "artifacts from all previous stages are passed by default".
+ - the jobs set their own report path to
+ `"knapsack/${TEST_TOOL}_${TEST_LEVEL}_${DATABASE}_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json"`.
+ - if knapsack is doing its job, test files that are run should be listed under
+ `Report specs`, not under `Leftover specs`.
+1. The `update-tests-metadata` job (which only runs on scheduled pipelines for
+ [the canonical project](https://gitlab.com/gitlab-org/gitlab) takes all the
+ `knapsack/rspec*.json` files and merge them all together into a single
+ `knapsack/report-master.json` file that is saved as artifact.
+
+After that, the next pipeline uses the up-to-date `knapsack/report-master.json` file.
+
+### Flaky tests
+
+#### Automatic skipping of flaky tests
+
+Tests that are [known to be flaky](../testing_guide/flaky_tests.md#automatic-retries-and-flaky-tests-detection) are
+skipped unless the `$SKIP_FLAKY_TESTS_AUTOMATICALLY` variable is set to `false` or if the `~"pipeline:run-flaky-tests"`
+label is set on the MR.
+
+See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1069).
+
+#### Automatic retry of failing tests in a separate process
+
+Unless `$RETRY_FAILED_TESTS_IN_NEW_PROCESS` variable is set to `false` (`true` by default), RSpec tests that failed are automatically retried once in a separate
+RSpec process. The goal is to get rid of most side-effects from previous tests that may lead to a subsequent test failure.
+
+We keep track of retried tests in the `$RETRIED_TESTS_REPORT_FILE` file saved as artifact by the `rspec:flaky-tests-report` job.
+
+See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1148).
+
+### Compatibility testing
+
+By default, we run all tests with the versions that runs on GitLab.com.
+
+Other versions (usually one back-compatible version, and one forward-compatible version) should be running in nightly scheduled pipelines.
+
+Exceptions to this general guideline should be motivated and documented.
+
+#### Single database testing
+
+By default, all tests run with [multiple databases](../database/multiple_databases.md).
+
+We also run tests with a single database in nightly scheduled pipelines, and in merge requests that touch database-related files.
+
+If you want to force tests to run with a single database, you can add the `pipeline:run-single-db` label to the merge request.
+
+### Monitoring
+
+The GitLab test suite is [monitored](../performance.md#rspec-profiling) for the `main` branch, and any branch
+that includes `rspec-profile` in their name.
+
+### Logging
+
+- Rails logging to `log/test.log` is disabled by default in CI
+ [for performance reasons](https://jtway.co/speed-up-your-rails-test-suite-by-6-in-1-line-13fedb869ec4).
+ To override this setting, provide the
+ `RAILS_ENABLE_TEST_LOG` environment variable.
+
+## Review app jobs
+
+Consult the [Review Apps](../testing_guide/review_apps.md) dedicated page for more information.
+
+If you want to force a Review App to be deployed regardless of your changes, you can add the `pipeline:run-review-app` label to the merge request.
+
+## As-if-FOSS jobs
+
+The `* as-if-foss` jobs run the GitLab test suite "as if FOSS", meaning as if the jobs would run in the context
+of `gitlab-org/gitlab-foss`. These jobs are only created in the following cases:
+
+- when the `pipeline:run-as-if-foss` label is set on the merge request
+- when the merge request is created in the `gitlab-org/security/gitlab` project
+- when any CI configuration file is changed (for example, `.gitlab-ci.yml` or `.gitlab/ci/**/*`)
+
+The `* as-if-foss` jobs are run in addition to the regular EE-context jobs. They have the `FOSS_ONLY='1'` variable
+set and get the `ee/` folder removed before the tests start running.
+
+The intent is to ensure that a change doesn't introduce a failure after `gitlab-org/gitlab` is synced to `gitlab-org/gitlab-foss`.
+
+## As-if-JH cross project downstream pipeline
+
+The `start-as-if-jh` job triggers a cross project downstream pipeline which
+runs the GitLab test suite "as if JiHu", meaning as if the pipeline would run
+in the context of [GitLab JH](../jh_features_review.md). These jobs are only
+created in the following cases:
+
+- when the `pipeline:run-as-if-jh` label is set on the merge request
+
+This pipeline runs under the context of a generated branch in the
+[GitLab JH validation](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation)
+project, which is a mirror of the
+[GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab).
+
+The generated branch name is prefixed with `as-if-jh/` along with the branch
+name in the merge request. This generated branch is based on the merge request
+branch, additionally adding changes downloaded from the
+[corresponding JH branch](#corresponding-jh-branch) on top to turn the whole
+pipeline as if JiHu.
+
+The intent is to ensure that a change doesn't introduce a failure after
+[GitLab](https://gitlab.com/gitlab-org/gitlab) is synchronized to
+[GitLab JH](https://jihulab.com/gitlab-cn/gitlab).
+
+### When to consider applying `pipeline:run-as-if-jh` label
+
+If a Ruby file is renamed and there's a corresponding [`prepend_mod` line](../jh_features_review.md#jh-features-based-on-ce-or-ee-features),
+it's likely that GitLab JH is relying on it and requires a corresponding
+change to rename the module or class it's prepending.
+
+### Corresponding JH branch
+
+You can create a corresponding JH branch on [GitLab JH](https://jihulab.com/gitlab-cn/gitlab) by
+appending `-jh` to the branch name. If a corresponding JH branch is found,
+as-if-jh pipeline grabs files from the respective branch, rather than from the
+default branch `main-jh`.
+
+NOTE:
+For now, CI will try to fetch the branch on the [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab), so it might take some time for the new JH branch to propagate to the mirror.
+
+NOTE:
+While [GitLab JH validation](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation) is a mirror of
+[GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab),
+it does not include any corresponding JH branch beside the default `main-jh`.
+This is why when we want to fetch corresponding JH branch we should fetch it
+from the main mirror, rather than the validation project.
+
+### How as-if-JH pipeline was configured
+
+The whole process looks like this:
+
+```mermaid
+flowchart TD
+ subgraph "JiHuLab.com"
+ JH["gitlab-cn/gitlab"]
+ end
+
+ subgraph "GitLab.com"
+ Mirror["gitlab-org/gitlab-jh-mirrors/gitlab"]
+ Validation["gitlab-org-sandbox/gitlab-jh-validation"]
+
+ subgraph MR["gitlab-org/gitlab merge request"]
+ Add["add-jh-files job"]
+ Prepare["prepare-as-if-jh-branch job"]
+ Add --"download artifacts"--> Prepare
+ end
+
+ Mirror --"pull mirror with master and main-jh"--> Validation
+ Mirror --"download JiHu files with ADD_JH_FILES_TOKEN"--> Add
+ Prepare --"push as-if-jh branches with AS_IF_JH_TOKEN"--> Validation
+ Validation --> Pipeline["as-if-jh pipeline"]
+ end
+
+ JH --"pull mirror with corresponding JH branches"--> Mirror
+```
+
+#### Tokens set in the project variables
+
+- `ADD_JH_FILES_TOKEN`: This is a [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab)
+ project token with `read_api` permission, to be able to download JiHu files.
+- `AS_IF_JH_TOKEN`: This is a [GitLab JH validation](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation)
+ project token with `write_repository` permission, to push generated `as-if-jh/*` branch.
+
+#### How we generate the as-if-JH branch
+
+First `add-jh-files` job will download the required JiHu files from the
+corresponding JH branch, saving in artifacts. Next `prepare-as-if-jh-branch`
+job will create a new branch from the merge request branch, commit the
+changes, and finally push the branch to the
+[validation project](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation).
+
+#### How we trigger and run the as-if-JH pipeline
+
+After having the `as-if-jh/*` branch, `start-as-if-jh` job will trigger a pipeline
+in the [validation project](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation)
+to run the cross-project downstream pipeline.
+
+#### How the GitLab JH mirror project is set up
+
+The [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab) project is private and CI is disabled.
+
+It's a pull mirror pulling from [GitLab JH](https://jihulab.com/gitlab-cn/gitlab),
+mirroring all branches, overriding divergent refs, triggering no pipelines
+when mirror is updated.
+
+The pulling user is [`@gitlab-jh-bot`](https://gitlab.com/gitlab-jh-bot), who
+is a maintainer in the project. The credentials can be found in the 1password
+engineering vault.
+
+No password is used from mirroring because GitLab JH is a public project.
+
+#### How the GitLab JH validation project is set up
+
+This [GitLab JH validation](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation) project is public and CI is enabled, without any project variables.
+
+It's a pull mirror pulling from [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab),
+mirroring only protected branches, `master` and `main-jh`, overriding
+divergent refs, triggering no pipelines when mirror is updated.
+
+The pulling user is [`@gitlab-jh-validation-bot`](https://gitlab.com/gitlab-jh-validation-bot), who is a maintainer in the project, and also a
+reporter in the
+[GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab).
+The credentials can be found in the 1password engineering vault.
+
+A personal access token from `@gitlab-jh-validation-bot` with
+`write_repository` permission is used as the password to pull changes from
+the GitLab JH mirror. Username is set with `gitlab-jh-validation-bot`.
+
+There is also a [pipeline schedule](https://gitlab.com/gitlab-org-sandbox/gitlab-jh-validation/-/pipeline_schedules)
+to run maintenance pipelines with variable `SCHEDULE_TYPE` set to `maintenance`
+running every day, updating cache.
+
+The default CI/CD configuration file is also set at `jh/.gitlab-ci.yml` so it
+runs exactly like [GitLab JH](https://jihulab.com/gitlab-cn/gitlab/-/blob/main-jh/jh/.gitlab-ci.yml).
+
+## Ruby 3.0 jobs
+
+You can add the `pipeline:run-in-ruby3` label to the merge request to switch
+the Ruby version used for running the whole test suite to 3.0. When you do
+this, the test suite will no longer run in Ruby 2.7 (default), and an
+additional job `verify-ruby-2.7` will also run and always fail to remind us to
+remove the label and run in Ruby 2.7 before merging the merge request.
+
+This should let us:
+
+- Test changes for Ruby 3.0
+- Make sure it will not break anything when it's merged into the default branch
+
+## `undercover` RSpec test
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74859) in GitLab 14.6.
+
+The `rspec:undercoverage` job runs [`undercover`](https://rubygems.org/gems/undercover)
+to detect, and fail if any changes introduced in the merge request has zero coverage.
+
+The `rspec:undercoverage` job obtains coverage data from the `rspec:coverage`
+job.
+
+In the event of an emergency, or false positive from this job, add the
+`pipeline:skip-undercoverage` label to the merge request to allow this job to
+fail.
+
+### Troubleshooting `rspec:undercoverage` failures
+
+The `rspec:undercoverage` job has [known bugs](https://gitlab.com/groups/gitlab-org/-/epics/8254)
+that can cause false positive failures. You can test coverage locally to determine if it's
+safe to apply `~"pipeline:skip-undercoverage"`. For example, using `<spec>` as the name of the
+test causing the failure:
+
+1. Run `SIMPLECOV=1 bundle exec rspec <spec>`.
+1. Run `scripts/undercoverage`.
+
+If these commands return `undercover: ✅ No coverage is missing in latest changes` then you can apply `~"pipeline:skip-undercoverage"` to bypass pipeline failures.
+
+## Ruby versions testing
+
+Our test suite runs against Ruby 2 in merge requests and default branch pipelines.
+
+We also run our test suite against Ruby 3 on another 2-hourly scheduled pipelines, as GitLab.com will soon run on Ruby 3.
+
+## PostgreSQL versions testing
+
+Our test suite runs against PG12 as GitLab.com runs on PG12 and
+[Omnibus defaults to PG12 for new installs and upgrades](../../administration/package_information/postgresql_versions.md).
+
+We do run our test suite against PG11 and PG13 on nightly scheduled pipelines.
+
+We also run our test suite against PG11 upon specific database library changes in MRs and `main` pipelines (with the `rspec db-library-code pg11` job).
+
+### Current versions testing
+
+| Where? | PostgreSQL version | Ruby version |
+|------------------------------------------------------------------------------------------------|-------------------------------------------------|--------------|
+| Merge requests | 12 (default version), 11 for DB library changes | 2.7 (default version) |
+| `master` branch commits | 12 (default version), 11 for DB library changes | 2.7 (default version) |
+| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour) | 12 (default version), 11 for DB library changes | 2.7 (default version) |
+| `maintenance` scheduled pipelines for the `ruby3` branch (every odd-numbered hour), see below. | 12 (default version), 11 for DB library changes | 3.0 (coded in the branch) |
+| `nightly` scheduled pipelines for the `master` branch | 12 (default version), 11, 13 | 2.7 (default version) |
+
+There are 2 pipeline schedules used for testing Ruby 3. One is triggering a
+pipeline in `ruby3-sync` branch, which updates the `ruby3` branch with latest
+`master`, and no pipelines will be triggered by this push. The other schedule
+is triggering a pipeline in `ruby3` 5 minutes after it, which is considered
+the maintenance schedule to run test suites and update cache.
+
+Any changes in `ruby3` are only for running the pipeline. It should
+never be merged back to `master`. Any other Ruby 3 changes should go into
+`master` directly, which should be compatible with Ruby 2.7.
+
+Previously, `ruby3-sync` was using a project token stored in `RUBY3_SYNC_TOKEN`
+(now backed up in `RUBY3_SYNC_TOKEN_NOT_USED`), however due to various
+permissions issues, we ended up using an access token from `gitlab-bot` so now
+`RUBY3_SYNC_TOKEN` is actually an access token from `gitlab-bot`.
+
+### Long-term plan
+
+We follow the [PostgreSQL versions shipped with Omnibus GitLab](../../administration/package_information/postgresql_versions.md):
+
+| PostgreSQL version | 14.1 (July 2021) | 14.2 (August 2021) | 14.3 (September 2021) | 14.4 (October 2021) | 14.5 (November 2021) | 14.6 (December 2021) |
+| -------------------| ---------------------- | ---------------------- | ---------------------- | ---------------------- | ---------------------- | ---------------------- |
+| PG12 | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` | MRs/`2-hour`/`nightly` |
+| PG11 | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` |
+| PG13 | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` |
+
+## Redis versions testing
+
+Our test suite runs against Redis 6 as GitLab.com runs on Redis 6 and
+[Omnibus defaults to Redis 6 for new installs and upgrades](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/config/software/redis.rb).
+
+We do run our test suite against Redis 5 on `nightly` scheduled pipelines, specifically when running backward-compatible and forward-compatible PostgreSQL jobs.
+
+### Current versions testing
+
+| Where? | Redis version |
+| ------ | ------------------ |
+| MRs | 6 |
+| `default branch` (non-scheduled pipelines) | 6 |
+| `nightly` scheduled pipelines | 5 |
+
+## Pipelines types for merge requests
+
+In general, pipelines for an MR fall into one of the following types (from shorter to longer), depending on the changes made in the MR:
+
+- [Documentation pipeline](#documentation-pipeline): For MRs that touch documentation.
+- [Backend pipeline](#backend-pipeline): For MRs that touch backend code.
+- [Frontend pipeline](#frontend-pipeline): For MRs that touch frontend code.
+- [End-to-end pipeline](#end-to-end-pipeline): For MRs that touch code in the `qa/` folder.
+
+A "pipeline type" is an abstract term that mostly describes the "critical path" (for example, the chain of jobs for which the sum
+of individual duration equals the pipeline's duration).
+We use these "pipeline types" in [metrics dashboards](https://app.periscopedata.com/app/gitlab/858266/GitLab-Pipeline-Durations) to detect what types and jobs need to be optimized first.
+
+An MR that touches multiple areas would be associated with the longest type applicable. For instance, an MR that touches backend
+and frontend would fall into the "Frontend" pipeline type since this type takes longer to finish than the "Backend" pipeline type.
+
+We use the [`rules:`](../../ci/yaml/index.md#rules) and [`needs:`](../../ci/yaml/index.md#needs) keywords extensively
+to determine the jobs that need to be run in a pipeline. Note that an MR that includes multiple types of changes would
+have a pipelines that include jobs from multiple types (for example, a combination of docs-only and code-only pipelines).
+
+Following are graphs of the critical paths for each pipeline type. Jobs that aren't part of the critical path are omitted.
+
+### Documentation pipeline
+
+[Reference pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipelines/432049110).
+
+```mermaid
+graph LR
+ classDef criticalPath fill:#f66;
+
+ 1-3["docs-lint links (5 minutes)"];
+ class 1-3 criticalPath;
+ click 1-3 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356757&udv=0"
+```
+
+### Backend pipeline
+
+[Reference pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipelines/433316063).
+
+```mermaid
+graph RL;
+ classDef criticalPath fill:#f66;
+
+ 1-3["compile-test-assets (6 minutes)"];
+ class 1-3 criticalPath;
+ click 1-3 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914317&udv=0"
+ 1-6["setup-test-env (4 minutes)"];
+ click 1-6 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914315&udv=0"
+ 1-14["retrieve-tests-metadata"];
+ click 1-14 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=8356697&udv=0"
+ 1-15["detect-tests"];
+ click 1-15 "https://app.periscopedata.com/app/gitlab/652085/EP---Jobs-Durations?widget=10113603&udv=1005715"
+
+ 2_5-1["rspec & db jobs (24 minutes)"];
+ class 2_5-1 criticalPath;
+ click 2_5-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations"
+ 2_5-1 --> 1-3 & 1-6 & 1-14 & 1-15;
+
+ 3_2-1["rspec:coverage (5.35 minutes)"];
+ class 3_2-1 criticalPath;
+ click 3_2-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=7248745&udv=0"
+ 3_2-1 -.->|"(don't use needs<br/>because of limitations)"| 2_5-1;
+
+ 4_3-1["rspec:undercoverage (3.5 minutes)"];
+ class 4_3-1 criticalPath;
+ click 4_3-1 "https://app.periscopedata.com/app/gitlab/652085/EP---Jobs-Durations?widget=13446492&udv=1005715"
+ 4_3-1 --> 3_2-1;
+
+```
+
+### Frontend pipeline
+
+[Reference pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipelines/431913287).
+
+```mermaid
+graph RL;
+ classDef criticalPath fill:#f66;
+
+ 1-2["build-qa-image (2 minutes)"];
+ click 1-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914325&udv=0"
+ 1-5["compile-production-assets (16 minutes)"];
+ class 1-5 criticalPath;
+ click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0"
+
+ 2_3-1["build-assets-image (1.3 minutes)"];
+ class 2_3-1 criticalPath;
+ 2_3-1 --> 1-5
+
+ 2_6-1["start-review-app-pipeline (49 minutes)"];
+ class 2_6-1 criticalPath;
+ click 2_6-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations"
+ 2_6-1 --> 2_3-1 & 1-2;
+```
+
+### End-to-end pipeline
+
+[Reference pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipelines/431918463).
+
+```mermaid
+graph RL;
+ classDef criticalPath fill:#f66;
+
+ 1-2["build-qa-image (2 minutes)"];
+ click 1-2 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914325&udv=0"
+ 1-5["compile-production-assets (16 minutes)"];
+ class 1-5 criticalPath;
+ click 1-5 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914312&udv=0"
+ 1-15["detect-tests"];
+ click 1-15 "https://app.periscopedata.com/app/gitlab/652085/EP---Jobs-Durations?widget=10113603&udv=1005715"
+
+ 2_3-1["build-assets-image (1.3 minutes)"];
+ class 2_3-1 criticalPath;
+ 2_3-1 --> 1-5
+
+ 2_4-1["e2e:package-and-test (102 minutes)"];
+ class 2_4-1 criticalPath;
+ click 2_4-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=6914305&udv=0"
+ 2_4-1 --> 1-2 & 2_3-1 & 1-15;
+```
+
+## CI configuration internals
+
+See the dedicated [CI configuration internals page](internals.md).
+
+## Performance
+
+See the dedicated [CI configuration performance page](performance.md).
+
+---
+
+[Return to Development documentation](../index.md)
diff --git a/doc/development/pipelines/internals.md b/doc/development/pipelines/internals.md
new file mode 100644
index 00000000000..2861e2f266b
--- /dev/null
+++ b/doc/development/pipelines/internals.md
@@ -0,0 +1,216 @@
+---
+stage: none
+group: Engineering Productivity
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# CI configuration internals
+
+## Workflow rules
+
+Pipelines for the GitLab project are created using the [`workflow:rules` keyword](../../ci/yaml/index.md#workflow)
+feature of the GitLab CI/CD.
+
+Pipelines are always created for the following scenarios:
+
+- `main` branch, including on schedules, pushes, merges, and so on.
+- Merge requests.
+- Tags.
+- Stable, `auto-deploy`, and security branches.
+
+Pipeline creation is also affected by the following CI/CD variables:
+
+- If `$FORCE_GITLAB_CI` is set, pipelines are created.
+- If `$GITLAB_INTERNAL` is not set, pipelines are not created.
+
+No pipeline is created in any other cases (for example, when pushing a branch with no
+MR for it).
+
+The source of truth for these workflow rules is defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml).
+
+## Default image
+
+The default image is defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml).
+
+<!-- vale gitlab.Spelling = NO -->
+
+It includes Ruby, Go, Git, Git LFS, Chrome, Node, Yarn, PostgreSQL, and Graphics Magick.
+
+<!-- vale gitlab.Spelling = YES -->
+
+The images used in our pipelines are configured in the
+[`gitlab-org/gitlab-build-images`](https://gitlab.com/gitlab-org/gitlab-build-images)
+project, which is push-mirrored to [`gitlab/gitlab-build-images`](https://dev.gitlab.org/gitlab/gitlab-build-images)
+for redundancy.
+
+The current version of the build images can be found in the
+["Used by GitLab section"](https://gitlab.com/gitlab-org/gitlab-build-images/blob/master/.gitlab-ci.yml).
+
+## Default variables
+
+In addition to the [predefined CI/CD variables](../../ci/variables/predefined_variables.md),
+each pipeline includes default variables defined in
+[`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml).
+
+## Stages
+
+The current stages are:
+
+- `sync`: This stage is used to synchronize changes from [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) to
+ [`gitlab-org/gitlab-foss`](https://gitlab.com/gitlab-org/gitlab-foss).
+- `prepare`: This stage includes jobs that prepare artifacts that are needed by
+ jobs in subsequent stages.
+- `build-images`: This stage includes jobs that prepare Docker images
+ that are needed by jobs in subsequent stages or downstream pipelines.
+- `fixtures`: This stage includes jobs that prepare fixtures needed by frontend tests.
+- `lint`: This stage includes linting and static analysis jobs.
+- `test`: This stage includes most of the tests, and DB/migration jobs.
+- `post-test`: This stage includes jobs that build reports or gather data from
+ the `test` stage's jobs (for example, coverage, Knapsack metadata, and so on).
+- `review`: This stage includes jobs that build the CNG images, deploy them, and
+ run end-to-end tests against Review Apps (see [Review Apps](../testing_guide/review_apps.md) for details).
+ It also includes Docs Review App jobs.
+- `qa`: This stage includes jobs that perform QA tasks against the Review App
+ that is deployed in stage `review`.
+- `post-qa`: This stage includes jobs that build reports or gather data from
+ the `qa` stage's jobs (for example, Review App performance report).
+- `pages`: This stage includes a job that deploys the various reports as
+ GitLab Pages (for example, [`coverage-ruby`](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/),
+ and `webpack-report` (found at `https://gitlab-org.gitlab.io/gitlab/webpack-report/`, but there is
+ [an issue with the deployment](https://gitlab.com/gitlab-org/gitlab/-/issues/233458)).
+- `notify`: This stage includes jobs that notify various failures to Slack.
+
+## Dependency Proxy
+
+Some of the jobs are using images from Docker Hub, where we also use
+`${GITLAB_DEPENDENCY_PROXY_ADDRESS}` as a prefix to the image path, so that we pull
+images from our [Dependency Proxy](../../user/packages/dependency_proxy/index.md).
+By default, this variable is set from the value of `${GITLAB_DEPENDENCY_PROXY}`.
+
+`${GITLAB_DEPENDENCY_PROXY}` is a group CI/CD variable defined in
+[`gitlab-org`](https://gitlab.com/gitlab-org) as
+`${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/`. This means when we use an image
+defined as:
+
+```yaml
+image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}alpine:edge
+```
+
+Projects in the `gitlab-org` group pull from the Dependency Proxy, while
+forks that reside on any other personal namespaces or groups fall back to
+Docker Hub unless `${GITLAB_DEPENDENCY_PROXY}` is also defined there.
+
+### Work around for when a pipeline is started by a Project access token user
+
+When a pipeline is started by a Project access token user (e.g. the `release-tools approver bot` user which
+automatically updates the Gitaly version used in the main project),
+[the Dependency proxy isn't accessible](https://gitlab.com/gitlab-org/gitlab/-/issues/332411#note_1130388163)
+and the job fails at the `Preparing the "docker+machine" executor` step.
+To work around that, we have a special workflow rule, that overrides the
+`${GITLAB_DEPENDENCY_PROXY_ADDRESS}` variable so that Depdendency proxy isn't used in that case:
+
+```yaml
+- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $GITLAB_USER_LOGIN =~ /project_\d+_bot\d*/'
+ variables:
+ GITLAB_DEPENDENCY_PROXY_ADDRESS: ""
+```
+
+NOTE:
+We don't directly override the `${GITLAB_DEPENDENCY_PROXY}` variable because group-level
+variables have higher precedence over `.gitlab-ci.yml` variables.
+
+## Common job definitions
+
+Most of the jobs [extend from a few CI definitions](../../ci/yaml/index.md#extends)
+defined in [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml)
+that are scoped to a single [configuration keyword](../../ci/yaml/index.md#job-keywords).
+
+| Job definitions | Description |
+|------------------|-------------|
+| `.default-retry` | Allows a job to [retry](../../ci/yaml/index.md#retry) upon `unknown_failure`, `api_failure`, `runner_system_failure`, `job_execution_timeout`, or `stuck_or_timeout_failure`. |
+| `.default-before_script` | Allows a job to use a default `before_script` definition suitable for Ruby/Rails tasks that may need a database running (for example, tests). |
+| `.setup-test-env-cache` | Allows a job to use a default `cache` definition suitable for setting up test environment for subsequent Ruby/Rails tasks. |
+| `.rails-cache` | Allows a job to use a default `cache` definition suitable for Ruby/Rails tasks. |
+| `.static-analysis-cache` | Allows a job to use a default `cache` definition suitable for static analysis tasks. |
+| `.coverage-cache` | Allows a job to use a default `cache` definition suitable for coverage tasks. |
+| `.qa-cache` | Allows a job to use a default `cache` definition suitable for QA tasks. |
+| `.yarn-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that do a `yarn install`. |
+| `.assets-compile-cache` | Allows a job to use a default `cache` definition suitable for frontend jobs that compile assets. |
+| `.use-pg11` | Allows a job to run the `postgres` 11 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
+| `.use-pg11-ee` | Same as `.use-pg11` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). |
+| `.use-pg12` | Allows a job to use the `postgres` 12 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
+| `.use-pg12-ee` | Same as `.use-pg12` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). |
+| `.use-pg13` | Allows a job to use the `postgres` 13 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). |
+| `.use-pg13-ee` | Same as `.use-pg13` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). |
+| `.use-kaniko` | Allows a job to use the `kaniko` tool to build Docker images. |
+| `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` CI/CD variable. |
+| `.use-docker-in-docker` | Allows a job to use Docker in Docker. |
+
+## `rules`, `if:` conditions and `changes:` patterns
+
+We're using the [`rules` keyword](../../ci/yaml/index.md#rules) extensively.
+
+All `rules` definitions are defined in
+[`rules.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml),
+then included in individual jobs via [`extends`](../../ci/yaml/index.md#extends).
+
+The `rules` definitions are composed of `if:` conditions and `changes:` patterns,
+which are also defined in
+[`rules.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml)
+and included in `rules` definitions via [YAML anchors](../../ci/yaml/yaml_optimization.md#anchors)
+
+### `if:` conditions
+
+<!-- vale gitlab.Substitutions = NO -->
+
+| `if:` conditions | Description | Notes |
+|------------------|-------------|-------|
+| `if-not-canonical-namespace` | Matches if the project isn't in the canonical (`gitlab-org/`) or security (`gitlab-org/security`) namespace. | Use to create a job for forks (by using `when: on_success|manual`), or **not** create a job for forks (by using `when: never`). |
+| `if-not-ee` | Matches if the project isn't EE (that is, project name isn't `gitlab` or `gitlab-ee`). | Use to create a job only in the FOSS project (by using `when: on_success|manual`), or **not** create a job if the project is EE (by using `when: never`). |
+| `if-not-foss` | Matches if the project isn't FOSS (that is, project name isn't `gitlab-foss`, `gitlab-ce`, or `gitlabhq`). | Use to create a job only in the EE project (by using `when: on_success|manual`), or **not** create a job if the project is FOSS (by using `when: never`). |
+| `if-default-refs` | Matches if the pipeline is for `master`, `main`, `/^[\d-]+-stable(-ee)?$/` (stable branches), `/^\d+-\d+-auto-deploy-\d+$/` (auto-deploy branches), `/^security\//` (security branches), merge requests, and tags. | Note that jobs aren't created for branches with this default configuration. |
+| `if-master-refs` | Matches if the current branch is `master` or `main`. | |
+| `if-master-push` | Matches if the current branch is `master` or `main` and pipeline source is `push`. | |
+| `if-master-schedule-maintenance` | Matches if the current branch is `master` or `main` and pipeline runs on a 2-hourly schedule. | |
+| `if-master-schedule-nightly` | Matches if the current branch is `master` or `main` and pipeline runs on a nightly schedule. | |
+| `if-auto-deploy-branches` | Matches if the current branch is an auto-deploy one. | |
+| `if-master-or-tag` | Matches if the pipeline is for the `master` or `main` branch or for a tag. | |
+| `if-merge-request` | Matches if the pipeline is for a merge request. | |
+| `if-merge-request-title-as-if-foss` | Matches if the pipeline is for a merge request and the MR has label ~"pipeline:run-as-if-foss" | |
+| `if-merge-request-title-update-caches` | Matches if the pipeline is for a merge request and the MR has label ~"pipeline:update-cache". | |
+| `if-merge-request-title-run-all-rspec` | Matches if the pipeline is for a merge request and the MR has label ~"pipeline:run-all-rspec". | |
+| `if-security-merge-request` | Matches if the pipeline is for a security merge request. | |
+| `if-security-schedule` | Matches if the pipeline is for a security scheduled pipeline. | |
+| `if-nightly-master-schedule` | Matches if the pipeline is for a `master` scheduled pipeline with `$NIGHTLY` set. | |
+| `if-dot-com-gitlab-org-schedule` | Limits jobs creation to scheduled pipelines for the `gitlab-org` group on GitLab.com. | |
+| `if-dot-com-gitlab-org-master` | Limits jobs creation to the `master` or `main` branch for the `gitlab-org` group on GitLab.com. | |
+| `if-dot-com-gitlab-org-merge-request` | Limits jobs creation to merge requests for the `gitlab-org` group on GitLab.com. | |
+| `if-dot-com-gitlab-org-and-security-tag` | Limits job creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | |
+| `if-dot-com-gitlab-org-and-security-merge-request` | Limit jobs creation to merge requests for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | |
+| `if-dot-com-gitlab-org-and-security-tag` | Limit jobs creation to tags for the `gitlab-org` and `gitlab-org/security` groups on GitLab.com. | |
+| `if-dot-com-ee-schedule` | Limits jobs to scheduled pipelines for the `gitlab-org/gitlab` project on GitLab.com. | |
+| `if-security-pipeline-merge-result` | Matches if the pipeline is for a security merge request triggered by `@gitlab-release-tools-bot`. | |
+
+<!-- vale gitlab.Substitutions = YES -->
+
+### `changes:` patterns
+
+| `changes:` patterns | Description |
+|------------------------------|--------------------------------------------------------------------------|
+| `ci-patterns` | Only create job for CI configuration-related changes. |
+| `ci-build-images-patterns` | Only create job for CI configuration-related changes related to the `build-images` stage. |
+| `ci-review-patterns` | Only create job for CI configuration-related changes related to the `review` stage. |
+| `ci-qa-patterns` | Only create job for CI configuration-related changes related to the `qa` stage. |
+| `yaml-lint-patterns` | Only create job for YAML-related changes. |
+| `docs-patterns` | Only create job for docs-related changes. |
+| `frontend-dependency-patterns` | Only create job when frontend dependencies are updated (that is, `package.json`, and `yarn.lock`). changes. |
+| `frontend-patterns-for-as-if-foss` | Only create job for frontend-related changes that have impact on FOSS. |
+| `backend-patterns` | Only create job for backend-related changes. |
+| `db-patterns` | Only create job for DB-related changes. |
+| `backstage-patterns` | Only create job for backstage-related changes (that is, Danger, fixtures, RuboCop, specs). |
+| `code-patterns` | Only create job for code-related changes. |
+| `qa-patterns` | Only create job for QA-related changes. |
+| `code-backstage-patterns` | Combination of `code-patterns` and `backstage-patterns`. |
+| `code-qa-patterns` | Combination of `code-patterns` and `qa-patterns`. |
+| `code-backstage-qa-patterns` | Combination of `code-patterns`, `backstage-patterns`, and `qa-patterns`. |
+| `static-analysis-patterns` | Only create jobs for Static Analytics configuration-related changes. |
diff --git a/doc/development/pipelines/performance.md b/doc/development/pipelines/performance.md
new file mode 100644
index 00000000000..1c6f9d78879
--- /dev/null
+++ b/doc/development/pipelines/performance.md
@@ -0,0 +1,151 @@
+---
+stage: none
+group: Engineering Productivity
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# CI configuration performance
+
+## Interruptible pipelines
+
+By default, all jobs are [interruptible](../../ci/yaml/index.md#interruptible), except the
+`dont-interrupt-me` job which runs automatically on `main`, and is `manual`
+otherwise.
+
+If you want a running pipeline to finish even if you push new commits to a merge
+request, be sure to start the `dont-interrupt-me` job before pushing.
+
+## Git fetch caching
+
+Because GitLab.com uses the [pack-objects cache](../../administration/gitaly/configure_gitaly.md#pack-objects-cache),
+concurrent Git fetches of the same pipeline ref are deduplicated on
+the Gitaly server (always) and served from cache (when available).
+
+This works well for the following reasons:
+
+- The pack-objects cache is enabled on all Gitaly servers on GitLab.com.
+- The CI/CD [Git strategy setting](../../ci/pipelines/settings.md#choose-the-default-git-strategy) for `gitlab-org/gitlab` is **Git clone**,
+ causing all jobs to fetch the same data, which maximizes the cache hit ratio.
+- We use [shallow clone](../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone) to avoid downloading the full Git
+ history for every job.
+
+## Caching strategy
+
+1. All jobs must only pull caches by default.
+1. All jobs must be able to pass with an empty cache. In other words, caches are only there to speed up jobs.
+1. We currently have several different cache definitions defined in
+ [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml),
+ with fixed keys:
+ - `.setup-test-env-cache`
+ - `.ruby-cache`
+ - `.rails-cache`
+ - `.static-analysis-cache`
+ - `.rubocop-cache`
+ - `.coverage-cache`
+ - `.ruby-node-cache`
+ - `.qa-cache`
+ - `.yarn-cache`
+ - `.assets-compile-cache` (the key includes `${NODE_ENV}` so it's actually two different caches).
+1. These cache definitions are composed of [multiple atomic caches](../../ci/caching/index.md#use-multiple-caches).
+1. Only the following jobs, running in 2-hourly `maintenance` scheduled pipelines, are pushing (that is, updating) to the caches:
+ - `update-setup-test-env-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rails.gitlab-ci.yml).
+ - `update-gitaly-binaries-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rails.gitlab-ci.yml).
+ - `update-rubocop-cache`, defined in [`.gitlab/ci/rails.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rails.gitlab-ci.yml).
+ - `update-qa-cache`, defined in [`.gitlab/ci/qa.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/qa.gitlab-ci.yml).
+ - `update-assets-compile-production-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml).
+ - `update-assets-compile-test-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml).
+ - `update-yarn-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml).
+ - `update-storybook-yarn-cache`, defined in [`.gitlab/ci/frontend.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/frontend.gitlab-ci.yml).
+1. These jobs can also be forced to run in merge requests with the `pipeline:update-cache` label (this can be useful to warm the caches in a MR that updates the cache keys).
+
+## Artifacts strategy
+
+We limit the artifacts that are saved and retrieved by jobs to the minimum to reduce the upload/download time and costs, as well as the artifacts storage.
+
+## Components caching
+
+Some external components (GitLab Workhorse and frontend assets) of GitLab need to be built from source as a preliminary step for running tests.
+
+## `cache-workhorse`
+
+In [this MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79766), and then
+[this MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96297),
+we introduced a new `cache-workhorse` job that:
+
+- runs automatically for all GitLab.com `gitlab-org/gitlab` scheduled pipelines
+- runs automatically for any `master` commit that touches the `workhorse/` folder
+- is manual for GitLab.com's `gitlab-org`'s MRs that touches caching-related files
+
+This job tries to download a generic package that contains GitLab Workhorse binaries needed in the GitLab test suite (under `tmp/tests/gitlab-workhorse`).
+
+- If the package URL returns a 404:
+ 1. It runs `scripts/setup-test-env`, so that the GitLab Workhorse binaries are built.
+ 1. It then creates an archive which contains the binaries and upload it [as a generic package](https://gitlab.com/gitlab-org/gitlab/-/packages/).
+- Otherwise, if the package already exists, it exits the job successfully.
+
+We also changed the `setup-test-env` job to:
+
+1. First download the GitLab Workhorse generic package build and uploaded by `cache-workhorse`.
+1. If the package is retrieved successfully, its content is placed in the right folder (for example, `tmp/tests/gitlab-workhorse`), preventing the building of the binaries when `scripts/setup-test-env` is run later on.
+1. If the package URL returns a 404, the behavior doesn't change compared to the current one: the GitLab Workhorse binaries are built as part of `scripts/setup-test-env`.
+
+NOTE:
+The version of the package is the workhorse tree SHA (for example, `git rev-parse HEAD:workhorse`).
+
+## `cache-assets`
+
+In [this MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96297),
+we introduced three new `cache-assets:test`, `cache-assets:test as-if-foss`,
+and `cache-assets:production` jobs that:
+
+- never run unless `$CACHE_ASSETS_AS_PACKAGE == "true"`
+- runs automatically for all GitLab.com `gitlab-org/gitlab` scheduled pipelines
+- runs automatically for any `master` commit that touches the assets-related folders
+- is manual for GitLab.com's `gitlab-org`'s MRs that touches caching-related files
+
+This job tries to download a generic package that contains GitLab compiled assets
+needed in the GitLab test suite (under `app/assets/javascripts/locale/**/app.js`,
+and `public/assets`).
+
+- If the package URL returns a 404:
+ 1. It runs `bin/rake gitlab:assets:compile`, so that the GitLab assets are compiled.
+ 1. It then creates an archive which contains the assets and uploads it [as a generic package](https://gitlab.com/gitlab-org/gitlab/-/packages/).
+ The package version is set to the assets folders' hash sum.
+- Otherwise, if the package already exists, it exits the job successfully.
+
+## `compile-*-assets`
+
+We also changed the `compile-test-assets`, `compile-test-assets as-if-foss`,
+and `compile-production-assets` jobs to:
+
+1. First download the "native" cache assets, which contain:
+ - The [compiled assets](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/global.gitlab-ci.yml#L86-87).
+ - A [`cached-assets-hash.txt` file](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/global.gitlab-ci.yml#L85)
+ containing the `SHA256` hexdigest of all the source files on which the assets depend on.
+ This list of files is a pessimistic list and the assets might not depend on
+ some of these files. At worst we compile the assets more often, which is better than
+ using outdated assets.
+
+ The file is [created after assets are compiled](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/frontend.gitlab-ci.yml#L83).
+1. We then we compute the `SHA256` hexdigest of all the source files the assets depend on, **for the current checked out branch**. We [store the hexdigest in the `GITLAB_ASSETS_HASH` variable](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/frontend.gitlab-ci.yml#L27).
+1. If `$CACHE_ASSETS_AS_PACKAGE == "true"`, we download the generic package built and uploaded by [`cache-assets:*`](#cache-assets).
+ - If the cache is up-to-date for the checked out branch, we download the native cache
+ **and** the cache package. We could optimize that by not downloading
+ the genetic package but the native cache is actually very often outdated because it's
+ rebuilt only every 2 hours.
+1. We [run the `assets_compile_script` function](https://gitlab.com/gitlab-org/gitlab/-/blob/a6910c9086bb28e553f5e747ec2dd50af6da3c6b/.gitlab/ci/frontend.gitlab-ci.yml#L35),
+ which [itself runs](https://gitlab.com/gitlab-org/gitlab/-/blob/c023191ef412e868ae957f3341208a41ca678403/scripts/utils.sh#L76)
+ the [`assets:compile` Rake task](https://gitlab.com/gitlab-org/gitlab/-/blob/c023191ef412e868ae957f3341208a41ca678403/lib/tasks/gitlab/assets.rake#L80-109).
+
+ This task is responsible for deciding if assets need to be compiled or not.
+ It [compares the `HEAD` `SHA256` hexdigest from `$GITLAB_ASSETS_HASH` with the `master` hexdigest from `cached-assets-hash.txt`](https://gitlab.com/gitlab-org/gitlab/-/blob/c023191ef412e868ae957f3341208a41ca678403/lib/tasks/gitlab/assets.rake#L86).
+1. If the hashes are the same, we don't compile anything. If they're different, we compile the assets.
+
+## Pre-clone step
+
+NOTE:
+We no longer use this optimization for `gitlab-org/gitlab` because the [pack-objects cache](../../administration/gitaly/configure_gitaly.md#pack-objects-cache)
+allows Gitaly to serve the full CI/CD fetch traffic now. See [Git fetch caching](#git-fetch-caching).
+
+The pre-clone step works by using the `CI_PRE_CLONE_SCRIPT` variable
+[defined by GitLab.com shared runners](../../ci/runners/saas/linux_saas_runner.md#pre-clone-script).
diff --git a/doc/development/prometheus_metrics.md b/doc/development/prometheus_metrics.md
index c2caa354567..d3d809c5386 100644
--- a/doc/development/prometheus_metrics.md
+++ b/doc/development/prometheus_metrics.md
@@ -36,7 +36,7 @@ After you add or change an existing common metric, you must [re-run the import s
Or, you can create a database migration:
```ruby
-class ImportCommonMetrics < Gitlab::Database::Migration[1.0]
+class ImportCommonMetrics < Gitlab::Database::Migration[2.0]
def up
::Gitlab::DatabaseImporters::CommonMetrics::Importer.new.execute
end
diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md
index 505f480c410..14fbe0e875b 100644
--- a/doc/development/rake_tasks.md
+++ b/doc/development/rake_tasks.md
@@ -85,6 +85,18 @@ To import these metrics, you can run:
bundle exec rake 'gitlab:seed:development_metrics[your_project_id]'
```
+#### Seed a project with vulnerabilities
+
+You can seed a project with [security vulnerabilities](../user/application_security/vulnerabilities/index.md).
+
+```shell
+# Seed all projects
+bin/rake 'gitlab:seed:vulnerabilities'
+
+# Seed a specific project
+bin/rake 'gitlab:seed:vulnerabilities[group-path/project-path]'
+```
+
### Automation
If you're very sure that you want to **wipe the current database** and refill
@@ -208,7 +220,7 @@ bundle exec rake rubocop:todo:generate\[Gitlab/NamespacedClass,Lint/Syntax\]
Some shells require brackets to be escaped or quoted.
-See [Resolving RuboCop exceptions](contributing/style_guides.md#resolving-rubocop-exceptions)
+See [Resolving RuboCop exceptions](../development/rubocop_development_guide.md#resolving-rubocop-exceptions)
on how to proceed from here.
### Run RuboCop in graceful mode
diff --git a/doc/development/reusing_abstractions.md b/doc/development/reusing_abstractions.md
index 2701192137c..58d1e20394c 100644
--- a/doc/development/reusing_abstractions.md
+++ b/doc/development/reusing_abstractions.md
@@ -151,7 +151,8 @@ When implementing a service class, consider:
developer's discretion, such as: `issue`, `project`, `merge_request`.
1. When service represents an action initiated by a user or executed in the
context of a user, the initializer must have the `current_user:` keyword argument.
- Services with `current_user:` argument run high-level business logic.
+ Services with the `current_user:` argument run high-level business logic
+ and must validate user authorization to perform their operations.
1. When service does not have a user context and it's not directly initiated
by a user (like background service or side-effects), the `current_user:`
argument is not needed. This describes low-level domain logic or instance-wide logic.
diff --git a/doc/development/routing.md b/doc/development/routing.md
index 16ed15fdcc6..8f475674c39 100644
--- a/doc/development/routing.md
+++ b/doc/development/routing.md
@@ -39,7 +39,7 @@ You can view and find routes from the console by running:
rails routes | grep crm
```
-You can also view routes in your browser by going to [http://gdk.test:3000/rails/info/routes](http://gdk.test:3000/rails/info/routes).
+You can also view routes in your browser by going to `http://gdk.test:3000/rails/info/routes`.
## Global routes
diff --git a/doc/development/rubocop_development_guide.md b/doc/development/rubocop_development_guide.md
new file mode 100644
index 00000000000..2ff94f65232
--- /dev/null
+++ b/doc/development/rubocop_development_guide.md
@@ -0,0 +1,114 @@
+---
+type: reference, dev
+stage: none
+group: Development
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# RuboCop rule development guide
+
+Our codebase style is defined and enforced by [RuboCop](https://github.com/rubocop-hq/rubocop).
+
+You can check for any offenses locally with `bundle exec rubocop --parallel`.
+On the CI, this is automatically checked by the `static-analysis` jobs.
+
+In addition, you can [integrate RuboCop](developing_with_solargraph.md) into
+supported IDEs using the [Solargraph](https://github.com/castwide/solargraph) gem.
+
+For RuboCop rules that we have not taken a decision on, follow the [Ruby style guide](backend/ruby_style_guide.md) to write idiomatic Ruby.
+
+Reviewers/maintainers should be tolerant and not too pedantic about style.
+
+Some RuboCop rules are disabled, and for those,
+reviewers/maintainers must not ask authors to use one style or the other, as both
+are accepted. This isn't an ideal situation because this leaves space for
+[bike-shedding](https://en.wiktionary.org/wiki/bikeshedding). Ideally we
+should enable all RuboCop rules to avoid style-related
+discussions, nitpicking, or back-and-forth in reviews. The
+[GitLab Ruby style guide](backend/ruby_style_guide.md) includes a non-exhaustive
+list of styles that commonly come up in reviews and are not enforced.
+
+Additionally, we have dedicated
+[test-specific style guides and best practices](testing_guide/index.md).
+
+## Creating new RuboCop cops
+
+Typically it is better for the linting rules to be enforced programmatically as it
+reduces the aforementioned [bike-shedding](https://en.wiktionary.org/wiki/bikeshedding).
+
+To that end, we encourage creation of new RuboCop rules in the codebase.
+
+Before adding a new cop to enforce a given style, make sure to discuss it with your team.
+
+We maintain cops across several Ruby code bases, and not all of them are
+specific to the GitLab application.
+When creating a new cop that could be applied to multiple applications, we encourage you
+to add it to our [`gitlab-styles`](https://gitlab.com/gitlab-org/gitlab-styles) gem.
+If the cop targets rules that only apply to the main GitLab application,
+it should be added to [GitLab](https://gitlab.com/gitlab-org/gitlab) instead.
+
+## Cop grace period
+
+A cop is in a _grace period_ if it is enabled and has `Details: grace period` defined in its TODO YAML configuration.
+
+On the default branch, offenses from cops in the [grace period](rake_tasks.md#run-rubocop-in-graceful-mode) do not fail the RuboCop CI job. Instead, the job notifies the `#f_rubocop` Slack channel. However, on other branches, the RuboCop job fails.
+
+A grace period can safely be lifted as soon as there are no warnings for 2 weeks in the `#f_rubocop` channel on Slack.
+
+## Enabling a new cop
+
+1. Enable the new cop in `.rubocop.yml` (if not already done via [`gitlab-styles`](https://gitlab.com/gitlab-org/ruby/gems/gitlab-styles)).
+1. [Generate TODOs for the new cop](rake_tasks.md#generate-initial-rubocop-todo-list).
+1. [Set the new cop to `grace period`](#cop-grace-period).
+1. Create an issue to fix TODOs and encourage community contributions (via ~"good for new contributors" and/or ~"Seeking community contributions"). [See some examples](https://gitlab.com/gitlab-org/gitlab/-/issues/?sort=created_date&state=opened&label_name%5B%5D=good%20for%20new%20contributors&label_name%5B%5D=static%20code%20analysis&first_page_size=20).
+1. Create an issue to remove `grace period` after 2 weeks of silence in the `#f_rubocop` Slack channel. [See an example](https://gitlab.com/gitlab-org/gitlab/-/issues/374903).
+
+## Silenced offenses
+
+When offenses are silenced for cops in the [grace period](#cop-grace-period),
+the `#f_rubocop` Slack channel receives a notification message every 2 hours.
+
+To fix this issue:
+
+1. Find cops with silenced offenses in the linked CI job.
+1. [Generate TODOs](rake_tasks.md#generate-initial-rubocop-todo-list) for these cops.
+
+### RuboCop node pattern
+
+When creating [node patterns](https://docs.rubocop.org/rubocop-ast/node_pattern.html) to match
+Ruby's AST, you can use [`scripts/rubocop-parse`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/rubocop-parse).
+This displays the AST of a Ruby expression to help you create the matcher.
+See also [!97024](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97024).
+
+## Resolving RuboCop exceptions
+
+When the number of RuboCop exceptions exceeds the default [`exclude-limit` of 15](https://docs.rubocop.org/rubocop/1.2/usage/basic_usage.html#command-line-flags),
+we may want to resolve exceptions over multiple commits. To minimize confusion,
+we should track our progress through the exception list.
+
+The preferred way to [generate the initial list or a list for specific RuboCop rules](rake_tasks.md#generate-initial-rubocop-todo-list)
+is to run the Rake task `rubocop:todo:generate`:
+
+```shell
+# Initial list
+bundle exec rake rubocop:todo:generate
+
+# List for specific RuboCop rules
+bundle exec rake 'rubocop:todo:generate[Gitlab/NamespacedClass,Lint/Syntax]'
+```
+
+This Rake task creates or updates the exception list in `.rubocop_todo/`. For
+example, the configuration for the RuboCop rule `Gitlab/NamespacedClass` is
+located in `.rubocop_todo/gitlab/namespaced_class.yml`.
+
+Make sure to commit any changes in `.rubocop_todo/` after running the Rake task.
+
+## Reveal existing RuboCop exceptions
+
+To reveal existing RuboCop exceptions in the code that have been excluded via `.rubocop_todo.yml` and
+`.rubocop_todo/**/*.yml`, set the environment variable `REVEAL_RUBOCOP_TODO` to `1`.
+
+This allows you to reveal existing RuboCop exceptions during your daily work cycle and fix them along the way.
+
+NOTE:
+Define permanent `Exclude`s in `.rubocop.yml` instead of `.rubocop_todo/**/*.yml`.
diff --git a/doc/development/sec/analyzer_development_guide.md b/doc/development/sec/analyzer_development_guide.md
index a35bc2b7237..af3a6f2b7d7 100644
--- a/doc/development/sec/analyzer_development_guide.md
+++ b/doc/development/sec/analyzer_development_guide.md
@@ -78,11 +78,23 @@ go build -o analyzer
./analyzer convert test/fixtures/app/spotbugsXml.Xml > ./gl-sast-report.json
```
+### Execution criteria
+
+[Enabling SAST](../../user/application_security/sast/index.md#configure-sast-manually) requires including a pre-defined [template](https://gitlab.com/gitlab-org/gitlab/-/blob/ee4d473eb9a39f2f84b719aa0ca13d2b8e11dc7e/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) to your GitLab CI/CD configuration.
+
+The following independent criteria determine which analyzer needs to be run on a project:
+
+1. The SAST template uses [`rules:exists`](../../ci/yaml/index.md#rulesexists) to determine which analyzer will be run based on the presence of certain files. For example, the Brakeman analyzer [runs when there are](https://gitlab.com/gitlab-org/gitlab/-/blob/ee4d473eb9a39f2f84b719aa0ca13d2b8e11dc7e/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml#L60) `.rb` files and a `Gemfile`.
+1. Each analyzer runs a customizable [match interface](https://gitlab.com/gitlab-org/security-products/analyzers/common/-/blob/master/search/search.go) before it performs the actual analysis. For example: [Flawfinder checks for C/C++ files](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder/-/blob/f972ac786268fb649553056a94cda05cdc1248b2/plugin/plugin.go#L14).
+1. For some analyzers that run on generic file extensions, there is a check based on a CI/CD variable. For example: Kubernetes manifests are written in YAML, so [Kubesec](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) runs only when [`SCAN_KUBERNETES_MANIFESTS` is set to true](../../user/application_security/sast/index.md#enabling-kubesec-analyzer).
+
+Step 1 helps prevent wastage of CI/CD minutes that would be spent running analyzers not suitable for the project. However, due to [technical limitations](https://gitlab.com/gitlab-org/gitlab/-/issues/227632), it cannot be used for large projects. Therefore, step 2 acts as final check to ensure a mismatched analyzer is able to exit early.
+
## How to test the analyzers
Video walkthrough of how Dependency Scanning analyzers are using [downstream pipeline](../../ci/pipelines/downstream_pipelines.md) feature to test analyzers using test projects:
-[![How Sec leverages the downstream pipeline feature of GitLab to test analyzers end to end](http://img.youtube.com/vi/KauRBlfUbDE/0.jpg)](http://www.youtube.com/watch?v=KauRBlfUbDE)
+[![How Sec leverages the downstream pipeline feature of GitLab to test analyzers end to end](https://img.youtube.com/vi/KauRBlfUbDE/0.jpg)](https://www.youtube.com/watch?v=KauRBlfUbDE)
### Testing local changes
@@ -118,6 +130,12 @@ To use Docker with `replace` in the `go.mod` file:
1. Update the `replace` statement to make sure it matches the destination of the `COPY` statement in the step above:
`replace gitlab.com/gitlab-org/security-products/analyzers/command/v3 => /command`
+## Analyzer scripts
+
+The [analyzer-scripts](https://gitlab.com/gitlab-org/secure/tools/analyzer-scripts) repository contains scripts that can be used to interact with most analyzers. They enable you to build, run, and debug analyzers in a GitLab CI-like environment, and are particularly useful for locally validating changes to an analyzer.
+
+For more information, refer to the [project README](https://gitlab.com/gitlab-org/secure/tools/analyzer-scripts/-/blob/master/README.md).
+
## Versioning and release process
Analyzers are independent projects that follow their own versioning. `Patch` version bumps tend to correspond to a `Minor` version bump of the underlying tools (i.e. [`bandit`](https://wiki.openstack.org/wiki/Security/Projects/Bandit)), allowing us greater flexibility in reserving `Minor` bumps for more significant changes to our scanners. In case of breaking changes imposed by the wrapped scanner, creating a new analyzer on a separate repository must be considered.
diff --git a/doc/development/sec/img/primary_identifier_changed_v15_6.png b/doc/development/sec/img/primary_identifier_changed_v15_6.png
new file mode 100644
index 00000000000..aa1a116c801
--- /dev/null
+++ b/doc/development/sec/img/primary_identifier_changed_v15_6.png
Binary files differ
diff --git a/doc/development/sec/index.md b/doc/development/sec/index.md
index c9805044e58..fc13c960451 100644
--- a/doc/development/sec/index.md
+++ b/doc/development/sec/index.md
@@ -64,7 +64,97 @@ After the data is available as a Report Artifact it can be processed by the GitL
Depending on the context, the security reports may be stored either in the database or stay as Report Artifacts for on-demand access.
+#### Security report ingestion overview
+
+For details on how GitLab processes the reports generated by the scanners, see
+[Security report ingestion overview](security_report_ingestion_overview.md).
+
## CI/CD template development
While CI/CD templates are the responsibility of the Verify section, many are critical to the Sec Section's feature usage.
If you are working with CI/CD templates, please read the [development guide for GitLab CI/CD templates](../cicd/templates.md).
+
+## Importance of the primary identifier
+
+Within analyzer JSON reports, the [`identifiers` field](../integrations/secure.md#identifiers) contains a collection of types and categories by which
+a vulnerability can be described (that is, a CWE family).
+
+The first item in the `identifiers` collection is known as the [primary identifier](../../user/application_security/terminology#primary-identifier),
+a critical component to both describing and tracking vulnerabilities.
+
+In most other cases, the `identifiers` collection is unordered, where the remaining secondary identifiers act as metadata for grouping vulnerabilities
+(see [Analyzer vulnerability translation](#analyzer-vulnerability-translation) below for the exception).
+
+Any time the primary identifier changes and a project pipeline is re-run, ingestion of the new report will “orphan” the previous DB record.
+Because our processing logic relies on generating a delta of two different vulnerabilities, it can end up looking rather confusing. For example:
+
+[!Screenshot of primary identifier mismatch in MR widget](img/primary_identifier_changed_v15_6.png)
+
+After being [merged](../integrations/secure.md#tracking-and-merging-vulnerabilities), the previous vulnerability is listed as "remediated" and the introduced as ["detected"](../../user/application_security/vulnerabilities/index.md#vulnerability-status-values).
+
+### Guiding principles for ensuring primary identifier stability
+
+- A primary identifier should never change unless we have a compelling reason.
+- Analyzer supporting vulnerability translation must include the legacy primary identifiers in a secondary position to prevent “orphaning” of results.
+- Beyond the primary identifier, the order of secondary identifiers does not matter.
+- The identifier is unique based on a combination of the `Type` and `Value` fields (see [identifier fingerprint](https://gitlab.com/gitlab-org/gitlab/-/blob/v15.5.1-ee/lib/gitlab/ci/reports/security/identifier.rb#L63)).
+- If we change the primary identifier, rolling back analyzers to previous versions will not fix the orphaned results. The data previously ingested into our database is an artifact of previous jobs with few ways of automating data migrations.
+
+### Analyzer vulnerability translation
+
+In the case of SAST's semgrep analyzer, there is a secondary identifier of particular importance: the identifier linking the report’s vulnerability
+to the legacy analyzer (that is, bandit or eslint).
+
+To [enable vulnerability translation](../../user/application_security/sast/analyzers.md#vulnerability-translation)
+the semgrep analyzer relies on a secondary identifier exactly matching the primary identifier of the legacy analyzer.
+
+For example, when [`eslint`](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) was previously used to generate vulnerability records,
+the [`semgrep`](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) analyzer must produce an identifier collection containing the
+original eslint primary identifier.
+
+Given the original `eslint` report:
+
+```json
+{
+ "version": "14.0.4",
+ "vulnerabilities": [
+ {
+ "identifiers": [
+ {
+ "type": "eslint_rule_id",
+ "name": "ESLint rule ID security/detect-eval-with-expression",
+ "value": "security/detect-eval-with-expression"
+ }
+ ]
+ }
+ ]
+}
+```
+
+The corresponding semgrep report must contain the `eslint_rule_id`:
+
+```json
+{
+ "version": "14.0.4",
+ "vulnerabilities": [
+ {
+ "identifiers": [
+ {
+ "type": "semgrep_id",
+ "name": "eslint.detect-eval-with-expression",
+ "value": "eslint.detect-eval-with-expression",
+ "url": "https://semgrep.dev/r/gitlab.eslint.detect-eval-with-expression"
+ },
+ {
+ "type": "eslint_rule_id",
+ "name": "ESLint rule ID security/detect-eval-with-expression",
+ "value": "security/detect-eval-with-expression"
+ }
+ ]
+ }
+ ]
+}
+```
+
+[Tracking of vulnerabilities](../integrations/secure.md#tracking-and-merging-vulnerabilities) relies on a combination of the two identifiers
+to remap DB records previously generated with the legacy analyzers to those generated with the new `semgrep` ones.
diff --git a/doc/development/sec/security_report_ingestion_overview.md b/doc/development/sec/security_report_ingestion_overview.md
new file mode 100644
index 00000000000..c1d977c2a17
--- /dev/null
+++ b/doc/development/sec/security_report_ingestion_overview.md
@@ -0,0 +1,74 @@
+---
+stage: Secure
+group: Threat Insights
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+type: concepts
+---
+
+# Security report ingestion overview
+
+## Definitions
+
+- **Vulnerability Finding** – an instance of `Vulnerabilities::Finding` class. This class was previously called `Vulnerabilities::Occurrence`; after renaming the class, we kept the associated table name `vulnerability_occurrences` due to the effort involved in renaming large tables.
+- **Vulnerability** – an instance of `Vulnerability` class. They are created based on information available in `Vulnerabilities::Finding` class. Every `Vulnerability` **must have** a corresponding `Vulnerabilities::Finding` object to be valid, however this is not enforced at the database level.
+- **Security Finding** – an instance of `Security::Finding` class. They store **partial** finding data to improve performance of the pipeline security report. We are working on extending this class to store almost all required information so we can stop relying on job artifacts.
+- **Feedback** – an instance of `Vulnerabilities::Feedback` class. They are created to keep track of users' interactions with Vulnerability Findings before they are promoted to a Vulnerability. We are in the process of removing this model via [Deprecate and remove Vulnerabilities::Feedback epic](https://gitlab.com/groups/gitlab-org/-/epics/5629).
+- **Issue Link** – an instance of `Vulnerabilities::IssueLink` class. They are used to link `Vulnerability` objects to `Issue` objects.
+
+## Vulnerability creation from security reports
+
+Assumptions:
+
+- Project uses GitLab CI
+- Project uses [security scanning tools](../../user/application_security)
+- No Vulnerabilities are present in the database
+- All pipelines perform security scans
+
+1. Code is pushed to a branch that's **not** the default branch.
+1. GitLab CI runs a new pipeline for that branch.
+1. Pipeline status transitions to any of [`::Ci::Pipeline.completed_statuses`](https://gitlab.com/gitlab-org/gitlab/-/blob/354261b2fe4fc5b86d1408467beadd90e466ce0a/app/models/concerns/ci/has_status.rb#L12).
+1. `Security::StoreScansWorker` is called and it schedules `Security::StoreScansService`.
+1. `Security::StoreScansService` calls `Security::StoreGroupedScansService`.
+1. `Security::StoreGroupedScansService` calls `Security::StoreScanService`.
+1. `Security::StoreScanService` calls `Security::StoreFindingsService`.
+1. At this point we have `Security::Finding` objects **only**.
+
+At this point, the following things can happen to the `Security::Finding`:
+
+- Dismissal
+- Issue creation
+- Promotion to a Vulnerability
+
+If the pipeline ran on the default branch then the following, additional steps are done:
+
+1. `Security::StoreScansService` gets called and schedules `Security::StoreSecurityReportsWorker`.
+1. `Security::StoreSecurityReportsWorker` executes `Security::Ingestion::IngestReportsService`.
+1. `Security::Ingestion::IngestReportsService` takes all reports from a given Pipeline and calls `Security::Ingestion::IngestReportService` and then calls `Security::Ingestion::MarkAsResolvedService`.
+1. `Security::Ingestion::IngestReportService` calls `Security::Ingestion::IngestReportSliceService` which executes a number of tasks for a report slice.
+
+### Dismissal
+
+If you select `Dismiss vulnerability`, a Feedback is created. You can also dismiss it with a comment.
+
+#### After Feedback removal
+
+If there is only a Security Finding, a Vulnerability Finding and a Vulnerability get created. At the same time we create a `Vulnerabilities::StateTransition` record to indicate the Vulnerability was dismissed.
+
+### Issue creation
+
+If you select `Create issue`, a Vulnerabilities::Feedback record is created as well. The Feedback has a different `feedback_type` and an `issue_id` that’s not `NULL`.
+
+NOTE:
+Vulnerabilities::Feedback are in the process of being [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/5629). This will later create a `Vulnerabilities::IssueLink` record.
+
+#### After Feedback removal
+
+If there's only a Security Finding, a Vulnerability Finding and a Vulnerability gets created. At the same time, we create an Issue and a Issue Link.
+
+## Promotion to a Vulnerability
+
+If the branch with a Security Finding gets merged into the default branch, all Security Findings get promoted into Vulnerabilities. Promotion is the process of creating Vulnerability Findings and Vulnerability records from those Security Findings.
+
+If there's a dismissal Feedback present for that Security Finding, the created Vulnerability is marked as dismissed.
+
+If there's an issue Feedback present for that Security Finding, we also create an Issue Link for that Vulnerability.
diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md
index 67f7c556055..c102e99720f 100644
--- a/doc/development/secure_coding_guidelines.md
+++ b/doc/development/secure_coding_guidelines.md
@@ -856,7 +856,7 @@ Working with archive files like `zip`, `tar`, `jar`, `war`, `cpio`, `apk`, `rar`
### Zip Slip
-In 2018, the security company Snyk [released a blog post](https://snyk.io/research/zip-slip-vulnerability) describing research into a widespread and critical vulnerability present in many libraries and applications which allows an attacker to overwrite arbitrary files on the server file system which, in many cases, can be leveraged to achieve remote code execution. The vulnerability was dubbed Zip Slip.
+In 2018, the security company Snyk [released a blog post](https://security.snyk.io/research/zip-slip-vulnerability) describing research into a widespread and critical vulnerability present in many libraries and applications which allows an attacker to overwrite arbitrary files on the server file system which, in many cases, can be leveraged to achieve remote code execution. The vulnerability was dubbed Zip Slip.
A Zip Slip vulnerability happens when an application extracts an archive without validating and sanitizing the filenames inside the archive for directory traversal sequences that change the file location when the file is extracted.
@@ -1218,7 +1218,7 @@ GitLab-specific example can be found in [this issue](https://gitlab.com/gitlab-o
**Example 3:** you need to fetch a remote file, and perform a `HEAD` request to get and validate the content length and content type. When you subsequently make a `GET` request, though, the file delivered is a different size or different file type. (This is stretching the definition of TOCTOU, but things _have_ changed between time of check and time of use).
-**Example 4:** you allow users to upvote a comment if they haven't already. The server is multi-threaded, and you aren't using transactions or an applicable database index. By repeatedly clicking upvote in quick succession a malicious user is able to add multiple upvotes: the requests arrive at the same time, the checks run in parallel and confirm that no upvote exists yet, and so each upvote is written to the database.
+**Example 4:** you allow users to upvote a comment if they haven't already. The server is multi-threaded, and you aren't using transactions or an applicable database index. By repeatedly selecting upvote in quick succession a malicious user is able to add multiple upvotes: the requests arrive at the same time, the checks run in parallel and confirm that no upvote exists yet, and so each upvote is written to the database.
Here's some pseudocode showing an example of a potential TOCTOU bug:
@@ -1271,7 +1271,7 @@ This sensitive data must be handled carefully to avoid leaks which could lead to
- The [Gitleaks Git hook](https://gitlab.com/gitlab-com/gl-security/security-research/gitleaks-endpoint-installer) is recommended for preventing credentials from being committed.
- Never log credentials under any circumstance. Issue [#353857](https://gitlab.com/gitlab-org/gitlab/-/issues/353857) is an example of credential leaks through log file.
- When credentials are required in a CI/CD job, use [masked variables](../ci/variables/index.md#mask-a-cicd-variable) to help prevent accidental exposure in the job logs. Be aware that when [debug logging](../ci/variables/index.md#debug-logging) is enabled, all masked CI/CD variables are visible in job logs. Also consider using [protected variables](../ci/variables/index.md#protected-cicd-variables) when possible so that sensitive CI/CD variables are only available to pipelines running on protected branches or protected tags.
-- Proper scanners must be enabled depending on what data those credentials are protecting. See the [Application Security Inventory Policy](https://about.gitlab.com/handbook/engineering/security/security-engineering-and-research/application-security/inventory.html#policies) and our [Data Classification Standards](https://about.gitlab.com/handbook/engineering/security/data-classification-standard.html#data-classification-standards).
+- Proper scanners must be enabled depending on what data those credentials are protecting. See the [Application Security Inventory Policy](https://about.gitlab.com/handbook/security/security-engineering-and-research/application-security/inventory.html#policies) and our [Data Classification Standards](https://about.gitlab.com/handbook/security/data-classification-standard.html#data-classification-standards).
- To store and/or share credentials between teams, refer to [1Password for Teams](https://about.gitlab.com/handbook/security/#1password-for-teams) and follow [the 1Password Guidelines](https://about.gitlab.com/handbook/security/#1password-guidelines).
- If you need to share a secret with a team member, use 1Password. Do not share a secret over email, Slack, or other service on the Internet.
@@ -1281,7 +1281,7 @@ This sensitive data must be handled carefully to avoid leaks which could lead to
- Avoid including credentials as part of an HTTP response unless it is absolutely necessary as part of the workflow. For example, generating a PAT for users.
- Avoid sending credentials in URL parameters, as these can be more easily logged inadvertently during transit.
-In the event of credential leak through an MR, issue, or any other medium, [reach out to SIRT team](https://about.gitlab.com/handbook/engineering/security/security-operations/sirt/#-engaging-sirt).
+In the event of credential leak through an MR, issue, or any other medium, [reach out to SIRT team](https://about.gitlab.com/handbook/security/security-operations/sirt/#-engaging-sirt).
## Serialization
diff --git a/doc/development/service_ping/implement.md b/doc/development/service_ping/implement.md
index 88a1454393f..5a564b2d83e 100644
--- a/doc/development/service_ping/implement.md
+++ b/doc/development/service_ping/implement.md
@@ -231,13 +231,6 @@ Examples:
estimate_batch_distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::Note.minimum(:id), finish: ::Note.maximum(:id))
```
-1. Execution of estimated batch counter with joined relation (`joins(:cluster)`),
- for a custom column (`'clusters.user_id'`):
-
- ```ruby
- estimate_batch_distinct_count(::Clusters::Applications::CertManager.where(time_period).available.joins(:cluster), 'clusters.user_id')
- ```
-
When instrumenting metric with usage of estimated batch counter please add
`_estimated` suffix to its name, for example:
@@ -266,7 +259,7 @@ Arguments:
#### Ordinary Redis counters
-Example of implementation: [`Gitlab::UsageDataCounters::WikiPageCounter`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/wiki_page_counter.rb), using Redis methods [`INCR`](https://redis.io/commands/incr) and [`GET`](https://redis.io/commands/get).
+Example of implementation: [`Gitlab::UsageDataCounters::WikiPageCounter`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/wiki_page_counter.rb), using Redis methods [`INCR`](https://redis.io/commands/incr/) and [`GET`](https://redis.io/commands/get/).
Events are handled by counter classes in the `Gitlab::UsageDataCounters` namespace, inheriting from `BaseCounter`, that are either:
@@ -813,7 +806,7 @@ and run a local container instance:
1. On your local machine, make sure you are signed in to the GitLab Docker registry. You can find the instructions for this in
[Authenticate to the GitLab Container Registry](../../user/packages/container_registry/index.md#authenticate-with-the-container-registry).
1. Once signed in, download the new image by using `docker pull registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<VERSION>`
-1. For more information about working with and running Omnibus GitLab containers in Docker, refer to [GitLab Docker images](https://docs.gitlab.com/omnibus/docker/README.html) in the Omnibus documentation.
+1. For more information about working with and running Omnibus GitLab containers in Docker, refer to [GitLab Docker images](../../install/docker.md) documentation.
### Test with GitLab development toolkits
@@ -849,105 +842,6 @@ You can then count each user that performed any combination of these actions.
To add data for aggregated metrics to the Service Ping payload,
create metric YAML definition file following [Aggregated metric instrumentation guide](metrics_instrumentation.md#aggregated-metrics).
-### (DEPRECATED) Defining aggregated metric via aggregated metric YAML config file
-
-WARNING:
-This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98206) in GitLab 15.5
-and is planned for removal in 15.5. Use [metrics definition YAMLs](https://gitlab.com/gitlab-org/gitlab/-/issues/370963) instead.
-
-To add data for aggregated metrics to the Service Ping payload, add a corresponding definition to:
-
-- [`config/metrics/aggregates/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/aggregates/) for metrics available in the Community Edition.
-- [`ee/config/metrics/aggregates/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/metrics/aggregates/) for metrics available in the Enterprise Edition.
-
-Each aggregate definition includes following parts:
-
-- `name`: Unique name under which the aggregate metric is added to the Service Ping payload.
-- `operator`: Operator that defines how the aggregated metric data is counted. Available operators are:
- - `OR`: Removes duplicates and counts all entries that triggered any of listed events.
- - `AND`: Removes duplicates and counts all elements that were observed triggering all of following events.
-- `time_frame`: One or more valid time frames. Use these to limit the data included in aggregated metric to events within a specific date-range. Valid time frames are:
- - `7d`: Last seven days of data.
- - `28d`: Last twenty eight days of data.
- - `all`: All historical data, only available for `database` sourced aggregated metrics.
-- `source`: Data source used to collect all events data included in aggregated metric. Valid data sources are:
- - [`database`](#database-sourced-aggregated-metrics)
- - [`redis`](#redis-sourced-aggregated-metrics)
-- `events`: list of events names to aggregate into metric. All events in this list must
- relay on the same data source. Additional data source requirements are described in the
- [Database sourced aggregated metrics](#database-sourced-aggregated-metrics) and
- [Redis sourced aggregated metrics](#redis-sourced-aggregated-metrics) sections.
-- `feature_flag`: Name of [development feature flag](../feature_flags/index.md#development-type)
- that is checked before metrics aggregation is performed. Corresponding feature flag
- should have `default_enabled` attribute set to `false`. The `feature_flag` attribute
- is optional and can be omitted. When `feature_flag` is missing, no feature flag is checked.
-
-Example aggregated metric entries:
-
-```yaml
-- name: example_metrics_union
- operator: OR
- events:
- - 'users_expanding_secure_security_report'
- - 'users_expanding_testing_code_quality_report'
- - 'users_expanding_testing_accessibility_report'
- source: redis
- time_frame:
- - 7d
- - 28d
-- name: example_metrics_intersection
- operator: AND
- source: database
- time_frame:
- - 28d
- - all
- events:
- - 'dependency_scanning_pipeline_all_time'
- - 'container_scanning_pipeline_all_time'
- feature_flag: example_aggregated_metric
-```
-
-Aggregated metrics collected in `7d` and `28d` time frames are added into Service Ping payload under the `aggregated_metrics` sub-key in the `counts_weekly` and `counts_monthly` top level keys.
-
-```ruby
-{
- :counts_monthly => {
- :deployments => 1003,
- :successful_deployments => 78,
- :failed_deployments => 275,
- :packages => 155,
- :personal_snippets => 2106,
- :project_snippets => 407,
- :aggregated_metrics => {
- :example_metrics_union => 7,
- :example_metrics_intersection => 2
- },
- :snippets => 2513
- }
-}
-```
-
-Aggregated metrics for `all` time frame are present in the `count` top level key, with the `aggregate_` prefix added to their name.
-
-For example:
-
-`example_metrics_intersection`
-
-Becomes:
-
-`counts.aggregate_example_metrics_intersection`
-
-```ruby
-{
- :counts => {
- :deployments => 11003,
- :successful_deployments => 178,
- :failed_deployments => 1275,
- :aggregate_example_metrics_intersection => 12
- }
-}
-```
-
### Redis sourced aggregated metrics
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/45979) in GitLab 13.6.
@@ -963,9 +857,7 @@ you must fulfill the following requirements:
### Database sourced aggregated metrics
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52784) in GitLab 13.9.
-> - It's [deployed behind a feature flag](../../user/feature_flags.md), disabled by default.
-> - It's enabled on GitLab.com.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52784) in GitLab 13.9.
To declare an aggregate of metrics based on events collected from database, follow
these steps:
@@ -1018,25 +910,9 @@ end
#### Add new aggregated metric definition
-After all metrics are persisted, you can add an aggregated metric definition at
-[`aggregated_metrics/`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/aggregates/).
-
+After all metrics are persisted, you can add an aggregated metric definition following [Aggregated metric instrumentation guide](metrics_instrumentation.md#aggregated-metrics).
To declare the aggregate of metrics collected with [Estimated Batch Counters](#estimated-batch-counters),
you must fulfill the following requirements:
- Metrics names listed in the `events:` attribute, have to use the same names you passed in the `metric_name` argument while persisting metrics in previous step.
- Every metric listed in the `events:` attribute, has to be persisted for **every** selected `time_frame:` value.
-
-Example definition:
-
-```yaml
-- name: example_metrics_intersection_database_sourced
- operator: AND
- source: database
- events:
- - 'dependency_scanning_pipeline'
- - 'container_scanning_pipeline'
- time_frame:
- - 28d
- - all
-```
diff --git a/doc/development/service_ping/index.md b/doc/development/service_ping/index.md
index 0a4e5998345..37e0b753448 100644
--- a/doc/development/service_ping/index.md
+++ b/doc/development/service_ping/index.md
@@ -115,7 +115,7 @@ sequenceDiagram
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/295289) in GitLab 15.2. [Feature flag `measure_service_ping_metric_collection`](https://gitlab.com/gitlab-org/gitlab/-/issues/358128) removed.
```ruby
- {
+ {
"metadata"=>
{
"uuid"=>"0000000-0000-0000-0000-000000000000",
@@ -504,7 +504,7 @@ Service Ping reporting process state is monitored with [internal SiSense dashboa
- [Product Intelligence Guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)
- [Snowplow Guide](../snowplow/index.md)
-- [Product Intelligence Direction](https://about.gitlab.com/direction/product-intelligence/)
+- [Product Intelligence Direction](https://about.gitlab.com/direction/analytics/product-intelligence/)
- [Data Analysis Process](https://about.gitlab.com/handbook/business-technology/data-team/#data-analysis-process/)
- [Data for Product Managers](https://about.gitlab.com/handbook/business-technology/data-team/programs/data-for-product-managers/)
- [Data Infrastructure](https://about.gitlab.com/handbook/business-technology/data-team/platform/infrastructure/)
diff --git a/doc/development/service_ping/metrics_instrumentation.md b/doc/development/service_ping/metrics_instrumentation.md
index 7a3f291460c..a9f236819fe 100644
--- a/doc/development/service_ping/metrics_instrumentation.md
+++ b/doc/development/service_ping/metrics_instrumentation.md
@@ -41,7 +41,7 @@ We have built a domain-specific language (DSL) to define the metrics instrumenta
You can use database metrics to track data kept in the database, for example, a count of issues that exist on a given instance.
- `operation`: Operations for the given `relation`, one of `count`, `distinct_count`, `sum`, and `average`.
-- `relation`: `ActiveRecord::Relation` for the objects we want to perform the `operation`.
+- `relation`: Assigns lambda that returns the `ActiveRecord::Relation` for the objects we want to perform the `operation`. The assigned lambda can accept up to one parameter. The parameter is hashed and stored under the `options` key in the metric definition.
- `start`: Specifies the start value of the batch counting, by default is `relation.minimum(:id)`.
- `finish`: Specifies the end value of the batch counting, by default is `relation.maximum(:id)`.
- `cache_start_and_finish_as`: Specifies the cache key for `start` and `finish` values and sets up caching them. Use this call when `start` and `finish` are expensive queries that should be reused between different metric calculations.
@@ -55,10 +55,10 @@ module Gitlab
module Usage
module Metrics
module Instrumentations
- class CountBoardsMetric < DatabaseMetric
+ class CountIssuesMetric < DatabaseMetric
operation :count
- relation { Board }
+ relation ->(options) { Issue.where(confidential: options[:confidential]) }
end
end
end
@@ -156,7 +156,7 @@ You can use Redis metrics to track events not kept in the database, for example,
[Example of a merge request that adds a `Redis` metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97009).
-Please note that `RedisMetric` class can only be used as the `instrumentation_class` for Redis metrics with simple counters classes (classes that only inherit `BaseCounter` and set `PREFIX` and `KNOWN_EVENTS` constants). In case the counter class has additional logic included in it, a new `instrumentation_class`, inheriting from `RedisMetric`, needs to be created. This new class needs to include the additional logic from the counter class.
+The `RedisMetric` class can only be used as the `instrumentation_class` for Redis metrics with simple counters classes (classes that only inherit `BaseCounter` and set `PREFIX` and `KNOWN_EVENTS` constants). In case the counter class has additional logic included in it, a new `instrumentation_class`, inheriting from `RedisMetric`, needs to be created. This new class needs to include the additional logic from the counter class.
Count unique values for `source_code_pushes` event.
@@ -256,6 +256,13 @@ options:
## Aggregated metrics
+<div class="video-fallback">
+ See the video from: <a href="https://www.youtube.com/embed/22LbYqHwtUQ">Product Intelligence Office Hours Oct 6th</a> for an aggregated metrics walk-through.
+</div>
+<figure class="video-container">
+ <iframe src="https://www.youtube.com/embed/22LbYqHwtUQ" frameborder="0" allowfullscreen="true"> </iframe>
+</figure>
+
The aggregated metrics feature provides insight into the number of data attributes, for example `pseudonymized_user_ids`, that occurred in a collection of events. For example, you can aggregate the number of users who perform multiple actions such as creating a new issue and opening
a new merge request.
diff --git a/doc/development/service_ping/metrics_lifecycle.md b/doc/development/service_ping/metrics_lifecycle.md
index f13aebeb16e..92c5d2d317d 100644
--- a/doc/development/service_ping/metrics_lifecycle.md
+++ b/doc/development/service_ping/metrics_lifecycle.md
@@ -25,6 +25,10 @@ Any such changes lead to inconsistent reports from multiple GitLab instances.
If there is a problem with an existing metric, it's best to deprecate the existing metric,
and use it, side by side, with the desired new metric.
+If you do need to change a metric, please notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) teams by `@` mentioning those groups in a comment on the MR.
+Many Service Ping metrics are relied upon for health score and XMAU reporting and
+unexpected changes to those metrics could break reporting.
+
Example:
Consider following change. Before GitLab 12.6, the `example_metric` was implemented as:
@@ -135,3 +139,6 @@ To remove a metric:
1. Remove any other records related to the metric:
- The feature flag YAML file at [`config/feature_flags/*/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/feature_flags).
- The entry in the known events YAML file at [`lib/gitlab/usage_data_counters/known_events/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/usage_data_counters/known_events).
+
+1. Notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) by `@` mentioning those groups in a comment on the MR.
+ Many Service Ping metrics are relied upon for health score and XMAU reporting and unexpected changes to those metrics could break reporting.
diff --git a/doc/development/service_ping/review_guidelines.md b/doc/development/service_ping/review_guidelines.md
index a1806551303..70f7f3dca54 100644
--- a/doc/development/service_ping/review_guidelines.md
+++ b/doc/development/service_ping/review_guidelines.md
@@ -68,6 +68,7 @@ are regular backend changes.
Read the [stages file](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml).
- Check the file location. Consider the time frame, and if the file should be under `ee`.
- Check the tiers.
+- If a metric was changed or removed: Make sure the MR author notified the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) by `@` mentioning those groups in a comment on the MR.
- Metrics instrumentations
- Recommend using metrics instrumentation for new metrics, [if possible](metrics_instrumentation.md#support-for-instrumentation-classes).
- Approve the MR, and relabel the MR with `~"product intelligence::approved"`.
diff --git a/doc/development/service_ping/troubleshooting.md b/doc/development/service_ping/troubleshooting.md
index 201d6a385eb..f8fd45e6062 100644
--- a/doc/development/service_ping/troubleshooting.md
+++ b/doc/development/service_ping/troubleshooting.md
@@ -120,3 +120,45 @@ To work around this bug, you have two options:
1. Expand **Usage Statistics**.
1. Clear the **Enable Service Ping** checkbox.
1. Select **Save Changes**.
+
+## Generate Service Ping
+
+### Generate or get the cached Service Ping in rails console
+
+Use the following method in the [rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session).
+
+```ruby
+Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values, cached: true)
+```
+
+### Generate a fresh new Service Ping
+
+Use the following method in the [rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session).
+
+This also refreshes the cached Service Ping displayed in the Admin Area.
+
+```ruby
+Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values)
+```
+
+### Generate and print
+
+Generates Service Ping data in JSON format.
+
+```shell
+gitlab-rake gitlab:usage_data:generate
+```
+
+Generates Service Ping data in YAML format:
+
+```shell
+gitlab-rake gitlab:usage_data:dump_sql_in_yaml
+```
+
+### Generate and send Service Ping
+
+Prints the metrics saved in `conversational_development_index_metrics`.
+
+```shell
+gitlab-rake gitlab:usage_data:generate_and_send
+```
diff --git a/doc/development/service_ping/usage_data.md b/doc/development/service_ping/usage_data.md
index 2f2df14f56d..1c7a212ed64 100644
--- a/doc/development/service_ping/usage_data.md
+++ b/doc/development/service_ping/usage_data.md
@@ -66,5 +66,4 @@ Examples:
```ruby
distinct_count(::Project, :creator_id)
distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id))
-distinct_count(::Clusters::Applications::CertManager.where(time_period).available.joins(:cluster), 'clusters.user_id')
```
diff --git a/doc/development/shell_commands.md b/doc/development/shell_commands.md
index 3935e98199a..d78a005d76b 100644
--- a/doc/development/shell_commands.md
+++ b/doc/development/shell_commands.md
@@ -71,6 +71,8 @@ FileUtils.touch myfile
This coding style could have prevented CVE-2013-4546.
+See also <https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93030>, and <https://starlabs.sg/blog/2022/07-gitlab-project-import-rce-analysis-cve-2022-2185/> for another example.
+
## Separate options from arguments with --
Make the difference between options and arguments clear to the argument parsers of system commands with `--`. This is supported by many but not all Unix commands.
diff --git a/doc/development/sidekiq/index.md b/doc/development/sidekiq/index.md
index e8c939571cf..a95e94cdd34 100644
--- a/doc/development/sidekiq/index.md
+++ b/doc/development/sidekiq/index.md
@@ -36,6 +36,31 @@ with back-off between each retry. 25 retries means that the last retry
would happen around three weeks after the first attempt (assuming all 24
prior retries failed).
+This means that a lot can happen in between the job being scheduled
+and its execution. Therefore, we must guard workers so they don't
+fail 25 times when the state changes after they are scheduled. For
+example, a job should not fail when the project it was scheduled for
+is deleted.
+
+Instead of:
+
+```ruby
+def perform(project_id)
+ project = Project.find(project_id)
+ # ...
+end
+```
+
+Do this:
+
+```ruby
+def perform(project_id)
+ project = Project.find_by_id(project_id)
+ return unless project
+ # ...
+end
+```
+
For most workers - especially [idempotent workers](idempotent_jobs.md) -
the default of 25 retries is more than sufficient. Many of our older
workers declare 3 retries, which used to be the default within the
diff --git a/doc/development/sidekiq/worker_attributes.md b/doc/development/sidekiq/worker_attributes.md
index 48a222d65a0..4fcd8e33d5c 100644
--- a/doc/development/sidekiq/worker_attributes.md
+++ b/doc/development/sidekiq/worker_attributes.md
@@ -277,7 +277,7 @@ they prefer read replicas and will wait for replicas to catch up:
| **Data Consistency** | **Description** |
|--------------|-----------------------------|
-| `:always` | The job is required to use the primary database (default). It should be used for workers that primarily perform writes or that have strict requirements around data consistency when reading their own writes. |
+| `:always` | The job is required to use the primary database (default). It should be used for workers that primarily perform writes, have strict requirements around data consistency when reading their own writes, or are cron jobs. |
| `:sticky` | The job prefers replicas, but switches to the primary for writes or when encountering replication lag. It should be used for jobs that require to be executed as fast as possible but can sustain a small initial queuing delay. |
| `:delayed` | The job prefers replicas, but switches to the primary for writes. When encountering replication lag before the job starts, the job is retried once. If the replica is still not up to date on the next retry, it switches to the primary. It should be used for jobs where delaying execution further typically does not matter, such as cache expiration or web hooks execution. |
diff --git a/doc/development/snowplow/event_dictionary_guide.md b/doc/development/snowplow/event_dictionary_guide.md
index c5a21f5a081..794a9a0160c 100644
--- a/doc/development/snowplow/event_dictionary_guide.md
+++ b/doc/development/snowplow/event_dictionary_guide.md
@@ -22,24 +22,24 @@ All event definitions are stored in the following directories:
Each event is defined in a separate YAML file consisting of the following fields:
-| Field | Required | Additional information |
-|------------------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `description` | yes | A description of the event. |
-| `category` | yes | The event category (see [Structured event taxonomy](index.md#structured-event-taxonomy)). |
-| `action` | yes | The event action (see [Structured event taxonomy](index.md#structured-event-taxonomy)). |
-| `label_description` | no | A description of the event label (see [Structured event taxonomy](index.md#structured-event-taxonomy)). |
-| `property_description` | no | A description of the event property (see [Structured event taxonomy](index.md#structured-event-taxonomy)). |
-| `value_description` | no | A description of the event value (see [Structured event taxonomy](index.md#structured-event-taxonomy)). |
-| `extra_properties` | no | The type and description of each extra property sent with the event. |
-| `identifiers` | no | A list of identifiers sent with the event. Can be set to one or more of `project`, `user`, or `namespace`. |
-| `iglu_schema_url` | no | The URL to the custom schema sent with the event, for example, `iglu:com.gitlab/gitlab_experiment/jsonschema/1-0-0`. |
-| `product_section` | yes | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml). |
-| `product_stage` | no | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the event. |
-| `product_group` | yes | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the event. |
-| `product_category` | no | The [product category](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/categories.yml) for the event. |
-| `milestone` | no | The milestone when the event is introduced. |
-| `introduced_by_url` | no | The URL to the merge request that introduced the event. |
-| `distributions` | yes | The [distributions](https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/#definitions) where the tracked feature is available. Can be set to one or more of `ce` or `ee`. |
+| Field | Required | Additional information |
+|------------------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `description` | yes | A description of the event. |
+| `category` | yes | The event category (see [Event schema](index.md#event-schema)). |
+| `action` | yes | The event action (see [Event schema](index.md#event-schema)). |
+| `label_description` | no | A description of the event label (see [Event schema](index.md#event-schema)). |
+| `property_description` | no | A description of the event property (see [Event schema](index.md#event-schema)). |
+| `value_description` | no | A description of the event value (see [Event schema](index.md#event-schema)). |
+| `extra_properties` | no | The type and description of each extra property sent with the event. |
+| `identifiers` | no | A list of identifiers sent with the event. Can be set to one or more of `project`, `user`, or `namespace`. |
+| `iglu_schema_url` | no | The URL to the custom schema sent with the event, for example, `iglu:com.gitlab/gitlab_experiment/jsonschema/1-0-0`. |
+| `product_section` | yes | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml). |
+| `product_stage` | no | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the event. |
+| `product_group` | yes | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the event. |
+| `product_category` | no | The [product category](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/categories.yml) for the event. |
+| `milestone` | no | The milestone when the event is introduced. |
+| `introduced_by_url` | no | The URL to the merge request that introduced the event. |
+| `distributions` | yes | The [distributions](https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/#definitions) where the tracked feature is available. Can be set to one or more of `ce` or `ee`. |
| `tiers` | yes | The [tiers]( https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/) where the tracked feature is available. Can be set to one or more of `free`, `premium`, or `ultimate`. |
### Example event definition
diff --git a/doc/development/snowplow/implementation.md b/doc/development/snowplow/implementation.md
index 26016eeba84..a80d6fe70ff 100644
--- a/doc/development/snowplow/implementation.md
+++ b/doc/development/snowplow/implementation.md
@@ -13,7 +13,7 @@ This page describes how to:
## Snowplow JavaScript frontend tracking
-GitLab provides a `Tracking` interface that wraps the [Snowplow JavaScript tracker](https://docs.snowplowanalytics.com/docs/collecting-data/collecting-from-own-applications/javascript-trackers/)
+GitLab provides a `Tracking` interface that wraps the [Snowplow JavaScript tracker](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/)
to track custom events.
For the recommended frontend tracking implementation, see [Usage recommendations](#usage-recommendations).
@@ -42,14 +42,14 @@ and the custom `extra` object. You can modify this object for any subsequent
structured event that fires, although this is not recommended.
Tracking implementations must have an `action` and a `category`. You can provide additional
-properties from the [structured event taxonomy](index.md#structured-event-taxonomy), in
+properties from the [event schema](index.md#event-schema), in
addition to an `extra` object that accepts key-value pairs.
-| Property | Type | Default value | Description |
-|:-----------|:-------|:---------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `category` | string | `document.body.dataset.page` | Page or subsection of a page in which events are captured. |
+| Property | Type | Default value | Description |
+|:-----------|:-------|:---------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `category` | string | `document.body.dataset.page` | Page or subsection of a page in which events are captured. |
| `action` | string | `'generic'` | Action the user is taking. Clicks must be `click` and activations must be `activate`. For example, focusing a form field is `activate_form_input`, and clicking a button is `click_button`. |
-| `data` | object | `{}` | Additional data such as `label`, `property`, `value` as described in [Structured event taxonomy](index.md#structured-event-taxonomy), `context` for custom contexts, and `extra` (key-value pairs object). |
+| `data` | object | `{}` | Additional data such as `label`, `property`, `value` as described in [Event schema](index.md#event-schema), `context` for custom contexts, and `extra` (key-value pairs object). |
### Usage recommendations
@@ -81,7 +81,7 @@ The following example shows `data-track-*` attributes assigned to a button:
| Attribute | Required | Description |
|:----------------------|:---------|:------------|
| `data-track-action` | true | Action the user is taking. Clicks must be prepended with `click` and activations must be prepended with `activate`. For example, focusing a form field is `activate_form_input` and clicking a button is `click_button`. Replaces `data-track-event`, which was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/290962) in GitLab 13.11. |
-| `data-track-label` | false | The specific element or object to act on. This can be: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown in the top bar; or the name or title attribute of a record being created. |
+| `data-track-label` | false | The specific element or object to act on. This can be: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown list in the top bar; or the name or title attribute of a record being created. |
| `data-track-property` | false | Any additional property of the element, or object being acted on. |
| `data-track-value` | false | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. If omitted, this is the element's `value` property or `undefined`. For checkboxes, the default value is the element's checked attribute or `0` when unchecked. The value is parsed as numeric before sending the event. |
| `data-track-extra` | false | A key-value pair object passed as a valid JSON string. This attribute is added to the `extra` property in our [`gitlab_standard`](schemas.md#gitlab_standard) schema. |
@@ -349,7 +349,7 @@ describe('MyTracking', () => {
### Form tracking
-To enable Snowplow automatic [form tracking](https://docs.snowplowanalytics.com/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracking-specific-events/#form-tracking):
+To enable Snowplow automatic [form tracking](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracking-specific-events/#form-tracking):
1. Call `Tracking.enableFormTracking` when the DOM is ready.
1. Provide a `config` object that includes at least one of the following elements:
@@ -389,7 +389,7 @@ describe('MyFormTracking', () => {
## Implement Ruby backend tracking
-`Gitlab::Tracking` is an interface that wraps the [Snowplow Ruby Tracker](https://docs.snowplowanalytics.com/docs/collecting-data/collecting-from-own-applications/ruby-tracker/) for tracking custom events.
+`Gitlab::Tracking` is an interface that wraps the [Snowplow Ruby Tracker](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/ruby-tracker/) for tracking custom events.
Backend tracking provides:
- User behavior tracking
@@ -415,7 +415,7 @@ Use the following arguments:
|------------|---------------------------|---------------|-----------------------------------------------------------------------------------------------------------------------------------|
| `category` | String | | Area or aspect of the application. For example, `HealthCheckController` or `Lfs::FileTransformer`. |
| `action` | String | | The action being taken. For example, a controller action such as `create`, or an Active Record callback. |
-| `label` | String | `nil` | The specific element or object to act on. This can be one of the following: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown in the top bar; or the name or title attribute of a record being created. |
+| `label` | String | `nil` | The specific element or object to act on. This can be one of the following: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown list in the top bar; or the name or title attribute of a record being created. |
| `property` | String | `nil` | Any additional property of the element, or object being acted on. |
| `value` | Numeric | `nil` | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. |
| `context` | Array\[SelfDescribingJSON\] | `nil` | An array of custom contexts to send with this event. Most events should not have any custom contexts. |
@@ -451,7 +451,7 @@ Before you test frontend events in development, you must:
1. Turn off ad blockers that could prevent Snowplow JavaScript from loading in your environment.
1. Turn off "Do Not Track" (DNT) in your browser.
-All URLs are pseudonymized. The entity identifier [replaces](https://docs.snowplowanalytics.com/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracker-setup/other-parameters-2/#Setting_a_custom_page_URL_and_referrer_URL) personally identifiable
+All URLs are pseudonymized. The entity identifier [replaces](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracker-setup/other-parameters-2/#setting-a-custom-page-url-and-referrer-url) personally identifiable
information (PII). PII includes usernames, group, and project names.
Page titles are hardcoded as `GitLab` for the same reason.
@@ -477,7 +477,7 @@ For a video tutorial, see the [Snowplow plugin walk through](https://www.youtube
#### Snowplow Micro
-[Snowplow Micro](https://snowplowanalytics.com/blog/2019/07/17/introducing-snowplow-micro/) is a
+[Snowplow Micro](https://snowplow.io/blog/introducing-snowplow-micro/) is a
Docker-based solution for testing backend and frontend in a local development environment. Snowplow Micro
records the same events as the full Snowplow pipeline. To query events, use the Snowplow Micro API.
diff --git a/doc/development/snowplow/index.md b/doc/development/snowplow/index.md
index 7327f18fcaf..8d05e05e592 100644
--- a/doc/development/snowplow/index.md
+++ b/doc/development/snowplow/index.md
@@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
Snowplow is an enterprise-grade marketing and Product Intelligence platform that tracks how users engage with our website and application.
-[Snowplow](https://snowplowanalytics.com) consists of several loosely-coupled sub-systems:
+[Snowplow](https://snowplow.io/) consists of several loosely-coupled sub-systems:
- **Trackers** fire Snowplow events. Snowplow has twelve trackers that cover web, mobile, desktop, server, and IoT.
- **Collectors** receive Snowplow events from trackers. We use different event collectors that synchronize events to Amazon S3, Apache Kafka, or Amazon Kinesis.
@@ -80,20 +80,21 @@ sequenceDiagram
For more details about the architecture, see [Snowplow infrastructure](infrastructure.md).
-## Structured event taxonomy
+## Event schema
-Click events must be consistent. If each feature captures events differently, it can be difficult
+All the events must be consistent. If each feature captures events differently, it can be difficult
to perform analysis.
-Each click event provides attributes that describe the event.
+Each event provides attributes that describe the event.
| Attribute | Type | Required | Description |
| --------- | ------- | -------- | ----------- |
| category | text | true | The page or backend section of the application. Unless infeasible, use the Rails page attribute by default in the frontend, and namespace + class name on the backend, for example, `Notes::CreateService`. |
| action | text | true | The action the user takes, or aspect that's being instrumented. The first word must describe the action or aspect. For example, clicks must be `click`, activations must be `activate`, creations must be `create`. Use underscores to describe what was acted on. For example, activating a form field is `activate_form_input`, an interface action like clicking on a dropdown list is `click_dropdown`, a behavior like creating a project record from the backend is `create_project`. |
-| label | text | false | The specific element or object to act on. This can be one of the following: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown in the top bar; or the name or title attribute of a record being created. For Service Ping metrics adapted to Snowplow events, this should be the full metric [key path](../service_ping/metrics_dictionary.md#metric-key_path) taken from its definition file. |
+| label | text | false | The specific element or object to act on. This can be one of the following: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown list in the top bar; or the name or title attribute of a record being created. For Service Ping metrics adapted to Snowplow events, this should be the full metric [key path](../service_ping/metrics_dictionary.md#metric-key_path) taken from its definition file. |
| property | text | false | Any additional property of the element, or object being acted on. For Service Ping metrics adapted to Snowplow events, this should be additional information or context that can help analyze the event. For example, in the case of `usage_activity_by_stage_monthly.create.merge_requests_users`, there are four different possible merge request actions: "create", "merge", "comment", and "close". Each of these would be a possible property value. |
| value | decimal | false | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. |
+| context | vector | false | Additional data in the form of a [self-describing JSON](https://docs.snowplow.io/docs/pipeline-components-and-applications/iglu/common-architecture/self-describing-json-schemas/) to describe the event if the attributes are not sufficient. Each context must have its schema defined to assure data integrity. Refer to the list of GitLab-defined contexts for more details. |
### Examples
@@ -185,16 +186,16 @@ LIMIT 100
### Web-specific parameters
-Snowplow JavaScript adds [web-specific parameters](https://docs.snowplowanalytics.com/docs/collecting-data/collecting-from-own-applications/snowplow-tracker-protocol/#Web-specific_parameters) to all web events by default.
+Snowplow JavaScript adds [web-specific parameters](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/snowplow-tracker-protocol/#Web-specific_parameters) to all web events by default.
## Related topics
-- [Snowplow data structure](https://docs.snowplowanalytics.com/docs/understanding-your-pipeline/canonical-event/)
+- [Snowplow data structure](https://docs.snowplow.io/docs/understanding-your-pipeline/canonical-event/)
- [Our Iglu schema registry](https://gitlab.com/gitlab-org/iglu)
- [List of events used in our codebase (Event Dictionary)](https://metrics.gitlab.com/snowplow/)
- [Product Intelligence Guide](https://about.gitlab.com/handbook/product/product-intelligence-guide/)
- [Service Ping Guide](../service_ping/index.md)
-- [Product Intelligence Direction](https://about.gitlab.com/direction/product-intelligence/)
+- [Product Intelligence Direction](https://about.gitlab.com/direction/analytics/product-intelligence/)
- [Data Analysis Process](https://about.gitlab.com/handbook/business-technology/data-team/#data-analysis-process/)
- [Data for Product Managers](https://about.gitlab.com/handbook/business-technology/data-team/programs/data-for-product-managers/)
- [Data Infrastructure](https://about.gitlab.com/handbook/business-technology/data-team/platform/infrastructure/)
diff --git a/doc/development/snowplow/review_guidelines.md b/doc/development/snowplow/review_guidelines.md
index 756dbd0ca92..ee2c1eb95df 100644
--- a/doc/development/snowplow/review_guidelines.md
+++ b/doc/development/snowplow/review_guidelines.md
@@ -35,7 +35,7 @@ events or touches Snowplow related files.
#### The Product Intelligence **reviewer** should
-- Check that the [event taxonomy](index.md#structured-event-taxonomy) is correct.
+- Check that the [event schema](index.md#event-schema) is correct.
- Check the [usage recommendations](implementation.md#usage-recommendations).
- Check that the [Event Dictionary](event_dictionary_guide.md) is up-to-date.
- If needed, check that the events are firing locally using one of the
diff --git a/doc/development/snowplow/schemas.md b/doc/development/snowplow/schemas.md
index 482c0e0105b..5a6cdea9fee 100644
--- a/doc/development/snowplow/schemas.md
+++ b/doc/development/snowplow/schemas.md
@@ -33,8 +33,8 @@ _\* Default value present for frontend events only_
## Default Schema
-Frontend events include a [web-specific schema](https://docs.snowplowanalytics.com/docs/understanding-your-pipeline/canonical-event/#Web-specific_fields) provided by Snowplow.
-All URLs are pseudonymized. The entity identifier [replaces](https://docs.snowplowanalytics.com/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracker-setup/other-parameters-2/#Setting_a_custom_page_URL_and_referrer_URL) personally identifiable
+Frontend events include a [web-specific schema](https://docs.snowplow.io/docs/understanding-your-pipeline/canonical-event/#web-specific-fields) provided by Snowplow.
+All URLs are pseudonymized. The entity identifier [replaces](https://docs.snowplow.io/docs/collecting-data/collecting-from-own-applications/javascript-trackers/javascript-tracker/javascript-tracker-v2/tracker-setup/other-parameters-2/#setting-a-custom-page-url-and-referrer-url) personally identifiable
information (PII). PII includes usernames, group, and project names.
Page titles are hardcoded as `GitLab` for the same reason.
@@ -172,3 +172,18 @@ Page titles are hardcoded as `GitLab` for the same reason.
| `v_collector` | **{dotted-circle}** | string | Collector version |
| `v_etl` | **{dotted-circle}** | string | ETL version |
| `v_tracker` | **{dotted-circle}** | string | Identifier for Snowplow tracker |
+
+## `gitlab_service_ping`
+
+Backend events converted from ServicePing (`redis` and `redis_hll`) must include [ServicePing context](https://gitlab.com/gitlab-org/iglu/-/tree/master/public/schemas/com.gitlab/gitlab_service_ping/jsonschema)
+using the [helper class](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/tracking/service_ping_context.rb).
+
+An example of converted `redis_hll` [event with context](https://gitlab.com/gitlab-org/gitlab/-/edit/master/app/controllers/concerns/product_analytics_tracking.rb#L58).
+
+| Field Name | Required | Type | Description |
+|---------------|:-------------------:|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `data_source` | **{check-circle}** | string (max 64 chars) | The `data_source` attribute from the metrics YAML definition. |
+| `event_name`* | **{dotted-circle}** | string (max 128 chars) | When there is a many-to-many relationship between events and metrics, this field contains the name of a Redis event that can be used for aggregations in downstream systems |
+| `key_path`* | **{dotted-circle}** | string (max 256 chars) | The `key_path` attribute from the metrics YAML definition |
+
+_\* Either `event_name` or `key_path` is required_
diff --git a/doc/development/spam_protection_and_captcha/exploratory_testing.md b/doc/development/spam_protection_and_captcha/exploratory_testing.md
index e17d25acb9d..f6e3e6814a8 100644
--- a/doc/development/spam_protection_and_captcha/exploratory_testing.md
+++ b/doc/development/spam_protection_and_captcha/exploratory_testing.md
@@ -91,7 +91,7 @@ CAPTCHA response string does not matter. It can be any string. If you use a
real, valid key pair, you must solve the CAPTCHA to obtain a
valid CAPTCHA response to use. You can do this once only, and only before it expires.
-To directly test the GraphQL API via [GraphQL Explorer](http://gdk.test:3000/-/graphql-explorer),
+To directly test the GraphQL API via GraphQL Explorer (`http://gdk.test:3000/-/graphql-explorer`),
get a reCAPTCHA response string via this form: `public/recaptcha.html` (`http://gdk.test:3000/recaptcha.html`):
```html
diff --git a/doc/development/sql.md b/doc/development/sql.md
index 5829d27b8ee..cdc952eb08b 100644
--- a/doc/development/sql.md
+++ b/doc/development/sql.md
@@ -103,7 +103,7 @@ transaction. Transactions for migrations can be disabled using the following
pattern:
```ruby
-class MigrationName < Gitlab::Database::Migration[1.0]
+class MigrationName < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
end
```
@@ -111,7 +111,7 @@ end
For example:
```ruby
-class AddUsersLowerUsernameEmailIndexes < Gitlab::Database::Migration[1.0]
+class AddUsersLowerUsernameEmailIndexes < Gitlab::Database::Migration[2.0]
disable_ddl_transaction!
def up
diff --git a/doc/development/stage_group_observability/dashboards/index.md b/doc/development/stage_group_observability/dashboards/index.md
index 9f9ba609271..c2da46bde7d 100644
--- a/doc/development/stage_group_observability/dashboards/index.md
+++ b/doc/development/stage_group_observability/dashboards/index.md
@@ -43,7 +43,7 @@ All metrics recorded in the GitLab production system have
[one-year retention](https://gitlab.com/gitlab-cookbooks/gitlab-prometheus/-/blob/31526b03fef823e2f9b3cda7c75dcd28a12418a3/attributes/prometheus.rb#L40).
You can also zoom in and filter the time range directly on a graph. For more information, see the
-[Grafana Time Range Controls](https://grafana.com/docs/grafana/latest/dashboards/time-range-controls/)
+[Grafana Time Range Controls](https://grafana.com/docs/grafana/latest/dashboards/use-dashboards/#set-dashboard-time-range)
documentation.
## Filters and annotations
@@ -51,7 +51,7 @@ documentation.
On each dashboard, there are two filters and some annotation switches on the top of the page.
Some special events are meaningful to development and operational activities.
-[Grafana annotations](https://grafana.com/docs/grafana/latest/dashboards/annotations/) mark them
+[Grafana annotations](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/annotate-visualizations/) mark them
directly on the graphs.
![Filters and annotations](img/stage_group_dashboards_filters.png)
diff --git a/doc/development/stage_group_observability/dashboards/stage_group_dashboard.md b/doc/development/stage_group_observability/dashboards/stage_group_dashboard.md
index fb5d5bbe379..abc46c52407 100644
--- a/doc/development/stage_group_observability/dashboards/stage_group_dashboard.md
+++ b/doc/development/stage_group_observability/dashboards/stage_group_dashboard.md
@@ -56,7 +56,7 @@ description, note the following:
To inspect the raw data of the panel for further calculation, select **Inspect** from the dropdown list of a panel.
Queries, raw data, and panel JSON structure are available.
-Read more at [Grafana panel inspection](http://grafana.com/docs/grafana/next/panels/query-a-data-source/).
+Read more at [Grafana panel inspection](https://grafana.com/docs/grafana/latest/panels-visualizations/query-transform-data/).
All the dashboards are powered by [Grafana](https://grafana.com/), a frontend for displaying metrics.
Grafana consumes the data returned from queries to backend Prometheus data source, then presents it
diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md
index f1a23f06699..b6bf3c7805a 100644
--- a/doc/development/testing_guide/best_practices.md
+++ b/doc/development/testing_guide/best_practices.md
@@ -110,7 +110,7 @@ If the specs fail the check they must be fixed before than can run in random ord
### Test speed
-GitLab has a massive test suite that, without [parallelization](../pipelines.md#test-suite-parallelization), can take hours
+GitLab has a massive test suite that, without [parallelization](../pipelines/index.md#test-suite-parallelization), can take hours
to run. It's important that we make an effort to write tests that are accurate
and effective _as well as_ fast.
@@ -282,7 +282,7 @@ end
#### Stubbing methods within factories
-You should avoid using `allow(object).to receive(:method)` in factories, as this makes the factory unable to be used with `let_it_be`.
+You should avoid using `allow(object).to receive(:method)` in factories, as this makes the factory unable to be used with `let_it_be`, as described in [common test setup](#common-test-setup).
Instead, you can use `stub_method` to stub the method:
@@ -425,6 +425,11 @@ results are available, and not just the first failure.
when you need an ID/IID/access level that doesn't actually exists. Using 123, 1234,
or even 999 is brittle as these IDs could actually exist in the database in the
context of a CI run.
+- All top-level `RSpec.describe` blocks should have [`feature_category`](https://about.gitlab.com/categories.json) metadata set.
+ Consider splitting the file in the case there are identified multiple feature categories in same file.
+ If no `feature_category` is identified then use `not_owned`. This information is used in flaky test
+ issues created in order to identify the group owning the feature.
+ Eg: `RSpec.describe Admin::Geo::SettingsController, :geo, feature_category: :geo_replication do`.
### Coverage
@@ -921,7 +926,8 @@ them unspecified, and look up the value after the row is created.
#### Redis
GitLab stores two main categories of data in Redis: cached items, and Sidekiq
-jobs.
+jobs. [View the full list of `Gitlab::Redis::Wrapper` descendants](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/redis.rb) that are backed by
+a separate Redis instance.
In most specs, the Rails cache is actually an in-memory store. This is replaced
between specs, so calls to `Rails.cache.read` and `Rails.cache.write` are safe.
@@ -961,6 +967,14 @@ it "really connects to Prometheus", :permit_dns do
And if you need more specific control, the DNS blocking is implemented in
`spec/support/helpers/dns_helpers.rb` and these methods can be called elsewhere.
+#### Rate Limiting
+
+[Rate limiting](../../security/rate_limits.md) is enabled in the test suite. Rate limits
+may be triggered in feature specs that use the `:js` trait. In most cases, triggering rate
+limiting can be avoided by marking the spec with the `:clean_gitlab_redis_rate_limiting`
+trait. This trait clears the rate limiting data stored in Redis cache between specs. If
+a single test triggers the rate limit, the `:disable_rate_limit` can be used instead.
+
#### Stubbing File methods
In the situations where you need to
@@ -1437,9 +1451,10 @@ GitLab uses [factory_bot](https://github.com/thoughtbot/factory_bot) as a test f
resulting record to pass validation.
- When instantiating from a factory, don't supply attributes that aren't
required by the test.
-- Prefer [implicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#implicit-definition)
- or [explicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#explicit-definition)
- association definitions instead of using `create` / `build` for association setup.
+- Prefer [implicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#implicit-definition),
+ [explicit](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#explicit-definition), or
+ [inline](https://github.com/thoughtbot/factory_bot/blob/master/GETTING_STARTED.md#inline-definition) associations
+ over `create` / `build` for association setup in callbacks.
See [issue #262624](https://gitlab.com/gitlab-org/gitlab/-/issues/262624) for further context.
- Factories don't have to be limited to `ActiveRecord` objects.
[See example](https://gitlab.com/gitlab-org/gitlab-foss/commit/0b8cefd3b2385a21cfed779bd659978c0402766d).
diff --git a/doc/development/testing_guide/contract/consumer_tests.md b/doc/development/testing_guide/contract/consumer_tests.md
index 67bb7160749..9c72e6835bd 100644
--- a/doc/development/testing_guide/contract/consumer_tests.md
+++ b/doc/development/testing_guide/contract/consumer_tests.md
@@ -213,7 +213,7 @@ pactWith(
const discussions = await getDiscussions({
url: provider.mockService.baseUrl,
});
-
+
expect(discussions).toEqual(Matchers.eachLike({
id: 'fd73763cbcbf7b29eb8765d969a38f7d735e222a',
project_id: 6954442,
diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md
index 37eda7f76f5..777a9f47339 100644
--- a/doc/development/testing_guide/end_to_end/best_practices.md
+++ b/doc/development/testing_guide/end_to_end/best_practices.md
@@ -17,8 +17,8 @@ In case custom inflection logic is needed, custom inflectors are added in the [q
## Link a test to its test case
Every test should have a corresponding test case in the [GitLab project Test Cases](https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases) as well as a results issue in the [Quality Test Cases project](https://gitlab.com/gitlab-org/quality/testcases/-/issues).
-If a test case issue does not yet exist you can create one yourself. To do so, create a new
-issue in the [Test Cases](https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases) GitLab project
+If a test case issue does not yet exist, any GitLab team member can create a new test case in
+the [Test Cases](https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases) GitLab project
with a placeholder title. After the test case URL is linked to a test in the code, when the test is
run in a pipeline that has reporting enabled, the `report-results` script automatically updates the
test case and the results issue.
diff --git a/doc/development/testing_guide/end_to_end/feature_flags.md b/doc/development/testing_guide/end_to_end/feature_flags.md
index 8e25c817938..6d826e170f6 100644
--- a/doc/development/testing_guide/end_to_end/feature_flags.md
+++ b/doc/development/testing_guide/end_to_end/feature_flags.md
@@ -160,7 +160,7 @@ For example:
```ruby
def initialize
- name_of_the_future_flag_activated = false
+ name_of_the_feature_flag_activated = false
...
end
```
diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md
index 1853ad47779..8ffe044c4d8 100644
--- a/doc/development/testing_guide/end_to_end/index.md
+++ b/doc/development/testing_guide/end_to_end/index.md
@@ -123,9 +123,8 @@ test or a group of tests that are different than the groups in any of the existi
For example, when we [dequarantine](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/debugging-qa-test-failures/#dequarantining-tests)
a flaky test we first want to make sure that it's no longer flaky.
-We can do that using the `ce:custom-parallel` and `ee:custom-parallel` jobs.
-Both are manual jobs that you can configure using custom variables.
-When selecting the name (not the play icon) of one of the parallel jobs,
+We can do that by running `_ee:quarantine` manual job.
+When selecting the name (not the play icon) of manual job,
you are prompted to enter variables. You can use any of
[the variables that can be used with `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-gitlab-environment-variables)
as well as these:
@@ -134,7 +133,7 @@ as well as these:
|-|-|
| `QA_SCENARIO` | The scenario to run (default `Test::Instance::Image`) |
| `QA_TESTS` | The tests to run (no default, which means run all the tests in the scenario). Use file paths as you would when running tests via RSpec, for example, `qa/specs/features/ee/browser_ui` would include all the `EE` UI tests. |
-| `QA_RSPEC_TAGS` | The RSpec tags to add (no default) |
+| `QA_RSPEC_TAGS` | The RSpec tags to add (default `--tag quarantine`) |
For now,
[manual jobs with custom variables don't use the same variable when retried](https://gitlab.com/gitlab-org/gitlab/-/issues/31367),
@@ -157,6 +156,19 @@ against the [Review App](../review_apps.md).
See [Review Apps](../review_apps.md) for more details about Review Apps.
+#### Selective test execution
+
+In order to limit amount of tests executed in a merge request, dynamic selection of which tests to execute is present. Algorithm of which tests to run is based
+on changed files and merge request labels. Following criteria determine which tests will run:
+
+1. Changes in `qa` framework code would execute the full suite
+1. Changes in particular `_spec.rb` file in `qa` folder would execute only that particular test
+1. Merge request with backend changes and label `devops::manage` would execute all e2e tests related to `manage` stage
+
+#### Overriding selective test execution
+
+To override selective test execution and trigger the full suite, label `pipeline:run-all-e2e` should be added to particular merge request.
+
### Run tests in parallel
To run tests in parallel on CI, the [Knapsack](https://github.com/KnapsackPro/knapsack)
@@ -184,7 +196,8 @@ Use these environment variables to configure metrics export:
| `QA_INFLUXDB_URL` | `true` | Should be set to `https://influxdb.quality.gitlab.net`. No default value. |
| `QA_INFLUXDB_TOKEN` | `true` | InfluxDB write token that can be found under `Influxdb auth tokens` document in `Gitlab-QA` `1Password` vault. No default value. |
| `QA_RUN_TYPE` | `false` | Arbitrary name for test execution, like `package-and-test`. Automatically inferred from the project name for live environment test executions. No default value. |
-| `QA_EXPORT_TEST_METRICS` | `false` | Flag to enable or disable metrics export. Defaults to `true`. |
+| `QA_EXPORT_TEST_METRICS` | `false` | Flag to enable or disable metrics export to InfluxDB. Defaults to `false`. |
+| `QA_SAVE_TEST_METRICS` | `false` | Flag to enable or disable saving metrics as JSON file. Defaults to `false`. |
## Test reports
diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
index b12c6bd443a..a7d1ece77b2 100644
--- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
+++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md
@@ -36,7 +36,7 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec
| `:only` | The test is only to be run in specific execution contexts. See [test execution context selection](execution_context_selection.md) for more information. |
| `:orchestrated` | The GitLab instance under test may be [configured by `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#orchestrated-tests) to be different to the default GitLab configuration, or `gitlab-qa` may launch additional services in separate Docker containers, or both. Tests tagged with `:orchestrated` are excluded when testing environments where we can't dynamically modify the GitLab configuration (for example, Staging). |
| `:packages` | The test requires a GitLab instance that has the [Package Registry](../../../administration/packages/index.md#gitlab-package-registry-administration) enabled. |
-| `:product_group` | Specifies what product group the test belongs to. See [Product sections, stages, groups, and categories](https://about.gitlab.com/handbook/product/categories) for the comprehensive groups list. |
+| `:product_group` | Specifies what product group the test belongs to. See [Product sections, stages, groups, and categories](https://about.gitlab.com/handbook/product/categories/) for the comprehensive groups list. |
| `:quarantine` | The test has been [quarantined](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/debugging-qa-test-failures/#quarantining-tests), runs in a separate job that only includes quarantined tests, and is allowed to fail. The test is skipped in its regular job so that if it fails it doesn't hold up the pipeline. Note that you can also [quarantine a test only when it runs in a specific context](execution_context_selection.md#quarantine-a-test-for-a-specific-environment). |
| `:relative_url` | The test requires a GitLab instance to be installed under a [relative URL](../../../install/relative_url.md). |
| `:reliable` | The test has been [promoted to a reliable test](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/reliable-tests/#promoting-an-existing-test-to-reliable) meaning it passes consistently in all pipelines, including merge requests. |
diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md
index f380b24d7a5..419942d6b8f 100644
--- a/doc/development/testing_guide/end_to_end/style_guide.md
+++ b/doc/development/testing_guide/end_to_end/style_guide.md
@@ -12,7 +12,7 @@ This document describes the conventions used at GitLab for writing End-to-end (E
### When to use `click_`?
-When clicking in a single link to navigate, use `click_`.
+When selecting a single link to navigate, use `click_`.
For example:
@@ -24,9 +24,9 @@ def click_ci_cd_pipelines
end
```
-From a testing perspective, if we want to check that clicking a link, or a button (a single interaction) is working as intended, we would want the test to read as:
+From a testing perspective, if we want to check that selecting a link, or a button (a single interaction) is working as intended, we would want the test to read as:
-- Click a certain element
+- Select a certain element
- Verify the action took place
### When to use `go_to_`?
@@ -47,7 +47,7 @@ end
`go_to_` fits the definition of interacting with multiple elements very well given it's more of a meta-navigation action that includes multiple interactions.
-Notice that in the above example, before clicking the `:operations_environments_link`, another element is hovered over.
+Notice that in the above example, before selecting the `:operations_environments_link`, another element is hovered over.
> We can create these methods as helpers to abstract multi-step navigation.
diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md
index 45f1d0ddf7e..cc62a0ebf03 100644
--- a/doc/development/testing_guide/flaky_tests.md
+++ b/doc/development/testing_guide/flaky_tests.md
@@ -11,9 +11,144 @@ info: To determine the technical writer assigned to the Stage/Group associated w
It's a test that sometimes fails, but if you retry it enough times, it passes,
eventually.
+## What are the potential cause for a test to be flaky?
+
+### Unclean environment
+
+**Label:** `flaky-test::unclean environment`
+
+**Description:** The environment got dirtied by a previous test. The actual cause is probably not the flaky test here.
+
+**Difficulty to reproduce:** Moderate. Usually, running the same spec files until the one that's failing reproduces the problem.
+
+**Resolution:** Fix the previous tests and/or places where the environment is modified, so that
+it's reset to a pristine test after each test.
+
+**Examples:**
+
+- [Example 1](https://gitlab.com/gitlab-org/gitlab/-/issues/378414#note_1142026988): A migration
+ test might roll-back the database, perform its testing, and then roll-up the database in an
+ inconsistent state, so that following tests might not know about certain columns.
+- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/issues/368500): A test modifies data that is
+ used by a following test.
+- [Example 3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103434#note_1172316521): A test for a database query passes in a fresh database, but in a
+ CI/CD pipeline where the database is used to process previous test sequences, the test fails. This likely
+ means that the query itself needs to be updated to work in a non-clean database.
+
+### Ordering assertion
+
+**Label:** `flaky-test::ordering assertion`
+
+**Description:** The test is expecting a specific order in the data under test yet the data is in
+a non-deterministic order.
+
+**Difficulty to reproduce:** Easy. Usually, running the test locally several times would reproduce
+the problem.
+
+**Resolution:** Depending on the problem, you might want to:
+
+- loosen the assertion if the test shouldn't care about ordering but only on the elements
+- fix the test by specifying a deterministic ordering
+- fix the app code by specifying a deterministic ordering
+
+**Examples:**
+
+- [Example 1](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10148/diffs): Without
+ specifying `ORDER BY`, database will not give deterministic ordering, or data race happening
+ in the tests.
+
+### Dataset-specific
+
+**Label:** `flaky-test::dataset-specific`
+
+**Description:** The test assumes the dataset is in a particular (usually limited) state, which
+might not be true depending on when the test run during the test suite.
+
+**Difficulty to reproduce:** Moderate, as the amount of data needed to reproduce the issue might be
+difficult to achieve locally.
+
+**Resolution:** Fix the test to not assume that the dataset is in a particular state, don't hardcode IDs.
+
+**Examples:**
+
+- [Example 1](https://gitlab.com/gitlab-org/gitlab/-/issues/378381): The database is recreated when
+ any table has more than 500 columns. It could pass in the merge request, but fail later in
+ `master` if the order of tests changes.
+- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91016/diffs): A test asserts
+ that trying to find a record with an unexisting ID retuns an error message. The test uses an
+ hardcoded ID that's supposed to not exist (e.g. `42`). If the test is run early in the test
+ suite, it might pass as not enough records were created before it, but as soon as it would run
+ later in the suite, there could be a record that actually has the ID `42`, hence the test would
+ start to fail.
+
+### Random input
+
+**Label:** `flaky-test::random input`
+
+**Description:** The test use random values, that sometimes match the expectations, and sometimes not.
+
+**Difficulty to reproduce:** Easy, as the test can be modified locally to use the "random value"
+used at the time the test failed
+
+**Resolution:** Once the problem is reproduced, it should be easy to debug and fix either the test
+or the app.
+
+**Examples:**
+
+- [Example 1](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/20121): The test isn't robust enough to handle a specific data, that only appears sporadically since the data input is random.
+
+### Unreliable DOM Selector
+
+**Label:** `flaky-test::unreliable dom selector`
+
+**Description:** The DOM selector used in the test is unreliable.
+
+**Difficulty to reproduce:** Moderate to difficult. Depending on whether the DOM selector is duplicated, or appears after a delay etc.
+
+**Resolution:** It really depends on the problem here. It could be to wait for requests to finish, to scroll down the page etc.
+
+**Examples:**
+
+- [Example 1](https://gitlab.com/gitlab-org/gitlab/-/issues/338341): A non-unique CSS selector
+ matching more than one element, or a non-waiting selector method that does not allow rendering
+ time before throwing an `element not found` error.
+- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101728/diffs): A CSS selector
+ only appears after a GraphQL requests has finished, and the UI has updated.
+
+### Datetime-sensitive
+
+**Label:** `flaky-test::datetime-sensitive`
+
+**Description:** The test is assuming a specific date or time.
+
+**Difficulty to reproduce:** Easy to moderate, depending on whether the test consistently fails after a certain date, or only fails at a given time or date.
+
+**Resolution:** Freezing the time is usually a good solution.
+
+**Examples:**
+
+- [Example 1](https://gitlab.com/gitlab-org/gitlab/-/issues/118612): A test that breaks after some time passed.
+
+### Unstable infrastructure
+
+**Label:** `flaky-test::unstable infrastructure`
+
+**Description:** The test fails from time to time due to infrastructure issues.
+
+**Difficulty to reproduce:** Hard. It's really hard to reproduce CI infrastructure issues. It might
+be possible by using containers locally.
+
+**Resolution:** Starting a conversation with the Infrastructure department in a dedicated issue is
+usually a good idea.
+
+**Examples:**
+
+- [Example 1](https://gitlab.com/gitlab-org/gitlab/-/issues/363214): The runner is under heavy load at this time.
+- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/issues/360559): The runner is having networking issues, making a job failing early
+
## Quarantined tests
-When a test frequently fails in `main`,
+When a test frequently fails in `master`,
create [a ~"failure::flaky-test" issue](https://about.gitlab.com/handbook/engineering/workflow/#broken-master).
If the test cannot be fixed in a timely fashion, there is an impact on the
@@ -62,7 +197,7 @@ For example, `FLAKY_RSPEC_GENERATE_REPORT=1 bin/rspec ...`.
The `rspec/flaky/report-suite.json` report is:
-- Used for [automatically skipping known flaky tests](../pipelines.md#automatic-skipping-of-flaky-tests).
+- Used for [automatically skipping known flaky tests](../pipelines/index.md#automatic-skipping-of-flaky-tests).
- [Imported into Snowflake](https://gitlab.com/gitlab-data/analytics/-/blob/master/extract/gitlab_flaky_tests/upload.py)
once per day, for monitoring with the [internal dashboard](https://app.periscopedata.com/app/gitlab/888968/EP---Flaky-tests).
diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md
index d7627e2064f..041b0f0a4f4 100644
--- a/doc/development/testing_guide/frontend_testing.md
+++ b/doc/development/testing_guide/frontend_testing.md
@@ -11,7 +11,7 @@ at GitLab. We use Jest for JavaScript unit and integration testing,
and RSpec feature tests with Capybara for e2e (end-to-end) integration testing.
Unit and feature tests need to be written for all new features.
-Most of the time, you should use [RSpec](https://github.com/rspec/rspec-rails#feature-specs) for your feature tests.
+Most of the time, you should use [RSpec](https://github.com/rspec/rspec-rails#feature-specs) for your feature tests. For more information on how to get started with feature tests, see [get started with feature tests](#get-started-with-feature-tests)
Regression tests should be written for bug fixes to prevent them from recurring
in the future.
@@ -458,7 +458,7 @@ If you cannot register handlers to the `Promise`, for example because it is exec
```javascript
it('waits for an Ajax call', async () => {
synchronousFunction();
-
+
await waitForPromises();
expect(something).toBe('done');
@@ -962,7 +962,7 @@ If an integration test depends on JavaScript to run correctly, you need to make
sure the spec is configured to enable JavaScript when the tests are run. If you
don't do this, the spec runner displays vague error messages.
-To enable a JavaScript driver in an `rspec` test, add `:js` to the
+To enable a JavaScript driver in an `RSpec` test, add `:js` to the
individual spec or the context block containing multiple specs that need
JavaScript enabled:
@@ -1242,6 +1242,332 @@ A good guideline to follow: the more complex the component you may want to steer
- To just have some kind of test written
- To capture highly volatile UI elements without stubbing them (Think of GitLab UI version updates)
+## Get started with feature tests
+
+### What is a feature test
+
+A [feature test](testing_levels.md#white-box-tests-at-the-system-level-formerly-known-as-system--feature-tests), also known as `white-box testing`, is a test that spawns a browser and has Capybara helpers. This means the test can:
+
+- Locate an element in the browser.
+- Click that element.
+- Call the API.
+
+Feature tests are expensive to run. You should make sure that you **really want** this type of test before running one.
+
+All of our feature tests are written in `Ruby` but often end up being written by `JavaScript` engineers, as they implement the user-facing feature. So, the following section assumes no prior knowledge of `Ruby` or `Capybara`, and provide a clear guideline on when and how to use these tests.
+
+### When to use feature tests
+
+You should use a feature test when the test:
+
+- Is across multiple components.
+- Requires that a user navigate across pages.
+- Is submitting a form and observing results elsewhere.
+- Would result in a huge number of mocking and stubbing with fake data and components if done as a unit test.
+
+Feature tests are especially useful when you want to test:
+
+- That multiple components are working together successfully.
+- Complex API interactions. Feature tests interact with the API, so they are slower but do not need any level of mocking or fixtures.
+
+### When not to use feature tests
+
+You should use `jest` and `vue-test-utils` unit tests instead of a feature test if you can get the same test results from these methods. Feature tests are quite expensive to run.
+
+You should use a unit test if:
+
+- The behavior you are implementing is all in one component.
+- You can simulate other components' behavior to trigger the desired effect.
+- You can already select UI elements in the virtual DOM to trigger the desired effects.
+
+Also, if a behavior in your new code needs multiple components to work together, you should consider testing your behavior higher in the component tree. For example, let's say that we have a component called `ParentComponent` with the code:
+
+```vue
+ <script>
+ export default{
+ name: ParentComponent,
+ data(){
+ return {
+ internalData: 'oldValue'
+ }
+ },
+ methods:{
+ changeSomeInternalData(newVal){
+ this.internalData = newVal
+ }
+ }
+ }
+ </script>
+ <template>
+ <div>
+ <child-component-1 @child-event="changeSomeInternalData" />
+ <child-component-2 :parent-data="internalData" />
+ </div>
+ </template>
+```
+
+In this example:
+
+- `ChildComponent1` emits an event.
+- `ParentComponent` changes its `internalData` value.
+- `ParentComponent` passes the props down to `ChildComponent2`.
+
+You can use a unit test instead by:
+
+- From inside the `ParentComponent` unit test file, emitting the expected event from `childComponent1`
+- Making sure the prop is passed down to `childComponent2`.
+
+Then each child component unit tests what happens when the event is emitted and when the prop changes.
+
+This example also applies at larger scale and with deeper component trees. It is definitely worth using unit tests and avoiding the extra cost of feature tests if you can:
+
+- Confidently mount child components.
+- Emit events or select elements in the virtual DOM.
+- Get the test behavior that you want.
+
+### Where to create your test
+
+Feature tests live in `spec/features` folder. You should look for existing files that can test the page you are adding a feature to. Within that folder, you can locate your section. For example, if you wanted to add a new feature test for the pipeline page, you would look in `spec/features/projects/pipelines` and see if the test you want to write exists here.
+
+### How to run a feature test
+
+1. Make sure that you have a working GDK environment.
+1. Start your `gdk` environment with `gdk start` command.
+1. In your terminal, run:
+
+ ```shell
+ bundle exec rspec path/to/file:line_of_my_test
+ ```
+
+You can also prefix this command with `WEBDRIVER_HEADLESS=0` which will run the test by opening an actual browser on your computer that you can see, which is very useful for debugging.
+
+### How to write a test
+
+#### Basic file structure
+
+1. Make all string literals unchangeable
+
+ In all feature tests, the very first line should be:
+
+ ```ruby
+ # frozen_string_literal: true
+ ```
+
+ This is in every `Ruby` file and makes all string literals unchangeable. There are also some performance benefits, but this is beyond the scope of this section.
+
+1. Import dependencies.
+
+ You should import the modules you need. You will most likely always need to require `spec_helper`:
+
+ ```ruby
+ require 'spec_helper'
+ ```
+
+ Import any other relevant module.
+
+1. Create a global scope for RSpec to define our tests, just like what we do in jest with the initial describe block.
+
+Then, you need to create the very first `RSpec` scope.
+
+```ruby
+RSpec.describe 'Pipeline', :js do
+ ...
+end
+```
+
+What is different though, is that just like everything in Ruby, this is actually a `class`. Which means that right at the top, you can `include` modules that you'd need for your test. For example, you could include the `RoutesHelpers` to navigate more easily.
+
+```ruby
+RSpec.describe 'Pipeline', :js do
+ include RoutesHelpers
+ ...
+end
+```
+
+After all of this implementation, we have a file that looks something like this:
+
+```ruby
+# frozen_string_literal: true
+
+require 'spec_helper'
+
+RSpec.describe 'Pipeline', :js do
+ include RoutesHelpers
+end
+```
+
+#### Seeding data
+
+Each test is in its own environment and so you must use a factory to seed the required data. To continue on with the pipeline example, let's say that you want a test that takes you to the main pipeline page, which is at the route `/namespace/project/-/pipelines/:id/`.
+
+Most feature tests at least require you to create a user, because you want to be signed in. You can skip this step if you don't have to be signed in, but as a general rule, you should **always create a user unless you are specifically testing a feature looked at by an anonymous user**. This makes sure that you explicitly set a level of permission that you can edit in the test as needed to change or test a new level of permission as the section changes. To create a user:
+
+```ruby
+ let(:user) { create(:user) }
+```
+
+This creates a variable that holds the newly created user and we can use `create` because we imported the `spec_helper`.
+
+However, we have not done anything with this user yet because it's just a variable. So, in the `before do` block of the spec, we could sign in with the user so that every spec starts with a signed in user.
+
+```ruby
+ let(:user) { create(:user) }
+
+ before do
+ sign_in(user)
+ end
+```
+
+Now that we have a user, we should look at what else we'd need before asserting anything on a pipeline page. If you look at the route `/namespace/project/-/pipelines/:id/` we can determine we need a project and a pipeline.
+
+So we'd create a project and pipeline, and link them together. Usually in factories, the child element requires its parent as an argument. In this case, a pipeline is a child of a project. So we can create the project first, and then when we create the pipeline, we are pass the project as an argument which "binds" the pipeline to the project. A pipeline is also owned by a user, so we need the user as well. For example, this creates a project and a pipeline:
+
+```ruby
+ let(:user) { create(:user) }
+ let(:project) { create(:project, :repository) }
+ let(:pipeline) { create(:ci_pipeline, project: project, ref: 'master', sha: project.commit.id, user: user) }
+```
+
+In the same spirit, you could then create a job (build) by using the build factory and passing the parent pipeline:
+
+```ruby
+ create(:ci_build, pipeline: pipeline, stage_idx: 10, stage: 'publish', name: 'CentOS')
+```
+
+There are many factories that already exists, so make sure to look at other existing files to see if what you need is available.
+
+#### Navigation
+
+You can navigate to a page by using the `visit` method and passing the path as an argument. Rails automatically generates helper paths, so make sure to use these instead of a hardcoded string. They are generated using the route model, so if we want to go to a pipeline, we'd use:
+
+```ruby
+ visit project_pipeline_path(project, pipeline)
+```
+
+Before executing any page interaction when navigating or making asynchronous call through the UI, make sure to use `wait_for_requests` before proceeding with further instructions.
+
+#### Elements interaction
+
+There are a lot of different ways to find and interact with elements. For example, you could use the basic `find` method with the `selector` and `text` parameter and then use the `.click` method
+
+```ruby
+ find('.gl-tab-nav-item', text: 'Tests').click
+```
+
+Alternatively, you could use `click_button` with a string of text that is found within the button, which is a more semantically meaningful way of clicking the element.
+
+```ruby
+ click_button 'Text inside the button element'
+```
+
+If you want to follow a link, then there is `click_link`:
+
+```ruby
+ click_link 'Text inside the link tag'
+```
+
+You can use `fill_in` to fill input / form elements. The first argument is the selector, the second is `with:` which is the value to pass in.
+
+```ruby
+ fill_in 'current_password', with: '123devops'
+```
+
+Alternatively, you can use the `find` selector paired with `send_keys` to add keys in a field without removing previous text, or `set` which completely replaces the value of the input element.
+
+All of these are valid selectors and methods. Pick whichever suits your needs and look around as there are many more useful ones!
+
+#### Assertions
+
+To assert anything in a page, you can always access `page` variable, which is automatically defines and actually means the page document. This means you can expect the `page` to have certain components like selectors or content. Here are a few examples:
+
+```ruby
+ # Finding an element by ID
+ expect(page).to have_selector('#js-pipeline-graph')
+```
+
+```ruby
+ # Finding by text
+ expect(page).to have_content('build')
+```
+
+```ruby
+ # Finding by `href` value
+ expect(page).to have_link(pipeline.ref)
+```
+
+```ruby
+ # Finding by CSS selector. This is a last resort.
+ # For example, when you cannot add attributes on the desired element.
+ expect(page).to have_css('.js-icon-retry')
+```
+
+```ruby
+ # Find by data-testid
+ # Like CSS selector, this is acceptable when there isn't a specific matcher available.
+ expect(page).to have_selector('[data-testid="pipeline-multi-actions-dropdown"]')
+```
+
+```ruby
+ # You can combine any of these selectors with `not_to` instead
+ expect(page).not_to have_selector('#js-pipeline-graph')
+```
+
+```ruby
+ # When a test case has back to back expectations,
+ # it is recommended to group them using `:aggregate_failures`
+ it 'shows the issue description and design references', :aggregate_failures do
+ expect(page).to have_text('The designs I mentioned')
+ expect(page).to have_link(design_tab_ref)
+ expect(page).to have_link(design_ref_a)
+ expect(page).to have_link(design_ref_b)
+ end
+```
+
+You can also create a sub-block to look into, to:
+
+- Scope down where you are making your assertions and reduce the risk of finding another element that was not intended.
+- Make sure an element is found within the right boundaries.
+
+```ruby
+ page.within('#js-pipeline-graph') do
+ ...
+ end
+```
+
+#### Feature flags
+
+By default, every feature flag is enabled **regardless of the YAML definition or the flags you've set manually in your GDK**. To test when a feature flag is disabled, you must manually stub the flag, ideally in a `before do` block.
+
+```ruby
+ stub_feature_flags(my_feature_flag: false)
+```
+
+If you are stubbing an `ee` feature flag, then use:
+
+```ruby
+ stub_licensed_features(my_feature_flag: false)
+```
+
+### Debugging
+
+You can run your spec with the prefix `WEBDRIVER_HEADLESS=0` to open an actual browser. However, the specs goes though the commands quickly and leaves you no time to look around.
+
+To avoid this problem, you can write `binding.pry` on the line where you want Capybara to stop execution. You are then inside the browser with normal usage. To understand why you cannot find certain elements, you can:
+
+- Select elements.
+- Use the console and network tab.
+- Execute selectors inside the browser console.
+
+Inside the terminal, where capybara is running, you can also execute `next` which goes line by line through the test. This way you can check every single interaction one by one to see what might be causing an issue.
+
+### Updating ChromeDriver
+
+On MacOS, if you get a ChromeDriver error, make sure to update it by running
+
+```shell
+ brew upgrade chromedriver
+```
+
---
[Return to Testing documentation](index.md)
diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md
index 753089fa673..5008564de01 100644
--- a/doc/development/testing_guide/index.md
+++ b/doc/development/testing_guide/index.md
@@ -43,12 +43,17 @@ system tests, parameterized tests etc.
Everything you should know about how to write good Frontend tests: Jest,
testing promises, stubbing etc.
+## [Getting started with Feature tests](frontend_testing.md#get-started-with-feature-tests)
+
+Need to get started with feature tests? Here are some general guidelines,
+tips and tricks to get the most out of white-box testing.
+
## [Flaky tests](flaky_tests.md)
What are flaky tests, the different kind of flaky tests we encountered, and what
we do about them.
-## [GitLab pipelines](../pipelines.md)
+## [GitLab pipelines](../pipelines/index.md)
How GitLab test suite is run in the CI context: setup, caches, artifacts,
parallelization, monitoring.
diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md
index d86d1c3c7b4..5edf18725c0 100644
--- a/doc/development/testing_guide/review_apps.md
+++ b/doc/development/testing_guide/review_apps.md
@@ -4,9 +4,9 @@ group: unassigned
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Review Apps
+# Using review apps in the development of GitLab
-Review Apps are deployed using the `start-review-app-pipeline` job which triggers a child pipeline containing a series of jobs to perform the various tasks needed to deploy a Review App.
+Review apps are deployed using the `start-review-app-pipeline` job which triggers a child pipeline containing a series of jobs to perform the various tasks needed to deploy a review app.
![start-review-app-pipeline job](img/review-app-parent-pipeline.png)
@@ -21,7 +21,7 @@ For any of the following scenarios, the `start-review-app-pipeline` job would be
- for scheduled pipelines
- the MR has the `pipeline:run-review-app` label set
-## QA runs on Review Apps
+## QA runs on review apps
On every [pipeline](https://gitlab.com/gitlab-org/gitlab/pipelines/125315730) in the `qa` stage (which comes after the
`review` stage), the `review-qa-smoke` and `review-qa-reliable` jobs are automatically started. The `review-qa-smoke` runs
@@ -34,7 +34,7 @@ You can also manually start the `review-qa-all`: it runs the full QA suite.
After the end-to-end test runs have finished, [Allure reports](https://github.com/allure-framework/allure2) are generated and published by
the `allure-report-qa-smoke`, `allure-report-qa-reliable`, and `allure-report-qa-all` jobs. A comment with links to the reports are added to the merge request.
-Errors can be found in the `gitlab-review-apps` Sentry project and [filterable by Review App URL](https://sentry.gitlab.net/gitlab/gitlab-review-apps/?query=url%3A%22https%3A%2F%2Fgitlab-review-require-ve-u92nn2.gitlab-review.app%2F%22) or [commit SHA](https://sentry.gitlab.net/gitlab/gitlab-review-apps/releases/6095b501da7/all-events/).
+Errors can be found in the `gitlab-review-apps` Sentry project and [filterable by review app URL](https://sentry.gitlab.net/gitlab/gitlab-review-apps/?query=url%3A%22https%3A%2F%2Fgitlab-review-require-ve-u92nn2.gitlab-review.app%2F%22) or [commit SHA](https://sentry.gitlab.net/gitlab/gitlab-review-apps/releases/6095b501da7/all-events/).
### Bypass failed review app deployment to merge a broken `master` fix
@@ -47,7 +47,7 @@ On every [pipeline](https://gitlab.com/gitlab-org/gitlab/pipelines/125315730) in
browser performance testing using a
[Sitespeed.io Container](../../ci/testing/browser_performance_testing.md).
-## Sample Data for Review Apps
+## Sample Data for review apps
Upon deployment of a review app, project data is created from the [`sample-gitlab-project`](https://gitlab.com/gitlab-org/sample-data-templates/sample-gitlab-project) template project. This aims to provide projects with prepopulated resources to facilitate manual and exploratory testing.
@@ -55,16 +55,16 @@ The sample projects will be created in the `root` user namespace and can be acce
## How to
-### Redeploy Review App from a clean slate
+### Redeploy review app from a clean slate
-To reset Review App and redeploy from a clean slate, do the following:
+To reset review app and redeploy from a clean slate, do the following:
1. Run `review-stop` job.
1. Re-deploy by running or retrying `review-deploy` job.
-Doing this will remove all existing data from a previously deployed Review App.
+Doing this will remove all existing data from a previously deployed review app.
-### Get access to the GCP Review Apps cluster
+### Get access to the GCP review apps cluster
You need to [open an access request (internal link)](https://gitlab.com/gitlab-com/access-requests/-/issues/new)
for the `gcp-review-apps-dev` GCP group and role.
@@ -74,7 +74,7 @@ This grants you the following permissions for:
- [Retrieving pod logs](#dig-into-a-pods-logs). Granted by [Viewer (`roles/viewer`)](https://cloud.google.com/iam/docs/understanding-roles#kubernetes-engine-roles).
- [Running a Rails console](#run-a-rails-console). Granted by [Kubernetes Engine Developer (`roles/container.pods.exec`)](https://cloud.google.com/iam/docs/understanding-roles#kubernetes-engine-roles).
-### Log into my Review App
+### Log into my review app
For GitLab Team Members only. If you want to sign in to the review app, review
the GitLab handbook information for the [shared 1Password account](https://about.gitlab.com/handbook/security/#1password-for-teams).
@@ -82,23 +82,23 @@ the GitLab handbook information for the [shared 1Password account](https://about
- The default username is `root`.
- The password can be found in the 1Password login item named `GitLab EE Review App`.
-### Enable a feature flag for my Review App
+### Enable a feature flag for my review app
-1. Open your Review App and sign in as documented above.
+1. Open your review app and sign in as documented above.
1. Create a personal access token.
1. Enable the feature flag using the [Feature flag API](../../api/features.md).
-### Find my Review App slug
+### Find my review app slug
1. Open the `review-deploy` job.
1. Look for `** Deploying review-*`.
1. For instance for `** Deploying review-1234-abc-defg... **`,
- your Review App slug would be `review-1234-abc-defg` in this case.
+ your review app slug would be `review-1234-abc-defg` in this case.
### Run a Rails console
1. Make sure you [have access to the cluster](#get-access-to-the-gcp-review-apps-cluster) and the `container.pods.exec` permission first.
-1. [Filter Workloads by your Review App slug](https://console.cloud.google.com/kubernetes/workload?project=gitlab-review-apps). For example, `review-qa-raise-e-12chm0`.
+1. [Filter Workloads by your review app slug](https://console.cloud.google.com/kubernetes/workload?project=gitlab-review-apps). For example, `review-qa-raise-e-12chm0`.
1. Find and open the `toolbox` Deployment. For example, `review-qa-raise-e-12chm0-toolbox`.
1. Select the Pod in the "Managed pods" section. For example, `review-qa-raise-e-12chm0-toolbox-d5455cc8-2lsvz`.
1. Select the `KUBECTL` dropdown list, then `Exec` -> `toolbox`.
@@ -111,7 +111,7 @@ the GitLab handbook information for the [shared 1Password account](https://about
### Dig into a Pod's logs
1. Make sure you [have access to the cluster](#get-access-to-the-gcp-review-apps-cluster) and the `container.pods.getLogs` permission first.
-1. [Filter Workloads by your Review App slug](https://console.cloud.google.com/kubernetes/workload?project=gitlab-review-apps). For example, `review-qa-raise-e-12chm0`.
+1. [Filter Workloads by your review app slug](https://console.cloud.google.com/kubernetes/workload?project=gitlab-review-apps). For example, `review-qa-raise-e-12chm0`.
1. Find and open the `migrations` Deployment. For example, `review-qa-raise-e-12chm0-migrations.1`.
1. Select the Pod in the "Managed pods" section. For example, `review-qa-raise-e-12chm0-migrations.1-nqwtx`.
1. Select `Container logs`.
@@ -149,11 +149,11 @@ subgraph "2. gitlab `review-prepare` stage"
end
subgraph "3. gitlab `review` stage"
- C["review-deploy<br><br>Helm deploys the Review App using the Cloud<br/>Native images built by the CNG-mirror pipeline.<br><br>Cloud Native images are deployed to the `review-apps`<br>Kubernetes (GKE) cluster, in the GCP `gitlab-review-apps` project."]
+ C["review-deploy<br><br>Helm deploys the review app using the Cloud<br/>Native images built by the CNG-mirror pipeline.<br><br>Cloud Native images are deployed to the `review-apps`<br>Kubernetes (GKE) cluster, in the GCP `gitlab-review-apps` project."]
end
subgraph "4. gitlab `qa` stage"
- E[review-qa-smoke, review-qa-reliable<br><br>gitlab-qa runs the smoke and reliable suites against the Review App.]
+ E[review-qa-smoke, review-qa-reliable<br><br>gitlab-qa runs the smoke and reliable suites against the review app.]
end
subgraph "CNG-mirror pipeline"
@@ -172,7 +172,7 @@ subgraph "CNG-mirror pipeline"
job [triggers a pipeline](https://gitlab.com/gitlab-org/build/CNG-mirror/pipelines/44364657)
in the [`CNG-mirror`](https://gitlab.com/gitlab-org/build/CNG-mirror) project.
- The `review-build-cng` job automatically starts only if your MR includes
- [CI or frontend changes](../pipelines.md#changes-patterns). In other cases, the job is manual.
+ [CI or frontend changes](../pipelines/internals.md#changes-patterns). In other cases, the job is manual.
- The [`CNG-mirror`](https://gitlab.com/gitlab-org/build/CNG-mirror/pipelines/44364657) pipeline creates the Docker images of
each component (for example, `gitlab-rails-ee`, `gitlab-shell`, `gitaly` etc.)
based on the commit from the [GitLab pipeline](https://gitlab.com/gitlab-org/gitlab/pipelines/125315730) and stores
@@ -180,10 +180,10 @@ subgraph "CNG-mirror pipeline"
- We use the [`CNG-mirror`](https://gitlab.com/gitlab-org/build/CNG-mirror) project so that the `CNG`, (Cloud
Native GitLab), project's registry is not overloaded with a lot of transient Docker images.
1. Once `review-build-cng` is done, the [`review-deploy`](https://gitlab.com/gitlab-org/gitlab/-/jobs/467724810) job
- deploys the Review App using [the official GitLab Helm chart](https://gitlab.com/gitlab-org/charts/gitlab/) to
+ deploys the review app using [the official GitLab Helm chart](https://gitlab.com/gitlab-org/charts/gitlab/) to
the [`review-apps`](https://console.cloud.google.com/kubernetes/clusters/details/us-central1-b/review-apps?project=gitlab-review-apps)
Kubernetes cluster on GCP.
- - The actual scripts used to deploy the Review App can be found at
+ - The actual scripts used to deploy the review app can be found at
[`scripts/review_apps/review-apps.sh`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/review_apps/review-apps.sh).
- These scripts are basically
[our official Auto DevOps scripts](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) where the
@@ -192,11 +192,11 @@ subgraph "CNG-mirror pipeline"
- Since we're using [the official GitLab Helm chart](https://gitlab.com/gitlab-org/charts/gitlab/), this means
you get a dedicated environment for your branch that's very close to what
it would look in production.
- - Each review app is deployed to its own Kubernetes namespace. The namespace is based on the Review App slug that is
+ - Each review app is deployed to its own Kubernetes namespace. The namespace is based on the review app slug that is
unique to each branch.
1. Once the [`review-deploy`](https://gitlab.com/gitlab-org/gitlab/-/jobs/467724810) job succeeds, you should be able to
- use your Review App thanks to the direct link to it from the MR widget. To log
- into the Review App, see "Log into my Review App?" below.
+ use your review app thanks to the direct link to it from the MR widget. To log
+ into the review app, see "Log into my review app?" below.
**Additional notes:**
@@ -213,23 +213,23 @@ subgraph "CNG-mirror pipeline"
`#quality` channel and/or create a ~Quality ~"type::bug" issue with a link to your
merge request.
- The manual `review-stop` can be used to
- stop a Review App manually, and is also started by GitLab once a merge
+ stop a review app manually, and is also started by GitLab once a merge
request's branch is deleted after being merged.
- The Kubernetes cluster is connected to the `gitlab` projects using the
[GitLab Kubernetes integration](../../user/infrastructure/clusters/index.md). This basically
- allows to have a link to the Review App directly from the merge request widget.
+ allows to have a link to the review app directly from the merge request widget.
-### Auto-stopping of Review Apps
+### Auto-stopping of review apps
-Review Apps are automatically stopped 2 days after the last deployment thanks to
+Review apps are automatically stopped 2 days after the last deployment thanks to
the [Environment auto-stop](../../ci/environments/index.md#stop-an-environment-after-a-certain-time-period) feature.
-If you need your Review App to stay up for a longer time, you can
+If you need your review app to stay up for a longer time, you can
[pin its environment](../../ci/environments/index.md#override-a-deployments-scheduled-stop-time) or retry the
`review-deploy` job to update the "latest deployed at" time.
The `review-cleanup` job that automatically runs in scheduled
-pipelines stops stale Review Apps after 5 days,
+pipelines stops stale review apps after 5 days,
deletes their environment after 6 days, and cleans up any dangling Helm releases
and Kubernetes resources after 7 days.
@@ -246,13 +246,13 @@ The Helm version used is defined in the
[`registry.gitlab.com/gitlab-org/gitlab-build-images:gitlab-helm3.5-kubectl1.17` image](https://gitlab.com/gitlab-org/gitlab-build-images/-/blob/master/Dockerfile.gitlab-helm3.5-kubectl1.17#L6)
used by the `review-deploy` and `review-stop` jobs.
-## Diagnosing unhealthy Review App releases
+## Diagnosing unhealthy review app releases
-If [Review App Stability](https://app.periscopedata.com/app/gitlab/496118/Engineering-Productivity-Sandbox?widget=6690556&udv=785399)
+If [review app stability](https://app.periscopedata.com/app/gitlab/496118/Engineering-Productivity-Sandbox?widget=6690556&udv=785399)
dips this may be a signal that the `review-apps` cluster is unhealthy.
-Leading indicators may be health check failures leading to restarts or majority failure for Review App deployments.
+Leading indicators may be health check failures leading to restarts or majority failure for review app deployments.
-The [Review Apps Overview dashboard](https://console.cloud.google.com/monitoring/classic/dashboards/6798952013815386466?project=gitlab-review-apps&timeDomain=1d)
+The [review apps Overview dashboard](https://console.cloud.google.com/monitoring/classic/dashboards/6798952013815386466?project=gitlab-review-apps&timeDomain=1d)
aids in identifying load spikes on the cluster, and if nodes are problematic or the entire cluster is trending towards unhealthy.
See the [review apps page of the Engineering Productivity Runbook](https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/blob/main/runbook/review-apps.md) for troubleshooting review app releases.
@@ -273,7 +273,7 @@ find a way to limit it to only us.**
## Other resources
-- [Review Apps integration for CE/EE (presentation)](https://docs.google.com/presentation/d/1QPLr6FO4LduROU8pQIPkX1yfGvD13GEJIBOenqoKxR8/edit?usp=sharing)
+- [Review apps integration for CE/EE (presentation)](https://docs.google.com/presentation/d/1QPLr6FO4LduROU8pQIPkX1yfGvD13GEJIBOenqoKxR8/edit?usp=sharing)
- [Stability issues](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/212)
### Helpful command line tools
diff --git a/doc/development/utilities.md b/doc/development/utilities.md
index 45a6b74f33a..551834670b3 100644
--- a/doc/development/utilities.md
+++ b/doc/development/utilities.md
@@ -199,10 +199,26 @@ Refer to [`strong_memoize.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/maste
end
strong_memoize_attr :result
- strong_memoize_attr :enabled?, :enabled
def enabled?
Feature.enabled?(:some_feature)
end
+ strong_memoize_attr :enabled?, :enabled
+ end
+ ```
+
+ There's also `strong_memoize_with` to help memoize methods that take arguments.
+ This should be used for methods that have a low number of possible values
+ as arguments or with consistent repeating arguments in a loop.
+
+ ```ruby
+ class Find
+ include Gitlab::Utils::StrongMemoize
+
+ def result(basic: true)
+ strong_memoize_with(:result, basic) do
+ search(basic)
+ end
+ end
end
```
diff --git a/doc/development/windows.md b/doc/development/windows.md
index 5f32848da79..bf56e16344a 100644
--- a/doc/development/windows.md
+++ b/doc/development/windows.md
@@ -132,7 +132,7 @@ PowerShell has aliases for all of the following commands so you don't have to le
- `ls` ---> `dir`
- `rm` ---> `del`
- `rm -rf nonemptydir` ---> `rmdir /S nonemptydir`
-- `/` ---> `\` (path separator)
+- `/` ---> <code>&#92;</code> (path separator)
- `cat` ---> `type`
- `mv` ---> `move`
- Redirection works the same (for example, `>` and `2>&1`)
diff --git a/doc/development/work_items.md b/doc/development/work_items.md
index eabad175bf7..a417e1d1349 100644
--- a/doc/development/work_items.md
+++ b/doc/development/work_items.md
@@ -55,6 +55,7 @@ To avoid confusion and ensure communication is efficient, we will use the follow
| legacy issue view | The existing view used to render issues and incidents | | |
| issue | The existing issue model | | |
| issuable | Any model currently using the issueable module (issues, epics and MRs) | _Incidents are an **issuable**_ | _Incidents are a **work item type**_ |
+| widget | A UI element to present or allow interaction with specific work item data | | |
Some terms have been used in the past but have since become confusing and are now discouraged.
@@ -138,6 +139,20 @@ To introduce a new WIT there are two options:
### Work item type widgets
+A widget is a single component that can exist on a work item. This component can be used on one or
+many work item types and can be lightly customized at the point of implementation.
+
+A widget contains both the frontend UI (if present) and the associated logic for presenting and
+managing any data used by the widget. There can be a one-to-many connection between the data model
+and widgets. It means there can be multiple widgets that use or manage the same data, and they could
+be present at the same time (for example, a read-only summary widget and an editable detail widget,
+or two widgets showing two different filtered views of the same model).
+
+Widgets should be differentiated by their **purpose**. When possible, this purpose should be
+abstracted to the highest reasonable level to maximize reusability. For example, the widget for
+managing "tasks" was built as "child items". Rather than managing one type of child, it's abstracted
+up to managing any children.
+
All WITs will share the same pool of predefined widgets and will be customized by
which widgets are active on a specific WIT. Every attribute (column or association)
will become a widget with self-encapsulated functionality regardless of the WIT it belongs to.
@@ -164,13 +179,17 @@ define these various types in a very flexible manner. Having GitLab use
this system first (without introducing customer customization) allows us to
better build out the initial system.
-NOTE:
-Currently work item's `base_type` is used to define static mapping of what
+Work item's `base_type` is used to define static mapping of what
widgets are available for each type (current status), this definition should be
-rather stored in database table. The exact structure of the WIT widgets
-metadata is still to be defined. `base_type` was added to help converting other
-types of resources (requirements and incidents) into work items. Eventually (when
-these resources become regular work items), `base_type` will be removed.
+rather stored in a database table. The exact structure of the WIT widgets metadata
+is [still to be defined](https://gitlab.com/gitlab-org/gitlab/-/issues/370599).
+`base_type` was added to help convert other types of resources (requirements
+and incidents) into work items. Eventually (when these resources become regular
+work items), `base_type` will be removed.
+
+Until the architecture of WIT widgets is finalized, we are holding off on the creation of new work item
+types. If a new work item type is absolutely necessary, please reach out to a
+member of the [Project Management Engineering Team](https://gitlab.com/gitlab-org/gitlab/-/issues/370599).
### Custom work item types
diff --git a/doc/development/workhorse/index.md b/doc/development/workhorse/index.md
index 0f4e55a002a..91795336f78 100644
--- a/doc/development/workhorse/index.md
+++ b/doc/development/workhorse/index.md
@@ -21,7 +21,7 @@ but that repository is no longer used for development.
## Install Workhorse
-To install GitLab Workhorse you need [Go 1.15 or newer](https://go.dev/dl) and
+To install GitLab Workhorse you need [Go 1.18 or newer](https://go.dev/dl) and
[GNU Make](https://www.gnu.org/software/make/).
To install into `/usr/local/bin` run `make install`.
diff --git a/doc/drawers/advanced_search_syntax.md b/doc/drawers/advanced_search_syntax.md
index 3c2ce212b99..7556c8bdfaf 100644
--- a/doc/drawers/advanced_search_syntax.md
+++ b/doc/drawers/advanced_search_syntax.md
@@ -3,39 +3,42 @@ stage: Data Stores
group: Global Search
info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments"
type: drawer
-source: /doc/user/search/global_search/advanced_search_syntax.md
+source: /doc/user/search/advanced_search.md
---
# Search tips
<!-- markdownlint-disable -->
-| Use | Description | Example |
-|------|-------------|---------|
-| `"` | Exact search | [`"gem sidekiq"`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=%22gem+sidekiq%22) |
-| <code>&#124;</code> | Or | [<code>display &#124; banner</code>](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+%7C+banner) |
-| `+` | And | [`display +banner`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=display+%2Bbanner&snippets=) |
-| `-` | Exclude | [`display -banner`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+-banner) |
-| `*` | Partial | [`bug error 50*`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=bug+error+50%2A&snippets=) |
-| `\` | Escape | [`\*md`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=%5C*md&group_id=9970&project_id=278964) |
+| Syntax | Description | Example |
+|--------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `"` | Exact search | [`"gem sidekiq"`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=%22gem+sidekiq%22) |
+| <code>&#124;</code> | Or | [<code>display &#124; banner</code>](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+%7C+banner) |
+| `+` | And | [`display +banner`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=display+%2Bbanner&snippets=) |
+| `-` | Exclude | [`display -banner`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+-banner) |
+| `*` | Partial | [`bug error 50*`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=bug+error+50%2A&snippets=) |
+| `\` | Escape | [`\*md`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=%5C*md&group_id=9970&project_id=278964) |
+| `#` | Issue ID | [`#23456`](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=%2323456&group_id=9970&project_id=278964) |
+| `!` | Merge request ID | [`!23456`](https://gitlab.com/search?snippets=&scope=merge_requests&repository_ref=&search=%2123456&group_id=9970&project_id=278964) |
## Code search
-| Use | Description | Example |
-|------|-------------|---------|
-| `filename:` | Filename | [`filename:*spec.rb`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=filename%3A*spec.rb&group_id=9970&project_id=278964) |
-| `path:` | Repository location | [`path:spec/workers/`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=path%3Aspec%2Fworkers&snippets=) |
-| `extension:` | File extension, without the `.` | [`extension:js`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=extension%3Ajs&snippets=) |
-| `blob:` | Git object ID | [`blob:998707*`](https://gitlab.com/search?snippets=false&scope=blobs&repository_ref=&search=blob%3A998707*&group_id=9970) |
+| Syntax | Description | Example |
+|--------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `filename:` | Filename | [`filename:*spec.rb`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=filename%3A*spec.rb&group_id=9970&project_id=278964) |
+| `path:` | Repository location | [`path:spec/workers/`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=path%3Aspec%2Fworkers&snippets=) |
+| `extension:` | File extension without `.` | [`extension:js`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=extension%3Ajs&snippets=) |
+| `blob:` | Git object ID | [`blob:998707*`](https://gitlab.com/search?snippets=false&scope=blobs&repository_ref=&search=blob%3A998707*&group_id=9970) |
-`extension` and `blob` return exact matches only.
+`extension:` and `blob:` return exact matches only.
## Examples
-| Use | Description |
-|------|-------------|
-| [`rails -filename:gemfile.lock`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=rails+-filename%3Agemfile.lock&snippets=) | Show _rails_ in all files except the _`gemfile.lock`_ file. |
-| [`RSpec.describe Resolvers -*builder`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=RSpec.describe+Resolvers+-*builder) | Show all _RSpec.describe Resolvers_ that don't start with _builder_. |
-| [<code>bug &#124; (display +banner)</code>](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+%7C+%28display+%2Bbanner%29&group_id=9970&project_id=278964) | Show _bug_ **or** _display_ **and** _banner_. |
+| Query | Description |
+|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------|
+| [`rails -filename:gemfile.lock`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=rails+-filename%3Agemfile.lock&snippets=) | Returns `rails` in all files except the `gemfile.lock` file. |
+| [`RSpec.describe Resolvers -*builder`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=RSpec.describe+Resolvers+-*builder) | Returns `RSpec.describe Resolvers` that does not start with `builder`. |
+| [<code>bug &#124; (display +banner)</code>](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+%7C+%28display+%2Bbanner%29&group_id=9970&project_id=278964) | Returns `bug` or both `display` and `banner`. |
+| [<code>helper -extension:yml -extension:js</code>](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=helper+-extension%3Ayml+-extension%3Ajs&snippets=) | Returns `helper` in all files except files with a `.yml` or `.js` extension. |
<!-- markdownlint-enable -->
diff --git a/doc/gitlab-basics/add-file.md b/doc/gitlab-basics/add-file.md
index 64384372a44..95b8b59a48d 100644
--- a/doc/gitlab-basics/add-file.md
+++ b/doc/gitlab-basics/add-file.md
@@ -90,6 +90,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/gitlab-basics/command-line-commands.md b/doc/gitlab-basics/command-line-commands.md
index 4b53535a711..07ab9365693 100644
--- a/doc/gitlab-basics/command-line-commands.md
+++ b/doc/gitlab-basics/command-line-commands.md
@@ -118,6 +118,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/gitlab-basics/start-using-git.md b/doc/gitlab-basics/start-using-git.md
index 056fad4061b..25ef094b2a7 100644
--- a/doc/gitlab-basics/start-using-git.md
+++ b/doc/gitlab-basics/start-using-git.md
@@ -443,6 +443,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/install/aws/gitlab_hybrid_on_aws.md b/doc/install/aws/gitlab_hybrid_on_aws.md
index ad3d28d15e3..7ae4391dde0 100644
--- a/doc/install/aws/gitlab_hybrid_on_aws.md
+++ b/doc/install/aws/gitlab_hybrid_on_aws.md
@@ -46,7 +46,7 @@ The Beta version deploys Aurora PostgreSQL, but the release version will deploy
| | [AWS Quick Start for GitLab Cloud Native Hybrid on EKS](https://aws-quickstart.github.io/quickstart-eks-gitlab/) | [GitLab Environment Toolkit (GET)](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) |
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
-| Overview and Vision | [AWS Quick Start](https://aws.amazon.com/quickstart/) | [GitLab Environment Toolkit](https://gitlab.com/gitlab-org/gitlab-environment-toolkit/-/blob/main/README.md) |
+| Overview and Vision | [AWS Quick Start](https://aws.amazon.com/quickstart/architecture/amazon-eks/) | [GitLab Environment Toolkit](https://gitlab.com/gitlab-org/gitlab-environment-toolkit/-/blob/main/README.md) |
| Licensing | [Open Source (Apache 2.0)](https://github.com/aws-quickstart/quickstart-eks-gitlab/blob/main/LICENSE.txt) | [GitLab Enterprise Edition license](https://gitlab.com/gitlab-org/gitlab-environment-toolkit/-/blob/main/LICENSE) ([GitLab Premium tier](https://gitlab.com/gitlab-org/gitlab-environment-toolkit/-/blob/main/README.md)) |
| GitLab Support | [GitLab Beta Support](../../policy/alpha-beta-support.md#beta-features) | [GitLab GA Support](../../policy/alpha-beta-support.md#generally-available-ga) |
| GitLab Reference Architecture Compliant | Yes | Yes |
diff --git a/doc/install/aws/manual_install_aws.md b/doc/install/aws/manual_install_aws.md
index 7dbb245dd99..5d8138e4705 100644
--- a/doc/install/aws/manual_install_aws.md
+++ b/doc/install/aws/manual_install_aws.md
@@ -147,7 +147,7 @@ We now create a VPC, a virtual networking environment that you control:
![Create VPC](img/create_vpc.png)
-1. Select the VPC, select **Actions**, select **Edit DNS resolution**, and enable DNS resolution. Hit **Save** when done.
+1. Select the VPC, select **Actions**, select **Edit DNS resolution**, and enable DNS resolution. Select **Save** when done.
### Subnets
@@ -226,7 +226,7 @@ it receive traffic from any destination.
route to show the options at the bottom.
1. Select the **Routes** tab, select **Edit routes > Add route** and set `0.0.0.0/0`
as the destination. In the target column, select the `gitlab-gateway` we created previously.
- Hit **Save routes** once done.
+ Select **Save routes** when done.
Next, we must associate the **public** subnets to the route table:
diff --git a/doc/install/docker.md b/doc/install/docker.md
index 93988454a27..11525842c6e 100644
--- a/doc/install/docker.md
+++ b/doc/install/docker.md
@@ -31,7 +31,7 @@ to community resources (such as IRC or forums) to seek help from other users.
## Prerequisites
-Docker is required. See the [official installation documentation](https://docs.docker.com/install/).
+Docker is required. See the [official installation documentation](https://docs.docker.com/get-docker/).
## Set up the volumes location
@@ -138,7 +138,7 @@ With [Docker Compose](https://docs.docker.com/compose/) you can easily configure
install, and upgrade your Docker-based GitLab installation:
1. [Install Docker Compose](https://docs.docker.com/compose/install/).
-1. Create a `docker-compose.yml` file (or [download an example](https://gitlab.com/gitlab-org/omnibus-gitlab/raw/master/docker/docker-compose.yml)):
+1. Create a `docker-compose.yml` file:
```yaml
version: '3.6'
@@ -211,7 +211,7 @@ and [Docker configurations](https://docs.docker.com/engine/swarm/configs/) to ef
Secrets can be used to securely pass your initial root password without exposing it as an environment variable.
Configurations can help you to keep your GitLab image as generic as possible.
-Here's an example that deploys GitLab with four runners as a [stack](https://docs.docker.com/get-started/part5/), using secrets and configurations:
+Here's an example that deploys GitLab with four runners as a [stack](https://docs.docker.com/get-started/swarm-deploy/#describe-apps-using-stack-files), using secrets and configurations:
1. [Set up a Docker swarm](https://docs.docker.com/engine/swarm/swarm-tutorial/).
1. Create a `docker-compose.yml` file:
diff --git a/doc/install/google_cloud_platform/index.md b/doc/install/google_cloud_platform/index.md
index 7ba1fbad5ea..d16ac3e2174 100644
--- a/doc/install/google_cloud_platform/index.md
+++ b/doc/install/google_cloud_platform/index.md
@@ -143,6 +143,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/install/installation.md b/doc/install/installation.md
index 6108ec8d0e0..c00a959e037 100644
--- a/doc/install/installation.md
+++ b/doc/install/installation.md
@@ -46,12 +46,12 @@ If the highest number stable branch is unclear, check the [GitLab blog](https://
## Software requirements
-| Software | Minimum version | Notes |
-| -------- | --------------- | ----- |
-| [Ruby](#2-ruby) | `2.7` | From GitLab 13.6, Ruby 2.7 is required. Ruby 3.0 is not supported yet (see [the relevant epic](https://gitlab.com/groups/gitlab-org/-/epics/5149) for the current status). You must use the standard MRI implementation of Ruby. We love [JRuby](https://www.jruby.org/) and [Rubinius](https://github.com/rubinius/rubinius#the-rubinius-language-platform), but GitLab needs several Gems that have native extensions. |
-| [Go](#3-go) | `1.17` | From GitLab 15.2, Go 1.17 or later is required. |
-| [Git](#git) | `2.33.x` | From GitLab 14.4, Git 2.33.x and later is required. It's highly recommended that you use the [Git version provided by Gitaly](#git). |
-| [Node.js](#4-node) | `14.15.0` | GitLab uses [webpack](https://webpack.js.org/) to compile frontend assets. Node.js 16.x is recommended, as it's faster. You can check which version you're running with `node -v`. You must update it to a newer version if needed. |
+| Software | Minimum version | Notes |
+| ------------------ | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| [Ruby](#2-ruby) | `2.7` | From GitLab 13.6, Ruby 2.7 is required. Ruby 3.0 is not supported yet (see [the relevant epic](https://gitlab.com/groups/gitlab-org/-/epics/5149) for the current status). You must use the standard MRI implementation of Ruby. We love [JRuby](https://www.jruby.org/) and [Rubinius](https://github.com/rubinius/rubinius#the-rubinius-language-platform), but GitLab needs several Gems that have native extensions. |
+| [Go](#3-go) | `1.18` | From GitLab 15.6, Go 1.18 or later is required. |
+| [Git](#git) | `2.37.x` | From GitLab 15.6, Git 2.37.x and later is required. It's highly recommended that you use the [Git version provided by Gitaly](#git). |
+| [Node.js](#4-node) | `14.15.0` | GitLab uses [webpack](https://webpack.js.org/) to compile frontend assets. Node.js 16.x is recommended, as it's faster. You can check which version you're running with `node -v`. You must update it to a newer version if needed. |
## GitLab directory structure
@@ -250,11 +250,11 @@ Linux. You can find downloads for other platforms at the
# Remove former Go installation folder
sudo rm -rf /usr/local/go
-curl --remote-name --location --progress-bar "https://go.dev/dl/go1.17.10.linux-amd64.tar.gz"
-echo '87fc728c9c731e2f74e4a999ef53cf07302d7ed3504b0839027bd9c10edaa3fd go1.17.10.linux-amd64.tar.gz' | shasum -a256 -c - && \
- sudo tar -C /usr/local -xzf go1.17.10.linux-amd64.tar.gz
+curl --remote-name --location --progress-bar "https://go.dev/dl/go1.18.8.linux-amd64.tar.gz"
+echo '4d854c7bad52d53470cf32f1b287a5c0c441dc6b98306dea27358e099698142a go1.18.8.linux-amd64.tar.gz' | shasum -a256 -c - && \
+ sudo tar -C /usr/local -xzf go1.18.8.linux-amd64.tar.gz
sudo ln -sf /usr/local/go/bin/{go,gofmt} /usr/local/bin/
-rm go1.17.10.linux-amd64.tar.gz
+rm go1.18.8.linux-amd64.tar.gz
```
## 4. Node
@@ -864,12 +864,6 @@ Check if GitLab and its environment are configured correctly:
sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
```
-### Compile GetText PO files
-
-```shell
-sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production
-```
-
### Compile Assets
```shell
diff --git a/doc/install/migrate/compare_sm_to_saas.md b/doc/install/migrate/compare_sm_to_saas.md
index e12861632b8..db5bb54386f 100644
--- a/doc/install/migrate/compare_sm_to_saas.md
+++ b/doc/install/migrate/compare_sm_to_saas.md
@@ -83,7 +83,7 @@ In a self-managed instance:
On GitLab SaaS:
-- You cannot use internal encryption key for the data store ([bring-your-own-key](https://about.gitlab.com/handbook/engineering/security/vulnerability_management/encryption-policy.html#rolling-your-own-crypto)).
+- You cannot use internal encryption key for the data store ([bring-your-own-key](https://about.gitlab.com/handbook/security/threat-management/vulnerability-management/encryption-policy.html#rolling-your-own-crypto)).
- You cannot view console logs.
- You cannot enforce jobs on every pipeline across the group or organization.
- You cannot configure or control data backups. You must use [group](../../api/group_import_export.md) and [project](../../api/project_import_export.md) export.
@@ -106,7 +106,7 @@ In a self-managed instance, you control the encryption type and configuration.
On GitLab SaaS:
-- An [Access Management Process](https://about.gitlab.com/handbook/engineering/security/#access-management-process) is in place.
+- An [Access Management Process](https://about.gitlab.com/handbook/security/#access-management-process) is in place.
- All data on GitLab.com is encrypted at rest by default. Access to encryption keys is strictly managed by GitLab.
- GitLab does not access your tenant data except as part of a verified service request from you.
diff --git a/doc/install/relative_url.md b/doc/install/relative_url.md
index 3fe34f6a9b0..6f4221f9e2e 100644
--- a/doc/install/relative_url.md
+++ b/doc/install/relative_url.md
@@ -134,6 +134,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/install/requirements.md b/doc/install/requirements.md
index 8a1533dc268..f581a1c50f9 100644
--- a/doc/install/requirements.md
+++ b/doc/install/requirements.md
@@ -55,7 +55,7 @@ Memory requirements are dependent on the number of users and expected workload.
The following is the recommended minimum Memory hardware guidance for a handful of example GitLab user base sizes.
- **4GB RAM** is the **required** minimum memory size and supports up to 500 users
- - Our [Memory Team](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/memory/) is working to reduce the memory requirement.
+ - Our [Memory Team](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/application_performance/) is working to reduce the memory requirement.
- 8GB RAM supports up to 1000 users
- More users? Consult the [reference architectures page](../administration/reference_architectures/index.md)
@@ -299,7 +299,7 @@ GitLab supports the following web browsers:
- [Google Chrome](https://www.google.com/chrome/)
- [Chromium](https://www.chromium.org/getting-involved/dev-channel/)
- [Apple Safari](https://www.apple.com/safari/)
-- [Microsoft Edge](https://www.microsoft.com/en-us/edge)
+- [Microsoft Edge](https://www.microsoft.com/en-us/edge?form=MA13FJ)
For the listed web browsers, GitLab supports:
@@ -322,6 +322,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/integration/advanced_search/elasticsearch.md b/doc/integration/advanced_search/elasticsearch.md
index 87b812f3f9b..94493aa6958 100644
--- a/doc/integration/advanced_search/elasticsearch.md
+++ b/doc/integration/advanced_search/elasticsearch.md
@@ -251,7 +251,7 @@ any subgroups and projects belonging to those subgroups to be indexed as well.
Advanced Search only provides cross-group code/commit search (global) if all name-spaces are indexed. In this particular scenario where only a subset of namespaces are indexed, a global search does not provide a code or commit scope. This is possible only in the scope of an indexed namespace. There is no way to code/commit search in multiple indexed namespaces (when only a subset of namespaces has been indexed). For example if two groups are indexed, there is no way to run a single code search on both. You can only run a code search on the first group and then on the second.
-You can filter the selection dropdown by writing part of the namespace or project name you're interested in.
+You can filter the selection dropdown list by writing part of the namespace or project name you're interested in.
![limit namespace filter](img/limit_namespace_filter.png)
diff --git a/doc/integration/advanced_search/elasticsearch_troubleshooting.md b/doc/integration/advanced_search/elasticsearch_troubleshooting.md
index 7136c273a2a..aa6613d6f1a 100644
--- a/doc/integration/advanced_search/elasticsearch_troubleshooting.md
+++ b/doc/integration/advanced_search/elasticsearch_troubleshooting.md
@@ -212,8 +212,8 @@ More [complex Elasticsearch API calls](https://www.elastic.co/guide/en/elasticse
If the results:
-- Sync up, please check that you are using [supported syntax](../../user/search/global_search/advanced_search_syntax.md). Note that Advanced Search does not support [exact substring matching](https://gitlab.com/gitlab-org/gitlab/-/issues/325234).
-- Do not match up, this indicates a problem with the documents generated from the project. It is best to [re-index that project](../advanced_search/elasticsearch.md#indexing-a-range-of-projects-or-a-specific-project)
+- Sync up, check that you are using [supported syntax](../../user/search/advanced_search.md#syntax). Advanced Search does not support [exact substring matching](https://gitlab.com/gitlab-org/gitlab/-/issues/325234).
+- Do not match up, this indicates a problem with the documents generated from the project. It is best to [re-index that project](../advanced_search/elasticsearch.md#indexing-a-range-of-projects-or-a-specific-project).
NOTE:
The above instructions are not to be used for scenarios that only index a [subset of namespaces](elasticsearch.md#limit-the-number-of-namespaces-and-projects-that-can-be-indexed).
diff --git a/doc/integration/bitbucket.md b/doc/integration/bitbucket.md
index 38d8f0049db..8019eccc421 100644
--- a/doc/integration/bitbucket.md
+++ b/doc/integration/bitbucket.md
@@ -38,7 +38,7 @@ you to use.
The URL to your GitLab installation, such as
`https://gitlab.example.com/users/auth`.
Leaving this field empty
- [results in an `Invalid redirect_uri` message](https://confluence.atlassian.com/bitbucket/oauth-faq-338365710.html).
+ results in an `Invalid redirect_uri` message.
WARNING:
To help prevent an [OAuth 2 covert redirect](https://oauth.net/advisories/2014-1-covert-redirect/)
diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md
index 0658c921610..1b0a1e50445 100644
--- a/doc/integration/gitlab.md
+++ b/doc/integration/gitlab.md
@@ -19,10 +19,11 @@ GitLab.com generates an application ID and secret key for you to use.
- Name: This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or something else descriptive.
- Redirect URI:
- ```plaintext
- http://your-gitlab.example.com/import/gitlab/callback
- http://your-gitlab.example.com/users/auth/gitlab/callback
- ```
+ ```plaintext
+ # You can also use a non-SSL URL, but you should use SSL URLs.
+ https://your-gitlab.example.com/import/gitlab/callback
+ https://your-gitlab.example.com/users/auth/gitlab/callback
+ ```
The first link is required for the importer and second for authentication.
diff --git a/doc/integration/gitpod.md b/doc/integration/gitpod.md
index 26d0da4b49d..0088d4b886d 100644
--- a/doc/integration/gitpod.md
+++ b/doc/integration/gitpod.md
@@ -44,7 +44,7 @@ With the Gitpod integration enabled for your GitLab instance, to enable it for y
For GitLab self-managed instances, a GitLab administrator needs to:
-1. Set up a Gitpod instance to integrate with GitLab. Refer to the [Gitpod documentation](https://www.gitpod.io/docs/self-hosted/latest)
+1. Set up a Gitpod instance to integrate with GitLab. Refer to the [Gitpod documentation](https://www.gitpod.io/docs/configure/self-hosted/latest)
to get your instance up and running.
1. Enable it in GitLab:
1. On the top bar, select **Main menu > Admin**.
diff --git a/doc/integration/index.md b/doc/integration/index.md
index 147edcc9e0f..b2a4201e88c 100644
--- a/doc/integration/index.md
+++ b/doc/integration/index.md
@@ -7,56 +7,67 @@ comments: false
# GitLab integrations **(FREE)**
-GitLab can be integrated with external services for enhanced functionality.
+You can integrate GitLab with external services for enhanced functionality.
## Services
-Services such as Campfire, Flowdock, Jira, Pivotal Tracker, and Slack are available as [integrations](../user/project/integrations/index.md).
+Services such as Campfire, Flowdock, Jira, Pivotal Tracker, and Slack
+are available as [integrations](../user/project/integrations/index.md).
## Issue trackers
-You can use an [external issue tracker](external-issue-tracker.md) at the same time as the GitLab
-issue tracker, or use only the external issue tracker.
+You can use an [external issue tracker](external-issue-tracker.md) with the GitLab
+issue tracker or use an external issue tracker only.
## Authentication sources
-GitLab can be configured to authenticate access requests with the following authentication sources:
+You can integrate GitLab with the following authentication sources:
- Enable the [Auth0 OmniAuth](auth0.md) provider.
-- Enable sign in with [Bitbucket](bitbucket.md) accounts.
-- Integrate with [Kerberos](kerberos.md).
-- Enable sign in via [LDAP](../administration/auth/ldap/index.md).
-- Enable [OAuth2 provider](oauth_provider.md) application creation.
-- Use [OmniAuth](omniauth.md) to enable sign in through Twitter, GitHub, GitLab.com, Google,
- Bitbucket, Facebook, SAML, Crowd, Azure, or Authentiq ID.
+- Enable sign-in with [Bitbucket](bitbucket.md) accounts.
+- Authenticate with [Kerberos](kerberos.md).
+- Enable sign-in with [LDAP](../administration/auth/ldap/index.md).
+- Enable creating [OAuth 2.0](oauth_provider.md) applications.
+- Use [OmniAuth](omniauth.md) to enable sign-in through:
+ - Authentiq ID
+ - Azure
+ - Bitbucket
+ - Crowd
+ - Facebook
+ - GitHub
+ - GitLab.com
+ - Google
+ - SAML
+ - Twitter
- Use GitLab as an [OpenID Connect](openid_connect_provider.md) identity provider.
-- Authenticate to [Vault](vault.md) through GitLab OpenID Connect.
-- Configure GitLab as a [SAML](saml.md) 2.0 Service Provider.
+- Authenticate with [Vault](vault.md) through GitLab OpenID Connect.
+- Configure GitLab as a [SAML 2.0](saml.md) Service Provider.
## Security enhancements
-GitLab can be integrated with the following external services to enhance security:
+You can integrate GitLab with the following security enhancements:
-- [Akismet](akismet.md) helps reduce spam.
-- Google [reCAPTCHA](recaptcha.md) helps verify new users.
+- [Akismet](akismet.md) to reduce spam.
+- Google [reCAPTCHA](recaptcha.md) to verify new users.
-GitLab also provides features to improve the security of your own application. For more details see [GitLab Secure](../user/application_security/index.md).
+GitLab also provides features to improve the security of your own application.
+For more details, see [Secure your application](../user/application_security/index.md).
## Security partners
-GitLab has integrated with several security partners. For more information, see
-[Security partners integration](security_partners/index.md).
+You can integrate GitLab with several security partners. For more information, see
+[Security partner integrations](security_partners/index.md).
## Continuous integration
-GitLab can be integrated with the following external services for continuous integration:
+You can integrate GitLab with the following external services for continuous integration:
- [Jenkins](jenkins.md) CI.
-- [Datadog](datadog.md), to monitor for CI/CD job failures and performance issues.
+- [Datadog](datadog.md) to monitor for CI/CD job failures and performance issues.
## Feature enhancements
-GitLab can be integrated with the following enhancements:
+You can integrate GitLab with the following feature enhancements:
- Add GitLab actions to [Gmail actions buttons](gmail_action_buttons_for_gitlab.md).
- Configure [PlantUML](../administration/integration/plantuml.md)
@@ -69,40 +80,28 @@ or [Kroki](../administration/integration/kroki.md) to use diagrams in AsciiDoc a
### SSL certificate errors
-When trying to integrate GitLab with services using self-signed certificates,
-SSL certificate errors can occur in different parts of the application. Sidekiq
-is a common culprit.
+When integrating GitLab with services using a self-signed certificate, you might
+encounter SSL certificate errors in different parts of the application.
-There are two approaches you can take to solve this:
+As a workaround, you can do one of the following:
-1. Add the root certificate to the trusted chain of the OS.
-1. If using Omnibus, you can add the certificate to the GitLab trusted certificates.
+- Add the certificate to the OS trusted chain. See:
+ - [Adding trusted root certificates to the server](https://manuals.gfi.com/en/kerio/connect/content/server-configuration/ssl-certificates/adding-trusted-root-certificates-to-the-server-1605.html)
+ - [How do you add a certificate authority (CA) to Ubuntu?](https://superuser.com/questions/437330/how-do-you-add-a-certificate-authority-ca-to-ubuntu)
+- In Omnibus GitLab, add the certificate to the Omnibus trusted chain:
+ 1. [Install the self-signed certificate](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates).
+ 1. Concatenate the self-signed certificate with the GitLab trusted certificate.
+ The self-signed certificate might be overwritten during upgrades.
-**OS main trusted chain**
+ ```shell
+ cat jira.pem >> /opt/gitlab/embedded/ssl/certs/cacert.pem
+ ```
-This [resource](https://manuals.gfi.com/en/kerio/connect/content/server-configuration/ssl-certificates/adding-trusted-root-certificates-to-the-server-1605.html)
-has all the information you need to add a certificate to the main trusted chain.
+ 1. Restart GitLab.
-This [answer](https://superuser.com/questions/437330/how-do-you-add-a-certificate-authority-ca-to-ubuntu)
-at Super User also has relevant information.
-
-**Omnibus Trusted Chain**
-
-[Install the self signed certificate or custom certificate authorities](https://docs.gitlab.com/omnibus/troubleshooting.html#using-self-signed-certificate-or-custom-certificate-authorities)
-in to Omnibus GitLab.
-
-It is enough to concatenate the certificate to the main trusted certificate
-however it may be overwritten during upgrades:
-
-```shell
-cat jira.pem >> /opt/gitlab/embedded/ssl/certs/cacert.pem
-```
-
-After that restart GitLab with:
-
-```shell
-sudo gitlab-ctl restart
-```
+ ```shell
+ sudo gitlab-ctl restart
+ ```
### Search Sidekiq logs in Kibana
@@ -112,4 +111,9 @@ To locate a specific integration in Kibana, use the following KQL search string:
`json.integration_class.keyword : "Integrations::Jira" and json.project_path : "path/to/project"`
```
-You can find information in `json.exception.backtrace`, `json.exception.class`, `json.exception.message`, and `json.message`.
+You can find information in:
+
+- `json.exception.backtrace`
+- `json.exception.class`
+- `json.exception.message`
+- `json.message`
diff --git a/doc/integration/jenkins_deprecated.md b/doc/integration/jenkins_deprecated.md
deleted file mode 100644
index 53f7162402b..00000000000
--- a/doc/integration/jenkins_deprecated.md
+++ /dev/null
@@ -1,13 +0,0 @@
----
-stage: Manage
-group: Integrations
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-remove_date: '2022-10-29'
-redirect_to: 'jenkins.md'
----
-
-# Jenkins CI service (removed) **(FREE)**
-
-This feature was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/1600)
-in GitLab 13.0.
-Use the [Jenkins integration](jenkins.md) instead.
diff --git a/doc/integration/jira/configure.md b/doc/integration/jira/configure.md
index 66339d5ec27..98d296db170 100644
--- a/doc/integration/jira/configure.md
+++ b/doc/integration/jira/configure.md
@@ -52,7 +52,7 @@ To configure your project:
can view all issues from the specified Jira project.
1. To enable [issue creation for vulnerabilities](../../user/application_security/vulnerabilities/index.md#create-a-jira-issue-for-a-vulnerability), select **Enable Jira issue creation from vulnerabilities**.
-1. Select the **Jira issue type**. If the dropdown is empty, select refresh (**{retry}**) and try again.
+1. Select the **Jira issue type**. If the dropdown list is empty, select refresh (**{retry}**) and try again.
1. To verify the Jira connection is working, select **Test settings**.
1. Select **Save changes**.
diff --git a/doc/integration/jira/index.md b/doc/integration/jira/index.md
index 5daad4094f4..63cd472b0b9 100644
--- a/doc/integration/jira/index.md
+++ b/doc/integration/jira/index.md
@@ -79,102 +79,3 @@ This method provides more fine-grained access control because access can be rest
Developers have built several third-party Jira integrations for GitLab that are
listed on the [Atlassian Marketplace](https://marketplace.atlassian.com/search?product=jira&query=gitlab).
-
-## Troubleshooting
-
-If these features do not work as expected, it is likely due to a problem with the way the integration settings were configured.
-
-### GitLab cannot comment on a Jira issue
-
-If GitLab cannot comment on Jira issues, make sure the Jira user you
-set up for the integration has permission to:
-
-- Post comments on a Jira issue.
-- Transition the Jira issue.
-
-Jira issue references and update comments do not work if the GitLab issue tracker is disabled.
-
-If you [restrict IP addresses for Jira access](https://support.atlassian.com/security-and-access-policies/docs/specify-ip-addresses-for-product-access/), make sure you add your self-managed IP addresses or [GitLab.com IP range](../../user/gitlab_com/index.md#ip-range) to the allowlist in Jira.
-
-### GitLab cannot close a Jira issue
-
-If GitLab cannot close a Jira issue:
-
-- Make sure the `Transition ID` you set in the Jira settings matches the one
- your project needs to close an issue.
-
-- Make sure the Jira issue is not already marked as resolved:
- - Check the Jira issue resolution field is not set.
- - Check the issue is not struck through in Jira lists.
-
-### CAPTCHA
-
-CAPTCHA may be triggered after several consecutive failed login attempts,
-which may lead to a `401 unauthorized` error when testing your Jira integration.
-If CAPTCHA has been triggered, you can't use Jira's REST API to
-authenticate with the Jira site.
-
-To fix this error, sign in to your Jira instance
-and complete the CAPTCHA.
-
-### Jira integration does not work for imported project
-
-There is a [known bug](https://gitlab.com/gitlab-org/gitlab/-/issues/341571)
-where the Jira integration sometimes does not work for a project that has been imported.
-As a workaround, disable the integration and then re-enable it.
-
-### Bulk change all Jira integrations to Jira instance-level values
-
-To change all Jira projects to use instance-level integration settings:
-
-1. In a [Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session), run the following:
-
- ```ruby
- jira_integration_instance_id = Integrations::Jira.find_by(instance: true).id
- Integrations::Jira.where(active: true, instance: false, template: false, inherit_from_id: nil).find_each do |integration|
- integration.update_attribute(:inherit_from_id, jira_integration_instance_id)
- end
- ```
-
-1. Modify and save the instance-level integration from the UI to propagate the changes to all group-level and project-level integrations.
-
-### Check if Jira Cloud is linked
-
-You can use the [Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session) to check if Jira Cloud is linked to:
-
-A specified namespace:
-
-```ruby
-JiraConnectSubscription.where(namespace: Namespace.by_path('group/subgroup'))
-```
-
-A specified project:
-
-```ruby
-Project.find_by_full_path('path/to/project').jira_subscription_exists?
-```
-
-Any namespace:
-
-```ruby
-installation = JiraConnectInstallation.find_by_base_url("https://customer_name.atlassian.net")
-installation.subscriptions
-```
-
-### Bulk update the service integration password for all projects
-
-To reset the Jira user's password for all projects with active Jira integrations,
-run the following in a [Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session):
-
-```ruby
-p = Project.find_by_sql("SELECT p.id FROM projects p LEFT JOIN services s ON p.id = s.project_id WHERE s.type = 'JiraService' AND s.active = true")
-
-p.each do |project|
- project.jira_integration.update_attribute(:password, '<your-new-password>')
-end
-```
-
-### `500 Whoops` when accessing a Jira issue in GitLab
-
-When accessing a Jira issue in GitLab, you might get a `500 Whoops, something went wrong on our end` error.
-Check [`production.log`](../../administration/logs/index.md#productionlog) to see if it contains a `:NoMethodError (undefined method 'duedate' for #<JIRA::Resource::Issue:0x00007f406d7b3180>)` exception. If that's the case, ensure the **Due date** field is visible for issues in the integrated Jira project.
diff --git a/doc/integration/jira/jira_server_configuration.md b/doc/integration/jira/jira_server_configuration.md
index 42de883753c..fd8e018fd0c 100644
--- a/doc/integration/jira/jira_server_configuration.md
+++ b/doc/integration/jira/jira_server_configuration.md
@@ -54,10 +54,12 @@ This process adds the `gitlab` user you created to a new group named `gitlab-dev
1. To add the `gitlab` user to the `gitlab-developers` group, select **Edit members**.
The `gitlab-developers` group should be listed in the leftmost box as a
selected group.
+<!-- vale gitlab.BadPlurals = NO -->
1. In the **Add members to selected group(s)** section, enter `gitlab`.
1. Select **Add selected users**.
The `gitlab` user appears in the **Group member(s)**
section.
+<!-- vale gitlab.BadPlurals = YES -->
![Jira added user to group](img/jira_added_user_to_group.png)
diff --git a/doc/integration/jira/troubleshooting.md b/doc/integration/jira/troubleshooting.md
new file mode 100644
index 00000000000..0e679693614
--- /dev/null
+++ b/doc/integration/jira/troubleshooting.md
@@ -0,0 +1,110 @@
+---
+stage: Manage
+group: Integrations
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Troubleshooting Jira integrations **(FREE)**
+
+This page contains a list of common issues you might encounter when working with Jira integrations.
+
+## GitLab cannot comment on a Jira issue
+
+If GitLab cannot comment on Jira issues, make sure the Jira user you
+set up for the integration has permission to:
+
+- Post comments on a Jira issue.
+- Transition the Jira issue.
+
+Jira issue references and update comments do not work if the GitLab issue tracker is disabled.
+
+If you [restrict IP addresses for Jira access](https://support.atlassian.com/security-and-access-policies/docs/specify-ip-addresses-for-product-access/), make sure you add your self-managed IP addresses or [GitLab.com IP range](../../user/gitlab_com/index.md#ip-range) to the allowlist in Jira.
+
+## GitLab cannot close a Jira issue
+
+If GitLab cannot close a Jira issue:
+
+- Make sure the `Transition ID` you set in the Jira settings matches the one
+ your project needs to close an issue.
+
+- Make sure the Jira issue is not already marked as resolved:
+ - Check the Jira issue resolution field is not set.
+ - Check the issue is not struck through in Jira lists.
+
+## CAPTCHA
+
+CAPTCHA might be triggered after several consecutive failed login attempts,
+which might lead to a `401 unauthorized` error when testing your Jira integration.
+If CAPTCHA has been triggered, you can't use the Jira REST API to
+authenticate with the Jira site.
+
+To fix this error, sign in to your Jira instance
+and complete the CAPTCHA.
+
+## Jira integration does not work for imported project
+
+There is a [known bug](https://gitlab.com/gitlab-org/gitlab/-/issues/341571)
+where the Jira integration sometimes does not work for a project that has been imported.
+As a workaround, disable the integration and then re-enable it.
+
+## Bulk change all Jira integrations to Jira instance-level values
+
+To change all Jira projects to use instance-level integration settings:
+
+1. In a [Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session), run the following:
+
+ ```ruby
+ jira_integration_instance_id = Integrations::Jira.find_by(instance: true).id
+ Integrations::Jira.where(active: true, instance: false, template: false, inherit_from_id: nil).find_each do |integration|
+ integration.update_attribute(:inherit_from_id, jira_integration_instance_id)
+ end
+ ```
+
+1. Modify and save the instance-level integration from the UI to propagate the changes to all group-level and project-level integrations.
+
+## Check if Jira Cloud is linked
+
+You can use the [Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session) to check if Jira Cloud is linked to:
+
+A specified namespace:
+
+```ruby
+JiraConnectSubscription.where(namespace: Namespace.by_path('group/subgroup'))
+```
+
+A specified project:
+
+```ruby
+Project.find_by_full_path('path/to/project').jira_subscription_exists?
+```
+
+Any namespace:
+
+```ruby
+installation = JiraConnectInstallation.find_by_base_url("https://customer_name.atlassian.net")
+installation.subscriptions
+```
+
+## Bulk update the service integration password for all projects
+
+To reset the Jira user's password for all projects with active Jira integrations,
+run the following in a [Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session):
+
+```ruby
+p = Project.find_by_sql("SELECT p.id FROM projects p LEFT JOIN services s ON p.id = s.project_id WHERE s.type = 'JiraService' AND s.active = true")
+
+p.each do |project|
+ project.jira_integration.update_attribute(:password, '<your-new-password>')
+end
+```
+
+## `500 Whoops` when accessing a Jira issue in GitLab
+
+When accessing a Jira issue in GitLab, you might get a `500 Whoops, something went wrong on our end` error.
+Check [`production.log`](../../administration/logs/index.md#productionlog) to see if it contains the following exception:
+
+```plaintext
+:NoMethodError (undefined method 'duedate' for #<JIRA::Resource::Issue:0x00007f406d7b3180>)
+```
+
+If that's the case, ensure the **Due date** field is visible for issues in the integrated Jira project.
diff --git a/doc/integration/oauth_provider.md b/doc/integration/oauth_provider.md
index 964c5edcbc1..bedcbf23163 100644
--- a/doc/integration/oauth_provider.md
+++ b/doc/integration/oauth_provider.md
@@ -38,7 +38,7 @@ GitLab supports several ways of adding a new OAuth 2 application to an instance:
- [Instance-wide applications](#instance-wide-applications)
The only difference between these methods is the [permission](../user/permissions.md)
-levels. The default callback URL is `http://your-gitlab.example.com/users/auth/gitlab/callback`.
+levels. The default callback URL is `https://your-gitlab.example.com/users/auth/gitlab/callback` (you can also use a non-SSL URL, but you should use SSL URLs).
## User owned applications
@@ -137,17 +137,3 @@ On self-managed GitLab, by default, this feature is not available. To make it av
On GitLab.com, this feature is not available.
By default, OAuth application secrets are stored as plain text in the database. When enabled, OAuth application secrets are stored in the database in hashed format and are only available to users immediately after creating OAuth applications.
-
-## Hashed OAuth tokens
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/364110) in GitLab 15.3 [with a flag](../administration/feature_flags.md) named `hash_oauth_tokens`. Enabled on GitLab.com. Disabled by default for self-managed.
-> - [Enabled by default on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 15.5.
-
-FLAG:
-On self-managed GitLab, by default, this feature is enabled. If you detect a problem, ask an administrator to
-[disable the feature flag](../administration/feature_flags.md) named `hash_oauth_tokens`. If the feature flag is disabled, any tokens that were stored
-in encrypted format are inaccessible. Users must reauthorize applications.
-On GitLab.com, this feature is enabled.
-
-By default, OAuth access tokens are stored in the database in PBKDF2+SHA512 format. GitLab administrators can disable this and OAuth access tokens are
-then stored in plaintext in the database.
diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md
index 55d1d1bcbb8..af039c8a009 100644
--- a/doc/integration/omniauth.md
+++ b/doc/integration/omniauth.md
@@ -43,23 +43,19 @@ GitLab supports the following OmniAuth providers.
Before you configure the OmniAuth provider,
configure the settings that are common for all providers.
-Setting | Description | Default value
----------------------------|-------------|--------------
-`allow_single_sign_on` | Enables you to list the providers that automatically create a GitLab account. The provider names are available in the **OmniAuth provider name** column in the [supported providers table](#supported-providers). | The default is `false`. If `false`, users must be created manually, or they can't sign in using OmniAuth.
-`auto_link_ldap_user` | If enabled, creates an LDAP identity in GitLab for users that are created through an OmniAuth provider. You can enable this setting if you have [LDAP integration](../administration/auth/ldap/index.md) enabled. Requires the `uid` of the user to be the same in both LDAP and the OmniAuth provider. | The default is `false`.
-`block_auto_created_users` | If enabled, blocks users that are automatically created from signing in until they are approved by an administrator. | The default is `true`. If you set the value to `false`, make sure you only define providers for `allow_single_sign_on` that you can control, like SAML or Google. Otherwise, any user on the internet can sign in to GitLab without an administrator's approval.
+Omnibus, Docker, and source | Helm chart | Description | Default value
+----------------------------|------------|-------------|-----------
+`allow_single_sign_on` | `allowSingleSignOn` | List of providers that automatically create a GitLab account. The provider names are available in the **OmniAuth provider name** column in the [supported providers table](#supported-providers). | `false`, which means that signing in using your OmniAuth provider account without a pre-existing GitLab account is not allowed. You must create a GitLab account first, and then connect it to your OmniAuth provider account through your profile settings.
+`auto_link_ldap_user` | `autoLinkLdapUser` | Creates an LDAP identity in GitLab for users that are created through an OmniAuth provider. You can enable this setting if you have [LDAP integration](../administration/auth/ldap/index.md) enabled. Requires the `uid` of the user to be the same in both LDAP and the OmniAuth provider. | `false`
+`block_auto_created_users` | `blockAutoCreatedUsers` | Blocks users that are automatically created from signing in until they are approved by an administrator. | `true`. If you set the value to `false`, make sure you define providers that you can control, like SAML or Google. Otherwise, any user on the internet can sign in to GitLab without an administrator's approval.
To change these settings:
-- **For Omnibus package**
-
- 1. Open the configuration file:
+ ::Tabs
- ```shell
- sudo editor /etc/gitlab/gitlab.rb
- ```
+ :::TabTitle Omnibus
- 1. Update the following section:
+ 1. Edit `/etc/gitlab/gitlab.rb` and update the following section:
```ruby
# CAUTION!
@@ -71,13 +67,47 @@ To change these settings:
gitlab_rails['omniauth_block_auto_created_users'] = true
```
-- **For installations from source**
+ 1. Reconfigure GitLab:
+
+ ```shell
+ sudo gitlab-ctl reconfigure
+ ```
+
+ :::TabTitle Helm chart
+
+ 1. Export the Helm values:
+
+ ```shell
+ helm get values gitlab > gitlab_values.yaml
+ ```
+
+ 1. Edit `gitlab_values.yaml`, and update the `omniauth` section under `globals.appConfig`:
+
+ ```yaml
+ global:
+ appConfig:
+ omniauth:
+ enabled: true
+ allowSingleSignOn: ['saml', 'twitter']
+ autoLinkLdapUser: false
+ blockAutoCreatedUsers: true
+ ```
+
+ For more details, see the
+ [globals documentation](https://docs.gitlab.com/charts/charts/globals.html#omniauth).
+
+ 1. Apply the new values:
+
+ ```shell
+ helm upgrade -f gitlab_values.yaml gitlab gitlab/gitlab
+ ```
+
+ :::TabTitle Source
1. Open the configuration file:
```shell
cd /home/git/gitlab
-
sudo -u git -H editor config/gitlab.yml
```
@@ -102,6 +132,14 @@ To change these settings:
block_auto_created_users: true
```
+ 1. Restart GitLab:
+
+ ```shell
+ sudo service gitlab restart
+ ```
+
+ ::EndTabs
+
After configuring these settings, you can configure
your chosen [provider](#supported-providers).
diff --git a/doc/integration/saml.md b/doc/integration/saml.md
index 0f7f3e336ef..fd01e9e0e56 100644
--- a/doc/integration/saml.md
+++ b/doc/integration/saml.md
@@ -660,7 +660,7 @@ balancer and include sensitive details in assertions that you do not want appear
in logs. Most organizations should not need additional encryption at this layer.
The SAML integration supports EncryptedAssertion. You should define the private
-key and the public certificate of your GitLab instance in the SAML settings:
+key and the public certificate of your GitLab instance in the SAML settings. When you define the key and certificate, replace all line feeds in the key file with `\n`. This makes the key file one long string with no line feeds.
```yaml
args: {
@@ -669,12 +669,8 @@ args: {
idp_sso_target_url: 'https://login.example.com/idp',
issuer: 'https://gitlab.example.com',
name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent',
- certificate: '-----BEGIN CERTIFICATE-----
- <redacted>
- -----END CERTIFICATE-----',
- private_key: '-----BEGIN PRIVATE KEY-----
- <redacted>
- -----END PRIVATE KEY-----'
+ certificate: '-----BEGIN CERTIFICATE-----\n<redacted>\n-----END CERTIFICATE-----',
+ private_key: '-----BEGIN PRIVATE KEY-----\n<redacted>\n-----END PRIVATE KEY-----'
}
```
@@ -703,12 +699,8 @@ args: {
idp_sso_target_url: 'https://login.example.com/idp',
issuer: 'https://gitlab.example.com',
name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent',
- certificate: '-----BEGIN CERTIFICATE-----
- <redacted>
- -----END CERTIFICATE-----',
- private_key: '-----BEGIN PRIVATE KEY-----
- <redacted>
- -----END PRIVATE KEY-----',
+ certificate: '-----BEGIN CERTIFICATE-----\n<redacted>\n-----END CERTIFICATE-----',
+ private_key: '-----BEGIN PRIVATE KEY-----\n<redacted>\n-----END PRIVATE KEY-----',
security: {
authn_requests_signed: true, # enable signature on AuthNRequest
want_assertions_signed: true, # enable the requirement of signed assertion
diff --git a/doc/integration/slash_commands.md b/doc/integration/slash_commands.md
index ff892f006a5..5eefa1138aa 100644
--- a/doc/integration/slash_commands.md
+++ b/doc/integration/slash_commands.md
@@ -32,7 +32,7 @@ Assuming `project-name` is the trigger command, the slash commands are:
| `/project-name deploy <from> to <to>` | [Deploys](#deploy-command) from the `<from>` environment to the `<to>` environment. |
| `/project-name run <job name> <arguments>` | Executes the [ChatOps](../ci/chatops/index.md) job `<job name>` on the default branch. |
-If you are using the [GitLab Slack application](../user/project/integrations/gitlab_slack_application.md) for
+If you are using the [GitLab for Slack app](../user/project/integrations/gitlab_slack_application.md) for
your GitLab.com projects, [add the `gitlab` keyword at the beginning of the command](../user/project/integrations/gitlab_slack_application.md#usage).
## Issue commands
diff --git a/doc/integration/vault.md b/doc/integration/vault.md
index ad807f9eb7a..2226dc4cfd4 100644
--- a/doc/integration/vault.md
+++ b/doc/integration/vault.md
@@ -18,7 +18,7 @@ GitLab by using our OpenID authentication feature.
## Prerequisites
-1. [Install Vault](https://www.vaultproject.io/docs/install).
+1. [Install Vault](https://developer.hashicorp.com/vault/docs/install).
1. Run Vault.
## Get the OpenID Connect client ID and secret from GitLab
@@ -29,7 +29,7 @@ for authenticating into Vault. To do this, sign in to GitLab and follow these st
1. In the top-right corner, select your avatar.
1. Select **Edit profile**.
1. On the left sidebar, select **Applications**.
-1. Fill out the application **Name** and [**Redirect URI**](https://www.vaultproject.io/docs/auth/jwt#redirect-uris).
+1. Fill out the application **Name** and [**Redirect URI**](https://developer.hashicorp.com/vault/docs/auth/jwt#redirect-uris).
1. Select the **OpenID** scope.
1. Select **Save application**.
1. Copy the **Client ID** and **Client Secret**, or keep the page open for reference.
@@ -78,7 +78,7 @@ Success! Data written to: auth/oidc/config
## Write the OIDC role configuration
-You must tell Vault the [**Redirect URIs**](https://www.vaultproject.io/docs/auth/jwt#redirect-uris)
+You must tell Vault the [**Redirect URIs**](https://developer.hashicorp.com/vault/docs/auth/jwt#redirect-uris)
and scopes given to GitLab when you created the application.
Run the following command in the terminal:
@@ -128,7 +128,7 @@ Otherwise, anyone with a public account can access your Vault instance.
## Sign in using the Vault CLI (optional)
-You can also sign into Vault using the [Vault CLI](https://www.vaultproject.io/docs/commands).
+You can also sign into Vault using the [Vault CLI](https://developer.hashicorp.com/vault/docs/commands).
1. To sign in with the role configuration you created in the previous example,
run the following command in your terminal:
@@ -143,7 +143,7 @@ You can also sign into Vault using the [Vault CLI](https://www.vaultproject.io/d
- `-method=oidc` to set Vault to use the `OIDC` sign-in method.
- `port=8250` to set the port that GitLab should redirect to. This port
number must match the port given to GitLab when listing
- [Redirect URIs](https://www.vaultproject.io/docs/auth/jwt#redirect-uris).
+ [Redirect URIs](https://developer.hashicorp.com/vault/docs/auth/jwt#redirect-uris).
After running this command, you should see a link in the terminal.
diff --git a/doc/legal/corporate_contributor_license_agreement.md b/doc/legal/corporate_contributor_license_agreement.md
index 257b26dd7c4..d98c8e54964 100644
--- a/doc/legal/corporate_contributor_license_agreement.md
+++ b/doc/legal/corporate_contributor_license_agreement.md
@@ -3,28 +3,29 @@ stage: none
group: unassigned
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
+<!-- vale off -->
# Corporate contributor license agreement
You accept and agree to the following terms and conditions for Your present and future Contributions submitted to GitLab B.V.. Except for the license granted herein to GitLab B.V. and recipients of software distributed by GitLab B.V., You reserve all right, title, and interest in and to Your Contributions.
-- **Definitions:**
+"1." **Definitions:**
"You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with GitLab B.V.. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
"Contribution" shall mean the code, documentation or other original works of authorship, including any modifications or additions to an existing work, that is submitted by You to GitLab B.V. for inclusion in, or documentation of, any of the products owned or managed by GitLab B.V. (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to GitLab B.V. or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, GitLab B.V. for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution."
-- **Grant of Copyright License:**
+"2." **Grant of Copyright License:**
Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works.
-- **Grant of Patent License:**
+"3." **Grant of Patent License:**
Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed.
You represent that You are legally entitled to grant the above license. You represent further that each of Your employees is authorized to submit Contributions on Your behalf, but excluding employees that are designated in writing by You as "Not authorized to submit Contributions on behalf of (name of Your corporation here)." Such designations of exclusion for unauthorized employees are to be submitted via email to `legal@gitlab.com`. It is Your responsibility to notify GitLab B.V. when any change is required to the list of designated employees excluded from submitting Contributions on Your behalf. Such notification should also be sent via email to `legal@gitlab.com`.
-- **Contributions:**
+"4." **Contributions:**
You represent that each of Your Contributions is Your original creation.
diff --git a/doc/legal/developer_certificate_of_origin.md b/doc/legal/developer_certificate_of_origin.md
index 5c9f7f1fed7..eec89ce403e 100644
--- a/doc/legal/developer_certificate_of_origin.md
+++ b/doc/legal/developer_certificate_of_origin.md
@@ -3,6 +3,7 @@ stage: none
group: unassigned
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
+<!-- vale off -->
# Developer Certificate of Origin Version 1.1
diff --git a/doc/legal/individual_contributor_license_agreement.md b/doc/legal/individual_contributor_license_agreement.md
index 125ab1e6a8e..5a6116befe0 100644
--- a/doc/legal/individual_contributor_license_agreement.md
+++ b/doc/legal/individual_contributor_license_agreement.md
@@ -3,28 +3,29 @@ stage: none
group: unassigned
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
+<!-- vale off -->
# Individual contributor license agreement
You accept and agree to the following terms and conditions for Your present and future Contributions submitted to GitLab B.V.. Except for the license granted herein to GitLab B.V. and recipients of software distributed by GitLab B.V., You reserve all right, title, and interest in and to Your Contributions.
-- **Definitions:**
+"1." **Definitions:**
"You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with GitLab B.V.. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
"Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to GitLab B.V. for inclusion in, or documentation of, any of the products owned or managed by GitLab B.V. (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to GitLab B.V. or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, GitLab B.V. for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution."
-- **Grant of Copyright License:**
+"2." **Grant of Copyright License:**
Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works.
-- **Grant of Patent License:**
+"3." **Grant of Patent License:**
Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed.
You represent that you are legally entitled to grant the above license. If your employer(s) has rights to intellectual property that you create that includes your Contributions, you represent that you have received permission to make Contributions on behalf of that employer, that your employer has waived such rights for your Contributions to GitLab B.V., or that your employer has executed a separate Corporate CLA with GitLab B.V..
-- **Contributions:**
+"4." **Contributions:**
You represent that each of Your Contributions is Your original creation. You represent that Your Contribution submissions include complete details of any third-party license or other restriction (including, but not limited to, related patents and trademarks) of which you are personally aware and which are associated with any part of Your Contributions.
diff --git a/doc/operations/error_tracking.md b/doc/operations/error_tracking.md
index 3717eb46184..85b64eb7b3e 100644
--- a/doc/operations/error_tracking.md
+++ b/doc/operations/error_tracking.md
@@ -32,7 +32,7 @@ For error tracking to work, you need two pieces:
### Deploying Sentry
-You can sign up to the cloud hosted [Sentry](https://sentry.io), deploy your own [on-premise instance](https://github.com/getsentry/onpremise/), or use GitLab to [install Sentry to a Kubernetes cluster](../user/infrastructure/clusters/manage/management_project_applications/sentry.md).
+You can sign up to the cloud hosted [Sentry](https://sentry.io) or deploy your own [on-premise instance](https://github.com/getsentry/onpremise/).
### Enabling Sentry
@@ -55,7 +55,7 @@ least Maintainer [permissions](../user/permissions.md) to enable the Sentry inte
1. Ensure the **Active** checkbox is selected.
1. In the **Sentry API URL** box, enter your Sentry hostname. For example, enter `https://sentry.example.com`. For the SaaS version of Sentry, the hostname is `https://sentry.io`.
1. In the **Auth Token** box, enter the token you previously generated.
-1. To test the connection to Sentry and populate the **Project** dropdown, select **Connect**.
+1. To test the connection to Sentry and populate the **Project** dropdown list, select **Connect**.
1. From the **Project** list, choose a Sentry project to link to your GitLab project.
1. Select **Save changes**.
@@ -131,12 +131,13 @@ If another event occurs, the error reverts to unresolved.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/329596) in GitLab 14.4.
> - [Disabled](https://gitlab.com/gitlab-org/gitlab/-/issues/353639) in GitLab 14.9 [with a flag](../administration/feature_flags.md) named `integrated_error_tracking`. Disabled by default.
+> - [Enabled on GitLab.com](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7586) in GitLab 15.6.
FLAG:
By default this feature is not available. To make it available on self-managed GitLab, ask an
administrator to [enable the feature flag](../administration/feature_flags.md)
named `integrated_error_tracking`. The feature is not ready for production use.
-On GitLab.com, please follow [our user guide](https://gitlab.com/gitlab-org/opstrace/opstrace/-/blob/main/docs/guides/user/error_tracking.md) to get started.
+On GitLab.com, this feature is available.
Integrated error tracking is a lightweight alternative to Sentry backend.
You still use Sentry SDK with your application. But you don't need to deploy Sentry
@@ -148,6 +149,14 @@ settings. By using a GitLab-provided DSN, your application connects to GitLab to
Those errors are stored in the GitLab database and rendered by the GitLab UI, in the same way as
Sentry integration.
+In GitLab 15.6 and later, the integrated error tracking is available as an
+[open Beta](../policy/alpha-beta-support.md#beta-features).
+It now uses a new backend based on the ClickHouse database that enables better scalability.
+Integrated error tracking remains limited in comparison to the Sentry backend, as only a small subset of the
+Sentry API is implemented.
+
+Changing the GitLab error UI to use the GitLab Observability UI is tracked in epic [19](https://gitlab.com/groups/gitlab-org/opstrace/-/epics/32).
+
### Project settings
You can find the feature configuration at **Settings > Monitor > Error Tracking**.
@@ -156,11 +165,11 @@ You can find the feature configuration at **Settings > Monitor > Error Tracking*
1. Select **GitLab** as the error tracking backend for your project:
- ![Error Tracking Settings](img/error_tracking_setting_v14_3.png)
+ ![Error Tracking Settings](img/error_tracking_setting_v14_3.png)
1. Select **Save changes**. After page reload you should see a text field with the DSN string. Copy it.
- ![Error Tracking Settings DSN](img/error_tracking_setting_dsn_v14_4.png)
+ ![Error Tracking Settings DSN](img/error_tracking_setting_dsn_v14_4.png)
1. Take the DSN from the previous step and configure your Sentry SDK with it. Errors are now
reported to the GitLab collector and are visible in the [GitLab UI](#error-tracking-list).
diff --git a/doc/operations/feature_flags.md b/doc/operations/feature_flags.md
index 0afba821363..fff2c8ffbf7 100644
--- a/doc/operations/feature_flags.md
+++ b/doc/operations/feature_flags.md
@@ -422,7 +422,7 @@ At the moment, the feature flag API falls into **Unauthenticated traffic (from a
in the [GitLab.com specific limits](../user/gitlab_com/index.md),
so it's **500 requests per minute**.
-Please note that the polling rate is configurable in SDKs. Provided that all clients are requesting from the same IP:
+The polling rate is configurable in SDKs. Provided that all clients are requesting from the same IP:
- Request once per minute ... 500 clients can be supported.
- Request once per 15 sec ... 125 clients can be supported.
diff --git a/doc/operations/incident_management/alerts.md b/doc/operations/incident_management/alerts.md
index 32603aa3753..d6293cf1479 100644
--- a/doc/operations/incident_management/alerts.md
+++ b/doc/operations/incident_management/alerts.md
@@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
Alerts are a critical entity in your incident management workflow. They represent a notable event that might indicate a service outage or disruption. GitLab provides a list view for triage and detail view for deeper investigation of what happened.
-## Alert List
+## Alert list
Users with at least the Developer role can
access the Alert list at **Monitor > Alerts** in your project's
@@ -103,31 +103,6 @@ When you upload an image, you can add text to the image and link it to the origi
If you add a link, it is shown above the uploaded image.
-#### View an alert's logs
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201846) in GitLab Ultimate 12.8.
-> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/217768) in GitLab 13.3.
-> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25455) from GitLab Ultimate to GitLab Free in 12.9.
-
-Viewing logs from a metrics panel can be useful if you're triaging an
-application incident and need to [explore logs](../metrics/dashboards/index.md#chart-context-menu)
-from across your application. These logs help you understand what's affecting
-your application's performance and how to resolve any problems.
-
-Prerequisite:
-
-- You must have at least the Developer role.
-
-To view the logs for an alert:
-
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Monitor > Alerts**.
-1. Select the alert you want to view.
-1. Below the title of the alert, select the **Metrics** tab.
-1. Select the [menu](../metrics/dashboards/index.md#chart-context-menu) of
- the metric chart to view options.
-1. Select **View logs**.
-
### Activity feed tab
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3066) in GitLab 13.1.
@@ -137,7 +112,7 @@ timeline of the alert's investigation and assignment history.
The following actions result in a system note:
-- [Updating the status of an alert](#update-an-alerts-status)
+- [Updating the status of an alert](#change-an-alerts-status)
- [Creating an incident based on an alert](#create-an-incident-from-an-alert)
- [Assignment of an alert to a user](#assign-an-alert)
- [Escalation of an alert to on-call responders](paging.md#escalating-an-alert)
@@ -148,25 +123,55 @@ The following actions result in a system note:
There are different actions available in GitLab to help triage and respond to alerts.
-### Update an alert's status
+### Change an alert's status
+
+You can change the status of an alert.
-**Triggered** is the default status for new alerts. For users with the Developer role or higher, the
-alert status can be updated from these locations:
+The available statuses are:
-- [Alert list](#alert-list): select the status dropdown corresponding to an alert, then select an
- alternate status.
-- [Alert details page](#alert-details-page): select **Edit** in the right-hand side bar, then select
- an alternate status.
+- Triggered (default for new alerts)
+- Acknowledged
+- Resolved
+
+Prerequisites:
+
+- You must have at least the Developer role.
+
+To change an alert's status:
+
+- From the [alert list](#alert-list):
+
+ 1. In the **Status** column, next to an alert, select the status dropdown list.
+ 1. Select a status.
+
+- From the [alert details page](#alert-details-page):
+
+ 1. On the right sidebar, select **Edit**.
+ 1. Select a status.
To stop email notifications for alert reoccurrences in projects with [email notifications enabled](paging.md#email-notifications-for-alerts),
-[change the alert's status](alerts.md#update-an-alerts-status) away from **Triggered**.
+change the alert's status away from **Triggered**.
+
+#### Resolve an alert by closing the linked incident
+
+Prerequisites:
+
+- You must have at least the Reporter role.
+
+When you close an [incident](incidents.md) that is linked to an alert,
+the linked alert's status changes to **Resolved**.
+You are then credited with the alert's status change.
+
+#### As an on-call responder **(PREMIUM)**
+
+On-call responders can respond to [alert pages](paging.md#escalating-an-alert)
+by changing the alert status.
-In projects with GitLab Premium, on-call responders can respond to [alert pages](paging.md#escalating-an-alert)
-by changing the status. Setting the status to:
+Changing the status has the following effects:
-- **Resolved** silences all on-call pages for the alert.
-- **Acknowledged** limits on-call pages based on the project's [escalation policy](escalation_policies.md).
-- **Triggered** from **Resolved** restarts the alert escalating from the beginning.
+- To **Acknowledged**: limits on-call pages based on the project's [escalation policy](escalation_policies.md).
+- To **Resolved**: silences all on-call pages for the alert.
+- From **Resolved** to **Triggered**: restarts the alert escalating.
In GitLab 15.1 and earlier, updating the status of an [alert with an associated incident](alerts.md#create-an-incident-from-an-alert)
also updates the incident status. In [GitLab 15.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/356057),
@@ -183,7 +188,7 @@ alert by selecting the **View Issue** button.
You can also [create incidents for alerts automatically](incidents.md#create-incidents-automatically).
-Closing a GitLab issue associated with an alert [changes the alert's status](#update-an-alerts-status) to
+Closing a GitLab issue associated with an alert [changes the alert's status](#change-an-alerts-status) to
**Resolved**. See [Alert List](#alert-list) for more details
about alert statuses.
@@ -213,7 +218,7 @@ To assign an alert:
GitLab creates a [to-do item](../../user/todos.md) for each user.
After completing their portion of investigating or fixing the alert, users can
-unassign themselves from the alert. To remove an assignee, select **Edit** next to the **Assignee** dropdown menu
+unassign themselves from the alert. To remove an assignee, select **Edit** next to the **Assignee** dropdown list
and clear the user from the list of assignees, or select **Unassigned**.
### Create a to-do item from an alert
diff --git a/doc/operations/incident_management/img/timeline_event_for_severity_change_v15_6.png b/doc/operations/incident_management/img/timeline_event_for_severity_change_v15_6.png
new file mode 100644
index 00000000000..ae9d446f8b5
--- /dev/null
+++ b/doc/operations/incident_management/img/timeline_event_for_severity_change_v15_6.png
Binary files differ
diff --git a/doc/operations/incident_management/incident_timeline_events.md b/doc/operations/incident_management/incident_timeline_events.md
index 5f98335d7aa..58448222356 100644
--- a/doc/operations/incident_management/incident_timeline_events.md
+++ b/doc/operations/incident_management/incident_timeline_events.md
@@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344059) in GitLab 15.2 [with a flag](../../administration/feature_flags.md) named `incident_timeline`. Enabled by default.
> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/353426) in GitLab 15.3.
-> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/353426) in GitLab 15.5. [Feature flag 'incident_timeline'](https://gitlab.com/gitlab-org/gitlab/-/issues/343386) removed.
+> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/353426) in GitLab 15.5. [Feature flag `incident_timeline`](https://gitlab.com/gitlab-org/gitlab/-/issues/343386) removed.
Incident timelines are an important part of record keeping for incidents.
Timelines can show executives and external viewers what happened during an incident,
@@ -74,6 +74,15 @@ To create a timeline event from a comment on the incident:
The comment is shown on the incident timeline as a timeline event.
+### When incident severity changes
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375280) in GitLab 15.6.
+
+A new timeline event is created when someone [changes the severity](incidents.md#change-severity)
+of an incident.
+
+![Incident timeline event for severity change](img/timeline_event_for_severity_change_v15_6.png)
+
## Delete an event
You can also delete timeline events.
diff --git a/doc/operations/incident_management/incidents.md b/doc/operations/incident_management/incidents.md
index edbe07f166c..a5d38b1a27c 100644
--- a/doc/operations/incident_management/incidents.md
+++ b/doc/operations/incident_management/incidents.md
@@ -35,7 +35,7 @@ To create an incident from the Issues List:
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230857) in GitLab 13.4.
1. Go to **Issues > List**, and select **New issue**.
-1. In the **Type** dropdown, select **Incident**. Only fields relevant to
+1. In the **Type** dropdown list, select **Incident**. Only fields relevant to
incidents are displayed on the page.
1. Create the incident as needed, and select **Create issue** to save the
incident.
diff --git a/doc/operations/incident_management/integrations.md b/doc/operations/incident_management/integrations.md
index ae6ffe549d5..a54556bd3a2 100644
--- a/doc/operations/incident_management/integrations.md
+++ b/doc/operations/incident_management/integrations.md
@@ -36,7 +36,7 @@ receive alert payloads in JSON format. You can always
1. Sign in to GitLab as a user with the Maintainer role
for a project.
1. Navigate to **Settings > Monitor** in your project.
-1. Expand the **Alerts** section, and in the **Select integration type** dropdown menu,
+1. Expand the **Alerts** section, and in the **Select integration type** dropdown list,
select **HTTP Endpoint**.
1. Toggle the **Active** alert setting. The URL and Authorization Key for the webhook configuration
are available in the **View credentials** tab after you save the integration. You must also input
@@ -57,7 +57,7 @@ and you can [customize the payload](#customize-the-alert-payload-outside-of-gitl
1. For each endpoint you want to create:
1. Select **Add new integration**.
- 1. In the **Select integration type** dropdown menu, select **HTTP Endpoint**.
+ 1. In the **Select integration type** dropdown list, select **HTTP Endpoint**.
1. Name the integration.
1. Toggle the **Active** alert setting. The **URL** and **Authorization Key** for the webhook
configuration are available in the **View credentials** tab after you save the integration.
diff --git a/doc/operations/incident_management/linked_resources.md b/doc/operations/incident_management/linked_resources.md
index 4c3ef34f634..a7be867608b 100644
--- a/doc/operations/incident_management/linked_resources.md
+++ b/doc/operations/incident_management/linked_resources.md
@@ -6,11 +6,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Linked resources in incidents **(PREMIUM)**
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230852) in GitLab 15.3 [with a flag](../../administration/feature_flags.md) named `incident_resource_links_widget`. Enabled on GitLab.com. Disabled on self-managed.
-
-FLAG:
-On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `incident_resource_links_widget`.
-On GitLab.com, this feature is available.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230852) in GitLab 15.3 [with a flag](../../administration/feature_flags.md) named `incident_resource_links_widget`. Disabled by default.
+> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/364755) in GitLab 15.3.
+> - [Generally available](ihttps://gitlab.com/gitlab-org/gitlab/-/issues/364755) in GitLab 15.5. Feature flag `incident_resource_links_widget` removed.
To help your team members find the important links without having to search through many comments,
you can add linked resources to an incident issue.
diff --git a/doc/operations/incident_management/paging.md b/doc/operations/incident_management/paging.md
index 55e1c9dfbcc..dcb17048d35 100644
--- a/doc/operations/incident_management/paging.md
+++ b/doc/operations/incident_management/paging.md
@@ -34,7 +34,7 @@ a single email notification for new alerts.
**Send a single email notification to Owners and Maintainers for new alerts** checkbox.
1. Select **Save changes**.
-[Update the alert's status](alerts.md#update-an-alerts-status) to manage email notifications for an alert.
+[Update the alert's status](alerts.md#change-an-alerts-status) to manage email notifications for an alert.
## Paging **(PREMIUM)**
@@ -46,7 +46,7 @@ can be automatically paged about critical problems through email.
When an alert is triggered, it begins escalating to the on-call responders immediately.
For each escalation rule in the project's escalation policy, the designated on-call
responders receive one email when the rule fires. You can respond to a page
-or stop alert escalations by [updating the alert's status](alerts.md#update-an-alerts-status).
+or stop alert escalations by [updating the alert's status](alerts.md#change-an-alerts-status).
### Escalating an incident
diff --git a/doc/operations/index.md b/doc/operations/index.md
index 4f0d843f66e..ff13c617ea7 100644
--- a/doc/operations/index.md
+++ b/doc/operations/index.md
@@ -57,35 +57,6 @@ and the work required to fix them - all without leaving GitLab.
- Discover and view errors generated by your applications with
[Error Tracking](error_tracking.md).
-## Trace application health and performance (DEPRECATED)
-
-> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346485) in GitLab 14.7.
-
-WARNING:
-This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346485)
-in GitLab 14.7, and is planned for removal in GitLab 15.0.
-
-Application tracing in GitLab is a way to measure an application's performance and
-health while it's running. After configuring your application to enable tracing, you
-gain in-depth insight into your application's layers. With application tracing,
-you can measure the execution time of a user journey for troubleshooting or
-optimization purposes.
-
-GitLab integrates with [Jaeger](https://www.jaegertracing.io/) - an open-source,
-end-to-end distributed tracing system tool used for monitoring and troubleshooting
-microservices-based distributed systems - and displays results within GitLab.
-
-- [Trace the performance and health](tracing.md) of a deployed application.
-
-<!--- start_remove The following content will be removed on remove_date: '2022-10-18'--->
-
-## Aggregate and store logs (removed) **(FREE SELF)**
-
-This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346485) in GitLab 14.7
-and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/360193) in GitLab 15.2.
-
-<!--- end_remove -->
-
## Manage your infrastructure in code
GitLab stores and executes your infrastructure as code, whether it's
@@ -105,5 +76,4 @@ an environment.
- Deploy to different [environments](../ci/environments/index.md).
- Connect your project to a [Kubernetes cluster](../user/infrastructure/clusters/index.md).
-- See how your application is used and analyze events with [Product Analytics](product_analytics.md).
- Create, toggle, and remove [Feature Flags](feature_flags.md).
diff --git a/doc/operations/metrics/dashboards/index.md b/doc/operations/metrics/dashboards/index.md
index 4e0b50d1e04..aef9bcb4b22 100644
--- a/doc/operations/metrics/dashboards/index.md
+++ b/doc/operations/metrics/dashboards/index.md
@@ -66,7 +66,7 @@ To create a new dashboard from the command line:
1. Save the file, commit, and push to your repository. The file must be present in your **default** branch.
1. Navigate to your project's **Monitor > Metrics** and choose the custom
- dashboard from the dropdown.
+ dashboard from the dropdown list.
Your custom dashboard is available at `https://example.com/project/-/metrics/custom_dashboard_name.yml`.
@@ -125,7 +125,7 @@ can manage [the settings](settings.md) for your metrics dashboard.
## Chart Context Menu
You can take action related to a chart's data by selecting the
-**{ellipsis_v}** **More actions** dropdown box above the upper right corner of
+**{ellipsis_v}** **More actions** dropdown list above the upper right corner of
any chart on a dashboard:
![Context Menu](img/panel_context_menu_v14_0.png)
diff --git a/doc/operations/metrics/dashboards/templating_variables.md b/doc/operations/metrics/dashboards/templating_variables.md
index aac22236dc7..a1c2ce063bc 100644
--- a/doc/operations/metrics/dashboards/templating_variables.md
+++ b/doc/operations/metrics/dashboards/templating_variables.md
@@ -67,7 +67,7 @@ WARNING:
This variable type is an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) feature, and is subject to change at any time
without prior notice!
-Each `custom` variable defined in the dashboard YAML creates a dropdown
+Each `custom` variable defined in the dashboard YAML creates a dropdown list
selector on the dashboard UI, allowing you to select a value for each variable.
The `custom` variable type supports a simple and a full syntax.
@@ -75,7 +75,7 @@ The `custom` variable type supports a simple and a full syntax.
### Simple syntax
This example creates a variable called `variable1`, with a default value of `value1`.
-The dashboard UI displays a dropdown with `value1`, `value2` and `value3`
+The dashboard UI displays a dropdown list with `value1`, `value2` and `value3`
as the choices.
```yaml
@@ -88,10 +88,10 @@ templating:
This example creates a variable called `variable1`, with a default value of `value_option_2`.
The label for the text box on the UI is the value of the `label` key.
-The dashboard UI displays a dropdown with `Option 1` and `Option 2`
+The dashboard UI displays a dropdown list with `Option 1` and `Option 2`
as the choices.
-If you select `Option 1` from the dropdown, the variable is replaced with `value option 1`.
+If you select `Option 1` from the dropdown list, the variable is replaced with `value option 1`.
Similarly, if you select `Option 2`, the variable is replaced with `value_option_2`:
```yaml
@@ -117,7 +117,7 @@ without prior notice!
### Full syntax
-This example creates a variable called `variable2`. The values of the dropdown are
+This example creates a variable called `variable2`. The values of the dropdown list are
all the different values of the `backend` label in the Prometheus series described by
`up{env="production"}`.
diff --git a/doc/operations/metrics/index.md b/doc/operations/metrics/index.md
index 3e5ea688fde..3c2790a96b7 100644
--- a/doc/operations/metrics/index.md
+++ b/doc/operations/metrics/index.md
@@ -148,7 +148,7 @@ suggested if this feature is used.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208976) in GitLab 12.9.
You can edit existing additional custom metrics for your dashboard by selecting the
-**{ellipsis_v}** **More actions** dropdown and selecting **Edit metric**.
+**{ellipsis_v}** **More actions** dropdown list and selecting **Edit metric**.
![Edit metric](img/prometheus_dashboard_edit_metric_link_v_12_9.png)
@@ -158,7 +158,7 @@ You can edit existing additional custom metrics for your dashboard by selecting
You can use keyboard shortcuts to interact more quickly with your currently-focused
chart panel. To activate keyboard shortcuts, use keyboard tabs to highlight the
-**{ellipsis_v}** **More actions** dropdown menu, or hover over the dropdown menu
+**{ellipsis_v}** **More actions** dropdown list, or hover over the dropdown list
with your mouse, then press the key corresponding to your desired action:
- **Expand panel** - <kbd>e</kbd>
diff --git a/doc/operations/product_analytics.md b/doc/operations/product_analytics.md
deleted file mode 100644
index f9b92f61fd2..00000000000
--- a/doc/operations/product_analytics.md
+++ /dev/null
@@ -1,45 +0,0 @@
----
-stage: Analytics
-group: Product Analytics
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
----
-
-# Product Analytics **(FREE)**
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225167) in GitLab 13.3 [with a flag](../administration/feature_flags.md) named `product_analytics`. Disabled by default.
-
-FLAG:
-On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `product_analytics`. On GitLab.com, this feature is not available. The feature is not ready for production use.
-
-GitLab enables you to go from planning an application to getting feedback. You can use
-Product Analytics to receive and analyze events sent from your application. This analysis
-provides observability information and feedback on how people use your product.
-
-Events are collected by a [Rails collector](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36443) and
-then processed with [Snowplow](https://github.com/snowplow/snowplow). Events are stored in a GitLab database.
-
-## View Product Analytics
-
-You can view the event data collected about your applications.
-
-Prerequisite:
-
-- You must have at least the Reporter role.
-
-To access Product Analytics:
-
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Monitor > Product Analytics**.
-
-The Product Analytics interface contains:
-
-- An Events tab that shows the recent events and a total count.
-- A Graph tab that shows graphs based on events of the last 30 days.
-- A Test tab that sends a sample event payload.
-- A Setup page containing the code to implement in your application.
-
-## Rate limits
-
-While Product Analytics is under development, it's rate-limited to
-**100 events per minute** per project. This limit prevents the events table in the
-database from growing too quickly.
diff --git a/doc/operations/tracing.md b/doc/operations/tracing.md
index 64d8f8a8707..b3c0763bbbc 100644
--- a/doc/operations/tracing.md
+++ b/doc/operations/tracing.md
@@ -2,50 +2,13 @@
stage: Monitor
group: Respond
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+remove_date: '2022-11-01'
+redirect_to: 'index.md'
---
-# Tracing (DEPRECATED) **(FREE SELF)**
+# Tracing (removed) **(FREE SELF)**
-> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/42645) from GitLab Ultimate to GitLab Free in 13.5.
-> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346540) in GitLab 14.7.
-> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/359904) behind a [feature flag](../administration/feature_flags.md) named `monitor_tracing` in GitLab 15.0. Disabled by default.
-
-WARNING:
-This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346540)
-in GitLab 14.7.
-It will be removed completely in GitLab 15.2.
-
-FLAG:
-On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `monitor_tracing`.
-On GitLab.com, this feature is not available.
-This feature is not recommended for production use.
-
-Tracing provides insight into the performance and health of a deployed application, tracking each
-function or microservice that handles a given request. Tracing makes it easy to understand the
-end-to-end flow of a request, regardless of whether you are using a monolithic or distributed
-system.
-
-## Install Jaeger
-
-[Jaeger](https://www.jaegertracing.io/) is an open source, end-to-end distributed tracing system
-used for monitoring and troubleshooting microservices-based distributed systems. To learn more about
-installing Jaeger, read the official
-[Getting Started documentation](https://www.jaegertracing.io/docs/latest/getting-started/).
-
-See also:
-
-- An [all-in-one Docker image](https://www.jaegertracing.io/docs/latest/getting-started/#all-in-one).
-- Deployment options for:
- - [Kubernetes](https://github.com/jaegertracing/jaeger-kubernetes).
- - [OpenShift](https://github.com/jaegertracing/jaeger-openshift).
-
-## Link to Jaeger
-
-GitLab provides an easy way to open the Jaeger UI from within your project:
-
-1. [Set up Jaeger](https://www.jaegertracing.io) and configure your application using one of the
- [client libraries](https://www.jaegertracing.io/docs/latest/client-libraries/).
-1. Navigate to your project's **Settings > Monitor** and provide the Jaeger URL.
-1. Select **Save changes** for the changes to take effect.
-1. You can now visit **Monitor > Tracing** in your project's sidebar and GitLab redirects you to
- the configured Jaeger URL.
+This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346540) in GitLab 14.7
+and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/346540) in GitLab 15.2.
+We are working on an alternative to replace tracing.
+To learn more, visit the [Observability direction page](https://about.gitlab.com/direction/monitor/observability/).
diff --git a/doc/raketasks/backup_gitlab.md b/doc/raketasks/backup_gitlab.md
index da004a1b774..47696fc1f99 100644
--- a/doc/raketasks/backup_gitlab.md
+++ b/doc/raketasks/backup_gitlab.md
@@ -139,7 +139,7 @@ For installation from source:
- `/home/git/gitlab/config/secrets.yml`
- `/home/git/gitlab/config/gitlab.yml`
-For [Docker installations](https://docs.gitlab.com/omnibus/docker/), you must
+For [Docker installations](../install/docker.md), you must
back up the volume where the configuration files are stored. If you created
the GitLab container according to the documentation, it should be in the
`/srv/gitlab/config` directory.
diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md
index 2fa6fc2e564..a67fec26a9b 100644
--- a/doc/raketasks/cleanup.md
+++ b/doc/raketasks/cleanup.md
@@ -152,6 +152,12 @@ NOTE:
These commands don't work for artifacts stored on
[object storage](../administration/object_storage.md).
+WARNING:
+Prior to GitLab 14.9, this task incorrectly deletes [pipeline artifacts](../ci/pipelines/pipeline_artifacts.md)
+[The bug fix](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81022) was
+also back-ported to 14.6.6, 14.7.5, and 14.8.3. Upgrade to a release with the bug
+fix to avoid data loss.
+
When you notice there are more job artifacts files and/or directories on disk than there
should be, you can run:
diff --git a/doc/raketasks/index.md b/doc/raketasks/index.md
index 12d2d8b101a..4a157688a41 100644
--- a/doc/raketasks/index.md
+++ b/doc/raketasks/index.md
@@ -46,7 +46,7 @@ The following Rake tasks are available for use with GitLab:
| [Reset user passwords](../security/reset_user_password.md#use-a-rake-task) | Reset user passwords using Rake. |
| [Uploads migrate](../administration/raketasks/uploads/migrate.md) | Migrate uploads between local storage and object storage. |
| [Uploads sanitize](../administration/raketasks/uploads/sanitize.md) | Remove EXIF data from images uploaded to earlier versions of GitLab. |
-| [Service Data](../administration/troubleshooting/gitlab_rails_cheat_sheet.md#generate-service-ping) | Generate and troubleshoot [Service Ping](../development/service_ping/index.md). |
+| [Service Data](../development/service_ping/troubleshooting.md#generate-service-ping) | Generate and troubleshoot [Service Ping](../development/service_ping/index.md). |
| [User management](user_management.md) | Perform user management tasks. |
| [Webhooks administration](web_hooks.md) | Maintain project webhooks. |
| [X.509 signatures](x509_signatures.md) | Update X.509 commit signatures, which can be useful if the certificate store changed. |
diff --git a/doc/security/crime_vulnerability.md b/doc/security/crime_vulnerability.md
index e2aa4b5d4ab..463ccb7b629 100644
--- a/doc/security/crime_vulnerability.md
+++ b/doc/security/crime_vulnerability.md
@@ -70,6 +70,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/security/index.md b/doc/security/index.md
index ff0769e0d93..38eb5337f5a 100644
--- a/doc/security/index.md
+++ b/doc/security/index.md
@@ -6,7 +6,7 @@ comments: false
type: index
---
-# Security **(FREE)**
+# Secure your installation **(FREE)**
- [Passwords and OAuth tokens storage](password_storage.md)
- [Password length limits](password_length_limits.md)
diff --git a/doc/security/information_exclusivity.md b/doc/security/information_exclusivity.md
index 37a9fdfdd81..16facadd782 100644
--- a/doc/security/information_exclusivity.md
+++ b/doc/security/information_exclusivity.md
@@ -35,6 +35,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/security/password_length_limits.md b/doc/security/password_length_limits.md
index fe51022eea7..0211a326e0a 100644
--- a/doc/security/password_length_limits.md
+++ b/doc/security/password_length_limits.md
@@ -12,17 +12,11 @@ By default, GitLab supports passwords with the following lengths:
- Minimum: 8 characters
- Maximum: 128 characters
-GitLab administrators can modify password lengths:
+You can only change the minimum password length. Changing the minimum length does not affect existing user passwords.
+Existing users are not asked to reset their password to adhere to the new limits. The new limit restriction applies only
+during new user sign-ups and when an existing user performs a password reset.
-- Using the GitLab UI. [From](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20661) GitLab
- 12.6, this is the only available option.
-- Using configuration file. Up to GitLab 12.5.
-
-Changing the minimum or maximum length does not affect existing user passwords. Existing users are
-not asked to reset their password to adhere to the new limits. The new limit restriction applies
-only during new user sign-ups and when an existing user performs a password reset.
-
-## Modify minimum password length using GitLab UI
+## Modify minimum password length
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20661) in GitLab 12.6
@@ -32,39 +26,9 @@ To change the minimum password length using GitLab UI:
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > General** and expand **Sign-up restrictions**.
-
- ![Minimum password length settings](../user/admin_area/img/minimum_password_length_settings_v12_6.png)
-
1. Enter a **Minimum password length** value greater than or equal to `8`.
1. Select **Save changes**.
-## Modify maximum password length using configuration file
-
-From GitLab 12.6, the minimum password length set in this configuration file is ignored. Minimum password lengths must instead be modified via the [GitLab UI](#modify-minimum-password-length-using-gitlab-ui).
-
-The user password length is set to a maximum of 128 characters by default.
-To change that for installations from source:
-
-1. Edit `devise_password_length.rb`:
-
- ```shell
- cd /home/git/gitlab
- sudo -u git -H cp config/initializers/devise_password_length.rb.example config/initializers/devise_password_length.rb
- sudo -u git -H editor config/initializers/devise_password_length.rb
- ```
-
-1. Change the new password length limits:
-
- ```ruby
- config.password_length = 12..135
- ```
-
- In this example, the minimum length is 12 characters, and the maximum length
- is 135 characters.
-
-1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source)
- for the changes to take effect.
-
<!-- ## Troubleshooting
Include any troubleshooting steps that you can foresee. If you know beforehand what issues
@@ -73,6 +37,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/security/password_storage.md b/doc/security/password_storage.md
index 6b20f8619ae..bd514de6e2c 100644
--- a/doc/security/password_storage.md
+++ b/doc/security/password_storage.md
@@ -11,7 +11,8 @@ GitLab administrators can configure how passwords and OAuth tokens are stored.
## Password storage
-> PBKDF2 and SHA512 [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360658) in GitLab 15.2 [with flags](../administration/feature_flags.md) named `pbkdf2_password_encryption` and `pbkdf2_password_encryption_write`. Disabled by default.
+> - PBKDF2+SHA512 [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360658) in GitLab 15.2 [with flags](../administration/feature_flags.md) named `pbkdf2_password_encryption` and `pbkdf2_password_encryption_write`. Disabled by default.
+> - Feature flags [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/101691) in GitLab 15.6 and PBKDF2+SHA512 was made available to all GitLab instances running in [FIPS mode](../development/fips_compliance.md).
GitLab stores user passwords in a hashed format to prevent passwords from being
stored as plain text.
@@ -23,17 +24,9 @@ library to hash user passwords. Created password hashes have these attributes:
- **BCrypt**: By default, the [`bcrypt`](https://en.wikipedia.org/wiki/Bcrypt) hashing
function is used to generate the hash of the provided password. This cryptographic hashing function is
strong and industry-standard.
- - **PBKDF2 and SHA512**: Starting in GitLab 15.2, PBKDF2 and SHA512 are supported
- behind the following feature flags (disabled by default):
- - `pbkdf2_password_encryption` - Enables reading and comparison of PBKDF2 + SHA512
- hashed passwords and supports fallback for BCrypt hashed passwords.
- - `pbkdf2_password_encryption_write` - Enables new passwords to be saved
- using PBKDF2 and SHA512, and existing BCrypt passwords to be migrated when users sign in.
-
- FLAG:
- On self-managed GitLab, by default this feature is not available. To make it available,
- ask an administrator to [enable the feature flags](../administration/feature_flags.md) named `pbkdf2_password_encryption` and `pbkdf2_password_encryption_write`.
-
+ - **PBKDF2+SHA512**: PBKDF2+SHA512 is supported:
+ - In GitLab 15.2 to GitLab 15.5 when `pbkdf2_password_encryption` and `pbkdf2_password_encryption_write` [feature flags](../administration/feature_flags.md) are enabled.
+ - In GitLab 15.6 and later when [FIPS mode](../development/fips_compliance.md) is enabled (feature flags are not required).
- **Stretching**: Password hashes are [stretched](https://en.wikipedia.org/wiki/Key_stretching)
to harden against brute-force attacks. By default, GitLab uses a stretching
factor of 10 for BCrypt and 20,000 for PBKDF2 + SHA512.
@@ -45,9 +38,7 @@ library to hash user passwords. Created password hashes have these attributes:
## OAuth access token storage
> - PBKDF2+SHA512 [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/364110) in GitLab 15.3 [with flag](../administration/feature_flags.md) named `hash_oauth_tokens`.
-> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98242) in GitLab 15.5.
-
-Depending on your version of GitLab and configuration, OAuth access tokens are stored in the database in PBKDF2+SHA512 format. For version information, see
-the relevant [OAuth provider documentation](../integration/oauth_provider.md#hashed-oauth-tokens).
+> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/367570) in GitLab 15.5.
+> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/367570) in GitLab 15.6.
-As with PBKDF2+SHA512 password storage, access token values are [stretched](https://en.wikipedia.org/wiki/Key_stretching) 20,000 times to harden against brute-force attacks.
+OAuth access tokens are stored in the database in PBKDF2+SHA512 format. As with PBKDF2+SHA512 password storage, access token values are [stretched](https://en.wikipedia.org/wiki/Key_stretching) 20,000 times to harden against brute-force attacks.
diff --git a/doc/security/passwords_for_integrated_authentication_methods.md b/doc/security/passwords_for_integrated_authentication_methods.md
index 961d147871e..68fb097ab9a 100644
--- a/doc/security/passwords_for_integrated_authentication_methods.md
+++ b/doc/security/passwords_for_integrated_authentication_methods.md
@@ -14,4 +14,4 @@ However, to maintain data consistency, GitLab requires passwords for all user ac
For such accounts, we use the [`friendly_token`](https://github.com/heartcombo/devise/blob/f26e05c20079c9acded3c0ee16da0df435a28997/lib/devise.rb#L492) method provided by the Devise gem to generate a random, unique and secure password and sets it as the account password during sign up.
-The length of the generated password is the set based on the value of [maximum password length](password_length_limits.md#modify-maximum-password-length-using-configuration-file) as set in the Device configuration. The default value is 128 characters.
+The length of the generated password is [128 characters](password_length_limits.md).
diff --git a/doc/security/ssh_keys_restrictions.md b/doc/security/ssh_keys_restrictions.md
index 5a8450bb8e7..f15d71461d4 100644
--- a/doc/security/ssh_keys_restrictions.md
+++ b/doc/security/ssh_keys_restrictions.md
@@ -71,6 +71,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/security/two_factor_authentication.md b/doc/security/two_factor_authentication.md
index cc06e824d14..95915c4e03b 100644
--- a/doc/security/two_factor_authentication.md
+++ b/doc/security/two_factor_authentication.md
@@ -99,11 +99,13 @@ This is a permanent and irreversible action. Users must reactivate 2FA to use it
### For a single user
-To disable 2FA for non-administrator users, we recommend using the [API endpoint](../api/users.md#disable-two-factor-authentication)
+To disable 2FA for non-administrator users, you should use the [API endpoint](../api/users.md#disable-two-factor-authentication)
instead of the Rails console.
Using the [Rails console](../administration/operations/rails_console.md), 2FA for a single user can be disabled.
Connect to the Rails console and run:
+**In GitLab 13.5 and later:**
+
```ruby
admin = User.find_by_username('<USERNAME>')
user_to_disable = User.find_by_username('<USERNAME>')
@@ -111,6 +113,8 @@ user_to_disable = User.find_by_username('<USERNAME>')
TwoFactor::DestroyService.new(admin, user: user_to_disable).execute
```
+The target user is notified that 2FA has been disabled.
+
### For all users
There may be some special situations where you want to disable 2FA for everyone
@@ -167,6 +171,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/security/unlock_user.md b/doc/security/unlock_user.md
index 4eded22ddaa..9a1f60f2462 100644
--- a/doc/security/unlock_user.md
+++ b/doc/security/unlock_user.md
@@ -17,7 +17,7 @@ Users are locked after ten failed sign-in attempts. These users remain locked:
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Users**.
1. Use the search bar to find the locked user.
-1. From the **User administration** dropdown select **Unlock**.
+1. From the **User administration** dropdown list select **Unlock**.
![Unlock a user from the Admin Area](img/unlock_user_v14_7.png)
@@ -66,6 +66,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/security/user_email_confirmation.md b/doc/security/user_email_confirmation.md
index 3f7c66b311b..ffc537c8f10 100644
--- a/doc/security/user_email_confirmation.md
+++ b/doc/security/user_email_confirmation.md
@@ -28,6 +28,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/security/user_file_uploads.md b/doc/security/user_file_uploads.md
index 63a5e51e3b5..db2948a8bd5 100644
--- a/doc/security/user_file_uploads.md
+++ b/doc/security/user_file_uploads.md
@@ -53,6 +53,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md
index 49ab4215ea5..eeb6720dfb7 100644
--- a/doc/security/webhooks.md
+++ b/doc/security/webhooks.md
@@ -99,6 +99,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/subscriptions/gitlab_com/index.md b/doc/subscriptions/gitlab_com/index.md
index c9b066144f3..6f1bf0df044 100644
--- a/doc/subscriptions/gitlab_com/index.md
+++ b/doc/subscriptions/gitlab_com/index.md
@@ -29,7 +29,7 @@ Members of every subgroup and project in the group:
To subscribe to GitLab SaaS:
-1. View the [GitLab SaaS feature comparison](https://about.gitlab.com/pricing/gitlab-com/feature-comparison/)
+1. View the [GitLab SaaS feature comparison](https://about.gitlab.com/pricing/feature-comparison/)
and decide which tier you want.
1. Create a user account for yourself by using the
[sign up page](https://gitlab.com/users/sign_up).
@@ -190,6 +190,30 @@ If you add a member to a group by using the [share a group with another group](.
- Remove the member from the shared group. You must be a group owner to do this.
- From the group's membership page, remove access from the entire shared group.
+## Seat usage alerts
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348481) in GitLab 15.2 [with a flag](../../administration/feature_flags.md) named `seat_flag_alerts`.
+> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/362041) in GitLab 15.4. Feature flag `seat_flag_alerts` removed.
+
+If you have the Owner role of the top-level group, an alert notifies you
+of your total seat usage.
+
+The alert displays on group, subgroup, and project
+pages, and only for top-level groups linked to subscriptions enrolled
+in [quarterly subscription reconciliations](../quarterly_reconciliation.md).
+After you dismiss the alert, it doesn't display until another seat is used.
+
+The alert displays based on the following seat usage. You cannot configure the
+amounts at which the alert displays.
+
+| Seats in subscription | Seat usage |
+|-----------------------|------------------------------------------------------------------------|
+| 0-15 | One seat remaining in the subscription. |
+| 16-25 | Two seats remaining in the subscription. |
+| 26-99 | 10% of seats have been used. |
+| 100-999 | 8% of seats have been used. |
+| 1000+ | 5% of seats have been used |
+
## Upgrade your GitLab SaaS subscription tier
To upgrade your [GitLab tier](https://about.gitlab.com/pricing/):
@@ -309,6 +333,10 @@ locked. Projects can only be unlocked by purchasing more storage subscription un
### Purchase more storage and transfer
+Prerequisite:
+
+- You must have at least the Owner role.
+
You can purchase a storage subscription for your personal or group namespace.
NOTE:
@@ -324,9 +352,9 @@ You can [cancel the subscription](#enable-or-disable-automatic-subscription-rene
1. Select **Purchase more storage** and you are taken to the Customers Portal.
1. Select **Add new subscription**.
1. Scroll to **Purchase add-on subscriptions** and select **Buy storage subscription**.
-1. In the **Subscription details** section select the name of the user or group from the dropdown.
+1. In the **Subscription details** section select the name of the user or group from the dropdown list.
1. Enter the desired quantity of storage packs.
-1. In the **Billing information** section select the payment method from the dropdown.
+1. In the **Billing information** section select the payment method from the dropdown list.
1. Select the **Privacy Policy** and **Terms of Service** checkbox.
1. Select **Buy subscription**.
1. Sign out of the Customers Portal.
@@ -388,3 +416,8 @@ If your credit card is declined when purchasing a GitLab subscription, possible
Check with your financial institution to confirm if any of these reasons apply. If they don't
apply, contact [GitLab Support](https://support.gitlab.com/hc/en-us/requests/new?ticket_form_id=360000071293).
+
+### Unable to link subcription to namespace
+
+If you cannot link a subscription to your namespace, ensure that you have the Owner role
+for that namespace.
diff --git a/doc/subscriptions/gitlab_dedicated/index.md b/doc/subscriptions/gitlab_dedicated/index.md
index 441a4d2aa30..c503c501eeb 100644
--- a/doc/subscriptions/gitlab_dedicated/index.md
+++ b/doc/subscriptions/gitlab_dedicated/index.md
@@ -30,7 +30,7 @@ GitLab Dedicated enables you to offload the operational overhead of managing the
- Backups: Regular backups taken and tested.
- Choice of cloud region: Upon onboarding, choose the cloud region where you want to deploy your instance. Some AWS regions have limited features and as a result, we are not able to deploy production instances to those regions. See below for the [full list of regions](#aws-regions-not-supported) not currently supported.
- Security: Data encrypted at rest and in transit using latest encryption standards.
-- Application: Self-managed [Ultimate feature set](https://about.gitlab.com/pricing/self-managed/feature-comparison/) with the exception of the unsupported features [listed below](#features-that-are-not-available).
+- Application: Self-managed [Ultimate feature set](https://about.gitlab.com/pricing/feature-comparison/) with the exception of the unsupported features [listed below](#features-that-are-not-available).
## Features that are not available
diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md
index 1df222ac349..47c1f730746 100644
--- a/doc/subscriptions/index.md
+++ b/doc/subscriptions/index.md
@@ -77,6 +77,8 @@ With the [Customers Portal](https://customers.gitlab.com/) you can:
- [Change the namespace the subscription is linked to](#change-the-linked-namespace)
- [Change customers portal account password](#change-customers-portal-account-password)
+The Customers Portal is available only to customers who purchased their subscription from GitLab. If you made your purchase through a partner or reseller, you must contact them directly for assistance with your subscription.
+
### Change account owner information
Account owner personal details are used on invoices. The account owner email
@@ -151,11 +153,15 @@ To change the namespace linked to a subscription:
[linked](#change-the-linked-account) GitLab SaaS account.
1. Navigate to the **Manage Purchases** page.
1. Select **Change linked namespace**.
-1. Select the desired group from the **This subscription is for** dropdown. For a group to appear
- here, you must have the Owner role
- for that group.
+1. Select the desired group from the **Select user or group** dropdown list. For a group to appear
+ here, you must have the Owner role for that group.
1. Select **Proceed to checkout**.
+If the group you want to link does not appear in the dropdown list, check:
+
+- You have [linked your Customers Portal account with your GitLab.com account](#change-the-linked-account).
+- That the linked account is a member of the group you want to select, and you are assigned the Owner role.
+
Subscription charges are calculated based on the total number of users in a group, including its subgroups and nested projects. If the [total number of users](gitlab_com/index.md#view-seat-usage) exceeds the number of seats in your subscription, your account is charged for the additional users and you need to pay for the overage before you can change the linked namespace.
Only one namespace can be linked to a subscription.
@@ -177,7 +183,7 @@ For qualifying non-profit educational institutions, the [GitLab for Education Pr
### GitLab for Open Source
-For qualifying open source projects, the [GitLab for Open Source Program](https://about.gitlab.com/solutions/open-source/) provides GitLab Ultimate, plus 50,000 CI/CD minutes per month. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Open Source Program page](https://about.gitlab.com/solutions/open-source/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/opensource-program/).
+For qualifying open source projects, the [GitLab for Open Source Program](https://about.gitlab.com/solutions/open-source/) provides GitLab Ultimate, plus 50,000 CI/CD minutes per month. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Open Source Program page](https://about.gitlab.com/solutions/open-source/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/opensource-program/).
#### Meeting GitLab for Open Source Program requirements
@@ -238,7 +244,7 @@ Exceptions to this public visibility requirement apply in select circumstances (
### GitLab for Startups
-For qualifying startups, the [GitLab for Startups](https://about.gitlab.com/solutions/startups/) program provides GitLab Ultimate, plus 50,000 CI/CD minutes per month for 12 months. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Startups Program page](https://about.gitlab.com/solutions/startups/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/startups-program/).
+For qualifying startups, the [GitLab for Startups](https://about.gitlab.com/solutions/startups/) program provides GitLab Ultimate, plus 50,000 CI/CD minutes per month for 12 months. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Startups Program page](https://about.gitlab.com/solutions/startups/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/startups-program/).
## Contact Support
@@ -259,6 +265,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/subscriptions/self_managed/index.md b/doc/subscriptions/self_managed/index.md
index 4538e8587c9..c4110fe4638 100644
--- a/doc/subscriptions/self_managed/index.md
+++ b/doc/subscriptions/self_managed/index.md
@@ -33,7 +33,7 @@ The cost of a GitLab self-managed subscription is determined by the following:
Pricing is [tier-based](https://about.gitlab.com/pricing/), so you can choose
the features that fit your budget. For information on the features available
for each tier, see the
-[GitLab self-managed feature comparison](https://about.gitlab.com/pricing/self-managed/feature-comparison/).
+[GitLab self-managed feature comparison](https://about.gitlab.com/pricing/feature-comparison/).
## Subscription seats
@@ -212,7 +212,7 @@ Example of a license sync request:
### Troubleshoot automatic subscription sync
If the sync job is not working, ensure you allow network traffic from your GitLab instance
-to IP address `104.18.26.123:443` (`customers.gitlab.com`).
+to IP addresses `172.64.146.11:443` and `104.18.41.245:443` (`customers.gitlab.com`).
## Manually sync your subscription details
@@ -460,3 +460,38 @@ If your credit card is declined when purchasing a GitLab subscription, possible
Check with your financial institution to confirm if any of these reasons apply. If they don't
apply, contact [GitLab Support](https://support.gitlab.com/hc/en-us/requests/new?ticket_form_id=360000071293).
+
+### Check daily and historical billable users
+
+Administrators can get a list of daily and historical billable users in your GitLab instance.
+
+1. [Start a Rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session).
+1. Count the number of users in the instance:
+
+ ```ruby
+ User.billable.count
+ ```
+
+1. Get the historical maximum number of users on the instance from the past year:
+
+ ```ruby
+ ::HistoricalData.max_historical_user_count(from: 1.year.ago.beginning_of_day, to: Time.current.end_of_day)
+ ```
+
+### Update daily billable and historical users
+
+Administrators can trigger a manual update of the daily and historical billable users in your GitLab instance.
+
+1. [Start a Rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session).
+1. Force an update of the daily billable users:
+
+ ```ruby
+ identifier = Analytics::UsageTrends::Measurement.identifiers[:billable_users]
+ ::Analytics::UsageTrends::CounterJobWorker.new.perform(identifier, User.minimum(:id), User.maximum(:id), Time.zone.now)
+ ```
+
+1. Force an update of the historical max billable users:
+
+ ```ruby
+ ::HistoricalDataWorker.new.perform
+ ```
diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md
index 56f3de528b8..8a041b08a4d 100644
--- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md
+++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md
@@ -101,13 +101,12 @@ or manually with Google Cloud Shell:
1. After the Cloud Shell starts, run these commands to install NGINX Ingress Controller:
```shell
- kubectl create ns gitlab-managed-apps
- helm repo add stable https://charts.helm.sh/stable
- helm repo update
- helm install ingress stable/nginx-ingress -n gitlab-managed-apps
+ helm upgrade --install ingress-nginx ingress-nginx \
+ --repo https://kubernetes.github.io/ingress-nginx \
+ --namespace gitlab-managed-apps --create-namespace
# Check that the ingress controller is installed successfully
- kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps
+ kubectl get service ingress-nginx-controller -n gitlab-managed-apps
```
## Configure Auto DevOps
@@ -118,7 +117,7 @@ Follow these steps to configure the base domain and other settings required for
get the external IP address with the following command:
```shell
- kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps -ojson | jq -r '.status.loadBalancer.ingress[].ip'
+ kubectl get service ingress-nginx-controller -n gitlab-managed-apps -ojson | jq -r '.status.loadBalancer.ingress[].ip'
```
Replace `gitlab-managed-apps` if you have overwritten your namespace.
diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md
index 0d4afc3c464..264ac790453 100644
--- a/doc/topics/autodevops/customize.md
+++ b/doc/topics/autodevops/customize.md
@@ -159,7 +159,7 @@ to `CI_COMMIT_SHA,CI_ENVIRONMENT_NAME`.
```
When `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` is set, Auto DevOps
-enables the experimental [Docker BuildKit](https://docs.docker.com/develop/develop-images/build_enhancements/)
+enables the experimental [Docker BuildKit](https://docs.docker.com/build/buildkit/)
feature to use the `--secret` flag.
## Custom Helm Chart
@@ -400,6 +400,7 @@ applications.
| `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | Used to set a username to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD`. |
| `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | Used to set a password to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME`. |
| `AUTO_DEVOPS_CHART_REPOSITORY_PASS_CREDENTIALS` | From GitLab 14.2, set to a non-empty value to enable forwarding of the Helm repository credentials to the chart server when the chart artifacts are on a different host than repository. |
+| `AUTO_DEVOPS_COMMON_NAME` | From GitLab 15.5, set to a valid domain name to customize the common name used for the TLS certificate. Defaults to `le-$CI_PROJECT_ID.$KUBE_INGRESS_BASE_DOMAIN`. Set to `false` to not set this alternative host on the Ingress. |
| `AUTO_DEVOPS_DEPLOY_DEBUG` | From GitLab 13.1, if this variable is present, Helm outputs debug logs. |
| `AUTO_DEVOPS_ALLOW_TO_FORCE_DEPLOY_V<N>` | From [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) v1.0.0, if this variable is present, a new major version of chart is forcibly deployed. For more information, see [Ignore warnings and continue deploying](upgrading_auto_deploy_dependencies.md#ignore-warnings-and-continue-deploying). |
| `BUILDPACK_URL` | Buildpack's full URL. [Must point to a URL supported by Pack or Herokuish](#custom-buildpacks). |
diff --git a/doc/topics/autodevops/multiple_clusters_auto_devops.md b/doc/topics/autodevops/multiple_clusters_auto_devops.md
index 49ded4bfb9a..af8c54a8edd 100644
--- a/doc/topics/autodevops/multiple_clusters_auto_devops.md
+++ b/doc/topics/autodevops/multiple_clusters_auto_devops.md
@@ -14,7 +14,7 @@ The [Deploy Job template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib
- `staging`
- `production`
-These environments are tied to jobs using [Auto Deploy](stages.md#auto-deploy), so they must have different deployment domains. You must define separate [`KUBE_CONTEXT`](../../user/clusters/agent/ci_cd_workflow.md#using-the-agent-with-auto-devops) and [`KUBE_INGRESS_BASE_DOMAIN`](requirements.md#auto-devops-base-domain) variables for each of the three environments.
+These environments are tied to jobs using [Auto Deploy](stages.md#auto-deploy), so they must have different deployment domains. You must define separate [`KUBE_CONTEXT`](../../user/clusters/agent/ci_cd_workflow.md#environments-that-use-auto-devops) and [`KUBE_INGRESS_BASE_DOMAIN`](requirements.md#auto-devops-base-domain) variables for each of the three environments.
## Deploy to different clusters
diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md
index 9458d1cc6f9..022ee131e40 100644
--- a/doc/topics/autodevops/stages.md
+++ b/doc/topics/autodevops/stages.md
@@ -316,7 +316,7 @@ To use a custom target instead of the auto-deployed review apps,
set a `DAST_WEBSITE` CI/CD variable to the URL for DAST to scan.
WARNING:
-If [DAST Full Scan](../../user/application_security/dast/index.md#full-scan) is
+If [DAST Full Scan](../../user/application_security/dast/proxy-based.md#full-scan) is
enabled, GitLab strongly advises **not**
to set `DAST_WEBSITE` to any staging or production environment. DAST Full Scan
actively attacks the target, which can take down your application and lead to
diff --git a/doc/topics/awesome_co.md b/doc/topics/awesome_co.md
index 0d725f64f3a..49e39542b2b 100644
--- a/doc/topics/awesome_co.md
+++ b/doc/topics/awesome_co.md
@@ -1,5 +1,5 @@
---
-stage: Ecosystem
+stage: Manage
group: Foundations
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
description: AwesomeCo test data harness created by the Test Data Working Group https://about.gitlab.com/company/team/structure/working-groups/demo-test-data/
diff --git a/doc/topics/git/how_to_install_git/index.md b/doc/topics/git/how_to_install_git/index.md
index 7d753374473..f3ea1431733 100644
--- a/doc/topics/git/how_to_install_git/index.md
+++ b/doc/topics/git/how_to_install_git/index.md
@@ -89,6 +89,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/topics/git/lfs/index.md b/doc/topics/git/lfs/index.md
index 1ab4a8a8acc..98710315652 100644
--- a/doc/topics/git/lfs/index.md
+++ b/doc/topics/git/lfs/index.md
@@ -43,6 +43,24 @@ Documentation for GitLab instance administrators is under [LFS administration do
[add the URL to Git configuration manually](#troubleshooting).
- [Group wikis](../../../user/project/wiki/group.md) do not support Git LFS.
+## How LFS objects affect repository size
+
+When you add an LFS object to a repository, GitLab:
+
+1. Creates an LFS object.
+1. Associates the LFS object with the repository.
+1. Queues a job to recalculate your project's statistics, including storage size and
+ LFS object storage. Your LFS object storage is the sum of the size of all LFS objects
+ associated with the repository.
+
+When your repository is forked, LFS objects from the upstream project are associated
+with the fork. When the fork is created, the LFS object storage for the fork is equal
+to the storage used by the upstream project. If new LFS objects are added to the fork,
+the total object storage changes for the fork, but not the upstream project.
+
+If you create a merge request from the fork back to the upstream project,
+any new LFS objects in the fork become associated with the upstream project.
+
## Using Git LFS
Let's take a look at the workflow for checking large files into your Git
diff --git a/doc/topics/git/lfs/migrate_to_git_lfs.md b/doc/topics/git/lfs/migrate_to_git_lfs.md
index 6e634cde8f0..07c89e52653 100644
--- a/doc/topics/git/lfs/migrate_to_git_lfs.md
+++ b/doc/topics/git/lfs/migrate_to_git_lfs.md
@@ -159,7 +159,7 @@ Consider an example upstream project, `git@gitlab.com:gitlab-tests/test-git-lfs-
1. Navigate to your project's **Settings > Repository** and
expand **Protected branches**.
- 1. Select the default branch from the **Branch** dropdown menu,
+ 1. Select the default branch from the **Branch** dropdown list,
and set up the
**Allowed to push** and **Allowed to merge** rules.
1. Select **Protect**.
@@ -172,7 +172,7 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/topics/git/merge_conflicts.md b/doc/topics/git/merge_conflicts.md
index 91b608ab705..ea37afe1b31 100644
--- a/doc/topics/git/merge_conflicts.md
+++ b/doc/topics/git/merge_conflicts.md
@@ -1,69 +1,11 @@
---
-stage: Create
-group: Source Code
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-comments: false
+redirect_to: '../../user/project/merge_requests/conflicts.md#resolve-conflicts-from-the-command-line'
+remove_date: '2023-02-01'
---
-# Merge conflicts **(FREE)**
+This document was moved to [another location](../../user/project/merge_requests/conflicts.md#resolve-conflicts-from-the-command-line).
-- Happen often
-- Learning to fix conflicts is hard
-- Practice makes perfect
-- Force push after fixing conflicts. Be careful!
-
-## Merge conflicts sample workflow
-
-1. Check out a new branch and edit `conflicts.rb`. Add 'Line4' and 'Line5'.
-1. Commit and push.
-1. Check out `main` and edit `conflicts.rb`. Add 'Line6' and 'Line7' below 'Line3'.
-1. Commit and push to `main``.
-1. Create a merge request and watch it fail.
-1. Rebase our new branch with `main`.
-1. Fix conflicts on the `conflicts.rb` file.
-1. Stage the file and continue rebasing.
-1. Force push the changes.
-1. Finally continue with the merge request.
-
-```shell
-git checkout -b conflicts_branch
-
-# vi conflicts.rb
-# Add 'Line4' and 'Line5'
-
-git commit -am "add line4 and line5"
-git push origin conflicts_branch
-
-git checkout main
-
-# vi conflicts.rb
-# Add 'Line6' and 'Line7'
-git commit -am "add line6 and line7"
-git push origin main
-```
-
-Create a merge request on the GitLab web UI, and a conflict warning displays.
-
-```shell
-git checkout conflicts_branch
-git fetch
-git rebase main
-
-# Fix conflicts by editing the files.
-
-git add conflicts.rb
-# No need to commit this file
-
-git rebase --continue
-
-# Remember that we have rewritten our commit history so we
-# need to force push so that our remote branch is restructured
-git push origin conflicts_branch -f
-```
-
-## Note
-
-- When to use `git merge` and when to use `git rebase`
-- Rebase when updating your branch with `main`
-- Merge when bringing changes from feature to `main`
-- Reference: <https://www.atlassian.com/git/tutorials/merging-vs-rebasing>
+<!-- This redirect file can be deleted after <2023-02-01>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/topics/git/numerous_undo_possibilities_in_git/index.md b/doc/topics/git/numerous_undo_possibilities_in_git/index.md
index e1e4300f42c..678e03a2c13 100644
--- a/doc/topics/git/numerous_undo_possibilities_in_git/index.md
+++ b/doc/topics/git/numerous_undo_possibilities_in_git/index.md
@@ -402,7 +402,7 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/topics/git/useful_git_commands.md b/doc/topics/git/useful_git_commands.md
index 26ad155054b..4156a3078da 100644
--- a/doc/topics/git/useful_git_commands.md
+++ b/doc/topics/git/useful_git_commands.md
@@ -217,6 +217,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/topics/release_your_application.md b/doc/topics/release_your_application.md
index 6f1c7bb4a40..851396f9bf3 100644
--- a/doc/topics/release_your_application.md
+++ b/doc/topics/release_your_application.md
@@ -28,7 +28,7 @@ deployment using GitLab CI/CD.
With the extensive integration between GitLab and Kubernetes, you can safely deploy your applications
to Kubernetes clusters using the [GitLab agent](../user/clusters/agent/install/index.md).
-#### GitOps deployments **(PREMIUM)**
+#### GitOps deployments
With the [GitLab agent for Kubernetes](../user/clusters/agent/install/index.md), you can perform
[pull-based deployments of Kubernetes manifests](../user/clusters/agent/gitops.md). This provides a scalable, secure,
diff --git a/doc/tutorials/agile_sprint.md b/doc/tutorials/agile_sprint.md
new file mode 100644
index 00000000000..8e1886cf464
--- /dev/null
+++ b/doc/tutorials/agile_sprint.md
@@ -0,0 +1,101 @@
+---
+stage: none
+group: Tutorials
+info: For assistance with this tutorial, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects.
+---
+
+# Tutorial: Use GitLab to run an Agile iteration
+
+To run an Agile development iteration in GitLab, you use multiple GitLab features
+that work together.
+
+To run an Agile iteration from GitLab:
+
+1. Create a group.
+1. Create a project.
+1. Set up an iteration cadence.
+1. Create scoped labels.
+1. Create your epics and issues.
+1. Create an issue board.
+
+After you've created these core components, you can begin running your iterations.
+
+## Create a group
+
+Iteration cadences are created at the group level, so start by
+[creating one](../user/group/manage.md#create-a-group) if you don't have one already.
+
+You use groups to manage one or more related projects at the same time.
+You add your users as members in the group, and assign them a role. Roles determine
+the [level of permissions](../user/permissions.md) each user has on the projects in the group.
+Membership automatically cascades down to all subgroups and projects.
+
+## Create a project
+
+Now [create one or more projects](../user/project/working_with_projects.md#create-a-project) in your group.
+There are several different ways to create a project. A project contains
+your code and pipelines, but also the issues that are used for planning your upcoming code changes.
+
+## Set up an iteration cadence
+
+Before you start creating epics or issues, create an
+[iteration cadence](../user/group/iterations/index.md#iteration-cadences).
+Iteration cadences contain the individual, sequential iteration timeboxes for planning and reporting
+on your issues.
+
+When creating an iteration cadence, you can decide whether to automatically manage the iterations or
+disable the automated scheduling to
+[manually manage the iterations](../user/group/iterations/index.md#manual-iteration-management).
+
+Similar to membership, iterations cascade down your group, subgroup, and project hierarchy. If your
+team works across many groups, subgroups, and projects, create the iteration cadence in the top-most
+group shared by all projects that contain the team's issues as illustrated by the diagram below.
+
+```mermaid
+graph TD
+ Group --> SubgroupA --> Project1
+ Group --> SubgroupB --> Project2
+ Group --> IterationCadence
+```
+
+## Create scoped labels
+
+You should also [create scoped labels](../user/project/labels.md) in the same group where you created
+your iteration cadence. Labels help you
+organize your epics, issues, and merge requests, as well as help you
+to visualize the flow of issues in boards. For example, you can use scoped labels like
+`workflow::planning`, `workflow::ready for development`, `workflow::in development`, and `workflow::complete`
+to indicate the status of an issue. You can also leverage scoped labels to denote the type of issue
+or epic such as `type::feature`, `type::defect`, and `type::maintenance`.
+
+## Create your epics and issues
+
+Now you can get started planning your iterations. Start by creating [epics](../user/group/epics/index.md)
+in the group where you created your iteration cadence,
+then create child [issues](../user/project/issues/index.md) in one or more of your projects.
+Add labels to each as needed.
+
+## Create an issue board
+
+[Issue boards](../user/project/issue_board.md) help you plan your upcoming iterations or visualize
+the workflow of the iteration currently in progress. List columns can be created based on label,
+assignee, iteration, or milestone. You can also filter the board by multiple attributes and group
+issues by their epic.
+
+In the group where you created your iteration cadence and labels,
+[create an issue board](../user/project/issue_board.md#create-an-issue-board) and name it
+"Iteration Planning." Then, create lists for each of your iterations. You can then drag issues from
+the "Open" list into iteration lists to schedule them for upcoming iterations.
+
+To visualize the workflow for issues in the current iteration, create another issue board called
+"Current Iteration." When you're creating the board:
+
+1. Select **Edit board**.
+1. Next to **Iteration**, select **Edit**.
+1. From the dropdown list, select **Current iteration**.
+1. Select **Save changes**.
+
+Your board will now only ever show issues that are in the current iteration.
+You can start adding lists for each of the `workflow::...` labels you created previously.
+
+Now you're ready to start development.
diff --git a/doc/tutorials/index.md b/doc/tutorials/index.md
index 92e96a059dc..9b264a8c636 100644
--- a/doc/tutorials/index.md
+++ b/doc/tutorials/index.md
@@ -1,6 +1,6 @@
---
stage: none
-group: unassigned
+group: Tutorials
info: For assistance with this tutorials page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects.
---
@@ -44,6 +44,8 @@ collaborating, and more.
|-------|-------------|--------------------|
| [Create a project from a template](https://gitlab.com/projects/new#create_from_template) | For hands-on learning, select **Sample GitLab Project** and create a project with example issues and merge requests. | **{star}** |
| [Migrate to GitLab](../user/project/import/index.md) | If you are coming to GitLab from another platform, you can import or convert your projects. | |
+| [Run an agile iteration](agile_sprint.md) | Use group, projects, and iterations to run an agile development iteration. |
+| <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Use GitLab for multi-team planning (SAFe)](https://www.youtube.com/watch?v=KmASFwSap7c) (37m 37s) | A use case of a multi-team organization that uses GitLab with [Scaled Agile Framework (SAFe)](https://about.gitlab.com/solutions/agile-delivery/scaled-agile/). |
## Use CI/CD pipelines
@@ -51,7 +53,7 @@ CI/CD pipelines are used to automatically build, test, and deploy your code.
| Topic | Description | Good for beginners |
|-------|-------------|--------------------|
-| [Get started: Create a pipeline](../ci/quick_start/index.md) | Create a `.gitlab-ci.yml` file and start a pipeline. | **{star}** |
+| [Create and run your first GitLab CI/CD pipeline](../ci/quick_start/index.md) | Create a `.gitlab-ci.yml` file and start a pipeline. | **{star}** |
| <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Get started: Learn about CI/CD](https://www.youtube.com/watch?v=sIegJaLy2ug) (9m 02s) | Learn about the `.gitlab-ci.yml` file and how it's used. | **{star}** |
| <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [CI deep dive](https://www.youtube.com/watch?v=ZVUbmVac-m8&list=PL05JrBw4t0KorkxIFgZGnzzxjZRCGROt_&index=27) (22m 51s) | Take a closer look at pipelines and continuous integration concepts. | |
| <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [CD deep dive](https://www.youtube.com/watch?v=Cn0rzND-Yjw&list=PL05JrBw4t0KorkxIFgZGnzzxjZRCGROt_&index=10) (47m 54s) | Learn about deploying in GitLab. | |
@@ -114,7 +116,6 @@ content:
- Find learning tracks and certification options at [GitLab Learn](https://about.gitlab.com/learn/).
GitLab learning platform login required (email and password for non-GitLab team members).
- For more information, see [First time login details](https://about.gitlab.com/handbook/people-group/learning-and-development/gitlab-learn/user/#first-time-login-to-gitlab-learn).
- Find recent tutorials on the GitLab blog by [searching by the `tutorial` tag](https://about.gitlab.com/blog/tags.html#tutorial).
diff --git a/doc/tutorials/make_your_first_git_commit.md b/doc/tutorials/make_your_first_git_commit.md
index cb6f41bff6c..3df65389c76 100644
--- a/doc/tutorials/make_your_first_git_commit.md
+++ b/doc/tutorials/make_your_first_git_commit.md
@@ -1,10 +1,10 @@
---
stage: none
-group: unassigned
+group: Tutorials
info: For assistance with this tutorial, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects.
---
-# Make your first Git commit
+# Tutorial: Make your first Git commit
This tutorial is going to teach you a little bit about how Git works. It walks
you through the steps of creating your own project, editing a file, and
@@ -37,11 +37,12 @@ Each time you push a change, Git records it as a unique *commit*. These commits
the history of when and how a file changed, and who changed it.
```mermaid
-graph TB
+graph LR
subgraph Repository commit history
- A(Author: Alex<br>Date: 3 Jan at 1PM<br>Commit message: Added sales figures for January<br> Commit ID: 123abc12) ---> B
- B(Author: Sam<br>Date: 4 Jan at 10AM<br>Commit message: Removed outdated marketing information<br> Commit ID: aabb1122) ---> C
- C(Author: Zhang<br>Date: 5 Jan at 3PM<br>Commit message: Added a new 'Invoices' file<br> Commit ID: ddee4455)
+ direction LR
+ A(Author: Alex<br>Date: 3 Jan at 1PM<br>Commit message: Added sales figures<br> Commit ID: 123abc12) ---> B
+ B(Author: Sam<br>Date: 4 Jan at 10AM<br>Commit message: Removed old info<br> Commit ID: aabb1122) ---> C
+ C(Author: Zhang<br>Date: 5 Jan at 3PM<br>Commit message: Added invoices<br> Commit ID: ddee4455)
end
```
@@ -54,15 +55,14 @@ of a repository are in a default branch. To make changes, you:
1. When you're ready, *merge* your branch into the default branch.
```mermaid
-flowchart TB
+flowchart LR
subgraph Default branch
A[Commit] --> B[Commit] --> C[Commit] --> D[Commit]
end
subgraph My branch
B --1. Create my branch--> E(Commit)
E --2. Add my commit--> F(Commit)
- F --2. Add my commit--> G(Commit)
- G --3. Merge my branch to default--> D
+ F --3. Merge my branch to default--> D
end
```
diff --git a/doc/tutorials/move_personal_project_to_a_group.md b/doc/tutorials/move_personal_project_to_a_group.md
index 2106f6ec1c9..ad86851393a 100644
--- a/doc/tutorials/move_personal_project_to_a_group.md
+++ b/doc/tutorials/move_personal_project_to_a_group.md
@@ -1,6 +1,6 @@
---
stage: none
-group: unassigned
+group: Tutorials
info: For assistance with this tutorial, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-other-projects-and-subjects.
---
diff --git a/doc/update/deprecations.md b/doc/update/deprecations.md
index ed9b812525e..353fbf191b5 100644
--- a/doc/update/deprecations.md
+++ b/doc/update/deprecations.md
@@ -45,8 +45,138 @@ sole discretion of GitLab Inc.
<div class="announcement-milestone">
+## Announced in 15.6
+
+<div class="deprecation removal-160 breaking-change">
+
+### Configuration fields in GitLab Runner Helm Chart
+
+End of Support: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)<br />
+Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)
+
+WARNING:
+This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
+Review the details carefully before upgrading.
+
+From GitLab 13.6, users can [specify any runner configuration in the GitLab Runner Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html). When we implemented this feature, we deprecated values in the GitLab Helm Chart configuration that were specific to GitLab Runner. These fields are deprecated and we plan to remove them in v1.0 of the GitLab Runner Helm chart.
+
+</div>
+
+<div class="deprecation removal-160 breaking-change">
+
+### GitLab Runner registration token in Runner Operator
+
+End of Support: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)<br />
+Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)
+
+WARNING:
+This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
+Review the details carefully before upgrading.
+
+The [`runner-registration-token`](https://docs.gitlab.com/runner/install/operator.html#install-the-kubernetes-operator) parameter that uses the OpenShift and k8s Vanilla Operator to install a runner on Kubernetes is deprecated. GitLab plans to introduce a new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/) in GitLab 15.8, which introduces a new method for registering runners and eliminates the legacy runner registration token.
+
+</div>
+
+<div class="deprecation removal-160 breaking-change">
+
+### `POST /api/v4/runners` method to register runners
+
+End of Support: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)<br />
+Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)
+
+WARNING:
+This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
+Review the details carefully before upgrading.
+
+The `POST` method operation on the `/api/v4/runners` endpoint is deprecated.
+This endpoint and method [registers](https://docs.gitlab.com/ee/api/runners.html#register-a-new-runner) a runner
+with a GitLab instance at the instance, group, or project level through the API. We plan to remove this endpoint
+and method in GitLab 16.0.
+
+In GitLab 15.8, we plan to implement a new method to bind runners to a GitLab instance,
+as part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/).
+This new architecture introduces a new method for registering runners and will eliminate the legacy
+[runner registration token](https://docs.gitlab.com/ee/security/token_overview.html#runner-registration-tokens).
+From GitLab 16.0 and later, the runner registration methods implemented by the new GitLab Runner token architecture will be the only supported methods.
+
+</div>
+
+<div class="deprecation removal-160 breaking-change">
+
+### `gitlab-runner register` command
+
+End of Support: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)<br />
+Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)
+
+WARNING:
+This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
+Review the details carefully before upgrading.
+
+The command to [register](https://docs.gitlab.com/runner/register/) a runner, `gitlab-runner register` is deprecated.
+GitLab plans to introduce a new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/) in GitLab 15.8,
+which introduces a new method for registering runners and eliminates the legacy
+[runner registration token](https://docs.gitlab.com/ee/security/token_overview.html#runner-registration-tokens).
+The new method will involve passing a [runner authentication token](https://docs.gitlab.com/ee/security/token_overview.html#runner-authentication-tokens-also-called-runner-tokens)
+to a new `gitlab-runner deploy` command.
+
+</div>
+
+<div class="deprecation removal-160 breaking-change">
+
+### `runnerRegistrationToken` parameter for GitLab Runner Helm Chart
+
+End of Support: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)<br />
+Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)
+
+WARNING:
+This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
+Review the details carefully before upgrading.
+
+The [`runnerRegistrationToken`](https://docs.gitlab.com/runner/install/kubernetes.html#required-configuration) parameter to use the GitLab Helm Chart to install a runner on Kubernetes is deprecated.
+
+As part of the new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/), in GitLab 15.8 we plan to introduce:
+
+- A new method to bind runners to a GitLab instance.
+- A unique system ID saved to the `config.toml`, which will ensure traceability between jobs and runners.
+From GitLab 16.0 and later, the methods to register runners introduced by the new GitLab Runner token architecture will be the only supported methods.
+
+</div>
+
+<div class="deprecation removal-160 breaking-change">
+
+### merge_status API field
+
+Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)
+
+WARNING:
+This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
+Review the details carefully before upgrading.
+
+The `merge_status` field in the [merge request API](https://docs.gitlab.com/ee/api/merge_requests.html#merge-status) has been deprecated in favor of the `detailed_merge_status` field which more correctly identifies all of the potential statuses that a merge request can be in. API users are encouraged to use the new `detailed_merge_status` field instead. The `merge_status` field will be removed in v5 of the GitLab REST API.
+
+</div>
+</div>
+
+<div class="announcement-milestone">
+
## Announced in 15.5
+<div class="deprecation removal-157 breaking-change">
+
+### File Type variable expansion in `.gitlab-ci.yml`
+
+Planned removal: GitLab <span class="removal-milestone">15.7</span> ()
+
+WARNING:
+This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
+Review the details carefully before upgrading.
+
+Previously, variables that referenced or applied alias file variables expanded the value of the `File` type variable. For example, the file contents. This behavior was incorrect because it did not comply with typical shell variable expansion rules. To leak secrets or sensitive information stored in `File` type variables, a user could run an $echo command with the variable as an input parameter.
+
+This breaking change fixes this issue but could disrupt user workflows that work around the behavior. With this change, job variable expansions that reference or apply alias file variables, expand to the file name or path of the `File` type variable, instead of its value, such as the file contents.
+
+</div>
+
<div class="deprecation removal-160 breaking-change">
### GraphQL field `confidential` changed to `internal` on notes
@@ -1063,7 +1193,7 @@ WARNING:
This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
Review the details carefully before upgrading.
-[Request profiling](https://docs.gitlab.com/ee/administration/monitoring/performance/request_profiling.html) is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0.
+[Request profiling](https://docs.gitlab.com/ee/administration/monitoring/performance/index.html) is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0.
We're working on [consolidating our profiling tools](https://gitlab.com/groups/gitlab-org/-/epics/7327) and making them more easily accessible.
We [evaluated](https://gitlab.com/gitlab-org/gitlab/-/issues/350152) the use of this feature and we found that it is not widely used.
@@ -1130,10 +1260,12 @@ In GitLab 15.4, GitLab SAST will no longer use the following analyzers:
- [Bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) (Python)
NOTE:
-This change was originally planned for GitLab 15.0 and has been postponed.
+This change was originally planned for GitLab 15.0 and was postponed to GitLab 15.4.
+See [the removal notice](./removals.md#sast-analyzer-consolidation-and-cicd-template-changes) for further details.
These analyzers will be removed from the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) and replaced with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep).
-They will no longer receive routine updates, except for security issues.
+Effective immediately, they will receive only security updates; other routine improvements or updates are not guaranteed.
+After these analyzers reach End of Support, no further updates will be provided.
We will not delete container images previously published for these analyzers; any such change would be announced as a [deprecation, removal, or breaking change announcement](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes).
We will also remove Java from the scope of the [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) analyzer and replace it with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep).
@@ -1656,7 +1788,7 @@ Planned removal: GitLab <span class="removal-milestone">15.0</span> (2022-05-22)
The Static Site Editor will no longer be available starting in GitLab 15.0. Improvements to the Markdown editing experience across GitLab will deliver smiliar benefit but with a wider reach. Incoming requests to the Static Site Editor will be redirected to the [Web IDE](https://docs.gitlab.com/ee/user/project/web_ide/index.html).
-Current users of the Static Site Editor can view the [documentation](https://docs.gitlab.com/ee/user/project/static_site_editor/) for more information, including how to remove the configuration files from existing projects.
+Current users of the Static Site Editor can view the [documentation](https://docs.gitlab.com/ee/user/project/web_ide/index.html) for more information, including how to remove the configuration files from existing projects.
</div>
@@ -1691,17 +1823,17 @@ only supported report file in 15.0, but this is the first step towards GitLab su
</div>
-<div class="deprecation removal-150 breaking-change">
+<div class="deprecation removal-160 breaking-change">
### merged_by API field
-Planned removal: GitLab <span class="removal-milestone">15.0</span> (2022-05-22)
+Planned removal: GitLab <span class="removal-milestone">16.0</span> (2023-05-22)
WARNING:
This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
Review the details carefully before upgrading.
-The `merged_by` field in the [merge request API](https://docs.gitlab.com/ee/api/merge_requests.html#list-merge-requests) is being deprecated and will be removed in GitLab 15.0. This field is being replaced with the `merge_user` field (already present in GraphQL) which more correctly identifies who merged a merge request when performing actions (merge when pipeline succeeds, add to merge train) other than a simple merge.
+The `merged_by` field in the [merge request API](https://docs.gitlab.com/ee/api/merge_requests.html#list-merge-requests) has been deprecated in favor of the `merge_user` field which more correctly identifies who merged a merge request when performing actions (merge when pipeline succeeds, add to merge train) other than a simple merge. API users are encouraged to use the new `merge_user` field instead. The `merged_by` field will be removed in v5 of the GitLab REST API.
</div>
</div>
@@ -2090,7 +2222,7 @@ WARNING:
This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
Review the details carefully before upgrading.
-[GitLab Serverless](https://docs.gitlab.com/ee/user/project/clusters/serverless/) is a feature set to support Knative-based serverless development with automatic deployments and monitoring.
+GitLab Serverless is a feature set to support Knative-based serverless development with automatic deployments and monitoring.
We decided to remove the GitLab Serverless features as they never really resonated with our users. Besides, given the continuous development of Kubernetes and Knative, our current implementations do not even work with recent versions.
diff --git a/doc/update/index.md b/doc/update/index.md
index 06609f03952..dbac4304897 100644
--- a/doc/update/index.md
+++ b/doc/update/index.md
@@ -143,6 +143,8 @@ GitLab 14.0 introduced [batched background migrations](../user/admin_area/monito
Some installations [may need to run GitLab 14.0 for at least a day](#1400) to complete the database changes introduced by that upgrade.
+Batched background migrations are handled by Sidekiq and [run in isolation](../development/database/batched_background_migrations.md#isolation), so an instance can remain operational while the migrations are processed. However, there may be performance degradation on larger instances that are heavily used while batched background migrations are run, so it's a good idea to [actively monitor the Sidekiq status](../user/admin_area/index.md#background-jobs) until all migrations are completed.
+
#### Check the status of batched background migrations
To check the status of batched background migrations:
@@ -376,24 +378,26 @@ Upgrading across multiple GitLab versions in one go is *only possible by accepti
The following examples assume downtime is acceptable while upgrading.
If you don't want any downtime, read how to [upgrade with zero downtime](zero_downtime.md).
+For a dynamic view of examples of supported upgrade paths, try the [Upgrade Path tool](https://gitlab-com.gitlab.io/support/toolbox/upgrade-path/) maintained by the [GitLab Support team](https://about.gitlab.com/handbook/support/#about-the-support-team). To share feedback and help improve the tool, create an issue or MR in the [upgrade-path project](https://gitlab.com/gitlab-com/support/toolbox/upgrade-path).
+
Find where your version sits in the upgrade path below, and upgrade GitLab
accordingly, while also consulting the
[version-specific upgrade instructions](#version-specific-upgrading-instructions):
-`8.11.Z` -> `8.12.0` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> [`11.11.8`](#1200) -> `12.0.12` -> [`12.1.17`](#1210) -> [`12.10.14`](#12100) -> `13.0.14` -> [`13.1.11`](#1310) -> [`13.8.8`](#1388) -> [`13.12.15`](#13120) -> [`14.0.12`](#1400) -> [`14.3.6`](#1430) -> [`14.9.5`](#1490) -> [`14.10.Z`](#14100) -> [`15.0.Z`](#1500) -> [`15.4.0`](#1540) -> [latest `15.Y.Z`](https://gitlab.com/gitlab-org/gitlab/-/releases)
+`8.11.Z` -> `8.12.0` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> [`11.11.8`](#1200) -> `12.0.12` -> [`12.1.17`](#1210) -> [`12.10.14`](#12100) -> `13.0.14` -> [`13.1.11`](#1310) -> [`13.8.8`](#1388) -> [`13.12.15`](#13120) -> [`14.0.12`](#1400) -> [`14.3.6`](#1430) -> [`14.9.5`](#1490) -> [`14.10.Z`](#14100) -> [`15.0.Z`](#1500) -> [`15.1.Z`](#1510) (for GitLab instances with multiple web nodes) -> [`15.4.0`](#1540) -> [latest `15.Y.Z`](https://gitlab.com/gitlab-org/gitlab/-/releases)
NOTE:
When not explicitly specified, upgrade GitLab to the latest available patch
release rather than the first patch release, for example `13.8.8` instead of `13.8.0`.
This includes versions you must stop at on the upgrade path as there may
be fixes for issues relating to the upgrade process.
+Specifically around a [major version](#upgrading-to-a-new-major-version),
+crucial database schema and migration patches are included in the latest patch releases.
The following table, while not exhaustive, shows some examples of the supported
upgrade paths.
Additional steps between the mentioned versions are possible. We list the minimally necessary steps only.
-For a dynamic view of examples of supported upgrade paths, try the [Upgrade Path tool](https://gitlab-com.gitlab.io/support/toolbox/upgrade-path/). The Upgrade Path tool is maintained by the [GitLab Support team](https://about.gitlab.com/handbook/support/#about-the-support-team). Share feedback and help improve the tool by raising an issue or MR in the [upgrade-path project](https://gitlab.com/gitlab-com/support/toolbox/upgrade-path).
-
| Target version | Your version | Supported upgrade path | Note |
| -------------- | ------------ | ---------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `15.1.0` | `14.6.2` | `14.6.2` -> `14.9.5` -> `14.10.5` -> `15.0.2` -> `15.1.0` | Three intermediate versions are required: `14.9` and `14.10`, `15.0`, then `15.1.0`. |
@@ -467,10 +471,32 @@ NOTE:
Specific information that follow related to Ruby and Git versions do not apply to [Omnibus installations](https://docs.gitlab.com/omnibus/)
and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with appropriate Ruby and Git versions and are not using system binaries for Ruby and Git. There is no need to install Ruby or Git when utilizing these two approaches.
+### 15.6.0
+
+- Git 2.37.0 and later is required by Gitaly. For installations from source, we recommend you use the [Git version provided by Gitaly](../install/installation.md#git).
+
+### 15.5.0
+
+- GitLab 15.4.0 introduced a default [Sidekiq routing rule](../administration/sidekiq/extra_sidekiq_routing.md) that routes all jobs to the `default` queue. For instances using [queue selectors](../administration/sidekiq/extra_sidekiq_processes.md#queue-selector), this will cause [performance problems](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1991) as some Sidekiq processes will be idle.
+ - The default routing rule has been reverted in 15.5.4, so upgrading to that version or later will return to the previous behavior.
+ - If a GitLab instance now listens only to the `default` queue (which is not currently recommended), it will be required to add this routing rule back in `/etc/gitlab/gitlab.rb`:
+
+ ```ruby
+ sidekiq['routing_rules'] = [['*', 'default']]
+ ```
+
### 15.4.0
- GitLab 15.4.0 includes a [batched background migration](#batched-background-migrations) to [remove incorrect values from `expire_at` in `ci_job_artifacts` table](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89318).
This migration might take hours or days to complete on larger GitLab instances.
+- By default, Gitaly and Praefect nodes use the time server at `pool.ntp.org`. If your instance can not connect to `pool.ntp.org`, [configure the `NTP_HOST` variable](../administration/gitaly/praefect.md#customize-time-server-setting).
+- GitLab 15.4.0 introduced a default [Sidekiq routing rule](../administration/sidekiq/extra_sidekiq_routing.md) that routes all jobs to the `default` queue. For instances using [queue selectors](../administration/sidekiq/extra_sidekiq_processes.md#queue-selector), this will cause [performance problems](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1991) as some Sidekiq processes will be idle.
+ - The default routing rule has been reverted in 15.4.5, so upgrading to that version or later will return to the previous behavior.
+ - If a GitLab instance now listens only to the `default` queue (which is not currently recommended), it will be required to add this routing rule back in `/etc/gitlab/gitlab.rb`:
+
+ ```ruby
+ sidekiq['routing_rules'] = [['*', 'default']]
+ ```
### 15.3.3
diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md
index 6e153d5132d..e0c0cdf31f9 100644
--- a/doc/update/patch_versions.md
+++ b/doc/update/patch_versions.md
@@ -60,11 +60,6 @@ sudo -u git -H bundle clean
# Run database migrations
sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
-# Compile GetText PO files
-# Internationalization was added in `v9.2.0` so this command is only
-# required for versions equal or major to it.
-sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production
-
# Clean up assets and cache
sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile cache:clear RAILS_ENV=production NODE_ENV=production NODE_OPTIONS="--max_old_space_size=4096"
```
diff --git a/doc/update/removals.md b/doc/update/removals.md
index 392259176bb..0bc82403f60 100644
--- a/doc/update/removals.md
+++ b/doc/update/removals.md
@@ -31,6 +31,18 @@ For removal reviewers (Technical Writers only):
https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-removals-doc
-->
+## Removed in 16.0
+
+### Changing merge request approvals with the `/approvals` API endpoint
+
+WARNING:
+This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
+Review the details carefully before upgrading.
+
+To change the approvals required for a merge request, you should no longer use the `/approvals` API endpoint, which was deprecated in GitLab 12.3.
+
+Instead, use the [`/approval_rules` endpoint](https://docs.gitlab.com/ee/api/merge_request_approvals.html#merge-request-level-mr-approvals) to [create](https://docs.gitlab.com/ee/api/merge_request_approvals.html#create-merge-request-level-rule) or [update](https://docs.gitlab.com/ee/api/merge_request_approvals.html#update-merge-request-level-rule) the approval rules for a merge request.
+
## Removed in 15.4
### SAST analyzer consolidation and CI/CD template changes
@@ -251,7 +263,7 @@ Elasticsearch 6.8 support has been removed in GitLab 15.0. Elasticsearch 6.8 has
If you use Elasticsearch 6.8, **you must upgrade your Elasticsearch version to 7.x** prior to upgrading to GitLab 15.0.
You should not upgrade to Elasticsearch 8 until you have completed the GitLab 15.0 upgrade.
-View the [version requirements](https://docs.gitlab.com/ee/integration/elasticsearch.html#version-requirements) for details.
+View the [version requirements](https://docs.gitlab.com/ee/integration/advanced_search/elasticsearch.html#version-requirements) for details.
### End of support for Python 3.6 in Dependency Scanning
@@ -444,7 +456,7 @@ WARNING:
This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
Review the details carefully before upgrading.
-[Request profiling](https://docs.gitlab.com/ee/administration/monitoring/performance/request_profiling.html) has been removed in GitLab 15.0.
+[Request profiling](https://docs.gitlab.com/ee/administration/monitoring/performance/index.html) has been removed in GitLab 15.0.
We're working on [consolidating our profiling tools](https://gitlab.com/groups/gitlab-org/-/epics/7327) and making them more easily accessible.
We [evaluated](https://gitlab.com/gitlab-org/gitlab/-/issues/350152) the use of this feature and we found that it is not widely used.
@@ -574,7 +586,7 @@ If you installed GitLab from source, verify manually that both servers are confi
### Static Site Editor
-The Static Site Editor was deprecated in GitLab 14.7 and the feature is being removed in GitLab 15.0. Incoming requests to the Static Site Editor will be redirected and open the target file to edit in the Web IDE. Current users of the Static Site Editor can view the [documentation](https://docs.gitlab.com/ee/user/project/static_site_editor/) for more information, including how to remove the configuration files from existing projects. We will continue investing in improvements to the Markdown editing experience by [maturing the Content Editor](https://gitlab.com/groups/gitlab-org/-/epics/5401) and making it available as a way to edit content across GitLab.
+The Static Site Editor was deprecated in GitLab 14.7 and the feature is being removed in GitLab 15.0. Incoming requests to the Static Site Editor will be redirected and open the target file to edit in the Web IDE. Current users of the Static Site Editor can view the [documentation](https://docs.gitlab.com/ee/user/project/web_ide/index.html) for more information, including how to remove the configuration files from existing projects. We will continue investing in improvements to the Markdown editing experience by [maturing the Content Editor](https://gitlab.com/groups/gitlab-org/-/epics/5401) and making it available as a way to edit content across GitLab.
### Support for `gitaly['internal_socket_dir']`
@@ -849,7 +861,7 @@ WARNING:
This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
Review the details carefully before upgrading.
-By default, the Code Quality feature has not provided support for Ruby 2.6+ if you're using the Code Quality template. To better support the latest versions of Ruby, the default RuboCop version is updated to add support for Ruby 2.4 through 3.0. As a result, support for Ruby 2.1, 2.2, and 2.3 is removed. You can re-enable support for older versions by [customizing your configuration](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html#rubocop-errors).
+By default, the Code Quality feature has not provided support for Ruby 2.6+ if you're using the Code Quality template. To better support the latest versions of Ruby, the default RuboCop version is updated to add support for Ruby 2.4 through 3.0. As a result, support for Ruby 2.1, 2.2, and 2.3 is removed. You can re-enable support for older versions by [customizing your configuration](https://docs.gitlab.com/ee/ci/testing/code_quality.html#rubocop-errors).
Relevant Issue: [Default `codeclimate-rubocop` engine does not support Ruby 2.6+](https://gitlab.com/gitlab-org/ci-cd/codequality/-/issues/28)
@@ -892,7 +904,7 @@ WARNING:
This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
Review the details carefully before upgrading.
-Browser Performance Testing has run in a job named `performance` by default. With the introduction of [Load Performance Testing](https://docs.gitlab.com/ee/user/project/merge_requests/load_performance_testing.html) in GitLab 13.2, this naming could be confusing. To make it clear which job is running [Browser Performance Testing](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html), the default job name is changed from `performance` to `browser_performance` in the template in GitLab 14.0.
+Browser Performance Testing has run in a job named `performance` by default. With the introduction of [Load Performance Testing](https://docs.gitlab.com/ee/ci/testing/code_quality.html) in GitLab 13.2, this naming could be confusing. To make it clear which job is running [Browser Performance Testing](https://docs.gitlab.com/ee/ci/testing/browser_performance_testing.html), the default job name is changed from `performance` to `browser_performance` in the template in GitLab 14.0.
Relevant Issue: [Rename default Browser Performance Testing job](https://gitlab.com/gitlab-org/gitlab/-/issues/225914)
@@ -1158,7 +1170,7 @@ WARNING:
This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
Review the details carefully before upgrading.
-The [deployment frequency project-level API](https://docs.gitlab.com/ee/api/dora4_project_analytics.html#list-project-deployment-frequencies) endpoint has been deprecated in favor of the [DORA 4 API](https://docs.gitlab.com/ee/api/dora/metrics.html), which consolidates all the metrics under one API with the specific metric as a required field. As a result, the timestamp field, which doesn't allow adding future extensions and causes performance issues, will be removed. With the old API, an example response would be `{ "2021-03-01": 3, "date": "2021-03-01", "value": 3 }`. The first key/value (`"2021-03-01": 3`) will be removed and replaced by the last two (`"date": "2021-03-01", "value": 3`).
+The [deployment frequency project-level API](https://docs.gitlab.com/ee/api/dora/metrics.html#list-project-deployment-frequencies) endpoint has been deprecated in favor of the [DORA 4 API](https://docs.gitlab.com/ee/api/dora/metrics.html), which consolidates all the metrics under one API with the specific metric as a required field. As a result, the timestamp field, which doesn't allow adding future extensions and causes performance issues, will be removed. With the old API, an example response would be `{ "2021-03-01": 3, "date": "2021-03-01", "value": 3 }`. The first key/value (`"2021-03-01": 3`) will be removed and replaced by the last two (`"date": "2021-03-01", "value": 3`).
### Release description in the Tags API
diff --git a/doc/update/upgrading_from_ce_to_ee.md b/doc/update/upgrading_from_ce_to_ee.md
index 9bb88755686..b99bb3d7992 100644
--- a/doc/update/upgrading_from_ce_to_ee.md
+++ b/doc/update/upgrading_from_ce_to_ee.md
@@ -75,9 +75,6 @@ sudo -u git -H bundle clean
# Run database migrations
sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
-# Compile GetText PO files
-sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production
-
# Update node dependencies and recompile assets
sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production NODE_OPTIONS="--max_old_space_size=4096"
diff --git a/doc/update/upgrading_from_source.md b/doc/update/upgrading_from_source.md
index bb9199560f9..f5b85330f3b 100644
--- a/doc/update/upgrading_from_source.md
+++ b/doc/update/upgrading_from_source.md
@@ -107,11 +107,11 @@ Download and install Go (for Linux, 64-bit):
# Remove former Go installation folder
sudo rm -rf /usr/local/go
-curl --remote-name --location --progress-bar "https://go.dev/dl/go1.17.10.linux-amd64.tar.gz"
-echo '87fc728c9c731e2f74e4a999ef53cf07302d7ed3504b0839027bd9c10edaa3fd go1.17.10.linux-amd64.tar.gz' | shasum -a256 -c - && \
- sudo tar -C /usr/local -xzf go1.17.10.linux-amd64.tar.gz
+curl --remote-name --location --progress-bar "https://go.dev/dl/go1.18.8.linux-amd64.tar.gz"
+echo '4d854c7bad52d53470cf32f1b287a5c0c441dc6b98306dea27358e099698142a go1.18.8.linux-amd64.tar.gz' | shasum -a256 -c - && \
+ sudo tar -C /usr/local -xzf go1.18.8.linux-amd64.tar.gz
sudo ln -sf /usr/local/go/bin/{go,gofmt} /usr/local/bin/
-rm go1.17.10.linux-amd64.tar.gz
+rm go1.18.8.linux-amd64.tar.gz
```
### 6. Update Git
@@ -310,9 +310,6 @@ sudo -u git -H bundle clean
# Run database migrations
sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
-# Compile GetText PO files
-sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production
-
# Update node dependencies and recompile assets
sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production NODE_OPTIONS="--max_old_space_size=4096"
diff --git a/doc/update/zero_downtime.md b/doc/update/zero_downtime.md
index aebd27f84a9..eb1d6cef606 100644
--- a/doc/update/zero_downtime.md
+++ b/doc/update/zero_downtime.md
@@ -395,7 +395,7 @@ HA.
#### In the application node
-According to [official Redis documentation](https://redis.io/docs/manual/admin/#upgrading-or-restarting-a-redis-instance-without-downtime),
+According to [official Redis documentation](https://redis.io/docs/management/admin/#upgrading-or-restarting-a-redis-instance-without-downtime),
the easiest way to update an HA instance using Sentinel is to upgrade the
secondaries one after the other, perform a manual failover from current
primary (running old version) to a recently upgraded secondary (running a new
diff --git a/doc/user/admin_area/appearance.md b/doc/user/admin_area/appearance.md
index 5513fa3585d..fc42c7770f2 100644
--- a/doc/user/admin_area/appearance.md
+++ b/doc/user/admin_area/appearance.md
@@ -94,6 +94,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/broadcast_messages.md b/doc/user/admin_area/broadcast_messages.md
index e5d0a6e297e..acb3e92fff8 100644
--- a/doc/user/admin_area/broadcast_messages.md
+++ b/doc/user/admin_area/broadcast_messages.md
@@ -111,6 +111,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/custom_project_templates.md b/doc/user/admin_area/custom_project_templates.md
index 551ed667250..de2856c2320 100644
--- a/doc/user/admin_area/custom_project_templates.md
+++ b/doc/user/admin_area/custom_project_templates.md
@@ -49,6 +49,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/diff_limits.md b/doc/user/admin_area/diff_limits.md
index 3a1ecac6ee6..3d7c49c1f2b 100644
--- a/doc/user/admin_area/diff_limits.md
+++ b/doc/user/admin_area/diff_limits.md
@@ -47,6 +47,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/email_from_gitlab.md b/doc/user/admin_area/email_from_gitlab.md
index c1d3521d60c..ba465fbea29 100644
--- a/doc/user/admin_area/email_from_gitlab.md
+++ b/doc/user/admin_area/email_from_gitlab.md
@@ -55,6 +55,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/geo_nodes.md b/doc/user/admin_area/geo_nodes.md
deleted file mode 100644
index 710f37bb344..00000000000
--- a/doc/user/admin_area/geo_nodes.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: 'geo_sites.md'
-remove_date: '2022-10-05'
----
-
-This document was moved to [another location](geo_sites.md).
-
-<!-- This redirect file can be deleted after <2022-10-05>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/user/admin_area/geo_sites.md b/doc/user/admin_area/geo_sites.md
index 093ed84f41a..f3be036fd38 100644
--- a/doc/user/admin_area/geo_sites.md
+++ b/doc/user/admin_area/geo_sites.md
@@ -112,6 +112,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/img/minimum_password_length_settings_v12_6.png b/doc/user/admin_area/img/minimum_password_length_settings_v12_6.png
deleted file mode 100644
index 27634a02a8a..00000000000
--- a/doc/user/admin_area/img/minimum_password_length_settings_v12_6.png
+++ /dev/null
Binary files differ
diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md
index e6ddb810933..4c34d82dc02 100644
--- a/doc/user/admin_area/index.md
+++ b/doc/user/admin_area/index.md
@@ -136,15 +136,15 @@ For each user, the following are listed:
1. Date of last activity
To edit a user, select the **Edit** button in that user's
-row. To delete the user, or delete the user and their contributions, select the cog dropdown in
+row. To delete the user, or delete the user and their contributions, select the cog dropdown list in
that user's row, and select the desired option.
To change the sort order:
-1. Select the sort dropdown.
+1. Select the sort dropdown list.
1. Select the desired order.
-By default the sort dropdown shows **Name**.
+By default the sort dropdown list shows **Name**.
To search for users, enter your criteria in the search field. The user search is case
insensitive, and applies partial matching to name and username. To search for an email address,
@@ -260,7 +260,7 @@ number of members, and whether the group is private, internal, or public. To edi
the **Edit** button in that group's row. To delete the group, select the **Delete** button in
that group's row.
-To change the sort order, select the sort dropdown and select the desired order. The default
+To change the sort order, select the sort dropdown list and select the desired order. The default
sort order is by **Last created**.
To search for groups by name, enter your criteria in the search field. The group search is case
diff --git a/doc/user/admin_area/labels.md b/doc/user/admin_area/labels.md
index 524546d447c..8e1ca979707 100644
--- a/doc/user/admin_area/labels.md
+++ b/doc/user/admin_area/labels.md
@@ -22,6 +22,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md
index 5211a18201b..5296a918f56 100644
--- a/doc/user/admin_area/license.md
+++ b/doc/user/admin_area/license.md
@@ -64,6 +64,17 @@ This error occurs when you use an activation code to activate your instance, but
You may have connectivity issues due to the following reasons:
-- **You have an offline environment**: Configure your setup to allow connection to GitLab servers. If connection to GitLab servers is not possible, contact [GitLab support](https://about.gitlab.com/support/#contact-support) to request a license key.
-- **Firewall settings**: Enable an encrypted HTTPS connection from your GitLab instance to `customers.gitlab.com` (with IP addresses 104.18.26.123 and 104.18.27.123) on port 443.
-- **Customers Portal is not operational**: To check for performance or service disruptions, check the Customers Portal [status](https://status.gitlab.com/).
+- **You have an offline environment**:
+ - Configure your setup to allow connection to GitLab servers. If connection to GitLab servers is not possible, contact your Sales Representative to request a license key. You can also contact [GitLab support](https://about.gitlab.com/support/#contact-support) if you need help finding your Sales Representative.
+- **Customers Portal is not operational**:
+ - To check for performance or service disruptions, check the Customers Portal [status](https://status.gitlab.com/).
+- **Firewall settings**:
+ - Check if your GitLab instance has an encrypted connection to `customers.gitlab.com` (with IP addresses 172.64.146.11 and 104.18.41.245) on port 443:
+
+ ```shell
+ curl --verbose "https://customers.gitlab.com/"
+ ```
+
+ - If the curl command returns a failure, either:
+ - [Configure a proxy](https://docs.gitlab.com/omnibus/settings/environment-variables.html) in `gitlab.rb` to point to your server.
+ - Contact your network administrator to make changes to the proxy.
diff --git a/doc/user/admin_area/license_file.md b/doc/user/admin_area/license_file.md
index e6186fb9805..d821d9bab23 100644
--- a/doc/user/admin_area/license_file.md
+++ b/doc/user/admin_area/license_file.md
@@ -25,7 +25,7 @@ Otherwise, to add your license:
1. Select **Add license**.
NOTE:
-For GitLab versions 14.1.x or newer, you can access the **Add License** page directly from the URL, `<YourGitLabURL>/admin/license/new`.
+In GitLab 14.1.x through 14.10.x, you can access the **Add License** page directly from the URL, `<YourGitLabURL>/admin/license/new`. In GitLab 15.0 and later, the path is `<YourGitLabURL>/admin/subscription`.
## Add your license file during installation
@@ -147,3 +147,81 @@ the license.
### `Start GitLab Ultimate trial` still displays after adding license
To fix this issue, restart [Puma or your entire GitLab instance](../../administration/restart_gitlab.md).
+
+### License commands in the rails console
+
+The following commands can be run in the [rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session).
+
+WARNING:
+Any command that changes data directly could be damaging if not run correctly, or under the right conditions.
+We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+
+#### See current license information
+
+```ruby
+# License information (name, company, email address)
+License.current.licensee
+
+# Plan:
+License.current.plan
+
+# Uploaded:
+License.current.created_at
+
+# Started:
+License.current.starts_at
+
+# Expires at:
+License.current.expires_at
+
+# Is this a trial license?
+License.current.trial?
+
+# License ID for lookup on CustomersDot
+License.current.license_id
+
+# License data in Base64-encoded ASCII format
+License.current.data
+```
+
+#### Check if a project feature is available on the instance
+
+Features listed in <https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/license.rb>.
+
+```ruby
+License.current.feature_available?(:jira_dev_panel_integration)
+```
+
+#### Check if a project feature is available in a project
+
+Features listed in [`license.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/license.rb).
+
+```ruby
+p = Project.find_by_full_path('<group>/<project>')
+p.feature_available?(:jira_dev_panel_integration)
+```
+
+#### Add a license through the console
+
+```ruby
+key = "<key>"
+license = License.new(data: key)
+license.save
+License.current # check to make sure it applied
+```
+
+This is needed for example in a known edge-case with
+[expired license and multiple LDAP servers](../../administration/auth/ldap/ldap-troubleshooting.md#expired-license-causes-errors-with-multiple-ldap-servers).
+
+#### Remove licenses
+
+To clean up the [License History table](../../user/admin_area/license_file.md#view-license-details-and-history):
+
+```ruby
+TYPE = :trial?
+# or :expired?
+
+License.select(&TYPE).each(&:destroy!)
+
+# or even License.all.each(&:destroy!)
+```
diff --git a/doc/user/admin_area/moderate_users.md b/doc/user/admin_area/moderate_users.md
index ace1c6be5f8..c0daf029b1f 100644
--- a/doc/user/admin_area/moderate_users.md
+++ b/doc/user/admin_area/moderate_users.md
@@ -54,7 +54,7 @@ To approve or reject a user sign up:
1. On the left sidebar, select **Overview > Users**.
1. Select the **Pending approval** tab.
1. Optional. Select a user.
-1. Select the **{settings}** **User administration** dropdown.
+1. Select the **{settings}** **User administration** dropdown list.
1. Select **Approve** or **Reject**.
Approving a user:
@@ -78,7 +78,7 @@ by removing them in LDAP, or directly from the Admin Area. To do this:
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Users**.
1. Optional. Select a user.
-1. Select the **{settings}** **User administration** dropdown.
+1. Select the **{settings}** **User administration** dropdown list.
1. Select **Block**.
A blocked user:
@@ -102,7 +102,7 @@ A blocked user can be unblocked from the Admin Area. To do this:
1. On the left sidebar, select **Overview > Users**.
1. Select the **Blocked** tab.
1. Optional. Select a user.
-1. Select the **{settings}** **User administration** dropdown.
+1. Select the **{settings}** **User administration** dropdown list.
1. Select **Unblock**.
The user's state is set to active and they consume a
@@ -156,13 +156,13 @@ A user can be deactivated from the Admin Area. To do this:
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Users**.
1. Optional. Select a user.
-1. Select the **{settings}** **User administration** dropdown.
+1. Select the **{settings}** **User administration** dropdown list.
1. Select **Deactivate**.
For the deactivation option to be visible to an administrator, the user:
- Must be currently active.
-- Must not have signed in, or have any activity, in the last 90 days.
+- Must not be [dormant](#automatically-deactivate-dormant-users).
NOTE:
Users can also be deactivated using the [GitLab API](../../api/users.md#deactivate-user).
@@ -203,7 +203,7 @@ To do this:
1. On the left sidebar, select **Overview > Users**.
1. Select the **Deactivated** tab.
1. Optional. Select a user.
-1. Select the **{settings}** **User administration** dropdown.
+1. Select the **{settings}** **User administration** dropdown list.
1. Select **Activate**.
The user's state is set to active and they consume a
@@ -235,7 +235,7 @@ Users can be banned using the Admin Area. To do this:
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Overview > Users**.
1. Optional. Select a user.
-1. Select the **{settings}** **User administration** dropdown.
+1. Select the **{settings}** **User administration** dropdown list.
1. Select **Ban user**.
The banned user does not consume a [seat](../../subscriptions/self_managed/index.md#billable-users).
@@ -248,7 +248,7 @@ A banned user can be unbanned using the Admin Area. To do this:
1. On the left sidebar, select **Overview > Users**.
1. Select the **Banned** tab.
1. Optional. Select a user.
-1. Select the **{settings}** **User administration** dropdown.
+1. Select the **{settings}** **User administration** dropdown list.
1. Select **Unban user**.
The user's state is set to active and they consume a
@@ -283,3 +283,68 @@ You can also delete a user and their contributions, such as merge requests, issu
NOTE:
Before 15.1, additionally groups of which deleted user were the only owner among direct members were deleted.
+
+## Troubleshooting
+
+When moderating users, you may need to perform bulk actions on them based on certain conditions. The following rails console scripts show some examples of this. You may [start a rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session) and use scripts similar to the following:
+
+### Deactivate users that have no recent activity
+
+Administrators can deactivate users that have no recent activity.
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+```ruby
+days_inactive = 90
+inactive_users = User.active.where("last_activity_on <= ?", days_inactive.days.ago)
+
+inactive_users.each do |user|
+ puts "user '#{user.username}': #{user.last_activity_on}"
+ user.deactivate!
+end
+```
+
+### Block users that have no recent activity
+
+Administrators can block users that have no recent activity.
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+```ruby
+days_inactive = 90
+inactive_users = User.active.where("last_activity_on <= ?", days_inactive.days.ago)
+
+inactive_users.each do |user|
+ puts "user '#{user.username}': #{user.last_activity_on}"
+ user.block!
+end
+```
+
+### Block or delete users that have no projects or groups
+
+Administrators can block or delete users that have no projects or groups.
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+```ruby
+users = User.where('id NOT IN (select distinct(user_id) from project_authorizations)')
+
+# How many users are removed?
+users.count
+
+# If that count looks sane:
+
+# You can either block the users:
+users.each { |user| user.blocked? ? nil : user.block! }
+
+# Or you can delete them:
+ # need 'current user' (your user) for auditing purposes
+current_user = User.find_by(username: '<your username>')
+
+users.each do |user|
+ DeleteUserWorker.perform_async(current_user.id, user.id)
+end
+```
diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md
index e6f9c045329..2939a8b0418 100644
--- a/doc/user/admin_area/monitoring/health_check.md
+++ b/doc/user/admin_area/monitoring/health_check.md
@@ -143,6 +143,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/reporting/git_abuse_rate_limit.md b/doc/user/admin_area/reporting/git_abuse_rate_limit.md
index 432205d8fa2..f700b8b1ea3 100644
--- a/doc/user/admin_area/reporting/git_abuse_rate_limit.md
+++ b/doc/user/admin_area/reporting/git_abuse_rate_limit.md
@@ -4,18 +4,20 @@ group: Anti-Abuse
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Git abuse rate limit **(ULTIMATE SELF)**
+# Git abuse rate limit (administration) **(ULTIMATE SELF)**
-> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with flags](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag` and `auto_ban_user_on_excessive_projects_download`. Both flags are disabled by default.
+> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with a flag](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. Disabled by default.
FLAG:
-On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flags](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag` and `auto_ban_user_on_excessive_projects_download`.
+On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. On GitLab.com, this feature is not available.
-Git abuse rate limiting is a feature to automatically [ban users](../moderate_users.md#ban-and-unban-users) who download more than a specified number of repositories in a given time. When the `git_abuse_rate_limit_feature_flag` feature flag is enabled, the administrator receives an email when a user is about to be banned.
+Git abuse rate limiting is a feature to automatically [ban users](../moderate_users.md#ban-and-unban-users) who download or clone more than a specified number of repositories in any project in the instance within a given time frame. Banned users cannot sign in to the instance and cannot access any non-public group via HTTP or SSH.
-When the `auto_ban_user_on_excessive_projects_download` is not enabled, the user is not banned automatically. You can use this setup to determine the correct values of the rate limit settings.
+If the `git_abuse_rate_limit_feature_flag` feature flag is enabled, all application administrators receive an email when a user is about to be banned.
-When both flags are enabled, the administrator receives an email when a user is about to be banned, and the user is automatically banned from the GitLab instance.
+If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, administrators are still notified. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning.
+
+If automatic banning is enabled, administrators receive an email when a user is about to be banned, and the user is automatically banned from the GitLab instance.
## Configure Git abuse rate limiting
@@ -24,6 +26,15 @@ When both flags are enabled, the administrator receives an email when a user is
1. Expand **Git abuse rate limit**.
1. Update the Git abuse rate limit settings:
1. Enter a number in the **Number of repositories** field, greater than or equal to `0` and less than or equal to `10,000`. This number specifies the maximum amount of unique repositories a user can download in the specified time period before they're banned. When set to `0`, Git abuse rate limiting is disabled.
- 1. Enter a number in the **Reporting time period (seconds)** field, greater than or equal to `0` and less than or equal to `86,400`. This number specifies the time in seconds a user can download the maximum amount of repositories before they're banned. When set to `0`, Git abuse rate limiting is disabled.
- 1. Optional. Exclude users by adding them to the **Excluded users** field. Excluded users are not automatically banned.
+ 1. Enter a number in the **Reporting time period (seconds)** field, greater than or equal to `0` and less than or equal to `86,400` (10 days). This number specifies the time in seconds a user can download the maximum amount of repositories before they're banned. When set to `0`, Git abuse rate limiting is disabled.
+ 1. Optional. Exclude up to `100` users by adding them to the **Excluded users** field. Excluded users are not automatically banned.
+ 1. Optional. Turn on the **Automatically ban users from this namespace when they exceed the specified limits** toggle to enable automatic banning.
1. Select **Save changes**.
+
+## Unban a user
+
+1. On the top bar, select **Main menu > Admin**.
+1. On the left sidebar, select **Overview > Users**.
+1. Select the **Banned** tab and search for the account you want to unban.
+1. From the **User administration** dropdown list select **Unban user**.
+1. On the confirmation dialog, select **Unban user**.
diff --git a/doc/user/admin_area/review_abuse_reports.md b/doc/user/admin_area/review_abuse_reports.md
index af2f6640f8e..b8531fded18 100644
--- a/doc/user/admin_area/review_abuse_reports.md
+++ b/doc/user/admin_area/review_abuse_reports.md
@@ -89,6 +89,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md
index cd1f20a2f4e..44a6bbb9d8e 100644
--- a/doc/user/admin_area/settings/account_and_limit_settings.md
+++ b/doc/user/admin_area/settings/account_and_limit_settings.md
@@ -259,18 +259,6 @@ Once a lifetime for access tokens is set, GitLab:
allowed lifetime. Three hours is given to allow administrators to change the allowed lifetime,
or remove it, before revocation takes place.
-<!-- start_remove The following content will be removed on remove_date: '2022-08-22' -->
-## Allow expired access tokens to be used (removed) **(ULTIMATE SELF)**
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214723) in GitLab 13.1.
-> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/296881) in GitLab 13.9.
-> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351962) in GitLab 14.8.
-> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351962) in GitLab 15.0.
-
-This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351962) in GitLab 14.8.
-This feature was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351962) in GitLab 15.0.
-<!-- end_remove -->
-
## Disable user profile name changes **(PREMIUM SELF)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24605) in GitLab 12.7.
@@ -288,11 +276,11 @@ When this ability is disabled, GitLab administrators can still use the
[Admin Area](../index.md#administering-users) or the
[API](../../../api/users.md#user-modification) to update usernames.
-## Prevent users from creating top-level groups
+## Prevent new users from creating top-level groups
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367754) in GitLab 15.5.
-By default, new users can create top-level groups. GitLab administrators can prevent users from creating top-level groups:
+By default, new users can create top-level groups. GitLab administrators can prevent new users from creating top-level groups:
- In GitLab 15.5 and later, using either:
- The GitLab UI using the steps in this section.
@@ -301,7 +289,7 @@ By default, new users can create top-level groups. GitLab administrators can pre
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > General**, then expand **Account and limit**.
-1. Clear the **Allow users to create top-level groups** checkbox.
+1. Clear the **Allow new users to create top-level groups** checkbox.
## Troubleshooting
diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md
index afc9a006b39..adca9c85af1 100644
--- a/doc/user/admin_area/settings/continuous_integration.md
+++ b/doc/user/admin_area/settings/continuous_integration.md
@@ -20,10 +20,10 @@ for all projects:
1. Check (or uncheck to disable) the box that says **Default to Auto DevOps pipeline for all projects**.
1. Optionally, set up the [Auto DevOps base domain](../../../topics/autodevops/requirements.md#auto-devops-base-domain)
which is used for Auto Deploy and Auto Review Apps.
-1. Hit **Save changes** for the changes to take effect.
+1. Select **Save changes** for the changes to take effect.
From now on, every existing project and newly created ones that don't have a
-`.gitlab-ci.yml`, uses the Auto DevOps pipelines.
+`.gitlab-ci.yml` use the Auto DevOps pipelines.
If you want to disable it for a specific project, you can do so in
[its settings](../../../topics/autodevops/index.md#enable-or-disable-auto-devops).
@@ -71,7 +71,7 @@ runner settings:
To view the rendered details:
1. On the top bar, select **Main menu**, and:
- - For a project, select ***Projects** and find your project.
+ - For a project, select **Projects** and find your project.
- For a group, select **Groups** and find your group.
1. On the left sidebar, select **Settings > CI/CD**.
1. Expand **Runners**.
@@ -132,7 +132,7 @@ NOTE:
Any changes to this setting applies to new artifacts only. The expiration time is not
be updated for artifacts created before this setting was changed.
The administrator may need to manually search for and expire previously-created
-artifacts, as described in the [troubleshooting documentation](../../../administration/troubleshooting/gitlab_rails_cheat_sheet.md#remove-artifacts-more-than-a-week-old).
+artifacts, as described in the [troubleshooting documentation](../../../administration/job_artifacts.md#delete-job-artifacts-from-jobs-completed-before-a-specific-date).
## Keep the latest artifacts for all jobs in the latest successful pipelines
@@ -174,7 +174,7 @@ To set the duration for which the jobs are considered as old and expired:
1. On the left sidebar, select **Settings > CI/CD**.
1. Expand the **Continuous Integration and Deployment** section.
1. Set the value of **Archive jobs**.
-1. Hit **Save changes** for the changes to take effect.
+1. Select **Save changes** for the changes to take effect.
After that time passes, the jobs are archived in the background and no longer able to be
retried. Make it empty to never expire jobs. It has to be no less than 1 day,
@@ -201,7 +201,7 @@ of your GitLab instance (`.gitlab-ci.yml` if not set):
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > CI/CD**.
1. Input the new file and path in the **Default CI/CD configuration file** field.
-1. Hit **Save changes** for the changes to take effect.
+1. Select **Save changes** for the changes to take effect.
It is also possible to specify a [custom CI/CD configuration file for a specific project](../../../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file).
@@ -244,7 +244,7 @@ To enable or disable the banner:
> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/352316) from GitLab Premium to GitLab Ultimate in 15.0.
NOTE:
-An alternative [compliance solution](../../group/manage.md#configure-a-compliance-pipeline)
+An alternative [compliance solution](../../group/compliance_frameworks.md#configure-a-compliance-pipeline)
is available. We recommend this alternative solution because it provides greater flexibility,
allowing required pipelines to be assigned to specific compliance framework labels.
@@ -272,7 +272,7 @@ To select a CI/CD template for the required pipeline configuration:
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > CI/CD**.
1. Expand the **Required pipeline configuration** section.
-1. Select a CI/CD template from the dropdown.
+1. Select a CI/CD template from the dropdown list.
1. Select **Save changes**.
## Package Registry configuration
diff --git a/doc/user/admin_area/settings/deprecated_api_rate_limits.md b/doc/user/admin_area/settings/deprecated_api_rate_limits.md
index 279cac95fc9..8bf0ffd21a5 100644
--- a/doc/user/admin_area/settings/deprecated_api_rate_limits.md
+++ b/doc/user/admin_area/settings/deprecated_api_rate_limits.md
@@ -40,10 +40,10 @@ To override the general user and IP rate limits for requests to deprecated API e
1. Select the checkboxes for the types of rate limits you want to enable:
- **Unauthenticated API request rate limit**
- **Authenticated API request rate limit**
-1. _If you enabled unauthenticated API request rate limits:_
+1. If you selected **unauthenticated**:
1. Select the **Maximum unauthenticated API requests per period per IP**.
1. Select the **Unauthenticated API rate limit period in seconds**.
-1. _If you enabled authenticated API request rate limits:_
+1. If you selected **authenticated**:
1. Select the **Maximum authenticated API requests per period per user**.
1. Select the **Authenticated API rate limit period in seconds**.
diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md
index 6a7c01ff98b..6d2a3c2cdae 100644
--- a/doc/user/admin_area/settings/email.md
+++ b/doc/user/admin_area/settings/email.md
@@ -92,6 +92,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/external_authorization.md b/doc/user/admin_area/settings/external_authorization.md
index 62d3d713616..a34ceac0d95 100644
--- a/doc/user/admin_area/settings/external_authorization.md
+++ b/doc/user/admin_area/settings/external_authorization.md
@@ -115,6 +115,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/files_api_rate_limits.md b/doc/user/admin_area/settings/files_api_rate_limits.md
index e5f5064304c..ef9a3674c49 100644
--- a/doc/user/admin_area/settings/files_api_rate_limits.md
+++ b/doc/user/admin_area/settings/files_api_rate_limits.md
@@ -36,10 +36,10 @@ To override the general user and IP rate limits for requests to the Repository f
1. Select the checkboxes for the types of rate limits you want to enable:
- **Unauthenticated API request rate limit**
- **Authenticated API request rate limit**
-1. _If you enabled unauthenticated API request rate limits:_
+1. If you selected **unauthenticated**:
1. Select the **Max unauthenticated API requests per period per IP**.
1. Select the **Unauthenticated API rate limit period in seconds**.
-1. _If you enabled authenticated API request rate limits:_
+1. If you selected **authenticated**:
1. Select the **Max authenticated API requests per period per user**.
1. Select the **Authenticated API rate limit period in seconds**.
diff --git a/doc/user/admin_area/settings/floc.md b/doc/user/admin_area/settings/floc.md
index 08f3e8c09b2..f8137afa40f 100644
--- a/doc/user/admin_area/settings/floc.md
+++ b/doc/user/admin_area/settings/floc.md
@@ -36,6 +36,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/help_page.md b/doc/user/admin_area/settings/help_page.md
index a4072f8680b..8d0fef398af 100644
--- a/doc/user/admin_area/settings/help_page.md
+++ b/doc/user/admin_area/settings/help_page.md
@@ -56,7 +56,7 @@ GitLab marketing-related entries are occasionally shown on the Help page. To hid
You can specify a custom URL to which users are directed when they:
-- Select **Support** from the Help dropdown.
+- Select **Support** from the Help dropdown list.
- Select **See our website for help** on the Help page.
1. On the top bar, select **Main menu > Admin**.
@@ -106,6 +106,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md
index 1bdacf486a2..08c4a0d0167 100644
--- a/doc/user/admin_area/settings/index.md
+++ b/doc/user/admin_area/settings/index.md
@@ -109,8 +109,8 @@ The **Metrics and profiling** settings contain:
Enable and configure Grafana.
- [Profiling - Performance bar](../../../administration/monitoring/performance/performance_bar.md#enable-the-performance-bar-for-non-administrators) -
Enable access to the Performance Bar for non-administrator users in a given group.
-- [Self monitoring](../../../administration/monitoring/gitlab_self_monitoring_project/index.md#create-the-self-monitoring-project) -
- Enable or disable instance self monitoring.
+- [Self-monitoring](../../../administration/monitoring/gitlab_self_monitoring_project/index.md#create-the-self-monitoring-project) -
+ Enable or disable instance self-monitoring.
- [Usage statistics](usage_statistics.md) - Enable or disable version check and Service Ping.
### Network
diff --git a/doc/user/admin_area/settings/instance_template_repository.md b/doc/user/admin_area/settings/instance_template_repository.md
index b74f4f68230..bf07c5b2808 100644
--- a/doc/user/admin_area/settings/instance_template_repository.md
+++ b/doc/user/admin_area/settings/instance_template_repository.md
@@ -28,7 +28,7 @@ To select a project to serve as the custom template repository:
1. Add custom templates to the selected repository.
After you add templates, you can use them for the entire instance.
-They are available in the [Web Editor's dropdown](../../project/repository/web_editor.md#template-dropdowns)
+They are available in the [Web Editor's dropdown list](../../project/repository/web_editor.md#template-dropdowns)
and through the [API settings](../../../api/settings.md).
## Supported file types and locations
@@ -67,9 +67,9 @@ extension and not be empty. So, the hierarchy should look like this:
|-- another_metrics-dashboard.yml
```
-Your custom templates are displayed on the dropdown menu when a new file is added through the GitLab UI:
+Your custom templates are displayed on the dropdown list when a new file is added through the GitLab UI:
-![Custom template dropdown menu](img/file_template_user_dropdown.png)
+![Custom template dropdown list](img/file_template_user_dropdown.png)
If this feature is disabled or no templates are present,
no **Custom** section displays in the selection dropdown.
@@ -82,6 +82,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/package_registry_rate_limits.md b/doc/user/admin_area/settings/package_registry_rate_limits.md
index 7e0c3482e3e..a5c5536f22d 100644
--- a/doc/user/admin_area/settings/package_registry_rate_limits.md
+++ b/doc/user/admin_area/settings/package_registry_rate_limits.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
type: reference
---
diff --git a/doc/user/admin_area/settings/sign_in_restrictions.md b/doc/user/admin_area/settings/sign_in_restrictions.md
index 320768e6e5a..4ea420d7ca6 100644
--- a/doc/user/admin_area/settings/sign_in_restrictions.md
+++ b/doc/user/admin_area/settings/sign_in_restrictions.md
@@ -32,27 +32,70 @@ In the event of an external authentication provider outage, use the [GitLab Rail
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2158) in GitLab 13.10.
-When this feature is enabled, instance administrators are limited as regular users. During that period,
-they do not have access to all projects, groups, or the **Admin Area** menu.
+If you are an administrator, you might want to work in GitLab without the access that
+comes from being an administrator. While you could create a separate user account that
+doesn't have administrator access, a more secure solution is to use *Admin Mode*.
-To access potentially dangerous resources, an administrator can activate Admin Mode by:
+With Admin Mode, your account does not have administrative access by default.
+You can continue to access groups and projects you are a member of, but to access
+administrative functionality, you must authenticate.
-- Selecting the *Enable Admin Mode* button
-- Trying to access any part of the UI that requires administrator access, specifically those which call `/admin` endpoints.
+When Admin Mode is enabled, it applies to all administrators on the instance.
-The main use case allows administrators to perform their regular tasks as a regular
-user, based on their memberships, without having to set up a second account for
-security reasons.
+When Admin Mode is enabled for an instance, administrators:
-When Admin Mode status is disabled, administrative users cannot access resources unless
-they've been explicitly granted access. For example, when Admin Mode is disabled, they
-get a `404` error if they try to open a private group or project, unless
-they are members of that group or project.
+- Are allowed to access group and projects for which they are members.
+- Cannot access the **Admin Area**.
-2FA should be enabled for administrators and is supported for the Admin Mode flow, as are
-OmniAuth providers and LDAP auth. The Admin Mode status is stored in the active user
-session and remains active until it is explicitly disabled (it will be disabled
-automatically after a timeout otherwise).
+### Enable Admin Mode for your instance
+
+Administrators can enable Admin Mode though the API, the Rails console, or the UI.
+
+#### Use the API to enable Admin Mode
+
+Make the following request to your instance endpoint:
+
+```shell
+curl --request PUT --header "PRIVATE-TOKEN:$ADMIN_TOKEN" "<gitlab.example.com>/api/v4/application/settings?admin_mode=true"
+```
+
+Replace `<gitlab.example.com>` with your instance URL.
+
+For more information, see the [list of settings that can be accessed through API calls](../../../api/settings.md).
+
+#### Use the Rails console to enable Admin Mode
+
+Open the [Rails console](../../../administration/operations/rails_console.md) and run the following:
+
+```ruby
+::Gitlab::CurrentSettings.update_attributes!(admin_mode: true)
+```
+
+#### Use the UI to enable Admin Mode
+
+To enable Admin Mode through the UI:
+
+1. On the top bar, select **Main menu > Admin**.
+1. On the left sidebar, select **Settings > General**.
+1. Expand **Sign-in restrictions**.
+1. In the **Admin Mode** section, select the **Require additional authentication for administrative tasks** checkbox.
+
+### Turn on Admin Mode for your session
+
+To turn on Admin Mode for your current session and access potentially dangerous resources:
+
+1. On the top bar, select **Enable Admin Mode**.
+1. Try to access any part of the UI with `/admin` in the URL (which requires administrator access).
+
+When Admin Mode status is disabled or turned off, administrators cannot access resources unless
+they've been explicitly granted access. For example, administrators get a `404` error
+if they try to open a private group or project, unless they are members of that group or project.
+
+2FA should be enabled for administrators. 2FA, OmniAuth providers, and LDAP
+authentication are supported by Admin Mode. Admin Mode status is stored in the current user session and remains active until either:
+
+- It is explicitly disabled.
+- It is disabled automatically after a timeout.
### Limitations of Admin Mode
diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md
index 28fb188731b..76415596dce 100644
--- a/doc/user/admin_area/settings/sign_up_restrictions.md
+++ b/doc/user/admin_area/settings/sign_up_restrictions.md
@@ -115,7 +115,7 @@ create or update pipelines until their email address is confirmed.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20661) in GitLab 12.6
-You can [change](../../../security/password_length_limits.md#modify-minimum-password-length-using-gitlab-ui)
+You can [change](../../../security/password_length_limits.md#modify-minimum-password-length)
the minimum number of characters a user must have in their password using the GitLab UI.
### Password complexity requirements **(PREMIUM SELF)**
@@ -198,6 +198,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/terms.md b/doc/user/admin_area/settings/terms.md
index 28fe352c684..9a02e50b23f 100644
--- a/doc/user/admin_area/settings/terms.md
+++ b/doc/user/admin_area/settings/terms.md
@@ -43,6 +43,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/third_party_offers.md b/doc/user/admin_area/settings/third_party_offers.md
index fbd282ed5ad..8d2ae72ba69 100644
--- a/doc/user/admin_area/settings/third_party_offers.md
+++ b/doc/user/admin_area/settings/third_party_offers.md
@@ -31,6 +31,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md
index fb027b462df..df60268a8bf 100644
--- a/doc/user/admin_area/settings/usage_statistics.md
+++ b/doc/user/admin_area/settings/usage_statistics.md
@@ -48,7 +48,7 @@ tier. Users can continue to access the features in a paid tier without sharing u
### Features available in 14.4 and later
- [Repository size limit](../settings/account_and_limit_settings.md#repository-size-limit).
-- [Group access restriction by IP address](../../group/access_and_permissions.md#restrict-group-access-by-ip-address).
+- [Group access restriction by IP address](../../group/access_and_permissions.md#restrict-access-to-groups-by-ip-address).
NOTE:
Registration is not yet required for participation, but may be added in a future milestone.
@@ -202,6 +202,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/user_and_ip_rate_limits.md b/doc/user/admin_area/settings/user_and_ip_rate_limits.md
index 7431fc329d1..e285275f5bb 100644
--- a/doc/user/admin_area/settings/user_and_ip_rate_limits.md
+++ b/doc/user/admin_area/settings/user_and_ip_rate_limits.md
@@ -225,6 +225,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md
index d3132ae03c6..6d878bcb01c 100644
--- a/doc/user/admin_area/settings/visibility_and_access_controls.md
+++ b/doc/user/admin_area/settings/visibility_and_access_controls.md
@@ -45,7 +45,7 @@ By default both administrators and anyone with the **Owner** role can delete a p
1. Expand the **Visibility and access controls** section.
1. Scroll to:
- (GitLab 15.1 and later) **Allowed to delete projects**, and select **Administrators**.
- - (GitLab 15.0 and earlier) **Default project deletion projection** and select **Only admins can delete project**.
+ - (GitLab 15.0 and earlier) **Default project deletion protection** and select **Only admins can delete project**.
1. Select **Save changes**.
## Deletion protection **(PREMIUM SELF)**
@@ -82,8 +82,8 @@ To configure delayed project deletion:
1. On the left sidebar, select **Settings > General**.
1. Expand the **Visibility and access controls** section.
1. Scroll to:
- - (GitLab 15.1 and later) **Deletion projection** and select keep deleted groups and projects, and select a retention period.
- - (GitLab 15.0 and earlier) **Default delayed project projection** and select **Enable delayed project deletion by
+ - (GitLab 15.1 and later) **Deletion protection** and select keep deleted groups and projects, and select a retention period.
+ - (GitLab 15.0 and earlier) **Default delayed project protection** and select **Enable delayed project deletion by
default for newly-created groups.** Then set a retention period in **Default deletion delay**.
1. Select **Save changes**.
@@ -262,7 +262,7 @@ These options specify the permitted types and lengths for SSH keys.
To specify a restriction for each key type:
-1. Select the desired option from the dropdown.
+1. Select the desired option from the dropdown list.
1. Select **Save changes**.
For more details, see [SSH key restrictions](../../../security/ssh_keys_restrictions.md).
@@ -280,13 +280,13 @@ work in every repository. They can only be re-enabled by an administrator user o
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87579) in GitLab 15.1 [with a flag](../../../administration/feature_flags.md) named `group_ip_restrictions_allow_global`. Disabled by default.
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/366445) in GitLab 15.4. [Feature flag `group_ip_restrictions_allow_global`](https://gitlab.com/gitlab-org/gitlab/-/issues/366445) removed.
-This setting allows you to set IP address ranges to be combined with group-level IP allowlists.
-It helps administrators prevent aspects of the GitLab installation from being blocked
-from working as intended when an IP allowlist is used.
+Administrators can set IP address ranges to be combined with [group-level IP restrictions](../../group/access_and_permissions.md#restrict-access-to-groups-by-ip-address).
+Use globally-allowed IP addresses to allow aspects of the GitLab installation to work even when group-level IP address
+restrictions are set.
-For example, if the GitLab Pages daemon runs on the `10.0.0.0/24` range, specify that range in this
-field, as otherwise any group-level restrictions that don't include that range cause the Pages
-daemon to be unable to fetch artifacts from the pipeline runs.
+For example, if the GitLab Pages daemon runs on the `10.0.0.0/24` range, you can specify that range as globally-allowed.
+This means GitLab Pages can still fetch artifacts from pipelines even if group-level IP address restrictions don't
+include the `10.0.0.0/24` range.
To add a IP address range to the group-level allowlist:
@@ -294,7 +294,11 @@ To add a IP address range to the group-level allowlist:
1. On the top bar, select **Main menu > Admin**.
1. On the left sidebar, select **Settings > General**.
1. Expand the **Visibility and access controls** section.
-1. In **Globally-allowed IP ranges**, provide a value.
+1. In **Globally-allowed IP ranges**, provide a list of IP address ranges. This list:
+ - Has no limit on the number of IP address ranges.
+ - Has a size limit of 1 GB.
+ - Applies to both SSH or HTTP authorized IP address ranges. You cannot split
+ this list by type of authorization.
1. Select **Save changes**.
<!-- ## Troubleshooting
@@ -305,6 +309,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/analytics/ci_cd_analytics.md b/doc/user/analytics/ci_cd_analytics.md
index de9275b8599..ef2d3a76990 100644
--- a/doc/user/analytics/ci_cd_analytics.md
+++ b/doc/user/analytics/ci_cd_analytics.md
@@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# CI/CD analytics **(FREE)**
-Use the CI/CD analytics page to view pipeline success rates and duration, and the history of [DORA metrics](index.md#devops-research-and-assessment-dora-key-metrics) over time.
+Use the CI/CD analytics page to view pipeline success rates and duration, and the history of DORA metrics over time.
## Pipeline success and duration charts
@@ -35,7 +35,7 @@ To view CI/CD analytics:
1. On the top bar, select **Main menu > Projects** and find your project.
1. On the left sidebar, select **Analytics > CI/CD Analytics**.
-## View deployment frequency chart **(ULTIMATE)**
+## View DORA deployment frequency chart **(ULTIMATE)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275991) in GitLab 13.8.
@@ -44,7 +44,7 @@ frequency to the `production` environment. The environment must be part of the
[production deployment tier](../../ci/environments/index.md#deployment-tier-of-environments)
for its deployment information to appear on the graphs.
- Deployment frequency is one of the four [DORA metrics](index.md#devops-research-and-assessment-dora-key-metrics) that DevOps teams use for measuring excellence in software delivery.
+ Deployment frequency is one of the four DORA metrics that DevOps teams use for measuring excellence in software delivery.
The deployment frequency chart is available for groups and projects.
@@ -56,7 +56,7 @@ To view the deployment frequency chart:
![Deployment frequency](img/deployment_frequency_charts_v13_12.png)
-## View lead time for changes chart **(ULTIMATE)**
+## View DORA lead time for changes chart **(ULTIMATE)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250329) in GitLab 13.11.
@@ -68,7 +68,7 @@ merge requests to be deployed to a production environment. This chart is availab
- For time periods in which no merge requests were deployed, the charts render a
red, dashed line.
- Lead time for changes is one of the four [DORA metrics](index.md#devops-research-and-assessment-dora-key-metrics) that DevOps teams use for measuring excellence in software delivery.
+ Lead time for changes is one of the four DORA metrics that DevOps teams use for measuring excellence in software delivery.
To view the lead time for changes chart:
@@ -78,13 +78,13 @@ To view the lead time for changes chart:
![Lead time](img/lead_time_chart_v13_11.png)
-## View time to restore service chart **(ULTIMATE)**
+## View DORA time to restore service chart **(ULTIMATE)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356959) in GitLab 15.1
The time to restore service chart shows information about the median time an incident was open in a production environment. This chart is available for groups and projects.
-Time to restore service is one of the four [DORA metrics](index.md#devops-research-and-assessment-dora-key-metrics) that DevOps teams use for measuring excellence in software delivery.
+Time to restore service is one of the four DORA metrics that DevOps teams use for measuring excellence in software delivery.
To view the time to restore service chart:
@@ -94,13 +94,13 @@ To view the time to restore service chart:
![Lead time](img/time_to_restore_service_charts_v15_1.png)
-## View change failure rate chart **(ULTIMATE)**
+## View DORA change failure rate chart **(ULTIMATE)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357072) in GitLab 15.2
The change failure rate chart shows information about the percentage of deployments that cause an incident in a production environment. This chart is available for groups and projects.
-Change failure rate is one of the four [DORA metrics](index.md#devops-research-and-assessment-dora-key-metrics) that DevOps teams use for measuring excellence in software delivery.
+Change failure rate is one of the four DORA metrics that DevOps teams use for measuring excellence in software delivery.
To view the change failure rate chart:
diff --git a/doc/user/analytics/dora_metrics.md b/doc/user/analytics/dora_metrics.md
new file mode 100644
index 00000000000..b5f37203817
--- /dev/null
+++ b/doc/user/analytics/dora_metrics.md
@@ -0,0 +1,127 @@
+---
+stage: Plan
+group: Optimize
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# DevOps Research and Assessment (DORA) metrics **(ULTIMATE)**
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275991) in GitLab 13.7.
+> - [Added support](https://gitlab.com/gitlab-org/gitlab/-/issues/291746) for lead time for changes in GitLab 13.10.
+
+The [DevOps Research and Assessment (DORA)](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance)
+team has identified four metrics that measure DevOps performance.
+Using these metrics helps improve DevOps efficiency and communicate performance to business stakeholders, which can accelerate business results.
+
+DORA includes four key metrics, divided into two core areas of DevOps:
+
+- [Deployment Frequency](#deployment-frequency) and [Lead Time for Change](#lead-time-for-changes) measure team velocity.
+- [Change Failure Rate](#change-failure-rate) and [Time to Restore Service](#time-to-restore-service) measure stability.
+
+For software leaders, tracking velocity alongside quality metrics ensures they're not sacrificing quality for speed.
+
+<div class="video-fallback">
+ For an overview, see <a href="https://www.youtube.com/watch?v=1BrcMV6rCDw">GitLab Speed Run: DORA metrics in GitLab One DevOps Platform</a>.
+</div>
+<figure class="video-container">
+ <iframe src="https://www.youtube.com/embed/1BrcMV6rCDw" frameborder="0" allowfullscreen="true"> </iframe>
+</figure>
+
+## DORA Metrics dashboard in Value Stream Analytics
+
+The four DORA metrics are available out-of-the-box in the [Value Stream Analytics (VSA) overview dashboard](../group/value_stream_analytics/index.md#view-dora-metrics-and-key-metrics-for-a-group).
+This helps you visualize the engineering work in the context of end-to-end value delivery.
+
+The One DevOps Platform [Value Stream Management](https://gitlab.com/gitlab-org/gitlab/-/value_stream_analytics) provides end-to-end visibility to the entire software delivery lifecycle.
+This enables teams and managers to understand all aspects of productivity, quality, and delivery, without the ["toolchain tax"](https://about.gitlab.com/solutions/value-stream-management/).
+
+## Deployment frequency
+
+Deployment frequency is the frequency of successful deployments to production (hourly, daily, weekly, monthly, or yearly).
+This measures how often you deliver value to end users. A higher deployment frequency means you can
+get feedback sooner and iterate faster to deliver improvements and features. GitLab measures this as the number of
+deployments to a production environment in the given time period.
+
+Deployment frequency displays in several charts:
+
+- [Group-level value stream analytics](../group/value_stream_analytics/index.md)
+- [Project-level value stream analytics](value_stream_analytics.md)
+- [CI/CD analytics](ci_cd_analytics.md)
+
+To retrieve metrics for deployment frequency, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs.
+
+## Lead time for changes
+
+DORA Lead time for changes measures the time to successfully deliver a commit into production.
+This metric reflects the efficiency of CI/CD pipelines.
+
+In GitLab, Lead time for changes calculates the median time it takes for a merge request to get merged into production.
+We measure **from** code committed **to** code successfully running in production, without adding the `coding_time` to the calculation.
+
+Over time, the lead time for changes should decrease, while your team's performance should increase.
+
+Lead time for changes displays in several charts:
+
+- [Group-level value stream analytics](../group/value_stream_analytics/index.md)
+- [Project-level value stream analytics](value_stream_analytics.md)
+- [CI/CD analytics](ci_cd_analytics.md)
+
+To retrieve metrics for lead time for changes, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs.
+
+- The definition of lead time for change can vary widely, which often creates confusion within the industry.
+- "Lead time for changes" is not the same as "Lead time". In the value stream, "Lead time" measures the time it takes for work on an issue to move from the moment it's requested (Issue created) to the moment it's fulfilled and delivered (Issue closed).
+
+## Time to restore service
+
+Time to restore service measures how long it takes an organization to recover from a failure in production.
+GitLab measures this as the average time required to close the incidents
+in the given time period. This assumes:
+
+- All incidents are related to a production environment.
+- Incidents and deployments have a strictly one-to-one relationship. An incident is related to only
+one production deployment, and any production deployment is related to no more than one incident).
+
+Time to restore service displays in several charts:
+
+- [Group-level value stream analytics](../group/value_stream_analytics/index.md)
+- [Project-level value stream analytics](value_stream_analytics.md)
+- [CI/CD analytics](ci_cd_analytics.md)
+
+To retrieve metrics for time to restore service, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs.
+
+## Change failure rate
+
+Change failure rate measures the percentage of deployments that cause a failure in production. GitLab measures this as the number
+of incidents divided by the number of deployments to a
+production environment in the given time period. This assumes:
+
+- All incidents are related to a production environment.
+- Incidents and deployments have a strictly one-to-one relationship. An incident is related to only
+one production deployment, and any production deployment is related to no
+more than one incident.
+
+To retrieve metrics for change failure rate, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs.
+
+### Insights: Custom DORA reporting
+
+Custom charts to visualize DORA data with Insights YAML-based reports.
+
+With this new visualization, software leaders can track metrics improvements, understand patterns in their metrics trends, and compare performance between groups and projects.
+
+### Measure DORA metrics without using GitLab CI/CD pipelines
+
+Deployment frequency is calculated based on the deployments record, which is created for typical push-based deployments.
+These deployment records are not created for pull-based deployments, for example when Container Images are connected to GitLab with an agent.
+
+To track DORA metrics in these cases, you can [create a deployment record](../../api/deployments.md#create-a-deployment) using the Deployments API.
+
+### Supported DORA metrics in GitLab
+
+| Metric | Level | API | UI chart | Comments |
+|---------------------------|-------------------|-----------------------------------------------------|------------------------|----------|
+| `deployment_frequency` | Project | [GitLab 13.7 and later](../../api/dora/metrics.md) | GitLab 14.8 and later | The previous API endpoint was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/323713) in 13.10. |
+| `deployment_frequency` | Group | [GitLab 13.10 and later](../../api/dora/metrics.md) | GitLab 13.12 and later | |
+| `lead_time_for_changes` | Project | [GitLab 13.10 and later](../../api/dora/metrics.md) | GitLab 13.11 and later | Unit in seconds. Aggregation method is median. |
+| `lead_time_for_changes` | Group | [GitLab 13.10 and later](../../api/dora/metrics.md) | GitLab 14.0 and later | Unit in seconds. Aggregation method is median. |
+| `time_to_restore_service` | Project and group | [GitLab 14.9 and later](../../api/dora/metrics.md) | GitLab 15.1 and later | Unit in days. Aggregation method is median. |
+| `change_failure_rate` | Project and group | [GitLab 14.10 and later](../../api/dora/metrics.md) | GitLab 15.2 and later | Percentage of deployments. |
diff --git a/doc/user/analytics/index.md b/doc/user/analytics/index.md
index 56adbff9f59..01fe466755c 100644
--- a/doc/user/analytics/index.md
+++ b/doc/user/analytics/index.md
@@ -34,7 +34,7 @@ GitLab provides several analytics features at the group level. Some of these fea
You can use GitLab to review analytics at the project level. Some of these features require you to use a higher tier than GitLab Free.
- [Application Security](../application_security/security_dashboard/index.md)
-- [CI/CD](ci_cd_analytics.md)
+- [CI/CD & DORA](ci_cd_analytics.md)
- [Code Review](code_review_analytics.md)
- [Insights](../project/insights/index.md)
- [Issue](../group/issues_analytics/index.md)
@@ -51,93 +51,6 @@ The following analytics features are available for users to create personalized
Be sure to review the documentation page for this feature for GitLab tier requirements.
-## DevOps Research and Assessment (DORA) key metrics **(ULTIMATE)**
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275991) in GitLab 13.7.
-> - [Added support](https://gitlab.com/gitlab-org/gitlab/-/issues/291746) for lead time for changes in GitLab 13.10.
-
-The [DevOps Research and Assessment (DORA)](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance)
-team developed several key metrics that you can use as performance indicators for software development
-teams.
-
-### Deployment frequency
-
-Deployment frequency is the frequency of successful deployments to production (hourly, daily, weekly, monthly, or yearly).
-This measures how often you deliver value to end users. A higher deployment frequency means you can
-get feedback sooner and iterate faster to deliver improvements and features. GitLab measures this as the number of
-deployments to a production environment in the given time period.
-
-Deployment frequency displays in several charts:
-
-- [Group-level value stream analytics](../group/value_stream_analytics/index.md)
-- [Project-level value stream analytics](value_stream_analytics.md)
-- [CI/CD analytics](ci_cd_analytics.md)
-
-To retrieve metrics for deployment frequency, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs.
-
-### Lead time for changes
-
-DORA Lead time for changes measures the time to successfully deliver a feature into production.
-This metric reflects the efficiency of CI/CD pipelines.
-
-In GitLab, Lead time for changes calculates the median time it takes for a merge request to get merged into production.
-We measure "FROM code committed TO code successfully running in production" without adding the "coding_time" to the calculation.
-
-Over time, the lead time for changes should decrease, while your team's performance should increase.
-
-Lead time for changes displays in several charts:
-
-- [Group-level value stream analytics](../group/value_stream_analytics/index.md)
-- [Project-level value stream analytics](value_stream_analytics.md)
-- [CI/CD analytics](ci_cd_analytics.md)
-
-To retrieve metrics for lead time for changes, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs.
-
-- The definition of lead time for change can vary widely, which often creates confusion within the industry.
-- "Lead time for changes" is not the same as "Lead time". In the value stream, "Lead time" measures the time it takes for work on issue to move from the moment it's requested (Issue created) to the time it's fulfilled and delivered (Issue closed).
-
-### Time to restore service
-
-Time to restore service measures how long it takes an organization to recover from a failure in production.
-GitLab measures this as the average time required to close the incidents
-in the given time period. This assumes:
-
-- All incidents are related to a production environment.
-- Incidents and deployments have a strictly one-to-one relationship. An incident is related to only
-one production deployment, and any production deployment is related to no more than one incident).
-
-Time to restore service displays in several charts:
-
-- [Group-level value stream analytics](../group/value_stream_analytics/index.md)
-- [Project-level value stream analytics](value_stream_analytics.md)
-- [CI/CD analytics](ci_cd_analytics.md)
-
-To retrieve metrics for time to restore service, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs.
-
-### Change failure rate
-
-Change failure rate measures the percentage of deployments that cause a failure in production. GitLab measures this as the number
-of incidents divided by the number of deployments to a
-production environment in the given time period. This assumes:
-
-- All incidents are related to a production environment.
-- Incidents and deployments have a strictly one-to-one relationship. An incident is related to only
-one production deployment, and any production deployment is related to no
-more than one incident.
-
-To retrieve metrics for change failure rate, use the [GraphQL](../../api/graphql/reference/index.md) or the [REST](../../api/dora/metrics.md) APIs.
-
-### Supported DORA metrics in GitLab
-
-| Metric | Level | API | UI chart | Comments |
-|---------------------------|-------------------|-----------------------------------------------------|------------------------|----------|
-| `deployment_frequency` | Project | [GitLab 13.7 and later](../../api/dora/metrics.md) | GitLab 14.8 and later | The previous API endpoint was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/323713) in 13.10. |
-| `deployment_frequency` | Group | [GitLab 13.10 and later](../../api/dora/metrics.md) | GitLab 13.12 and later | |
-| `lead_time_for_changes` | Project | [GitLab 13.10 and later](../../api/dora/metrics.md) | GitLab 13.11 and later | Unit in seconds. Aggregation method is median. |
-| `lead_time_for_changes` | Group | [GitLab 13.10 and later](../../api/dora/metrics.md) | GitLab 14.0 and later | Unit in seconds. Aggregation method is median. |
-| `time_to_restore_service` | Project and group | [GitLab 14.9 and later](../../api/dora/metrics.md) | GitLab 15.1 and later | Unit in days. Aggregation method is median. |
-| `change_failure_rate` | Project and group | [GitLab 14.10 and later](../../api/dora/metrics.md) | GitLab 15.2 and later | Percentage of deployments. |
-
## Definitions
We use the following terms to describe GitLab analytics:
diff --git a/doc/user/application_security/api_fuzzing/create_har_files.md b/doc/user/application_security/api_fuzzing/create_har_files.md
index 445bcfb0444..e7eaf47121b 100644
--- a/doc/user/application_security/api_fuzzing/create_har_files.md
+++ b/doc/user/application_security/api_fuzzing/create_har_files.md
@@ -93,7 +93,7 @@ used in [Web API Fuzz Testing](index.md#http-archive-har).
1. Select **Export Data > Current Workspace**.
1. Select requests to include in the HAR file.
1. Select **Export**.
-1. In the **Select Export Type** dropdown select **HAR -- HTTP Archive Format**.
+1. In the **Select Export Type** dropdown list select **HAR -- HTTP Archive Format**.
1. Select **Done**.
1. Enter a location and filename for the HAR file.
@@ -109,7 +109,7 @@ responses in HAR format.
have an account, first create an account.
1. Browse pages that call an API. Fiddler automatically captures the requests.
1. Select one or more requests, then from the context menu, select **Export > Selected Sessions**.
-1. In the **Choose Format** dropdown select **HTTPArchive v1.2**.
+1. In the **Choose Format** dropdown list select **HTTPArchive v1.2**.
1. Enter a filename and select **Save**.
Fiddler shows a popup message confirming the export has succeeded.
diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md
index d542c2d4be5..03eed6fdbf8 100644
--- a/doc/user/application_security/api_fuzzing/index.md
+++ b/doc/user/application_security/api_fuzzing/index.md
@@ -1211,7 +1211,7 @@ Example usage for setting a `body-json` override:
}
```
-Note that each JSON property name in the object `body-json` is set to a [JSON Path](https://goessner.net/articles/JsonPath/)
+Each JSON property name in the object `body-json` is set to a [JSON Path](https://goessner.net/articles/JsonPath/)
expression. The JSON Path expression `$.credentials.access-token` identifies the node to be
overridden with the value `iddqd!42.$`. The override engine uses `body-json` when the request body
has only [JSON](https://www.json.org/json-en.html) content.
@@ -1250,7 +1250,7 @@ the second entry overrides an XML element:
}
```
-Note that each JSON property name in the object `body-xml` is set to an
+Each JSON property name in the object `body-xml` is set to an
[XPath v2](https://www.w3.org/TR/xpath20/)
expression. The XPath expression `/credentials/@isEnabled` identifies the attribute node to override
with the value `true`. The XPath expression `/credentials/access-token/text()` identifies the
@@ -1392,7 +1392,7 @@ It is also possible to write messages from your script to a log file that is col
Adding some basic logging to your overrides script is useful in case the script fails unexpectedly during normal running of the job. The log file is automatically included as an artifact of the job, allowing you to download it after the job has finished.
-Following our example, we provided `renew_token.py` in the environmental variable `FUZZAPI_OVERRIDES_CMD`. Please notice two things in the script:
+Following our example, we provided `renew_token.py` in the environmental variable `FUZZAPI_OVERRIDES_CMD`. Notice two things in the script:
- Log file is saved in the location indicated by the environment variable `CI_PROJECT_DIR`.
- Log filename should match `gl-*.log`.
@@ -1871,7 +1871,7 @@ variables:
##### Excluding two URLs and their child resources
-In order to exclude the URLs: `http://target/api/buy` and `http://target/api/sell`, and their child resources. To provide multiple URLs we use the `,` character as follows:
+To exclude the URLs: `http://target/api/buy` and `http://target/api/sell`, and their child resources. To provide multiple URLs we use the `,` character as follows:
```yaml
stages:
@@ -1888,7 +1888,11 @@ variables:
##### Excluding URL using regular expressions
-In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there.
+To exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more), we could use `https://target/api/v.*/user/create$`. In the previous regular expression:
+
+- `.` indicates any character.
+- `*` indicates zero or more times.
+- `$` indicates that the URL should end there.
```yaml
stages:
@@ -2211,9 +2215,235 @@ Setting `SECURE_ANALYZERS_PREFIX` changes the Docker image registry location for
For more information, see [Offline environments](../offline_deployments/index.md).
+## Performance tuning and testing speed
+
+Security tools that perform API fuzz testing, such as API Fuzzing, perform testing by sending requests to an instance of your running application. The requests are mutated by our fuzzing engine to trigger unexpected behavior that might exist in your application. The speed of an API fuzzing test depends on the following:
+
+- How many requests per second can be sent to your application by our tooling
+- How fast your application responds to requests
+- How many requests must be sent to test the application
+ - How many operations your API is comprised of
+ - How many fields are in each operation (think JSON bodies, headers, query string, cookies, etc.)
+
+If API Fuzzing testing job still takes longer than expected after following the advice in this performance guide, reach out to support for further assistance.
+
+### Diagnosing performance issues
+
+The first step to resolving performance issues is to understand what is contributing to the slower-than-expected testing time. Some common issues we see are:
+
+- API Fuzzing is running on a slow or single-CPU GitLab Runner (GitLab Shared Runners are single-CPU)
+- The application deployed to a slow/single-CPU instance and is not able to keep up with the testing load
+- The application contains a slow operation that impacts the overall test speed (> 1/2 second)
+- The application contains an operation that returns a large amount of data (> 500K+)
+- The application contains a large number of operations (> 40)
+
+#### The application contains a slow operation that impacts the overall test speed (> 1/2 second)
+
+The API Fuzzing job output contains helpful information about how fast we are testing, how fast each operation being tested responds, and summary information. Let's take a look at some sample output to see how it can be used in tracking down performance issues:
+
+```shell
+API Security: Loaded 10 operations from: assets/har-large-response/large_responses.har
+API Security:
+API Security: Testing operation [1/10]: 'GET http://target:7777/api/large_response_json'.
+API Security: - Parameters: (Headers: 4, Query: 0, Body: 0)
+API Security: - Request body size: 0 Bytes (0 bytes)
+API Security:
+API Security: Finished testing operation 'GET http://target:7777/api/large_response_json'.
+API Security: - Excluded Parameters: (Headers: 0, Query: 0, Body: 0)
+API Security: - Performed 767 requests
+API Security: - Average response body size: 130 MB
+API Security: - Average call time: 2 seconds and 82.69 milliseconds (2.082693 seconds)
+API Security: - Time to complete: 14 minutes, 8 seconds and 788.36 milliseconds (848.788358 seconds)
+```
+
+This job console output snippet starts by telling us how many operations were found (10), followed by notifications that testing has started on a specific operation and a summary of the operation has been completed. The summary is the most interesting part of this log output. In the summary, we can see that it took API Fuzzing 767 requests to fully test this operation and its related fields. We can also see that the average response time was 2 seconds and the time to complete was 14 minutes for this one operation.
+
+An average response time of 2 seconds is a good initial indicator that this specific operation takes a long time to test. Further, we can see that the response body size is quite large. The large body size is the culprit here, transferring that much data on each request is what takes the majority of that 2 seconds.
+
+For this issue, the team might decide to:
+
+- Use a multi-CPU runner. Using a multi-CPU runner allows API Fuzzing to parallelize the work being performed. This helps lower the test time, but getting the test down under 10 minutes might still be problematic without moving to a high CPU machine due to how long the operation takes to test.
+ - Trade off between how many CPUs and cost.
+- [Exclude this operation](#excluding-slow-operations) from the API Fuzzing test. While this is the simplest, it has the downside of a gap in security test coverage.
+- [Exclude the operation from feature branch API Fuzzing tests, but include it in the default branch test](#excluding-operations-in-feature-branches-but-not-default-branch).
+- [Split up the API Fuzzing testing into multiple jobs](#splitting-a-test-into-multiple-jobs).
+
+The likely solution is to use a combination of these solutions to reach an acceptable test time, assuming your team's requirements are in the 5-7 minute range.
+
+### Addressing performance issues
+
+The following sections document various options for addressing performance issues for API Fuzzing:
+
+- [Using a multi-CPU Runner](#using-a-multi-cpu-runner)
+- [Excluding slow operations](#excluding-slow-operations)
+- [Splitting a test into multiple jobs](#splitting-a-test-into-multiple-jobs)
+- [Excluding operations in feature branches, but not default branch](#excluding-operations-in-feature-branches-but-not-default-branch)
+
+#### Using a multi-CPU Runner
+
+One of the easiest performance boosts can be achieved using a multi-CPU runner with API Fuzzing. This table shows statistics collected during benchmarking of a Java Spring Boot REST API. In this benchmark, the target and API Fuzzing share a single runner instance.
+
+| CPU Count | Request per Second |
+|----------------------|--------------------|
+| 1 CPU (Shared Runner)| 75 |
+| 4 CPU | 255 |
+| 8 CPU | 400 |
+
+As we can see from this table, increasing the CPU count of the runner can have a large impact on testing speed/performance.
+
+To use a multi-CPU typically requires deploying a self-managed GitLab Runner onto a multi-CPU machine or cloud compute instance.
+
+When multiple types of GitLab Runners are available for use, the various instances are commonly set up with tags that can be used in the job definition to select a type of runner.
+
+Here is an example job definition for API Fuzzing that adds a `tags` section with the tag `multi-cpu`. The job automatically extends the job definition included through the API Fuzzing template.
+
+```yaml
+apifuzzer_fuzz:
+ tags:
+ - multi-cpu
+```
+
+To verify that API Fuzzing can detect multiple CPUs in the runner, download the `gl-api-security-scanner.log` file from a completed job's artifacts. Search the file for the string `Starting work item processor` and inspect the reported max DOP (degree of parallelism). The max DOP should be greater than or equal to the number of CPUs assigned to the runner. The value is never lower than 2, even on single CPU runners, unless forced through a configuration variable. If the value reported is less than the number of CPUs assigned to the runner, then something is wrong with the runner deployment. If unable to identify the problem, open a ticket with support to assist.
+
+Example log entry:
+
+`17:00:01.084 [INF] <Peach.Web.Core.Services.WebRunnerMachine> Starting work item processor with 2 max DOP`
+
+#### Excluding slow operations
+
+In the case of one or two slow operations, the team might decide to skip testing the operations. Excluding the operation is done using the `FUZZAPI_EXCLUDE_PATHS` configuration [variable as explained in this section.](#exclude-paths)
+
+In this example, we have an operation that returns a large amount of data. The operation is `GET http://target:7777/api/large_response_json`. To exclude it we provide the `FUZZAPI_EXCLUDE_PATHS` configuration variable with the path portion of our operation URL `/api/large_response_json`.
+
+To verify the operation is excluded, run the API Fuzzing job and review the job console output. It includes a list of included and excluded operations at the end of the test.
+
+```yaml
+apifuzzer_fuzz:
+ variables:
+ FUZZAPI_EXCLUDE_PATHS: /api/large_response_json
+```
+
+Excluding operations from testing could allow some vulnerabilities to go undetected.
+{: .alert .alert-warning}
+
+#### Splitting a test into multiple jobs
+
+Splitting a test into multiple jobs is supported by API Fuzzing through the use of [`FUZZAPI_EXCLUDE_PATHS`](#exclude-paths) and [`FUZZAPI_EXCLUDE_URLS`](#exclude-urls). When splitting a test up, a good pattern is to disable the `apifuzzer_fuzz` job and replace it with two jobs with identifying names. In this example we have two jobs, each job is testing a version of the API, so our names reflect that. However, this technique can be applied to any situation, not just with versions of an API.
+
+The rules we are using in the `apifuzzer_v1` and `apifuzzer_v2` jobs are copied from the [API Fuzzing template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml).
+
+```yaml
+# Disable the main apifuzzer_fuzz job
+apifuzzer_fuzz:
+ rules:
+ - if: $CI_COMMIT_BRANCH
+ when: never
+
+apifuzzer_v1:
+ extends: apifuzzer_fuzz
+ variables:
+ FUZZAPI_EXCLUDE_PATHS: /api/v1/**
+ rules:
+ rules:
+ - if: $API_FUZZING_DISABLED
+ when: never
+ - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH &&
+ $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
+ when: never
+ - if: $CI_COMMIT_BRANCH &&
+ $CI_GITLAB_FIPS_MODE == "true"
+ variables:
+ FUZZAPI_IMAGE_SUFFIX: "-fips"
+ - if: $CI_COMMIT_BRANCH
+
+apifuzzer_v2:
+ variables:
+ FUZZAPI_EXCLUDE_PATHS: /api/v2/**
+ rules:
+ rules:
+ - if: $API_FUZZING_DISABLED
+ when: never
+ - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH &&
+ $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
+ when: never
+ - if: $CI_COMMIT_BRANCH &&
+ $CI_GITLAB_FIPS_MODE == "true"
+ variables:
+ FUZZAPI_IMAGE_SUFFIX: "-fips"
+ - if: $CI_COMMIT_BRANCH
+```
+
+#### Excluding operations in feature branches, but not default branch
+
+In the case of one or two slow operations, the team might decide to skip testing the operations, or exclude them from feature branch tests, but include them for default branch tests. Excluding the operation is done using the `FUZZAPI_EXCLUDE_PATHS` configuration [variable as explained in this section.](#exclude-paths)
+
+In this example, we have an operation that returns a large amount of data. The operation is `GET http://target:7777/api/large_response_json`. To exclude it we provide the `FUZZAPI_EXCLUDE_PATHS` configuration variable with the path portion of our operation URL `/api/large_response_json`. Our configuration disables the main `apifuzzer_fuzz` job and creates two new jobs `apifuzzer_main` and `apifuzzer_branch`. The `apifuzzer_branch` is set up to exclude the long operation and only run on non-default branches (e.g. feature branches). The `apifuzzer_main` branch is set up to only execute on the default branch (`main` in this example). The `apifuzzer_branch` jobs run faster, allowing for quick development cycles, while the `apifuzzer_main` job which only runs on default branch builds, takes longer to run.
+
+To verify the operation is excluded, run the API Fuzzing job and review the job console output. It includes a list of included and excluded operations at the end of the test.
+
+```yaml
+# Disable the main job so we can create two jobs with
+# different names
+apifuzzer_fuzz:
+ rules:
+ - if: $CI_COMMIT_BRANCH
+ when: never
+
+# API Fuzzing for feature branch work, excludes /api/large_response_json
+apifuzzer_branch:
+ extends: apifuzzer_fuzz
+ variables:
+ FUZZAPI_EXCLUDE_PATHS: /api/large_response_json
+ rules:
+ rules:
+ - if: $API_FUZZING_DISABLED
+ when: never
+ - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH &&
+ $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
+ when: never
+ - if: $CI_COMMIT_BRANCH &&
+ $CI_GITLAB_FIPS_MODE == "true"
+ variables:
+ FUZZAPI_IMAGE_SUFFIX: "-fips"
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+ when: never
+ - if: $CI_COMMIT_BRANCH
+
+# API Fuzzing for default branch (main in our case)
+# Includes the long running operations
+apifuzzer_main:
+ extends: apifuzzer_fuzz
+ rules:
+ - if: $API_FUZZING_DISABLED
+ when: never
+ - if: $API_FUZZING_DISABLED_FOR_DEFAULT_BRANCH &&
+ $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
+ when: never
+ - if: $CI_COMMIT_BRANCH &&
+ $CI_GITLAB_FIPS_MODE == "true"
+ variables:
+ FUZZAPI_IMAGE_SUFFIX: "-fips"
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+```
+
## Troubleshooting
-### `Error waiting for API Security 'http://127.0.0.1:5000' to become available`
+### API Fuzzing job times out after N hours
+
+The top two reasons for the API Fuzzing job timing out are slow operations (> 1 second) and using a single-CPU runner for API Fuzzing (GitLab shared runners are single-CPU). Before you can diagnose the problem further, the job must complete so the output can be analyzed. We recommend to start with a multi-CPU runner first, then exclude portions of your API operations until the job completes and the output can be further reviewed.
+
+See the following documentation sections for assistance:
+
+- [Performance tuning and testing speed](#performance-tuning-and-testing-speed)
+- [Using a multi-CPU Runner](#using-a-multi-cpu-runner)
+- [Excluding operations by path](#exclude-paths)
+- [Excluding slow operations](#excluding-slow-operations)
+
+### API Fuzzing job takes too long to complete
+
+See [Performance Tuning and Testing Speed](#performance-tuning-and-testing-speed)
+
+### Error waiting for API Security 'http://127.0.0.1:5000' to become available
A bug exists in versions of the API Fuzzing analyzer prior to v1.6.196 that can cause a background process to fail under certain conditions. The solution is to update to a newer version of the API Fuzzing analyzer.
@@ -2281,16 +2511,16 @@ For OpenAPI Specifications that are generated automatically validation errors ar
1. Identify the validation errors.
1. Use the [Swagger Editor](https://editor.swagger.io/) to identify validation problems in your specification. The visual nature of the Swagger Editor makes it easier to understand what needs to change.
- 1. Alternatively, you can check the log output and look for schema validation warnings. They are prefixed with messages such as `OpenAPI 2.0 schema validation error` or `OpenAPI 3.0.x schema validation error`. Each failed validation provides extra information about `location` and `description`. Note that JSON Schema validation messages might not be easy to understand. This is why we recommend the use of editors to validate schema documents.
+ 1. Alternatively, you can check the log output and look for schema validation warnings. They are prefixed with messages such as `OpenAPI 2.0 schema validation error` or `OpenAPI 3.0.x schema validation error`. Each failed validation provides extra information about `location` and `description`. JSON Schema validation messages can be complex, and editors can help you validate schema documents.
1. Review the documentation for the OpenAPI generation your framework/tech stack is using. Identify the changes needed to produce a correct OpenAPI document.
-1. Once the validation issues are resolved, re-run your pipeline.
+1. After the validation issues are resolved, re-run your pipeline.
**For manually created OpenAPI Specifications**
1. Identify the validation errors.
1. The simplest solution is to use a visual tool to edit and validate the OpenAPI document. For example the [Swagger Editor](https://editor.swagger.io/) highlights schema errors and possible solutions.
- 1. Alternatively, you can check the log output and look for schema validation warnings. They are prefixed with messages such as `OpenAPI 2.0 schema validation error` or `OpenAPI 3.0.x schema validation error`. Each failed validation provides extra information about `location` and `description`. Correct each of the validation failures and then resubmit the OpenAPI doc. Note that JSON Schema validation message might not be easy to understand. This is why we recommend the use of editors to validate document.
-1. Once the validation issues are resolved, re-run your pipeline.
+ 1. Alternatively, you can check the log output and look for schema validation warnings. They are prefixed with messages such as `OpenAPI 2.0 schema validation error` or `OpenAPI 3.0.x schema validation error`. Each failed validation provides extra information about `location` and `description`. Correct each of the validation failures and then resubmit the OpenAPI doc. JSON Schema validation messages can be complex, and editors can help you validate schema documents.
+1. After the validation issues are resolved, re-run your pipeline.
### `Failed to start scanner session (version header not found)`
@@ -2312,7 +2542,10 @@ The API Fuzzing analyzer outputs an error message when it cannot determine the t
There is an order of precedence in which the API Fuzzing analyzer tries to get the target API when checking the different sources. First, it will try to use the `FUZZAPI_TARGET_URL`. If the environment variable has not been set, then the API Fuzzing analyzer will attempt to use the `environment_url.txt` file. If there is no file `environment_url.txt`, the API Fuzzing analyzer will then use the OpenAPI document contents and the URL provided in `FUZZAPI_OPENAPI` (if a URL is provided) to try to compute the target API.
-The best-suited solution will depend on whether or not your target API changes for each deployment. In static environments, the target API is the same for each deployment, in this case please refer to the [static environment solution](#static-environment-solution). If the target API changes for each deployment a [dynamic environment solution](#dynamic-environment-solutions) should be applied.
+The best-suited solution depends on whether or not your target API changes for each deployment:
+
+- If the target API is the same for each deployment (a static environment), use the [static environment solution](#static-environment-solution).
+- If the target API changes for each deployment, use a [dynamic environment solution](#dynamic-environment-solutions).
#### Static environment solution
@@ -2417,10 +2650,10 @@ API Fuzzing uses the specified media types in the OpenAPI document to generate r
## Get support or request an improvement
-To get support for your particular problem please use the [getting help channels](https://about.gitlab.com/get-help/).
+To get support for your particular problem use the [getting help channels](https://about.gitlab.com/get-help/).
The [GitLab issue tracker on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues) is the right place for bugs and feature proposals about API Security and API Fuzzing.
-Please use `~"Category:API Security"` [label](../../../development/contributing/issue_workflow.md#labels) when opening a new issue regarding API fuzzing to ensure it is quickly reviewed by the right people. Please refer to our [review response SLO](https://about.gitlab.com/handbook/engineering/workflow/code-review/#review-response-slo) to understand when you should receive a response.
+Use `~"Category:API Security"` [label](../../../development/contributing/issue_workflow.md#labels) when opening a new issue regarding API fuzzing to ensure it is quickly reviewed by the right people. Refer to our [review response SLO](https://about.gitlab.com/handbook/engineering/workflow/code-review/#review-response-slo) to understand when you should receive a response.
[Search the issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues) for similar entries before submitting your own, there's a good chance somebody else had the same issue or feature proposal. Show your support with an award emoji and or join the discussion.
@@ -2432,7 +2665,7 @@ When experiencing a behavior not working as expected, consider providing context
- Scanner log file available as a job artifact named `gl-api-security-scanner.log`.
WARNING:
-**Sanitize data attached to a support issue**. Please remove sensitive information, including: credentials, passwords, tokens, keys, and secrets.
+**Sanitize data attached to a support issue**. Remove sensitive information, including: credentials, passwords, tokens, keys, and secrets.
## Glossary
diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md
index 5eb1b93eb76..d9ba4640855 100644
--- a/doc/user/application_security/configuration/index.md
+++ b/doc/user/application_security/configuration/index.md
@@ -55,7 +55,7 @@ You can configure the following security controls:
- [Dynamic Application Security Testing](../dast/index.md) (DAST)
- Select **Enable DAST** to configure DAST for the current project.
- Select **Manage scans** to manage the saved DAST scans, site profiles, and scanner profiles.
- For more details, read [DAST on-demand scans](../dast/index.md#on-demand-scans).
+ For more details, read [DAST on-demand scans](../dast/proxy-based.md#on-demand-scans).
- [Dependency Scanning](../dependency_scanning/index.md)
- Select **Configure with a merge request** to create a merge request with the changes required to
enable Dependency Scanning. For more details, see [Enable Dependency Scanning via an automatic merge request](../dependency_scanning/index.md#enable-dependency-scanning-via-an-automatic-merge-request).
diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md
index f6bd6157a28..6fc01a716b2 100644
--- a/doc/user/application_security/container_scanning/index.md
+++ b/doc/user/application_security/container_scanning/index.md
@@ -173,7 +173,7 @@ container_scanning:
before_script:
- ruby -r open-uri -e "IO.copy_stream(URI.open('https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip'), 'awscliv2.zip')"
- unzip awscliv2.zip
- - ./aws/install
+ - sudo ./aws/install
- aws --version
- export AWS_ECR_PASSWORD=$(aws ecr get-login-password --region region)
@@ -259,7 +259,7 @@ including a large number of false positives.
| `CS_IMAGE_SUFFIX` | `""` | Suffix added to `CS_ANALYZER_IMAGE`. If set to `-fips`, `FIPS-enabled` image is used for scan. See [FIPS-enabled images](#fips-enabled-images) for more details. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7630) in GitLab 14.10. | All |
| `CS_IGNORE_UNFIXED` | `"false"` | Ignore vulnerabilities that are not fixed. | All |
| `CS_REGISTRY_INSECURE` | `"false"` | Allow access to insecure registries (HTTP only). Should only be set to `true` when testing the image locally. Works with all scanners, but the registry must listen on port `80/tcp` for Trivy to work. | All |
-| `CS_SEVERITY_THRESHOLD` | `UNKNOWN` | Severity level threshold. The scanner outputs vulnerabilities with severity level higher than or equal to this threshold. Supported levels are Unknown, Low, Medium, High, and Critical. | Trivy |
+| `CS_SEVERITY_THRESHOLD` | `UNKNOWN` | Severity level threshold. The scanner outputs vulnerabilities with severity level higher than or equal to this threshold. Supported levels are `UNKNOWN`, `LOW`, `MEDIUM`, `HIGH`, and `CRITICAL`. | Trivy |
| <!-- start_remove The following content will be removed on remove_date: '2023-08-22' --> `DOCKER_IMAGE` | `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_IMAGE`. The Docker image to be scanned. If set, this variable overrides the `$CI_APPLICATION_REPOSITORY` and `$CI_APPLICATION_TAG` variables. | All |
| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_REGISTRY_PASSWORD`. Password for accessing a Docker registry requiring authentication. The default is only set if `$DOCKER_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All |
| `DOCKER_USER` | `$CI_REGISTRY_USER` | **Deprecated** will be removed in GitLab 16.0. Replaced by `CS_REGISTRY_USER`. Username for accessing a Docker registry requiring authentication. The default is only set if `$DOCKER_IMAGE` resides at [`$CI_REGISTRY`](../../../ci/variables/predefined_variables.md). Not supported when [FIPS mode](../../../development/fips_compliance.md#enable-fips-mode) is enabled. | All |
@@ -359,46 +359,12 @@ The container-scanning analyzer can use different scanners, depending on the val
The following options are available:
-| Scanner name | `CS_ANALYZER_IMAGE` |
-| ------------ | ------------------- |
-| Default ([Trivy](https://github.com/aquasecurity/trivy)) | `registry.gitlab.com/security-products/container-scanning:5` |
+| Scanner name | `CS_ANALYZER_IMAGE` |
+|----------------------------------------------------------|--------------------------------------------------------------------|
+| Default ([Trivy](https://github.com/aquasecurity/trivy)) | `registry.gitlab.com/security-products/container-scanning:5` |
| [Grype](https://github.com/anchore/grype) | `registry.gitlab.com/security-products/container-scanning/grype:5` |
| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:5` |
-If you're migrating from a GitLab 13.x release to a GitLab 14.x release and have customized the
-`container_scanning` job or its CI variables, you might need to perform these migration steps in
-your CI file:
-
-1. Remove these variables:
-
- - `CS_MAJOR_VERSION`
- - `CS_PROJECT`
- - `SECURE_ANALYZERS_PREFIX`
-
-1. Review the `CS_ANALYZER_IMAGE` variable. It no longer depends on the variables above and its new
- default value is `registry.gitlab.com/security-products/container-scanning:5`. If you have an
- offline environment, see
- [Running container scanning in an offline environment](#running-container-scanning-in-an-offline-environment).
-
-1. If present, remove the `.cs_common` and `container_scanning_new` configuration sections.
-
-1. If the `container_scanning` section is present, it's safer to create one from scratch based on
- the new version of the [`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml).
- Once finished, it should not have any variables that are only applicable to Klar or Clair. For a
- complete list of supported variables, see [available variables](#available-cicd-variables).
-
-1. Make any [necessary customizations](#customizing-the-container-scanning-settings)
- to the chosen scanner. We recommend that you minimize such customizations, as they might require
- changes in future GitLab major releases.
-
-1. Trigger a new run of a pipeline that includes the `container_scanning` job. Inspect the job
- output and ensure that the log messages do not mention Clair.
-
-NOTE:
-Prior to the GitLab 14.0 release, any variable defined under the scope `container_scanning` is not
-considered for scanners other than Clair. In GitLab 14.0 and later, all variables can be defined
-either as a global variable or under `container_scanning`.
-
### Setting the default branch image
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338877) in GitLab 14.5.
diff --git a/doc/user/application_security/dast/browser_based.md b/doc/user/application_security/dast/browser_based.md
index 5a4acc78728..c0a97c0ff92 100644
--- a/doc/user/application_security/dast/browser_based.md
+++ b/doc/user/application_security/dast/browser_based.md
@@ -36,8 +36,8 @@ current DAST scanner is much more effective at finding and testing every page in
To enable the browser-based analyzer:
1. Ensure the DAST [prerequisites](index.md#prerequisites) are met.
-1. Include the [DAST CI/CD template](index.md#include-the-dast-template).
-1. Set the target website using the [`DAST_WEBSITE` CI/CD variable](index.md#available-cicd-variables).
+1. Include the [DAST CI/CD template](proxy-based.md#include-the-dast-template).
+1. Set the target website using the [`DAST_WEBSITE` CI/CD variable](proxy-based.md#available-cicd-variables).
1. Set the CI/CD variable `DAST_BROWSER_SCAN` to `true`.
Example extract of `.gitlab-ci.yml` file:
@@ -79,7 +79,7 @@ The browser-based crawler can be configured using CI/CD variables.
| `DAST_BROWSER_ELEMENT_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `600ms` | The maximum amount of time to wait for an element before determining it is ready for analysis. |
| `DAST_BROWSER_PAGE_READY_SELECTOR` | selector | `css:#page-is-ready` | Selector that when detected as visible on the page, indicates to the analyzer that the page has finished loading and the scan can continue. Note: When this selector is set, but the element is not found, the scanner waits for the period defined in `DAST_BROWSER_STABILITY_TIMEOUT` before continuing the scan. This can significantly increase scanning time if the element is not present on multiple pages within the site. |
-The [DAST variables](index.md#available-cicd-variables) `SECURE_ANALYZERS_PREFIX`, `DAST_FULL_SCAN_ENABLED`, `DAST_AUTO_UPDATE_ADDONS`, `DAST_EXCLUDE_RULES`, `DAST_REQUEST_HEADERS`, `DAST_HTML_REPORT`, `DAST_MARKDOWN_REPORT`, `DAST_XML_REPORT`,
+The [DAST variables](proxy-based.md#available-cicd-variables) `SECURE_ANALYZERS_PREFIX`, `DAST_FULL_SCAN_ENABLED`, `DAST_AUTO_UPDATE_ADDONS`, `DAST_EXCLUDE_RULES`, `DAST_REQUEST_HEADERS`, `DAST_HTML_REPORT`, `DAST_MARKDOWN_REPORT`, `DAST_XML_REPORT`,
`DAST_AUTH_URL`, `DAST_USERNAME`, `DAST_PASSWORD`, `DAST_USERNAME_FIELD`, `DAST_PASSWORD_FIELD`, `DAST_FIRST_SUBMIT_FIELD`, `DAST_SUBMIT_FIELD`, `DAST_EXCLUDE_URLS`, `DAST_AUTH_VERIFICATION_URL`, `DAST_BROWSER_AUTH_VERIFICATION_SELECTOR`, `DAST_BROWSER_AUTH_VERIFICATION_LOGIN_FORM`, `DAST_BROWSER_AUTH_REPORT`,
`DAST_INCLUDE_ALPHA_VULNERABILITIES`, `DAST_PATHS_FILE`, `DAST_PATHS`, `DAST_ZAP_CLI_OPTIONS`, and `DAST_ZAP_LOG_CONFIGURATION` are also compatible with browser-based crawler scans.
diff --git a/doc/user/application_security/dast/dast_troubleshooting.md b/doc/user/application_security/dast/dast_troubleshooting.md
index 194761797de..61a7520bf7c 100644
--- a/doc/user/application_security/dast/dast_troubleshooting.md
+++ b/doc/user/application_security/dast/dast_troubleshooting.md
@@ -20,7 +20,7 @@ A DAST job has two executing processes:
Enable the `DAST_DEBUG` CI/CD variable to debug scripts. This can help when troubleshooting the job,
and outputs statements indicating what percentage of the scan is complete.
-For details on using variables, see [Overriding the DAST template](index.md#customize-dast-settings).
+For details on using variables, see [Overriding the DAST template](proxy-based.md#customize-dast-settings).
Debug mode of the ZAP server can be enabled using the `DAST_ZAP_LOG_CONFIGURATION` variable.
The following table outlines examples of values that can be set and the effect that they have on the output that is logged.
diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md
index 6d1b7beefc7..d78a8fca98f 100644
--- a/doc/user/application_security/dast/index.md
+++ b/doc/user/application_security/dast/index.md
@@ -13,16 +13,7 @@ application server or incorrect assumptions about security controls may not be
visible from the source code.
Dynamic Application Security Testing (DAST) examines applications for
-vulnerabilities like these in deployed environments. DAST uses the open source
-tool [OWASP Zed Attack Proxy](https://www.zaproxy.org/) for analysis.
-
-After DAST creates its report, GitLab evaluates it for discovered
-vulnerabilities between the source and target branches. Relevant
-findings are noted in the merge request.
-
-The comparison logic uses only the latest pipeline executed for the target
-branch's base commit. Running the pipeline on other commits has no effect on
-the merge request.
+vulnerabilities like these in deployed environments.
NOTE:
To learn how four of the top six attacks were application-based and how
@@ -30,16 +21,88 @@ to protect your organization, download our
["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/)
whitepaper.
-## DAST application analysis
+## GitLab DAST
+
+GitLab provides the following DAST analyzers, one or more of which may be useful depending on the kind of application you're testing.
+
+For scanning websites, use one of:
+
+- The [DAST proxy-based analyzer](proxy-based.md) for scanning traditional applications serving simple HTML. The proxy-based analyzer can be run automatically or on-demand.
+- The [DAST browser-based analyzer](browser_based.md) for scanning applications that make heavy use of JavaScript. This includes single page web applications.
+
+For scanning APIs, use:
+
+- The [DAST API analyzer](../dast_api/index.md) for scanning web APIs. Web API technologies such as GraphQL, REST, and SOAP are supported.
+
+Analyzers follow the architectural patterns described in [Secure your application](../index.md).
+Each analyzer can be configured in the pipeline using a CI template and runs the scan in a Docker container. Scans output a [DAST report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportsdast)
+which GitLab uses to determine discovered vulnerabilities based on differences between scan results on the source and target branches.
+
+### Getting started
+
+#### Prerequisites
+
+- [GitLab Runner](../../../ci/runners/index.md) available, with the
+ [`docker` executor](https://docs.gitlab.com/runner/executors/docker.html) on Linux/amd64.
+- Target application deployed. For more details, read [Deployment options](#application-deployment-options).
+- `dast` stage added to the CI/CD pipeline definition. This should be added after the deploy step, for example:
+
+ ```yaml
+ stages:
+ - build
+ - test
+ - deploy
+ - dast
+ ```
+
+#### Recommendations
+
+- Take care if your pipeline is configured to deploy to the same web server in each run. Running a DAST scan while a server is being updated will lead to inaccurate and non-deterministic results.
+- Configure runners to use the [always pull policy](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy) to run the latest versions of the analyzers.
+- By default, DAST downloads all artifacts defined by previous jobs in the pipeline. If
+ your DAST job does not rely on `environment_url.txt` to define the URL under test or any other files created
+ in previous jobs, we recommend you don't download artifacts. To avoid downloading
+ artifacts, extend the analyzer CI/CD job to specify no dependencies. For example, for the DAST proxy-based analyzer add the following to your `.gitlab-ci.yml` file:
+
+ ```yaml
+ dast:
+ dependencies: []
+ ```
+
+#### Analyzer configuration
+
+Please see [DAST proxy-based analyzer](proxy-based.md), [DAST browser-based analyzer](browser_based.md) or [DAST API analyzer](../dast_api/index.md) for
+analyzer-specific configuration instructions.
+
+### View scan results
-DAST can analyze applications in two ways:
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36332) in GitLab 13.1.
+
+Detected vulnerabilities are shown in [Merge requests](../index.md#view-security-scan-information-in-merge-requests), the [Pipeline security tab](../index.md#view-security-scan-information-in-the-pipeline-security-tab),
+and the [Vulnerability report](../index.md#view-security-scan-information-in-the-vulnerability-report).
-- Passive scan only (DAST default). DAST executes
- [ZAP's Baseline Scan](https://www.zaproxy.org/docs/docker/baseline-scan/) and doesn't
- actively attack your application.
-- Passive and active scan. DAST can be [configured](#full-scan) to also perform an active scan
- to attack your application and produce a more extensive security report. It can be very
- useful when combined with [Review Apps](../../../ci/review_apps/index.md).
+1. To see all vulnerabilities detected, either:
+ - From your project, select **Security & Compliance**, then **Vulnerability report**.
+ - From your pipeline, click on the **Security** tab.
+ - From the merge request, go to the **Security scanning** widget and click **Full report** tab.
+
+1. Select a DAST vulnerability's description. The following fields are examples of what a DAST analyzer may produce to aid investigation and rectification of the underlying cause. Each analyzer may output different fields.
+
+ | Field | Description |
+ |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------ |
+ | Description | Description of the vulnerability. |
+ | Evidence | Evidence of the data found that verified the vulnerability. Often a snippet of the request or response, this can be used to help verify that the finding is a vulnerability. |
+ | Identifiers | Identifiers of the vulnerability. |
+ | Links | Links to further details of the detected vulnerability. |
+ | Method | HTTP method used to detect the vulnerability. |
+ | Project | Namespace and project in which the vulnerability was detected. |
+ | Request Headers | Headers of the request. |
+ | Response Headers | Headers of the response received from the application. |
+ | Response Status | Response status received from the application. |
+ | Scanner Type | Type of vulnerability report. |
+ | Severity | Severity of the vulnerability. |
+ | Solution | Details of a recommended solution to the vulnerability. |
+ | URL | URL at which the vulnerability was detected. |
NOTE:
A pipeline may consist of multiple jobs, including SAST and DAST scanning. If any job
@@ -48,17 +111,19 @@ example, if the DAST job finishes but the SAST job fails, the security dashboard
results. On failure, the analyzer outputs an
[exit code](../../../development/integrations/secure.md#exit-code).
-## Prerequisites
+#### List URLs scanned
-- [GitLab Runner](../../../ci/runners/index.md) available, with the
-[`docker` executor](https://docs.gitlab.com/runner/executors/docker.html) on Linux/amd64.
-- Target application deployed. For more details, read [Deployment options](#deployment-options).
-- DAST runs in the `dast` stage, which must be added manually to your `.gitlab-ci.yml`.
+When DAST completes scanning, the merge request page states the number of URLs scanned.
+Select **View details** to view the web console output which includes the list of scanned URLs.
+
+![DAST Widget](img/dast_urls_scanned_v12_10.png)
-### Deployment options
+### Application deployment options
+
+DAST requires a deployed application to be available to scan.
Depending on the complexity of the target application, there are a few options as to how to deploy and configure
-the DAST template. We provided a set of example applications with their configurations in our
+the DAST template. A set of example applications have been provided with their configurations in the
[DAST demonstrations](https://gitlab.com/gitlab-org/security-products/demos/dast/) project.
#### Review Apps
@@ -77,6 +142,8 @@ After your Docker build job completes and your image is added to your container
By using service definitions in your `.gitlab-ci.yml`, you can scan services with the DAST analyzer.
+When adding a `services` section to the job, the `alias` is used to define the hostname that can be used to access the service. In the following example, the `alias: yourapp` portion of the `dast` job definition means that the URL to the deployed application will use `yourapp` as the hostname (`https://yourapp/`).
+
```yaml
stages:
- build
@@ -105,6 +172,7 @@ dast:
alias: yourapp
variables:
+ DAST_WEBSITE: https://yourapp
DAST_FULL_SCAN_ENABLED: "true" # do a full scan
DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
```
@@ -122,1323 +190,3 @@ services: # use services to link the container to the dast job
- name: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
alias: yourapp
```
-
-### DAST job order
-
-When using the `DAST.gitlab-ci.yml` template, the `dast` stage is run last as shown in
-the example below. To ensure DAST scans the latest code, deploy your application
-in a stage before the `dast` stage.
-
-```yaml
- stages:
- - build
- - test
- - deploy
- - dast
-```
-
-Take care if your pipeline is configured to deploy to the same web server in each run. Running a
-pipeline while another is still running could result in one pipeline overwriting the code from
-another pipeline. The site to be scanned should be excluded from changes for the duration of a DAST
-scan. The only changes to the site should be from the DAST scanner.
-
-Changes to the site during a scan from any of the following could lead to inaccurate results:
-
-- Users.
-- Scheduled tasks.
-- Database changes.
-- Code changes.
-- Other pipelines.
-- Other scanners.
-
-## DAST run options
-
-You can use DAST to examine your web application:
-
-- Automatically, initiated by a merge request.
-- Manually, initiated on demand.
-
-Some of the differences between these run options:
-
-| Automatic scan | On-demand scan |
-|:-----------------------------------------------------------------|:------------------------------|
-| DAST scan is initiated by a merge request. | DAST scan is initiated manually, outside the DevOps life cycle. |
-| CI/CD variables are sourced from `.gitlab-ci.yml`. | CI/CD variables are provided in the UI. |
-| All [DAST CI/CD variables](#available-cicd-variables) available. | Subset of [DAST CI/CD variables](#available-cicd-variables) available. |
-| `DAST.gitlab-ci.yml` template. | `DAST-On-Demand-Scan.gitlab-ci.yml` template. |
-
-### Enable automatic DAST run
-
-To enable DAST to run automatically, either:
-
-- Enable [Auto DAST](../../../topics/autodevops/stages.md#auto-dast) (provided
- by [Auto DevOps](../../../topics/autodevops/index.md)).
-- [Include the DAST template](#include-the-dast-template) in your existing
- `.gitlab-ci.yml` file.
-- [Configure DAST using the UI](#configure-dast-using-the-ui).
-
-#### Include the DAST template
-
-> - This template was [updated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62597) to DAST_VERSION: 2 in GitLab 14.0.
-> - This template was [updated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87183) to DAST_VERSION: 3 in GitLab 15.0.
-
-If you want to manually add DAST to your application, the DAST job is defined
-in a CI/CD template file. Updates to the template are provided with GitLab
-upgrades, allowing you to benefit from any improvements and additions.
-
-To include the DAST template:
-
-1. Select the CI/CD template you want to use:
-
- - [`DAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml):
- Stable version of the DAST CI/CD template.
- - [`DAST.latest.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST.latest.gitlab-ci.yml):
- Latest version of the DAST template. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254325)
- in GitLab 13.8).
-
- WARNING:
- The latest version of the template may include breaking changes. Use the
- stable template unless you need a feature provided only in the latest template.
-
- For more information about template versioning, see the
- [CI/CD documentation](../../../development/cicd/templates.md#latest-version).
-
-1. Add a `dast` stage to your GitLab CI stages configuration:
-
- ```yaml
- stages:
- - dast
- ```
-
-1. Add the template to GitLab, based on your version of GitLab:
-
- - In GitLab 11.9 and later, [include](../../../ci/yaml/index.md#includetemplate)
- the template by adding the following to your `.gitlab-ci.yml` file:
-
- ```yaml
- include:
- - template: <template_file.yml>
-
- variables:
- DAST_WEBSITE: https://example.com
- ```
-
- - In GitLab 11.8 and earlier, add the contents of the template to your
- `.gitlab_ci.yml` file.
-
-1. Define the URL to be scanned by DAST by using one of these methods:
-
- - Set the `DAST_WEBSITE` [CI/CD variable](../../../ci/yaml/index.md#variables).
- If set, this value takes precedence.
-
- - Add the URL in an `environment_url.txt` file at the root of your project. This is
- useful for testing in dynamic environments. To run DAST against an application
- dynamically created during a GitLab CI/CD pipeline, a job that runs prior to
- the DAST scan must persist the application's domain in an `environment_url.txt`
- file. DAST automatically parses the `environment_url.txt` file to find its
- scan target.
-
- For example, in a job that runs prior to DAST, you could include code that
- looks similar to:
-
- ```yaml
- script:
- - echo http://${CI_PROJECT_ID}-${CI_ENVIRONMENT_SLUG}.domain.com > environment_url.txt
- artifacts:
- paths: [environment_url.txt]
- when: always
- ```
-
- You can see an example of this in our
- [Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml)
- file.
-
-The included template creates a `dast` job in your CI/CD pipeline and scans
-your project's running application for possible vulnerabilities.
-
-The results are saved as a
-[DAST report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportsdast)
-that you can later download and analyze. Due to implementation limitations, we
-always take the latest DAST artifact available. Behind the scenes, the
-[GitLab DAST Docker image](https://gitlab.com/security-products/dast)
-is used to run the tests on the specified URL and scan it for possible
-vulnerabilities.
-
-By default, the DAST template uses the latest major version of the DAST Docker
-image. Using the `DAST_VERSION` variable, you can choose how DAST updates:
-
-- Automatically update DAST with new features and fixes by pinning to a major
- version (such as `1`).
-- Only update fixes by pinning to a minor version (such as `1.6`).
-- Prevent all updates by pinning to a specific version (such as `1.6.4`).
-
-Find the latest DAST versions on the [Releases](https://gitlab.com/gitlab-org/security-products/dast/-/releases)
-page.
-
-#### Configure DAST using the UI
-
-You can enable or configure DAST settings using the UI. The generated settings are formatted so they
-can be conveniently pasted into the `.gitlab-ci.yml` file.
-
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Security & Compliance > Configuration**.
-1. In the **Dynamic Application Security Testing (DAST)** section, select **Enable DAST** or
- **Configure DAST**.
-1. Select the desired **Scanner profile**, or select **Create scanner profile** and save a
- scanner profile. For more details, see [scanner profiles](#scanner-profile).
-1. Select the desired **Site profile**, or select **Create site profile** and save a site
- profile. For more details, see [site profiles](#site-profile).
-1. Select **Generate code snippet**. A modal opens with the YAML snippet corresponding to the
- options you selected.
-1. Do one of the following:
- 1. To copy the snippet to your clipboard, select **Copy code only**.
- 1. To add the snippet to your project's `.gitlab-ci.yml` file, select
- **Copy code and open `.gitlab-ci.yml` file**. The Pipeline Editor opens.
- 1. Paste the snippet into the `.gitlab-ci.yml` file.
- 1. Select the **Lint** tab to confirm the edited `.gitlab-ci.yml` file is valid.
- 1. Select the **Edit** tab, then select **Commit changes**.
-
-When the snippet is committed to the `.gitlab-ci.yml` file, pipelines include a DAST job.
-
-#### Crawling web applications dependent on JavaScript
-
-GitLab has released a new browser-based crawler, an add-on to DAST that uses a browser to crawl web applications for content. This crawler replaces the standard DAST Spider and Ajax Crawler, and uses the same authentication mechanisms as a normal DAST scan.
-
-The browser-based crawler crawls websites by browsing web pages as a user would. This approach works well with web applications that make heavy use of JavaScript, such as Single Page Applications.
-
-For more details, including setup instructions, see [DAST browser-based crawler](browser_based.md).
-
-### Full scan
-
-DAST can be configured to perform [ZAP Full Scan](https://www.zaproxy.org/docs/docker/full-scan/), which
-includes both passive and active scanning against the same target website:
-
-```yaml
-include:
- - template: DAST.gitlab-ci.yml
-
-variables:
- DAST_FULL_SCAN_ENABLED: "true"
- DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
-```
-
-If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some
-tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/2020/08/31/how-to-configure-dast-full-scans-for-complex-web-applications/).
-
-### API scan
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10928) in GitLab 12.10.
-> - A new DAST API scanning engine was introduced in GitLab 13.10.
-
-Using an API specification as a scan's target is a useful way to seed URLs for scanning an API.
-Vulnerability rules in an API scan are different than those in a normal website scan.
-
-A new DAST API scanning engine is available in GitLab 13.12 and later. For more details, see [DAST API scanning engine](../dast_api). The new scanning engine supports REST, SOAP, GraphQL, and generic APIs using forms, XML, and JSON. Testing can be performed using OpenAPI, Postman Collections, and HTTP Archive (HAR) documents.
-
-The target API instance's base URL is provided by using the `DAST_API_TARGET_URL` variable or an `environment_url.txt` file.
-
-#### Specification format
-
-API scans support OpenAPI V2 and OpenAPI V3 specifications. You can define these specifications using `JSON` or `YAML`.
-
-#### Import API specification from a URL
-
-If your API specification is accessible at a URL, you can pass that URL in directly as the target.
-The specification does not have to be hosted on the same host as the API being tested.
-
-```yaml
-include:
- - template: DAST-API.gitlab-ci.yml
-
-variables:
- DAST_API_SPECIFICATION: http://my.api/api-specification.yml
-```
-
-#### Import API specification from a file
-
-If your API specification file is in your repository, you can provide its filename as the target.
-
-```yaml
-dast:
- variables:
- GIT_STRATEGY: fetch
- DAST_API_SPECIFICATION: api-specification.yml
-```
-
-#### Full API scan
-
-API scans support full scanning, which can be enabled by using the `DAST_FULL_SCAN_ENABLED`
-CI/CD variable. Domain validation is not supported for full API scans.
-
-#### Host override
-
-Specifications often define a host, which contains a domain name and a port. The
-host referenced may be different than the host of the API's review instance.
-This can cause incorrect URLs to be imported, or a scan on an incorrect host.
-Use the `DAST_API_HOST_OVERRIDE` CI/CD variable to override these values.
-
-WARNING:
-When using the API host override feature, you cannot use the `$DAST_WEBSITE` variable to override the hostname.
-A host override is _only_ supported when importing the API specification from a URL. Attempts to override the
-host throw an error when the API specification is imported from a file. This is due to a limitation in the
-ZAP OpenAPI extension.
-
-For example, with a OpenAPI V3 specification containing:
-
-```yaml
-servers:
- - url: https://api.host.com
-```
-
-If the test version of the API is running at `https://api-test.host.com`, then
-the following DAST configuration can be used:
-
-```yaml
-include:
- - template: DAST-API.gitlab-ci.yml
-
-variables:
- DAST_API_SPECIFICATION: http://api-test.host.com/api-specification.yml
- DAST_API_HOST_OVERRIDE: api-test.host.com
-```
-
-#### Authentication using headers
-
-Tokens in request headers are often used as a way to authenticate API requests.
-You can achieve this by using the `DAST_REQUEST_HEADERS` CI/CD variable.
-Headers are applied to every request DAST makes.
-
-```yaml
-include:
- - template: DAST-API.gitlab-ci.yml
-
-variables:
- DAST_API_SPECIFICATION: http://api-test.api.com/api-specification.yml
- DAST_REQUEST_HEADERS: "Authorization: Bearer my.token"
-```
-
-### URL scan
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4.
-> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/273141) in GitLab 13.11.
-
-A URL scan allows you to specify which parts of a website are scanned by DAST.
-
-#### Define the URLs to scan
-
-URLs to scan can be specified by either of the following methods:
-
-- Use `DAST_PATHS_FILE` CI/CD variable to specify the name of a file containing the paths.
-- Use `DAST_PATHS` variable to list the paths.
-
-##### Use `DAST_PATHS_FILE` CI/CD variable
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6.
-
-To define the URLs to scan in a file, create a plain text file with one path per line.
-
-```plaintext
-page1.html
-/page2.html
-category/shoes/page1.html
-```
-
-To scan the URLs in that file, set the CI/CD variable `DAST_PATHS_FILE` to the path of that file.
-The file can be checked into the project repository or generated as an artifact by a job that
-runs before DAST.
-
-By default, DAST scans do not clone the project repository. Instruct the DAST job to clone
-the project by setting `GIT_STRATEGY` to fetch. Give a file path relative to `CI_PROJECT_DIR` to `DAST_PATHS_FILE`.
-
-```yaml
-include:
- - template: DAST.gitlab-ci.yml
-
-variables:
- GIT_STRATEGY: fetch
- DAST_PATHS_FILE: url_file.txt # url_file.txt lives in the root directory of the project
- DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
-```
-
-##### Use `DAST_PATHS` CI/CD variable
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4.
-
-To specify the paths to scan in a CI/CD variable, add a comma-separated list of the paths to the `DAST_PATHS`
-variable. Note that you can only scan paths of a single host.
-
-```yaml
-include:
- - template: DAST.gitlab-ci.yml
-
-variables:
- DAST_PATHS: "/page1.html,/category1/page1.html,/page3.html"
- DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
-```
-
-When using `DAST_PATHS` and `DAST_PATHS_FILE`, note the following:
-
-- `DAST_WEBSITE` must be defined when using either `DAST_PATHS_FILE` or `DAST_PATHS`. The paths listed in either use `DAST_WEBSITE` to build the URLs to scan
-- Spidering is disabled when `DAST_PATHS` or `DAST_PATHS_FILE` are defined
-- `DAST_PATHS_FILE` and `DAST_PATHS` cannot be used together
-- The `DAST_PATHS` variable has a limit of about 130kb. If you have a list or paths
- greater than this, use `DAST_PATHS_FILE`.
-
-#### Full Scan
-
-To perform a [full scan](#full-scan) on the listed paths, use the `DAST_FULL_SCAN_ENABLED` CI/CD variable.
-
-### List URLs scanned
-
-When DAST completes scanning, the merge request page states the number of URLs scanned.
-Select **View details** to view the web console output which includes the list of scanned URLs.
-
-![DAST Widget](img/dast_urls_scanned_v12_10.png)
-
-### View details of a vulnerability detected by DAST
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36332) in GitLab 13.1.
-
-Vulnerabilities detected by DAST occur in the live web application. Addressing these types of
-vulnerabilities requires specific information. DAST provides the information required to
-investigate and rectify the underlying cause.
-
-To view details of vulnerabilities detected by DAST:
-
-1. To see all vulnerabilities detected, either:
- - Go to your project and select **Security & Compliance**.
- - Go to the merge request and select the **Security** tab.
-
-1. Select a vulnerability's description. The following details are provided:
-
- | Field | Description |
- |:-----------------|:------------------------------------------------------------------ |
- | Description | Description of the vulnerability. |
- | Project | Namespace and project in which the vulnerability was detected. |
- | Method | HTTP method used to detect the vulnerability. |
- | URL | URL at which the vulnerability was detected. |
- | Request Headers | Headers of the request. |
- | Response Status | Response status received from the application. |
- | Response Headers | Headers of the response received from the application. |
- | Evidence | Evidence of the data found that verified the vulnerability. Often a snippet of the request or response, this can be used to help verify that the finding is a vulnerability. |
- | Identifiers | Identifiers of the vulnerability. |
- | Severity | Severity of the vulnerability. |
- | Scanner Type | Type of vulnerability report. |
- | Links | Links to further details of the detected vulnerability. |
- | Solution | Details of a recommended solution to the vulnerability (optional). |
-
-## Customize DAST settings
-
-You can customize the behavior of DAST using both CI/CD variables and command-line options. Use of CI/CD
-variables overrides the values contained in the DAST template.
-
-### Customize DAST using CI/CD variables
-
-WARNING:
-Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except)
-is no longer supported. You must use [`rules`](../../../ci/yaml/index.md#rules) instead.
-
-The DAST settings can be changed through CI/CD variables by using the
-[`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. For details of
-all DAST CI/CD variables, read [Available CI/CD variables](#available-cicd-variables).
-
-For example:
-
-```yaml
-include:
- - template: DAST.gitlab-ci.yml
-
-variables:
- DAST_WEBSITE: https://example.com
- DAST_SPIDER_MINS: 120
- DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
-```
-
-Because the template is [evaluated before](../../../ci/yaml/index.md#include) the pipeline
-configuration, the last mention of the variable takes precedence.
-
-#### Enable or disable rules
-
-A complete list of the rules that DAST uses to scan for vulnerabilities can be
-found in the [ZAP documentation](https://www.zaproxy.org/docs/alerts/).
-
-`DAST_EXCLUDE_RULES` disables the rules with the given IDs.
-
-`DAST_ONLY_INCLUDE_RULES` restricts the set of rules used in the scan to
-those with the given IDs.
-
-`DAST_EXCLUDE_RULES` and `DAST_ONLY_INCLUDE_RULES` are mutually exclusive and a
-DAST scan with both configured exits with an error.
-
-By default, several rules are disabled because they either take a long time to
-run or frequently generate false positives. The complete list of disabled rules
-can be found in [`exclude_rules.yml`](https://gitlab.com/gitlab-org/security-products/dast/-/blob/main/src/config/exclude_rules.yml).
-
-The lists for `DAST_EXCLUDE_RULES` and `DAST_ONLY_INCLUDE_RULES` **must** be enclosed in double
-quotes (`"`), otherwise they are interpreted as numeric values.
-
-#### Hide sensitive information
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36332) in GitLab 13.1.
-
-HTTP request and response headers may contain sensitive information, including cookies and
-authorization credentials. By default, the following headers are masked:
-
-- `Authorization`.
-- `Proxy-Authorization`.
-- `Set-Cookie` (values only).
-- `Cookie` (values only).
-
-Using the [`DAST_MASK_HTTP_HEADERS` CI/CD variable](#available-cicd-variables), you can list the
-headers whose values you want masked. For details on how to mask headers, see
-[Customizing the DAST settings](#customize-dast-settings).
-
-#### Use Mutual TLS
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299596) in GitLab 14.8.
-
-Mutual TLS allows a target application server to verify that requests are from a known source. Browser-based scans do not support Mutual TLS.
-
-**Requirements**
-
-- Base64-encoded PKCS12 certificate
-- Password of the base64-encoded PKCS12 certificate
-
-To enable Mutual TLS:
-
-1. If the PKCS12 certificate is not already base64-encoded, convert it to base64 encoding. For security reasons, we recommend encoding the certificate locally, **not** using a web-hosted conversion service. For example, to encode the certificate on either macOS or Linux:
-
- ```shell
- base64 <path-to-pkcs12-certificate-file>
- ```
-
-1. Create a [masked variable](../../../ci/variables/index.md) named `DAST_PKCS12_CERTIFICATE_BASE64` and store the base64-encoded PKCS12 certificate's value in that variable.
-1. Create a masked variable `DAST_PKCS12_PASSWORD` and store the PKCS12 certificate's password in that variable.
-
-#### Available CI/CD variables
-
-These CI/CD variables are specific to DAST. They can be used to customize the behavior of DAST to your requirements.
-
-WARNING:
-All customization of GitLab security scanning tools should be tested in a merge request before
-merging these changes to the default branch. Failure to do so can give unexpected results,
-including a large number of false positives.
-
-| CI/CD variable | Type | Description |
-|:-------------------------------------------------|:--------------|:------------------------------|
-| `DAST_ADVERTISE_SCAN` | boolean | Set to `true` to add a `Via` header to every request sent, advertising that the request was sent as part of a GitLab DAST scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334947) in GitLab 14.1. |
-| `DAST_AGGREGATE_VULNERABILITIES` | boolean | Vulnerability aggregation is set to `true` by default. To disable this feature and see each vulnerability individually set to `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254043) in GitLab 14.0. |
-| `DAST_API_HOST_OVERRIDE` <sup>1</sup> | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080`. |
-| `DAST_API_SPECIFICATION` <sup>1</sup> | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. The variable `DAST_WEBSITE` must be specified if this is omitted. |
-| `DAST_AUTH_REPORT` <sup>2</sup> | boolean | Used in combination with exporting the `gl-dast-debug-auth-report.html` artifact to aid in debugging authentication issues. |
-| `DAST_AUTH_EXCLUDE_URLS` <sup>2</sup> | URLs | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/289959)** in GitLab 14.0. Replaced by `DAST_EXCLUDE_URLS`. The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. |
-| `DAST_AUTH_URL` <sup>1,2</sup> | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Not supported for API scans. Example: `https://login.example.com`. |
-| `DAST_AUTH_VERIFICATION_LOGIN_FORM` <sup>2</sup> | boolean | Verifies successful authentication by checking for the lack of a login form once the login form has been submitted. |
-| `DAST_AUTH_VERIFICATION_SELECTOR` <sup>2</sup> | selector | Verifies successful authentication by checking for presence of a selector once the login form has been submitted. Example: `css:.user-photo`. |
-| `DAST_AUTH_VERIFICATION_URL` <sup>1,2</sup> | URL | A URL only accessible to logged in users that DAST can use to confirm successful authentication. If provided, DAST exits if it cannot access the URL. Example: `"http://example.com/loggedin_page"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. |
-| `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false`. |
-| `DAST_BROWSER_PATH_TO_LOGIN_FORM` <sup>1,2</sup> | selector | Comma-separated list of selectors that are selected prior to attempting to enter `DAST_USERNAME` and `DAST_PASSWORD` into the login form. Example: `"css:.navigation-menu,css:.login-menu-item"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326633) in GitLab 14.1. |
-| `DAST_DEBUG` <sup>1</sup> | boolean | Enable debug message output. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
-| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). For example, `HTTP Parameter Override` has a rule ID of `10026`. Cannot be used when `DAST_ONLY_INCLUDE_RULES` is set. **Note:** In earlier versions of GitLab the excluded rules were executed but vulnerabilities they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. |
-| `DAST_EXCLUDE_URLS` <sup>1,2</sup> | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. Example, `http://example.com/sign-out`. |
-| `DAST_FIRST_SUBMIT_FIELD` <sup>2</sup> | string | The `id` or `name` of the element that when selected submits the username form of a multi-page login process. For example, `css:button[type='user-submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. |
-| `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | boolean | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/293595)** in GitLab 14.0. Set to `true` to require domain validation when running DAST full scans. Not supported for API scans. Default: `false` |
-| `DAST_FULL_SCAN_ENABLED` <sup>1</sup> | boolean | Set to `true` to run a [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of a [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Default: `false` |
-| `DAST_HTML_REPORT` | string | The filename of the HTML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
-| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
-| `DAST_MARKDOWN_REPORT` | string | The filename of the Markdown report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
-| `DAST_MASK_HTTP_HEADERS` | string | Comma-separated list of request and response headers to be masked (GitLab 13.1). Must contain **all** headers to be masked. Refer to [list of headers that are masked by default](#hide-sensitive-information). |
-| `DAST_MAX_URLS_PER_VULNERABILITY` | number | The maximum number of URLs reported for a single vulnerability. `DAST_MAX_URLS_PER_VULNERABILITY` is set to `50` by default. To list all the URLs set to `0`. [Introduced](https://gitlab.com/gitlab-org/security-products/dast/-/merge_requests/433) in GitLab 13.12. |
-| `DAST_ONLY_INCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to configure the scan to run only them. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). Cannot be used when `DAST_EXCLUDE_RULES` is set. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250651) in GitLab 13.12. |
-| `DAST_PASSWORD` <sup>1,2</sup> | string | The password to authenticate to in the website. Example: `P@55w0rd!` |
-| `DAST_PASSWORD_FIELD` <sup>1,2</sup> | string | The selector of password field at the sign-in HTML form. Example: `id:password` |
-| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4. |
-| `DAST_PATHS_FILE` | string | The file path containing the paths within `DAST_WEBSITE` to scan. The file must be plain text with one path per line. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6. |
-| `DAST_PKCS12_CERTIFICATE_BASE64` | string | The PKCS12 certificate used for sites that require Mutual TLS. Must be encoded as base64 text. |
-| `DAST_PKCS12_PASSWORD` | string | The password of the certificate used in `DAST_PKCS12_CERTIFICATE_BASE64`. |
-| `DAST_REQUEST_HEADERS` <sup>1</sup> | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` |
-| `DAST_SKIP_TARGET_CHECK` | boolean | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229067) in GitLab 13.8. |
-| `DAST_SPIDER_MINS` <sup>1</sup> | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
-| `DAST_SPIDER_START_AT_HOST` | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` is reset to `http://test.site` before scan. Default: `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258805) in GitLab 13.6. |
-| `DAST_SUBMIT_FIELD` <sup>2</sup> | string | The `id` or `name` of the element that when selected submits the login form or the password form of a multi-page login process. For example, `css:button[type='submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. |
-| `DAST_TARGET_AVAILABILITY_TIMEOUT` <sup>1</sup> | number | Time limit in seconds to wait for target availability. |
-| `DAST_USE_AJAX_SPIDER` <sup>1</sup> | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
-| `DAST_USERNAME` <sup>1,2</sup> | string | The username to authenticate to in the website. Example: `admin` |
-| `DAST_USERNAME_FIELD` <sup>1,2</sup> | string | The selector of username field at the sign-in HTML form. Example: `name:username` |
-| `DAST_XML_REPORT` | string | The filename of the XML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
-| `DAST_WEBSITE` <sup>1</sup> | URL | The URL of the website to scan. The variable `DAST_API_SPECIFICATION` must be specified if this is omitted. |
-| `DAST_ZAP_CLI_OPTIONS` | string | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
-| `DAST_ZAP_LOG_CONFIGURATION` | string | Set to a semicolon-separated list of additional log4j properties for the ZAP Server. Example: `logger.httpsender.name=org.parosproxy.paros.network.HttpSender;logger.httpsender.level=debug;logger.sitemap.name=org.parosproxy.paros.model.SiteMap;logger.sitemap.level=debug;` |
-| `SECURE_ANALYZERS_PREFIX` | URL | Set the Docker registry base address from which to download the analyzer. |
-
-1. Available to an on-demand DAST scan.
-1. Used for authentication.
-
-### Customize DAST using command-line options
-
-Not all DAST configuration is available via CI/CD variables. To find out all
-possible options, run the following configuration.
-Available command-line options are printed to the job log:
-
-```yaml
-include:
- template: DAST.gitlab-ci.yml
-
-dast:
- script:
- - /analyze --help
-```
-
-You must then overwrite the `script` command to pass in the appropriate
-argument. For example, vulnerability definitions in alpha can be included with
-`-a`. The following configuration includes those definitions:
-
-```yaml
-include:
- template: DAST.gitlab-ci.yml
-
-dast:
- script:
- - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)}
- - /analyze -a -t $DAST_WEBSITE
-```
-
-### Custom ZAProxy configuration
-
-The ZAProxy server contains many [useful configurable values](https://gitlab.com/gitlab-org/gitlab/-/issues/36437#note_245801885).
-Many key/values for `-config` remain undocumented, but there is an untested list of
-[possible keys](https://gitlab.com/gitlab-org/gitlab/-/issues/36437#note_244981023).
-Note that these options are not supported by DAST, and may break the DAST scan
-when used. An example of how to rewrite the Authorization header value with `TOKEN` follows:
-
-```yaml
-include:
- template: DAST.gitlab-ci.yml
-
-variables:
- DAST_ZAP_CLI_OPTIONS: "-config replacer.full_list(0).description=auth -config replacer.full_list(0).enabled=true -config replacer.full_list(0).matchtype=REQ_HEADER -config replacer.full_list(0).matchstr=Authorization -config replacer.full_list(0).regex=false -config replacer.full_list(0).replacement=TOKEN"
-```
-
-## Authentication
-
-NOTE:
-We highly recommend you configure the scanner to authenticate to the application. If you don't, it cannot check most of the application for security risks, as most
-of your application is likely not accessible without authentication. We also recommend
-you periodically confirm the scanner's authentication is still working, as this tends to break over
-time due to authentication changes to the application.
-
-Create masked CI/CD variables to pass the credentials that DAST uses.
-To create masked variables for the username and password, see [Create a custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables).
-The key of the username variable must be `DAST_USERNAME`,
-and the key of the password variable must be `DAST_PASSWORD`.
-
-After DAST has authenticated with the application, all cookies are collected from the web browser.
-For each cookie a matching session token is created for use by ZAP. This ensures ZAP is recognized
-by the application as correctly authenticated.
-
-Authentication supports single form logins, multi-step login forms, and authenticating to URLs outside of the configured target URL.
-
-WARNING:
-**Never** run an authenticated scan against a production server. When an authenticated
-scan is run, it may perform *any* function that the authenticated user can. This
-includes actions like modifying and deleting data, submitting forms, and following links.
-Only run an authenticated scan against a test server.
-
-### SSO
-
-DAST can authenticate to websites making use of SSO, with the following restrictions:
-
-- DAST cannot bypass a CAPTCHA if the authentication flow includes one.
-- DAST cannot handle multi-factor authentication like one-time passwords (OTP) by using SMS or authenticator apps.
-- DAST must get a cookie, or a local or session storage, with a sufficiently random value.
-
-The [authentication debug output](index.md#configure-the-authentication-debug-output) can be helpful for troubleshooting SSO authentication
-with DAST.
-
-### Log in using automatic detection of the login form
-
-By providing a `DAST_USERNAME`, `DAST_PASSWORD`, and `DAST_AUTH_URL`, DAST attempts to authenticate to the
-target application by locating the login form based on a determination about whether or not the form contains username or password fields.
-
-Automatic detection is "best-effort", and depending on the application being scanned may provide either a resilient login experience or one that fails to authenticate the user.
-
-Login process:
-
-1. The `DAST_AUTH_URL` is loaded into the browser, and any forms on the page are located.
- 1. If a form contains a username and password field, `DAST_USERNAME` and `DAST_PASSWORD` is inputted into the respective fields, the form submit button is selected and the user is logged in.
- 1. If a form contains only a username field, it is assumed that the login form is multi-step.
- 1. The `DAST_USERNAME` is inputted into the username field and the form submit button is selected.
- 1. The subsequent pages loads where it is expected that a form exists and contains a password field. If found, `DAST_PASSWORD` is inputted, form submit button is selected and the user is logged in.
-
-### Log in using explicit selection of the login form
-
-By providing a `DAST_USERNAME_FIELD`, `DAST_PASSWORD_FIELD`, and `DAST_SUBMIT_FIELD`, in addition to the fields required for automatic login,
-DAST attempts to authenticate to the target application by locating the login form based on the selectors provided.
-Most applications benefit from this approach to authentication.
-
-Login process:
-
-1. The `DAST_AUTH_URL` is loaded into the browser, and any forms on the page are located.
- 1. If the `DAST_FIRST_SUBMIT_FIELD` is not defined, then `DAST_USERNAME` is inputted into `DAST_USERNAME_FIELD`, `DAST_PASSWORD` is inputted into `DAST_PASSWORD_FIELD`, `DAST_SUBMIT_FIELD` is selected and the user is logged in.
- 1. If the `DAST_FIRST_SUBMIT_FIELD` is defined, then it is assumed that the login form is multi-step.
- 1. The `DAST_USERNAME` is inputted into the `DAST_USERNAME_FIELD` field and the `DAST_FIRST_SUBMIT_FIELD` is selected.
- 1. The subsequent pages loads where the `DAST_PASSWORD` is inputted into the `DAST_PASSWORD_FIELD` field, the `DAST_SUBMIT_FIELD` is selected and the user is logged in.
-
-### Verifying successful login
-
-Once the login form has been submitted, DAST determines if the login was successful. Unsuccessful attempts at authentication cause the scan to halt.
-
-Following the submission of the login form, authentication is determined to be unsuccessful when:
-
-- A `400` or `500` series HTTP response status code is returned.
-- A new cookie/browser storage value determined to be sufficiently random has not been set.
-
-In addition to these checks, the user can configure their own verification checks.
-Each of the following checks can be used in conjunction with one another, if none are configured by default the presence of a login form is checked.
-
-#### Verifying based on the URL
-
-When `DAST_AUTH_VERIFICATION_URL` is configured, the URL displayed in the browser tab post login form submission is directly compared to the URL in the CI/CD variable.
-If these are not exactly the same, authentication is deemed to be unsuccessful.
-
-For example:
-
-```yaml
-include:
- - template: DAST.gitlab-ci.yml
-
-dast:
- variables:
- DAST_WEBSITE: "https://example.com"
- DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
- ...
- DAST_AUTH_VERIFICATION_URL: "https://example.com/user/welcome"
-```
-
-#### Verify based on presence of an element
-
-When `DAST_AUTH_VERIFICATION_SELECTOR` is configured, the page displayed in the browser tab is searched for an element described by the selector in the CI/CD variable.
-If no element is found, authentication is deemed to be unsuccessful.
-
-For example:
-
-```yaml
-include:
- - template: DAST.gitlab-ci.yml
-
-dast:
- variables:
- DAST_WEBSITE: "https://example.com"
- DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
- ...
- DAST_AUTH_VERIFICATION_SELECTOR: "css:.welcome-user"
-```
-
-#### Verify based on presence of a login form
-
-When `DAST_AUTH_VERIFICATION_LOGIN_FORM` is configured, the page displayed in the browser tab is searched for a form that is detected to be a login form.
-If any such form is found, authentication is deemed to be unsuccessful.
-
-For example:
-
-```yaml
-include:
- - template: DAST.gitlab-ci.yml
-
-dast:
- variables:
- DAST_WEBSITE: "https://example.com"
- DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
- ...
- DAST_AUTH_VERIFICATION_LOGIN_FORM: "true"
-```
-
-### View the login form
-
-Many web applications show the user the login form in a pop-up (modal) window.
-For these applications, navigating to the form requires both:
-
-- A starting URL.
-- A list of elements to select to display the modal window.
-
-When `DAST_BROWSER_PATH_TO_LOGIN_FORM` is present, like in this example:
-
-```yaml
-include:
- - template: DAST.gitlab-ci.yml
-
-dast:
- variables:
- DAST_WEBSITE: "https://my.site.com"
- DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
- ...
- DAST_AUTH_URL: "https://my.site.com/admin"
- DAST_BROWSER_PATH_TO_LOGIN_FORM: "css:.navigation-menu,css:.login-menu-item"
-```
-
-DAST performs these actions:
-
-1. Load the `DAST_AUTH_URL` page, such as `https://my.site.com/admin`.
-1. After the page loads, DAST selects elements found by the selectors described
- in `DAST_BROWSER_PATH_TO_LOGIN_FORM`. This example opens the navigation menu
- and selects the login menu, to display the login modal window.
-1. To continue the authentication process, DAST fills in the username and password
- on the login form.
-
-### Configure the authentication debug output
-
-It is often difficult to understand the cause of an authentication failure when running DAST in a CI/CD pipeline.
-To assist users in debugging authentication issues, a debug report can be generated and saved as a job artifact.
-This HTML report contains all steps made during the login process, along with HTTP requests and responses, the Document Object Model (DOM) and screenshots.
-
-![dast-auth-report](img/dast_auth_report.jpg)
-
-An example configuration where the authentication debug report is exported may look like the following:
-
-```yaml
-dast:
- variables:
- DAST_WEBSITE: "https://example.com"
- DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
- ...
- DAST_AUTH_REPORT: "true"
- artifacts:
- paths: [gl-dast-debug-auth-report.html]
- when: always
-```
-
-### Selectors
-
-Selectors are used by CI/CD variables to specify the location of an element displayed on a page in a browser.
-Selectors have the format `type`:`search string`. The crawler searches for the selector using the search string based on the type.
-
-| Selector type | Example | Description |
-| ------------- | ---------------------------------- | ----------- |
-| `css` | `css:.password-field` | Searches for a HTML element having the supplied CSS selector. Selectors should be as specific as possible for performance reasons. |
-| `id` | `id:element` | Searches for an HTML element with the provided element ID. |
-| `name` | `name:element` | Searches for an HTML element with the provided element name. |
-| `xpath` | `xpath://input[@id="my-button"]/a` | Searches for a HTML element with the provided XPath. Note that XPath searches are expected to be less performant than other searches. |
-| None provided | `a.click-me` | Defaults to searching using a CSS selector. |
-
-#### Find selectors with Google Chrome
-
-Chrome DevTools element selector tool is an effective way to find a selector.
-
-1. Open Chrome and navigate to the page where you would like to find a selector, for example, the login page for your site.
-1. Open the `Elements` tab in Chrome DevTools with the keyboard shortcut `Command + Shift + c` in macOS or `Ctrl + Shift + c` in Windows.
-1. Select the `Select an element in the page to select it` tool.
- ![search-elements](img/dast_auth_browser_scan_search_elements.png)
-1. Select the field on your page that you would like to know the selector for.
-1. Once the tool is active, highlight a field you wish to view the details of.
- ![highlight](img/dast_auth_browser_scan_highlight.png)
-1. Once highlighted, you can see the element's details, including attributes that would make a good candidate for a selector.
-
-In this example, the `id="user_login"` appears to be a good candidate. You can use this as a selector as the DAST username field by setting
-`DAST_USERNAME_FIELD: "id:user_login"`.
-
-#### Choose the right selector
-
-Judicious choice of selector leads to a scan that is resilient to the application changing.
-
-In order of preference, it is recommended to choose as selectors:
-
-- `id` fields. These are generally unique on a page, and rarely change.
-- `name` fields. These are generally unique on a page, and rarely change.
-- `class` values specific to the field, such as the selector `"css:.username"` for the `username` class on the username field.
-- Presence of field specific data attributes, such as the selector, `"css:[data-username]"` when the `data-username` field has any value on the username field.
-- Multiple `class` hierarchy values, such as the selector `"css:.login-form .username"` when there are multiple elements with class `username` but only one nested inside the element with the class `login-form`.
-
-When using selectors to locate specific fields we recommend you avoid searching on:
-
-- Any `id`, `name`, `attribute`, `class` or `value` that is dynamically generated.
-- Generic class names, such as `column-10` and `dark-grey`.
-- XPath searches as they are less performant than other selector searches.
-- Unscoped searches, such as those beginning with `css:*` and `xpath://*`.
-
-### Bleeding-edge vulnerability definitions
-
-ZAP first creates rules in the `alpha` class. After a testing period with
-the community, they are promoted to `beta`. DAST uses `beta` definitions by
-default. To request `alpha` definitions, use the
-`DAST_INCLUDE_ALPHA_VULNERABILITIES` CI/CD variable as shown in the
-following configuration:
-
-```yaml
-include:
- template: DAST.gitlab-ci.yml
-
-variables:
- DAST_INCLUDE_ALPHA_VULNERABILITIES: "true"
-```
-
-### Cloning the project's repository
-
-The DAST job does not require the project's repository to be present when running, so by default
-[`GIT_STRATEGY`](../../../ci/runners/configure_runners.md#git-strategy) is set to `none`.
-
-## On-demand scans
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2.
-> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3.
-> - The saved scans feature was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5100) in GitLab 13.9.
-> - The option to select a branch was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4847) in GitLab 13.10.
-> - DAST branch selection [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/322672) in GitLab 13.11.
-> - Auditing for DAST profile management was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1.
-
-An on-demand DAST scan runs outside the DevOps life cycle. Changes in your repository don't trigger
-the scan. You must either start it manually, or schedule it to run.
-
-An on-demand DAST scan:
-
-- Can run a specific combination of a [site profile](#site-profile) and a
- [scanner profile](#scanner-profile).
-- Is associated with your project's default branch.
-- Is saved on creation so it can be run later.
-
-### On-demand scan modes
-
-An on-demand scan can be run in active or passive mode:
-
-- _Passive mode_ is the default and runs a ZAP Baseline Scan.
-- _Active mode_ runs a ZAP Full Scan which is potentially harmful to the site being scanned. To
- minimize the risk of accidental damage, running an active scan requires a [validated site profile](#site-profile-validation).
-
-### View on-demand DAST scans
-
-To view running completed and scheduled on-demand DAST scans for a project, go to
-**Security & Compliance > On-demand Scans** in the left sidebar.
-
-- To view both running and completed scans, select **All**.
-- To view running scans only, select **Running**.
-- To view finished scans, select **Finished**. A finished scan is a scan that either succeeded,
- failed, or was canceled.
-- To view scheduled scans, select **Scheduled**. It shows on-demand scans that have a schedule
- set up. Those are _not_ included in the **All** tab.
-- To view saved on-demand scan profiles, select **Scan library**.
- Those are _not_ included in the **All** tab.
-
-#### Cancel an on-demand scan
-
-To cancel a pending or running on-demand scan, select **Cancel** (**{cancel}**) in the
-on-demand scans list.
-
-#### Retry an on-demand scan
-
-To retry a scan that failed or succeeded with warnings, select **Retry** (**{retry}**) in the
-on-demand scans list.
-
-#### View an on-demand scan's results
-
-To view a finished scan's results, select **View results** in the on-demand scans list.
-
-#### Edit an on-demand scan
-
-To edit an on-demand scan's settings, select **Edit** (**{pencil}**) in the **Scheduled** tab.
-
-### Run an on-demand DAST scan
-
-Prerequisites:
-
-- You must have permission to run an on-demand DAST scan against a protected branch. The default
- branch is automatically protected. For more information, read
- [Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches).
-- A [scanner profile](#create-a-scanner-profile).
-- A [site profile](#create-a-site-profile).
-- If you are running an active scan the site profile must have been [validated](#validate-a-site-profile).
-
-You can run an on-demand scan immediately, once at a scheduled date and time or at a specified
-frequency:
-
-- Every day
-- Every week
-- Every month
-- Every 3 months
-- Every 6 months
-- Every year
-
-To run an on-demand scan immediately, either:
-
-- [Create and run an on-demand scan immediately](#create-and-run-an-on-demand-scan-immediately).
-- [Run a previously saved on-demand scan](#run-a-saved-on-demand-scan).
-
-To run an on-demand scan either at a scheduled date or frequency, read
-[Schedule an on-demand scan](#schedule-an-on-demand-scan).
-
-#### Create and run an on-demand scan immediately
-
-1. From your project's home page, go to **Security & Compliance > On-demand Scans** in the left
- sidebar.
-1. Select **New scan**.
-1. Complete the **Scan name** and **Description** fields.
-1. In GitLab 13.10 and later, select the desired branch from the **Branch** dropdown.
-1. In **Scanner profile**, select a scanner profile from the dropdown.
-1. In **Site profile**, select a site profile from the dropdown.
-1. To run the on-demand scan immediately, select **Save and run scan**. Otherwise, select
- **Save scan** to [run](#run-a-saved-on-demand-scan) it later.
-
-The on-demand DAST scan runs and the project's dashboard shows the results.
-
-#### Run a saved on-demand scan
-
-To run a saved on-demand scan:
-
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Security & Compliance > On-demand Scans**.
-1. Select the **Scan library** tab.
-1. In the scan's row, select **Run scan**.
-
- If the branch saved in the scan no longer exists, you must first
- [edit the scan](#edit-an-on-demand-scan), select a new branch, and save the edited scan.
-
-The on-demand DAST scan runs, and the project's dashboard shows the results.
-
-#### Schedule an on-demand scan
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.3. [Deployed behind the `dast_on_demand_scans_scheduler` flag](../../../administration/feature_flags.md), disabled by default.
-> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.4.
-> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.4.
-> - [Feature flag `dast_on_demand_scans_scheduler` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.5.
-
-To schedule a scan:
-
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Security & Compliance > On-demand Scans**.
-1. Select **New scan**.
-1. Complete the **Scan name** and **Description** text boxes.
-1. In GitLab 13.10 and later, from the **Branch** dropdown list, select the desired branch.
-1. In the **Scanner profile** section, from the dropdown list, select a scanner profile.
-1. In the **Site profile** section, from the dropdown list, select a site profile.
-1. Select **Schedule scan**.
-1. In the **Start time** section, select a time zone, date, and time.
-1. From the **Repeats** dropdown list, select your desired frequency:
- - To run the scan once, select **Never**.
- - For a recurring scan, select any other option.
-1. To run the on-demand scan immediately, select **Save and run scan**. To [run](#run-a-saved-on-demand-scan) it according to the schedule you set, select
- **Save scan**.
-
-#### List saved on-demand scans
-
-To list saved on-demand scans:
-
-1. From your project's home page, go to **Security & Compliance > On-demand Scans**.
-1. Select the **Scan library** tab.
-
-#### View details of an on-demand scan
-
-To view details of an on-demand scan:
-
-1. From your project's home page, go to **Security & Compliance > On-demand Scans**.
-1. Select the **Scan library** tab.
-1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Edit**.
-
-#### Edit an on-demand scan
-
-To edit an on-demand scan:
-
-1. From your project's home page, go to **Security & Compliance > On-demand Scans**.
-1. Select the **Scan library** tab.
-1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Edit**.
-1. Edit the form.
-1. Select **Save scan**.
-
-#### Delete an on-demand scan
-
-To delete an on-demand scan:
-
-1. From your project's home page, go to **Security & Compliance > On-demand Scans**.
-1. Select the **Scan library** tab.
-1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Delete**.
-1. Select **Delete** to confirm the deletion.
-
-## Site profile
-
-A site profile defines the attributes and configuration details of the deployed application,
-website, or API to be scanned by DAST. A site profile can be referenced in `.gitlab-ci.yml` and
-on-demand scans.
-
-A site profile contains:
-
-- **Profile name**: A name you assign to the site to be scanned. While a site profile is referenced
- in either `.gitlab-ci.yml` or an on-demand scan, it **cannot** be renamed.
-- **Site type**: The type of target to be scanned, either website or API scan.
-- **Target URL**: The URL that DAST runs against.
-- **Excluded URLs**: A comma-separated list of URLs to exclude from the scan.
-- **Request headers**: A comma-separated list of HTTP request headers, including names and values. These headers are added to every request made by DAST.
-- **Authentication**:
- - **Authenticated URL**: The URL of the page containing the sign-in HTML form on the target website. The username and password are submitted with the login form to create an authenticated scan.
- - **Username**: The username used to authenticate to the website.
- - **Password**: The password used to authenticate to the website.
- - **Username form field**: The name of username field at the sign-in HTML form.
- - **Password form field**: The name of password field at the sign-in HTML form.
- - **Submit form field**: The `id` or `name` of the element that when selected submits the sign-in HTML form.
-
-When an API site type is selected, a [host override](#host-override) is used to ensure the API being scanned is on the same host as the target. This is done to reduce the risk of running an active scan against the wrong API.
-
-When configured, request headers and password fields are encrypted using [`aes-256-gcm`](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) before being stored in the database.
-This data can only be read and decrypted with a valid secrets file.
-
-### Site profile validation
-
-> - Site profile validation [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233020) in GitLab 13.8.
-> - Meta tag validation [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6460) in GitLab 14.2.
-
-Site profile validation reduces the risk of running an active scan against the wrong website. A site
-must be validated before an active scan can run against it. The site validation methods are as
-follows:
-
-- _Text file validation_ requires a text file be uploaded to the target site. The text file is
- allocated a name and content that is unique to the project. The validation process checks the
- file's content.
-- _Header validation_ requires the header `Gitlab-On-Demand-DAST` be added to the target site,
- with a value unique to the project. The validation process checks that the header is present, and
- checks its value.
-- _Meta tag validation_ requires the meta tag named `gitlab-dast-validation` be added to the target site,
- with a value unique to the project. Make sure it's added to the `<head>` section of the page. The validation process checks that the meta tag is present, and
- checks its value.
-
-All these methods are equivalent in functionality. Use whichever is feasible.
-
-In [GitLab 14.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/324990), site profile
-validation happens in a CI job using the [GitLab Runner](../../../ci/runners/index.md).
-
-### Create a site profile
-
-To create a site profile:
-
-1. From your project's home page, go to **Security & Compliance > Configuration**.
-1. Select **Manage** in the **DAST Profiles** row.
-1. Select **New > Site Profile**.
-1. Complete the fields then select **Save profile**.
-
-The site profile is created.
-
-### Edit a site profile
-
-If a site profile is linked to a security policy, a user cannot edit the profile from this page. See
-[Scan execution policies](../policies/scan-execution-policies.md)
-for more information.
-
-When a validated site profile's file, header, or meta tag is edited, the site's
-[validation status](#site-profile-validation) is revoked.
-
-To edit a site profile:
-
-1. From your project's home page, go to **Security & Compliance > Configuration**.
-1. In the **DAST Profiles** row select **Manage**.
-1. Select the **Site Profiles** tab.
-1. In the profile's row select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**.
-1. Edit the fields then select **Save profile**.
-
-### Delete a site profile
-
-If a site profile is linked to a security policy, a user cannot delete the profile from this page.
-See [Scan execution policies](../policies/scan-execution-policies.md)
-for more information.
-
-To delete a site profile:
-
-1. From your project's home page, go to **Security & Compliance > Configuration**.
-1. In the **DAST Profiles** row select **Manage**.
-1. Select the **Site Profiles** tab.
-1. In the profile's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**.
-1. Select **Delete** to confirm the deletion.
-
-### Validate a site profile
-
-Validating a site is required to run an active scan.
-
-To validate a site profile:
-
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Security & Compliance > Configuration**.
-1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**.
-1. Select the **Site Profiles** tab.
-1. In the profile's row, select **Validate**.
-1. Select the validation method.
- 1. For **Text file validation**:
- 1. Download the validation file listed in **Step 2**.
- 1. Upload the validation file to the host, to the location in **Step 3** or any location you
- prefer.
- 1. If required, edit the file location in **Step 3**.
- 1. Select **Validate**.
- 1. For **Header validation**:
- 1. Select the clipboard icon in **Step 2**.
- 1. Edit the header of the site to validate, and paste the clipboard content.
- 1. Select the input field in **Step 3** and enter the location of the header.
- 1. Select **Validate**.
- 1. For **Meta tag validation**:
- 1. Select the clipboard icon in **Step 2**.
- 1. Edit the content of the site to validate, and paste the clipboard content.
- 1. Select the input field in **Step 3** and enter the location of the meta tag.
- 1. Select **Validate**.
-
-The site is validated and an active scan can run against it. A site profile's validation status is
-revoked only when it's revoked manually, or its file, header, or meta tag is edited.
-
-### Retry a failed validation
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322609) in GitLab 14.3.
-> - [Deployed behind the `dast_failed_site_validations` flag](../../../administration/feature_flags.md), enabled by default.
-> - [Feature flag `dast_failed_site_validations` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323961) in GitLab 14.4.
-
-Failed site validation attempts are listed on the **Site profiles** tab of the **Manage profiles**
-page.
-
-To retry a site profile's failed validation:
-
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Security & Compliance > Configuration**.
-1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**.
-1. Select the **Site Profiles** tab.
-1. In the profile's row, select **Retry validation**.
-
-### Revoke a site profile's validation status
-
-WARNING:
-When a site profile's validation status is revoked, all site profiles that share the same URL also
-have their validation status revoked.
-
-To revoke a site profile's validation status:
-
-1. From your project's home page, go to **Security & Compliance > Configuration**.
-1. In the **DAST Profiles** row select **Manage**.
-1. Beside the validated profile, select **Revoke validation**.
-
-The site profile's validation status is revoked.
-
-### Validated site profile headers
-
-The following are code samples of how you can provide the required site profile header in your
-application.
-
-#### Ruby on Rails example for on-demand scan
-
-Here's how you can add a custom header in a Ruby on Rails application:
-
-```ruby
-class DastWebsiteTargetController < ActionController::Base
- def dast_website_target
- response.headers['Gitlab-On-Demand-DAST'] = '0dd79c9a-7b29-4e26-a815-eaaf53fcab1c'
- head :ok
- end
-end
-```
-
-#### Django example for on-demand scan
-
-Here's how you can add a
-[custom header in Django](https://docs.djangoproject.com/en/2.2/ref/request-response/#setting-header-fields):
-
-```python
-class DastWebsiteTargetView(View):
- def head(self, *args, **kwargs):
- response = HttpResponse()
- response['Gitlab-On-Demand-DAST'] = '0dd79c9a-7b29-4e26-a815-eaaf53fcab1c'
-
- return response
-```
-
-#### Node (with Express) example for on-demand scan
-
-Here's how you can add a
-[custom header in Node (with Express)](https://expressjs.com/en/5x/api.html#res.append):
-
-```javascript
-app.get('/dast-website-target', function(req, res) {
- res.append('Gitlab-On-Demand-DAST', '0dd79c9a-7b29-4e26-a815-eaaf53fcab1c')
- res.send('Respond to DAST ping')
-})
-```
-
-## Scanner profile
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222767) in GitLab 13.4.
-> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/225804) in GitLab 13.5: scan mode, AJAX spider, debug messages.
-
-A scanner profile defines the configuration details of a security scanner. A scanner profile can be
-referenced in `.gitlab-ci.yml` and on-demand scans.
-
-A scanner profile contains:
-
-- **Profile name:** A name you give the scanner profile. For example, "Spider_15". While a scanner
- profile is referenced in either `.gitlab-ci.yml` or an on-demand scan, it **cannot** be renamed.
-- **Scan mode:** A passive scan monitors all HTTP messages (requests and responses) sent to the target. An active scan attacks the target to find potential vulnerabilities.
-- **Spider timeout:** The maximum number of minutes allowed for the spider to traverse the site.
-- **Target timeout:** The maximum number of seconds DAST waits for the site to be available before
- starting the scan.
-- **AJAX spider:** Run the AJAX spider, in addition to the traditional spider, to crawl the target site.
-- **Debug messages:** Include debug messages in the DAST console output.
-
-### Create a scanner profile
-
-To create a scanner profile:
-
-1. From your project's home page, go to **Security & Compliance > Configuration**.
-1. In the **DAST Profiles** row, select **Manage**.
-1. Select **New > Scanner Profile**.
-1. Complete the form. For details of each field, see [Scanner profile](#scanner-profile).
-1. Select **Save profile**.
-
-### Edit a scanner profile
-
-If a scanner profile is linked to a security policy, a user cannot edit the profile from this page.
-See [Scan execution policies](../policies/scan-execution-policies.md)
-for more information.
-
-To edit a scanner profile:
-
-1. From your project's home page, go to **Security & Compliance > Configuration**.
-1. In the **DAST Profiles** row, select **Manage**.
-1. Select the **Scanner Profiles** tab.
-1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**.
-1. Edit the form.
-1. Select **Save profile**.
-
-### Delete a scanner profile
-
-If a scanner profile is linked to a security policy, a user cannot delete the profile from this
-page. See [Scan execution policies](../policies/scan-execution-policies.md)
-for more information.
-
-To delete a scanner profile:
-
-1. From your project's home page, go to **Security & Compliance > Configuration**.
-1. In the **DAST Profiles** row, select **Manage**.
-1. Select the **Scanner Profiles** tab.
-1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**.
-1. Select **Delete**.
-
-## Auditing
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1.
-
-The creation, updating, and deletion of DAST profiles, DAST scanner profiles,
-and DAST site profiles are included in the [audit log](../../../administration/audit_events.md).
-
-## Reports
-
-The DAST tool outputs a `gl-dast-report.json` report file containing details of the scan and its results.
-This file is included in the job's artifacts. JSON is the default format, but
-you can output the report in Markdown, HTML, and XML formats. To specify an alternative
-format, use a [CI/CD variable](#available-cicd-variables). You can also use a CI/CD variable
-to configure the job to output the `gl-dast-debug-auth-report.html` file which helps when debugging
-authentication issues.
-
-For details of the report's schema, see the [schema for DAST reports](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/dast-report-format.json). Example reports can be found in the
-[DAST repository](https://gitlab.com/gitlab-org/security-products/dast/-/tree/main/test/end-to-end/expect).
-
-WARNING:
-The JSON report artifacts are not a public API of DAST and their format is expected to change in the
-future.
-
-## Optimizing DAST
-
-By default, DAST downloads all artifacts defined by previous jobs in the pipeline. If
-your DAST job does not rely on `environment_url.txt` to define the URL under test or any other files created
-in previous jobs, we recommend you don't download artifacts. To avoid downloading
-artifacts, add the following to your `.gitlab-ci.yml` file:
-
-```yaml
-dast:
- dependencies: []
-```
diff --git a/doc/user/application_security/dast/proxy-based.md b/doc/user/application_security/dast/proxy-based.md
new file mode 100644
index 00000000000..ec98b809fb7
--- /dev/null
+++ b/doc/user/application_security/dast/proxy-based.md
@@ -0,0 +1,1247 @@
+---
+stage: Secure
+group: Dynamic Analysis
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+type: reference, howto
+---
+
+# DAST proxy-based analyzer **(ULTIMATE)**
+
+The DAST proxy-based analyzer can be added to your [GitLab CI/CD](../../../ci/index.md) pipeline.
+This helps you discover vulnerabilities in web applications that do not use JavaScript heavily. For applications that do,
+please see the [DAST browser-based analyzer](browser_based.md).
+
+WARNING:
+Do not run DAST scans against a production server. Not only can it perform *any* function that
+a user can, such as clicking buttons or submitting forms, but it may also trigger bugs, leading to modification or loss of production data. Only run DAST scans against a test server.
+
+The analyzer uses the [OWASP Zed Attack Proxy](https://www.zaproxy.org/) (ZAP) to scan in two different ways:
+
+- Passive scan only (default). DAST executes
+ [ZAP's Baseline Scan](https://www.zaproxy.org/docs/docker/baseline-scan/) and doesn't
+ actively attack your application.
+- Passive and active (or full) scan. DAST can be [configured](#full-scan) to also perform an active scan
+ to attack your application and produce a more extensive security report. It can be very
+ useful when combined with [Review Apps](../../../ci/review_apps/index.md).
+
+## DAST run options
+
+You can use DAST to examine your web application:
+
+- Automatically, initiated by a merge request.
+- Manually, initiated on demand.
+
+Some of the differences between these run options:
+
+| Automatic scan | On-demand scan |
+|:-----------------------------------------------------------------|:------------------------------|
+| DAST scan is initiated by a merge request. | DAST scan is initiated manually, outside the DevOps life cycle. |
+| CI/CD variables are sourced from `.gitlab-ci.yml`. | CI/CD variables are provided in the UI. |
+| All [DAST CI/CD variables](#available-cicd-variables) available. | Subset of [DAST CI/CD variables](#available-cicd-variables) available. |
+| `DAST.gitlab-ci.yml` template. | `DAST-On-Demand-Scan.gitlab-ci.yml` template. |
+
+### Enable automatic DAST run
+
+To enable DAST to run automatically, either:
+
+- Enable [Auto DAST](../../../topics/autodevops/stages.md#auto-dast) (provided
+ by [Auto DevOps](../../../topics/autodevops/index.md)).
+- [Include the DAST template](#include-the-dast-template) in your existing
+ `.gitlab-ci.yml` file.
+- [Configure DAST using the UI](#configure-dast-using-the-ui).
+
+#### Include the DAST template
+
+> - This template was [updated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62597) to DAST_VERSION: 2 in GitLab 14.0.
+> - This template was [updated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87183) to DAST_VERSION: 3 in GitLab 15.0.
+
+If you want to manually add DAST to your application, the DAST job is defined
+in a CI/CD template file. Updates to the template are provided with GitLab
+upgrades, allowing you to benefit from any improvements and additions.
+
+To include the DAST template:
+
+1. Select the CI/CD template you want to use:
+
+ - [`DAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml):
+ Stable version of the DAST CI/CD template.
+ - [`DAST.latest.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST.latest.gitlab-ci.yml):
+ Latest version of the DAST template. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254325)
+ in GitLab 13.8).
+
+ WARNING:
+ The latest version of the template may include breaking changes. Use the
+ stable template unless you need a feature provided only in the latest template.
+
+ For more information about template versioning, see the
+ [CI/CD documentation](../../../development/cicd/templates.md#latest-version).
+
+1. Add a `dast` stage to your GitLab CI stages configuration:
+
+ ```yaml
+ stages:
+ - dast
+ ```
+
+1. Add the template to GitLab, based on your version of GitLab:
+
+ - In GitLab 11.9 and later, [include](../../../ci/yaml/index.md#includetemplate)
+ the template by adding the following to your `.gitlab-ci.yml` file:
+
+ ```yaml
+ include:
+ - template: <template_file.yml>
+
+ variables:
+ DAST_WEBSITE: https://example.com
+ ```
+
+ - In GitLab 11.8 and earlier, add the contents of the template to your
+ `.gitlab_ci.yml` file.
+
+1. Define the URL to be scanned by DAST by using one of these methods:
+
+ - Set the `DAST_WEBSITE` [CI/CD variable](../../../ci/yaml/index.md#variables).
+ If set, this value takes precedence.
+
+ - Add the URL in an `environment_url.txt` file at the root of your project. This is
+ useful for testing in dynamic environments. To run DAST against an application
+ dynamically created during a GitLab CI/CD pipeline, a job that runs prior to
+ the DAST scan must persist the application's domain in an `environment_url.txt`
+ file. DAST automatically parses the `environment_url.txt` file to find its
+ scan target.
+
+ For example, in a job that runs prior to DAST, you could include code that
+ looks similar to:
+
+ ```yaml
+ script:
+ - echo http://${CI_PROJECT_ID}-${CI_ENVIRONMENT_SLUG}.domain.com > environment_url.txt
+ artifacts:
+ paths: [environment_url.txt]
+ when: always
+ ```
+
+ You can see an example of this in our
+ [Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml)
+ file.
+
+The included template creates a `dast` job in your CI/CD pipeline and scans
+your project's running application for possible vulnerabilities.
+
+The results are saved as a
+[DAST report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportsdast)
+that you can later download and analyze. Due to implementation limitations, we
+always take the latest DAST artifact available. Behind the scenes, the
+[GitLab DAST Docker image](https://gitlab.com/security-products/dast)
+is used to run the tests on the specified URL and scan it for possible
+vulnerabilities.
+
+By default, the DAST template uses the latest major version of the DAST Docker
+image. Using the `DAST_VERSION` variable, you can choose how DAST updates:
+
+- Automatically update DAST with new features and fixes by pinning to a major
+ version (such as `1`).
+- Only update fixes by pinning to a minor version (such as `1.6`).
+- Prevent all updates by pinning to a specific version (such as `1.6.4`).
+
+Find the latest DAST versions on the [Releases](https://gitlab.com/gitlab-org/security-products/dast/-/releases)
+page.
+
+#### Configure DAST using the UI
+
+You can enable or configure DAST settings using the UI. The generated settings are formatted so they
+can be conveniently pasted into the `.gitlab-ci.yml` file.
+
+1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, select **Security & Compliance > Configuration**.
+1. In the **Dynamic Application Security Testing (DAST)** section, select **Enable DAST** or
+ **Configure DAST**.
+1. Select the desired **Scanner profile**, or select **Create scanner profile** and save a
+ scanner profile. For more details, see [scanner profiles](#scanner-profile).
+1. Select the desired **Site profile**, or select **Create site profile** and save a site
+ profile. For more details, see [site profiles](#site-profile).
+1. Select **Generate code snippet**. A modal opens with the YAML snippet corresponding to the
+ options you selected.
+1. Do one of the following:
+ 1. To copy the snippet to your clipboard, select **Copy code only**.
+ 1. To add the snippet to your project's `.gitlab-ci.yml` file, select
+ **Copy code and open `.gitlab-ci.yml` file**. The Pipeline Editor opens.
+ 1. Paste the snippet into the `.gitlab-ci.yml` file.
+ 1. Select the **Lint** tab to confirm the edited `.gitlab-ci.yml` file is valid.
+ 1. Select the **Edit** tab, then select **Commit changes**.
+
+When the snippet is committed to the `.gitlab-ci.yml` file, pipelines include a DAST job.
+
+### API scan
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10928) in GitLab 12.10.
+> - A new DAST API scanning engine was introduced in GitLab 13.10.
+
+Using an API specification as a scan's target is a useful way to seed URLs for scanning an API.
+Vulnerability rules in an API scan are different than those in a normal website scan.
+
+A new DAST API scanning engine is available in GitLab 13.12 and later. For more details, see [DAST API scanning engine](../dast_api). The new scanning engine supports REST, SOAP, GraphQL, and generic APIs using forms, XML, and JSON. Testing can be performed using OpenAPI, Postman Collections, and HTTP Archive (HAR) documents.
+
+The target API instance's base URL is provided by using the `DAST_API_TARGET_URL` variable or an `environment_url.txt` file.
+
+#### Specification format
+
+API scans support OpenAPI V2 and OpenAPI V3 specifications. You can define these specifications using `JSON` or `YAML`.
+
+#### Import API specification from a URL
+
+If your API specification is accessible at a URL, you can pass that URL in directly as the target.
+The specification does not have to be hosted on the same host as the API being tested.
+
+```yaml
+include:
+ - template: DAST-API.gitlab-ci.yml
+
+variables:
+ DAST_API_SPECIFICATION: http://my.api/api-specification.yml
+```
+
+#### Import API specification from a file
+
+If your API specification file is in your repository, you can provide its filename as the target.
+
+```yaml
+dast:
+ variables:
+ GIT_STRATEGY: fetch
+ DAST_API_SPECIFICATION: api-specification.yml
+```
+
+#### Full API scan
+
+API scans support full scanning, which can be enabled by using the `DAST_FULL_SCAN_ENABLED`
+CI/CD variable. Domain validation is not supported for full API scans.
+
+#### Host override
+
+Specifications often define a host, which contains a domain name and a port. The
+host referenced may be different than the host of the API's review instance.
+This can cause incorrect URLs to be imported, or a scan on an incorrect host.
+Use the `DAST_API_HOST_OVERRIDE` CI/CD variable to override these values.
+
+WARNING:
+When using the API host override feature, you cannot use the `$DAST_WEBSITE` variable to override the hostname.
+A host override is _only_ supported when importing the API specification from a URL. Attempts to override the
+host throw an error when the API specification is imported from a file. This is due to a limitation in the
+ZAP OpenAPI extension.
+
+For example, with a OpenAPI V3 specification containing:
+
+```yaml
+servers:
+ - url: https://api.host.com
+```
+
+If the test version of the API is running at `https://api-test.host.com`, then
+the following DAST configuration can be used:
+
+```yaml
+include:
+ - template: DAST-API.gitlab-ci.yml
+
+variables:
+ DAST_API_SPECIFICATION: http://api-test.host.com/api-specification.yml
+ DAST_API_HOST_OVERRIDE: api-test.host.com
+```
+
+#### Authentication using headers
+
+Tokens in request headers are often used as a way to authenticate API requests.
+You can achieve this by using the `DAST_REQUEST_HEADERS` CI/CD variable.
+Headers are applied to every request DAST makes.
+
+```yaml
+include:
+ - template: DAST-API.gitlab-ci.yml
+
+variables:
+ DAST_API_SPECIFICATION: http://api-test.api.com/api-specification.yml
+ DAST_REQUEST_HEADERS: "Authorization: Bearer my.token"
+```
+
+### URL scan
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4.
+> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/273141) in GitLab 13.11.
+
+A URL scan allows you to specify which parts of a website are scanned by DAST.
+
+#### Define the URLs to scan
+
+URLs to scan can be specified by either of the following methods:
+
+- Use `DAST_PATHS_FILE` CI/CD variable to specify the name of a file containing the paths.
+- Use `DAST_PATHS` variable to list the paths.
+
+##### Use `DAST_PATHS_FILE` CI/CD variable
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6.
+
+To define the URLs to scan in a file, create a plain text file with one path per line.
+
+```plaintext
+page1.html
+/page2.html
+category/shoes/page1.html
+```
+
+To scan the URLs in that file, set the CI/CD variable `DAST_PATHS_FILE` to the path of that file.
+The file can be checked into the project repository or generated as an artifact by a job that
+runs before DAST.
+
+By default, DAST scans do not clone the project repository. Instruct the DAST job to clone
+the project by setting `GIT_STRATEGY` to fetch. Give a file path relative to `CI_PROJECT_DIR` to `DAST_PATHS_FILE`.
+
+```yaml
+include:
+ - template: DAST.gitlab-ci.yml
+
+variables:
+ GIT_STRATEGY: fetch
+ DAST_PATHS_FILE: url_file.txt # url_file.txt lives in the root directory of the project
+ DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
+```
+
+##### Use `DAST_PATHS` CI/CD variable
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4.
+
+To specify the paths to scan in a CI/CD variable, add a comma-separated list of the paths to the `DAST_PATHS`
+variable. Note that you can only scan paths of a single host.
+
+```yaml
+include:
+ - template: DAST.gitlab-ci.yml
+
+variables:
+ DAST_PATHS: "/page1.html,/category1/page1.html,/page3.html"
+ DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
+```
+
+When using `DAST_PATHS` and `DAST_PATHS_FILE`, note the following:
+
+- `DAST_WEBSITE` must be defined when using either `DAST_PATHS_FILE` or `DAST_PATHS`. The paths listed in either use `DAST_WEBSITE` to build the URLs to scan
+- Spidering is disabled when `DAST_PATHS` or `DAST_PATHS_FILE` are defined
+- `DAST_PATHS_FILE` and `DAST_PATHS` cannot be used together
+- The `DAST_PATHS` variable has a limit of about 130kb. If you have a list or paths
+ greater than this, use `DAST_PATHS_FILE`.
+
+#### Full Scan
+
+To perform a [full scan](#full-scan) on the listed paths, use the `DAST_FULL_SCAN_ENABLED` CI/CD variable.
+
+## Customize DAST settings
+
+You can customize the behavior of DAST using both CI/CD variables and command-line options. Use of CI/CD
+variables overrides the values contained in the DAST template.
+
+### Customize DAST using CI/CD variables
+
+WARNING:
+Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/index.md#only--except)
+is no longer supported. You must use [`rules`](../../../ci/yaml/index.md#rules) instead.
+
+The DAST settings can be changed through CI/CD variables by using the
+[`variables`](../../../ci/yaml/index.md#variables) parameter in `.gitlab-ci.yml`. For details of
+all DAST CI/CD variables, read [Available CI/CD variables](#available-cicd-variables).
+
+For example:
+
+```yaml
+include:
+ - template: DAST.gitlab-ci.yml
+
+variables:
+ DAST_WEBSITE: https://example.com
+ DAST_SPIDER_MINS: 120
+ DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
+```
+
+Because the template is [evaluated before](../../../ci/yaml/index.md#include) the pipeline
+configuration, the last mention of the variable takes precedence.
+
+#### Enable or disable rules
+
+A complete list of the rules that DAST uses to scan for vulnerabilities can be
+found in the [ZAP documentation](https://www.zaproxy.org/docs/alerts/).
+
+`DAST_EXCLUDE_RULES` disables the rules with the given IDs.
+
+`DAST_ONLY_INCLUDE_RULES` restricts the set of rules used in the scan to
+those with the given IDs.
+
+`DAST_EXCLUDE_RULES` and `DAST_ONLY_INCLUDE_RULES` are mutually exclusive and a
+DAST scan with both configured exits with an error.
+
+By default, several rules are disabled because they either take a long time to
+run or frequently generate false positives. The complete list of disabled rules
+can be found in [`exclude_rules.yml`](https://gitlab.com/gitlab-org/security-products/dast/-/blob/main/src/config/exclude_rules.yml).
+
+The lists for `DAST_EXCLUDE_RULES` and `DAST_ONLY_INCLUDE_RULES` **must** be enclosed in double
+quotes (`"`), otherwise they are interpreted as numeric values.
+
+#### Hide sensitive information
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36332) in GitLab 13.1.
+
+HTTP request and response headers may contain sensitive information, including cookies and
+authorization credentials. By default, the following headers are masked:
+
+- `Authorization`.
+- `Proxy-Authorization`.
+- `Set-Cookie` (values only).
+- `Cookie` (values only).
+
+Using the [`DAST_MASK_HTTP_HEADERS` CI/CD variable](#available-cicd-variables), you can list the
+headers whose values you want masked. For details on how to mask headers, see
+[Customizing the DAST settings](#customize-dast-settings).
+
+#### Use Mutual TLS
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299596) in GitLab 14.8.
+
+Mutual TLS allows a target application server to verify that requests are from a known source. Browser-based scans do not support Mutual TLS.
+
+**Requirements**
+
+- Base64-encoded PKCS12 certificate
+- Password of the base64-encoded PKCS12 certificate
+
+To enable Mutual TLS:
+
+1. If the PKCS12 certificate is not already base64-encoded, convert it to base64 encoding. For security reasons, we recommend encoding the certificate locally, **not** using a web-hosted conversion service. For example, to encode the certificate on either macOS or Linux:
+
+ ```shell
+ base64 <path-to-pkcs12-certificate-file>
+ ```
+
+1. Create a [masked variable](../../../ci/variables/index.md) named `DAST_PKCS12_CERTIFICATE_BASE64` and store the base64-encoded PKCS12 certificate's value in that variable.
+1. Create a masked variable `DAST_PKCS12_PASSWORD` and store the PKCS12 certificate's password in that variable.
+
+#### Available CI/CD variables
+
+These CI/CD variables are specific to DAST. They can be used to customize the behavior of DAST to your requirements.
+
+WARNING:
+All customization of GitLab security scanning tools should be tested in a merge request before
+merging these changes to the default branch. Failure to do so can give unexpected results,
+including a large number of false positives.
+
+| CI/CD variable | Type | Description |
+|:-------------------------------------------------|:--------------|:------------------------------|
+| `DAST_ADVERTISE_SCAN` | boolean | Set to `true` to add a `Via` header to every request sent, advertising that the request was sent as part of a GitLab DAST scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334947) in GitLab 14.1. |
+| `DAST_AGGREGATE_VULNERABILITIES` | boolean | Vulnerability aggregation is set to `true` by default. To disable this feature and see each vulnerability individually set to `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254043) in GitLab 14.0. |
+| `DAST_API_HOST_OVERRIDE` <sup>1</sup> | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080`. |
+| `DAST_API_SPECIFICATION` <sup>1</sup> | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. The variable `DAST_WEBSITE` must be specified if this is omitted. |
+| `DAST_AUTH_REPORT` <sup>2</sup> | boolean | Used in combination with exporting the `gl-dast-debug-auth-report.html` artifact to aid in debugging authentication issues. |
+| `DAST_AUTH_EXCLUDE_URLS` <sup>2</sup> | URLs | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/289959)** in GitLab 14.0. Replaced by `DAST_EXCLUDE_URLS`. The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. |
+| `DAST_AUTH_URL` <sup>1,2</sup> | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Not supported for API scans. Example: `https://login.example.com`. |
+| `DAST_AUTH_VERIFICATION_LOGIN_FORM` <sup>2</sup> | boolean | Verifies successful authentication by checking for the lack of a login form once the login form has been submitted. |
+| `DAST_AUTH_VERIFICATION_SELECTOR` <sup>2</sup> | selector | Verifies successful authentication by checking for presence of a selector once the login form has been submitted. Example: `css:.user-photo`. |
+| `DAST_AUTH_VERIFICATION_URL` <sup>1,2</sup> | URL | A URL only accessible to logged in users that DAST can use to confirm successful authentication. If provided, DAST exits if it cannot access the URL. Example: `"http://example.com/loggedin_page"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. |
+| `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false`. |
+| `DAST_BROWSER_PATH_TO_LOGIN_FORM` <sup>1,2</sup> | selector | Comma-separated list of selectors that are selected prior to attempting to enter `DAST_USERNAME` and `DAST_PASSWORD` into the login form. Example: `"css:.navigation-menu,css:.login-menu-item"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326633) in GitLab 14.1. |
+| `DAST_DEBUG` <sup>1</sup> | boolean | Enable debug message output. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). For example, `HTTP Parameter Override` has a rule ID of `10026`. Cannot be used when `DAST_ONLY_INCLUDE_RULES` is set. **Note:** In earlier versions of GitLab the excluded rules were executed but vulnerabilities they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. |
+| `DAST_EXCLUDE_URLS` <sup>1,2</sup> | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. Example, `http://example.com/sign-out`. |
+| `DAST_FIRST_SUBMIT_FIELD` <sup>2</sup> | string | The `id` or `name` of the element that when selected submits the username form of a multi-page login process. For example, `css:button[type='user-submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. |
+| `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | boolean | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/293595)** in GitLab 14.0. Set to `true` to require domain validation when running DAST full scans. Not supported for API scans. Default: `false` |
+| `DAST_FULL_SCAN_ENABLED` <sup>1</sup> | boolean | Set to `true` to run a [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of a [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Default: `false` |
+| `DAST_HTML_REPORT` | string | The filename of the HTML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_MARKDOWN_REPORT` | string | The filename of the Markdown report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_MASK_HTTP_HEADERS` | string | Comma-separated list of request and response headers to be masked (GitLab 13.1). Must contain **all** headers to be masked. Refer to [list of headers that are masked by default](#hide-sensitive-information). |
+| `DAST_MAX_URLS_PER_VULNERABILITY` | number | The maximum number of URLs reported for a single vulnerability. `DAST_MAX_URLS_PER_VULNERABILITY` is set to `50` by default. To list all the URLs set to `0`. [Introduced](https://gitlab.com/gitlab-org/security-products/dast/-/merge_requests/433) in GitLab 13.12. |
+| `DAST_ONLY_INCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to configure the scan to run only them. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). Cannot be used when `DAST_EXCLUDE_RULES` is set. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250651) in GitLab 13.12. |
+| `DAST_PASSWORD` <sup>1,2</sup> | string | The password to authenticate to in the website. Example: `P@55w0rd!` |
+| `DAST_PASSWORD_FIELD` <sup>1,2</sup> | string | The selector of password field at the sign-in HTML form. Example: `id:password` |
+| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4. |
+| `DAST_PATHS_FILE` | string | The file path containing the paths within `DAST_WEBSITE` to scan. The file must be plain text with one path per line. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6. |
+| `DAST_PKCS12_CERTIFICATE_BASE64` | string | The PKCS12 certificate used for sites that require Mutual TLS. Must be encoded as base64 text. |
+| `DAST_PKCS12_PASSWORD` | string | The password of the certificate used in `DAST_PKCS12_CERTIFICATE_BASE64`. |
+| `DAST_REQUEST_HEADERS` <sup>1</sup> | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` |
+| `DAST_SKIP_TARGET_CHECK` | boolean | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229067) in GitLab 13.8. |
+| `DAST_SPIDER_MINS` <sup>1</sup> | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_SPIDER_START_AT_HOST` | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` is reset to `http://test.site` before scan. Default: `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258805) in GitLab 13.6. |
+| `DAST_SUBMIT_FIELD` <sup>2</sup> | string | The `id` or `name` of the element that when selected submits the login form or the password form of a multi-page login process. For example, `css:button[type='submit']`. [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9894) in GitLab 12.4. |
+| `DAST_TARGET_AVAILABILITY_TIMEOUT` <sup>1</sup> | number | Time limit in seconds to wait for target availability. |
+| `DAST_USE_AJAX_SPIDER` <sup>1</sup> | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_USERNAME` <sup>1,2</sup> | string | The username to authenticate to in the website. Example: `admin` |
+| `DAST_USERNAME_FIELD` <sup>1,2</sup> | string | The selector of username field at the sign-in HTML form. Example: `name:username` |
+| `DAST_XML_REPORT` | string | The filename of the XML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_WEBSITE` <sup>1</sup> | URL | The URL of the website to scan. The variable `DAST_API_SPECIFICATION` must be specified if this is omitted. |
+| `DAST_ZAP_CLI_OPTIONS` | string | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_ZAP_LOG_CONFIGURATION` | string | Set to a semicolon-separated list of additional log4j properties for the ZAP Server. Example: `logger.httpsender.name=org.parosproxy.paros.network.HttpSender;logger.httpsender.level=debug;logger.sitemap.name=org.parosproxy.paros.model.SiteMap;logger.sitemap.level=debug;` |
+| `SECURE_ANALYZERS_PREFIX` | URL | Set the Docker registry base address from which to download the analyzer. |
+
+1. Available to an on-demand DAST scan.
+1. Used for authentication.
+
+### Customize DAST using command-line options
+
+Not all DAST configuration is available via CI/CD variables. To find out all
+possible options, run the following configuration.
+Available command-line options are printed to the job log:
+
+```yaml
+include:
+ template: DAST.gitlab-ci.yml
+
+dast:
+ script:
+ - /analyze --help
+```
+
+You must then overwrite the `script` command to pass in the appropriate
+argument. For example, vulnerability definitions in alpha can be included with
+`-a`. The following configuration includes those definitions:
+
+```yaml
+include:
+ template: DAST.gitlab-ci.yml
+
+dast:
+ script:
+ - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)}
+ - /analyze -a -t $DAST_WEBSITE
+```
+
+### Custom ZAProxy configuration
+
+The ZAProxy server contains many [useful configurable values](https://gitlab.com/gitlab-org/gitlab/-/issues/36437#note_245801885).
+Many key/values for `-config` remain undocumented, but there is an untested list of
+[possible keys](https://gitlab.com/gitlab-org/gitlab/-/issues/36437#note_244981023).
+Note that these options are not supported by DAST, and may break the DAST scan
+when used. An example of how to rewrite the Authorization header value with `TOKEN` follows:
+
+```yaml
+include:
+ template: DAST.gitlab-ci.yml
+
+variables:
+ DAST_ZAP_CLI_OPTIONS: "-config replacer.full_list(0).description=auth -config replacer.full_list(0).enabled=true -config replacer.full_list(0).matchtype=REQ_HEADER -config replacer.full_list(0).matchstr=Authorization -config replacer.full_list(0).regex=false -config replacer.full_list(0).replacement=TOKEN"
+```
+
+## Authentication
+
+NOTE:
+We highly recommend you configure the scanner to authenticate to the application. If you don't, it cannot check most of the application for security risks, as most
+of your application is likely not accessible without authentication. We also recommend
+you periodically confirm the scanner's authentication is still working, as this tends to break over
+time due to authentication changes to the application.
+
+Create masked CI/CD variables to pass the credentials that DAST uses.
+To create masked variables for the username and password, see [Create a custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables).
+The key of the username variable must be `DAST_USERNAME`,
+and the key of the password variable must be `DAST_PASSWORD`.
+
+After DAST has authenticated with the application, all cookies are collected from the web browser.
+For each cookie a matching session token is created for use by ZAP. This ensures ZAP is recognized
+by the application as correctly authenticated.
+
+Authentication supports single form logins, multi-step login forms, and authenticating to URLs outside of the configured target URL.
+
+WARNING:
+**Never** run an authenticated scan against a production server. When an authenticated
+scan is run, it may perform *any* function that the authenticated user can. This
+includes actions like modifying and deleting data, submitting forms, and following links.
+Only run an authenticated scan against a test server.
+
+### SSO
+
+DAST can authenticate to websites making use of SSO, with the following restrictions:
+
+- DAST cannot bypass a CAPTCHA if the authentication flow includes one.
+- DAST cannot handle multi-factor authentication like one-time passwords (OTP) by using SMS or authenticator apps.
+- DAST must get a cookie, or a local or session storage, with a sufficiently random value.
+
+The [authentication debug output](#configure-the-authentication-debug-output) can be helpful for troubleshooting SSO authentication
+with DAST.
+
+### Log in using automatic detection of the login form
+
+By providing a `DAST_USERNAME`, `DAST_PASSWORD`, and `DAST_AUTH_URL`, DAST attempts to authenticate to the
+target application by locating the login form based on a determination about whether or not the form contains username or password fields.
+
+Automatic detection is "best-effort", and depending on the application being scanned may provide either a resilient login experience or one that fails to authenticate the user.
+
+Login process:
+
+1. The `DAST_AUTH_URL` is loaded into the browser, and any forms on the page are located.
+ 1. If a form contains a username and password field, `DAST_USERNAME` and `DAST_PASSWORD` is inputted into the respective fields, the form submit button is selected and the user is logged in.
+ 1. If a form contains only a username field, it is assumed that the login form is multi-step.
+ 1. The `DAST_USERNAME` is inputted into the username field and the form submit button is selected.
+ 1. The subsequent pages loads where it is expected that a form exists and contains a password field. If found, `DAST_PASSWORD` is inputted, form submit button is selected and the user is logged in.
+
+### Log in using explicit selection of the login form
+
+By providing a `DAST_USERNAME_FIELD`, `DAST_PASSWORD_FIELD`, and `DAST_SUBMIT_FIELD`, in addition to the fields required for automatic login,
+DAST attempts to authenticate to the target application by locating the login form based on the selectors provided.
+Most applications benefit from this approach to authentication.
+
+Login process:
+
+1. The `DAST_AUTH_URL` is loaded into the browser, and any forms on the page are located.
+ 1. If the `DAST_FIRST_SUBMIT_FIELD` is not defined, then `DAST_USERNAME` is inputted into `DAST_USERNAME_FIELD`, `DAST_PASSWORD` is inputted into `DAST_PASSWORD_FIELD`, `DAST_SUBMIT_FIELD` is selected and the user is logged in.
+ 1. If the `DAST_FIRST_SUBMIT_FIELD` is defined, then it is assumed that the login form is multi-step.
+ 1. The `DAST_USERNAME` is inputted into the `DAST_USERNAME_FIELD` field and the `DAST_FIRST_SUBMIT_FIELD` is selected.
+ 1. The subsequent pages loads where the `DAST_PASSWORD` is inputted into the `DAST_PASSWORD_FIELD` field, the `DAST_SUBMIT_FIELD` is selected and the user is logged in.
+
+### Verifying successful login
+
+Once the login form has been submitted, DAST determines if the login was successful. Unsuccessful attempts at authentication cause the scan to halt.
+
+Following the submission of the login form, authentication is determined to be unsuccessful when:
+
+- A `400` or `500` series HTTP response status code is returned.
+- A new cookie/browser storage value determined to be sufficiently random has not been set.
+
+In addition to these checks, the user can configure their own verification checks.
+Each of the following checks can be used in conjunction with one another, if none are configured by default the presence of a login form is checked.
+
+#### Verifying based on the URL
+
+When `DAST_AUTH_VERIFICATION_URL` is configured, the URL displayed in the browser tab post login form submission is directly compared to the URL in the CI/CD variable.
+If these are not exactly the same, authentication is deemed to be unsuccessful.
+
+For example:
+
+```yaml
+include:
+ - template: DAST.gitlab-ci.yml
+
+dast:
+ variables:
+ DAST_WEBSITE: "https://example.com"
+ DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
+ ...
+ DAST_AUTH_VERIFICATION_URL: "https://example.com/user/welcome"
+```
+
+#### Verify based on presence of an element
+
+When `DAST_AUTH_VERIFICATION_SELECTOR` is configured, the page displayed in the browser tab is searched for an element described by the selector in the CI/CD variable.
+If no element is found, authentication is deemed to be unsuccessful.
+
+For example:
+
+```yaml
+include:
+ - template: DAST.gitlab-ci.yml
+
+dast:
+ variables:
+ DAST_WEBSITE: "https://example.com"
+ DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
+ ...
+ DAST_AUTH_VERIFICATION_SELECTOR: "css:.welcome-user"
+```
+
+#### Verify based on presence of a login form
+
+When `DAST_AUTH_VERIFICATION_LOGIN_FORM` is configured, the page displayed in the browser tab is searched for a form that is detected to be a login form.
+If any such form is found, authentication is deemed to be unsuccessful.
+
+For example:
+
+```yaml
+include:
+ - template: DAST.gitlab-ci.yml
+
+dast:
+ variables:
+ DAST_WEBSITE: "https://example.com"
+ DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
+ ...
+ DAST_AUTH_VERIFICATION_LOGIN_FORM: "true"
+```
+
+### View the login form
+
+Many web applications show the user the login form in a pop-up (modal) window.
+For these applications, navigating to the form requires both:
+
+- A starting URL.
+- A list of elements to select to display the modal window.
+
+When `DAST_BROWSER_PATH_TO_LOGIN_FORM` is present, like in this example:
+
+```yaml
+include:
+ - template: DAST.gitlab-ci.yml
+
+dast:
+ variables:
+ DAST_WEBSITE: "https://my.site.com"
+ DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
+ ...
+ DAST_AUTH_URL: "https://my.site.com/admin"
+ DAST_BROWSER_PATH_TO_LOGIN_FORM: "css:.navigation-menu,css:.login-menu-item"
+```
+
+DAST performs these actions:
+
+1. Load the `DAST_AUTH_URL` page, such as `https://my.site.com/admin`.
+1. After the page loads, DAST selects elements found by the selectors described
+ in `DAST_BROWSER_PATH_TO_LOGIN_FORM`. This example opens the navigation menu
+ and selects the login menu, to display the login modal window.
+1. To continue the authentication process, DAST fills in the username and password
+ on the login form.
+
+### Configure the authentication debug output
+
+It is often difficult to understand the cause of an authentication failure when running DAST in a CI/CD pipeline.
+To assist users in debugging authentication issues, a debug report can be generated and saved as a job artifact.
+This HTML report contains all steps made during the login process, along with HTTP requests and responses, the Document Object Model (DOM) and screenshots.
+
+![dast-auth-report](img/dast_auth_report.jpg)
+
+An example configuration where the authentication debug report is exported may look like the following:
+
+```yaml
+dast:
+ variables:
+ DAST_WEBSITE: "https://example.com"
+ DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler
+ ...
+ DAST_AUTH_REPORT: "true"
+ artifacts:
+ paths: [gl-dast-debug-auth-report.html]
+ when: always
+```
+
+### Selectors
+
+Selectors are used by CI/CD variables to specify the location of an element displayed on a page in a browser.
+Selectors have the format `type`:`search string`. The crawler searches for the selector using the search string based on the type.
+
+| Selector type | Example | Description |
+| ------------- | ---------------------------------- | ----------- |
+| `css` | `css:.password-field` | Searches for a HTML element having the supplied CSS selector. Selectors should be as specific as possible for performance reasons. |
+| `id` | `id:element` | Searches for an HTML element with the provided element ID. |
+| `name` | `name:element` | Searches for an HTML element with the provided element name. |
+| `xpath` | `xpath://input[@id="my-button"]/a` | Searches for a HTML element with the provided XPath. Note that XPath searches are expected to be less performant than other searches. |
+| None provided | `a.click-me` | Defaults to searching using a CSS selector. |
+
+#### Find selectors with Google Chrome
+
+Chrome DevTools element selector tool is an effective way to find a selector.
+
+1. Open Chrome and navigate to the page where you would like to find a selector, for example, the login page for your site.
+1. Open the `Elements` tab in Chrome DevTools with the keyboard shortcut `Command + Shift + c` in macOS or `Ctrl + Shift + c` in Windows.
+1. Select the `Select an element in the page to select it` tool.
+ ![search-elements](img/dast_auth_browser_scan_search_elements.png)
+1. Select the field on your page that you would like to know the selector for.
+1. Once the tool is active, highlight a field you wish to view the details of.
+ ![highlight](img/dast_auth_browser_scan_highlight.png)
+1. Once highlighted, you can see the element's details, including attributes that would make a good candidate for a selector.
+
+In this example, the `id="user_login"` appears to be a good candidate. You can use this as a selector as the DAST username field by setting
+`DAST_USERNAME_FIELD: "id:user_login"`.
+
+#### Choose the right selector
+
+Judicious choice of selector leads to a scan that is resilient to the application changing.
+
+In order of preference, it is recommended to choose as selectors:
+
+- `id` fields. These are generally unique on a page, and rarely change.
+- `name` fields. These are generally unique on a page, and rarely change.
+- `class` values specific to the field, such as the selector `"css:.username"` for the `username` class on the username field.
+- Presence of field specific data attributes, such as the selector, `"css:[data-username]"` when the `data-username` field has any value on the username field.
+- Multiple `class` hierarchy values, such as the selector `"css:.login-form .username"` when there are multiple elements with class `username` but only one nested inside the element with the class `login-form`.
+
+When using selectors to locate specific fields we recommend you avoid searching on:
+
+- Any `id`, `name`, `attribute`, `class` or `value` that is dynamically generated.
+- Generic class names, such as `column-10` and `dark-grey`.
+- XPath searches as they are less performant than other selector searches.
+- Unscoped searches, such as those beginning with `css:*` and `xpath://*`.
+
+### Bleeding-edge vulnerability definitions
+
+ZAP first creates rules in the `alpha` class. After a testing period with
+the community, they are promoted to `beta`. DAST uses `beta` definitions by
+default. To request `alpha` definitions, use the
+`DAST_INCLUDE_ALPHA_VULNERABILITIES` CI/CD variable as shown in the
+following configuration:
+
+```yaml
+include:
+ template: DAST.gitlab-ci.yml
+
+variables:
+ DAST_INCLUDE_ALPHA_VULNERABILITIES: "true"
+```
+
+### Cloning the project's repository
+
+The DAST job does not require the project's repository to be present when running, so by default
+[`GIT_STRATEGY`](../../../ci/runners/configure_runners.md#git-strategy) is set to `none`.
+
+## On-demand scans
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2.
+> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3.
+> - The saved scans feature was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5100) in GitLab 13.9.
+> - The option to select a branch was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4847) in GitLab 13.10.
+> - DAST branch selection [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/322672) in GitLab 13.11.
+> - Auditing for DAST profile management was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1.
+
+An on-demand DAST scan runs outside the DevOps life cycle. Changes in your repository don't trigger
+the scan. You must either start it manually, or schedule it to run.
+
+An on-demand DAST scan:
+
+- Can run a specific combination of a [site profile](#site-profile) and a
+ [scanner profile](#scanner-profile).
+- Is associated with your project's default branch.
+- Is saved on creation so it can be run later.
+
+### On-demand scan modes
+
+An on-demand scan can be run in active or passive mode:
+
+- _Passive mode_ is the default and runs a ZAP Baseline Scan.
+- _Active mode_ runs a ZAP Full Scan which is potentially harmful to the site being scanned. To
+ minimize the risk of accidental damage, running an active scan requires a [validated site profile](#site-profile-validation).
+
+### View on-demand DAST scans
+
+To view running completed and scheduled on-demand DAST scans for a project, go to
+**Security & Compliance > On-demand Scans** in the left sidebar.
+
+- To view both running and completed scans, select **All**.
+- To view running scans only, select **Running**.
+- To view finished scans, select **Finished**. A finished scan is a scan that either succeeded,
+ failed, or was canceled.
+- To view scheduled scans, select **Scheduled**. It shows on-demand scans that have a schedule
+ set up. Those are _not_ included in the **All** tab.
+- To view saved on-demand scan profiles, select **Scan library**.
+ Those are _not_ included in the **All** tab.
+
+#### Cancel an on-demand scan
+
+To cancel a pending or running on-demand scan, select **Cancel** (**{cancel}**) in the
+on-demand scans list.
+
+#### Retry an on-demand scan
+
+To retry a scan that failed or succeeded with warnings, select **Retry** (**{retry}**) in the
+on-demand scans list.
+
+#### View an on-demand scan's results
+
+To view a finished scan's results, select **View results** in the on-demand scans list.
+
+#### Edit an on-demand scan
+
+To edit an on-demand scan's settings, select **Edit** (**{pencil}**) in the **Scheduled** tab.
+
+### Run an on-demand DAST scan
+
+Prerequisites:
+
+- You must have permission to run an on-demand DAST scan against a protected branch. The default
+ branch is automatically protected. For more information, read
+ [Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches).
+- A [scanner profile](#create-a-scanner-profile).
+- A [site profile](#create-a-site-profile).
+- If you are running an active scan the site profile must have been [validated](#validate-a-site-profile).
+
+You can run an on-demand scan immediately, once at a scheduled date and time or at a specified
+frequency:
+
+- Every day
+- Every week
+- Every month
+- Every 3 months
+- Every 6 months
+- Every year
+
+To run an on-demand scan immediately, either:
+
+- [Create and run an on-demand scan immediately](#create-and-run-an-on-demand-scan-immediately).
+- [Run a previously saved on-demand scan](#run-a-saved-on-demand-scan).
+
+To run an on-demand scan either at a scheduled date or frequency, read
+[Schedule an on-demand scan](#schedule-an-on-demand-scan).
+
+#### Create and run an on-demand scan immediately
+
+1. From your project's home page, go to **Security & Compliance > On-demand Scans** in the left
+ sidebar.
+1. Select **New scan**.
+1. Complete the **Scan name** and **Description** fields.
+1. In GitLab 13.10 and later, select the desired branch from the **Branch** dropdown list.
+1. In **Scanner profile**, select a scanner profile from the dropdown list.
+1. In **Site profile**, select a site profile from the dropdown list.
+1. To run the on-demand scan immediately, select **Save and run scan**. Otherwise, select
+ **Save scan** to [run](#run-a-saved-on-demand-scan) it later.
+
+The on-demand DAST scan runs and the project's dashboard shows the results.
+
+#### Run a saved on-demand scan
+
+To run a saved on-demand scan:
+
+1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, select **Security & Compliance > On-demand Scans**.
+1. Select the **Scan library** tab.
+1. In the scan's row, select **Run scan**.
+
+ If the branch saved in the scan no longer exists, you must first
+ [edit the scan](#edit-an-on-demand-scan), select a new branch, and save the edited scan.
+
+The on-demand DAST scan runs, and the project's dashboard shows the results.
+
+#### Schedule an on-demand scan
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.3. [Deployed behind the `dast_on_demand_scans_scheduler` flag](../../../administration/feature_flags.md), disabled by default.
+> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.4.
+> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.4.
+> - [Feature flag `dast_on_demand_scans_scheduler` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.5.
+
+To schedule a scan:
+
+1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, select **Security & Compliance > On-demand Scans**.
+1. Select **New scan**.
+1. Complete the **Scan name** and **Description** text boxes.
+1. In GitLab 13.10 and later, from the **Branch** dropdown list, select the desired branch.
+1. In the **Scanner profile** section, from the dropdown list, select a scanner profile.
+1. In the **Site profile** section, from the dropdown list, select a site profile.
+1. Select **Schedule scan**.
+1. In the **Start time** section, select a time zone, date, and time.
+1. From the **Repeats** dropdown list, select your desired frequency:
+ - To run the scan once, select **Never**.
+ - For a recurring scan, select any other option.
+1. To run the on-demand scan immediately, select **Save and run scan**. To [run](#run-a-saved-on-demand-scan) it according to the schedule you set, select
+ **Save scan**.
+
+#### List saved on-demand scans
+
+To list saved on-demand scans:
+
+1. From your project's home page, go to **Security & Compliance > On-demand Scans**.
+1. Select the **Scan library** tab.
+
+#### View details of an on-demand scan
+
+To view details of an on-demand scan:
+
+1. From your project's home page, go to **Security & Compliance > On-demand Scans**.
+1. Select the **Scan library** tab.
+1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Edit**.
+
+#### Edit an on-demand scan
+
+To edit an on-demand scan:
+
+1. From your project's home page, go to **Security & Compliance > On-demand Scans**.
+1. Select the **Scan library** tab.
+1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Edit**.
+1. Edit the form.
+1. Select **Save scan**.
+
+#### Delete an on-demand scan
+
+To delete an on-demand scan:
+
+1. From your project's home page, go to **Security & Compliance > On-demand Scans**.
+1. Select the **Scan library** tab.
+1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Delete**.
+1. Select **Delete** to confirm the deletion.
+
+## Site profile
+
+> - Scan method [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/345837) in GitLab 15.6.
+> - File URL [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/345837) in GitLab 15.6.
+
+A site profile defines the attributes and configuration details of the deployed application,
+website, or API to be scanned by DAST. A site profile can be referenced in `.gitlab-ci.yml` and
+on-demand scans.
+
+A site profile contains:
+
+- **Profile name**: A name you assign to the site to be scanned. While a site profile is referenced
+ in either `.gitlab-ci.yml` or an on-demand scan, it **cannot** be renamed.
+- **Site type**: The type of target to be scanned, either website or API scan.
+- **Target URL**: The URL that DAST runs against.
+- **Excluded URLs**: A comma-separated list of URLs to exclude from the scan.
+- **Request headers**: A comma-separated list of HTTP request headers, including names and values. These headers are added to every request made by DAST.
+- **Authentication**:
+ - **Authenticated URL**: The URL of the page containing the sign-in HTML form on the target website. The username and password are submitted with the login form to create an authenticated scan.
+ - **Username**: The username used to authenticate to the website.
+ - **Password**: The password used to authenticate to the website.
+ - **Username form field**: The name of username field at the sign-in HTML form.
+ - **Password form field**: The name of password field at the sign-in HTML form.
+ - **Submit form field**: The `id` or `name` of the element that when selected submits the sign-in HTML form.
+
+- **Scan method**: A type of method to perform API testing. The supported methods are OpenAPI, Postman Collections, and HTTP Archive (HAR) documents.
+- **File URL**: The URL of the OpenAPI, Postman Collection, or HTTP Archive file.
+
+When an API site type is selected, a [host override](#host-override) is used to ensure the API being scanned is on the same host as the target. This is done to reduce the risk of running an active scan against the wrong API.
+
+When configured, request headers and password fields are encrypted using [`aes-256-gcm`](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) before being stored in the database.
+This data can only be read and decrypted with a valid secrets file.
+
+### Site profile validation
+
+> - Site profile validation [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233020) in GitLab 13.8.
+> - Meta tag validation [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6460) in GitLab 14.2.
+
+Site profile validation reduces the risk of running an active scan against the wrong website. A site
+must be validated before an active scan can run against it. The site validation methods are as
+follows:
+
+- _Text file validation_ requires a text file be uploaded to the target site. The text file is
+ allocated a name and content that is unique to the project. The validation process checks the
+ file's content.
+- _Header validation_ requires the header `Gitlab-On-Demand-DAST` be added to the target site,
+ with a value unique to the project. The validation process checks that the header is present, and
+ checks its value.
+- _Meta tag validation_ requires the meta tag named `gitlab-dast-validation` be added to the target site,
+ with a value unique to the project. Make sure it's added to the `<head>` section of the page. The validation process checks that the meta tag is present, and
+ checks its value.
+
+All these methods are equivalent in functionality. Use whichever is feasible.
+
+In [GitLab 14.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/324990), site profile
+validation happens in a CI job using the [GitLab Runner](../../../ci/runners/index.md).
+
+### Create a site profile
+
+To create a site profile:
+
+1. From your project's home page, go to **Security & Compliance > Configuration**.
+1. Select **Manage** in the **DAST Profiles** row.
+1. Select **New > Site Profile**.
+1. Complete the fields then select **Save profile**.
+
+The site profile is created.
+
+### Edit a site profile
+
+If a site profile is linked to a security policy, a user cannot edit the profile from this page. See
+[Scan execution policies](../policies/scan-execution-policies.md)
+for more information.
+
+When a validated site profile's file, header, or meta tag is edited, the site's
+[validation status](#site-profile-validation) is revoked.
+
+To edit a site profile:
+
+1. From your project's home page, go to **Security & Compliance > Configuration**.
+1. In the **DAST Profiles** row select **Manage**.
+1. Select the **Site Profiles** tab.
+1. In the profile's row select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**.
+1. Edit the fields then select **Save profile**.
+
+### Delete a site profile
+
+If a site profile is linked to a security policy, a user cannot delete the profile from this page.
+See [Scan execution policies](../policies/scan-execution-policies.md)
+for more information.
+
+To delete a site profile:
+
+1. From your project's home page, go to **Security & Compliance > Configuration**.
+1. In the **DAST Profiles** row select **Manage**.
+1. Select the **Site Profiles** tab.
+1. In the profile's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**.
+1. Select **Delete** to confirm the deletion.
+
+### Validate a site profile
+
+Validating a site is required to run an active scan.
+
+To validate a site profile:
+
+1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, select **Security & Compliance > Configuration**.
+1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**.
+1. Select the **Site Profiles** tab.
+1. In the profile's row, select **Validate**.
+1. Select the validation method.
+ 1. For **Text file validation**:
+ 1. Download the validation file listed in **Step 2**.
+ 1. Upload the validation file to the host, to the location in **Step 3** or any location you
+ prefer.
+ 1. If required, edit the file location in **Step 3**.
+ 1. Select **Validate**.
+ 1. For **Header validation**:
+ 1. Select the clipboard icon in **Step 2**.
+ 1. Edit the header of the site to validate, and paste the clipboard content.
+ 1. Select the input field in **Step 3** and enter the location of the header.
+ 1. Select **Validate**.
+ 1. For **Meta tag validation**:
+ 1. Select the clipboard icon in **Step 2**.
+ 1. Edit the content of the site to validate, and paste the clipboard content.
+ 1. Select the input field in **Step 3** and enter the location of the meta tag.
+ 1. Select **Validate**.
+
+The site is validated and an active scan can run against it. A site profile's validation status is
+revoked only when it's revoked manually, or its file, header, or meta tag is edited.
+
+### Retry a failed validation
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322609) in GitLab 14.3.
+> - [Deployed behind the `dast_failed_site_validations` flag](../../../administration/feature_flags.md), enabled by default.
+> - [Feature flag `dast_failed_site_validations` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323961) in GitLab 14.4.
+
+Failed site validation attempts are listed on the **Site profiles** tab of the **Manage profiles**
+page.
+
+To retry a site profile's failed validation:
+
+1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, select **Security & Compliance > Configuration**.
+1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**.
+1. Select the **Site Profiles** tab.
+1. In the profile's row, select **Retry validation**.
+
+### Revoke a site profile's validation status
+
+WARNING:
+When a site profile's validation status is revoked, all site profiles that share the same URL also
+have their validation status revoked.
+
+To revoke a site profile's validation status:
+
+1. From your project's home page, go to **Security & Compliance > Configuration**.
+1. In the **DAST Profiles** row select **Manage**.
+1. Beside the validated profile, select **Revoke validation**.
+
+The site profile's validation status is revoked.
+
+### Validated site profile headers
+
+The following are code samples of how you can provide the required site profile header in your
+application.
+
+#### Ruby on Rails example for on-demand scan
+
+Here's how you can add a custom header in a Ruby on Rails application:
+
+```ruby
+class DastWebsiteTargetController < ActionController::Base
+ def dast_website_target
+ response.headers['Gitlab-On-Demand-DAST'] = '0dd79c9a-7b29-4e26-a815-eaaf53fcab1c'
+ head :ok
+ end
+end
+```
+
+#### Django example for on-demand scan
+
+Here's how you can add a
+[custom header in Django](https://docs.djangoproject.com/en/2.2/ref/request-response/#setting-header-fields):
+
+```python
+class DastWebsiteTargetView(View):
+ def head(self, *args, **kwargs):
+ response = HttpResponse()
+ response['Gitlab-On-Demand-DAST'] = '0dd79c9a-7b29-4e26-a815-eaaf53fcab1c'
+
+ return response
+```
+
+#### Node (with Express) example for on-demand scan
+
+Here's how you can add a
+[custom header in Node (with Express)](https://expressjs.com/en/5x/api.html#res.append):
+
+```javascript
+app.get('/dast-website-target', function(req, res) {
+ res.append('Gitlab-On-Demand-DAST', '0dd79c9a-7b29-4e26-a815-eaaf53fcab1c')
+ res.send('Respond to DAST ping')
+})
+```
+
+## Scanner profile
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222767) in GitLab 13.4.
+> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/225804) in GitLab 13.5: scan mode, AJAX spider, debug messages.
+
+A scanner profile defines the configuration details of a security scanner. A scanner profile can be
+referenced in `.gitlab-ci.yml` and on-demand scans.
+
+A scanner profile contains:
+
+- **Profile name:** A name you give the scanner profile. For example, "Spider_15". While a scanner
+ profile is referenced in either `.gitlab-ci.yml` or an on-demand scan, it **cannot** be renamed.
+- **Scan mode:** A passive scan monitors all HTTP messages (requests and responses) sent to the target. An active scan attacks the target to find potential vulnerabilities.
+- **Spider timeout:** The maximum number of minutes allowed for the spider to traverse the site.
+- **Target timeout:** The maximum number of seconds DAST waits for the site to be available before
+ starting the scan.
+- **AJAX spider:** Run the AJAX spider, in addition to the traditional spider, to crawl the target site.
+- **Debug messages:** Include debug messages in the DAST console output.
+
+### Create a scanner profile
+
+To create a scanner profile:
+
+1. From your project's home page, go to **Security & Compliance > Configuration**.
+1. In the **DAST Profiles** row, select **Manage**.
+1. Select **New > Scanner Profile**.
+1. Complete the form. For details of each field, see [Scanner profile](#scanner-profile).
+1. Select **Save profile**.
+
+### Edit a scanner profile
+
+If a scanner profile is linked to a security policy, a user cannot edit the profile from this page.
+See [Scan execution policies](../policies/scan-execution-policies.md)
+for more information.
+
+To edit a scanner profile:
+
+1. From your project's home page, go to **Security & Compliance > Configuration**.
+1. In the **DAST Profiles** row, select **Manage**.
+1. Select the **Scanner Profiles** tab.
+1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**.
+1. Edit the form.
+1. Select **Save profile**.
+
+### Delete a scanner profile
+
+If a scanner profile is linked to a security policy, a user cannot delete the profile from this
+page. See [Scan execution policies](../policies/scan-execution-policies.md)
+for more information.
+
+To delete a scanner profile:
+
+1. From your project's home page, go to **Security & Compliance > Configuration**.
+1. In the **DAST Profiles** row, select **Manage**.
+1. Select the **Scanner Profiles** tab.
+1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**.
+1. Select **Delete**.
+
+## Auditing
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1.
+
+The creation, updating, and deletion of DAST profiles, DAST scanner profiles,
+and DAST site profiles are included in the [audit log](../../../administration/audit_events.md).
+
+## Reports
+
+The DAST tool outputs a `gl-dast-report.json` report file containing details of the scan and its results.
+This file is included in the job's artifacts. JSON is the default format, but
+you can output the report in Markdown, HTML, and XML formats. To specify an alternative
+format, use a [CI/CD variable](#available-cicd-variables). You can also use a CI/CD variable
+to configure the job to output the `gl-dast-debug-auth-report.html` file which helps when debugging
+authentication issues.
+
+For details of the report's schema, see the [schema for DAST reports](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/dast-report-format.json). Example reports can be found in the
+[DAST repository](https://gitlab.com/gitlab-org/security-products/dast/-/tree/main/test/end-to-end/expect).
+
+WARNING:
+The JSON report artifacts are not a public API of DAST and their format is expected to change in the
+future.
diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md
index eae32f789b8..d77be0f0ca9 100644
--- a/doc/user/application_security/dast_api/index.md
+++ b/doc/user/application_security/dast_api/index.md
@@ -7,20 +7,19 @@ type: reference, howto
# DAST API **(ULTIMATE)**
-You can add dynamic application security testing (DAST) of web APIs to your
-[GitLab CI/CD](../../../ci/index.md) pipelines. This helps you discover bugs and potential security
-issues that other QA processes may miss.
+> DAST API analyzer [became the default analyzer for on-demand DAST API scans](https://gitlab.com/groups/gitlab-org/-/epics/4254) in GitLab 15.6.
-We recommend that you use DAST API testing in addition to [GitLab Secure](../index.md)'s
-other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/index.md),
-you can run DAST API tests as part your CI/CD workflow.
+Perform Dynamic Application Security Testing (DAST) of web APIs to help discover bugs and potential
+security issues that other QA processes may miss. Use DAST API tests in addition to
+[GitLab Secure](../index.md)'s other security scanners and your own test processes. You can run DAST
+API tests either as part your CI/CD workflow, [on-demand](../dast/proxy-based.md#on-demand-scans), or both.
WARNING:
-Do not run DAST API testing against a production server. Not only can it perform *any* function that
+Do not run DAST API testing against a production server. Not only can it perform _any_ function that
the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting
data. Only run DAST API against a test server.
-You can run DAST API scanning against the following web API types:
+DAST API can test the following web API types:
- REST API
- SOAP
@@ -29,9 +28,9 @@ You can run DAST API scanning against the following web API types:
## When DAST API scans run
-DAST API scanning runs in the `dast` stage by default. To ensure DAST API scanning examines the latest
-code, ensure your CI/CD pipeline deploys changes to a test environment in a stage before the `dast`
-stage.
+When run in your CI/CD pipeline, DAST API scanning runs in the `dast` stage by default. To ensure
+DAST API scanning examines the latest code, ensure your CI/CD pipeline deploys changes to a test
+environment in a stage before the `dast` stage.
If your pipeline is configured to deploy to the same web server on each run, running a pipeline
while another is still running could cause a race condition in which one pipeline overwrites the
@@ -2057,9 +2056,232 @@ Setting `SECURE_ANALYZERS_PREFIX` changes the Docker image registry location for
For more information, see [Offline environments](../offline_deployments/index.md).
+## Performance tuning and testing speed
+
+Security tools that perform dynamic analysis testing, such as DAST API, perform testing by sending requests to an instance of your running application. The requests are engineered to test for specific vulnerabilities that might exist in your application. The speed of a dynamic analysis test depends on the following:
+
+- How many requests per second can be sent to your application by our tooling
+- How fast your application responds to requests
+- How many requests must be sent to test the application
+ - How many operations your API is comprised of
+ - How many fields are in each operation (think JSON bodies, headers, query string, cookies, etc.)
+
+If the DAST API testing job still takes longer than expected reach after following the advice in this performance guide, reach out to support for further assistance.
+
+### Diagnosing performance issues
+
+The first step to resolving performance issues is to understand what is contributing to the slower-than-expected testing time. Some common issues we see are:
+
+- DAST API is running on a slow or single-CPU GitLab Runner (GitLab Shared Runners are single-CPU)
+- The application deployed to a slow/single-CPU instance and is not able to keep up with the testing load
+- The application contains a slow operation that impacts the overall test speed (> 1/2 second)
+- The application contains an operation that returns a large amount of data (> 500K+)
+- The application contains a large number of operations (> 40)
+
+#### The application contains a slow operation that impacts the overall test speed (> 1/2 second)
+
+The DAST API job output contains helpful information about how fast we are testing, how fast each operation being tested responds, and summary information. Let's take a look at some sample output to see how it can be used in tracking down performance issues:
+
+```shell
+API Security: Loaded 10 operations from: assets/har-large-response/large_responses.har
+API Security:
+API Security: Testing operation [1/10]: 'GET http://target:7777/api/large_response_json'.
+API Security: - Parameters: (Headers: 4, Query: 0, Body: 0)
+API Security: - Request body size: 0 Bytes (0 bytes)
+API Security:
+API Security: Finished testing operation 'GET http://target:7777/api/large_response_json'.
+API Security: - Excluded Parameters: (Headers: 0, Query: 0, Body: 0)
+API Security: - Performed 767 requests
+API Security: - Average response body size: 130 MB
+API Security: - Average call time: 2 seconds and 82.69 milliseconds (2.082693 seconds)
+API Security: - Time to complete: 14 minutes, 8 seconds and 788.36 milliseconds (848.788358 seconds)
+```
+
+This job console output snippet starts by telling us how many operations were found (10), followed by notifications that testing has started on a specific operation and a summary of the operation has been completed. The summary is the most interesting part of this log output. In the summary, we can see that it took DAST API 767 requests to fully test this operation and its related fields. We can also see that the average response time was 2 seconds and the time to complete was 14 minutes for this one operation.
+
+An average response time of 2 seconds is a good initial indicator that this specific operation takes a long time to test. Further, we can see that the response body size is quite large. The large body size is the culprit here, transferring that much data on each request is what takes the majority of that 2 seconds.
+
+For this issue, the team might decide to:
+
+- Use a multi-CPU runner. Using a multi-CPU runner allows DAST API to parallelize the work being performed. This helps lower the test time, but getting the test down under 10 minutes might still be problematic without moving to a high CPU machine due to how long the operation takes to test.
+ - Trade off between how many CPUs and cost.
+- [Exclude this operation](#excluding-slow-operations) from the DAST API test. While this is the simplest, it has the downside of a gap in security test coverage.
+- [Exclude the operation from feature branch DAST API tests, but include it in the default branch test](#excluding-operations-in-feature-branches-but-not-default-branch).
+- [Split up the DAST API testing into multiple jobs](#splitting-a-test-into-multiple-jobs).
+
+The likely solution is to use a combination of these solutions to reach an acceptable test time, assuming your team's requirements are in the 5-7 minute range.
+
+### Addressing performance issues
+
+The following sections document various options for addressing performance issues for DAST API:
+
+- [Using a multi-CPU Runner](#using-a-multi-cpu-runner)
+- [Excluding slow operations](#excluding-slow-operations)
+- [Splitting a test into multiple jobs](#splitting-a-test-into-multiple-jobs)
+- [Excluding operations in feature branches, but not default branch](#excluding-operations-in-feature-branches-but-not-default-branch)
+
+#### Using a multi-CPU Runner
+
+One of the easiest performance boosts can be achieved using a multi-CPU runner with DAST API. This table shows statistics collected during benchmarking of a Java Spring Boot REST API. In this benchmark, the target and DAST API share a single runner instance.
+
+| CPU Count | Request per Second |
+|----------------------|--------------------|
+| 1 CPU (Shared Runner)| 75 |
+| 4 CPU | 255 |
+| 8 CPU | 400 |
+
+As we can see from this table, increasing the CPU count of the runner can have a large impact on testing speed/performance.
+
+To use a multi-CPU typically requires deploying a self-managed GitLab Runner onto a multi-CPU machine or cloud compute instance.
+
+When multiple types of GitLab Runners are available for use, the various instances are commonly set up with tags that can be used in the job definition to select a type of runner.
+
+Here is an example job definition for DAST API that adds a `tags` section with the tag `multi-cpu`. The job automatically extends the job definition included through the DAST API template.
+
+```yaml
+dast_api:
+ tags:
+ - multi-cpu
+```
+
+To verify that DAST API can detect multiple CPUs in the runner, download the `gl-api-security-scanner.log` file from a completed job's artifacts. Search the file for the string `Starting work item processor` and inspect the reported max DOP (degree of parallelism). The max DOP should be greater than or equal to the number of CPUs assigned to the runner. The value is never lower than 2, even on single CPU runners, unless forced through a configuration variable. If the value reported is less than the number of CPUs assigned to the runner, then something is wrong with the runner deployment. If unable to identify the problem, open a ticket with support to assist.
+
+Example log entry:
+
+`17:00:01.084 [INF] <Peach.Web.Core.Services.WebRunnerMachine> Starting work item processor with 2 max DOP`
+
+#### Excluding slow operations
+
+In the case of one or two slow operations, the team might decide to skip testing the operations. Excluding the operation is done using the `DAST_API_EXCLUDE_PATHS` configuration [variable as explained in this section.](#exclude-paths)
+
+In this example, we have an operation that returns a large amount of data. The operation is `GET http://target:7777/api/large_response_json`. To exclude it we provide the `DAST_API_EXCLUDE_PATHS` configuration variable with the path portion of our operation URL `/api/large_response_json`.
+
+To verify the operation is excluded, run the DAST API job and review the job console output. It includes a list of included and excluded operations at the end of the test.
+
+```yaml
+dast_api:
+ variables:
+ DAST_API_EXCLUDE_PATHS: /api/large_response_json
+```
+
+Excluding operations from testing could allow some vulnerabilities to go undetected.
+{: .alert .alert-warning}
+
+#### Splitting a test into multiple jobs
+
+Splitting a test into multiple jobs is supported by DAST API through the use of [`DAST_API_EXCLUDE_PATHS`](#exclude-paths) and [`DAST_API_EXCLUDE_URLS`](#exclude-urls). When splitting a test up, a good pattern is to disable the `dast_api` job and replace it with two jobs with identifying names. In this example we have two jobs, each job is testing a version of the API, so our names reflect that. However, this technique can be applied to any situation, not just with versions of an API.
+
+The rules we are using in the `dast_api_v1` and `dast_api_v2` jobs are copied from the [DAST API template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml).
+
+```yaml
+# Disable the main dast_api job
+dast_api:
+ rules:
+ - if: $CI_COMMIT_BRANCH
+ when: never
+
+dast_api_v1:
+ extends: dast_api
+ variables:
+ DAST_API_EXCLUDE_PATHS: /api/v1/**
+ rules:
+ - if: $DAST_API_DISABLED
+ when: never
+ - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH &&
+ $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
+ when: never
+ - if: $CI_COMMIT_BRANCH &&
+ $CI_GITLAB_FIPS_MODE == "true"
+ variables:
+ DAST_API_IMAGE_SUFFIX: "-fips"
+ - if: $CI_COMMIT_BRANCH
+
+dast_api_v2:
+ variables:
+ DAST_API_EXCLUDE_PATHS: /api/v2/**
+ rules:
+ - if: $DAST_API_DISABLED
+ when: never
+ - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH &&
+ $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
+ when: never
+ - if: $CI_COMMIT_BRANCH &&
+ $CI_GITLAB_FIPS_MODE == "true"
+ variables:
+ DAST_API_IMAGE_SUFFIX: "-fips"
+ - if: $CI_COMMIT_BRANCH
+```
+
+#### Excluding operations in feature branches, but not default branch
+
+In the case of one or two slow operations, the team might decide to skip testing the operations, or exclude them from feature branch tests, but include them for default branch tests. Excluding the operation is done using the `DAST_API_EXCLUDE_PATHS` configuration [variable as explained in this section.](#exclude-paths)
+
+In this example, we have an operation that returns a large amount of data. The operation is `GET http://target:7777/api/large_response_json`. To exclude it we provide the `DAST_API_EXCLUDE_PATHS` configuration variable with the path portion of our operation URL `/api/large_response_json`. Our configuration disables the main `dast_api` job and creates two new jobs `dast_api_main` and `dast_api_branch`. The `dast_api_branch` is set up to exclude the long operation and only run on non-default branches (e.g. feature branches). The `dast_api_main` branch is set up to only execute on the default branch (`main` in this example). The `dast_api_branch` jobs run faster, allowing for quick development cycles, while the `dast_api_main` job which only runs on default branch builds, takes longer to run.
+
+To verify the operation is excluded, run the DAST API job and review the job console output. It includes a list of included and excluded operations at the end of the test.
+
+```yaml
+# Disable the main job so we can create two jobs with
+# different names
+dast_api:
+ rules:
+ - if: $CI_COMMIT_BRANCH
+ when: never
+
+# DAST API for feature branch work, excludes /api/large_response_json
+dast_api_branch:
+ extends: dast_api
+ variables:
+ DAST_API_EXCLUDE_PATHS: /api/large_response_json
+ rules:
+ - if: $DAST_API_DISABLED
+ when: never
+ - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH &&
+ $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
+ when: never
+ - if: $CI_COMMIT_BRANCH &&
+ $CI_GITLAB_FIPS_MODE == "true"
+ variables:
+ DAST_API_IMAGE_SUFFIX: "-fips"
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+ when: never
+ - if: $CI_COMMIT_BRANCH
+
+# DAST API for default branch (main in our case)
+# Includes the long running operations
+dast_api_main:
+ extends: dast_api
+ rules:
+ - if: $DAST_API_DISABLED
+ when: never
+ - if: $DAST_API_DISABLED_FOR_DEFAULT_BRANCH &&
+ $CI_DEFAULT_BRANCH == $CI_COMMIT_REF_NAME
+ when: never
+ - if: $CI_COMMIT_BRANCH &&
+ $CI_GITLAB_FIPS_MODE == "true"
+ variables:
+ DAST_API_IMAGE_SUFFIX: "-fips"
+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+```
+
## Troubleshooting
-### `Error waiting for API Security 'http://127.0.0.1:5000' to become available`
+### DAST API job times out after N hours
+
+The top two reasons for the DAST API job timing out are slow operations (> 1 second) and using a single-CPU runner for DAST API (GitLab shared runners are single-CPU). Before you can diagnose the problem further, the job must complete so the output can be analyzed. We recommend to start with a multi-CPU runner first, then exclude portions of your API operations until the job completes and the output can be further reviewed.
+
+See the following documentation sections for assistance:
+
+- [Performance tuning and testing speed](#performance-tuning-and-testing-speed)
+- [Using a multi-CPU Runner](#using-a-multi-cpu-runner)
+- [Excluding operations by path](#exclude-paths)
+- [Excluding slow operations](#excluding-slow-operations)
+
+### DAST API job takes too long to complete
+
+See [Performance Tuning and Testing Speed](#performance-tuning-and-testing-speed)
+
+### Error waiting for API Security 'http://127.0.0.1:5000' to become available
A bug exists in versions of the DAST API analyzer prior to v1.6.196 that can cause a background process to fail under certain conditions. The solution is to update to a newer version of the DAST API analyzer.
diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md
index a3c6c46b081..4ed9ceedb4d 100644
--- a/doc/user/application_security/dependency_scanning/index.md
+++ b/doc/user/application_security/dependency_scanning/index.md
@@ -221,7 +221,7 @@ table.supported-languages ul {
<td>N</td>
</tr>
<tr>
- <td rowspan="2">JavaScript</td>
+ <td rowspan="2">JavaScript and TypeScript</td>
<td>All versions</td>
<td><a href="https://www.npmjs.com/">npm</a></td>
<td>
@@ -501,11 +501,11 @@ From GitLab 14.8 the `gemnasium` analyzer scans supported JavaScript projects fo
#### Go
-When scanning a Go project, gemnasium invokes a builder and attempts to generate a [build list](https://go.dev/ref/mod#glos-build-list) using
-[Minimal Version Selection](https://go.dev/ref/mod#glos-minimal-version-selection). If a non-fatal error is encountered, the build process signals
-that the execution should proceed and falls back to parsing the available `go.sum` file.
+Multiple files are supported. When a `go.mod` file is detected, the analyzer attempts to generate a [build list](https://go.dev/ref/mod#glos-build-list) using
+[Minimal Version Selection](https://go.dev/ref/mod#glos-minimal-version-selection). If a non-fatal error is encountered, the analyzer falls back to parsing the
+available `go.sum` file. The process is repeated for every detected `go.mod` and `go.sum` file.
-#### PHP, Go, C, C++, .NET, C&#35;, Ruby, JavaScript
+#### PHP, C, C++, .NET, C&#35;, Ruby, JavaScript
The analyzer for these languages supports multiple lockfiles.
@@ -525,7 +525,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md)
To enable dependency scanning for GitLab 11.9 and later, you must
[include](../../../ci/yaml/index.md#includetemplate) the
-[`Dependency-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml)
+[`Dependency-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml)
that is provided as a part of your GitLab installation.
For GitLab versions earlier than 11.9, you can copy and use the job as defined
that template.
@@ -534,7 +534,7 @@ Add the following to your `.gitlab-ci.yml` file:
```yaml
include:
- - template: Security/Dependency-Scanning.gitlab-ci.yml
+ - template: Jobs/Dependency-Scanning.gitlab-ci.yml
```
The included template creates dependency scanning jobs in your CI/CD
@@ -624,7 +624,6 @@ The following variables allow configuration of global dependency scanning settin
| ----------------------------|------------ |
| `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. The bundle of certificates provided here is also used by other tools during the scanning process, such as `git`, `yarn`, or `npm`. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. |
| `DS_EXCLUDED_ANALYZERS` | Specify the analyzers (by name) to exclude from Dependency Scanning. For more information, see [Dependency Scanning Analyzers](analyzers.md). |
-| `DS_DEFAULT_ANALYZERS` | This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/287691) in GitLab 14.0 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/333299) in 15.0. Use `DS_EXCLUDED_ANALYZERS` instead. |
| `DS_EXCLUDED_PATHS` | Exclude files and directories from the scan based on the paths. A comma-separated list of patterns. Patterns can be globs (see [`doublestar.Match`](https://pkg.go.dev/github.com/bmatcuk/doublestar/v4@v4.0.2#Match) for supported patterns), or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"`. |
| `DS_IMAGE_SUFFIX` | Suffix added to the image name. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354796) in GitLab 14.10.) Automatically set to `"-fips"` when FIPS mode is enabled. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357922) in GitLab 15.0.) |
| `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). |
@@ -641,6 +640,7 @@ The following variables are used for configuring specific analyzers (used for a
| `GEMNASIUM_DB_REMOTE_URL` | `gemnasium` | `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` | Repository URL for fetching the Gemnasium database. |
| `GEMNASIUM_DB_REF_NAME` | `gemnasium` | `master` | Branch name for remote repository database. `GEMNASIUM_DB_REMOTE_URL` is required. |
| `DS_REMEDIATE` | `gemnasium` | `"true"`, `"false"` in FIPS mode | Enable automatic remediation of vulnerable dependencies. Not supported in FIPS mode. |
+| `DS_REMEDIATE_TIMEOUT` | `gemnasium` | `5m` | Timeout for auto-remediation. |
| `GEMNASIUM_LIBRARY_SCAN_ENABLED` | `gemnasium` | `"true"` | Enable detecting vulnerabilities in vendored JavaScript libraries. For now, `gemnasium` leverages [`Retire.js`](https://github.com/RetireJS/retire.js) to do this job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350512) in GitLab 14.8. |
| `DS_JAVA_VERSION` | `gemnasium-maven` | `17` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`, `17`. Available versions in FIPS-enabled image: `8`, `11`, `17`. |
| `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that are passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repositories). |
@@ -1311,6 +1311,11 @@ gemnasium-python-dependency_scanning:
- apt-get update && apt-get install -y libpq-dev
```
+### `NoSuchOptionException` when using `poetry config http-basic` with `CI_JOB_TOKEN`
+
+This error can occur when the automatically generated `CI_JOB_TOKEN` starts with a hyphen (`-`).
+To avoid this error, follow [Poetry's configuration advice](https://python-poetry.org/docs/repositories/#configuring-credentials).
+
### Error: Project has `<number>` unresolved dependencies
The error message `Project has <number> unresolved dependencies` indicates a dependency resolution problem caused by your `gradle.build` or `gradle.build.kts` file. In the current release, `gemnasium-maven` cannot continue processing when an unresolved dependency is encountered. However, There is an [open issue](https://gitlab.com/gitlab-org/gitlab/-/issues/337083) to allow `gemnasium-maven` to recover from unresolved dependency errors and produce a dependency graph. Until this issue has been resolved, you'll need to consult the [Gradle dependency resolution docs](https://docs.gradle.org/current/userguide/dependency_resolution.html) for details on how to fix your `gradle.build` file.
diff --git a/doc/user/application_security/get-started-security.md b/doc/user/application_security/get-started-security.md
index b6213a98f91..5774bf940b0 100644
--- a/doc/user/application_security/get-started-security.md
+++ b/doc/user/application_security/get-started-security.md
@@ -36,7 +36,7 @@ The following steps will help you get the most from GitLab application security
remediating existing vulnerabilities and preventing the introduction of new ones.
1. Enable other scan types such as [SAST](sast/index.md), [DAST](dast/index.md),
[Fuzz testing](coverage_fuzzing/index.md), or [Container Scanning](container_scanning/index.md).
-1. Use [Compliance Pipelines](../group/manage.md#configure-a-compliance-pipeline)
+1. Use [Compliance Pipelines](../group/compliance_frameworks.md#configure-a-compliance-pipeline)
or [Scan Execution Policies](policies/scan-execution-policies.md) to enforce required scan types
and ensure separation of duties between security and engineering.
1. Consider enabling [Review Apps](../../development/testing_guide/review_apps.md) to allow for DAST
diff --git a/doc/user/application_security/iac_scanning/index.md b/doc/user/application_security/iac_scanning/index.md
index 150c2b732d8..1c14c529523 100644
--- a/doc/user/application_security/iac_scanning/index.md
+++ b/doc/user/application_security/iac_scanning/index.md
@@ -4,25 +4,27 @@ group: Static Analysis
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Infrastructure as Code (IaC) Scanning
+# Infrastructure as Code (IaC) Scanning **(FREE)**
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6655) in GitLab 14.5.
Infrastructure as Code (IaC) Scanning scans your IaC configuration files for known vulnerabilities.
-Currently, IaC scanning supports configuration files for Terraform, Ansible, AWS CloudFormation, and Kubernetes.
+IaC Scanning supports configuration files for Terraform, Ansible, AWS CloudFormation, and Kubernetes.
## Requirements
IaC Scanning runs in the `test` stage, which is available by default. If you redefine the stages in the `.gitlab-ci.yml` file, the `test` stage is required.
-To run IaC scanning jobs, by default, you need GitLab Runner with the
+We recommend a minimum of 4GB RAM to ensure consistent performance.
+
+To run IaC Scanning jobs, by default, you need GitLab Runner with the
[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or
[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor.
If you're using the shared runners on GitLab.com, this is enabled by default.
WARNING:
-Our IaC scanning jobs require a Linux/amd64 container type. Windows containers are not yet supported.
+Our IaC Scanning jobs require a Linux/amd64 container type. Windows containers are not supported.
WARNING:
If you use your own runners, make sure the Docker version installed
@@ -30,20 +32,20 @@ is **not** `19.03.0`. See [troubleshooting information](../sast/index.md#error-r
## Supported languages and frameworks
-GitLab IaC scanning supports a variety of IaC configuration files. Our IaC security scanners also feature automatic language detection which works even for mixed-language projects. If any supported configuration files are detected in project source code we automatically run the appropriate IaC analyzers.
+GitLab IaC Scanning supports a variety of IaC configuration files. Our IaC security scanners also feature automatic language detection which works even for mixed-language projects. If any supported configuration files are detected in project source code we automatically run the appropriate IaC analyzers.
-| Configuration File Type | Scan tool | Introduced in GitLab Version |
-|------------------------------------------|----------------------------------|-------------------------------|
-| Ansible | [KICS](https://kics.io/) | 14.5 |
-| AWS CloudFormation | [KICS](https://kics.io/) | 14.5 |
-| Azure Resource Manager <sup>1</sup> | [KICS](https://kics.io/) | 14.5 |
-| Dockerfile | [KICS](https://kics.io/) | 14.5 |
-| Google Deployment Manager | [KICS](https://kics.io/) | 14.5 |
-| Kubernetes | [KICS](https://kics.io/) | 14.5 |
-| OpenAPI | [KICS](https://kics.io/) | 14.5 |
-| Terraform <sup>2</sup> | [KICS](https://kics.io/) | 14.5 |
+| Configuration file type | Scan tool | Introduced in GitLab version |
+| ----------------------------------- | ------------------------ | ---------------------------- |
+| Ansible | [KICS](https://kics.io/) | 14.5 |
+| AWS CloudFormation | [KICS](https://kics.io/) | 14.5 |
+| Azure Resource Manager <sup>1</sup> | [KICS](https://kics.io/) | 14.5 |
+| Dockerfile | [KICS](https://kics.io/) | 14.5 |
+| Google Deployment Manager | [KICS](https://kics.io/) | 14.5 |
+| Kubernetes | [KICS](https://kics.io/) | 14.5 |
+| OpenAPI | [KICS](https://kics.io/) | 14.5 |
+| Terraform <sup>2</sup> | [KICS](https://kics.io/) | 14.5 |
-1. IaC scanning can analyze Azure Resource Manager templates in JSON format. If you write templates in the [Bicep](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview) language, you must use [the bicep CLI](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-cli) to convert your Bicep files into JSON before GitLab IaC scanning can analyze them.
+1. IaC Scanning can analyze Azure Resource Manager templates in JSON format. If you write templates in the [Bicep](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview) language, you must use [the bicep CLI](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-cli) to convert your Bicep files into JSON before GitLab IaC Scanning can analyze them.
1. Terraform modules in a custom registry are not scanned for vulnerabilities. You can follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/357004) for the proposed feature.
### Supported distributions
@@ -77,7 +79,7 @@ Different features are available in different [GitLab tiers](https://about.gitla
as shown in the following table:
| Capability | In Free & Premium | In Ultimate |
-|:----------------------------------------------------------------|:--------------------|:-------------------|
+| :-------------------------------------------------------------- | :------------------ | :----------------- |
| [Configure IaC scanner](#configuration) | **{check-circle}** | **{check-circle}** |
| Download [JSON Report](#reports-json-format) | **{check-circle}** | **{check-circle}** |
| See new findings in merge request widget | **{dotted-circle}** | **{check-circle}** |
@@ -98,14 +100,14 @@ To configure IaC Scanning for a project you can:
### Configure IaC Scanning manually
To enable IaC Scanning you must [include](../../../ci/yaml/index.md#includetemplate) the
-[`SAST-IaC.gitlab-ci.yml template`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml) provided as part of your GitLab installation. Here is an example of how to include it:
+[`SAST-IaC.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml) provided as part of your GitLab installation. Here is an example of how to include it:
```yaml
include:
- template: Jobs/SAST-IaC.gitlab-ci.yml
```
-The included template creates IaC scanning jobs in your CI/CD pipeline and scans
+The included template creates IaC Scanning jobs in your CI/CD pipeline and scans
your project's configuration files for possible vulnerabilities.
The results are saved as a
@@ -121,7 +123,123 @@ To enable IaC Scanning in a project, you can create a merge request:
1. In the **Infrastructure as Code (IaC) Scanning** row, select **Configure with a merge request**.
1. Review and merge the merge request to enable IaC Scanning.
-Pipelines now include an IaC job.
+Pipelines now include an IaC Scanning job.
+
+## Customize rulesets **(ULTIMATE)**
+
+> [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/235359) support for overriding rules in GitLab 14.8.
+
+You can customize the default IaC Scanning rules provided with GitLab.
+
+The following customization options can be used separately, or together:
+
+- [Disable predefined rules](#disable-predefined-analyzer-rules).
+- [Override predefined rules](#override-predefined-analyzer-rules).
+
+### Disable predefined analyzer rules
+
+If there are specific IaC Scanning rules that you don't want active, you can disable them.
+
+To disable analyzer rules:
+
+1. Create a `.gitlab` directory at the root of your project, if one doesn't already exist.
+1. Create a custom ruleset file named `sast-ruleset.toml` in the `.gitlab` directory, if
+ one doesn't already exist.
+1. Set the `disabled` flag to `true` in the context of a `ruleset` section.
+1. In one or more `ruleset` subsections, list the rules to disable. Every
+ `ruleset.identifier` section has:
+ - A `type` field for the rule. For IaC Scanning, the identifier type is `kics_id`.
+ - A `value` field for the rule identifier. KICS rule identifiers are alphanumeric strings. To find the rule identifier, you can:
+ - Find it in the [JSON report artifact](#reports-json-format).
+ - Search for the rule name in the [list of KICS queries](https://docs.kics.io/latest/queries/all-queries/) and copy the alphanumeric identifier that's shown. The rule name is shown on the [Vulnerability Page](../vulnerabilities/index.md) when a rule violation is detected.
+
+In the following example `sast-ruleset.toml` file, the disabled rules are assigned to
+the `kics` analyzer by matching the `type` and `value` of identifiers:
+
+```toml
+[kics]
+ [[kics.ruleset]]
+ disable = true
+ [kics.ruleset.identifier]
+ type = "kics_id"
+ value = "8212e2d7-e683-49bc-bf78-d6799075c5a7"
+
+ [[kics.ruleset]]
+ disable = true
+ [kics.ruleset.identifier]
+ type = "kics_id"
+ value = "b03a748a-542d-44f4-bb86-9199ab4fd2d5"
+```
+
+### Override predefined analyzer rules
+
+If there are specific IaC Scanning rules you want to customize, you can override them. For
+example, you might lower the severity of a rule or link to your own documentation about how to fix a finding.
+
+To override rules:
+
+1. Create a `.gitlab` directory at the root of your project, if one doesn't already exist.
+1. Create a custom ruleset file named `sast-ruleset.toml` in the `.gitlab` directory, if
+ one doesn't already exist.
+1. In one or more `ruleset.identifier` subsections, list the rules to override. Every
+ `ruleset.identifier` section has:
+ - A `type` field for the rule. For IaC Scanning, the identifier type is `kics_id`.
+ - A `value` field for the rule identifier. KICS rule identifiers are alphanumeric strings. To find the rule identifier, you can:
+ - Find it in the [JSON report artifact](#reports-json-format).
+ - Search for the rule name in the [list of KICS queries](https://docs.kics.io/latest/queries/all-queries/) and copy the alphanumeric identifier that's shown. The rule name is shown on the [Vulnerability Page](../vulnerabilities/index.md) when a rule violation is detected.
+1. In the `ruleset.override` context of a `ruleset` section,
+ provide the keys to override. Any combination of keys can be
+ overridden. Valid keys are:
+ - description
+ - message
+ - name
+ - severity (valid options are: Critical, High, Medium, Low, Unknown, Info)
+
+In the following example `sast-ruleset.toml` file, rules are matched by the `type` and
+`value` of identifiers and then overridden:
+
+```toml
+[kics]
+ [[kics.ruleset]]
+ [kics.ruleset.identifier]
+ type = "kics_id"
+ value = "8212e2d7-e683-49bc-bf78-d6799075c5a7"
+ [kics.ruleset.override]
+ description = "OVERRIDDEN description"
+ message = "OVERRIDDEN message"
+ name = "OVERRIDDEN name"
+ severity = "Info"
+```
+
+## Pinning to specific analyzer version
+
+The GitLab-managed CI/CD template specifies a major version and automatically pulls the latest analyzer release within that major version.
+
+In some cases, you may need to use a specific version.
+For example, you might need to avoid a regression in a later release.
+
+To override the automatic update behavior, set the `SAST_ANALYZER_IMAGE_TAG` CI/CD variable
+in your CI/CD configuration file after you include the [`SAST-IaC.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml).
+
+Only set this variable within a specific job.
+If you set it [at the top level](../../../ci/variables/index.md#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file), the version you set will be used for other SAST analyzers.
+
+You can set the tag to:
+
+- A major version, like `3`. Your pipelines will use any minor or patch updates that are released within this major version.
+- A minor version, like `3.7`. Your pipelines will use any patch updates that are released within this minor version.
+- A patch version, like `3.7.0`. Your pipelines won't receive any updates.
+
+This example uses a specific minor version of the `KICS` analyzer:
+
+```yaml
+include:
+ - template: Security/SAST-IaC.gitlab-ci.yml
+
+kics-iac-sast:
+ variables:
+ SAST_ANALYZER_IMAGE_TAG: "3.1"
+```
## Reports JSON format
diff --git a/doc/user/application_security/img/secure_tools_and_cicd_stages.png b/doc/user/application_security/img/secure_tools_and_cicd_stages.png
index 3dbfd835baf..3284b376adc 100644
--- a/doc/user/application_security/img/secure_tools_and_cicd_stages.png
+++ b/doc/user/application_security/img/secure_tools_and_cicd_stages.png
Binary files differ
diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md
index 809fa5f3764..5ddfa99fc81 100644
--- a/doc/user/application_security/index.md
+++ b/doc/user/application_security/index.md
@@ -227,7 +227,7 @@ The artifact generated by the secure analyzer contains all findings it discovers
> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4393) in GitLab Free 13.5.
> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/273205) in 13.6.
-> - Report download dropdown [added](https://gitlab.com/gitlab-org/gitlab/-/issues/273418) in 13.7.
+> - Report download dropdown list [added](https://gitlab.com/gitlab-org/gitlab/-/issues/273418) in 13.7.
> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/249550) in GitLab 13.9.
### All tiers
@@ -403,7 +403,7 @@ Learn more on overriding security jobs:
- [Overriding Dependency Scanning jobs](dependency_scanning/index.md#overriding-dependency-scanning-jobs).
- [Overriding Container Scanning jobs](container_scanning/index.md#overriding-the-container-scanning-template).
- [Overriding Secret Detection jobs](secret_detection/index.md#configure-scan-settings).
-- [Overriding DAST jobs](dast/index.md#customize-dast-settings).
+- [Overriding DAST jobs](dast/proxy-based.md#customize-dast-settings).
- [Overriding License Compliance jobs](../compliance/license_compliance/index.md#overriding-the-template).
All the security scanning tools define their stage, so this error can occur with all of them.
@@ -487,7 +487,7 @@ Security and compliance teams must ensure that security scans:
GitLab provides two methods of accomplishing this, each with advantages and disadvantages.
-- [Compliance framework pipelines](../group/manage.md#configure-a-compliance-pipeline)
+- [Compliance framework pipelines](../group/compliance_frameworks.md#configure-a-compliance-pipeline)
are recommended when:
- Scan execution enforcement is required for any scanner that uses a GitLab template, such as SAST IaC, DAST, Dependency Scanning,
@@ -499,8 +499,8 @@ GitLab provides two methods of accomplishing this, each with advantages and disa
are recommended when:
- Scan execution enforcement is required for DAST which uses a DAST site or scan profile.
- - Scan execution enforcement is required for SAST, Secret Detection, or Container Scanning with project-specific variable
- customizations. To accomplish this, users must create a separate security policy per project.
+ - Scan execution enforcement is required for SAST, Secret Detection, Dependency Scanning, or Container Scanning with project-specific
+variable customizations. To accomplish this, users must create a separate security policy per project.
- Scans are required to run on a regular, scheduled cadence.
- Either solution can be used equally well when:
@@ -514,7 +514,7 @@ Additional details about the differences between the two solutions are outlined
| | Compliance Framework Pipelines | Scan Execution Policies |
| ------ | ------ | ------ |
-| **Flexibility** | Supports anything that can be done in a CI file. | Limited to only the items for which GitLab has explicitly added support. DAST, SAST, Secret Detection, and Container Scanning scans are supported. |
+| **Flexibility** | Supports anything that can be done in a CI file. | Limited to only the items for which GitLab has explicitly added support. DAST, SAST, Secret Detection, Dependency Scanning, and Container Scanning scans are supported. |
| **Usability** | Requires knowledge of CI YAML. | Follows a `rules` and `actions`-based YAML structure. |
| **Inclusion in CI pipeline** | The compliance pipeline is executed instead of the project's `.gitlab-ci.yml` file. To include the project's `.gitlab-ci.yml` file, use an `include` statement. Defined variables aren't allowed to be overwritten by the included project's YAML file. | Forced inclusion of a new job into the CI pipeline. DAST jobs that must be customized on a per-project basis can have project-level Site Profiles and Scan Profiles defined. To ensure separation of duties, these profiles are immutable when referenced in a scan execution policy. All jobs can be customized as part of the security policy itself with the same variables that are normally available to the CI job. |
| **Schedulable** | Can be scheduled through a scheduled pipeline on the group. | Can be scheduled natively through the policy configuration itself. |
diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md
index b1315329071..f6d22ab28cd 100644
--- a/doc/user/application_security/policies/index.md
+++ b/doc/user/application_security/policies/index.md
@@ -57,7 +57,7 @@ project and a project that you would like to designate as the security policy pr
1. On the top bar, select **Main menu > Projects** and find your project.
1. On the left sidebar, select **Security & Compliance > Policies**.
1. Select **Edit Policy Project**, and search for and select the
- project you would like to link from the dropdown menu.
+ project you would like to link from the dropdown list.
1. Select **Save**.
To unlink a security policy project, follow the same steps but instead select the trash can icon in
diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md
index 41d25dfa8c8..f950d5116b1 100644
--- a/doc/user/application_security/policies/scan-execution-policies.md
+++ b/doc/user/application_security/policies/scan-execution-policies.md
@@ -15,7 +15,7 @@ with a long, random job name. In the unlikely event of a job name collision, the
any pre-existing job in the pipeline. If a policy is created at the group-level, it will apply to every child
project or subgroup. A group-level policy cannot be edited from a child project or subgroup.
-This feature has some overlap with [compliance framework pipelines](../../group/manage.md#configure-a-compliance-pipeline),
+This feature has some overlap with [compliance framework pipelines](../../group/compliance_frameworks.md#configure-a-compliance-pipeline),
as we have not [unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312).
For details on the similarities and differences between these features, see
[Enforce scan execution](../index.md#enforce-scan-execution).
@@ -129,14 +129,14 @@ rule in the defined policy are met.
| Field | Type | Possible values | Description |
|-------|------|-----------------|-------------|
-| `scan` | `string` | `dast`, `secret_detection`, `sast`, `container_scanning` | The action's type. |
-| `site_profile` | `string` | Name of the selected [DAST site profile](../dast/index.md#site-profile). | The DAST site profile to execute the DAST scan. This field should only be set if `scan` type is `dast`. |
-| `scanner_profile` | `string` or `null` | Name of the selected [DAST scanner profile](../dast/index.md#scanner-profile). | The DAST scanner profile to execute the DAST scan. This field should only be set if `scan` type is `dast`.|
+| `scan` | `string` | `dast`, `secret_detection`, `sast`, `container_scanning`, `dependency_scanning` | The action's type. |
+| `site_profile` | `string` | Name of the selected [DAST site profile](../dast/proxy-based.md#site-profile). | The DAST site profile to execute the DAST scan. This field should only be set if `scan` type is `dast`. |
+| `scanner_profile` | `string` or `null` | Name of the selected [DAST scanner profile](../dast/proxy-based.md#scanner-profile). | The DAST scanner profile to execute the DAST scan. This field should only be set if `scan` type is `dast`.|
| `variables` | `object` | | A set of CI variables, supplied as an array of `key: value` pairs, to apply and enforce for the selected scan. The `key` is the variable name, with its `value` provided as a string. This parameter supports any variable that the GitLab CI job supports for the specified scan. |
Note the following:
-- You must create the [site profile](../dast/index.md#site-profile) and [scanner profile](../dast/index.md#scanner-profile)
+- You must create the [site profile](../dast/proxy-based.md#site-profile) and [scanner profile](../dast/proxy-based.md#scanner-profile)
with selected names for each project that is assigned to the selected Security Policy Project.
Otherwise, the policy is not applied and a job with an error message is created instead.
- Once you associate the site profile and scanner profile by name in the policy, it is not possible
@@ -152,7 +152,7 @@ Note the following:
mode when executed as part of a scheduled scan.
- A container scanning scan that is configured for the `pipeline` rule type ignores the agent defined in the `agents` object. The `agents` object is only considered for `schedule` rule types.
An agent with a name provided in the `agents` object must be created and configured for the project.
-- The SAST scan uses the default template and runs in a [child pipeline](../../../ci/pipelines/downstream_pipelines.md#parent-child-pipelines).
+- The Dependency Scanning and SAST scans use the default templates and run in a [child pipeline](../../../ci/pipelines/downstream_pipelines.md#parent-child-pipelines).
## Example security policies project
diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md
index 45b715a52b8..7482df18cc3 100644
--- a/doc/user/application_security/policies/scan-result-policies.md
+++ b/doc/user/application_security/policies/scan-result-policies.md
@@ -6,6 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Scan result policies **(ULTIMATE)**
+> Group-level scan result policies [introduced](https://gitlab.com/groups/gitlab-org/-/epics/7622) in GitLab 15.6.
+
You can use scan result policies to take action based on scan results. For example, one type of scan
result policy is a security approval policy that allows approval to be required based on the
findings of one or more security scan jobs. Scan result policies are evaluated after a CI scanning
@@ -20,7 +22,8 @@ job is fully executed. The following video gives you an overview of GitLab scan
## Scan result policy editor
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77814) in GitLab 14.8.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77814) in GitLab 14.8.
+> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/369473) in GitLab 15.6.
NOTE:
Only project Owners have the [permissions](../../permissions.md#project-members-permissions)
@@ -38,6 +41,9 @@ before the policy changes take effect.
The [policy editor](index.md#policy-editor) supports YAML mode and rule mode.
+NOTE:
+Propagating scan result policies created for groups with a large number of projects will take a while to complete.
+
## Scan result policies schema
The YAML file with scan result policies consists of an array of objects matching the scan result
diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md
index dfb096f14cc..e83825636bf 100644
--- a/doc/user/application_security/sast/analyzers.md
+++ b/doc/user/application_security/sast/analyzers.md
@@ -12,7 +12,7 @@ Static Application Security Testing (SAST) uses analyzers
to detect vulnerabilities in source code. Each analyzer is a wrapper around a [scanner](../terminology/index.md#scanner), a third-party code analysis tool.
The analyzers are published as Docker images that SAST uses to launch dedicated containers for each
-analysis.
+analysis. We recommend a minimum of 4GB RAM to ensure consistent performance of the analyzers.
SAST default images are maintained by GitLab, but you can also integrate your own custom image.
@@ -134,6 +134,14 @@ In GitLab 15.4, we [removed the deprecated analyzers](https://gitlab.com/gitlab-
To preview the upcoming changes to the CI/CD configuration in GitLab 15.3 or earlier:
1. Open an MR to switch from the Stable CI/CD template, `SAST.gitlab-ci.yaml`, to [the Latest template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.latest.gitlab-ci.yml), `SAST.latest.gitlab-ci.yaml`.
+ - On GitLab.com, use the latest template directly:
+
+ ```yaml
+ include:
+ template: 'Jobs/SAST.latest.gitlab-ci.yaml'
+ ```
+
+ - On a Self-Managed instance, download the template from GitLab.com:
```yaml
include:
diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md
index a7624db4604..b1bc9794ced 100644
--- a/doc/user/application_security/sast/index.md
+++ b/doc/user/application_security/sast/index.md
@@ -70,44 +70,45 @@ is **not** `19.03.0`. See [troubleshooting information](#error-response-from-dae
## Supported languages and frameworks
-GitLab SAST supports a variety of languages, package managers, and frameworks. Our SAST security scanners also feature automatic language detection which works even for mixed-language projects. If any supported language is detected in project source code we automatically run the appropriate SAST analyzers.
-
-You can also [view our language roadmap](https://about.gitlab.com/direction/secure/static-analysis/sast/#language-support) and [request other language support by opening an issue](https://gitlab.com/groups/gitlab-org/-/epics/297).
-
-| Language (package managers) / framework | Scan tool | Introduced in GitLab Version |
-|------------------------------------------------|-----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
-| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 |
-| .NET Framework<sup>1</sup> | [Security Code Scan](https://security-code-scan.github.io) | 13.0 |
-| .NET (all versions, C# only) | [Semgrep](https://semgrep.dev) | 15.4 |
-| Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 |
-| C | [Semgrep](https://semgrep.dev) | 14.2 |
-| C/C++ | [Flawfinder](https://github.com/david-a-wheeler/flawfinder) | 10.7 |
-| Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.1 |
-| Go<sup>3</sup> | [Gosec](https://github.com/securego/gosec) | 10.7 |
-| Go | [Semgrep](https://semgrep.dev) | 14.4 |
-| Groovy<sup>2</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Maven, SBT) |
-| Helm Charts | [Kubesec](https://github.com/controlplaneio/kubesec) | 13.1 |
-| Java (any build system) | [Semgrep](https://semgrep.dev) | 14.10 |
-| Java<sup>2, 3</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (SBT) |
-| Java (Android) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 |
-| JavaScript<sup>3</sup> | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8 |
-| JavaScript | [Semgrep](https://semgrep.dev) | 13.10 |
-| Kotlin (Android) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 |
-| Kotlin (General)<sup>2</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 13.11 |
-| Kubernetes manifests | [Kubesec](https://github.com/controlplaneio/kubesec) | 12.6 |
-| Node.js | [NodeJsScan](https://github.com/ajinabraham/NodeJsScan) | 11.1 |
-| Objective-C (iOS) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 |
-| PHP | [phpcs-security-audit](https://github.com/FloeDesignTechnologies/phpcs-security-audit) | 10.8 |
-| Python<sup>3</sup> | [bandit](https://github.com/PyCQA/bandit) | 10.3 |
-| Python | [Semgrep](https://semgrep.dev) | 13.9 |
-| React<sup>3</sup> | [ESLint react plugin](https://github.com/yannickcr/eslint-plugin-react) | 12.5 |
-| React | [Semgrep](https://semgrep.dev) | 13.10 |
-| Ruby | [brakeman](https://brakemanscanner.org) | 13.9 |
-| Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 |
-| Scala<sup>2</sup> | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Gradle, Maven) |
-| Swift (iOS) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 |
-| TypeScript<sup>3</sup> | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.9, [merged](https://gitlab.com/gitlab-org/gitlab/-/issues/36059) with ESLint in 13.2 |
-| TypeScript | [Semgrep](https://semgrep.dev) | 13.10 |
+GitLab SAST supports scanning a variety of programming languages and frameworks.
+Once you [enable SAST](#configuration), the right set of analyzers runs automatically even if your project uses more than one language.
+
+Check the [SAST direction page](https://about.gitlab.com/direction/secure/static-analysis/sast/#language-support) to learn more about our plans for language support in SAST.
+
+| Language / framework | [Analyzer](analyzers.md) used for scanning | Minimum supported GitLab version |
+|------------------------------|--------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
+| .NET Core | [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | 11.0 |
+| .NET Framework<sup>1</sup> | [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | 13.0 |
+| .NET (all versions, C# only) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 15.4 |
+| Apex (Salesforce) | [PMD](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) | 12.1 |
+| C | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 14.2 |
+| C/C++ | [Flawfinder](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) | 10.7 |
+| Elixir (Phoenix) | [Sobelow](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) | 11.1 |
+| Go<sup>3</sup> | [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) | 10.7 |
+| Go | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 14.4 |
+| Groovy<sup>2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 11.3 (Gradle) & 11.9 (Maven, SBT) |
+| Helm Charts | [Kubesec](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) | 13.1 |
+| Java (any build system) | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 14.10 |
+| Java<sup>2, 3</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (SBT) |
+| Java (Android) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 |
+| JavaScript<sup>3</sup> | [ESLint security plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 11.8 |
+| JavaScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.10 |
+| Kotlin (Android) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 |
+| Kotlin (General)<sup>2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 13.11 |
+| Kubernetes manifests | [Kubesec](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) | 12.6 |
+| Node.js | [NodeJsScan](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan) | 11.1 |
+| Objective-C (iOS) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 |
+| PHP | [phpcs-security-audit](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) | 10.8 |
+| Python<sup>3</sup> | [bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) | 10.3 |
+| Python | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.9 |
+| React<sup>3</sup> | [ESLint react plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 12.5 |
+| React | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.10 |
+| Ruby | [brakeman](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman) | 13.9 |
+| Ruby on Rails | [brakeman](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman) | 10.3 |
+| Scala<sup>2</sup> | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) with the find-sec-bugs plugin | 11.0 (SBT) & 11.9 (Gradle, Maven) |
+| Swift (iOS) | [MobSF (beta)](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) | 13.5 |
+| TypeScript<sup>3</sup> | [ESLint security plugin](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | 11.9, [merged](https://gitlab.com/gitlab-org/gitlab/-/issues/36059) with ESLint in 13.2 |
+| TypeScript | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) with GitLab-managed rules | 13.10 |
1. .NET 4 support is limited. The analyzer runs in a Linux container and does not have access to Windows-specific libraries or features. Use the Semgrep-based scanner if you need .NET 4 support.
1. The SpotBugs-based analyzer supports [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/), and [SBT](https://www.scala-sbt.org/). It can also be used with variants like the
@@ -116,7 +117,7 @@ You can also [view our language roadmap](https://about.gitlab.com/direction/secu
and the [Maven wrapper](https://github.com/takari/maven-wrapper). However, SpotBugs has [limitations](https://gitlab.com/gitlab-org/gitlab/-/issues/350801) when used against [Ant](https://ant.apache.org/)-based projects. We recommend using the Semgrep-based analyzer for Ant-based Java projects.
1. These analyzers reached [End of Support](https://about.gitlab.com/handbook/product/gitlab-the-product/#end-of-support) status [in GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/issues/352554).
-### Multi-project support
+## Multi-project support
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4895) in GitLab 13.7.
@@ -136,16 +137,52 @@ The following analyzers have multi-project support:
- SpotBugs
- Sobelow
-#### Enable multi-project support for Security Code Scan
+### Enable multi-project support for Security Code Scan
Multi-project support in the Security Code Scan requires a Solution (`.sln`) file in the root of
the repository. For details on the Solution format, see the Microsoft reference [Solution (`.sln`) file](https://learn.microsoft.com/en-us/visualstudio/extensibility/internals/solution-dot-sln-file?view=vs-2019).
-### Supported distributions
+## False positive detection **(ULTIMATE)**
+
+> Introduced in GitLab 14.2.
+
+Vulnerabilities that have been detected and are false positives will be flagged as false positives in the security dashboard.
+
+False positive detection is available in a subset of the [supported languages](#supported-languages-and-frameworks) and [analyzers](analyzers.md):
+
+- Ruby, in the Brakeman-based analyzer
+
+![SAST false-positives show in Vulnerability Pages](img/sast_vulnerability_page_fp_detection_v15_2.png)
+
+## Advanced vulnerability tracking **(ULTIMATE)**
+
+> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5144) in GitLab 14.2.
+
+Source code is volatile; as developers make changes, source code may move within files or between files.
+Security analyzers may have already reported vulnerabilities that are being tracked in the [Vulnerability Report](../vulnerability_report/index.md).
+These vulnerabilities are linked to specific problematic code fragments so that they can be found and fixed.
+If the code fragments are not tracked reliably as they move, vulnerability management is harder because the same vulnerability could be reported again.
+
+GitLab SAST uses an advanced vulnerability tracking algorithm to more accurately identify when the same vulnerability has moved within a file due to refactoring or unrelated changes.
+
+Advanced vulnerability tracking is available in a subset of the [supported languages](#supported-languages-and-frameworks) and [analyzers](analyzers.md):
+
+- C, in the Semgrep-based analyzer only
+- Go, in the Gosec- and Semgrep-based analyzers
+- Java, in the Semgrep-based analyzer only
+- JavaScript, in the Semgrep-based analyzer only
+- Python, in the Semgrep-based analyzer only
+- Ruby, in the Brakeman-based analyzer
+
+Support for more languages and analyzers is tracked in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/5144).
+
+For more information, see the confidential project `https://gitlab.com/gitlab-org/security-products/post-analyzers/tracking-calculator`. The content of this project is available only to GitLab team members.
+
+## Supported distributions
The default scanner images are build off a base Alpine image for size and maintainability.
-#### FIPS-enabled images
+### FIPS-enabled images
> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6479) in GitLab 14.10.
@@ -169,17 +206,14 @@ A FIPS-compliant image is only available for the Semgrep-based analyzer.
To use SAST in a FIPS-compliant manner, you must [exclude other analyzers from running](analyzers.md#customize-analyzers).
-### Making SAST analyzers available to all GitLab tiers
-
-All open source (OSS) analyzers have been moved to the GitLab Free tier as of GitLab 13.3.
-
-#### Summary of features per tier
+## Summary of features per tier
Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/),
as shown in the following table:
| Capability | In Free & Premium | In Ultimate |
|:---------------------------------------------------------------- -|:--------------------|:-------------------|
+| Automatically scan code with [appropriate analyzers](#supported-languages-and-frameworks) | **{check-circle}** | **{check-circle}** |
| [Configure SAST scanners](#configuration) | **{check-circle}** | **{check-circle}** |
| [Customize SAST settings](#available-cicd-variables) | **{check-circle}** | **{check-circle}** |
| Download [JSON Report](#reports-json-format) | **{check-circle}** | **{check-circle}** |
@@ -207,14 +241,14 @@ To configure SAST for a project you can:
### Configure SAST manually
To enable SAST you must [include](../../../ci/yaml/index.md#includetemplate)
-the [`SAST.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml)
+the [`SAST.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml)
provided as a part of your GitLab installation.
Add the following to your `.gitlab-ci.yml` file:
```yaml
include:
- - template: Security/SAST.gitlab-ci.yml
+ - template: Jobs/SAST.gitlab-ci.yml
```
The included template creates SAST jobs in your CI/CD pipeline and scans
@@ -300,14 +334,26 @@ spotbugs-sast:
FAIL_NEVER: 1
```
-#### Pinning to minor image version
+### Pinning to minor image version
+
+The GitLab-managed CI/CD template specifies a major version and automatically pulls the latest analyzer release within that major version.
-While our templates use `MAJOR` version pinning to always ensure the latest analyzer
-versions are pulled, there are certain cases where it can be beneficial to pin
-an analyzer to a specific release. To do so, override the `SAST_ANALYZER_IMAGE_TAG` CI/CD variable
-in the job template directly.
+In some cases, you may need to use a specific version.
+For example, you might need to avoid a regression in a later release.
-In the example below, we pin to a minor version of the `semgrep` analyzer and a specific patch version of the `brakeman` analyzer:
+To override the automatic update behavior, set the `SAST_ANALYZER_IMAGE_TAG` CI/CD variable
+in your CI/CD configuration file after you include the [`SAST.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml).
+
+Only set this variable within a specific job.
+If you set it [at the top level](../../../ci/variables/index.md#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file), the version you set will be used for other SAST analyzers.
+
+You can set the tag to:
+
+- A major version, like `3`. Your pipelines will use any minor or patch updates that are released within this major version.
+- A minor version, like `3.7`. Your pipelines will use any patch updates that are released within this minor version.
+- A patch version, like `3.7.0`. Your pipelines won't receive any updates.
+
+This example uses a specific minor version of the `semgrep` analyzer and a specific patch version of the `brakeman` analyzer:
```yaml
include:
@@ -315,47 +361,13 @@ include:
semgrep-sast:
variables:
- SAST_ANALYZER_IMAGE_TAG: "2.16"
+ SAST_ANALYZER_IMAGE_TAG: "3.7"
brakeman-sast:
variables:
- SAST_ANALYZER_IMAGE_TAG: "2.21.1"
+ SAST_ANALYZER_IMAGE_TAG: "3.1.1"
```
-### False Positive Detection **(ULTIMATE)**
-
-> Introduced in GitLab 14.2.
-
-Vulnerabilities that have been detected and are false positives will be flagged as false positives in the security dashboard.
-
-False positive detection is available in a subset of the [supported languages](#supported-languages-and-frameworks) and [analyzers](analyzers.md):
-
-- Ruby, in the Brakeman-based analyzer
-
-![SAST false-positives show in Vulnerability Pages](img/sast_vulnerability_page_fp_detection_v15_2.png)
-
-### Advanced vulnerability tracking **(ULTIMATE)**
-
-> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5144) in GitLab 14.2.
-
-Source code is volatile; as developers make changes, source code may move within files or between files.
-Security analyzers may have already reported vulnerabilities that are being tracked in the [Vulnerability Report](../vulnerability_report/index.md).
-These vulnerabilities are linked to specific problematic code fragments so that they can be found and fixed.
-If the code fragments are not tracked reliably as they move, vulnerability management is harder because the same vulnerability could be reported again.
-
-GitLab SAST uses an advanced vulnerability tracking algorithm to more accurately identify when the same vulnerability has moved within a file due to refactoring or unrelated changes.
-
-Advanced vulnerability tracking is available in a subset of the [supported languages](#supported-languages-and-frameworks) and [analyzers](analyzers.md):
-
-- C, in the Semgrep-based analyzer only
-- Go, in the Gosec- and Semgrep-based analyzers
-- Java, in the Semgrep-based analyzer only
-- JavaScript, in the Semgrep-based analyzer only
-- Python, in the Semgrep-based analyzer only
-- Ruby, in the Brakeman-based analyzer
-
-Support for more languages and analyzers is tracked in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/5144).
-
### Using CI/CD variables to pass credentials for private repositories
Some analyzers require downloading the project's dependencies to
@@ -665,16 +677,16 @@ import the following default SAST analyzer images from `registry.gitlab.com` int
[local Docker container registry](../../packages/container_registry/index.md):
```plaintext
-registry.gitlab.com/security-products/brakeman:2
-registry.gitlab.com/security-products/flawfinder:2
-registry.gitlab.com/security-products/kubesec:2
-registry.gitlab.com/security-products/nodejs-scan:2
-registry.gitlab.com/security-products/phpcs-security-audit:2
-registry.gitlab.com/security-products/pmd-apex:2
-registry.gitlab.com/security-products/security-code-scan:2
-registry.gitlab.com/security-products/semgrep:2
-registry.gitlab.com/security-products/sobelow:2
-registry.gitlab.com/security-products/spotbugs:2
+registry.gitlab.com/security-products/brakeman:3
+registry.gitlab.com/security-products/flawfinder:3
+registry.gitlab.com/security-products/kubesec:3
+registry.gitlab.com/security-products/nodejs-scan:3
+registry.gitlab.com/security-products/phpcs-security-audit:3
+registry.gitlab.com/security-products/pmd-apex:3
+registry.gitlab.com/security-products/security-code-scan:3
+registry.gitlab.com/security-products/semgrep:3
+registry.gitlab.com/security-products/sobelow:3
+registry.gitlab.com/security-products/spotbugs:3
```
The process for importing Docker images into a local offline Docker registry depends on
diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md
index df6bb19ac25..8a066cf1be1 100644
--- a/doc/user/application_security/secret_detection/index.md
+++ b/doc/user/application_security/secret_detection/index.md
@@ -15,18 +15,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w
> `secret_detection_default_branch` and `secret_detection` were consolidated into one job,
> `secret_detection`.
-People may accidentally commit secrets to
-remote Git repositories. Secrets include keys, passwords, API tokens, and other sensitive
-information. Anyone with access to the repository could use the secrets for malicious purposes.
-Exposed secrets are compromised and must be replaced, which can be costly.
-
-To help prevent secrets from being committed to a Git repository, you can use Secret Detection
-to scan your repository for secrets. Scanning is language
-and framework agnostic, but does not support scanning binary files.
-
-Secret Detection uses a specific analyzer containing the
-[Gitleaks](https://github.com/zricethezav/gitleaks) tool to scan the repository for secrets, in a
-`secret-detection` job. The results are saved as a
+People may accidentally commit secrets (such as keys, passwords, and API tokens) to remote Git repositories.
+
+Anyone with access to the repository could use the secrets for malicious purposes. Exposed secrets
+must be considered compromised and be replaced, which can be costly.
+
+To help prevent secrets from being committed to a Git repository, you can use Secret Detection to
+scan your repository for secrets. Scanning is language and framework agnostic, but does not support
+scanning binary files.
+
+Secret Detection uses an analyzer containing the [Gitleaks](https://github.com/zricethezav/gitleaks)
+tool to scan the repository for secrets. Detection occurs in the `secret-detection` job. The results
+are saved as a
[Secret Detection report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportssecret_detection)
that you can later download and analyze. Due to implementation limitations, we always take the
latest Secret Detection artifact available.
@@ -38,7 +38,7 @@ All identified secrets are reported in the:
- Merge request widget
- Pipelines' **Security** tab
-- [Security Dashboard](../security_dashboard/index.md)
+- [Vulnerability Report](../vulnerability_report/index.md)
![Secret Detection in merge request widget](img/secret_detection_v13_2.png)
@@ -61,7 +61,7 @@ Different features are available in different [GitLab tiers](https://about.gitla
| Download [JSON Report](../sast/index.md#reports-json-format) | **{check-circle}** Yes | **{check-circle}** Yes |
| See new findings in the merge request widget | **{dotted-circle}** No | **{check-circle}** Yes |
| View identified secrets in the pipelines' **Security** tab | **{dotted-circle}** No | **{check-circle}** Yes |
-| [Manage vulnerabilities](../vulnerabilities/index.md) | **{dotted-circle}** No | **{check-circle}** Yes |
+| [Manage vulnerabilities](../vulnerability_report/index.md) | **{dotted-circle}** No | **{check-circle}** Yes |
| [Access the Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** No | **{check-circle}** Yes |
| [Customize Secret Detection rulesets](#custom-rulesets) | **{dotted-circle}** No | **{check-circle}** Yes |
@@ -88,13 +88,13 @@ To enable Secret Detection, either:
### Enable Secret Detection by including the template
-We recommend this method if you have an existing GitLab CI/CD configuration file.
+You should use this method if you have an existing GitLab CI/CD configuration file.
Add the following extract to your `.gitlab-ci.yml` file:
```yaml
include:
- - template: Security/Secret-Detection.gitlab-ci.yml
+ - template: Jobs/Secret-Detection.gitlab-ci.yml
```
Pipelines now include a Secret Detection job, and the results are included in the merge request
@@ -122,7 +122,34 @@ widget.
## Responding to a leaked secret
-If the scanner detects a secret we recommend you rotate it immediately. [Purging a file from the repository's history](../../project/repository/reducing_the_repo_size_using_git.md#purge-files-from-repository-history) may not be effective in removing all references to the file. Also, the secret remains in any forks of the repository.
+If the scanner detects a secret you should rotate it immediately. [Purging a file from the repository's history](../../project/repository/reducing_the_repo_size_using_git.md#purge-files-from-repository-history) may not be effective in removing all references to the file. Also, the secret remains in any forks of the repository.
+
+## Pinning to specific analyzer version
+
+The GitLab-managed CI/CD template specifies a major version and automatically pulls the latest analyzer release within that major version.
+
+In some cases, you may need to use a specific version.
+For example, you might need to avoid a regression in a later release.
+
+To override the automatic update behavior, set the `SECRETS_ANALYZER_VERSION` CI/CD variable
+in your CI/CD configuration file after you include the [`Secret-Detection.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Secret-Detection.gitlab-ci.yml).
+
+You can set the tag to:
+
+- A major version, like `4`. Your pipelines will use any minor or patch updates that are released within this major version.
+- A minor version, like `4.5`. Your pipelines will use any patch updates that are released within this minor version.
+- A patch version, like `4.5.0`. Your pipelines won't receive any updates.
+
+This example uses a specific minor version of the analyzer:
+
+```yaml
+include:
+ - template: Security/Secret-Detection.gitlab-ci.yml
+
+secret_detection:
+ variables:
+ SECRETS_ANALYZER_VERSION: "4.5"
+```
## Configure scan settings
@@ -154,11 +181,13 @@ secret_detection:
SECRET_DETECTION_HISTORIC_SCAN: "true"
```
-### Ignoring Secrets
+### Ignore secrets
-You might want to add a fake secret to your code base. For instance, you can use a fake secret as an example in your documentation or test suite.
+In some instances, you might want to ignore a secret. For example, you may have a fake secret in an
+example or a test suite. In these instances, you want to ignore the secret, instead of having it
+reported as a vulnerability.
-In these cases, Secret Detection can ignore the fake secret and not report it as a vulnerability. To ignore a secret, add `gitleaks:allow` as a comment to the line that contains the secret.
+To ignore a secret, add `gitleaks:allow` as a comment to the line that contains the secret.
For example:
@@ -172,7 +201,7 @@ Secret Detection can be customized by defining available CI/CD variables:
| CI/CD variable | Default value | Description |
|-----------------------------------|---------------|-------------|
-| `SECRET_DETECTION_EXCLUDED_PATHS` | "" | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs (see [`doublestar.Match`](https://pkg.go.dev/github.com/bmatcuk/doublestar/v4@v4.0.2#Match) for supported patterns), or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225273) in GitLab 13.3. |
+| `SECRET_DETECTION_EXCLUDED_PATHS` | "" | Exclude vulnerabilities from output based on the paths. The paths are a comma-separated list of patterns. Patterns can be globs (see [`doublestar.Match`](https://pkg.go.dev/github.com/bmatcuk/doublestar/v4@v4.0.2#Match) for supported patterns), or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225273) in GitLab 13.3. |
| `SECRET_DETECTION_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. |
| `SECRET_DETECTION_IMAGE_SUFFIX` | "" | Suffix added to the image name. If set to `-fips`, `FIPS-enabled` images are used for scan. See [Use FIPS-enabled images](#use-fips-enabled-images) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355519) in GitLab 14.10. |
| `SECRET_DETECTION_LOG_OPTIONS` | "" | [`git log`](https://git-scm.com/docs/git-log) options used to define commit ranges. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350660) in GitLab 15.1.|
@@ -214,7 +243,7 @@ By default, Secret Detection scans only the current state of the Git repository.
contained in the repository's history are not detected. To address this, Secret Detection can
scan the Git repository's full history.
-We recommend you do a full history scan only once, after enabling Secret Detection. A full history
+You should do a full history scan only once, after enabling Secret Detection. A full history
can take a long time, especially for larger repositories with lengthy Git histories. After
completing an initial full history scan, use only standard Secret Detection as part of your
pipeline.
@@ -349,15 +378,15 @@ In the `secret-detection-ruleset.toml` file, do one of the following:
## Running Secret Detection in an offline environment **(PREMIUM SELF)**
-For self-managed GitLab instances in an environment with limited, restricted, or intermittent access
-to external resources through the internet, some configuration changes are required for the Secret
-Detection job to run successfully. The instructions in this section must be completed together with
-the instructions detailed in [offline environments](../offline_deployments/index.md).
+An offline environment has limited, restricted, or intermittent access to external resources through
+the internet. For self-managed GitLab instances in such an environment, Secret Detection requires
+some configuration changes. The instructions in this section must be completed together with the
+instructions detailed in [offline environments](../offline_deployments/index.md).
### Configure GitLab Runner
By default, a runner tries to pull Docker images from the GitLab container registry even if a local
-copy is available. We recommend using this default setting, to ensure Docker images remain current.
+copy is available. You should use this default setting, to ensure Docker images remain current.
However, if no network connectivity is available, you must change the default GitLab Runner
`pull_policy` variable.
@@ -379,11 +408,11 @@ Prerequisites:
[local Docker container registry](../../packages/container_registry/index.md):
```plaintext
- registry.gitlab.com/security-products/secret-detection:3
+ registry.gitlab.com/security-products/secrets:4
```
The Secret Detection analyzer's image is [periodically updated](../index.md#vulnerability-scanner-maintenance)
- so you may need to periodically update the local copy.
+ so you should periodically update the local copy.
1. Set the CI/CD variable `SECURE_ANALYZERS_PREFIX` to the local Docker container registry.
@@ -454,14 +483,14 @@ secrets. If the number of commits in a merge request is greater than the value o
[`GIT_DEPTH` CI/CD variable](../../../ci/runners/configure_runners.md#shallow-cloning), Secret
Detection [fails to detect secrets](#error-couldnt-run-the-gitleaks-command-exit-status-2).
-For example, if a pipeline is triggered from a merge request containing 60 commits and the
-`GIT_DEPTH` variable's value is less than 60, the Secret Detection job fails as the clone is not
-deep enough to contain all of the relevant commits. To veridy the current value, see
+For example, you could have a pipeline triggered from a merge request containing 60 commits and the
+`GIT_DEPTH` variable set to less than 60. In that case the Secret Detection job fails because the
+clone is not deep enough to contain all of the relevant commits. To verify the current value, see
[pipeline configuration](../../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone).
-To confirm this as the cause of the error, set the [logging level](#set-the-logging-level) to `debug`, then
-rerun the pipeline. The logs should look similar to the following example. The text "object not
-found" is a symptom of this error.
+To confirm this as the cause of the error, set the [logging level](#set-the-logging-level) to
+`debug`, then rerun the pipeline. The logs should look similar to the following example. The text
+"object not found" is a symptom of this error.
```plaintext
ERRO[2020-11-18T18:05:52Z] object not found
diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md
index af98fc783e7..7c44b49b78c 100644
--- a/doc/user/application_security/security_dashboard/index.md
+++ b/doc/user/application_security/security_dashboard/index.md
@@ -153,7 +153,7 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/application_security/terminology/index.md b/doc/user/application_security/terminology/index.md
index 085a762fffa..f156d60be8f 100644
--- a/doc/user/application_security/terminology/index.md
+++ b/doc/user/application_security/terminology/index.md
@@ -230,7 +230,7 @@ support for cheap scan is proposed in issue [349926](https://gitlab.com/gitlab-o
### Pre-filter
-An irreversible action that is done to filter out target(s) before analysis occurs. This is usually provided to allow
+An irreversible action that is done to filter out targets before analysis occurs. This is usually provided to allow
the user to reduce scope and noise as well as speed up the analysis. This should not be done if a record is needed as
we currently do not store anything related to the skipped/excluded code or assets.
diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md
index 36f9578f999..e75d0a45f7d 100644
--- a/doc/user/application_security/vulnerabilities/severities.md
+++ b/doc/user/application_security/vulnerabilities/severities.md
@@ -42,15 +42,13 @@ the following tables:
| [`sobelow`](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) | **{check-circle}** Yes | Not applicable | Hardcodes all severity levels to `Unknown` |
| [`nodejs-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan) | **{check-circle}** Yes | String | `INFO`, `WARNING`, `ERROR` |
| [`flawfinder`](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) | **{check-circle}** Yes | Integer | `0`, `1`, `2`, `3`, `4`, `5` |
-| [`eslint`](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) | **{check-circle}** Yes | Not applicable | Hardcodes all severity levels to `Unknown` |
| [`SpotBugs`](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) | **{check-circle}** Yes | Integer | `1`, `2`, `3`, `11`, `12`, `18` |
-| [`gosec`](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) | **{check-circle}** Yes | String | `HIGH`, `MEDIUM`, `LOW` |
-| [`bandit`](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) | **{check-circle}** Yes | String | `HIGH`, `MEDIUM`, `LOW` |
| [`phpcs-security-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) | **{check-circle}** Yes | String | `ERROR`, `WARNING` |
| [`pmd-apex`](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) | **{check-circle}** Yes | Integer | `1`, `2`, `3`, `4`, `5` |
| [`kubesec`](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) | **{check-circle}** Yes | String | `CriticalSeverity`, `InfoSeverity` |
| [`secrets`](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) | **{check-circle}** Yes | Not applicable | Hardcodes all severity levels to `Critical` |
| [`semgrep`](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) | **{check-circle}** Yes | String | `error`, `warning`, `note`, `none` |
+| [`kics`](https://gitlab.com/gitlab-org/security-products/analyzers/kics) | **{check-circle}** Yes | String | `error`, `warning`, `note`, `none` (gets mapped to `info` in [analyzer version 3.7.0 and later](https://gitlab.com/gitlab-org/security-products/analyzers/kics/-/releases/v3.7.0)) |
## Dependency Scanning
diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md
index 59851fd192c..2b78dde4f63 100644
--- a/doc/user/application_security/vulnerability_report/index.md
+++ b/doc/user/application_security/vulnerability_report/index.md
@@ -216,6 +216,10 @@ Fields included are:
- [CVE](https://cve.mitre.org/) (Common Vulnerabilities and Exposures)
- [CWE](https://cwe.mitre.org/) (Common Weakness Enumeration)
- Other identifiers
+- Detected At
+- Location
+- Activity
+- Comments
NOTE:
Full details are available through our
diff --git a/doc/user/award_emojis.md b/doc/user/award_emojis.md
index d0aa9fa6119..5df38550c40 100644
--- a/doc/user/award_emojis.md
+++ b/doc/user/award_emojis.md
@@ -4,23 +4,23 @@ group: Project Management
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Award emoji **(FREE)**
+# Award emojis **(FREE)**
When you're collaborating online, you get fewer opportunities for high-fives
-and thumbs-ups. Emoji can be awarded to [issues](project/issues/index.md), [merge requests](project/merge_requests/index.md),
+and thumbs-ups. Emojis can be awarded to [issues](project/issues/index.md), [merge requests](project/merge_requests/index.md),
[snippets](snippets.md), and anywhere you can have a thread.
![Award emoji](img/award_emoji_select_v14_6.png)
-Award emoji make it much easier to give and receive feedback without a long
+Award emojis make it much easier to give and receive feedback without a long
comment thread.
For information on the relevant API, see [Award Emoji API](../api/award_emoji.md).
## Sort issues and merge requests on vote count
-You can quickly sort issues and merge requests by the number of votes they
-have received. The sort options can be found in the dropdown menu as "Most
+You can quickly sort issues and merge requests by the number of votes ("thumbs up" and "thumbs down" emoji) they
+have received. The sort options can be found in the dropdown list as "Most
popular" and "Least popular".
![Votes sort options](img/award_emoji_votes_sort_options.png)
@@ -29,9 +29,9 @@ The total number of votes is not summed up. An issue with 18 upvotes and 5
downvotes is considered more popular than an issue with 17 upvotes and no
downvotes.
-## Award emoji for comments
+## Award emojis for comments
-Award emoji can also be applied to individual comments when you want to
+Award emojis can also be applied to individual comments when you want to
celebrate an accomplishment or agree with an opinion.
To add an award emoji:
@@ -40,3 +40,15 @@ To add an award emoji:
1. Select an emoji from the dropdown list.
To remove an award emoji, select the emoji again.
+
+## Custom emojis
+
+You can upload custom emojis to a GitLab instance with the GraphQL API.
+For more, visit [Use custom emojis with GraphQL](../api/graphql/custom_emoji.md).
+
+Custom emojis don't show in the emoji picker.
+To use them in a text box, type the filename without the extension and surrounded by colons.
+For example, for a file named `thank-you.png`, type `:thank-you:`.
+
+For the list of custom emojis available for GitLab.com, visit
+[the `custom_emoji` project](https://gitlab.com/custom_emoji/custom_emoji/-/tree/main/img).
diff --git a/doc/user/clusters/agent/ci_cd_workflow.md b/doc/user/clusters/agent/ci_cd_workflow.md
index 7a3c09687a5..454be3c53c7 100644
--- a/doc/user/clusters/agent/ci_cd_workflow.md
+++ b/doc/user/clusters/agent/ci_cd_workflow.md
@@ -58,7 +58,8 @@ Authorization configuration can take one or two minutes to propagate.
### Authorize the agent to access your projects
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327850) in GitLab 14.4.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327850) in GitLab 14.4.
+> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/346566) to remove hierarchy restrictions in GitLab 15.6.
To authorize the agent to access the GitLab project where you keep Kubernetes manifests:
@@ -72,7 +73,7 @@ To authorize the agent to access the GitLab project where you keep Kubernetes ma
- id: path/to/project
```
- - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is.
+ - Authorized projects must have the same root group as the agent's configuration project.
- You can install additional agents into the same cluster to accommodate additional hierarchies.
- You can authorize up to 100 projects.
@@ -81,7 +82,8 @@ Choose the context to run `kubectl` commands from your CI/CD scripts.
### Authorize the agent to access projects in your groups
-> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3.
+> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3.
+> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/346566) to remove hierarchy restrictions in GitLab 15.6.
To authorize the agent to access all of the GitLab projects in a group or subgroup:
@@ -95,7 +97,7 @@ To authorize the agent to access all of the GitLab projects in a group or subgro
- id: path/to/group/subgroup
```
- - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is.
+ - Authorized groups must have the same root group as the agent's configuration project.
- You can install additional agents into the same cluster to accommodate additional hierarchies.
- All of the subgroups of an authorized group also have access to the same agent (without being specified individually).
- You can authorize up to 100 groups.
@@ -125,6 +127,30 @@ deploy:
If you are not sure what your agent's context is, open a terminal and connect to your cluster.
Run `kubectl config get-contexts`.
+### Environments that use Auto DevOps
+
+If Auto DevOps is enabled, you must define the CI/CD variable `KUBE_CONTEXT`.
+Set the value of `KUBE_CONTEXT` to the context of the agent you want Auto DevOps to use:
+
+```yaml
+deploy:
+ variables:
+ KUBE_CONTEXT: <path_to_agent_config_repository>:<agent_name>
+```
+
+You can assign different agents to separate Auto DevOps jobs. For instance,
+Auto DevOps can use one agent for `staging` jobs, and another agent for `production` jobs.
+To use multiple agents, define an [environment-scoped CI/CD variable](../../../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable)
+for each agent. For example:
+
+1. Define two variables named `KUBE_CONTEXT`.
+1. For the first variable:
+ 1. Set the `environment` to `staging`.
+ 1. Set the value to the context of your staging agent.
+1. For the second variable:
+ 1. Set the `environment` to `production`.
+ 1. Set the value to the context of your production agent.
+
### Environments with both certificate-based and agent-based connections
When you deploy to an environment that has both a
@@ -156,20 +182,6 @@ deploy:
# ... rest of your job configuration
```
-### Using the agent with Auto DevOps
-
-If Auto DevOps is enabled, you must define the `KUBE_CONTEXT` CI/CD variable. Set the value of `KUBE_CONTEXT` to the context of the agent you want to use in your Auto DevOps pipeline jobs (`<PATH_TO_AGENT_CONFIG_REPOSITORY>:<AGENT_NAME>`).
-
-You can also use different agents for different Auto DevOps jobs. For instance, you can use one agent for `staging` jobs and a different agent for `production` jobs. To use multiple agents, define a unique CI/CD variable for each agent.
-
-For example:
-
-1. Add two [environment-scoped CI/CD variables](../../../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable) and name both `KUBE_CONTEXT`.
-1. Set the `environment` of the first variable to `staging`. Set the value of the variable to `<PATH_TO_AGENT_CONFIGURATION_PROJECT>:<STAGING_AGENT_NAME>`.
-1. Set the `environment` of the second variable to `production`. Set the value of the variable to `<PATH_TO_AGENT_CONFIGURATION_PROJECT>:<PRODUCTION_AGENT_NAME>`.
-
-When the `staging` job runs, it will connect to the cluster via the agent named `<STAGING_AGENT_NAME>`, and when the `production` job runs it will connect to the cluster via the agent named `<PRODUCTION_AGENT_NAME>`.
-
## Restrict project and group access by using impersonation **(PREMIUM)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345014) in GitLab 14.5.
@@ -281,13 +293,6 @@ See the [official Kubernetes documentation for details](https://kubernetes.io/do
## Troubleshooting
-### `kubectl` commands not supported
-
-The commands `kubectl exec`, `kubectl cp`, `kubectl attach`, `kubectl run --attach=true` and `kubectl port-forward` are not supported.
-Anything that uses these API endpoints does not work, because they use the deprecated
-SPDY protocol.
-[An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/346248) to add support for these commands.
-
### Grant write permissions to `~/.kube/cache`
Tools like `kubectl`, Helm, `kpt`, and `kustomize` cache information about
diff --git a/doc/user/clusters/agent/gitops/helm.md b/doc/user/clusters/agent/gitops/helm.md
index af9f80618b5..0ec87376636 100644
--- a/doc/user/clusters/agent/gitops/helm.md
+++ b/doc/user/clusters/agent/gitops/helm.md
@@ -54,16 +54,50 @@ gitops:
path: dir-in-project/with/charts
namespace: my-ns
max_history: 1
+ values:
+ - inline:
+ someKey: example value
```
| Keyword | Description |
|--|--|
| `charts` | List of charts you want to be applied in your cluster. Charts are applied concurrently. |
| `release_name` | Required. Name of the release to use when applying the chart. |
-| `id` | Required. ID of the project where Helm chart is committed. No authentication mechanisms are currently supported. |
-| `path` | Optional. Path of the chart in the project repository. Root of the repository is used by default. This is the directory with the `Chart.yaml` file. |
+| `values` | Optional. [Custom values](#custom-values) for the release. An array of objects. Only supports `inline` values. |
| `namespace` | Optional. Namespace to use when applying the chart. Defaults to `default`. |
| `max_history` | Optional. Maximum number of release [revisions to store in the cluster](https://helm.sh/docs/helm/helm_history/). |
+| `source` | Required. From where the chart should get installed. Only supports project sources. |
+| `source.project.id` | Required. ID of the project where Helm chart is committed. Authentication is not supported. |
+| `source.project.path` | Optional. Path of the chart in the project repository. Root of the repository is used by default. Should be the directory with the `Chart.yaml` file. |
+
+## Custom values
+
+> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/merge_requests/766) in GitLab 15.6. Requires both GitLab and the installed agent to be version 15.6 or later.
+
+To customize the values for a release, set the `values` key. It must be
+an array of objects. Each object must have exactly one top-level key that describes
+where the values come from. The supported top-level keys are:
+
+- `inline`: Specify the values inline in YAML format, similar to a Helm values
+ file.
+
+When installing a chart with custom values:
+
+- Custom values get merged on top of the chart's default `values.yaml` file.
+- Values from subsequent entries in the `values` array overwrite values from
+ previous entries.
+
+Example:
+
+```yaml
+gitops:
+ charts:
+ - release_name: some-release
+ values:
+ - inline:
+ someKey: example value
+ # ...
+```
## Automatic drift remediation
@@ -98,7 +132,7 @@ The following are known issues:
[this epic](https://gitlab.com/groups/gitlab-org/-/epics/7704).
- Values for the chart must be in a `values.yaml` file. This file must be with the chart,
in the same project and path.
-- Because of drift detection and remediation, release history, stored in the cluster, is not useful.
+- Because of drift detection and remediation, the release history stored in the cluster is not useful.
A new release is created every five minutes and the oldest release is discarded.
Eventually history consists only of the same information.
View [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/372023) for details.
diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md
index 7fdf0bb2bf0..8f4855a7f93 100644
--- a/doc/user/clusters/agent/index.md
+++ b/doc/user/clusters/agent/index.md
@@ -69,14 +69,15 @@ This workflow has a weaker security model and is not recommended for production
GitLab supports the following Kubernetes versions. You can upgrade your
Kubernetes version to a supported version at any time:
-- 1.24 (support ends on September 22, 2023 or when 1.27 becomes supported)
+- 1.25 (support ends on October 22, 2023 or when 1.28 becomes supported)
+- 1.24 (support ends on July 22, 2023 or when 1.27 becomes supported)
- 1.23 (support ends on February 22, 2023 or when 1.26 becomes supported)
-- 1.22 (support ends on October 22, 2022)
-- 1.21 (support ends on August 22, 2022)
GitLab aims to support a new minor Kubernetes version three months after its initial release. GitLab supports at least three production-ready Kubernetes minor
versions at any given time.
+When installing the agent, use a Helm version compatible with your Kubernetes version. Other versions of Helm might not work. For a list of compatible versions, see the [Helm version support policy](https://helm.sh/docs/topics/version_skew/).
+
Support for deprecated APIs can be removed from the GitLab codebase when we drop support for the Kubernetes version that only supports the deprecated API.
Some GitLab features might work on versions not listed here. [This epic](https://gitlab.com/groups/gitlab-org/-/epics/4827) tracks support for Kubernetes versions.
diff --git a/doc/user/clusters/agent/install/index.md b/doc/user/clusters/agent/install/index.md
index 19628419784..2030052e3b0 100644
--- a/doc/user/clusters/agent/install/index.md
+++ b/doc/user/clusters/agent/install/index.md
@@ -45,6 +45,8 @@ For configuration settings, the agent uses a YAML file in the GitLab project. Yo
- You use [a GitOps workflow](../gitops.md#gitops-workflow-steps).
- You use [a GitLab CI/CD workflow](../ci_cd_workflow.md#gitlab-cicd-workflow-steps) and want to authorize a different project to use the agent.
+Otherwise it is optional.
+
To create an agent configuration file:
1. Choose a name for your agent. The agent name follows the
@@ -90,7 +92,7 @@ You must register an agent before you can install the agent in your cluster. To
- If you already have an [agent configuration file](#create-an-agent-configuration-file), select it from the list.
1. Select **Register an agent**.
1. GitLab generates an access token for the agent. You need this token to install the agent
- in your cluster and to [update the agent](#update-the-agent-version) to another version.
+ in your cluster.
WARNING:
Securely store the agent access token. A bad actor can use this token to access source code in the agent's configuration project, access source code in any public project on the GitLab instance, or even, under very specific conditions, obtain a Kubernetes manifest.
diff --git a/doc/user/clusters/agent/troubleshooting.md b/doc/user/clusters/agent/troubleshooting.md
index 0c26c533cc3..6b5f58f8f76 100644
--- a/doc/user/clusters/agent/troubleshooting.md
+++ b/doc/user/clusters/agent/troubleshooting.md
@@ -208,3 +208,24 @@ kubectl delete jobs -l app.kubernetes.io/managed-by=starboard -n gitlab-agent
```
[We're working on making the cleanup of these jobs more robust.](https://gitlab.com/gitlab-org/gitlab/-/issues/362016)
+
+## Inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)
+
+```json
+{
+ "error":"inventory policy prevented actuation (strategy: Apply, status: Empty, policy: MustMatch)",
+ "group":"networking.k8s.io",
+ "kind":"Deployment",
+ "name":"resource-name",
+ "namespace":"namespace",
+ "status":"Skipped",
+ "timestamp":"2022-10-29T15:34:21Z",
+ "type":"apply"
+}
+```
+
+This error occurs when the GitLab agent tries to update an object and the object doesn't have the required annotations. To fix this error, you can:
+
+- Add the required annotations manually.
+- Delete the object and let the agent recreate it.
+- Change your [`inventory_policy`](../../infrastructure/clusters/deploy/inventory_object.md#inventory_policy-options) setting.
diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md
index df338e8fcee..18317ae09ed 100644
--- a/doc/user/clusters/management_project.md
+++ b/doc/user/clusters/management_project.md
@@ -63,7 +63,7 @@ To associate a cluster management project with your cluster:
page.
- [Instance-level cluster](../instance/clusters/index.md), on the top bar, select **Main menu > Admin > Kubernetes**.
1. Expand **Advanced settings**.
-1. From the **Cluster management project** dropdown, select the cluster management project
+1. From the **Cluster management project** dropdown list, select the cluster management project
you created in the previous step.
### Configuring your pipeline
diff --git a/doc/user/clusters/management_project_template.md b/doc/user/clusters/management_project_template.md
index bdd11f11f9c..cbe577b9b74 100644
--- a/doc/user/clusters/management_project_template.md
+++ b/doc/user/clusters/management_project_template.md
@@ -103,7 +103,6 @@ The [built-in supported applications](https://gitlab.com/gitlab-org/project-temp
- [GitLab Runner](../infrastructure/clusters/manage/management_project_applications/runner.md)
- [Ingress](../infrastructure/clusters/manage/management_project_applications/ingress.md)
- [Prometheus](../infrastructure/clusters/manage/management_project_applications/prometheus.md)
-- [Sentry](../infrastructure/clusters/manage/management_project_applications/sentry.md)
- [Vault](../infrastructure/clusters/manage/management_project_applications/vault.md)
Each application has an `applications/{app}/values.yaml` file.
diff --git a/doc/user/compliance/compliance_report/index.md b/doc/user/compliance/compliance_report/index.md
index ac4b20b5166..0d33dfce30b 100644
--- a/doc/user/compliance/compliance_report/index.md
+++ b/doc/user/compliance/compliance_report/index.md
@@ -94,9 +94,7 @@ Our criteria for the separation of duties is as follows:
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213364) in GitLab 13.3.
> - Chain of Custody reports sent using email [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342594) in GitLab 15.3 with a flag named `async_chain_of_custody_report`. Disabled by default.
-
-FLAG:
-On self-managed GitLab, by default sending Chain of Custody reports through email is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `async_chain_of_custody_report`. On GitLab.com, this feature is not available.
+> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/370100) in GitLab 15.5. Feature flag `async_chain_of_custody_report` removed.
The Chain of Custody report allows customers to export a list of merge commits within the group.
The data provides a comprehensive view with respect to merge commits. It includes the merge commit SHA,
@@ -112,7 +110,7 @@ To generate the Chain of Custody report:
The Chain of Custody report is either:
- Available for download.
-- Sent through email. Requires GitLab 15.3 and later with `async_chain_of_custody_report` feature flag enabled.
+- Sent through email. Requires GitLab 15.5 and later.
### Commit-specific Chain of Custody report
@@ -131,7 +129,7 @@ Authenticated group owners can generate a commit-specific Chain of Custody repor
The Chain of Custody report is either:
- Available for download.
-- Sent through email. Requires GitLab 15.3 and later with `async_chain_of_custody_report` feature flag enabled.
+- Sent through email. Requires GitLab 15.5 and later.
- Using a direct link: `https://gitlab.com/groups/<group-name>/-/security/merge_commit_reports.csv?commit_sha={optional_commit_sha}`, passing in an optional value to the
`commit_sha` query parameter.
diff --git a/doc/user/crm/index.md b/doc/user/crm/index.md
index d7ab2195e2d..2d1aeaa1c07 100644
--- a/doc/user/crm/index.md
+++ b/doc/user/crm/index.md
@@ -79,6 +79,21 @@ To edit an existing contact:
You can also [edit](../../api/graphql/reference/index.md#mutationcustomerrelationscontactupdate)
contacts using the GraphQL API.
+#### Change the state of a contact
+
+Each contact can be in one of two states:
+
+- **Active**: contacts in this state can be added to an issue.
+- **Inactive**: contacts in this state cannot be added to an issue.
+
+To change the state of a contact:
+
+1. On the top bar, select **Main menu > Groups** and find your group.
+1. On the left sidebar, select **Customer relations > Contacts**.
+1. Next to the contact you wish to edit, select **Edit** (**{pencil}**).
+1. Select or clear the **Active** checkbox.
+1. Select **Save changes**.
+
## Organizations
### View organizations
@@ -153,7 +168,7 @@ API.
### Add contacts to an issue
-To add contacts to an issue use the `/add_contacts [contact:address@example.com]`
+To add [active](#change-the-state-of-a-contact) contacts to an issue use the `/add_contacts [contact:address@example.com]`
[quick action](../project/quick_actions.md).
You can also add, remove, or replace issue contacts using the
@@ -175,10 +190,15 @@ API.
> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/352123) in GitLab 15.0.
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352123) in GitLab 15.2. [Feature flag `contacts_autocomplete`](https://gitlab.com/gitlab-org/gitlab/-/issues/352123) removed.
-When you use the `/add_contacts` or `/remove_contacts` quick actions, follow them with `[contact:` and an autocomplete list appears:
+When you use the `/add_contacts` quick action, follow it with `[contact:` and an autocomplete list with the [active](#change-the-state-of-a-contact) contacts appears:
```plaintext
/add_contacts [contact:
+```
+
+When you use the `/remove_contacts` quick action, follow it with `[contact:` and an autocomplete list with the contacts added to the issue appears:
+
+```plaintext
/remove_contacts [contact:
```
diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md
index 13d5b27ad41..d9cacb6395d 100644
--- a/doc/user/discussions/index.md
+++ b/doc/user/discussions/index.md
@@ -44,8 +44,9 @@ You can quickly see which comments involve you, because
mentions for yourself (the user currently signed in) are highlighted
in a different color.
-Avoid mentioning `@all` in issues and merge requests, because it sends an email notification
-to all the members of that project's group. This might be interpreted as spam.
+Avoid mentioning `@all` in issues and merge requests. It sends an email notification
+to all members of that project's parent group, not only the participants of the project,
+and may be interpreted as spam.
Notifications and mentions can be disabled in
[a group's settings](../group/manage.md#disable-email-notifications).
@@ -77,7 +78,7 @@ To add a commit diff comment:
You can select multiple lines by dragging the **Comment** (**{comment}**) icon.
1. Enter your comment and select **Start a review** or **Add comment now**.
-The comment is displayed on the merge request's **Discussions** tab.
+The comment is displayed on the merge request's **Overview** tab.
The comment is not displayed on your project's **Repository > Commits** page.
@@ -143,7 +144,7 @@ If you edit an existing comment to add a user mention that wasn't there before,
- Creates a to-do item for the mentioned user.
- Does not send a notification email.
-## Prevent comments by locking an issue
+## Prevent comments by locking the discussion
You can prevent public comments in an issue or merge request.
When you do, only project members can add and edit comments.
@@ -153,6 +154,8 @@ Prerequisite:
- In merge requests, you must have at least the Developer role.
- In issues, you must have at least the Reporter role.
+To lock an issue or merge request:
+
1. On the right sidebar, next to **Lock issue** or **Lock merge request**, select **Edit**.
1. On the confirmation dialog, select **Lock**.
@@ -160,6 +163,9 @@ Notes are added to the page details.
If an issue or merge request is locked and closed, you cannot reopen it.
+<!-- Delete when the `moved_mr_sidebar` feature flag is removed -->
+If you don't see this action on the right sidebar, your project or instance might have [moved sidebar actions](../project/merge_requests/index.md#move-sidebar-actions) enabled.
+
## Add an internal note
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207473) in GitLab 13.9 [with a flag](../../administration/feature_flags.md) named `confidential_notes`. Disabled by default.
@@ -167,12 +173,9 @@ If an issue or merge request is locked and closed, you cannot reopen it.
> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87403) from "confidential comments" to "internal notes" in GitLab 15.0.
> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87383) in GitLab 15.0.
> - [Feature flag `confidential_notes`](https://gitlab.com/gitlab-org/gitlab/-/issues/362712) removed in GitLab 15.2.
+> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/363045) permissions in GitLab 15.5 to at least the Reporter role. In GitLab 15.4 and earlier, issue or epic authors and assignees could also read and create internal notes.
-You can add an internal note **to an issue or an epic**. It's then visible only to the following people:
-
-- Project members who have at least the Reporter role
-- Issue or epic author
-- Users assigned to the issue or epic
+You can add an internal note **to an issue or an epic**. It's then visible only to project members who have at least the Reporter role.
Keep in mind:
@@ -181,10 +184,7 @@ Keep in mind:
Prerequisites:
-- You must either:
- - Have at least the Reporter role for the project.
- - Be the issue or epic assignee.
- - Be the issue or epic author.
+- You must have at least the Reporter role for the project.
To add an internal note:
@@ -200,14 +200,14 @@ You can also mark an [issue as confidential](../project/issues/confidential_issu
For issues and merge requests with many comments, you can filter the page to show comments only.
-1. Open a merge request's **Discussion** tab, or epic or issue's **Overview** tab.
+1. Open the **Overview** tab in a merge request, issue, or epic.
1. On the right side of the page, select from the filter:
- **Show all activity**: Display all user comments and system notes.
(issue updates, mentions from other issues, changes to the description, and so on).
- **Show comments only**: Display only user comments.
- **Show history only**: Display only activity notes.
-![Notes filters dropdown options](img/index_notes_filters.png)
+![Notes filters dropdown list options](img/index_notes_filters.png)
GitLab saves your preference, so it persists when you visit the same page again
from any device you're logged into.
diff --git a/doc/user/feature_highlight.md b/doc/user/feature_highlight.md
deleted file mode 100644
index ef96d2524a6..00000000000
--- a/doc/user/feature_highlight.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: 'index.md'
-remove_date: '2022-10-29'
----
-
-This document was moved to [another location](index.md).
-
-<!-- This redirect file can be deleted after <2022-10-29>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/user/free_user_limit.md b/doc/user/free_user_limit.md
index 3fbfb2e1aa7..35777847947 100644
--- a/doc/user/free_user_limit.md
+++ b/doc/user/free_user_limit.md
@@ -29,6 +29,34 @@ If you need more time to manage your members, or to try GitLab features
with a team of more than five members, you can [start a trial](https://gitlab.com/-/trial_registrations/new?glm_source=docs.gitlab.com&glm_content=free-user-limit).
A trial lasts for 30 days and includes an unlimited number of members.
+## Determining namespace user counts
+
+Every unique user of a top-level namespace with private visibility counts towards the five-user limit. This includes every user of a group, subgroup, and project within a namespace.
+
+For example:
+
+The group `example-1` has:
+
+- One group owner, `A`.
+- One subgroup called `subgroup-1` with one member, `B`.
+ - `subgroup-1` inherits `A` as a member from `example-1`.
+- One project in `subgroup-1` called `project-1` with two members, `C` and `D`.
+ - `project-1` inherits `A` and `B` as members from `subgroup-1`.
+
+The namespace `example-1` has four unique members: `A`, `B`, `C`, and `D`. Because `example-1` has only four unique members, it is not impacted by the five-user limit.
+
+The group `example-2` has:
+
+- One group owner, `A`.
+- One subgroup called `subgroup-2` with one member, `B`.
+ - `subgroup-2` inherits `A` as a member from `example-2`.
+- One project in `subgroup-2` called `project-2a` with two members, `C` and `D`.
+ - `project-2a` inherits `A` and `B` as members from `subgroup-2`.
+- One project in `subgroup-2` called `project-2b` with two members, `E` and `F`.
+ - `project-2b` inherits `A` and `B` as members from `subgroup-2`.
+
+The namespace `example-2` has six unique members: `A`, `B`, `C`, `D`, `E`, and `F`. Because `example-2` has six unique users, it is impacted by the five-user limit.
+
## Related topics
- [GitLab SaaS Free tier frequently asked questions](https://about.gitlab.com/pricing/faq-efficient-free-tier/)
diff --git a/doc/user/group/access_and_permissions.md b/doc/user/group/access_and_permissions.md
index 395ed3c91c7..13a1fd31ee4 100644
--- a/doc/user/group/access_and_permissions.md
+++ b/doc/user/group/access_and_permissions.md
@@ -56,18 +56,20 @@ To change the permitted Git access protocols for a group:
1. Choose the permitted protocols from **Enabled Git access protocols**.
1. Select **Save changes**.
-## Restrict group access by IP address **(PREMIUM)**
+## Restrict access to groups by IP address **(PREMIUM)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1985) in GitLab 12.0.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/215410) from GitLab Ultimate to GitLab Premium in 13.1.
-To ensure only people from your organization can access particular
-resources, you can restrict access to groups by IP address. This group-level setting
-applies to:
+To ensure only people from your organization can access particular resources, you can restrict access to groups by IP
+address. This group-level setting applies to:
- The GitLab UI, including subgroups, projects, and issues.
- [In GitLab 12.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/12874), the API.
+Administrators can combine restricted access by IP address with
+[globally-allowed IP addresses](../admin_area/settings/visibility_and_access_controls.md#configure-globally-allowed-ip-address-ranges).
+
### Security implications
You should consider some security implications before configuring IP address restrictions.
@@ -94,7 +96,12 @@ To restrict group access by IP address:
1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Settings > General**.
1. Expand the **Permissions and group features** section.
-1. In the **Restrict access by IP address** field, enter IPv4 or IPv6 address ranges in CIDR notation.
+1. In the **Restrict access by IP address** field, enter a list of IPv4 or IPv6
+ address ranges in CIDR notation. This list:
+ - Has no limit on the number of IP address ranges.
+ - Has a size limit of 1 GB.
+ - Applies to both SSH or HTTP authorized IP address ranges. You cannot split
+ this list by type of authorization.
1. Select **Save changes**.
In self-managed installations of GitLab 15.1 and later, you can also configure
@@ -290,5 +297,4 @@ If a user sees a 404 when they would normally expect access, and the problem is
- `json.message`: `'Attempting to access IP restricted group'`
- `json.allowed`: `false`
-In viewing the log entries, compare the `remote.ip` with the list of
-[allowed IPs](#restrict-group-access-by-ip-address) for the group.
+In viewing the log entries, compare the `remote.ip` with the list of [allowed IP addresses](#restrict-access-to-groups-by-ip-address) for the group.
diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md
index c6cc828302f..62f5a3ba54f 100644
--- a/doc/user/group/clusters/index.md
+++ b/doc/user/group/clusters/index.md
@@ -187,6 +187,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/group/compliance_frameworks.md b/doc/user/group/compliance_frameworks.md
new file mode 100644
index 00000000000..7bd545003db
--- /dev/null
+++ b/doc/user/group/compliance_frameworks.md
@@ -0,0 +1,262 @@
+---
+stage: Govern
+group: Compliance
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Compliance frameworks **(PREMIUM)**
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276221) in GitLab 13.9.
+> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/287779) in GitLab 13.12.
+
+You can create a compliance framework that is a label to identify that your project has certain compliance
+requirements or needs additional oversight. The label can optionally enforce
+[compliance pipeline configuration](#configure-a-compliance-pipeline) to the projects on which it is
+[applied](../project/settings/index.md#add-a-compliance-framework-to-a-project).
+
+Group owners can create, edit, and delete compliance frameworks:
+
+1. On the top bar, select **Main menu > Groups > View all groups** and find your group.
+1. On the left sidebar, select **Settings** > **General**.
+1. Expand the **Compliance frameworks** section.
+1. Create, edit, or delete compliance frameworks.
+
+## Set a default compliance framework
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375036) in GitLab 15.6.
+
+Group owners can set a default compliance framework. The default framework is applied to all the new projects
+that are created within that group. It does not affect the framework applied to the existing projects. The default
+framework cannot be deleted.
+
+### Example GraphQL mutations for setting a default compliance framework
+
+Creating a new compliance framework and setting it as the default framework for the group.
+
+```graphql
+mutation {
+ createComplianceFramework(
+ input: {params: {name: "SOX", description: "Sarbanes-Oxley Act", color: "#87CEEB", default: true}, namespacePath: "gitlab-org"}
+ ) {
+ framework {
+ id
+ name
+ default
+ description
+ color
+ pipelineConfigurationFullPath
+ }
+ errors
+ }
+}
+```
+
+Setting an existing compliance framework as the default framework the group.
+
+```graphql
+mutation {
+ updateComplianceFramework(
+ input: {id: "gid://gitlab/ComplianceManagement::Framework/<id>", params: {default: true}}
+ ) {
+ complianceFramework {
+ id
+ name
+ default
+ description
+ color
+ pipelineConfigurationFullPath
+ }
+ }
+}
+```
+
+## Configure a compliance pipeline **(ULTIMATE)**
+
+> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3156) in GitLab 13.9, disabled behind `ff_evaluate_group_level_compliance_pipeline` [feature flag](../../administration/feature_flags.md).
+> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/300324) in GitLab 13.11.
+> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/331231) in GitLab 14.2.
+
+Group owners can configure a compliance pipeline in a project separate to other projects. By default, the compliance
+pipeline configuration (`.gitlab-ci.yml` file) is run instead of the pipeline configuration of labeled projects.
+
+However, the compliance pipeline configuration can reference the `.gitlab-ci.yml` file of the labeled projects so that:
+
+- The compliance pipeline can also run jobs of labeled project pipelines. This allows for centralized control of
+ pipeline configuration.
+- Jobs and variables defined in the compliance pipeline can't be changed by variables in the labeled project's
+ `.gitlab-ci.yml` file.
+
+See [example configuration](#example-configuration) for help configuring a compliance pipeline that runs jobs from
+labeled project pipeline configuration.
+
+To configure a compliance pipeline:
+
+1. On the top bar, select **Main menu > Groups > View all groups** and find your group.
+1. On the left sidebar, select **Settings** > **General**.
+1. Expand the **Compliance frameworks** section.
+1. In **Compliance pipeline configuration (optional)**, add the path to the compliance framework configuration. Use the
+ `path/file.y[a]ml@group-name/project-name` format. For example:
+
+ - `.compliance-ci.yml@gitlab-org/gitlab`.
+ - `.compliance-ci.yaml@gitlab-org/gitlab`.
+
+This configuration is inherited by projects where the compliance framework label is
+[applied](../project/settings/index.md#add-a-compliance-framework-to-a-project). In projects with the applied compliance
+framework label, the compliance pipeline configuration is run instead of the labeled project's own pipeline configuration.
+
+The user running the pipeline in the labeled project must at least have the Reporter role on the compliance project.
+
+When used to enforce scan execution, this feature has some overlap with
+[scan execution policies](../application_security/policies/scan-execution-policies.md). We have not
+[unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312). For details on
+the similarities and differences between these features, see [Enforce scan execution](../application_security/index.md#enforce-scan-execution).
+
+### Example configuration
+
+The following example `.compliance-gitlab-ci.yml` includes the `include` keyword to ensure labeled project pipeline
+configuration is also executed.
+
+```yaml
+# Allows compliance team to control the ordering and interweaving of stages/jobs.
+# Stages without jobs defined will remain hidden.
+stages:
+ - pre-compliance
+ - build
+ - test
+ - pre-deploy-compliance
+ - deploy
+ - post-compliance
+
+variables: # Can be overridden by setting a job-specific variable in project's local .gitlab-ci.yml
+ FOO: sast
+
+sast: # None of these attributes can be overridden by a project's local .gitlab-ci.yml
+ variables:
+ FOO: sast
+ image: ruby:2.6
+ stage: pre-compliance
+ rules:
+ - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"
+ when: never
+ - when: always # or when: on_success
+ allow_failure: false
+ before_script:
+ - "# No before scripts."
+ script:
+ - echo "running $FOO"
+ after_script:
+ - "# No after scripts."
+
+sanity check:
+ image: ruby:2.6
+ stage: pre-deploy-compliance
+ rules:
+ - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"
+ when: never
+ - when: always # or when: on_success
+ allow_failure: false
+ before_script:
+ - "# No before scripts."
+ script:
+ - echo "running $FOO"
+ after_script:
+ - "# No after scripts."
+
+audit trail:
+ image: ruby:2.7
+ stage: post-compliance
+ rules:
+ - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"
+ when: never
+ - when: always # or when: on_success
+ allow_failure: false
+ before_script:
+ - "# No before scripts."
+ script:
+ - echo "running $FOO"
+ after_script:
+ - "# No after scripts."
+
+include: # Execute individual project's configuration (if project contains .gitlab-ci.yml)
+ project: '$CI_PROJECT_PATH'
+ file: '$CI_CONFIG_PATH'
+ ref: '$CI_COMMIT_REF_NAME' # Must be defined or MR pipelines always use the use default branch
+```
+
+#### CF pipelines in Merge Requests originating in project forks
+
+When an MR originates in a fork, the branch to be merged usually only exists in the fork.
+When creating such an MR against a project with CF pipelines, the above snippet will fail with a
+`Project <project-name> reference <branch-name> does not exist!` error message.
+This is because in the context of the target project, `$CI_COMMIT_REF_NAME` evaluates to a non-existing branch name.
+
+To get the correct context, use `$CI_MERGE_REQUEST_SOURCE_PROJECT_PATH` instead of `$CI_PROJECT_PATH`.
+This variable is only availabe in
+[merge request pipelines](../../ci/pipelines/merge_request_pipelines.md).
+
+For example, for a configuration that supports both merge request pipelines originating in project forks and branch pipelines,
+you need to [combine both `include` directives with `rules:if`](../../ci/yaml/includes.md#use-rules-with-include):
+
+```yaml
+include: # Execute individual project's configuration (if project contains .gitlab-ci.yml)
+ - project: '$CI_MERGE_REQUEST_SOURCE_PROJECT_PATH'
+ file: '$CI_CONFIG_PATH'
+ ref: '$CI_COMMIT_REF_NAME'
+ rules:
+ - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
+ - project: '$CI_PROJECT_PATH'
+ file: '$CI_CONFIG_PATH'
+ ref: '$CI_COMMIT_REF_NAME'
+ rules:
+ - if: $CI_PIPELINE_SOURCE != 'merge_request_event'
+```
+
+## Ensure compliance jobs are always run
+
+Compliance pipelines [use GitLab CI/CD](../../ci/index.md) to give you an incredible amount of flexibility
+for defining any sort of compliance jobs you like. Depending on your goals, these jobs
+can be configured to be:
+
+- Modified by users.
+- Non-modifiable.
+
+Generally, if a value in a compliance job:
+
+- Is set, it cannot be changed or overridden by project-level configurations.
+- Is not set, a project-level configuration may set.
+
+Either might be wanted or not depending on your use case.
+
+There are a few best practices for ensuring that these jobs are always run exactly
+as you define them and that downstream, project-level pipeline configurations
+cannot change them:
+
+- Add [a `rules:when:always` block](../../ci/yaml/index.md#when) to each of your compliance jobs. This ensures they are
+ non-modifiable and are always run.
+- Explicitly set any [variables](../../ci/yaml/index.md#variables) the job references. This:
+ - Ensures that project-level pipeline configurations do not set them and alter their
+ behavior.
+ - Includes any jobs that drive the logic of your job.
+- Explicitly set the [container image](../../ci/yaml/index.md#image) to run the job in. This ensures that your script
+ steps execute in the correct environment.
+- Explicitly set any relevant GitLab pre-defined [job keywords](../../ci/yaml/index.md#job-keywords).
+ This ensures that your job uses the settings you intend and that they are not overridden by
+ project-level pipelines.
+
+## Avoid parent and child pipelines in GitLab 14.7 and earlier
+
+NOTE:
+This advice does not apply to GitLab 14.8 and later because [a fix](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78878) added
+compatibility for combining compliance pipelines, and parent and child pipelines.
+
+Compliance pipelines start on the run of _every_ pipeline in a labeled project. This means that if a pipeline in the labeled project
+triggers a child pipeline, the compliance pipeline runs first. This can trigger the parent pipeline, instead of the child pipeline.
+
+Therefore, in projects with compliance frameworks, we recommend replacing
+[parent-child pipelines](../../ci/pipelines/downstream_pipelines.md#parent-child-pipelines) with the following:
+
+- Direct [`include`](../../ci/yaml/index.md#include) statements that provide the parent pipeline with child pipeline configuration.
+- Child pipelines placed in another project that are run using the [trigger API](../../ci/triggers/index.md) rather than the parent-child
+ pipeline feature.
+
+This alternative ensures the compliance pipeline does not re-start the parent pipeline.
diff --git a/doc/user/group/contribution_analytics/index.md b/doc/user/group/contribution_analytics/index.md
index 280781a4cad..b1efd2e9251 100644
--- a/doc/user/group/contribution_analytics/index.md
+++ b/doc/user/group/contribution_analytics/index.md
@@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3090) in GitLab 12.2 for subgroups.
-With Contribution Analytics, you can get an overview of the [contribution events](../../profile/index.md#user-contribution-events) in your
+With Contribution Analytics, you can get an overview of the [contribution events](../../profile/contributions_calendar.md#user-contribution-events) in your
group.
- Analyze your team's contributions over a period of time.
@@ -43,7 +43,7 @@ You can choose from the following three periods:
- Last month
- Last three months
-Select the desired period from the calendar dropdown.
+Select the desired period from the calendar dropdown list.
![Contribution analytics choose period](img/group_stats_cal.png)
@@ -62,6 +62,10 @@ Contributions per group member are also presented in tabular format. Select a co
![Contribution analytics contributions table](img/group_stats_table.png)
+## Contribution analytics GraphQL API
+
+To retrieve metrics for user contributions, use the [GraphQL](../../../api/graphql/reference/index.md#groupcontributions) API.
+
<!-- ## Troubleshooting
Include any troubleshooting steps that you can foresee. If you know beforehand what issues
@@ -70,6 +74,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/group/custom_project_templates.md b/doc/user/group/custom_project_templates.md
index 7f77c2147e1..547e64df7c5 100644
--- a/doc/user/group/custom_project_templates.md
+++ b/doc/user/group/custom_project_templates.md
@@ -78,6 +78,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/group/epics/epic_boards.md b/doc/user/group/epics/epic_boards.md
index 78cc65f1923..64addd524ad 100644
--- a/doc/user/group/epics/epic_boards.md
+++ b/doc/user/group/epics/epic_boards.md
@@ -30,7 +30,7 @@ To create a new epic board:
1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Epics > Boards**.
-1. In the upper left corner, select the dropdown with the current board name.
+1. In the upper left corner, select the dropdown list with the current board name.
1. Select **Create new board**.
1. Enter the new board's title.
1. Optional. To hide the Open or Closed lists, clear the **Show the Open list** and
@@ -54,7 +54,7 @@ Prerequisites:
To delete the active epic board:
-1. Select the dropdown with the current board name in the upper left corner of the epic boards page.
+1. Select the dropdown list with the current board name in the upper left corner of the epic boards page.
1. Select **Delete board**.
1. Select **Delete**.
@@ -80,7 +80,7 @@ To create a new list:
1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Epics > Boards**.
1. In the upper-right corner, select **Create list**.
-1. In the **New list** column expand the **Select a label** dropdown and select the label to use as
+1. In the **New list** column expand the **Select a label** dropdown list and select the label to use as
list scope.
1. Select **Add to board**.
diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md
index da6e675f0eb..21c95f37aeb 100644
--- a/doc/user/group/epics/index.md
+++ b/doc/user/group/epics/index.md
@@ -37,15 +37,9 @@ Also, read more about possible [planning hierarchies](../planning_hierarchy/inde
### Child issues from different group hierarchies
-<!-- When feature flag is removed, integrate this info as a sentence in
-https://docs.gitlab.com/ee/user/group/epics/manage_epics.html#add-an-existing-issue-to-an-epic -->
-
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371081) in GitLab 15.5 [with a flag](../../../administration/feature_flags.md) named `epic_issues_from_different_hierarchies`. Disabled by default.
> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/373304) in GitLab 15.5.
-
-FLAG:
-On self-managed GitLab, by default this feature is unavailable. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `epic_issues_from_different_hierarchies`.
-On GitLab.com, this feature is available.
+> - Feature flag `epic_issues_from_different_hierarchies` removed in GitLab 15.6.
You can add issues from a different group hierarchy to an epic.
To do it, paste the issue URL when
@@ -77,6 +71,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md
index 0a12f551588..61aa5c5fe02 100644
--- a/doc/user/group/epics/manage_epics.md
+++ b/doc/user/group/epics/manage_epics.md
@@ -24,8 +24,8 @@ To create an epic in the group you're in:
1. Get to the New Epic form:
- Go to your group and from the left sidebar select **Epics**. Then select **New epic**.
- - From an epic in your group, select **New epic**.
- - From anywhere, in the top menu, select **New...** (**{plus-square}**) **> New epic**.
+ - From an epic in your group, select the vertical ellipsis (**{ellipsis_v}**). Then select **New epic**.
+ - From anywhere, in the top menu, select **New...** (**{plus-square}**). Then select **New epic**.
- In an empty [roadmap](../roadmap/index.md), select **New epic**.
1. Enter a title.
@@ -257,7 +257,7 @@ To filter:
1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Epics**.
1. Select the field **Search or filter results**.
-1. From the dropdown menu, select the scope or enter plain text to search by epic title or description.
+1. From the dropdown list, select the scope or enter plain text to search by epic title or description.
1. Press <kbd>Enter</kbd> on your keyboard. The list is filtered.
## Sort the list of epics
@@ -282,10 +282,10 @@ You can reverse the default order and interact with the activity feed sorted by
at the top. Your preference is saved via local storage and automatically applied to every epic and issue
you view.
-To change the activity sort order, select the **Oldest first** dropdown menu and select either oldest
+To change the activity sort order, select the **Oldest first** dropdown list and select either oldest
or newest items to be shown first.
-![Issue activity sort order dropdown button](img/epic_activity_sort_order_v13_2.png)
+![Issue activity sort order dropdown list](img/epic_activity_sort_order_v13_2.png)
## Make an epic confidential
@@ -311,6 +311,8 @@ To make an epic confidential:
- **In an existing epic:** on the right sidebar, select **Edit** next to **Confidentiality**, and then
select **Turn on**.
+In GitLab 15.6 and later, you can also use the `/confidential` [quick action](../../../user/project/quick_actions.md).
+
## Manage issues assigned to an epic
This section collects instructions for all the things you can do with [issues](../../project/issues/index.md)
@@ -339,9 +341,8 @@ automatically added to the epic.
#### Add an existing issue to an epic
-You can add existing issues to an epic, including issues in a project in an epic's group, or any of
-the epic's subgroups. Newly added issues appear at the top of the list of
-issues in the **Epics and Issues** tab.
+You can add existing issues to an epic, including issues in a project from a [different group hierarchy](index.md#child-issues-from-different-group-hierarchies).
+Newly added issues appear at the top of the list of issues in the **Epics and Issues** tab.
An epic contains a list of issues and an issue can be associated with at most one epic.
When you add a new issue that's already linked to an epic, the issue is automatically unlinked from its
@@ -359,7 +360,8 @@ To add an existing issue to an epic:
1. Identify the issue to be added, using either of the following methods:
- Paste the link of the issue.
- Search for the desired issue by entering part of the issue's title, then selecting the desired
- match ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9126) in GitLab 12.5).
+ match ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9126) in GitLab 12.5). Issues
+ from different group hierarchies do not appear in search results. To add such an issue, enter its full URL.
If there are multiple issues to be added, press <kbd>Space</kbd> and repeat this step.
1. Select **Add**.
@@ -486,9 +488,26 @@ New child epics appear at the top of the list of epics in the **Epics and Issues
When you add an epic that's already linked to a parent epic, the link to its current parent is removed.
-Epics can contain multiple nested child epics, up to a total of seven levels deep.
+Epics can contain multiple nested child epics, up to a total of 7 levels deep.
+
+The maximum number of direct child epics is 100.
+
+### Child epics from other groups
+
+> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8502) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `child_epics_from_different_hierarchies`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available per group, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `child_epics_from_different_hierarchies`.
+On GitLab.com, this feature is not available. The feature is not ready for production use.
+
+You can add a child epic that belongs to a group that is different from the parent epic's group.
-Maximum number of direct child epics is 100.
+Prerequisites:
+
+- You must have at least the Reporter role for both the child and parent epics' groups.
+- Multi-level child epics must be available for both the child and parent epics' groups.
+
+To add a child epic from another group, paste the epic's URL when [adding an existing epic](#add-a-child-epic-to-an-epic).
### Add a child epic to an epic
@@ -496,14 +515,19 @@ Prerequisites:
- You must have at least the Reporter role for the parent epic's group.
-To add a child epic to an epic:
+To add a new epic as child epic:
-1. Select **Add**.
-1. Select **Add a new epic**.
+1. In an epic, in the **Child issues and epics** section, select **Add > Add a new epic**.
+1. Select a group from the dropdown. The epic's group is selected by default.
+1. Enter a title for the new epic.
+1. Select **Create epic**.
+
+To add an existing epic as child epic:
+
+1. In an epic, in the **Child issues and epics** section, select **Add > Add an existing epic**.
1. Identify the epic to be added, using either of the following methods:
- Paste the link of the epic.
- - Search for the desired issue by entering part of the epic's title, then selecting the desired
- match ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9126) in GitLab 12.5).
+ - Search for the desired issue by entering part of the epic's title, then selecting the desired match. This search is only available for epics within the same group hierarchy.
If there are multiple epics to be added, press <kbd>Space</kbd> and repeat this step.
1. Select **Add**.
diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md
index ee50cfcf182..6dcc01242c2 100644
--- a/doc/user/group/import/index.md
+++ b/doc/user/group/import/index.md
@@ -4,7 +4,7 @@ group: Import
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Migrating groups **(FREE)**
+# Migrating groups with GitLab Migration **(FREE)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/249160) in GitLab 13.7 for group resources [with a flag](../../feature_flags.md) named `bulk_import`. Disabled by default.
> - Group items [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/338985) in GitLab 14.3.
@@ -18,15 +18,15 @@ this feature, ask an administrator to [enable the feature flag](../../../adminis
`bulk_import_projects`. On GitLab.com, migrating group resources is available but migrating project resources is not
available.
-Users with the Owner role on a top-level group can migrate it to:
+Users with the Owner role on a top-level group can use GitLab Migration to migrate the group to:
- Another top-level group.
- The subgroup of any existing top-level group.
- Another GitLab instance, including GitLab.com.
-Migrating groups using the method documented here is not the same as [migrating groups using file exports](../settings/import_export.md).
+Migrating groups using GitLab Migration is not the same as [migrating groups using file exports](../settings/import_export.md).
Importing and exporting groups using file exports requires you to export a group to a file and then import that file in
-another GitLab instance. Migrating groups using the method documented here automates this step.
+another GitLab instance. Migrating groups using GitLab Migration automates this step.
## Import your groups into GitLab
@@ -139,6 +139,7 @@ Migrating projects with file exports uses the same export and import mechanisms
- Issue resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4)
- Issue resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4)
- Issue resource iteration events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4)
+ - Merge request URL references ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267947) in GitLab 15.6)
- Labels ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339419) in GitLab 14.4)
- LFS Objects ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339405) in GitLab 14.8)
- Members ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/341886) in GitLab 14.8)
@@ -148,6 +149,7 @@ Migrating projects with file exports uses the same export and import mechanisms
- Merge request approvers ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339520) in GitLab 15.3)
- Merge request resource state events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4)
- Merge request resource milestone events ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/291983) in GitLab 15.4)
+ - Issue URL references ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267947) in GitLab 15.6)
- Migrate Push Rules ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339403) in GitLab 14.6)
- Pull Requests (including external pull requests) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339409) in GitLab 14.5)
- Pipeline History ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339412) in GitLab 14.6)
diff --git a/doc/user/group/insights/index.md b/doc/user/group/insights/index.md
index b4bca919498..9eb6d4387c1 100644
--- a/doc/user/group/insights/index.md
+++ b/doc/user/group/insights/index.md
@@ -79,6 +79,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/group/issues_analytics/index.md b/doc/user/group/issues_analytics/index.md
index 4764625ff83..dbade014ec2 100644
--- a/doc/user/group/issues_analytics/index.md
+++ b/doc/user/group/issues_analytics/index.md
@@ -55,6 +55,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md
index a6435ae2aa3..518370fc7ac 100644
--- a/doc/user/group/iterations/index.md
+++ b/doc/user/group/iterations/index.md
@@ -278,8 +278,8 @@ To group issues by label:
1. On the top bar, select **Main menu > Groups** and find your group.
1. On the left sidebar, select **Issues > Iterations**.
-1. In the **Group by** dropdown, select **Label**.
-1. Select the **Filter by label** dropdown.
-1. Select the labels you want to group by in the labels dropdown.
+1. In the **Group by** dropdown list, select **Label**.
+1. Select the **Filter by label** dropdown list.
+1. Select the labels you want to group by in the labels dropdown list.
You can also search for labels by typing in the search input.
1. Select any area outside the label dropdown list. The page is now grouped by the selected labels.
diff --git a/doc/user/group/manage.md b/doc/user/group/manage.md
index f11d9035a52..a63e6c6dd7f 100644
--- a/doc/user/group/manage.md
+++ b/doc/user/group/manage.md
@@ -66,6 +66,7 @@ This action removes the group. It also adds a background job to delete all proje
Specifically:
- In [GitLab 12.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/33257), on [GitLab Premium](https://about.gitlab.com/pricing/premium/) or higher tiers, this action adds a background job to mark a group for deletion. By default, the job schedules the deletion 7 days in the future. You can modify this waiting period through the [instance settings](../admin_area/settings/visibility_and_access_controls.md#deletion-protection).
+
- In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/39504), if the user who sets up the deletion is removed from the group before the
deletion happens, the job is cancelled, and the group is no longer scheduled for deletion.
@@ -201,6 +202,13 @@ To remove a member from a group:
- To unassign the user from linked issues and merge requests, select the **Also unassign this user from linked issues and merge requests** checkbox.
1. Select **Remove member**.
+## Ensure removed users cannot invite themselves back
+
+Malicious users with the Maintainer or Owner role could exploit a race condition that allows
+them to invite themselves back to a group or project that a GitLab administrator has removed them from.
+
+To avoid this problem, GitLab administrators can [ensure removed users cannot invite themselves back](../project/members/index.md#ensure-removed-users-cannot-invite-themselves-back).
+
## Add projects to a group
There are two different ways to add a new project to a group:
@@ -255,6 +263,12 @@ If you are changing the path so it can be claimed by another group or user,
you must rename the group too. Both names and paths must
be unique.
+After you change the group path, the new group path is a new namespace and you must update the existing project URL in the following resources:
+
+- [Include statements](../../ci/yaml/includes.md#include-a-single-configuration-file).
+- Docker image references in CI files.
+- Variables that specify a project or namespace.
+
To retain ownership of the original namespace and protect the URL redirects,
create a new group and transfer projects to it instead.
@@ -303,9 +317,7 @@ for the group's projects to meet your group's needs.
[Feature flag `invite_members_group_modal`](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) removed.
Similar to how you [share a project with a group](../project/members/share_project_with_groups.md),
-you can share a group with another group. To invite a group, you must be a member of it. Members get direct access
-to the shared group. This includes members who inherited group membership from a parent group.
-
+you can share a group with another group. To invite a group, you must be a member of it.
To share a given group, for example, `Frontend` with another group, for example,
`Engineering`:
@@ -320,7 +332,7 @@ After sharing the `Frontend` group with the `Engineering` group:
- The **Groups** tab lists the `Engineering` group.
- The **Groups** tab lists a group regardless of whether it is a public or private group.
-- All members of the `Engineering` group have access to the `Frontend` group. The same access levels of the members apply up to the maximum access level selected when sharing the group.
+- All direct members of the `Engineering` group have access to the `Frontend` group. Direct members of `Engineering` that gain access to the `Frontend` group keep their same access level as in `Engineering`, but up to the maximum access level selected when sharing the group. Inherited members of the `Engineering` group do not gain access to the `Frontend` group.
## Transfer a group
@@ -385,214 +397,6 @@ To enable delayed deletion of projects in a group:
NOTE:
In GitLab 13.11 and above the group setting for delayed project deletion is inherited by subgroups. As discussed in [Cascading settings](../../development/cascading_settings.md) inheritance can be overridden, unless enforced by an ancestor.
-## Compliance frameworks **(PREMIUM)**
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276221) in GitLab 13.9.
-> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/287779) in GitLab 13.12.
-
-You can create a compliance framework that is a label to identify that your project has certain compliance
-requirements or needs additional oversight. The label can optionally enforce
-[compliance pipeline configuration](#configure-a-compliance-pipeline) to the projects on which it is
-[applied](../project/settings/index.md#add-a-compliance-framework-to-a-project).
-
-Group owners can create, edit, and delete compliance frameworks:
-
-1. On the top bar, select **Main menu > Groups > View all groups** and find your group.
-1. On the left sidebar, select **Settings** > **General**.
-1. Expand the **Compliance frameworks** section.
-1. Create, edit, or delete compliance frameworks.
-
-### Configure a compliance pipeline **(ULTIMATE)**
-
-> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3156) in GitLab 13.9, disabled behind `ff_evaluate_group_level_compliance_pipeline` [feature flag](../../administration/feature_flags.md).
-> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/300324) in GitLab 13.11.
-> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/331231) in GitLab 14.2.
-
-Group owners can configure a compliance pipeline in a project separate to other projects. By default, the compliance
-pipeline configuration (`.gitlab-ci.yml` file) is run instead of the pipeline configuration of labeled projects.
-
-However, the compliance pipeline configuration can reference the `.gitlab-ci.yml` file of the labeled projects so that:
-
-- The compliance pipeline can also run jobs of labeled project pipelines. This allows for centralized control of
- pipeline configuration.
-- Jobs and variables defined in the compliance pipeline can't be changed by variables in the labeled project's
- `.gitlab-ci.yml` file.
-
-See [example configuration](#example-configuration) for help configuring a compliance pipeline that runs jobs from
-labeled project pipeline configuration.
-
-To configure a compliance pipeline:
-
-1. On the top bar, select **Main menu > Groups > View all groups** and find your group.
-1. On the left sidebar, select **Settings** > **General**.
-1. Expand the **Compliance frameworks** section.
-1. In **Compliance pipeline configuration (optional)**, add the path to the compliance framework configuration. Use the
- `path/file.y[a]ml@group-name/project-name` format. For example:
-
- - `.compliance-ci.yml@gitlab-org/gitlab`.
- - `.compliance-ci.yaml@gitlab-org/gitlab`.
-
-This configuration is inherited by projects where the compliance framework label is
-[applied](../project/settings/index.md#add-a-compliance-framework-to-a-project). In projects with the applied compliance
-framework label, the compliance pipeline configuration is run instead of the labeled project's own pipeline configuration.
-
-The user running the pipeline in the labeled project must at least have the Reporter role on the compliance project.
-
-When used to enforce scan execution, this feature has some overlap with
-[scan execution policies](../application_security/policies/scan-execution-policies.md). We have not
-[unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312). For details on
-the similarities and differences between these features, see [Enforce scan execution](../application_security/index.md#enforce-scan-execution).
-
-#### Example configuration
-
-The following example `.compliance-gitlab-ci.yml` includes the `include` keyword to ensure labeled project pipeline
-configuration is also executed.
-
-```yaml
-# Allows compliance team to control the ordering and interweaving of stages/jobs.
-# Stages without jobs defined will remain hidden.
-stages:
- - pre-compliance
- - build
- - test
- - pre-deploy-compliance
- - deploy
- - post-compliance
-
-variables: # Can be overridden by setting a job-specific variable in project's local .gitlab-ci.yml
- FOO: sast
-
-sast: # None of these attributes can be overridden by a project's local .gitlab-ci.yml
- variables:
- FOO: sast
- image: ruby:2.6
- stage: pre-compliance
- rules:
- - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"
- when: never
- - when: always # or when: on_success
- allow_failure: false
- before_script:
- - "# No before scripts."
- script:
- - echo "running $FOO"
- after_script:
- - "# No after scripts."
-
-sanity check:
- image: ruby:2.6
- stage: pre-deploy-compliance
- rules:
- - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"
- when: never
- - when: always # or when: on_success
- allow_failure: false
- before_script:
- - "# No before scripts."
- script:
- - echo "running $FOO"
- after_script:
- - "# No after scripts."
-
-audit trail:
- image: ruby:2.7
- stage: post-compliance
- rules:
- - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_PIPELINE_SOURCE == "push"
- when: never
- - when: always # or when: on_success
- allow_failure: false
- before_script:
- - "# No before scripts."
- script:
- - echo "running $FOO"
- after_script:
- - "# No after scripts."
-
-include: # Execute individual project's configuration (if project contains .gitlab-ci.yml)
- project: '$CI_PROJECT_PATH'
- file: '$CI_CONFIG_PATH'
- ref: '$CI_COMMIT_REF_NAME' # Must be defined or MR pipelines always use the use default branch
-```
-
-##### CF pipelines in Merge Requests originating in project forks
-
-When an MR originates in a fork, the branch to be merged usually only exists in the fork.
-When creating such an MR against a project with CF pipelines, the above snippet will fail with a
-`Project <project-name> reference <branch-name> does not exist!` error message.
-This is because in the context of the target project, `$CI_COMMIT_REF_NAME` evaluates to a non-existing branch name.
-
-To get the correct context, use `$CI_MERGE_REQUEST_SOURCE_PROJECT_PATH` instead of `$CI_PROJECT_PATH`.
-This variable is only availabe in
-[merge request pipelines](../../ci/pipelines/merge_request_pipelines.md).
-
-For example, for a configuration that supports both branch pipelines, as well as merge request pipelines originating in project forks,
-you need to [combine both `include` directives with `rules:if`](../../ci/yaml/includes.md#use-rules-with-include):
-
-```yaml
-include: # Execute individual project's configuration (if project contains .gitlab-ci.yml)
- - project: '$CI_MERGE_REQUEST_SOURCE_PROJECT_PATH'
- file: '$CI_CONFIG_PATH'
- ref: '$CI_COMMIT_REF_NAME'
- rules:
- - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
- - project: '$CI_PROJECT_PATH'
- file: '$CI_CONFIG_PATH'
- ref: '$CI_COMMIT_REF_NAME'
- rules:
- - if: $CI_PIPELINE_SOURCE != 'merge_request_event'
-```
-
-### Ensure compliance jobs are always run
-
-Compliance pipelines [use GitLab CI/CD](../../ci/index.md) to give you an incredible amount of flexibility
-for defining any sort of compliance jobs you like. Depending on your goals, these jobs
-can be configured to be:
-
-- Modified by users.
-- Non-modifiable.
-
-Generally, if a value in a compliance job:
-
-- Is set, it cannot be changed or overridden by project-level configurations.
-- Is not set, a project-level configuration may set.
-
-Either might be wanted or not depending on your use case.
-
-There are a few best practices for ensuring that these jobs are always run exactly
-as you define them and that downstream, project-level pipeline configurations
-cannot change them:
-
-- Add [a `rules:when:always` block](../../ci/yaml/index.md#when) to each of your compliance jobs. This ensures they are
- non-modifiable and are always run.
-- Explicitly set any [variables](../../ci/yaml/index.md#variables) the job references. This:
- - Ensures that project-level pipeline configurations do not set them and alter their
- behavior.
- - Includes any jobs that drive the logic of your job.
-- Explicitly set the [container image](../../ci/yaml/index.md#image) to run the job in. This ensures that your script
- steps execute in the correct environment.
-- Explicitly set any relevant GitLab pre-defined [job keywords](../../ci/yaml/index.md#job-keywords).
- This ensures that your job uses the settings you intend and that they are not overridden by
- project-level pipelines.
-
-### Avoid parent and child pipelines in GitLab 14.7 and earlier
-
-NOTE:
-This advice does not apply to GitLab 14.8 and later because [a fix](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78878) added
-compatibility for combining compliance pipelines, and parent and child pipelines.
-
-Compliance pipelines start on the run of _every_ pipeline in a labeled project. This means that if a pipeline in the labeled project
-triggers a child pipeline, the compliance pipeline runs first. This can trigger the parent pipeline, instead of the child pipeline.
-
-Therefore, in projects with compliance frameworks, we recommend replacing
-[parent-child pipelines](../../ci/pipelines/downstream_pipelines.md#parent-child-pipelines) with the following:
-
-- Direct [`include`](../../ci/yaml/index.md#include) statements that provide the parent pipeline with child pipeline configuration.
-- Child pipelines placed in another project that are run using the [trigger API](../../ci/triggers/index.md) rather than the parent-child
- pipeline feature.
-
-This alternative ensures the compliance pipeline does not re-start the parent pipeline.
-
## Disable email notifications
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23585) in GitLab 12.2.
@@ -722,7 +526,7 @@ This includes projects shared with the group, but it **excludes** projects in
subgroups or parent groups of the group being configured.
You can configure this feature for both subgroups and immediate parent groups. A project
-in a subgroup has access to the templates for that subgroup, as well as
+in a subgroup has access to the templates for that subgroup and
any immediate parent groups.
To learn how to create templates for issues and merge requests, see
@@ -811,7 +615,7 @@ Group.find_by_sql("SELECT * FROM namespaces WHERE name LIKE '%oup'")
If transferring a group doesn't work through the UI or API, you may want to attempt the transfer in a [Rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session):
WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
```ruby
user = User.find_by_username('<username>')
@@ -842,8 +646,27 @@ At times, a group deletion may get stuck. If needed, in a [Rails console session
you can attempt to delete a group using the following command:
WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
```ruby
GroupDestroyWorker.new.perform(group_id, user_id)
```
+
+### Find a user's maximum permissions for a group or project
+
+Administrators can find a user's maximum permissions for a group or project.
+
+1. Start a [Rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session).
+1. Run the following commands:
+
+ ```ruby
+ user = User.find_by_username 'username'
+ project = Project.find_by_full_path 'group/project'
+ user.max_member_access_for_project project.id
+ ```
+
+ ```ruby
+ user = User.find_by_username 'username'
+ group = Group.find_by_full_path 'group'
+ user.max_member_access_for_group group.id
+ ```
diff --git a/doc/user/group/planning_hierarchy/index.md b/doc/user/group/planning_hierarchy/index.md
index baa2a4d6046..f48a027ab2d 100644
--- a/doc/user/group/planning_hierarchy/index.md
+++ b/doc/user/group/planning_hierarchy/index.md
@@ -58,10 +58,10 @@ Epic "1"*-- "0..*" Issue
In an issue, you can view the parented epic above the issue in the right sidebar under **Epic**.
-![epics state dropdown](img/issue-view-parent-epic-in-sidebar_v14_6.png)
+![epics state dropdown list](img/issue-view-parent-epic-in-sidebar_v14_6.png)
## View ancestry of an epic
In an epic, you can view the ancestors as parents in the right sidebar under **Ancestors**.
-![epics state dropdown](img/epic-view-ancestors-in-sidebar_v14_6.png)
+![epics state dropdown list](img/epic-view-ancestors-in-sidebar_v14_6.png)
diff --git a/doc/user/group/reporting/git_abuse_rate_limit.md b/doc/user/group/reporting/git_abuse_rate_limit.md
new file mode 100644
index 00000000000..1cf3a9dbe7d
--- /dev/null
+++ b/doc/user/group/reporting/git_abuse_rate_limit.md
@@ -0,0 +1,36 @@
+---
+stage: Anti-Abuse
+group: Anti-Abuse
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Git abuse rate limit **(ULTIMATE SELF)**
+
+> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with a flag](../../../administration/feature_flags.md) named `limit_unique_project_downloads_per_namespace_user`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `limit_unique_project_downloads_per_namespace_user`. On GitLab.com, this feature is not available.
+
+Git abuse rate limiting is a feature to automatically ban users who download or clone more than a specified number of repositories in a group or any of its subgroups within a given time frame. Banned users cannot access the main group or any of its non-public subgroups via HTTP or SSH. Access to unrelated groups is unaffected.
+
+If the `limit_unique_project_downloads_per_namespace_user` feature flag is enabled, all users with the Owner role for the main group receive an email when a user is about to be banned.
+
+If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, users with the Owner role for the main group are still notified. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning.
+
+If automatic banning is enabled, users with the Owner role for the main group receive an email when a user is about to be banned, and the user is automatically banned from the group and its subgroups.
+
+## Configure Git abuse rate limiting
+
+1. On the left sidebar, select **Settings > Reporting**.
+1. Update the Git abuse rate limit settings:
+ 1. Enter a number in the **Number of repositories** field, greater than or equal to `0` and less than or equal to `10,000`. This number specifies the maximum amount of unique repositories a user can download in the specified time period before they're banned. When set to `0`, Git abuse rate limiting is disabled.
+ 1. Enter a number in the **Reporting time period (seconds)** field, greater than or equal to `0` and less than or equal to `86,400` (10 days). This number specifies the time in seconds a user can download the maximum amount of repositories before they're banned. When set to `0`, Git abuse rate limiting is disabled.
+ 1. Optional. Exclude up to `100` users by adding them to the **Excluded users** field. Excluded users are not automatically banned.
+ 1. Optional. Turn on the **Automatically ban users from this namespace when they exceed the specified limits** toggle to enable automatic banning.
+1. Select **Save changes**.
+
+## Unban a user
+
+1. On the left sidebar, select **Group information > Members**.
+1. Select the **Banned** tab.
+1. For the account you want to unban, select **Unban**.
diff --git a/doc/user/group/repositories_analytics/index.md b/doc/user/group/repositories_analytics/index.md
index 0a4d7f5eae4..9971457f2ac 100644
--- a/doc/user/group/repositories_analytics/index.md
+++ b/doc/user/group/repositories_analytics/index.md
@@ -58,7 +58,7 @@ To get the report:
1. Select the projects and date range you want to include in the report.
1. Select **Download test coverage data (.csv)**.
-The projects dropdown shows up to 100 projects from your group. If the project you want to check is not in the dropdown list, you can select **All projects** to download the report for all projects in your group, including any projects that are not listed. There is a plan to improve this behavior in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/250684).
+The projects dropdown list shows up to 100 projects from your group. If the project you want to check is not in the dropdown list, you can select **All projects** to download the report for all projects in your group, including any projects that are not listed. There is a plan to improve this behavior in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/250684).
For each day that a coverage report was generated by a job in a project's pipeline, a row in the CSV includes:
@@ -82,6 +82,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/group/roadmap/img/roadmap_blocked_icon_v15_5.png b/doc/user/group/roadmap/img/roadmap_blocked_icon_v15_5.png
index 52130672e5a..625e4afc714 100644
--- a/doc/user/group/roadmap/img/roadmap_blocked_icon_v15_5.png
+++ b/doc/user/group/roadmap/img/roadmap_blocked_icon_v15_5.png
Binary files differ
diff --git a/doc/user/group/roadmap/index.md b/doc/user/group/roadmap/index.md
index 28431cb6b9a..3a9d0c833c1 100644
--- a/doc/user/group/roadmap/index.md
+++ b/doc/user/group/roadmap/index.md
@@ -164,6 +164,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/group/saml_sso/group_sync.md b/doc/user/group/saml_sso/group_sync.md
index 3a50470ce50..001c73b6979 100644
--- a/doc/user/group/saml_sso/group_sync.md
+++ b/doc/user/group/saml_sso/group_sync.md
@@ -77,12 +77,8 @@ Users granted:
### Automatic member removal
After a group sync, for GitLab subgroups, users who are not members of a mapped SAML
-group are removed from the group.
-
-FLAG:
-In [GitLab 15.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/364144), on GitLab.com, users in the top-level
-group are assigned the [default membership role](index.md#role) rather than removed. This setting is enabled with the
-`saml_group_sync_retain_default_membership` feature flag and can be configured by GitLab.com administrators only.
+group are removed from the group. Users in the top-level group are assigned the
+[default membership role](index.md#role).
For example, in the following diagram:
diff --git a/doc/user/group/saml_sso/img/member_enterprise_badge_v14_0.png b/doc/user/group/saml_sso/img/member_enterprise_badge_v14_0.png
deleted file mode 100644
index f9534b14a51..00000000000
--- a/doc/user/group/saml_sso/img/member_enterprise_badge_v14_0.png
+++ /dev/null
Binary files differ
diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md
index fa6f378e811..1c5e7ff0115 100644
--- a/doc/user/group/saml_sso/index.md
+++ b/doc/user/group/saml_sso/index.md
@@ -121,11 +121,18 @@ It can also help to compare the XML response from your provider with our [exampl
> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/211962) in GitLab 13.8 with allowing group owners to not go through SSO.
> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/9152) in GitLab 13.11 with enforcing open SSO session to use Git if this setting is switched on.
> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/339888) in GitLab 14.7 to not enforce SSO checks for Git activity originating from CI/CD jobs.
-> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/215155) in GitLab 15.5 [with a flag](../../../administration/feature_flags.md) named `transparent_sso_enforcement` to include transparent enforcement even when SSO enforcement is not enabled. Enabled on GitLab.com.
+> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/215155) in GitLab 15.5 [with a flag](../../../administration/feature_flags.md) named `transparent_sso_enforcement` to include transparent enforcement even when SSO enforcement is not enabled. Disabled on GitLab.com.
+
+FLAG:
+On self-managed GitLab, transparent SSO enforcement is unavailable. On GitLab.com, transparent SSO enforcement is unavailable and can be configured by GitLab.com administrators only.
SSO is enforced when users access groups and projects in the organization's group hierarchy. Users can view other groups and projects without SSO sign in.
-When SAML SSO is enabled, SSO is enforced for each user with an existing SAML identity.
+SSO is enforced for each user with an existing SAML identity when the following is enabled:
+
+- SAML SSO.
+- The `:transparent_sso_enforcement` feature flag.
+
A user has a SAML identity if one or both of the following are true:
- They have signed in to GitLab by using their GitLab group's single sign-on URL.
@@ -142,6 +149,15 @@ However, users are not prompted to sign in through SSO on each visit. GitLab che
has authenticated through SSO. If it's been more than 1 day since the last sign-in, GitLab
prompts the user to sign in again through SSO.
+When the transparent SSO enforcement feature flag is enabled, SSO is enforced as follows:
+
+| Project/Group visibility | Enforce SSO setting | Member with identity | Member without identity | Non-member or not signed in |
+|--------------------------|---------------------|--------------------| ------ |------------------------------|
+| Private | Off | Enforced | Not enforced | No access |
+| Private | On | Enforced | Enforced | No access |
+| Public | Off | Enforced | Not enforced | Not enforced |
+| Public | On | Enforced | Enforced | Not enforced |
+
An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/297389) to add a similar SSO requirement for API activity.
SSO enforcement has the following effects when enabled:
@@ -306,9 +322,11 @@ To migrate users to a new email domain, users must:
## User access and management
-> SAML user provisioning [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/268142) in GitLab 13.7.
+> - SAML user provisioning [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/268142) in GitLab 13.7.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325712) in GitLab 14.0, GitLab users created by [SAML SSO](index.md#user-access-and-management) or SCIM provisioning are displayed with an **Enterprise** badge in the **Members** view.
-Once Group SSO is configured and enabled, users can access the GitLab.com group through the identity provider's dashboard. If [SCIM](scim_setup.md) is configured, see the [user access and linking setup section on the SCIM page](scim_setup.md#user-access-and-linking-setup).
+After group SSO is configured and enabled, users can access the GitLab.com group through the identity provider's dashboard.
+If [SCIM](scim_setup.md) is configured, see [user access](scim_setup.md#user-access) on the SCIM page.
When a user tries to sign in with Group SSO, GitLab attempts to find or create a user based on the following:
@@ -422,7 +440,7 @@ To rescind a user's access to the group when only SAML SSO is configured, either
1. The GitLab.com group.
- Use Group Sync at the top-level of your group to [automatically remove the user](group_sync.md#automatic-member-removal).
-To rescind a user's access to the group when also using SCIM, refer to [Blocking access](scim_setup.md#blocking-access).
+To rescind a user's access to the group when also using SCIM, refer to [Remove access](scim_setup.md#remove-access).
### Unlinking accounts
diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md
index 55990336a50..18af39f4271 100644
--- a/doc/user/group/saml_sso/scim_setup.md
+++ b/doc/user/group/saml_sso/scim_setup.md
@@ -168,13 +168,16 @@ Prerequisites:
OneLogin provides a **GitLab (SaaS)** app in their catalog, which includes a SCIM integration. Contact OneLogin if you
encounter issues.
-## User access and linking setup
+## User access
-During the synchronization process, all of your users get GitLab accounts, welcoming them
-to their respective groups, with an invitation email. When implementing SCIM provisioning,
-you may want to warn your security-conscious employees about this email.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325712) in GitLab 14.0, GitLab users created by [SAML SSO](index.md#user-access-and-management) or SCIM provisioning are displayed with an **Enterprise** badge in the **Members** view.
-The following diagram is a general outline on what happens when you add users to your SCIM app:
+During the synchronization process, all new users:
+
+- Receive GitLab accounts.
+- Are welcomed to their groups with an invitation email. You may want to warn your employees to expect this email.
+
+The following diagram describes what happens when you add users to your SCIM app:
```mermaid
graph TD
@@ -186,29 +189,38 @@ graph TD
During provisioning:
- Both primary and secondary emails are considered when checking whether a GitLab user account exists.
-- Duplicate usernames are also handled, by adding suffix `1` upon user creation. For example,
- due to already existing `test_user` username, `test_user1` is used.
+- Duplicate usernames are handled by adding suffix `1` when creating the user. For example, if `test_user` already
+ exists, `test_user1` is used. If `test_user1` already exists, GitLab increments the suffix until an unused username
+ is found.
-If [Group SAML](index.md) has been configured and you have an existing GitLab.com account, you can link your SCIM and SAML identities:
+On subsequent visits, new and existing users can access groups either:
-1. Update the [primary email](../../profile/index.md#change-your-primary-email) address in your GitLab.com user account to match the
- user profile email address in your identity provider.
-1. [Link your SAML identity](index.md#linking-saml-to-your-existing-gitlabcom-account).
+- Through the identity provider's dashboard.
+- By visiting links directly.
-We recommend users do this prior to turning on sync, because while synchronization is active, there may be provisioning errors for existing users.
+For role information, see the [Group SAML](index.md#user-access-and-management) page.
-New users and existing users on subsequent visits can access the group through the identity provider's dashboard or by visiting links directly.
+### Link SCIM and SAML identities
-[In GitLab 14.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/325712), GitLab users created by [SAML SSO](index.md#user-access-and-management) or SCIM provisioning display with an **Enterprise** badge in the **Members** view.
+If [group SAML](index.md) is configured and you have an existing GitLab.com account, users can link their SCIM and SAML
+identities. Users should do this before synchronization is turned on because there can be provisioning errors for
+existing users when synchronization is active.
+
+To link your SCIM and SAML identities:
+
+1. Update the [primary email](../../profile/index.md#change-your-primary-email) address in your GitLab.com user account
+ to match the user profile email address in your identity provider.
+1. [Link your SAML identity](index.md#linking-saml-to-your-existing-gitlabcom-account).
-![Enterprise badge for users created with a SCIM identity](img/member_enterprise_badge_v14_0.png)
+### Remove access
-For role information, see the [Group SAML page](index.md#user-access-and-management)
+Remove or deactivate a user on the identity provider to remove their access to:
-### Blocking access
+- The top-level group.
+- All subgroups and projects.
-To rescind access to the top-level group, all subgroups, and projects, remove or deactivate the user
-on the identity provider. After the identity provider performs a sync, based on its configured schedule, the user's membership is revoked and they lose access.
+After the identity provider performs a sync based on its configured schedule, the user's membership is revoked and they
+lose access.
NOTE:
Deprovisioning does not delete the GitLab user account.
diff --git a/doc/user/group/saml_sso/troubleshooting.md b/doc/user/group/saml_sso/troubleshooting.md
index bde5ed1762a..7dafd2c5075 100644
--- a/doc/user/group/saml_sso/troubleshooting.md
+++ b/doc/user/group/saml_sso/troubleshooting.md
@@ -19,9 +19,10 @@ SAML responses are base64 encoded, so we recommend the following browser plugins
- [SAML-tracer](https://addons.mozilla.org/en-US/firefox/addon/saml-tracer/) for Firefox.
- [SAML Message Decoder](https://chrome.google.com/webstore/detail/saml-message-decoder/mpabchoaimgbdbbjjieoaeiibojelbhm?hl=en) for Chrome.
-Specific attention should be paid to:
+Pay specific attention to:
-- The NameID, which we use to identify which user is signing in. If the user has previously signed in, this [must match the value we have stored](#verifying-nameid).
+- The `NameID`, which we use to identify which user is signing in. If the user has previously signed in, this
+ [must match the value we have stored](#verify-nameid).
- The presence of a `X509Certificate`, which we require to verify the response signature.
- The `SubjectConfirmation` and `Conditions`, which can cause errors if misconfigured.
@@ -32,7 +33,7 @@ using an identity provider.
To generate a SAML Response:
-1. Install one of the browser debugging tools previously mentioned.
+1. Install one of the [browser debugging tools](#saml-debugging-tools).
1. Open a new browser tab.
1. Open the SAML tracer console:
- Chrome: On a context menu on the page, select **Inspect**, then select the **SAML** tab in the opened developer
@@ -43,16 +44,17 @@ To generate a SAML Response:
[example SAML response](index.md#example-saml-response).
1. Within the SAML tracer, select the **Export** icon to save the response in JSON format.
-## GitLab SAML Testing Environments
+## Testing GitLab SAML
-To troubleshoot, [a complete GitLab with SAML testing environment using Docker compose](https://gitlab.com/gitlab-com/support/toolbox/replication/tree/master/compose_files)
-is available.
+You can use one of the following to troubleshoot SAML:
-If you only require a SAML provider for testing, a [quick start guide to start a Docker container](../../../administration/troubleshooting/test_environments.md#saml) with a plug and play SAML 2.0 Identity Provider (identity provider) is available.
+- A [complete GitLab with SAML testing environment using Docker compose](https://gitlab.com/gitlab-com/support/toolbox/replication/tree/master/compose_files).
+- A [quick start guide to start a Docker container](../../../administration/troubleshooting/test_environments.md#saml)
+ with a plug and play SAML 2.0 identity provider if you only require a SAML provider.
+- A local environment by
+ [enabling SAML for groups on a self-managed instance](../../../integration/saml.md#configuring-group-saml-on-a-self-managed-gitlab-instance).
-You can test the SaaS feature locally by [enabling SAML for groups on a self-managed instance](../../../integration/saml.md#configuring-group-saml-on-a-self-managed-gitlab-instance).
-
-## Verifying configuration
+## Verify configuration
For convenience, we've included some [example resources](../../../user/group/saml_sso/example_saml_config.md) used by our Support Team. While they may help you verify the SAML app configuration, they are not guaranteed to reflect the current state of third-party products.
@@ -82,7 +84,7 @@ in case the customer has [configured SAML Group Sync](group_sync.md):
- `json.class`: `GroupSamlGroupSyncWorker`
- `json.args`: `<user ID> or <group ID>`
-In the relevant log entry, the:
+In the relevant log entry, the:
- `json.args` are in the form `<userID>, <group ID>,
[group link ID 1, group link ID 2, ..., group link ID N]`.
@@ -148,13 +150,13 @@ If you do not wish to use that GitLab user with the SAML login, you can [unlink
### Message: "SAML authentication failed: User has already been taken"
-The user that you're signed in with already has SAML linked to a different identity, or the NameID value has changed.
+The user that you're signed in with already has SAML linked to a different identity, or the `NameID` value has changed.
Here are possible causes and solutions:
| Cause | Solution |
| ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| You've tried to link multiple SAML identities to the same user, for a given identity provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](index.md#unlinking-accounts) from this GitLab account before attempting to sign in again. |
-| The NameID changes every time the user requests SSO identification | [Check the NameID](#verifying-nameid) is not set with `Transient` format, or the NameID is not changing on subsequent requests.|
+| The `NameID` changes every time the user requests SSO identification | [Check the `NameID`](#verify-nameid) is not set with `Transient` format, or the `NameID` is not changing on subsequent requests.|
### Message: "SAML authentication failed: Email has already been taken"
@@ -171,9 +173,9 @@ User accounts are created in one of the following ways:
### Message: "SAML authentication failed: Extern UID has already been taken, User has already been taken"
-Getting both of these errors at the same time suggests the NameID capitalization provided by the identity provider didn't exactly match the previous value for that user.
+Getting both of these errors at the same time suggests the `NameID` capitalization provided by the identity provider didn't exactly match the previous value for that user.
-This can be prevented by configuring the NameID to return a consistent value. Fixing this for an individual user involves changing the identifier for the user. For GitLab.com, the user needs to [unlink their SAML from the GitLab account](index.md#unlinking-accounts).
+This can be prevented by configuring the `NameID` to return a consistent value. Fixing this for an individual user involves changing the identifier for the user. For GitLab.com, the user needs to [unlink their SAML from the GitLab account](index.md#unlinking-accounts).
### Message: "Request to link SAML account must be authorized"
@@ -196,15 +198,15 @@ to [reset their password](https://gitlab.com/users/password/new) if both:
## Other user sign in issues
-### Verifying NameID
+### Verify `NameID`
-In troubleshooting, any authenticated user can use the API to verify the NameID GitLab already has linked to the user by visiting [`https://gitlab.com/api/v4/user`](https://gitlab.com/api/v4/user) and checking the `extern_uid` under identities.
+In troubleshooting, any authenticated user can use the API to verify the `NameID` GitLab already has linked to their user by visiting [`https://gitlab.com/api/v4/user`](https://gitlab.com/api/v4/user) and checking the `extern_uid` under identities.
For self-managed, administrators can use the [users API](../../../api/users.md) to see the same information.
When using SAML for groups, group members of a role with the appropriate permissions can make use of the [members API](../../../api/members.md) to view group SAML identity information for members of the group.
-This can then be compared to the NameID being sent by the identity provider by decoding the message with a [SAML debugging tool](#saml-debugging-tools). We require that these match to identify users.
+This can then be compared to the `NameID` being sent by the identity provider by decoding the message with a [SAML debugging tool](#saml-debugging-tools). We require that these match to identify users.
### Stuck in a login "loop"
diff --git a/doc/user/group/saml_sso/troubleshooting_scim.md b/doc/user/group/saml_sso/troubleshooting_scim.md
index 6f8aed4b386..22562c51e9e 100644
--- a/doc/user/group/saml_sso/troubleshooting_scim.md
+++ b/doc/user/group/saml_sso/troubleshooting_scim.md
@@ -8,93 +8,120 @@ info: To determine the technical writer assigned to the Stage/Group associated w
This section contains possible solutions for problems you might encounter.
-## How come I can't add a user after I removed them?
+## User cannot be added after they are removed
-As outlined in the [Blocking access section](scim_setup.md#blocking-access), when you remove a user, they are removed from the group. However, their account is not deleted.
+When you remove a user, they are removed from the group but their account is not deleted
+(see [remove access](scim_setup.md#remove-access)).
When the user is added back to the SCIM app, GitLab cannot create a new user because the user already exists.
-Solution: Have a user sign in directly to GitLab, then [manually link](scim_setup.md#user-access-and-linking-setup) their account.
+To solve this problem:
-## How do I diagnose why a user is unable to sign in
+1. Have the user sign in directly to GitLab.
+1. [Manually link](scim_setup.md#link-scim-and-saml-identities) their account.
-Ensure that the user has been added to the SCIM app.
+## User cannot sign in
-If you receive "User is not linked to a SAML account", then most likely the user already exists in GitLab. Have the user follow the [User access and linking setup](scim_setup.md#user-access-and-linking-setup) instructions.
+The following are possible solutions for problems where users cannot sign in:
-The **Identity** (`extern_uid`) value stored by GitLab is updated by SCIM whenever `id` or `externalId` changes. Users cannot sign in unless the GitLab Identity (`extern_uid`) value matches the `NameId` sent by SAML.
+- Ensure that the user was added to the SCIM app.
+- If you receive the `User is not linked to a SAML account` error, the user probably already exists in GitLab. Have the
+ user follow the [Link SCIM and SAML identities](scim_setup.md#link-scim-and-saml-identities) instructions.
+- The **Identity** (`extern_uid`) value stored by GitLab is updated by SCIM whenever `id` or `externalId` changes. Users
+ cannot sign in unless the GitLab Identity (`extern_uid`) value matches the `NameId` sent by SAML. This value is also
+ used by SCIM to match users on the `id`, and is updated by SCIM whenever the `id` or `externalId` values change.
+- The SCIM `id` and SCIM `externalId` must be configured to the same value as the SAML `NameId`. You can trace SAML responses
+ using [debugging tools](troubleshooting.md#saml-debugging-tools), and check any errors against the
+ [SAML troubleshooting](troubleshooting.md) information.
-This value is also used by SCIM to match users on the `id`, and is updated by SCIM whenever the `id` or `externalId` values change.
+## Unsure if user's SAML `NameId` matches the SCIM `externalId`
-It is important that this SCIM `id` and SCIM `externalId` are configured to the same value as the SAML `NameId`. SAML responses can be traced using [debugging tools](troubleshooting.md#saml-debugging-tools), and any errors can be checked against our [SAML troubleshooting docs](troubleshooting.md).
+To check if a user's SAML `NameId` matches their SCIM `externalId`:
-## How do I verify user's SAML NameId matches the SCIM externalId
+- Administrators can use the Admin Area to [list SCIM identities for a user](../../admin_area/index.md#user-identities).
+- Group owners can see the list of users and the identifier stored for each user in the group SAML SSO Settings page.
+- You can use the [SCIM API](../../../api/scim.md) to manually retrieve the `external_uid` GitLab has stored for users and compare the value for each user from the [SAML API](../../../api/saml.md) .
+- Have the user use a [SAML Tracer](troubleshooting.md#saml-debugging-tools) and compare the `external_uid` to
+ the value returned as the SAML `NameId`.
-Administrators can use the Admin Area to [list SCIM identities for a user](../../admin_area/index.md#user-identities).
+## Mismatched SCIM `extern_uid` and SAML `NameId`
-Group owners can see the list of users and the `externalId` stored for each user in the group SAML SSO Settings page.
+Whether the value was changed or you need to map to a different field, the following must map to the same field:
-A possible alternative is to use the [SCIM API](../../../api/scim.md) to manually retrieve the `externalId` we have stored for users, also called the `external_uid` or `NameId`.
+- `id`
+- `externalId`
+- `NameId`
-To see how the `external_uid` compares to the value returned as the SAML NameId, you can have the user use a [SAML Tracer](troubleshooting.md#saml-debugging-tools).
-
-## Update or fix mismatched SCIM externalId and SAML NameId
-
-Whether the value was changed or you need to map to a different field, ensure `id`, `externalId`, and `NameId` all map to the same field.
-
-If the GitLab `externalId` doesn't match the SAML NameId, it needs to be updated in order for the user to sign in. Ideally your identity provider is configured to do such an update, but in some cases it may be unable to do so, such as when looking up a user fails due to an ID change.
+If the GitLab `extern_uid` doesn't match the SAML `NameId`, it must be updated for the user to sign in. Your identity
+provider should be configured to do this update. In some cases the identity provider cannot do the update, for example
+when a user lookup fails because of an ID change.
Be cautious if you revise the fields used by your SCIM identity provider, typically `id` and `externalId`.
-We use these IDs to look up users. If the identity provider does not know the current values for these fields,
+GitLab uses these IDs to look up users. If the identity provider does not know the current values for these fields,
that provider may create duplicate users.
-If the `externalId` for a user is not correct, and also doesn't match the SAML NameID,
-you can address the problem in the following ways:
+If the `extern_uid` for a user is not correct, and also doesn't match the SAML `NameID`, either:
+
+- Have users unlink and relink themselves, based on the
+ [SAML authentication failed: User has already been taken](troubleshooting.md#message-saml-authentication-failed-user-has-already-been-taken)
+ section.
+- Unlink all users simultaneously by removing all users from the SCIM app while provisioning is turned on.
+- Use the [SCIM API](../../../api/scim.md) to manually correct the `extern_uid` stored for users to match the SAML
+ `NameId`. To look up a user, you must know the desired value that matches the `NameId` as well as the current
+ `extern_uid`.
-- You can have users unlink and relink themselves, based on the ["SAML authentication failed: User has already been taken"](troubleshooting.md#message-saml-authentication-failed-user-has-already-been-taken) section.
-- You can unlink all users simultaneously, by removing all users from the SAML app while provisioning is turned on.
-- Use the [SCIM API](../../../api/scim.md) to manually correct the `externalId` stored for users to match the SAML `NameId`.
- To look up a user, you need to know the desired value that matches the `NameId` as well as the current `externalId`.
+You must not:
-It is important not to update these to incorrect values, since this causes users to be unable to sign in. It is also important not to assign a value to the wrong user, as this causes users to get signed into the wrong account.
+- Update these to incorrect values because this causes users to be unable to sign in.
+- Assign a value to the wrong user because this causes users to be signed in to the wrong account.
-## I need to change my SCIM app
+## Change SCIM app
-Individual users can follow the instructions in the ["SAML authentication failed: User has already been taken"](index.md#change-the-saml-app) section.
+When the SCIM app changes:
-Alternatively, users can be removed from the SCIM app which de-links all removed users. Sync can then be turned on for the new SCIM app to [link existing users](scim_setup.md#user-access-and-linking-setup).
+- Users can follow the instructions in the [Change the SAML app](index.md#change-the-saml-app) section.
+- Administrators of the identity provider can:
+ 1. Remove users from the SCIM app, which unlinks all removed users.
+ 1. Turn on sync for the new SCIM app to [link existing users](scim_setup.md#link-scim-and-saml-identities).
-## The SCIM app is throwing `"User has already been taken","status":409` error message
+## SCIM app returns `"User has already been taken","status":409` error
Changing the SAML or SCIM configuration or provider can cause the following problems:
-| Problem | Solution |
-| ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| SAML and SCIM identity mismatch. | First [verify that the user's SAML NameId matches the SCIM externalId](#how-do-i-verify-users-saml-nameid-matches-the-scim-externalid) and then [update or fix the mismatched SCIM externalId and SAML NameId](#update-or-fix-mismatched-scim-externalid-and-saml-nameid). |
-| SCIM identity mismatch between GitLab and the identity provider SCIM app. | You can confirm whether you're hitting the error because of your SCIM identity mismatch between your SCIM app and GitLab.com by using the [SCIM API](../../../api/scim.md) which shows up in the `id` key and compares it with the user `externalId` in the SCIM app. You can use the same [SCIM API](../../../api/scim.md) to update the SCIM `id` for the user on GitLab.com. |
+- SAML and SCIM identity mismatch. To solve this problem:
+ 1. [Verify that the user's SAML `NameId` matches the SCIM `extern_uid`](#unsure-if-users-saml-nameid-matches-the-scim-externalid).
+ 1. [Update or fix the mismatched SCIM `extern_uid` and SAML `NameId`](#mismatched-scim-extern_uid-and-saml-nameid).
+- SCIM identity mismatch between GitLab and the identity provider SCIM app. To solve this problem:
+ 1. Use the [SCIM API](../../../api/scim.md), which displays the user's `extern_uid` stored in GitLab and compares it with the user `externalId` in
+ the SCIM app.
+ 1. Use the same SCIM API to update the SCIM `extern_uid` for the user on GitLab.com.
## Search Rails logs for SCIM requests
GitLab.com administrators can search for SCIM requests in the `api_json.log` using the `pubsub-rails-inf-gprd-*` index in
-[Kibana](https://about.gitlab.com/handbook/support/workflows/kibana.html#using-kibana). Use the following filters based on the internal
-[SCIM API](../../../development/internal_api/index.md#scim-api):
+[Kibana](https://about.gitlab.com/handbook/support/workflows/kibana.html#using-kibana). Use the following filters based
+on the internal [SCIM API](../../../development/internal_api/index.md#scim-api):
- `json.path`: `/scim/v2/groups/<group-path>`
- `json.params.value`: `<externalId>`
-In a relevant log entry, the `json.params.value` shows the values of SCIM parameters GitLab receives. These values can be used to verify if SCIM parameters configured in an
-identity provider's SCIM app are communicated to GitLab as intended. For example, we can use these values as a definitive source on why an account was provisioned with a certain
-set of details. This information can help where an account was SCIM provisioned with details that appear to be incongruent with what might have been configured within an identity
-provider's SCIM app.
+In a relevant log entry, the `json.params.value` shows the values of SCIM parameters GitLab receives. Use these values
+to verify if SCIM parameters configured in an identity provider's SCIM app are communicated to GitLab as intended.
-## Azure
+For example, use these values as a definitive source on why an account was provisioned with a certain set of
+details. This information can help where an account was SCIM provisioned with details that do not match
+the SCIM app configuration.
-### How do I verify my SCIM configuration is correct?
+## Azure Active Directory
-Review the following:
+The following troubleshooting information is specifically for SCIM provisioned through Azure Active Directory.
-- Ensure that the SCIM value for `id` matches the SAML value for `NameId`.
-- Ensure that the SCIM value for `externalId` matches the SAML value for `NameId`.
+### Verify my SCIM configuration is correct
+
+Ensure that:
+
+- The matching precedence for `externalId` is 1.
+- The SCIM value for `externalId` matches the SAML value for `NameId`.
Review the following SCIM parameters for sensible values:
@@ -102,28 +129,37 @@ Review the following SCIM parameters for sensible values:
- `displayName`
- `emails[type eq "work"].value`
-### Testing Azure connection: invalid credentials
+### `invalid credentials` error when testing connection
+
+When testing the connection, you may encounter an error:
+
+```plaintext
+You appear to have entered invalid credentials. Please confirm
+you are using the correct information for an administrative account
+```
-When testing the connection, you may encounter an error: **You appear to have entered invalid credentials. Please confirm you are using the correct information for an administrative account**. If `Tenant URL` and `secret token` are correct, check whether your group path contains characters that may be considered invalid JSON primitives (such as `.`). Removing such characters from the group path typically resolves the error.
+If `Tenant URL` and `secret token` are correct, check whether your group path contains characters that may be considered
+invalid JSON primitives (such as `.`). Removing or URL encoding these characters in the group path typically resolves the error.
-### (Field) can't be blank sync error
+### `(Field) can't be blank` sync error
-When checking the Audit Events for the Provisioning, you can sometimes see the
-error `Namespace can't be blank, Name can't be blank, and User can't be blank.`
+When checking the Audit Events for the provisioning, you sometimes see a `Namespace can't be blank, Name can't be blank,
+and User can't be blank.` error.
-This is likely caused because not all required fields (such as first name and last name) are present for all users being mapped.
+This error can occur because not all required fields (such as first name and last name) are present for all users
+being mapped.
As a workaround, try an alternate mapping:
-1. Follow the Azure mapping instructions from above.
+1. Follow the [Azure mapping instructions](scim_setup.md#configure-attribute-mappings).
1. Delete the `name.formatted` target attribute entry.
1. Change the `displayName` source attribute to have `name.formatted` target attribute.
-### Failed to match an entry in the source and target systems Group 'Group-Name'
+### `Failed to match an entry in the source and target systems Group 'Group-Name'` error
-Group provisioning in Azure can fail with the `Failed to match an entry in the source and target systems Group 'Group-Name'` error message,
-and the error response can include a HTML result of the GitLab URL `https://gitlab.com/users/sign_in`.
+Group provisioning in Azure can fail with the `Failed to match an entry in the source and target systems Group 'Group-Name'`
+error. The error response can include a HTML result of the GitLab URL `https://gitlab.com/users/sign_in`.
-This error is harmless and occurs because Group provisioning was turned on but GitLab SCIM integration does not support it nor require it. To
-remove the error, follow the instructions in the Azure configuration guide to disable the option
-[`Synchronize Azure Active Directory Groups to AppName`](scim_setup.md#configure-azure-active-directory).
+This error is harmless and occurs because group provisioning was turned on but GitLab SCIM integration does not support
+it nor require it. To remove the error, follow the instructions in the Azure configuration guide to disable the option
+to [synchronize Azure Active Directory groups to AppName](scim_setup.md#configure-azure-active-directory).
diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md
index 58f5e476f26..95c8e60af5d 100644
--- a/doc/user/group/subgroups/index.md
+++ b/doc/user/group/subgroups/index.md
@@ -212,6 +212,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md
index fcf4189aa32..980618ac3ae 100644
--- a/doc/user/group/value_stream_analytics/index.md
+++ b/doc/user/group/value_stream_analytics/index.md
@@ -104,7 +104,7 @@ The **Overview** dashboard shows the following key metrics that measure team per
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355304) time to restore service tile in GitLab 15.0.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357071) change failure rate tile in GitLab 15.0.
-The value stream analytics **Overview** dashboard displays the following [DORA](../../../user/analytics/index.md) metrics:
+The value stream analytics **Overview** dashboard displays the following [DORA](../../../user/analytics/dora_metrics.md) metrics:
- Deployment Frequency.
- Lead time for changes.
diff --git a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md
index 998548e3a5e..df785cce406 100644
--- a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md
+++ b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md
@@ -91,7 +91,7 @@ Refer to the [Civo Terraform provider](https://registry.terraform.io/providers/c
After configuring your project, manually trigger the provisioning of your cluster. In GitLab:
1. On the left sidebar, go to **CI/CD > Pipelines**.
-1. Next to **Play** (**{play}**), select the dropdown icon (**{chevron-lg-down}**).
+1. Next to **Play** (**{play}**), select the dropdown list icon (**{chevron-lg-down}**).
1. Select **Deploy** to manually trigger the deployment job.
When the pipeline finishes successfully, you can see your new cluster:
diff --git a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md
index 164b426fd24..8a5c32150c9 100644
--- a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md
+++ b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md
@@ -92,7 +92,7 @@ View the [AWS Terraform provider](https://registry.terraform.io/providers/hashic
After configuring your project, manually trigger the provisioning of your cluster. In GitLab:
1. On the left sidebar, go to **CI/CD > Pipelines**.
-1. Next to **Play** (**{play}**), select the dropdown icon (**{chevron-lg-down}**).
+1. Next to **Play** (**{play}**), select the dropdown list icon (**{chevron-lg-down}**).
1. Select **Deploy** to manually trigger the deployment job.
When the pipeline finishes successfully, you can view the new cluster:
diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md
index d04b14bf80f..6b8e2003b8d 100644
--- a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md
+++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md
@@ -20,7 +20,7 @@ by using the GitLab agent for Kubernetes.
**Prerequisites:**
-- A [Google Cloud Platform (GCP) service account](https://cloud.google.com/docs/authentication/getting-started).
+- A [Google Cloud Platform (GCP) service account](https://cloud.google.com/docs/authentication#service-accounts).
- [A runner](https://docs.gitlab.com/runner/install/) you can use to run the GitLab CI/CD pipeline.
**Steps:**
@@ -71,7 +71,7 @@ To create a GitLab agent for Kubernetes:
To set up your project to communicate to GCP and the GitLab API:
-1. To authenticate GCP with GitLab, create a [GCP service account](https://cloud.google.com/docs/authentication/getting-started)
+1. To authenticate GCP with GitLab, create a [GCP service account](https://cloud.google.com/docs/authentication#service-accounts)
with following roles: `Compute Network Viewer`, `Kubernetes Engine Admin`, `Service Account User`, and `Service Account Admin`. Both User and Admin
service accounts are necessary. The User role impersonates the [default service account](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account)
when [creating the node pool](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/using_gke_with_terraform#node-pool-management).
@@ -117,7 +117,7 @@ Refer to the [Google Terraform provider](https://registry.terraform.io/providers
After configuring your project, manually trigger the provisioning of your cluster. In GitLab:
1. On the left sidebar, go to **CI/CD > Pipelines**.
-1. Next to **Play** (**{play}**), select the dropdown icon (**{chevron-lg-down}**).
+1. Next to **Play** (**{play}**), select the dropdown list icon (**{chevron-lg-down}**).
1. Select **Deploy** to manually trigger the deployment job.
When the pipeline finishes successfully, you can see your new cluster:
diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md
index bf86fdb5a8f..14d3a7996e0 100644
--- a/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md
+++ b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md
@@ -22,5 +22,5 @@ of your cluster.
You can customize the installation of Ingress by updating the
`applications/ingress/values.yaml` file in your cluster
management project. Refer to the
-[chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress)
+[chart](https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx)
for the available configuration options.
diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md
index 72a44ef2a21..c2190ad7cfa 100644
--- a/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md
+++ b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md
@@ -36,17 +36,17 @@ Vault application causes downtime.
To optimally use Vault in a production environment, it's ideal to have a good understanding
of the internals of Vault and how to configure it. This can be done by reading
the [Vault Configuration guide](../../../../../ci/secrets/index.md#configure-your-vault-server),
-the [Vault documentation](https://www.vaultproject.io/docs/internals) and
+the [Vault documentation](https://developer.hashicorp.com/vault/docs/internals) and
the Vault Helm chart [`values.yaml` file](https://github.com/hashicorp/vault-helm/blob/v0.3.3/values.yaml).
At a minimum, most users set up:
-- A [seal](https://www.vaultproject.io/docs/configuration/seal) for extra encryption
+- A [seal](https://developer.hashicorp.com/vault/docs/configuration/seal) for extra encryption
of the main key.
-- A [storage backend](https://www.vaultproject.io/docs/configuration/storage) that's
+- A [storage backend](https://developer.hashicorp.com/vault/docs/configuration/storage) that's
suitable for environment and storage security requirements.
-- [HA Mode](https://www.vaultproject.io/docs/concepts/ha).
-- The [Vault UI](https://www.vaultproject.io/docs/configuration/ui).
+- [HA Mode](https://developer.hashicorp.com/vault/docs/concepts/ha).
+- The [Vault UI](https://developer.hashicorp.com/vault/docs/configuration/ui).
The following is an example values file (`applications/vault/values.yaml`)
that configures Google Key Management Service for auto-unseal, using a Google Cloud Storage backend, enabling
@@ -86,7 +86,7 @@ server:
```
After you have successfully installed Vault, you must
-[initialize the Vault](https://learn.hashicorp.com/tutorials/vault/getting-started-deploy#initializing-the-vault)
+[initialize the Vault](https://developer.hashicorp.com/vault/tutorials/getting-started/getting-started-deploy#initializing-the-vault)
and obtain the initial root token. You need access to your Kubernetes cluster that
Vault has been deployed into to do this. To initialize the Vault, get a
shell to one of the Vault pods running inside Kubernetes (typically this is done
diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md
index c2285ecc773..866b16652fa 100644
--- a/doc/user/infrastructure/iac/index.md
+++ b/doc/user/infrastructure/iac/index.md
@@ -99,8 +99,8 @@ in the template you fetched to customize your configuration.
- To collaborate on Terraform code changes and Infrastructure-as-Code workflows, use the
[Terraform integration in merge requests](mr_integration.md).
- To manage GitLab resources like users, groups, and projects, use the
- [GitLab Terraform provider](https://github.com/gitlabhq/terraform-provider-gitlab). It is released separately from GitLab
- and its documentation is available on [the Terraform docs site](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs).
+ [GitLab Terraform provider](https://gitlab.com/gitlab-org/terraform-provider-gitlab).
+ The GitLab Terraform provider documentation is available on [the Terraform docs site](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs).
- [Create a new cluster on Amazon Elastic Kubernetes Service (EKS)](../clusters/connect/new_eks_cluster.md).
- [Create a new cluster on Google Kubernetes Engine (GKE)](../clusters/connect/new_gke_cluster.md).
- [Troubleshoot](troubleshooting.md) issues with GitLab and Terraform.
diff --git a/doc/user/infrastructure/iac/terraform_state.md b/doc/user/infrastructure/iac/terraform_state.md
index 9cb3fdd7aaf..a690cc78121 100644
--- a/doc/user/infrastructure/iac/terraform_state.md
+++ b/doc/user/infrastructure/iac/terraform_state.md
@@ -59,7 +59,7 @@ encrypt plan output or modify the project visibility settings.
To configure GitLab CI/CD as a backend:
1. In your Terraform project, in a `.tf` file like `backend.tf`,
- define the [HTTP backend](https://www.terraform.io/docs/language/settings/backends/http.html):
+ define the [HTTP backend](https://developer.hashicorp.com/terraform/language/settings/backends/http):
```hcl
terraform {
diff --git a/doc/user/infrastructure/iac/troubleshooting.md b/doc/user/infrastructure/iac/troubleshooting.md
index a1eecb909fb..3921c6a7dc8 100644
--- a/doc/user/infrastructure/iac/troubleshooting.md
+++ b/doc/user/infrastructure/iac/troubleshooting.md
@@ -121,3 +121,13 @@ To resolve this, ensure that:
- If you have set the `TF_HTTP_PASSWORD` CI/CD variable, make sure that you either:
- Set the same value as `TF_PASSWORD`
- Remove `TF_HTTP_PASSWORD` variable if your CI/CD job does not explicitly use it.
+
+### Enable Developer role access to destructive commands
+
+To permit a user with the Developer role to run destructive commands, you need a workaround:
+
+1. [Create a project access token](../../project/settings/project_access_tokens.md#create-a-project-access-token) with `api` scope.
+1. Add `TF_USERNAME` and `TF_PASSWORD` to your CI/CD variables:
+ 1. Set the value of `TF_USERNAME` to the username of your project access token.
+ 1. Set the value of `TF_PASSWORD` to the password of your project access token.
+ 1. Optional. Protect the variables to make them only available in pipelines that run on protected branches or protected tags.
diff --git a/doc/user/markdown.md b/doc/user/markdown.md
index 1a743ae99bb..b6f3ba1cfdd 100644
--- a/doc/user/markdown.md
+++ b/doc/user/markdown.md
@@ -331,7 +331,7 @@ However, you cannot mix the wrapping tags:
```
If your diff includes words in `` `code` `` font, make sure to escape each backtick `` ` `` with a
-backslash `\`. Otherwise the diff highlight does not render correctly:
+backslash <code>&#92;</code>. Otherwise the diff highlight does not render correctly:
```markdown
- {+ Just regular text +}
@@ -1160,17 +1160,15 @@ These details <em>remain</em> <strong>hidden</strong> until expanded.
Markdown inside these tags is also supported.
-NOTE:
-If your Markdown isn't rendering correctly, try adding
-`{::options parse_block_html="true" /}` to the top of the page, and add
-`markdown="span"` to the opening summary tag like this: `<summary markdown="span">`.
-
-Remember to leave a blank line after the `</summary>` tag and before the `</details>` tag,
-as shown in the example:
+Remember to leave a blank line before and after any Markdown sections, as shown in the example:
````html
<details>
-<summary>Click this to collapse/fold.</summary>
+<summary>
+
+Click this to _collapse/fold._
+
+</summary>
These details _remain_ **hidden** until expanded.
@@ -1187,7 +1185,7 @@ works correctly in GitLab.
-->
<details>
-<summary>Click this to collapse/fold.</summary>
+<summary>Click this to <em>collapse/fold.</em></summary>
These details <em>remain</em> <b>hidden</b> until expanded.
diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md
index 5eafb74a144..49c54ec191e 100644
--- a/doc/user/packages/composer_repository/index.md
+++ b/doc/user/packages/composer_repository/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -25,58 +25,7 @@ client uses, see the [Composer API documentation](../../../api/packages/composer
Composer v2.0 is recommended. Composer v1.0 is supported, but it has lower performance when working
in groups with very large numbers of packages.
-## Create a Composer package
-
-If you do not have a Composer package, create one and check it in to
-a repository. This example shows a GitLab repository, but the repository
-can be any public or private repository.
-
-WARNING:
-If you are using a GitLab repository, the project must have been created from
-a group's namespace, rather than a user's namespace. Composer packages
-[can't be published to projects created from a user's namespace](https://gitlab.com/gitlab-org/gitlab/-/issues/235467).
-
-1. Create a directory called `my-composer-package` and change to that directory:
-
- ```shell
- mkdir my-composer-package && cd my-composer-package
- ```
-
-1. Run [`composer init`](https://getcomposer.org/doc/03-cli.md#init) and answer the prompts.
-
- For namespace, enter your unique [namespace](../../../user/namespace/index.md), like your GitLab username or group name.
-
- A file called `composer.json` is created:
-
- ```json
- {
- "name": "<namespace>/composer-test",
- "description": "Library XY",
- "type": "library",
- "license": "GPL-3.0-only",
- "authors": [
- {
- "name": "John Doe",
- "email": "john@example.com"
- }
- ],
- "require": {}
- }
- ```
-
-1. Run Git commands to tag the changes and push them to your repository:
-
- ```shell
- git init
- git add composer.json
- git commit -m 'Composer package test'
- git tag v1.0.0
- git remote add origin git@gitlab.example.com:<namespace>/<project-name>.git
- git push --set-upstream origin main
- git push origin v1.0.0
- ```
-
-The package is now in your GitLab Package Registry.
+Learn how to [build a Composer package](../workflows/build_packages.md#composer).
## Publish a Composer package by using the API
@@ -110,7 +59,7 @@ To publish the package with a personal access token:
- `<personal-access-token>` is your personal access token.
- `<project_id>` is your project ID.
- `<tag>` is the Git tag name of the version you want to publish.
- To publish a branch, use `branch=<branch>` instead of `tag=<tag>`.
+ To publish a branch, use `branch=<branch>` instead of `tag=<tag>`.
To publish the package with a deploy token:
@@ -125,7 +74,7 @@ To publish the package with a deploy token:
- `<deploy-token>` is your deploy token
- `<project_id>` is your project ID.
- `<tag>` is the Git tag name of the version you want to publish.
- To publish a branch, use `branch=<branch>` instead of `tag=<tag>`.
+ To publish a branch, use `branch=<branch>` instead of `tag=<tag>`.
You can view the published package by going to **Packages and registries > Package Registry** and
selecting the **Composer** tab.
diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md
index cd5ce9a1135..3d3fe35fd65 100644
--- a/doc/user/packages/conan_repository/index.md
+++ b/doc/user/packages/conan_repository/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -29,105 +29,7 @@ Package Registry.
For documentation of the specific API endpoints that the Conan package manager
client uses, see the [Conan API documentation](../../../api/packages/conan.md).
-## Build a Conan package
-
-This section explains how to install Conan and build a package for your C/C++
-project.
-
-If you already use Conan and know how to build your own packages, go to the
-[next section](#add-the-package-registry-as-a-conan-remote).
-
-### Install Conan
-
-Download the Conan package manager to your local development environment by
-following the instructions at [conan.io](https://conan.io/downloads.html).
-
-When installation is complete, verify you can use Conan in your terminal by
-running:
-
-```shell
-conan --version
-```
-
-The Conan version is printed in the output:
-
-```plaintext
-Conan version 1.20.5
-```
-
-### Install CMake
-
-When you develop with C++ and Conan, you can select from many available
-compilers. This example uses the CMake build system generator.
-
-To install CMake:
-
-- For Mac, use [Homebrew](https://brew.sh/) and run `brew install cmake`.
-- For other operating systems, follow the instructions at [cmake.org](https://cmake.org/install/).
-
-When installation is complete, verify you can use CMake in your terminal by
-running:
-
-```shell
-cmake --version
-```
-
-The CMake version is printed in the output.
-
-### Create a project
-
-To test the Package Registry, you need a C++ project. If you don't already have
-one, you can clone the Conan [hello world starter project](https://github.com/conan-io/hello).
-
-### Build a package
-
-To build a package:
-
-1. Open a terminal and navigate to your project's root folder.
-1. Generate a new recipe by running `conan new` with a package name and version:
-
- ```shell
- conan new Hello/0.1 -t
- ```
-
-1. Create a package for the recipe by running `conan create` with the Conan user
- and channel:
-
- ```shell
- conan create . mycompany/beta
- ```
-
- NOTE:
- If you use an [instance remote](#add-a-remote-for-your-instance), you must
- follow a specific [naming convention](#package-recipe-naming-convention-for-instance-remotes).
-
-A package with the recipe `Hello/0.1@mycompany/beta` is created.
-
-For more details about creating and managing Conan packages, see the
-[Conan documentation](https://docs.conan.io/en/latest/creating_packages.html).
-
-#### Package without a username and a channel
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345055) in GitLab 14.6.
-
-Even though they are [recommended](https://docs.conan.io/en/latest/reference/conanfile/attributes.html#user-channel)
-to distinguish your package from a similarly named existing package,
-the username and channel are not mandatory fields for a Conan package.
-
-You can create a package without a username and channel by removing them from
-the `create` command:
-
-```shell
-conan create .
-```
-
-The username _and_ the channel must be blank. If only one of these fields is
-blank, the request is rejected.
-
-NOTE:
-Empty usernames and channels can only be used if you use a [project remote](#add-a-remote-for-your-project).
-If you use an [instance remote](#add-a-remote-for-your-instance), the username
-and the channel must be set.
+Learn how to [build a Conan package](../workflows/build_packages.md#conan).
## Add the Package Registry as a Conan remote
@@ -193,12 +95,12 @@ recipe `user` must be the plus sign (`+`) separated project path.
Example recipe names:
-| Project | Package | Supported |
-| ---------------------------------- | ----------------------------------------------- | --------- |
-| `foo/bar` | `my-package/1.0.0@foo+bar/stable` | Yes |
-| `foo/bar-baz/buz` | `my-package/1.0.0@foo+bar-baz+buz/stable` | Yes |
-| `gitlab-org/gitlab-ce` | `my-package/1.0.0@gitlab-org+gitlab-ce/stable` | Yes |
-| `gitlab-org/gitlab-ce` | `my-package/1.0.0@foo/stable` | No |
+| Project | Package | Supported |
+| ---------------------- | ---------------------------------------------- | --------- |
+| `foo/bar` | `my-package/1.0.0@foo+bar/stable` | Yes |
+| `foo/bar-baz/buz` | `my-package/1.0.0@foo+bar-baz+buz/stable` | Yes |
+| `gitlab-org/gitlab-ce` | `my-package/1.0.0@gitlab-org+gitlab-ce/stable` | Yes |
+| `gitlab-org/gitlab-ce` | `my-package/1.0.0@foo/stable` | No |
[Project remotes](#add-a-remote-for-your-project) have a more flexible naming
convention.
diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md
index 1ac38979950..65e7918d827 100644
--- a/doc/user/packages/container_registry/index.md
+++ b/doc/user/packages/container_registry/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -70,7 +70,7 @@ To download and run a container image hosted in the GitLab Container Registry:
[Authentication](#authenticate-with-the-container-registry) is needed to download images from private repository.
For more information on running Docker containers, visit the
-[Docker documentation](https://docs.docker.com/engine/userguide/intro/).
+[Docker documentation](https://docs.docker.com/get-started/).
## Image naming convention
@@ -81,9 +81,9 @@ Images follow this naming convention:
```
If your project is `gitlab.example.com/mynamespace/myproject`, for example,
-then your image must be named `gitlab.example.com/mynamespace/myproject/my-app` at a minimum.
+then your image must be named `gitlab.example.com/mynamespace/myproject` at a minimum.
-You can append additional names to the end of an image name, up to three levels deep.
+You can append additional names to the end of an image name, up to two levels deep.
For example, these are all valid image names for images within the project named `myproject`:
@@ -523,7 +523,7 @@ for more details about the permissions that this setting grants to users.
1. Go to your project's **Settings > General** page.
1. Expand the section **Visibility, project features, permissions**.
-1. Under **Container Registry**, select an option from the dropdown:
+1. Under **Container Registry**, select an option from the dropdown list:
- **Everyone With Access** (Default): The Container Registry is visible to everyone with access
to the project. If the project is public, the Container Registry is also public. If the project
@@ -556,7 +556,7 @@ this setting. However, disabling the Container Registry disables all Container R
## Troubleshooting the GitLab Container Registry
-## Migrating OCI container images to GitLab Container Registry
+### Migrating OCI container images to GitLab Container Registry
Migrating built container images to the GitLab registry is not a current feature. However, an [epic](https://gitlab.com/groups/gitlab-org/-/epics/5210) is open to track the work on this feature.
diff --git a/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md b/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md
index ff89e5f361f..74cbcba2ffc 100644
--- a/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md
+++ b/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -90,7 +90,7 @@ build process instead of trying to minify images afterward.
### Use multi-stage builds
-With [multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/),
+With [multi-stage builds](https://docs.docker.com/build/building/multi-stage/),
you use multiple `FROM` statements in your Dockerfile. Each `FROM` instruction can use a different
base, and each begins a new build stage. You can selectively copy artifacts from one stage to
another, leaving behind everything you don't want in the final image. This is especially useful when
diff --git a/doc/user/packages/container_registry/reduce_container_registry_storage.md b/doc/user/packages/container_registry/reduce_container_registry_storage.md
index 9c981aeac53..23d835ddf5f 100644
--- a/doc/user/packages/container_registry/reduce_container_registry_storage.md
+++ b/doc/user/packages/container_registry/reduce_container_registry_storage.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/debian_repository/index.md b/doc/user/packages/debian_repository/index.md
index 90fc0bc3bb1..7db2a341743 100644
--- a/doc/user/packages/debian_repository/index.md
+++ b/doc/user/packages/debian_repository/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md
index 3fb22437eb0..b7a9729c8ba 100644
--- a/doc/user/packages/dependency_proxy/index.md
+++ b/doc/user/packages/dependency_proxy/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -299,7 +299,7 @@ hub_docker_quota_check:
## Troubleshooting
-## Authentication error: "HTTP Basic: Access Denied"
+### Authentication error: "HTTP Basic: Access Denied"
If you receive an `HTTP Basic: Access denied` error when authenticating against the Dependency Proxy, refer to the [two-factor authentication troubleshooting guide](../../profile/account/two_factor_authentication.md#troubleshooting).
diff --git a/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md b/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md
index 1239d1a97ae..3aaaf0c0186 100644
--- a/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md
+++ b/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md
index 930e0760eb3..563f35f2f4f 100644
--- a/doc/user/packages/generic_packages/index.md
+++ b/doc/user/packages/generic_packages/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/go_proxy/index.md b/doc/user/packages/go_proxy/index.md
index 6f6dc084720..a147c3656b7 100644
--- a/doc/user/packages/go_proxy/index.md
+++ b/doc/user/packages/go_proxy/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/harbor_container_registry/index.md b/doc/user/packages/harbor_container_registry/index.md
index 217d3d57416..f159522d77d 100644
--- a/doc/user/packages/harbor_container_registry/index.md
+++ b/doc/user/packages/harbor_container_registry/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Container Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/helm_repository/index.md b/doc/user/packages/helm_repository/index.md
index 521f04226df..bba68494c2d 100644
--- a/doc/user/packages/helm_repository/index.md
+++ b/doc/user/packages/helm_repository/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -134,6 +134,8 @@ for any related errors. If you see `Validation failed: Version is invalid`, it m
version in your `Chart.yaml` file does not follow [Helm Chart versioning specifications](https://helm.sh/docs/topics/charts/#charts-and-versioning).
To fix the error, use the correct version syntax and upload the chart again.
+Support for providing better error messages for package processing errors in the UI is proposed in issue [330515](https://gitlab.com/gitlab-org/gitlab/-/issues/330515).
+
### `helm push` results in an error
Helm 3.7 introduced a breaking change for the `helm-push` plugin. You can update the
diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md
index 84a10943879..7418977e0d1 100644
--- a/doc/user/packages/index.md
+++ b/doc/user/packages/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/infrastructure_registry/index.md b/doc/user/packages/infrastructure_registry/index.md
index 8129b42905a..ac31b491a0e 100644
--- a/doc/user/packages/infrastructure_registry/index.md
+++ b/doc/user/packages/infrastructure_registry/index.md
@@ -1,6 +1,6 @@
---
-stage: Configure
-group: Configure
+stage: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md
index ec56255999a..2aa71e111fb 100644
--- a/doc/user/packages/maven_repository/index.md
+++ b/doc/user/packages/maven_repository/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -14,177 +14,7 @@ Then, install the packages whenever you need to use them as a dependency.
For documentation of the specific API endpoints that the Maven package manager
client uses, see the [Maven API documentation](../../../api/packages/maven.md).
-## Build a Maven package
-
-This section explains how to install Maven and build a package.
-
-If you already use Maven and know how to build your own packages, go to the
-[next section](#authenticate-to-the-package-registry-with-maven).
-
-Maven repositories work well with Gradle, too. To set up a Gradle project, see [get started with Gradle](#build-a-java-project-with-gradle).
-
-### Install Maven
-
-The required minimum versions are:
-
-- Java 11.0.5+
-- Maven 3.6+
-
-Follow the instructions at [maven.apache.org](https://maven.apache.org/install.html)
-to download and install Maven for your local development environment. After
-installation is complete, verify you can use Maven in your terminal by running:
-
-```shell
-mvn --version
-```
-
-The output should be similar to:
-
-```shell
-Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-04T20:00:29+01:00)
-Maven home: /Users/<your_user>/apache-maven-3.6.1
-Java version: 12.0.2, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk-12.0.2.jdk/Contents/Home
-Default locale: en_GB, platform encoding: UTF-8
-OS name: "mac os x", version: "10.15.2", arch: "x86_64", family: "mac"
-```
-
-### Create a project
-
-Follow these steps to create a Maven project that can be
-published to the GitLab Package Registry.
-
-1. Open your terminal and create a directory to store the project.
-1. From the new directory, run this Maven command to initialize a new package:
-
- ```shell
- mvn archetype:generate -DgroupId=com.mycompany.mydepartment -DartifactId=my-project -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
- ```
-
- The arguments are:
-
- - `DgroupId`: A unique string that identifies your package. Follow
- the [Maven naming conventions](https://maven.apache.org/guides/mini/guide-naming-conventions.html).
- - `DartifactId`: The name of the `JAR`, appended to the end of the `DgroupId`.
- - `DarchetypeArtifactId`: The archetype used to create the initial structure of
- the project.
- - `DinteractiveMode`: Create the project using batch mode (optional).
-
-This message indicates that the project was set up successfully:
-
-```shell
-...
-[INFO] ------------------------------------------------------------------------
-[INFO] BUILD SUCCESS
-[INFO] ------------------------------------------------------------------------
-[INFO] Total time: 3.429 s
-[INFO] Finished at: 2020-01-28T11:47:04Z
-[INFO] ------------------------------------------------------------------------
-```
-
-In the folder where you ran the command, a new directory should be displayed.
-The directory name should match the `DartifactId` parameter, which in this case,
-is `my-project`.
-
-## Build a Java project with Gradle
-
-This section explains how to install Gradle and initialize a Java project.
-
-If you already use Gradle and know how to build your own packages, go to the
-[next section](#authenticate-to-the-package-registry-with-maven).
-
-### Install Gradle
-
-If you want to create a new Gradle project, you must install Gradle. Follow
-instructions at [gradle.org](https://gradle.org/install/) to download and install
-Gradle for your local development environment.
-
-In your terminal, verify you can use Gradle by running:
-
-```shell
-gradle -version
-```
-
-To use an existing Gradle project, in the project directory,
-on Linux execute `gradlew`, or on Windows execute `gradlew.bat`.
-
-The output should be similar to:
-
-```plaintext
-------------------------------------------------------------
-Gradle 6.0.1
-------------------------------------------------------------
-
-Build time: 2019-11-18 20:25:01 UTC
-Revision: fad121066a68c4701acd362daf4287a7c309a0f5
-
-Kotlin: 1.3.50
-Groovy: 2.5.8
-Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019
-JVM: 11.0.5 (Oracle Corporation 11.0.5+10)
-OS: Windows 10 10.0 amd64
-```
-
-### Create a Java project
-
-Follow these steps to create a Maven project that can be
-published to the GitLab Package Registry.
-
-1. Open your terminal and create a directory to store the project.
-1. From this new directory, run this Maven command to initialize a new package:
-
- ```shell
- gradle init
- ```
-
- The output should be:
-
- ```plaintext
- Select type of project to generate:
- 1: basic
- 2: application
- 3: library
- 4: Gradle plugin
- Enter selection (default: basic) [1..4]
- ```
-
-1. Enter `3` to create a new Library project. The output should be:
-
- ```plaintext
- Select implementation language:
- 1: C++
- 2: Groovy
- 3: Java
- 4: Kotlin
- 5: Scala
- 6: Swift
- ```
-
-1. Enter `3` to create a new Java Library project. The output should be:
-
- ```plaintext
- Select build script DSL:
- 1: Groovy
- 2: Kotlin
- Enter selection (default: Groovy) [1..2]
- ```
-
-1. Enter `1` to create a new Java Library project that is described in Groovy DSL, or `2` to create one that is described in Kotlin DSL. The output should be:
-
- ```plaintext
- Select test framework:
- 1: JUnit 4
- 2: TestNG
- 3: Spock
- 4: JUnit Jupiter
- ```
-
-1. Enter `1` to initialize the project with JUnit 4 testing libraries. The output should be:
-
- ```plaintext
- Project name (default: test):
- ```
-
-1. Enter a project name or press <kbd>Enter</kbd> to use the directory name as project name.
+Learn how to build a [Maven](../workflows/build_packages.md#maven) or [Gradle](../workflows/build_packages.md#gradle) package.
## Authenticate to the Package Registry with Maven
@@ -561,11 +391,11 @@ for download.
**Only packages that have the same path as the project** are exposed by
the instance-level endpoint.
-| Project | Package | Instance-level endpoint available |
-| ------- | ------- | --------------------------------- |
-| `foo/bar` | `foo/bar/1.0-SNAPSHOT` | Yes |
-| `gitlab-org/gitlab` | `foo/bar/1.0-SNAPSHOT` | No |
-| `gitlab-org/gitlab` | `gitlab-org/gitlab/1.0-SNAPSHOT` | Yes |
+| Project | Package | Instance-level endpoint available |
+| ------------------- | -------------------------------- | --------------------------------- |
+| `foo/bar` | `foo/bar/1.0-SNAPSHOT` | Yes |
+| `gitlab-org/gitlab` | `foo/bar/1.0-SNAPSHOT` | No |
+| `gitlab-org/gitlab` | `gitlab-org/gitlab/1.0-SNAPSHOT` | Yes |
This example shows how relevant `repository` section of your `pom.xml`.
You still need a project-specific URL in the `distributionManagement` section.
@@ -838,7 +668,7 @@ dependencies {
### Request forwarding to Maven Central
-> [Introduced](<https://gitlab.com/gitlab-org/gitlab/-/issues/362657>) behind a [feature flag](../../feature_flags.md), disabled by default in GitLab 15.4
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362657) behind a [feature flag](../../feature_flags.md), disabled by default in GitLab 15.4
FLAG:
On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `maven_central_request_forwarding`.
@@ -1053,20 +883,20 @@ that you can use when performing tasks with GitLab CI/CD.
- Specify where to find the `pom.xml` file (`-f,--file`):
- ```yaml
- package:
- script:
- - 'mvn --no-transfer-progress -f helloworld/pom.xml package'
- ```
+ ```yaml
+ package:
+ script:
+ - 'mvn --no-transfer-progress -f helloworld/pom.xml package'
+ ```
- Specify where to find the user settings (`-s,--settings`) instead of
[the default location](https://maven.apache.org/settings.html). There's also a `-gs,--global-settings` option:
- ```yaml
- package:
- script:
- - 'mvn -s settings/ci.xml package'
- ```
+ ```yaml
+ package:
+ script:
+ - 'mvn -s settings/ci.xml package'
+ ```
### Verify your Maven settings
diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md
index 44266559999..5d2efc52ba9 100644
--- a/doc/user/packages/npm_registry/index.md
+++ b/doc/user/packages/npm_registry/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -20,72 +20,7 @@ WARNING:
Never hardcode GitLab tokens (or any tokens) directly in `.npmrc` files or any other files that can
be committed to a repository.
-## Build an npm package
-
-This section covers how to install npm or Yarn and build a package for your
-JavaScript project.
-
-If you already use npm and know how to build your own packages, go to
-the [next section](#authenticate-to-the-package-registry).
-
-### Install npm
-
-Install Node.js and npm in your local development environment by following
-the instructions at [npmjs.com](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm/).
-
-When installation is complete, verify you can use npm in your terminal by
-running:
-
-```shell
-npm --version
-```
-
-The npm version is shown in the output:
-
-```plaintext
-6.10.3
-```
-
-### Install Yarn
-
-As an alternative to npm, you can install Yarn in your local environment by following the
-instructions at [classic.yarnpkg.com](https://classic.yarnpkg.com/en/docs/install).
-
-When installation is complete, verify you can use Yarn in your terminal by
-running:
-
-```shell
-yarn --version
-```
-
-The Yarn version is shown in the output:
-
-```plaintext
-1.19.1
-```
-
-### Create a project
-
-To create a project:
-
-1. Create an empty directory.
-1. Go to the directory and initialize an empty package by running:
-
- ```shell
- npm init
- ```
-
- Or if you're using Yarn:
-
- ```shell
- yarn init
- ```
-
-1. Enter responses to the questions. Ensure the **package name** follows
- the [naming convention](#package-naming-convention) and is scoped to the
- project or group where the registry exists.
-
-A `package.json` file is created.
+Learn how to build an [npm](../workflows/build_packages.md#npm) or [yarn](../workflows/build_packages.md#yarn) package.
## Use the GitLab endpoint for npm packages
@@ -244,15 +179,15 @@ In this case, the `@scope` needs to be `@registries-group` and not `@source-code
For example, if your project is `https://gitlab.example.com/my-org/engineering-group/team-amazing/analytics`,
the root namespace is `my-org`. When you publish a package, it must have `my-org` as the scope.
-| Project | Package | Supported |
-| ---------------------- | ----------------------- | --------- |
-| `my-org/bar` | `@my-org/bar` | Yes |
-| `my-org/bar/baz` | `@my-org/baz` | Yes |
-| `My-Org/Bar/baz` | `@my-org/Baz` | Yes |
-| `My-Org/Bar/baz` | `@My-Org/Baz` | Yes |
-| `my-org/bar/buz` | `@my-org/anything` | Yes |
-| `gitlab-org/gitlab` | `@gitlab-org/gitlab` | Yes |
-| `gitlab-org/gitlab` | `@foo/bar` | No |
+| Project | Package | Supported |
+| ------------------- | -------------------- | --------- |
+| `my-org/bar` | `@my-org/bar` | Yes |
+| `my-org/bar/baz` | `@my-org/baz` | Yes |
+| `My-Org/Bar/baz` | `@my-org/Baz` | Yes |
+| `My-Org/Bar/baz` | `@My-Org/Baz` | Yes |
+| `my-org/bar/buz` | `@my-org/anything` | Yes |
+| `gitlab-org/gitlab` | `@gitlab-org/gitlab` | Yes |
+| `gitlab-org/gitlab` | `@foo/bar` | No |
In GitLab, this regex validates all package names from all package managers:
@@ -290,7 +225,9 @@ You can also define `"publishConfig"` for your project in `package.json`. For ex
```json
{
-"publishConfig": { "@foo:registry":" https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/" }
+ "publishConfig": {
+ "@foo:registry": " https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/"
+ }
}
```
@@ -342,13 +279,13 @@ To publish and install with the project-level npm endpoint, set the following co
```yaml
npmScopes:
foo:
- npmRegistryServer: "https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/"
- npmPublishRegistry: "https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/"
+ npmRegistryServer: 'https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/'
+ npmPublishRegistry: 'https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/'
npmRegistries:
//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:
npmAlwaysAuth: true
- npmAuthToken: "<your_token>"
+ npmAuthToken: '<your_token>'
```
For the instance-level npm endpoint, use this Yarn 2 configuration in `.yarnrc.yml`:
@@ -356,12 +293,12 @@ For the instance-level npm endpoint, use this Yarn 2 configuration in `.yarnrc.y
```yaml
npmScopes:
foo:
- npmRegistryServer: "https://gitlab.example.com/api/v4/packages/npm/"
+ npmRegistryServer: 'https://gitlab.example.com/api/v4/packages/npm/'
npmRegistries:
//gitlab.example.com/api/v4/packages/npm/:
npmAlwaysAuth: true
- npmAuthToken: "<your_token>"
+ npmAuthToken: '<your_token>'
```
In this configuration:
@@ -561,7 +498,7 @@ should look like:
{
"name": "@foo/my-package",
"version": "1.0.0",
- "description": "Example package for GitLab npm registry",
+ "description": "Example package for GitLab npm registry"
}
```
diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md
index d4b87d70447..956202bb990 100644
--- a/doc/user/packages/nuget_repository/index.md
+++ b/doc/user/packages/nuget_repository/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -22,47 +22,7 @@ The Package Registry works with:
For documentation of the specific API endpoints that these
clients use, see the [NuGet API documentation](../../../api/packages/nuget.md).
-## Install NuGet
-
-The required minimum versions are:
-
-- [NuGet CLI 5.1 or later](https://www.nuget.org/downloads). If you have
- [Visual Studio](https://visualstudio.microsoft.com/vs/), the NuGet CLI is
- probably already installed.
-- Alternatively, you can use [.NET SDK 3.0 or later](https://dotnet.microsoft.com/download/dotnet/3.0),
- which installs the NuGet CLI.
-- NuGet protocol version 3 or later.
-
-Verify that the [NuGet CLI](https://www.nuget.org/) is installed by running:
-
-```shell
-nuget help
-```
-
-The output should be similar to:
-
-```plaintext
-NuGet Version: 5.1.0.6013
-usage: NuGet <command> [args] [options]
-Type 'NuGet help <command>' for help on a specific command.
-
-Available commands:
-
-[output truncated]
-```
-
-### Install NuGet on macOS
-
-For macOS, you can use [Mono](https://www.mono-project.com/) to run the
-NuGet CLI.
-
-1. If you use Homebrew, to install Mono, run `brew install mono`.
-1. Download the Windows C# binary `nuget.exe` from the [NuGet CLI page](https://www.nuget.org/downloads).
-1. Run this command:
-
- ```shell
- mono nuget.exe
- ```
+Learn how to [install NuGet](../workflows/build_packages.md#nuget).
## Use the GitLab endpoint for NuGet Packages
@@ -162,6 +122,7 @@ To use the [project-level](#use-the-gitlab-endpoint-for-nuget-packages) NuGet en
1. In the **NuGet** section, select **Sources** to view a list of all your NuGet sources.
1. Select **Add**.
1. Complete the following fields:
+
- **Name**: Name for the source.
- **Location**: `https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json`,
where `<your_project_id>` is your project ID, and `gitlab.example.com` is
@@ -191,6 +152,7 @@ To use the [group-level](#use-the-gitlab-endpoint-for-nuget-packages) NuGet endp
1. In the **NuGet** section, select **Sources** to view a list of all your NuGet sources.
1. Select **Add**.
1. Complete the following fields:
+
- **Name**: Name for the source.
- **Location**: `https://gitlab.example.com/api/v4/groups/<your_group_id>/-/packages/nuget/index.json`,
where `<your_group_id>` is your group ID, and `gitlab.example.com` is
diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md
index 2d8cb46f933..8e160cbb195 100644
--- a/doc/user/packages/package_registry/index.md
+++ b/doc/user/packages/package_registry/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/package_registry/reduce_package_registry_storage.md b/doc/user/packages/package_registry/reduce_package_registry_storage.md
index e6996d9dc3e..1085cf5c239 100644
--- a/doc/user/packages/package_registry/reduce_package_registry_storage.md
+++ b/doc/user/packages/package_registry/reduce_package_registry_storage.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md
index fb1b9ce78ab..f6ed9654882 100644
--- a/doc/user/packages/pypi_repository/index.md
+++ b/doc/user/packages/pypi_repository/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -20,148 +20,7 @@ The Package Registry works with:
For documentation of the specific API endpoints that the `pip` and `twine`
clients use, see the [PyPI API documentation](../../../api/packages/pypi.md).
-## Build a PyPI package
-
-This section explains how to create a PyPI package.
-
-If you already use PyPI and know how to build your own packages, go to the
-[next section](#authenticate-with-the-package-registry).
-
-### Install pip and twine
-
-Install a recent version of [pip](https://pypi.org/project/pip/) and
-[twine](https://pypi.org/project/twine/).
-
-### Create a project
-
-Create a test project.
-
-1. Open your terminal.
-1. Create a directory called `MyPyPiPackage`, and then go to that directory:
-
- ```shell
- mkdir MyPyPiPackage && cd MyPyPiPackage
- ```
-
-1. Create another directory and go to it:
-
- ```shell
- mkdir mypypipackage && cd mypypipackage
- ```
-
-1. Create the required files in this directory:
-
- ```shell
- touch __init__.py
- touch greet.py
- ```
-
-1. Open the `greet.py` file, and then add:
-
- ```python
- def SayHello():
- print("Hello from MyPyPiPackage")
- return
- ```
-
-1. Open the `__init__.py` file, and then add:
-
- ```python
- from .greet import SayHello
- ```
-
-1. To test the code, in your `MyPyPiPackage` directory, start the Python prompt.
-
- ```shell
- python
- ```
-
-1. Run this command:
-
- ```python
- >>> from mypypipackage import SayHello
- >>> SayHello()
- ```
-
-A message indicates that the project was set up successfully:
-
-```plaintext
-Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
-[Clang 6.0 (clang-600.0.57)] on darwin
-Type "help", "copyright", "credits" or "license" for more information.
->>> from mypypipackage import SayHello
->>> SayHello()
-Hello from MyPyPiPackage
-```
-
-### Create a package
-
-After you create a project, you can create a package.
-
-1. In your terminal, go to the `MyPyPiPackage` directory.
-1. Create a `pyproject.toml` file:
-
- ```shell
- touch pyproject.toml
- ```
-
- This file contains all the information about the package. For more information
- about this file, see [creating `pyproject.toml`](https://packaging.python.org/en/latest/tutorials/packaging-projects/#creating-pyproject-toml).
- Because GitLab identifies packages based on
- [Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names),
- ensure your package name meets these requirements. See the [installation section](#authenticate-with-a-ci-job-token)
- for details.
-
-1. Open the `pyproject.toml` file, and then add basic information:
-
- ```toml
- [build-system]
- requires = ["setuptools>=61.0"]
- build-backend = "setuptools.build_meta"
-
- [project]
- name = "mypypipackage"
- version = "0.0.1"
- authors = [
- { name="Example Author", email="author@example.com" },
- ]
- description = "A small example package"
- requires-python = ">=3.7"
- classifiers = [
- "Programming Language :: Python :: 3",
- "Operating System :: OS Independent",
- ]
-
- [tool.setuptools.packages]
- find = {}
- ```
-
-1. Save the file.
-1. Install the package build library:
-
- ```shell
- pip install build
- ```
-
-1. Build the package:
-
- ```shell
- python -m build
- ```
-
-The output should be visible in a newly-created `dist` folder:
-
-```shell
-ls dist
-```
-
-The output should appear similar to the following:
-
-```plaintext
-mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz
-```
-
-The package is now ready to be published to the Package Registry.
+Learn how to [build a PyPI package](../workflows/build_packages.md#pypi).
## Authenticate with the Package Registry
diff --git a/doc/user/packages/rubygems_registry/index.md b/doc/user/packages/rubygems_registry/index.md
index f21b210f4f5..7710ad3db01 100644
--- a/doc/user/packages/rubygems_registry/index.md
+++ b/doc/user/packages/rubygems_registry/index.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/terraform_module_registry/index.md b/doc/user/packages/terraform_module_registry/index.md
index 6dd118c39b4..2b99ff807ec 100644
--- a/doc/user/packages/terraform_module_registry/index.md
+++ b/doc/user/packages/terraform_module_registry/index.md
@@ -1,6 +1,6 @@
---
-stage: Configure
-group: Configure
+stage: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
@@ -24,9 +24,11 @@ When you publish a Terraform Module, if it does not exist, it is created.
Prerequisites:
-- A package with the same name and version must not already exist in the top-level namespace.
+- The package name and version [must be unique in the top-level namespace](../infrastructure_registry/index.md#how-module-resolution-works).
- Your project and group names must not include a dot (`.`). For example, `source = "gitlab.example.com/my.group/project.name"`.
- You must [authenticate with the API](../../../api/index.md#authentication). If authenticating with a deploy token, it must be configured with the `write_package_registry` scope.
+- The name of a module [must be unique within the scope of its group](../infrastructure_registry/index.md#how-module-resolution-works), otherwise an
+ [error occurs](#troubleshooting).
```plaintext
PUT /projects/:id/packages/terraform/modules/:module-name/:module-system/:module-version/file
@@ -141,3 +143,7 @@ For examples of the Terraform module registry, check the projects below:
- The [_GitLab local file_ project](https://gitlab.com/mattkasa/gitlab-local-file) creates a minimal Terraform module and uploads it into the Terraform module registry using GitLab CI/CD.
- The [_Terraform module test_ project](https://gitlab.com/mattkasa/terraform-module-test) uses the module from the previous example.
+
+## Troubleshooting
+
+- Publishing a module with a duplicate name results in a `{"message":"Access Denied"}` error. There's an ongoing discussion about allowing duplicate module names [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/368040).
diff --git a/doc/user/packages/workflows/build_packages.md b/doc/user/packages/workflows/build_packages.md
new file mode 100644
index 00000000000..ec971195ea9
--- /dev/null
+++ b/doc/user/packages/workflows/build_packages.md
@@ -0,0 +1,504 @@
+---
+stage: Package
+group: Package Registry
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Build packages
+
+Learn how to install and build packages different package formats.
+
+- [Composer](#composer)
+- [Conan](#conan)
+- [Maven](#maven)
+- [Gradle](#gradle)
+- [npm](#npm)
+- [Yarn](#yarn)
+- [NuGet](#nuget)
+- [PyPI](#pypi)
+
+## Composer
+
+1. Create a directory called `my-composer-package` and change to that directory:
+
+ ```shell
+ mkdir my-composer-package && cd my-composer-package
+ ```
+
+1. Run [`composer init`](https://getcomposer.org/doc/03-cli.md#init) and answer the prompts.
+
+ For namespace, enter your unique [namespace](../../../user/namespace/index.md), like your GitLab username or group name.
+
+ A file called `composer.json` is created:
+
+ ```json
+ {
+ "name": "<namespace>/composer-test",
+ "description": "Library XY",
+ "type": "library",
+ "license": "GPL-3.0-only",
+ "authors": [
+ {
+ "name": "John Doe",
+ "email": "john@example.com"
+ }
+ ],
+ "require": {}
+ }
+ ```
+
+## Conan
+
+### Install Conan
+
+Download the Conan package manager to your local development environment by
+following the instructions at [conan.io](https://conan.io/downloads.html).
+
+When installation is complete, verify you can use Conan in your terminal by
+running:
+
+```shell
+conan --version
+```
+
+The Conan version is printed in the output:
+
+```plaintext
+Conan version 1.20.5
+```
+
+### Install CMake
+
+When you develop with C++ and Conan, you can select from many available
+compilers. This example uses the CMake build system generator.
+
+To install CMake:
+
+- For Mac, use [Homebrew](https://brew.sh/) and run `brew install cmake`.
+- For other operating systems, follow the instructions at [cmake.org](https://cmake.org/install/).
+
+When installation is complete, verify you can use CMake in your terminal by
+running:
+
+```shell
+cmake --version
+```
+
+The CMake version is printed in the output.
+
+### Create a project
+
+To test the Package Registry, you need a C++ project. If you don't already have
+one, you can clone the Conan [hello world starter project](https://github.com/conan-io/hello).
+
+### Build a Conan package
+
+To build a package:
+
+1. Open a terminal and navigate to your project's root folder.
+1. Generate a new recipe by running `conan new` with a package name and version:
+
+ ```shell
+ conan new Hello/0.1 -t
+ ```
+
+1. Create a package for the recipe by running `conan create` with the Conan user
+ and channel:
+
+ ```shell
+ conan create . mycompany/beta
+ ```
+
+ NOTE:
+ If you use an [instance remote](../conan_repository/index.md#add-a-remote-for-your-instance), you must
+ follow a specific [naming convention](../conan_repository/index.md#package-recipe-naming-convention-for-instance-remotes).
+
+A package with the recipe `Hello/0.1@mycompany/beta` is created.
+
+For more details about creating and managing Conan packages, see the
+[Conan documentation](https://docs.conan.io/en/latest/creating_packages.html).
+
+## Maven
+
+### Install Maven
+
+The required minimum versions are:
+
+- Java 11.0.5+
+- Maven 3.6+
+
+Follow the instructions at [maven.apache.org](https://maven.apache.org/install.html)
+to download and install Maven for your local development environment. After
+installation is complete, verify you can use Maven in your terminal by running:
+
+```shell
+mvn --version
+```
+
+The output should be similar to:
+
+```shell
+Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-04T20:00:29+01:00)
+Maven home: /Users/<your_user>/apache-maven-3.6.1
+Java version: 12.0.2, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk-12.0.2.jdk/Contents/Home
+Default locale: en_GB, platform encoding: UTF-8
+OS name: "mac os x", version: "10.15.2", arch: "x86_64", family: "mac"
+```
+
+### Build a Maven package
+
+1. Open your terminal and create a directory to store the project.
+1. From the new directory, run this Maven command to initialize a new package:
+
+ ```shell
+ mvn archetype:generate -DgroupId=com.mycompany.mydepartment -DartifactId=my-project -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
+ ```
+
+ The arguments are:
+
+ - `DgroupId`: A unique string that identifies your package. Follow
+ the [Maven naming conventions](https://maven.apache.org/guides/mini/guide-naming-conventions.html).
+ - `DartifactId`: The name of the `JAR`, appended to the end of the `DgroupId`.
+ - `DarchetypeArtifactId`: The archetype used to create the initial structure of
+ the project.
+ - `DinteractiveMode`: Create the project using batch mode (optional).
+
+This message indicates that the project was set up successfully:
+
+```shell
+...
+[INFO] ------------------------------------------------------------------------
+[INFO] BUILD SUCCESS
+[INFO] ------------------------------------------------------------------------
+[INFO] Total time: 3.429 s
+[INFO] Finished at: 2020-01-28T11:47:04Z
+[INFO] ------------------------------------------------------------------------
+```
+
+In the folder where you ran the command, a new directory should be displayed.
+The directory name should match the `DartifactId` parameter, which in this case,
+is `my-project`.
+
+## Gradle
+
+### Install Gradle
+
+If you want to create a new Gradle project, you must install Gradle. Follow
+instructions at [gradle.org](https://gradle.org/install/) to download and install
+Gradle for your local development environment.
+
+In your terminal, verify you can use Gradle by running:
+
+```shell
+gradle -version
+```
+
+To use an existing Gradle project, in the project directory,
+on Linux execute `gradlew`, or on Windows execute `gradlew.bat`.
+
+The output should be similar to:
+
+```plaintext
+------------------------------------------------------------
+Gradle 6.0.1
+------------------------------------------------------------
+
+Build time: 2019-11-18 20:25:01 UTC
+Revision: fad121066a68c4701acd362daf4287a7c309a0f5
+
+Kotlin: 1.3.50
+Groovy: 2.5.8
+Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019
+JVM: 11.0.5 (Oracle Corporation 11.0.5+10)
+OS: Windows 10 10.0 amd64
+```
+
+### Create a package
+
+1. Open your terminal and create a directory to store the project.
+1. From this new directory, run this command to initialize a new package:
+
+ ```shell
+ gradle init
+ ```
+
+ The output should be:
+
+ ```plaintext
+ Select type of project to generate:
+ 1: basic
+ 2: application
+ 3: library
+ 4: Gradle plugin
+ Enter selection (default: basic) [1..4]
+ ```
+
+1. Enter `3` to create a new Library project. The output should be:
+
+ ```plaintext
+ Select implementation language:
+ 1: C++
+ 2: Groovy
+ 3: Java
+ 4: Kotlin
+ 5: Scala
+ 6: Swift
+ ```
+
+1. Enter `3` to create a new Java Library project. The output should be:
+
+ ```plaintext
+ Select build script DSL:
+ 1: Groovy
+ 2: Kotlin
+ Enter selection (default: Groovy) [1..2]
+ ```
+
+1. Enter `1` to create a new Java Library project that is described in Groovy DSL, or `2` to create one that is described in Kotlin DSL. The output should be:
+
+ ```plaintext
+ Select test framework:
+ 1: JUnit 4
+ 2: TestNG
+ 3: Spock
+ 4: JUnit Jupiter
+ ```
+
+1. Enter `1` to initialize the project with JUnit 4 testing libraries. The output should be:
+
+ ```plaintext
+ Project name (default: test):
+ ```
+
+1. Enter a project name or press <kbd>Enter</kbd> to use the directory name as project name.
+
+## npm
+
+### Install npm
+
+Install Node.js and npm in your local development environment by following
+the instructions at [npmjs.com](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm/).
+
+When installation is complete, verify you can use npm in your terminal by
+running:
+
+```shell
+npm --version
+```
+
+The npm version is shown in the output:
+
+```plaintext
+6.10.3
+```
+
+### Create an npm package
+
+1. Create an empty directory.
+1. Go to the directory and initialize an empty package by running:
+
+ ```shell
+ npm init
+ ```
+
+1. Enter responses to the questions. Ensure the **package name** follows
+ the [naming convention](../npm_registry/index.md#package-naming-convention) and is scoped to the project or group where the registry exists.
+
+## Yarn
+
+### Install Yarn
+
+As an alternative to npm, you can install Yarn in your local environment by following the
+instructions at [classic.yarnpkg.com](https://classic.yarnpkg.com/en/docs/install).
+
+When installation is complete, verify you can use Yarn in your terminal by
+running:
+
+```shell
+yarn --version
+```
+
+The Yarn version is shown in the output:
+
+```plaintext
+1.19.1
+```
+
+### Create a package
+
+1. Create an empty directory.
+1. Go to the directory and initialize an empty package by running:
+
+ ```shell
+ yarn init
+ ```
+
+1. Enter responses to the questions. Ensure the **package name** follows
+ the [naming convention](../npm_registry/index.md#package-naming-convention) and is scoped to the
+ project or group where the registry exists.
+
+A `package.json` file is created.
+
+## NuGet
+
+### Install NuGet
+
+Follow the instructions from [Microsoft](https://learn.microsoft.com/en-us/nuget/install-nuget-client-tools) to install NuGet. If you have
+[Visual Studio](https://visualstudio.microsoft.com/vs/), NuGet is
+probably already installed.
+
+Verify that the [NuGet CLI](https://www.nuget.org/) is installed by running:
+
+```shell
+nuget help
+```
+
+The output should be similar to:
+
+```plaintext
+NuGet Version: 5.1.0.6013
+usage: NuGet <command> [args] [options]
+Type 'NuGet help <command>' for help on a specific command.
+
+Available commands:
+
+[output truncated]
+```
+
+## PyPI
+
+### Install pip and twine
+
+Install a recent version of [pip](https://pypi.org/project/pip/) and
+[twine](https://pypi.org/project/twine/).
+
+### Create a project
+
+Create a test project.
+
+1. Open your terminal.
+1. Create a directory called `MyPyPiPackage`, and then go to that directory:
+
+ ```shell
+ mkdir MyPyPiPackage && cd MyPyPiPackage
+ ```
+
+1. Create another directory and go to it:
+
+ ```shell
+ mkdir mypypipackage && cd mypypipackage
+ ```
+
+1. Create the required files in this directory:
+
+ ```shell
+ touch __init__.py
+ touch greet.py
+ ```
+
+1. Open the `greet.py` file, and then add:
+
+ ```python
+ def SayHello():
+ print("Hello from MyPyPiPackage")
+ return
+ ```
+
+1. Open the `__init__.py` file, and then add:
+
+ ```python
+ from .greet import SayHello
+ ```
+
+1. To test the code, in your `MyPyPiPackage` directory, start the Python prompt.
+
+ ```shell
+ python
+ ```
+
+1. Run this command:
+
+ ```python
+ >>> from mypypipackage import SayHello
+ >>> SayHello()
+ ```
+
+A message indicates that the project was set up successfully:
+
+```plaintext
+Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18)
+[Clang 6.0 (clang-600.0.57)] on darwin
+Type "help", "copyright", "credits" or "license" for more information.
+>>> from mypypipackage import SayHello
+>>> SayHello()
+Hello from MyPyPiPackage
+```
+
+### Create a PyPI package
+
+After you create a project, you can create a package.
+
+1. In your terminal, go to the `MyPyPiPackage` directory.
+1. Create a `pyproject.toml` file:
+
+ ```shell
+ touch pyproject.toml
+ ```
+
+ This file contains all the information about the package. For more information
+ about this file, see [creating `pyproject.toml`](https://packaging.python.org/en/latest/tutorials/packaging-projects/#creating-pyproject-toml).
+ Because GitLab identifies packages based on
+ [Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names),
+ ensure your package name meets these requirements. See the [installation section](../pypi_repository/index.md#authenticate-with-a-ci-job-token)
+ for details.
+
+1. Open the `pyproject.toml` file, and then add basic information:
+
+ ```toml
+ [build-system]
+ requires = ["setuptools>=61.0"]
+ build-backend = "setuptools.build_meta"
+
+ [project]
+ name = "mypypipackage"
+ version = "0.0.1"
+ authors = [
+ { name="Example Author", email="author@example.com" },
+ ]
+ description = "A small example package"
+ requires-python = ">=3.7"
+ classifiers = [
+ "Programming Language :: Python :: 3",
+ "Operating System :: OS Independent",
+ ]
+
+ [tool.setuptools.packages]
+ find = {}
+ ```
+
+1. Save the file.
+1. Install the package build library:
+
+ ```shell
+ pip install build
+ ```
+
+1. Build the package:
+
+ ```shell
+ python -m build
+ ```
+
+The output should be visible in a newly-created `dist` folder:
+
+```shell
+ls dist
+```
+
+The output should appear similar to the following:
+
+```plaintext
+mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz
+```
+
+The package is now ready to be published to the Package Registry.
diff --git a/doc/user/packages/workflows/project_registry.md b/doc/user/packages/workflows/project_registry.md
index 25c64c93b73..df4b087f6d5 100644
--- a/doc/user/packages/workflows/project_registry.md
+++ b/doc/user/packages/workflows/project_registry.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/packages/workflows/working_with_monorepos.md b/doc/user/packages/workflows/working_with_monorepos.md
index 5606e257ea8..572cd309e67 100644
--- a/doc/user/packages/workflows/working_with_monorepos.md
+++ b/doc/user/packages/workflows/working_with_monorepos.md
@@ -1,6 +1,6 @@
---
stage: Package
-group: Package
+group: Package Registry
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/permissions.md b/doc/user/permissions.md
index 102abf2b427..8e152a8c190 100644
--- a/doc/user/permissions.md
+++ b/doc/user/permissions.md
@@ -64,7 +64,7 @@ The following table lists project permissions available for each role:
| [Analytics](analytics/index.md):<br>View [code review analytics](analytics/code_review_analytics.md) | | ✓ | ✓ | ✓ | ✓ |
| [Analytics](analytics/index.md):<br>View [repository analytics](analytics/repository_analytics.md) | | ✓ | ✓ | ✓ | ✓ |
| [Application security](application_security/index.md):<br>View licenses in [dependency list](application_security/dependency_list/index.md) | | | ✓ | ✓ | ✓ |
-| [Application security](application_security/index.md):<br>Create and run [on-demand DAST scans](application_security/dast/index.md#on-demand-scans) | | | ✓ | ✓ | ✓ |
+| [Application security](application_security/index.md):<br>Create and run [on-demand DAST scans](application_security/dast/proxy-based.md#on-demand-scans) | | | ✓ | ✓ | ✓ |
| [Application security](application_security/index.md):<br>Manage [security policy](application_security/policies/index.md) | | | ✓ | ✓ | ✓ |
| [Application security](application_security/index.md):<br>View [dependency list](application_security/dependency_list/index.md) | | | ✓ | ✓ | ✓ |
| [Application security](application_security/index.md):<br>Create a [CVE ID Request](application_security/cve_id_request.md) | | | | ✓ | ✓ |
@@ -73,16 +73,17 @@ The following table lists project permissions available for each role:
| [Clusters](infrastructure/clusters/index.md):<br>Manage clusters | | | | ✓ | ✓ |
| [Container Registry](packages/container_registry/index.md):<br>Create, edit, delete [cleanup policies](packages/container_registry/index.md#delete-images-by-using-a-cleanup-policy) | | | | ✓ | ✓ |
| [Container Registry](packages/container_registry/index.md):<br>Push an image to the Container Registry | | | ✓ | ✓ | ✓ |
-| [Container Registry](packages/container_registry/index.md):<br>Pull an image from the Container Registry | ✓ (*20*) | ✓ (*20*) | ✓ | ✓ | ✓ |
+| [Container Registry](packages/container_registry/index.md):<br>Pull an image from the Container Registry | ✓ (*19*) | ✓ (*19*) | ✓ | ✓ | ✓ |
| [Container Registry](packages/container_registry/index.md):<br>Remove a Container Registry image | | | ✓ | ✓ | ✓ |
-| [GitLab Pages](project/pages/index.md):<br>View Pages protected by [access control](project/pages/introduction.md#gitlab-pages-access-control) | ✓ | ✓ | ✓ | ✓ | ✓ |
+| [GitLab Pages](project/pages/index.md):<br>View Pages protected by [access control](project/pages/pages_access_control.md) | ✓ | ✓ | ✓ | ✓ | ✓ |
| [GitLab Pages](project/pages/index.md):<br>Manage | | | | ✓ | ✓ |
| [GitLab Pages](project/pages/index.md):<br>Manage GitLab Pages domains and certificates | | | | ✓ | ✓ |
| [GitLab Pages](project/pages/index.md):<br>Remove GitLab Pages | | | | ✓ | ✓ |
| [Incident Management](../operations/incident_management/index.md):<br>View [alerts](../operations/incident_management/alerts.md) | | ✓ | ✓ | ✓ | ✓ |
| [Incident Management](../operations/incident_management/index.md):<br>Assign an alert | ✓ | ✓ | ✓ | ✓ | ✓ |
+| [Incident Management](../operations/incident_management/index.md):<br>[Change an alert status](../operations/incident_management/alerts.md#change-an-alerts-status) | | ✓ | ✓ | ✓ | ✓ |
| [Incident Management](../operations/incident_management/index.md):<br>View [incident](../operations/incident_management/incidents.md) | ✓ | ✓ | ✓ | ✓ | ✓ |
-| [Incident Management](../operations/incident_management/index.md):<br>Create [incident](../operations/incident_management/incidents.md) | (*16*) | ✓ | ✓ | ✓ | ✓ |
+| [Incident Management](../operations/incident_management/index.md):<br>Create [incident](../operations/incident_management/incidents.md) | | ✓ | ✓ | ✓ | ✓ |
| [Incident Management](../operations/incident_management/index.md):<br>View [on-call schedules](../operations/incident_management/oncall_schedules.md) | | ✓ | ✓ | ✓ | ✓ |
| [Incident Management](../operations/incident_management/index.md):<br>Participate in on-call rotation | ✓ | ✓ | ✓ | ✓ | ✓ |
| [Incident Management](../operations/incident_management/index.md):<br>View [escalation policies](../operations/incident_management/escalation_policies.md) | | ✓ | ✓ | ✓ | ✓ |
@@ -91,16 +92,16 @@ The following table lists project permissions available for each role:
| [Issue boards](project/issue_board.md):<br>Create or delete lists | | ✓ | ✓ | ✓ | ✓ |
| [Issue boards](project/issue_board.md):<br>Move issues between lists | | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>Add Labels | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ |
-| [Issues](project/issues/index.md):<br>Add to epic | | ✓ (*23*) | ✓ (*23*) | ✓ (*23*) | ✓ (*23*) |
+| [Issues](project/issues/index.md):<br>Add to epic | | ✓ (*22*) | ✓ (*22*) | ✓ (*22*) | ✓ (*22*) |
| [Issues](project/issues/index.md):<br>Assign | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ |
-| [Issues](project/issues/index.md):<br>Create (*18*) | ✓ | ✓ | ✓ | ✓ | ✓ |
+| [Issues](project/issues/index.md):<br>Create (*17*) | ✓ | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>Create [confidential issues](project/issues/confidential_issues.md) | ✓ | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>View [Design Management](project/issues/design_management.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>View [related issues](project/issues/related_issues.md) | ✓ | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>Set [weight](project/issues/issue_weight.md) | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>Set [parent epic](group/epics/manage_epics.md#add-an-existing-issue-to-an-epic) | | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>View [confidential issues](project/issues/confidential_issues.md) | (*2*) | ✓ | ✓ | ✓ | ✓ |
-| [Issues](project/issues/index.md):<br>Close / reopen (*19*) | | ✓ | ✓ | ✓ | ✓ |
+| [Issues](project/issues/index.md):<br>Close / reopen (*18*) | | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>Lock threads | | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>Manage [related issues](project/issues/related_issues.md) | | ✓ | ✓ | ✓ | ✓ |
| [Issues](project/issues/index.md):<br>Manage tracker | | ✓ | ✓ | ✓ | ✓ |
@@ -118,7 +119,7 @@ The following table lists project permissions available for each role:
| [Merge requests](project/merge_requests/index.md):<br>Apply code change suggestions | | | ✓ | ✓ | ✓ |
| [Merge requests](project/merge_requests/index.md):<br>Approve (*8*) | | | ✓ | ✓ | ✓ |
| [Merge requests](project/merge_requests/index.md):<br>Assign | | | ✓ | ✓ | ✓ |
-| [Merge requests](project/merge_requests/index.md):<br>Create (*17*) | | | ✓ | ✓ | ✓ |
+| [Merge requests](project/merge_requests/index.md):<br>Create (*16*) | | | ✓ | ✓ | ✓ |
| [Merge requests](project/merge_requests/index.md):<br>Add labels | | | ✓ | ✓ | ✓ |
| [Merge requests](project/merge_requests/index.md):<br>Lock threads | | | ✓ | ✓ | ✓ |
| [Merge requests](project/merge_requests/index.md):<br>Manage or accept | | | ✓ | ✓ | ✓ |
@@ -153,7 +154,7 @@ The following table lists project permissions available for each role:
| [Projects](project/index.md):<br>View project [Audit Events](../administration/audit_events.md) | | | ✓ (*10*) | ✓ | ✓ |
| [Projects](project/index.md):<br>Add [deploy keys](project/deploy_keys/index.md) | | | | ✓ | ✓ |
| [Projects](project/index.md):<br>Add new [team members](project/members/index.md) | | | | ✓ | ✓ |
-| [Projects](project/index.md):<br>Manage [team members](project/members/index.md) | | | | ✓ (*21*) | ✓ |
+| [Projects](project/index.md):<br>Manage [team members](project/members/index.md) | | | | ✓ (*20*) | ✓ |
| [Projects](project/index.md):<br>Change [project features visibility](public_access.md) level | | | | ✓ (*13*) | ✓ |
| [Projects](project/index.md):<br>Configure [webhooks](project/integrations/webhooks.md) | | | | ✓ | ✓ |
| [Projects](project/index.md):<br>Delete [wiki](project/wiki/index.md) pages | | | ✓ | ✓ | ✓ |
@@ -161,7 +162,7 @@ The following table lists project permissions available for each role:
| [Projects](project/index.md):<br>Edit project badges | | | | ✓ | ✓ |
| [Projects](project/index.md):<br>Edit project settings | | | | ✓ | ✓ |
| [Projects](project/index.md):<br>Export project | | | | ✓ | ✓ |
-| [Projects](project/index.md):<br>Manage [project access tokens](project/settings/project_access_tokens.md) (*11*) | | | | ✓ (*21*) | ✓ |
+| [Projects](project/index.md):<br>Manage [project access tokens](project/settings/project_access_tokens.md) (*11*) | | | | ✓ (*20*) | ✓ |
| [Projects](project/index.md):<br>Manage [Project Operations](../operations/index.md) | | | | ✓ | ✓ |
| [Projects](project/index.md):<br>Rename project | | | | ✓ | ✓ |
| [Projects](project/index.md):<br>Share (invite) projects with groups | | | | ✓ (*7*) | ✓ (*7*) |
@@ -203,10 +204,10 @@ The following table lists project permissions available for each role:
| [Security dashboard](application_security/security_dashboard/index.md):<br>Use security dashboard | | | ✓ | ✓ | ✓ |
| [Security dashboard](application_security/security_dashboard/index.md):<br>View vulnerability | | | ✓ | ✓ | ✓ |
| [Security dashboard](application_security/security_dashboard/index.md):<br>View vulnerability findings in [dependency list](application_security/dependency_list/index.md) | | | ✓ | ✓ | ✓ |
-| [Tasks](tasks.md):<br>Create (*18*) | ✓ | ✓ | ✓ | ✓ | ✓ |
+| [Tasks](tasks.md):<br>Create (*17*) | ✓ | ✓ | ✓ | ✓ | ✓ |
| [Tasks](tasks.md):<br>Edit | | ✓ | ✓ | ✓ | ✓ |
| [Tasks](tasks.md):<br>Remove from issue | | ✓ | ✓ | ✓ | ✓ |
-| [Tasks](tasks.md):<br>Delete (*22*) | | | | | ✓ |
+| [Tasks](tasks.md):<br>Delete (*21*) | | | | | ✓ |
| [Terraform](infrastructure/index.md):<br>Read Terraform state | | | ✓ | ✓ | ✓ |
| [Terraform](infrastructure/index.md):<br>Manage Terraform state | | | | ✓ | ✓ |
| [Test cases](../ci/test_cases/index.md):<br>Archive | | ✓ | ✓ | ✓ | ✓ |
@@ -239,15 +240,13 @@ The following table lists project permissions available for each role:
Developer role.
15. Guest users can only set metadata (for example, labels, assignees, or milestones)
when creating an issue. They cannot change the metadata on existing issues.
-16. In GitLab 14.5 or later, Guests are not allowed to [create incidents](../operations/incident_management/incidents.md#incident-creation).
- In GitLab 15.1 and later, a Guest who created an issue that was promoted to an incident cannot edit, close, or reopen their incident.
-17. In projects that accept contributions from external members, users can create, edit, and close their own merge requests.
-18. Authors and assignees can modify the title and description even if they don't have the Reporter role.
-19. Authors and assignees can close and reopen issues even if they don't have the Reporter role.
-20. The ability to view the Container Registry and pull images is controlled by the [Container Registry's visibility permissions](packages/container_registry/index.md#container-registry-visibility-permissions).
-21. Maintainers cannot create, demote, or remove Owners, and they cannot promote users to the Owner role. They also cannot approve Owner role access requests.
-22. Authors of tasks can delete them even if they don't have the Owner role, but they have to have at least the Guest role for the project.
-23. You must have permission to [view the epic](group/epics/manage_epics.md#who-can-view-an-epic).
+16. In projects that accept contributions from external members, users can create, edit, and close their own merge requests.
+17. Authors and assignees can modify the title and description even if they don't have the Reporter role.
+18. Authors and assignees can close and reopen issues even if they don't have the Reporter role.
+19. The ability to view the Container Registry and pull images is controlled by the [Container Registry's visibility permissions](packages/container_registry/index.md#container-registry-visibility-permissions).
+20. Maintainers cannot create, demote, or remove Owners, and they cannot promote users to the Owner role. They also cannot approve Owner role access requests.
+21. Authors of tasks can delete them even if they don't have the Owner role, but they have to have at least the Guest role for the project.
+22. You must have permission to [view the epic](group/epics/manage_epics.md#who-can-view-an-epic).
<!-- markdownlint-enable MD029 -->
@@ -470,7 +469,7 @@ project and should only have access to that project.
External users:
-- Cannot create project, groups, and snippets within their personal namespaces.
+- Cannot create project, groups, and snippets in their personal namespaces.
- Can only create projects (including forks), subgroups, and snippets within top-level groups to which they are explicitly granted access.
- Can only access public projects and projects to which they are explicitly granted access,
thus hiding all other internal or private ones from them (like being
diff --git a/doc/user/product_analytics/index.md b/doc/user/product_analytics/index.md
new file mode 100644
index 00000000000..8e340fff32a
--- /dev/null
+++ b/doc/user/product_analytics/index.md
@@ -0,0 +1,48 @@
+---
+stage: Analyze
+group: Product Analytics
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+---
+
+# Product analytics **(ULTIMATE)** **Alpha**
+
+> Introduced in GitLab 15.4 [with a flag](../../administration/feature_flags.md) named `cube_api_proxy`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `cube_api_proxy`.
+On GitLab.com, this feature is not available.
+This feature is not ready for production use.
+
+## Overview
+
+You can view the [product category](https://about.gitlab.com/direction/analytics/product-analytics/) page for more information about our direction. This page is a work in progress and will be updated as we add more features.
+
+## Product analytics dashboards
+
+Each project can define an unlimited number of dashboards. These dashboards are defined using our YAML schema and stored
+in the `.gitlab/product_analytics/dashboards/` directory. The name of the file is the name of the dashboard, and visualizations are shared across dashboards..
+
+Project maintainers can enforce approval rules on dashboard changes, and dashboards can be versioned in source control.
+
+### Define a dashboard
+
+To define a dashboard:
+
+1. In `.gitlab/product_analytics/dashboards/`, create a directory named like the dashboard. Each dashboard should have its own directory.
+1. In the new directory, create a `.yaml` file with the same name as the directory. This file contains the dashboard definition, and must conform to the JSON schema defined in `ee/app/validators/json_schemas/product_analytics_dashboard.json`.
+1. In the `.gitlab/product_analytics/dashboards/visualizations/` directory, create a `yaml` file. This file defines the visualization type for the dashboard, and must conform to the schema in
+`ee/app/validators/json_schemas/product_analytics_visualization.json`.
+
+The example below includes three dashboards and one visualization that applies to all dashboards.
+
+```plaintext
+.gitlab/product_analytics/dashboards
+├── conversion_funnels
+│ └── conversion_funnels.yaml
+├── demographic_breakdown
+│ └── demographic_breakdown.yaml
+├── north_star_metrics
+| └── north_star_metrics.yaml
+├── visualizations
+│ └── example_line_chart.yaml
+```
diff --git a/doc/user/profile/account/create_accounts.md b/doc/user/profile/account/create_accounts.md
index f2636fdaa68..60b8fe8ab88 100644
--- a/doc/user/profile/account/create_accounts.md
+++ b/doc/user/profile/account/create_accounts.md
@@ -49,3 +49,19 @@ Users are:
- Created when first signing with [Group SAML](../../group/saml_sso/index.md).
- Automatically created by [SCIM](../../group/saml_sso/scim_setup.md) when the user is created in
the identity provider.
+
+## Create users through the Rails console
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+To create a user through the Rails console:
+
+1. [Start a Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session).
+1. Run the following commands:
+
+ ```ruby
+ u = User.new(username: 'test_user', email: 'test@example.com', name: 'Test User', password: 'password', password_confirmation: 'password')
+ u.skip_confirmation! # Use it only if you wish user to be automatically confirmed. If skipped, user receives confirmation e-mail
+ u.save!
+ ```
diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md
index 3dc768f6606..39826bf59c4 100644
--- a/doc/user/profile/account/two_factor_authentication.md
+++ b/doc/user/profile/account/two_factor_authentication.md
@@ -61,7 +61,6 @@ To enable 2FA with a one-time password:
1. Install a compatible application. For example:
- Cloud-based (recommended because you can restore access if you lose the hardware device):
- [Authy](https://authy.com/)
- - [Duo Mobile](https://duo.com/product/multi-factor-authentication-mfa/duo-mobile-app)
- Other:
- [Google Authenticator](https://support.google.com/accounts/answer/1066447?hl=en)
- [Microsoft Authenticator](https://www.microsoft.com/en-us/security/mobile-authenticator-app)
@@ -73,6 +72,9 @@ To enable 2FA with a one-time password:
1. Enter your current password.
1. Select **Submit**.
+NOTE:
+DUO [cannot be used for 2FA](https://gitlab.com/gitlab-org/gitlab/-/issues/15760).
+
If you entered the correct pin, GitLab displays a list of [recovery codes](#recovery-codes). Download them and keep them
in a safe place.
diff --git a/doc/user/profile/active_sessions.md b/doc/user/profile/active_sessions.md
index d0600e5e80d..430d1c3dc9f 100644
--- a/doc/user/profile/active_sessions.md
+++ b/doc/user/profile/active_sessions.md
@@ -51,6 +51,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/profile/contributions_calendar.md b/doc/user/profile/contributions_calendar.md
new file mode 100644
index 00000000000..6df7ad56c5e
--- /dev/null
+++ b/doc/user/profile/contributions_calendar.md
@@ -0,0 +1,136 @@
+---
+stage: Manage
+group: Workspace
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+type: concepts, howto
+---
+
+# Contributions calendar **(FREE)**
+
+The contributions calendar displays a [user's events](#user-contribution-events) from the past 12 months.
+This includes contributions made in forked and [private](#show-private-contributions-on-your-user-profile-page) repositories.
+
+![Contributions calendar](img/contributions_calendar_v15_6.png)
+
+The gradient color of the tiles represents the number of contributions made per day. The gradient ranges from blank (0 contributions) to dark blue (more than 30 contributions).
+
+NOTE:
+The contribution calendar only displays contributions from the last 12 months, but issue [24264](https://gitlab.com/gitlab-org/gitlab/-/issues/24264) proposes to change this to more than 12 months. General improvements to the user profile are proposed in issue [8488](https://gitlab.com/groups/gitlab-org/-/epics/8488).
+
+## User contribution events
+
+GitLab tracks the following contribution events:
+
+- `approved`
+ - Merge request
+- `closed`
+ - [Epic](../group/epics/index.md)
+ - Issue
+ - Merge request
+ - Milestone
+- `commented` on any `Noteable` record.
+ - Alert
+ - Commit
+ - Design
+ - Issue
+ - Merge request
+ - Snippet
+- `created`
+ - Design
+ - [Epic](../group/epics/index.md)
+ - Issue
+ - Merge request
+ - Milestone
+ - Project
+ - Wiki page
+- `destroyed`
+ - Design
+ - Milestone
+ - Wiki page
+- `expired`
+ - Project membership
+- `joined`
+ - Project membership
+- `left`
+ - Project membership
+- `merged`
+ - Merge request
+- `pushed` commits to (or deleted commits from) a repository, individually or in bulk.
+ - Project
+- `reopened`
+ - [Epic](../group/epics/index.md)
+ - Issue
+ - Merge request
+ - Milestone
+- `updated`
+ - Design
+ - Wiki page
+
+### View daily contributions
+
+To view your daily contributions:
+
+1. On the top bar, in the top-right corner, select your avatar.
+1. Select your name from the dropdown list.
+1. In the contributions calendar:
+ - To view the number of contributions for a specific day, hover over a tile.
+ - To view all contributions for a specific day, select a tile. A list displays all contributions and the time they were made.
+
+### Show private contributions on your user profile page
+
+The contributions calendar graph and recent activity list displays your
+[contribution actions](#user-contribution-events) to private projects.
+
+To view private contributions:
+
+1. On the top bar, in the top-right corner, select your avatar.
+1. Select **Edit profile**.
+1. In the **Main settings** section, select the **Include private contributions on my profile** checkbox.
+1. Select **Update profile settings**.
+
+## User activity
+
+### Follow a user's activity
+
+You can follow users whose activity you're interested in.
+In [GitLab 15.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/360755),
+the maximum number of users you can follow is 300.
+
+To follow a user, either:
+
+- From a user's profile, select **Follow**.
+- Hover over a user's name, and select **Follow** ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/76050)
+ in GitLab 15.0).
+
+To view the activity of users you follow:
+
+1. In the GitLab menu, select **Activity**.
+1. Select the **Followed users** tab.
+
+### Retrieve user activity as a feed
+
+GitLab provides RSS feeds of user activity. To subscribe to the
+RSS feed of a user's activity:
+
+1. Go to the [user's profile](index.md#access-your-user-profile).
+1. In the top right, select the feed symbol **{rss}** to display the results as an RSS feed in Atom format.
+
+The URL of the result contains both a feed token, and
+the user's activity that you're authorized to view.
+You can add this URL to your feed reader.
+
+### Reset the user activity feed token
+
+Feed tokens are sensitive and can reveal information from confidential issues.
+If you think your feed token has been exposed, you should reset it.
+
+To reset your feed token:
+
+1. On the top bar, in the top right corner, select your avatar.
+1. Select **Edit profile**.
+1. On the left sidebar, select **Access Tokens**.
+1. Scroll down. In the **Feed token** section, select the
+ **reset this token** link.
+1. On the confirmation box, select **OK**.
+
+A new token is generated.
diff --git a/doc/user/profile/img/contributions_calendar_v15_6.png b/doc/user/profile/img/contributions_calendar_v15_6.png
new file mode 100644
index 00000000000..363636ec0cd
--- /dev/null
+++ b/doc/user/profile/img/contributions_calendar_v15_6.png
Binary files differ
diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md
index 66f6b4b52de..4adf6c351df 100644
--- a/doc/user/profile/index.md
+++ b/doc/user/profile/index.md
@@ -137,7 +137,7 @@ To add links to other accounts:
## Show private contributions on your user profile page
-In the user contribution calendar graph and recent activity list, you can see your [contribution actions](#user-contribution-events) to private projects.
+In the user contribution calendar graph and recent activity list, you can see your [contribution actions](contributions_calendar.md#user-contribution-events) to private projects.
To show private contributions:
@@ -311,7 +311,7 @@ git config --global user.email <your email address>
## User activity
-GitLab tracks user contribution activity. You can follow or unfollow other users from either:
+GitLab tracks [user contribution activity](contributions_calendar.md). You can follow or unfollow other users from either:
- Their [user profiles](#access-your-user-profile).
- The small popover that appears when you hover over a user's name ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/76050)
@@ -326,83 +326,6 @@ To view a user's activity in a top-level Activity view:
1. In the GitLab menu, select **Activity**.
1. Select the **Followed users** tab.
-### User contribution events
-
-Each of these contribution events is tracked:
-
-- `approved`
- - Merge request
-- `closed`
- - [Epic](../group/epics/index.md)
- - Issue
- - Merge request
- - Milestone
-- `commented` on any `Noteable` record.
- - Alert
- - Commit
- - Design
- - Issue
- - Merge request
- - Snippet
-- `created`
- - Design
- - [Epic](../group/epics/index.md)
- - Issue
- - Merge request
- - Milestone
- - Project
- - Wiki page
-- `destroyed`
- - Design
- - Milestone
- - Wiki page
-- `expired`
- - Project membership
-- `joined`
- - Project membership
-- `left`
- - Project membership
-- `merged`
- - Merge request
-- `pushed` commits to (or deleted commits from) a repository, individually or in bulk.
- - Project
-- `reopened`
- - [Epic](../group/epics/index.md)
- - Issue
- - Merge request
- - Milestone
-- `updated`
- - Design
- - Wiki page
-
-### Retrieve user activity as a feed
-
-GitLab provides RSS feeds of user activity. To subscribe to the
-RSS feed of a user's activity:
-
-1. Go to the [user's profile](#access-your-user-profile).
-1. In the top right, select the feed symbol **{rss}** to display the results as an RSS feed in Atom format.
-
-The URL of the result contains both a feed token, and
-the user's activity that you're authorized to view.
-You can add this URL to your feed reader.
-
-### Reset the user activity feed token
-
-Feed tokens are sensitive and can reveal information from confidential issues.
-If you think your feed token has been exposed, you should reset it.
-
-To reset your feed token:
-
-1. On the top bar, in the top right corner, select your avatar.
-1. Select **Edit profile**.
-1. On the left sidebar, select **Access Tokens**.
-1. Scroll down. In the **Feed token** section, select the
- **reset this token** link.
-1. On the confirmation box, select **OK**.
-
-A new token is generated.
-
## Troubleshooting
### Why do I keep getting signed out?
diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md
index 7942ee38be9..1deb4842107 100644
--- a/doc/user/profile/notifications.md
+++ b/doc/user/profile/notifications.md
@@ -109,7 +109,7 @@ To select a notification level for a group, use either of these methods:
Or:
1. On the top bar, select **Main menu > Groups** and find your group.
-1. Select the notification dropdown, next to the bell icon (**{notifications}**).
+1. Select the notification dropdown list, next to the bell icon (**{notifications}**).
1. Select the desired [notification level](#notification-levels).
#### Change email address used for group notifications
@@ -140,7 +140,7 @@ To select a notification level for a project, use either of these methods:
Or:
1. On the top bar, select **Main menu > Projects** and find your project.
-1. Select the notification dropdown, next to the bell icon (**{notifications}**).
+1. Select the notification dropdown list, next to the bell icon (**{notifications}**).
1. Select the desired [notification level](#notification-levels).
<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>
@@ -194,6 +194,7 @@ Users are notified of the following events:
| Personal access tokens expiring soon | User | Security email, always sent. |
| Personal access tokens have been created | User | Security email, always sent. _[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/337591) in GitLab 14.9._ |
| Personal access tokens have expired | User | Security email, always sent. |
+| Personal access token has been revoked | User | Security email, always sent. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98911) in GitLab 15.5. |
| Project access level changed | User | Sent when user project access level is changed. |
| SSH key has expired | User | Security email, always sent. _[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.12._ |
| Two-factor authentication disabled | User | Security email, always sent. |
@@ -227,20 +228,20 @@ to change their user notification settings to **Watch** instead.
### Edit notification settings for issues, merge requests, and epics
-To enable notifications on a specific issue, merge request, or epic, you must turn on the
-**Notifications** toggle in the right sidebar.
+To toggle notifications on an issue, merge request, or epic: on the right sidebar, turn on or off the **Notifications** toggle.
-- To subscribe, **turn on** if you are not a participant in the discussion, but want to receive
- notifications on each update.
+When you **turn on** notifications, you start receiving notifications on each update, even if you
+haven't participated in the discussion.
+When you turn notifications on in an epic, you aren't automatically subscribed to the issues linked
+to the epic.
- When you turn notifications on in an epic, you aren't automatically subscribed to the issues linked
- to the epic.
-
-- To unsubscribe, **turn off** if you are receiving notifications for updates but no longer want to
- receive them.
+When you **turn off** notifications, you stop receiving notifications for updates.
+Turning this toggle off only unsubscribes you from updates related to this issue, merge request, or epic.
+Learn how to [opt out of all emails from GitLab](#opt-out-of-all-gitlab-emails).
- Turning this toggle off only unsubscribes you from updates related to this issue, merge request, or epic.
- Learn how to [opt out of all emails from GitLab](#opt-out-of-all-gitlab-emails).
+<!-- Delete when the `moved_mr_sidebar` feature flag is removed -->
+If you don't see this action on the right sidebar, your project or instance may have
+enabled a feature flag for [moved sidebar actions](../project/merge_requests/index.md#move-sidebar-actions).
### Notification events on issues, merge requests, and epics
@@ -356,13 +357,13 @@ a merge request or an issue.
The following table lists all GitLab-specific email headers:
| Header | Description |
-|---------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------|
+| ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| `List-Id` | The path of the project in an RFC 2919 mailing list identifier. You can use it for email organization with filters. |
| `X-GitLab-(Resource)-ID` | The ID of the resource the notification is for. The resource, for example, can be `Issue`, `MergeRequest`, `Commit`, or another such resource. |
| `X-GitLab-Discussion-ID` | The ID of the thread the comment belongs to, in notification emails for comments. |
| `X-GitLab-Group-Id` | The group's ID. Only present on notification emails for [epics](../group/epics/index.md). |
| `X-GitLab-Group-Path` | The group's path. Only present on notification emails for [epics](../group/epics/index.md) |
-| [`X-GitLab-NotificationReason`](#x-gitlab-notificationreason) | The reason for the notification. This can be `mentioned`, `assigned`, or `own_activity`. |
+| `X-GitLab-NotificationReason` | The reason for the notification. [See possible values.](#x-gitlab-notificationreason). |
| `X-GitLab-Pipeline-Id` | The ID of the pipeline the notification is for, in notification emails for pipelines. |
| `X-GitLab-Project-Id` | The project's ID. |
| `X-GitLab-Project-Path` | The project's path. |
@@ -376,21 +377,35 @@ The value is one of the following, in order of priority:
- `own_activity`
- `assigned`
+- `review_requested`
- `mentioned`
+- `subscribed`
The reason for the notification is also included in the footer of the notification email.
For example, an email with the reason `assigned` has this sentence in the footer:
> You are receiving this email because you have been assigned an item on \<configured GitLab hostname>.
-For example, an alert notification email can have one of
-[the alert's](../../operations/incident_management/alerts.md) statuses:
+#### On-call alerts notifications **(PREMIUM)**
+
+An [on-call alert](../../operations/incident_management/oncall_schedules.md)
+notification email can have one of [the alert's](../../operations/incident_management/alerts.md) statuses:
- `alert_triggered`
- `alert_acknowledged`
- `alert_resolved`
- `alert_ignored`
+#### Incident escalation notifications **(PREMIUM)**
+
+An [incident escalation](../../operations/incident_management/escalation_policies.md)
+notification email can have one of [the incident's](../../operations/incident_management/incidents.md) status:
+
+- `incident_triggered`
+- `incident_acknowledged`
+- `incident_resolved`
+- `incident_ignored`
+
Expanding the list of events included in the `X-GitLab-NotificationReason` header is tracked in
[issue 20689](https://gitlab.com/gitlab-org/gitlab/-/issues/20689).
diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md
index c7fe68c0609..507ad6378bc 100644
--- a/doc/user/profile/personal_access_tokens.md
+++ b/doc/user/profile/personal_access_tokens.md
@@ -89,8 +89,10 @@ At any time, you can revoke a personal access token.
## View the last time a token was used
-Token usage is updated once every 24 hours. It is updated each time the token is used to request
-[API resources](../../api/api_resources.md) and the [GraphQL API](../../api/graphql/index.md).
+Token usage information is updated every 24 hours. GitLab considers a token used when the token is used to:
+
+- Authenticate with the [REST](../../api/index.md) or [GraphQL](../../api/graphql/index.md) APIs.
+- Perform a Git operation.
To view the last time a token was used:
@@ -195,17 +197,29 @@ This code can be shortened into a single-line shell command using the
sudo gitlab-rails runner "PersonalAccessToken.find_by_token('token-string-here123').revoke!"
```
-<!-- ## Troubleshooting
+## Troubleshooting
+
+### Unrevoke a personal access token **(FREE SELF)**
+
+If a personal access token is revoked accidentally by any method, administrators can unrevoke that token.
+
+WARNING:
+Running the following commands changes data directly. This could be damaging if not done correctly, or under the right conditions. You should first run these commands in a test environment with a backup of the instance ready to be restored, just in case.
+
+1. Open a [Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session).
+1. Unrevoke the token:
-Include any troubleshooting steps that you can foresee. If you know beforehand what issues
-one might have when setting this up, or when something is changed, or on upgrading, it's
-important to describe those, too. Think of things that may go wrong and include them here.
-This is important to minimize requests for support, and to avoid doc comments with
-questions that you know someone might ask.
+ ```ruby
+ token = PersonalAccessToken.find_by_token('<token_string>')
+ token.update!(revoked:false)
+ ```
+
+ For example, to unrevoke a token of `token-string-here123`:
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
-If you have none to add when creating a doc, leave this section in place
-but commented out to help encourage others to add to it in the future. -->
+ ```ruby
+ token = PersonalAccessToken.find_by_token('token-string-here123')
+ token.update!(revoked:false)
+ ```
## Alternatives to personal access tokens
diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md
index 6849918886a..dce8684d993 100644
--- a/doc/user/profile/preferences.md
+++ b/doc/user/profile/preferences.md
@@ -137,7 +137,7 @@ You can include the following options for your default dashboard view:
### Group overview content
-The **Group overview content** dropdown allows you to choose what information is
+The **Group overview content** dropdown list allows you to choose what information is
displayed on a group's home page.
You can choose between 2 options:
@@ -230,6 +230,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/profile/user_passwords.md b/doc/user/profile/user_passwords.md
index 04d149c9709..b8dbdcdd956 100644
--- a/doc/user/profile/user_passwords.md
+++ b/doc/user/profile/user_passwords.md
@@ -60,7 +60,8 @@ Self-managed installations can configure the following additional password requi
## Block weak passwords
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23610) in GitLab 15.4 [with a flag](../../administration/feature_flags.md) named `block_weak_passwords`, weak passwords aren't accepted. Disabled by default.
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23610) in GitLab 15.4 [with a flag](../../administration/feature_flags.md) named `block_weak_passwords`, weak passwords aren't accepted. Disabled by default on self-managed.
+> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/363445) on GitLab.com.
FLAG:
On self-managed GitLab, by default blocking weak passwords is not available. To make it available, ask an administrator
diff --git a/doc/user/project/badges.md b/doc/user/project/badges.md
index d43af92e9c6..5d1d10fc37d 100644
--- a/doc/user/project/badges.md
+++ b/doc/user/project/badges.md
@@ -128,7 +128,7 @@ To add a new badge to a group or project with a custom image:
1. Select **Add badge**.
To learn how to use custom images generated via a pipeline, see our documentation on
-[accessing the latest job artifacts by URL](../../ci/pipelines/job_artifacts.md#access-the-latest-job-artifacts-by-url).
+[accessing the latest job artifacts by URL](../../ci/pipelines/job_artifacts.md#access-the-latest-job-artifacts).
## API
diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md
index 363197107f9..0b0555806ce 100644
--- a/doc/user/project/clusters/add_existing_cluster.md
+++ b/doc/user/project/clusters/add_existing_cluster.md
@@ -69,7 +69,7 @@ To add a Kubernetes cluster to your project, group, or instance:
1. Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster.
1. Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster.
1. **Main menu > Admin > Kubernetes** page, for an instance-level cluster.
-1. On the **Kubernetes clusters** page, select the **Connect with a certificate** option from the **Actions** dropdown menu.
+1. On the **Kubernetes clusters** page, select the **Connect with a certificate** option from the **Actions** dropdown list.
1. On the **Connect a cluster** page, fill in the details:
1. **Kubernetes cluster name** (required) - The name you wish to give the cluster.
1. **Environment scope** (required) - The
diff --git a/doc/user/project/clusters/deploy_to_cluster.md b/doc/user/project/clusters/deploy_to_cluster.md
index e8acf5a2727..65e4a5d9fde 100644
--- a/doc/user/project/clusters/deploy_to_cluster.md
+++ b/doc/user/project/clusters/deploy_to_cluster.md
@@ -131,7 +131,7 @@ However, sometimes GitLab cannot create them. In such instances, your job can fa
This job failed because the necessary resources were not successfully created.
```
-To find the cause of this error when creating a namespace and service account, check the [logs](../../../administration/logs/index.md#kuberneteslog).
+To find the cause of this error when creating a namespace and service account, check the [logs](../../../administration/logs/index.md#kuberneteslog-deprecated).
Reasons for failure include:
diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md
deleted file mode 100644
index 51e4f1c2db2..00000000000
--- a/doc/user/project/clusters/kubernetes_pod_logs.md
+++ /dev/null
@@ -1,12 +0,0 @@
----
-stage: Monitor
-group: Respond
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
-remove_date: '2022-10-18'
-redirect_to: '../../clusters/agent/index.md'
----
-
-# Kubernetes Logs (removed) **(FREE SELF)**
-
-This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5
-and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/360193) in GitLab 15.2.
diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md
index c4ca82f7db4..df0ffff8561 100644
--- a/doc/user/project/clusters/runbooks/index.md
+++ b/doc/user/project/clusters/runbooks/index.md
@@ -166,11 +166,11 @@ the components outlined above and the pre-loaded demo runbook.
[GitLab Access Token](../../../profile/personal_access_tokens.md)
and your Project ID in the **Setup** section of the demo runbook:
- 1. Double-click the **DevOps-Runbook-Demo** folder located on the left panel.
+ 1. Select the **DevOps-Runbook-Demo** folder located on the left panel.
![demo runbook](img/demo-runbook.png)
- 1. Double-click the `Nurtch-DevOps-Demo.ipynb` runbook.
+ 1. Select the `Nurtch-DevOps-Demo.ipynb` runbook.
![sample runbook](img/sample-runbook.png)
diff --git a/doc/user/project/import/clearcase.md b/doc/user/project/import/clearcase.md
index 1dc62cbbe35..0878476d59f 100644
--- a/doc/user/project/import/clearcase.md
+++ b/doc/user/project/import/clearcase.md
@@ -27,7 +27,7 @@ The following table illustrates the main differences between ClearCase and Git:
| Server | UNIX, Windows legacy systems | UNIX, macOS |
| License | Proprietary | GPL |
-_Taken from the slides [ClearCase and the journey to Git](https://docplayer.net/42708453-Clearcase-the-journey-to-git-migrating-your-skills-and-vobs-to-git.html) provided by [collab.net](https://www.collab.net/)_
+_Taken from the slides [ClearCase and the journey to Git](https://docplayer.net/42708453-Clearcase-the-journey-to-git-migrating-your-skills-and-vobs-to-git.html) provided by `collab.net`_
## Why migrate
diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md
index 03f6fd20b0a..c0b13c76322 100644
--- a/doc/user/project/import/github.md
+++ b/doc/user/project/import/github.md
@@ -112,8 +112,8 @@ If you are not using the GitHub integration, you can still perform an authorizat
1. Select **Generate token**.
1. Copy the token hash.
1. Go back to GitLab and provide the token to the GitHub importer.
-1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads your repositories' information.
- Once done, you are taken to the importer page to select the repositories to import.
+1. Select **List Your GitHub Repositories** and wait while GitLab reads your repositories' information.
+ When done, you are taken to the importer page to select the repositories to import.
To use a newer personal access token in imports after previously performing these steps, sign out of
your GitLab account and sign in again, or revoke the older personal access token in GitHub.
@@ -202,18 +202,21 @@ The following items of a project are imported:
- Merge Request description. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18052) in GitLab 15.5.
All attachment imports are disabled by default behind
- `github_importer_attachments_import` [feature flag](../../../administration/feature_flags.md). From GitLab 15.5, can be imported
- [as an additional item](#select-additional-items-to-import). The feature flag was removed.
+ `github_importer_attachments_import` [feature flag](../../../administration/feature_flags.md). From GitLab 15.5, can
+ be imported [as an additional item](#select-additional-items-to-import). The feature flag was removed.
- Pull request review comments.
- Regular issue and pull request comments.
- [Git Large File Storage (LFS) Objects](../../../topics/git/lfs/index.md).
-- Pull request reviews (GitLab.com and GitLab 13.7 and later).
-- Pull request "merged by" information (GitLab.com and GitLab 13.7 and later).
-- Pull request comments replies in discussions ([GitLab.com and GitLab 14.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/336596)).
-- Diff Notes suggestions ([GitLab.com and GitLab 14.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/340624)).
-- Issue events and pull requests events. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7673) in GitLab 15.4 with `github_importer_issue_events_import`
- [feature flag](../../../administration/feature_flags.md) disabled by default.
- From GitLab 15.5, can be imported [as an additional item](#select-additional-items-to-import). The feature flag was removed.
+- Pull request reviews.
+- Pull request assigned reviewers. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355137) in GitLab 15.6.
+- Pull request "merged by" information.
+- Pull request comments replies in discussions. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336596) in
+ GitLab 14.5.
+- Diff Notes suggestions. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340624) in GitLab 14.7.
+- Issue events and pull requests events. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7673) in GitLab 15.4
+ with `github_importer_issue_events_import` [feature flag](../../../administration/feature_flags.md) disabled by default.
+ From GitLab 15.5, can be imported [as an additional item](#select-additional-items-to-import). The feature flag was
+ removed.
References to pull requests and issues are preserved. Each imported repository maintains visibility level unless that
[visibility level is restricted](../../public_access.md#restrict-use-of-public-or-internal-projects), in which case it
@@ -225,7 +228,13 @@ Supported GitHub branch protection rules are mapped to GitLab branch protection
- GitHub rule **Require conversation resolution before merging** for the project's default branch is mapped to the [**All threads must be resolved** GitLab setting](../../discussions/index.md#prevent-merge-unless-all-threads-are-resolved). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371110) in GitLab 15.5.
- GitHub rule **Require a pull request before merging** is mapped to the **No one** option in the **Allowed to push** list of the branch protection rule. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370951) in GitLab 15.5.
-- GitHub rule **Require signed commits** for the project's default branch is mapped to the **Reject unsigned commits** GitLab setting. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370949) in GitLab 15.5.
+- GitHub rule **Require a pull request before merging - Require review from Code Owners** is mapped to the
+ [**Code owner approval** branch protection rule](../protected_branches.md#require-code-owner-approval-on-a-protected-branch). Requires GitLab Premium or higher.
+ [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/376683) in GitLab 15.6.
+- GitHub rule **Require signed commits** for the project's default branch is mapped to the **Reject unsigned commits** GitLab push rule. Requires GitLab Premium or higher.
+ [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370949) in GitLab 15.5.
+- GitHub rule **Allow force pushes - Everyone** is mapped to the [**Allowed to force push** branch protection rule](../protected_branches.md#allow-force-push-on-a-protected-branch). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370943) in GitLab 15.6.
+- GitHub rule **Allow force pushes - Specify who can force push** is proposed in issue [370945](https://gitlab.com/gitlab-org/gitlab/-/issues/370945).
- Support for GitHub rule **Require status checks to pass before merging** was proposed in issue [370948](https://gitlab.com/gitlab-org/gitlab/-/issues/370948). However, this rule cannot be translated during project import into GitLab due to technical difficulties.
You can still create [status checks](../merge_requests/status_checks.md) in GitLab yourself.
@@ -263,6 +272,112 @@ To disable the feature, run this command:
Feature.disable(:github_importer_lower_per_page_limit, group)
```
+## Import from GitHub Enterprise on an internal network
+
+If your GitHub Enterprise instance is on a internal network that is unaccessible to the internet, you can use a reverse proxy
+to allow GitLab.com to access the instance.
+
+The proxy needs to:
+
+- Forward requests to the GitHub Enterprise instance.
+- Convert to the public proxy hostname all occurrences of the internal hostname in:
+ - The API response body.
+ - The API response `Link` header.
+
+GitHub API uses the `Link` header for pagination.
+
+After configuring the proxy, test it by making API requests. Below there are some examples of commands to test the API:
+
+```shell
+curl --header "Authorization: Bearer <YOUR-TOKEN>" "https://{PROXY_HOSTNAME}/user"
+
+### URLs in the response body should use the proxy hostname
+
+{
+ "login": "example_username",
+ "id": 1,
+ "url": "https://{PROXY_HOSTNAME}/users/example_username",
+ "html_url": "https://{PROXY_HOSTNAME}/example_username",
+ "followers_url": "https://{PROXY_HOSTNAME}/api/v3/users/example_username/followers",
+ ...
+ "created_at": "2014-02-11T17:03:25Z",
+ "updated_at": "2022-10-18T14:36:27Z"
+}
+```
+
+```shell
+curl --head --header "Authorization: Bearer <YOUR-TOKEN>" "https://{PROXY_DOMAIN}/api/v3/repos/{repository_path}/pulls?states=all&sort=created&direction=asc"
+
+### Link header should use the proxy hostname
+
+HTTP/1.1 200 OK
+Date: Tue, 18 Oct 2022 21:42:55 GMT
+Server: GitHub.com
+Content-Type: application/json; charset=utf-8
+Cache-Control: private, max-age=60, s-maxage=60
+...
+X-OAuth-Scopes: repo
+X-Accepted-OAuth-Scopes:
+github-authentication-token-expiration: 2022-11-22 18:13:46 UTC
+X-GitHub-Media-Type: github.v3; format=json
+X-RateLimit-Limit: 5000
+X-RateLimit-Remaining: 4997
+X-RateLimit-Reset: 1666132381
+X-RateLimit-Used: 3
+X-RateLimit-Resource: core
+Link: <https://{PROXY_DOMAIN}/api/v3/repositories/1/pulls?page=2>; rel="next", <https://{PROXY_DOMAIN}/api/v3/repositories/1/pulls?page=11>; rel="last"
+```
+
+Also test that cloning the repository using the proxy does not fail:
+
+```shell
+git clone -c http.extraHeader="Authorization: basic <base64 encode YOUR-TOKEN>" --mirror https://{PROXY_DOMAIN}/{REPOSITORY_PATH}.git
+```
+
+### Sample reverse proxy configuration
+
+The following configuration is an example on how to configure Apache HTTP Server as a reverse proxy
+
+WARNING:
+For simplicity, the snippet does not have configuration to encrypt the connection between the client and the proxy. However, for security reasons you should include that
+configuration. See [sample Apache TLS/SSL configuration](https://ssl-config.mozilla.org/#server=apache&version=2.4.41&config=intermediate&openssl=1.1.1k&guideline=5.6).
+
+```plaintext
+# Required modules
+LoadModule filter_module lib/httpd/modules/mod_filter.so
+LoadModule reflector_module lib/httpd/modules/mod_reflector.so
+LoadModule substitute_module lib/httpd/modules/mod_substitute.so
+LoadModule deflate_module lib/httpd/modules/mod_deflate.so
+LoadModule headers_module lib/httpd/modules/mod_headers.so
+LoadModule proxy_module lib/httpd/modules/mod_proxy.so
+LoadModule proxy_connect_module lib/httpd/modules/mod_proxy_connect.so
+LoadModule proxy_http_module lib/httpd/modules/mod_proxy_http.so
+LoadModule ssl_module lib/httpd/modules/mod_ssl.so
+
+<VirtualHost GITHUB_ENTERPRISE_HOSTNAME:80>
+ ServerName GITHUB_ENTERPRISE_HOSTNAME
+
+ # Enables reverse-proxy configuration with SSL support
+ SSLProxyEngine On
+ ProxyPass "/" "https://GITHUB_ENTERPRISE_HOSTNAME/"
+ ProxyPassReverse "/" "https://GITHUB_ENTERPRISE_HOSTNAME/"
+
+ # Replaces occurrences of the local GitHub Enterprise URL with the Proxy URL
+ # GitHub Enterprise compresses the responses, the filters INFLATE and DEFLATE needs to be used to
+ # decompress and compress the response back
+ AddOutputFilterByType INFLATE;SUBSTITUTE;DEFLATE application/json
+ Substitute "s|https://GITHUB_ENTERPRISE_HOSTNAME|https://PROXY_HOSTNAME|ni"
+ SubstituteMaxLineLength 50M
+
+ # GitHub API uses the response header "Link" for the API pagination
+ # For example:
+ # <https://example.com/api/v3/repositories/1/issues?page=2>; rel="next", <https://example.com/api/v3/repositories/1/issues?page=3>; rel="last"
+ # The directive below replaces all occurrences of the GitHub Enterprise URL with the Proxy URL if the
+ # response header Link is present
+ Header edit* Link "https://GITHUB_ENTERPRISE_HOSTNAME" "https://PROXY_HOSTNAME"
+</VirtualHost>
+```
+
## Automate group and project import **(PREMIUM)**
For information on automating user, group, and project import API calls, see
@@ -279,8 +394,8 @@ repository to be imported manually. Administrators can manually import the repos
1. Run the following series of commands in the console:
```ruby
- project_id = <PROJECT_ID>
- github_access_token = <GITHUB_ACCESS_TOKEN>
+ project_id = <PROJECT_ID>
+ github_access_token = <GITHUB_ACCESS_TOKEN>
github_repository_path = '<GROUP>/<REPOSITORY>'
github_repository_url = "https://#{github_access_token}@github.com/#{github_repository_path}.git"
diff --git a/doc/user/project/import/tfvc.md b/doc/user/project/import/tfvc.md
index 3625355340b..0a03513467e 100644
--- a/doc/user/project/import/tfvc.md
+++ b/doc/user/project/import/tfvc.md
@@ -7,7 +7,7 @@ type: concepts
# Migrate from TFVC to Git **(FREE)**
-Team Foundation Server (TFS), renamed [Azure DevOps Server](https://azure.microsoft.com/en-us/services/devops/server/)
+Team Foundation Server (TFS), renamed [Azure DevOps Server](https://azure.microsoft.com/en-us/products/devops/server/)
in 2019, is a set of tools developed by Microsoft which also includes
[Team Foundation Version Control](https://learn.microsoft.com/en-us/azure/devops/repos/tfvc/what-is-tfvc?view=azure-devops)
(TFVC), a centralized version control system similar to Git.
diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md
index 07c99653a0e..8d6fdf882b7 100644
--- a/doc/user/project/integrations/gitlab_slack_application.md
+++ b/doc/user/project/integrations/gitlab_slack_application.md
@@ -4,35 +4,35 @@ group: Integrations
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# GitLab Slack application **(FREE SAAS)**
+# GitLab for Slack app **(FREE SAAS)**
NOTE:
-The GitLab Slack application is only configurable for GitLab.com. It will **not**
-work for on-premises installations where you can configure the
-[Slack slash commands](slack_slash_commands.md) integration instead. We're planning
-to make this configurable for all GitLab installations, but there's
-no ETA - see [#28164](https://gitlab.com/gitlab-org/gitlab/-/issues/28164).
+The GitLab for Slack app is only configurable for GitLab.com. It does **not**
+work for on-premises installations where you can configure
+[Slack slash commands](slack_slash_commands.md) instead. See
+[Slack application integration for self-managed instances](https://gitlab.com/gitlab-org/gitlab/-/issues/28164)
+for our plans to make the app configurable for all GitLab installations.
-Slack provides a native application which you can enable via your project's
+Slack provides a native application that you can enable with your project's
integrations on GitLab.com.
## Slack App Directory
-The simplest way to enable the GitLab Slack application for your workspace is to
-install the [GitLab application](https://slack-platform.slack.com/apps/A676ADMV5-gitlab) from
-the [Slack App Directory](https://slack.com/apps).
+To enable the GitLab for Slack app for your workspace,
+install the [GitLab application](https://slack-platform.slack.com/apps/A676ADMV5-gitlab)
+from the [Slack App Directory](https://slack.com/apps).
-Selecting install takes you to the [GitLab Slack application landing page](https://gitlab.com/-/profile/slack/edit)
-where you can select a project to enable the GitLab Slack application for.
+On the [GitLab for Slack app landing page](https://gitlab.com/-/profile/slack/edit),
+you can select a GitLab project to link with your Slack workspace.
## Configuration
-Alternatively, you can configure the Slack application with a project's
+Alternatively, you can configure the GitLab for Slack app with your project's
integration settings.
-Keep in mind that you must have the appropriate permissions for your Slack
-workspace to be able to install a new application. Read more in Slack's
-documentation on [Adding an app to your workspace](https://slack.com/help/articles/202035138-Add-apps-to-your-Slack-workspace).
+You must have the appropriate permissions for your Slack
+workspace to be able to install a new application. See
+[Add apps to your Slack workspace](https://slack.com/help/articles/202035138-Add-apps-to-your-Slack-workspace).
To enable the GitLab integration for your Slack workspace:
@@ -41,23 +41,21 @@ To enable the GitLab integration for your Slack workspace:
1. Select **Install Slack app**.
1. Select **Allow** on Slack's confirmation screen.
-That's all! You can now start using the Slack slash commands.
-
You can also select **Reinstall Slack app** to update the app in your Slack workspace
-to the latest version. See the [Version history](#version-history) for details.
+to the latest version. See [Version history](#version-history) for details.
## Create a project alias for Slack
To create a project alias on GitLab.com for Slack integration:
1. Go to your project's home page.
-1. Go to **Settings > Integrations** (only visible on GitLab.com)
+1. Go to **Settings > Integrations** (only visible on GitLab.com).
1. On the **Integrations** page, select **Slack application**.
1. The current **Project Alias**, if any, is displayed. To edit this value,
select **Edit**.
1. Enter your desired alias, and select **Save changes**.
-Some Slack commands require a project alias, and fail with the following error
+Some Slack commands require a project alias and fail with the following error
if the project alias is incorrect or missing from the command:
```plaintext
@@ -66,17 +64,15 @@ GitLab error: project or alias not found
## Usage
-After confirming the installation, you, and everyone else in your Slack workspace,
-can use all the [slash commands](../../../integration/slash_commands.md).
-
-When you perform your first slash command, you are asked to authorize your
-Slack user on GitLab.com.
+After installing the app, everyone in your Slack workspace can
+use the [slash commands](../../../integration/slash_commands.md).
+When you perform your first slash command, you are asked to
+authorize your Slack user on GitLab.com.
The only difference with the [manually configurable Slack slash commands](slack_slash_commands.md)
-is that all the commands should be prefixed with the `/gitlab` keyword.
-
-For example, to show the issue number `1001` under the `gitlab-org/gitlab`
-project, you would do:
+is that you must prefix all commands with the `/gitlab` keyword. For example,
+to show the issue number `1001` under the `gitlab-org/gitlab`
+project, you must run the following command:
```plaintext
/gitlab gitlab-org/gitlab issue show 1001
@@ -84,15 +80,11 @@ project, you would do:
## Version history
-### 15.0+
-
-In GitLab 15.0 the Slack app is updated to [Slack's new granular permissions app model](https://medium.com/slack-developer-blog/more-precision-less-restrictions-a3550006f9c3).
-
-There is no change in functionality. A reinstall is not required but recommended.
+In GitLab 15.0 and later, the GitLab for Slack app is updated to [Slack's new granular permissions model](https://medium.com/slack-developer-blog/more-precision-less-restrictions-a3550006f9c3). While there is no change in functionality, you should reinstall the app.
## Troubleshooting
-When you work with the Slack app, the
+When you work with the GitLab for Slack app, the
[App Home](https://api.slack.com/start/overview#app_home) might not display properly.
As a workaround, ensure your app is up to date.
diff --git a/doc/user/project/integrations/hangouts_chat.md b/doc/user/project/integrations/hangouts_chat.md
index 1be0db223ac..c6e102b1d74 100644
--- a/doc/user/project/integrations/hangouts_chat.md
+++ b/doc/user/project/integrations/hangouts_chat.md
@@ -28,7 +28,7 @@ notifications to Google Chat:
To enable the integration in Google Chat:
1. Enter the room where you want to receive notifications from GitLab.
-1. Open the room dropdown menu on the top-left and select **Manage webhooks**.
+1. Open the room dropdown list on the top-left and select **Manage webhooks**.
1. Enter the name for your webhook, for example "GitLab integration".
1. Optional. Add an avatar for your bot.
1. Select **Save**.
diff --git a/doc/user/project/integrations/mlflow_client.md b/doc/user/project/integrations/mlflow_client.md
new file mode 100644
index 00000000000..82bfd08e926
--- /dev/null
+++ b/doc/user/project/integrations/mlflow_client.md
@@ -0,0 +1,81 @@
+---
+stage: Create
+group: Incubation
+info: Machine Learning Experiment Tracking is a GitLab Incubation Engineering program. No technical writer assigned to this group.
+---
+
+# MLFlow Client Integration **(FREE)**
+
+> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8560) in GitLab 15.6 as an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) release [with a flag](../../../administration/feature_flags.md) named `ml_experiment_tracking`. Disabled by default.
+
+DISCLAIMER:
+MLFlow Client Integration is an experimental feature being developed by the Incubation Engineering Department,
+and will receive significant changes over time.
+
+[MLFlow](https://mlflow.org/) is one of the most popular open source tools for Machine Learning Experiment Tracking.
+GitLabs works as a backend to the MLFlow Client, [logging experiments](../ml/experiment_tracking/index.md).
+Setting up your integrations requires minimal changes to existing code.
+
+GitLab plays the role of proxy server, both for artifact storage and tracking data. It reflects the
+MLFlow [Scenario 5](https://www.mlflow.org/docs/latest/tracking.html#scenario-5-mlflow-tracking-server-enabled-with-proxied-artifact-storage-access).
+
+## Enable MFlow Client Integration
+
+Complete this task to enable MFlow Client Integration.
+
+Prerequisites:
+
+- A [personal access token](../../../user/profile/personal_access_tokens.md) for the project, with minimum access level of `api`.
+- The project ID. To find the project ID, on the top bar, select **Main menu > Projects** and find your project. On the left sidebar, select **Settings > General**.
+
+1. Set the tracking URI and token environment variables on the host that runs the code (your local environment, CI pipeline, or remote host).
+
+ For example:
+
+ ```shell
+ export MLFLOW_TRACKING_URI="http://<your gitlab endpoint>/api/v4/projects/<your project id>/ml/mlflow"
+ export MLFLOW_TRACKING_TOKEN="<your_access_token>"
+ ```
+
+1. If your training code contains the call to `mlflow.set_tracking_uri()`, remove it.
+
+When running the training code, MLFlow will create experiments, runs, log parameters, metrics,
+and artifacts on GitLab.
+
+After experiments are logged, they are listed under `/<your project>/-/ml/experiments`. Runs are registered as Model Candidates,
+that can be explored by selecting an experiment.
+
+## Limitations
+
+- The API GitLab supports is the one defined at MLFlow version 1.28.0.
+- API endpoints not listed above are not supported.
+- During creation of experiments and runs, tags are ExperimentTags and RunTags are ignored.
+- MLFLow Model Registry is not supported.
+
+## Supported methods and caveats
+
+This is a list of methods we support from the MLFlow client. Other methods might be supported but were not
+tested. More information can be found in the [MLFlow Documentation](https://www.mlflow.org/docs/1.28.0/python_api/mlflow.html).
+
+### `set_experiment`
+
+Accepts both experiment_name and experiment_id
+
+### `start_run()`
+
+- Nested runs have not been tested.
+- `run_name` is not supported
+
+### `log_param()`, `log_params()`, `log_metric()`, `log_metrics()`
+
+Work as defined by the documentation
+
+### `log_artifact()`, `log_artifacts()`
+
+`artifact_path` must be empty string.
+
+### `log_model()`
+
+This is an experimental method in MLFlow, and partial support is offered. It stores the model artifacts, but does
+not log the model information. The `artifact_path` parameter must be set to `''`, because Generic Packages do not support folder
+structure.
diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md
index 99466a67417..947210541f4 100644
--- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md
+++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md
@@ -38,7 +38,7 @@ Next, the Ingress needs to be annotated for Prometheus monitoring. Two new annot
- `prometheus.io/scrape: "true"`
- `prometheus.io/port: "10254"`
-Managing these settings depends on how NGINX Ingress has been deployed. If you have deployed via the [official Helm chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress), metrics can be enabled with `controller.stats.enabled` along with the required annotations. Alternatively it is possible to edit the NGINX Ingress YAML directly in the [Kubernetes dashboard](https://github.com/kubernetes/dashboard).
+Managing these settings depends on how NGINX Ingress has been deployed. If you have deployed via the [official Helm chart](https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx), metrics can be enabled with `controller.stats.enabled` along with the required annotations. Alternatively it is possible to edit the NGINX Ingress YAML directly in the [Kubernetes dashboard](https://github.com/kubernetes/dashboard).
## Specifying the Environment label
diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md
index e26f93351a1..e6f2ac2753a 100644
--- a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md
+++ b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md
@@ -38,7 +38,7 @@ Next, the Ingress needs to be annotated for Prometheus monitoring. Two new annot
- `prometheus.io/scrape: "true"`
- `prometheus.io/port: "10254"`
-Managing these settings depends on how NGINX Ingress has been deployed. If you have deployed via the [official Helm chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress), metrics can be enabled with `controller.stats.enabled` along with the required annotations. Alternatively it is possible edit the NGINX Ingress YAML directly in the [Kubernetes dashboard](https://github.com/kubernetes/dashboard).
+Managing these settings depends on how NGINX Ingress has been deployed. If you have deployed via the [official Helm chart](https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx), metrics can be enabled with `controller.stats.enabled` along with the required annotations. Alternatively it is possible edit the NGINX Ingress YAML directly in the [Kubernetes dashboard](https://github.com/kubernetes/dashboard).
## Specifying the Environment label
diff --git a/doc/user/project/integrations/servicenow.md b/doc/user/project/integrations/servicenow.md
index d528d1a5547..fcc8db7cb87 100644
--- a/doc/user/project/integrations/servicenow.md
+++ b/doc/user/project/integrations/servicenow.md
@@ -35,7 +35,5 @@ You can:
For more information, refer to the following ServiceNow resources:
- [ServiceNow DevOps home page](https://www.servicenow.com/products/devops.html)
-- [Install DevOps](https://docs.servicenow.com/bundle/paris-devops/page/product/enterprise-dev-ops/task/activate-dev-ops.html)
-- [Install DevOps Integrations](https://docs.servicenow.com/bundle/paris-devops/page/product/enterprise-dev-ops/task/activate-dev-ops-integrations.html)
+- [ServiceNow DevOps documentation](https://docs.servicenow.com/bundle/tokyo-devops/page/product/enterprise-dev-ops/concept/dev-ops-bundle-landing-page.html)
- [GitLab SCM and Continuous Integration for DevOps](https://store.servicenow.com/sn_appstore_store.do#!/store/application/54dc4eacdbc2dcd02805320b7c96191e/)
-- [Model a GitLab CI pipeline in DevOps](https://docs.servicenow.com/bundle/paris-devops/page/product/enterprise-dev-ops/task/model-gitlab-pipeline-dev-ops.html).
diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md
index 9fe0c76ec4f..d34c558ebbc 100644
--- a/doc/user/project/integrations/slack.md
+++ b/doc/user/project/integrations/slack.md
@@ -4,9 +4,9 @@ group: Integrations
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Slack notifications service **(FREE)**
+# Slack notifications integration **(FREE)**
-The Slack notifications service enables your GitLab project to send events
+The Slack notifications integration enables your GitLab project to send events
(such as issue creation) to your existing Slack team as notifications. Setting up
Slack notifications requires configuration changes for both Slack and GitLab.
@@ -45,7 +45,7 @@ to control GitLab from Slack. Slash commands are configured separately.
1. Optional. In **Username**, enter the username of the Slack bot that sends
the notifications.
1. Select the **Notify only broken pipelines** checkbox to notify only on failures.
-1. In the **Branches for which notifications are to be sent** dropdown, select which types of branches
+1. In the **Branches for which notifications are to be sent** dropdown list, select which types of branches
to send notifications for.
1. Leave the **Labels to be notified** field blank to get all notifications, or
add labels that the issue or merge request must have to trigger a
@@ -91,7 +91,7 @@ the error message and keep troubleshooting from there.
You might see an entry like the following in your Sidekiq log:
```plaintext
-2019-01-10_13:22:08.42572 2019-01-10T13:22:08.425Z 6877 TID-abcdefg Integrations::ExecuteWorker JID-3bade5fb3dd47a85db6d78c5 ERROR: {:class=>"Integrations::ExecuteWorker :service_class=>"SlackService", :message=>"SSL_connect returned=1 errno=0 state=error: certificate verify failed"}
+2019-01-10_13:22:08.42572 2019-01-10T13:22:08.425Z 6877 TID-abcdefg Integrations::ExecuteWorker JID-3bade5fb3dd47a85db6d78c5 ERROR: {:class=>"Integrations::ExecuteWorker :integration_class=>"SlackService", :message=>"SSL_connect returned=1 errno=0 state=error: certificate verify failed"}
```
This issue occurs when there is a problem with GitLab communicating with Slack,
@@ -128,20 +128,20 @@ the GitLab OpenSSL trust store is incorrect. Typical causes are:
- Overriding the trust store with `gitlab_rails['env'] = {"SSL_CERT_FILE" => "/path/to/file.pem"}`.
- Accidentally modifying the default CA bundle `/opt/gitlab/embedded/ssl/certs/cacert.pem`.
-### Bulk update to disable the Slack Notification service
+### Bulk update to disable the Slack Notification integration
To disable notifications for all projects that have Slack integration enabled,
[start a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session) and use a script similar to the following:
WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
```ruby
# Grab all projects that have the Slack notifications enabled
p = Project.find_by_sql("SELECT p.id FROM projects p LEFT JOIN integrations s ON p.id = s.project_id WHERE s.type_new = 'Slack' AND s.active = true")
-# Disable the service on each of the projects that were found.
+# Disable the integration on each of the projects that were found.
p.each do |project|
- project.slack_service.update!(:active, false)
+ project.slack_integration.update!(:active, false)
end
```
diff --git a/doc/user/project/integrations/slack_slash_commands.md b/doc/user/project/integrations/slack_slash_commands.md
index cb698ac0ee0..f4c789239f3 100644
--- a/doc/user/project/integrations/slack_slash_commands.md
+++ b/doc/user/project/integrations/slack_slash_commands.md
@@ -14,7 +14,7 @@ GitLab can also send events (for example, `issue created`) to Slack as notificat
The [Slack notifications service](slack.md) is configured separately.
NOTE:
-For GitLab.com, use the [GitLab Slack app](gitlab_slack_application.md) instead.
+For GitLab.com, use the [GitLab for Slack app](gitlab_slack_application.md) instead.
## Configure GitLab and Slack
diff --git a/doc/user/project/integrations/unify_circuit.md b/doc/user/project/integrations/unify_circuit.md
index c13f642d9e9..77530b4b417 100644
--- a/doc/user/project/integrations/unify_circuit.md
+++ b/doc/user/project/integrations/unify_circuit.md
@@ -22,7 +22,7 @@ In GitLab:
1. Select the checkboxes corresponding to the GitLab events you want to receive in Unify Circuit.
1. Paste the **Webhook URL** that you copied from the Unify Circuit configuration step.
1. Select the **Notify only broken pipelines** checkbox to notify only on failures.
-1. In the **Branches for which notifications are to be sent** dropdown, select which types of branches to send notifications for.
+1. In the **Branches for which notifications are to be sent** dropdown list, select which types of branches to send notifications for.
1. Select `Save changes` or optionally select **Test settings**.
Your Unify Circuit conversation now starts receiving GitLab event notifications.
diff --git a/doc/user/project/integrations/webex_teams.md b/doc/user/project/integrations/webex_teams.md
index 930ca8e99b8..be4528ba70d 100644
--- a/doc/user/project/integrations/webex_teams.md
+++ b/doc/user/project/integrations/webex_teams.md
@@ -13,7 +13,7 @@ You can configure GitLab to send notifications to a Webex Teams space:
## Create a webhook for the space
-1. Go to the [Incoming Webhooks app page](https://apphub.webex.com/applications/incoming-webhooks-cisco-systems-38054-23307).
+1. Go to the [Incoming Webhooks app page](https://apphub.webex.com/applications/incoming-webhooks-cisco-systems-38054-23307-75252).
1. Select **Connect**, and sign in to Webex Teams if required.
1. Enter a name for the webhook and select the space to receive the notifications.
1. Select **ADD**.
diff --git a/doc/user/project/integrations/webhook_events.md b/doc/user/project/integrations/webhook_events.md
index c0f0f5a0cd4..60187b9a682 100644
--- a/doc/user/project/integrations/webhook_events.md
+++ b/doc/user/project/integrations/webhook_events.md
@@ -611,7 +611,8 @@ Payload example:
"name": "User1",
"username": "user1",
"avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40\u0026d=identicon"
- }
+ },
+ "detailed_merge_status": "checking"
}
}
```
@@ -947,7 +948,8 @@ Payload example:
"type": "ProjectLabel",
"group_id": 41
}],
- "action": "open"
+ "action": "open",
+ "detailed_merge_status": "mergeable"
},
"labels": [{
"id": 206,
@@ -1017,7 +1019,7 @@ Payload example:
```
NOTE:
-The fields `assignee_id`, and `state` are deprecated.
+The fields `assignee_id`, `state`, `merge_status` are deprecated.
## Wiki page events
@@ -1132,6 +1134,7 @@ Payload example:
"target_project_id": 1,
"state": "opened",
"merge_status": "can_be_merged",
+ "detailed_merge_status": "mergeable",
"url": "http://192.168.64.1:3005/gitlab-org/gitlab-test/merge_requests/1"
},
"user":{
@@ -1359,6 +1362,15 @@ Payload example:
## Job events
+- Number of retries [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/382046) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md)
+ named `job_webhook_retries_count`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available,
+ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named
+`job_webhook_retries_count`.
+On GitLab.com, this feature is not available.
+
Job events are triggered when the status of a job changes.
The `commit.id` in the payload is the ID of the pipeline, not the ID of the commit.
@@ -1391,6 +1403,7 @@ Payload example:
"build_duration": null,
"build_allow_failure": false,
"build_failure_reason": "script_failure",
+ "retries_count": 2, // 2 indicates this is the 2nd retry of this job
"pipeline_id": 2366,
"project_id": 380,
"project_name": "gitlab-org/gitlab-test",
diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md
index 9fc9d6e2eda..ef6957ac2d8 100644
--- a/doc/user/project/integrations/webhooks.md
+++ b/doc/user/project/integrations/webhooks.md
@@ -115,6 +115,15 @@ a test request to re-enable a [disabled webhook](#re-enable-disabled-webhooks).
For example, to test `push events`, your project should have at least one commit. The webhook uses this commit in the webhook.
+NOTE:
+Testing is not supported for some types of events for project and groups webhooks.
+Read more in [issue 379201](https://gitlab.com/gitlab-org/gitlab/-/issues/379201).
+
+Prerequisites:
+
+- To test project webhooks, you must have at least the Maintainer role for the project.
+- To test group webhooks, you must have the Owner role for the group.
+
To test a webhook:
1. In your project or group, on the left sidebar, select **Settings > Webhooks**.
@@ -177,9 +186,13 @@ that the request is legitimate.
## Filter push events by branch
-Push events can be filtered by branch using a branch name or wildcard pattern
-to limit which push events are sent to your webhook endpoint. By default,
-all push events are sent to your webhook endpoint. You can configure branch filtering
+You can filter push events by branch. Use one of the following options to filter which push events are sent to your webhook endpoint:
+
+- **All branches**: push events from all branches.
+- **Wildcard pattern**: push events from a branch that matches a wildcard pattern (for example, `*-stable` or `production/*`).
+- **Regular expression**: push events from a branch that matches a regular expression (for example, `(feature|hotfix)/*`).
+
+You can configure branch filtering
in the [webhook settings](#configure-a-webhook-in-gitlab) in your project.
## How image URLs are displayed in the webhook body
@@ -233,6 +246,11 @@ Webhook requests to your endpoint include the following headers:
GitLab records the history of each webhook request.
You can view requests made in the last 2 days in the **Recent events** table.
+Prerequisites:
+
+- To troubleshoot project webhooks, you must have at least the Maintainer role for the project.
+- To troubleshoot group webhooks, you must have the Owner role for the group.
+
To view the table:
1. In your project or group, on the left sidebar, select **Settings > Webhooks**.
diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md
index 1cf902d2290..c2952b23615 100644
--- a/doc/user/project/issue_board.md
+++ b/doc/user/project/issue_board.md
@@ -707,6 +707,14 @@ A few things to remember:
## Troubleshooting issue boards
+### `There was a problem fetching users` on group issue board when filtering by Author or Assignee
+
+If you get a banner with `There was a problem fetching users` error when filtering by author or assignee on
+group issue board, make sure that you are added as a member to the current group.
+Non-members do not have permission to list group members when filtering by author or assignee on issue boards.
+
+To fix this error, you should add all of your users to the top-level group with at least the Guest role.
+
### Use Rails console to fix issue boards not loading and timing out
If you see issue board not loading and timing out in UI, use Rails console to call the Issue Rebalancing service to fix it:
diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md
index b1bb3f0dbf8..2b302a60d63 100644
--- a/doc/user/project/issues/confidential_issues.md
+++ b/doc/user/project/issues/confidential_issues.md
@@ -16,9 +16,9 @@ keep security vulnerabilities private or prevent surprises from leaking out.
You can make an issue confidential when you create or edit an issue.
When you create a new issue, a checkbox right below the text area is available
-to mark the issue as confidential. Check that box and hit the **Create issue**
-button to create the issue. For existing issues, edit them, check the
-confidential checkbox and hit **Save changes**.
+to mark the issue as confidential. Check that box and select **Create issue**
+to create the issue. For existing issues, edit them, check the
+confidential checkbox and select **Save changes**.
When you create a confidential issue in a project, the project becomes listed in the **Contributed projects** section in your [profile](../../profile/index.md). **Contributed projects** does not show information about the confidential issue; it only shows the project name.
diff --git a/doc/user/project/issues/design_management.md b/doc/user/project/issues/design_management.md
index 593557967ed..27d935d0ed1 100644
--- a/doc/user/project/issues/design_management.md
+++ b/doc/user/project/issues/design_management.md
@@ -4,7 +4,7 @@ group: Product Planning
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Design Management **(FREE)**
+# Design management **(FREE)**
> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/660) in GitLab 12.2.
> - Support for SVGs [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12771) in GitLab 12.4.
diff --git a/doc/user/project/issues/due_dates.md b/doc/user/project/issues/due_dates.md
index 6293fe981de..f41c90a74a5 100644
--- a/doc/user/project/issues/due_dates.md
+++ b/doc/user/project/issues/due_dates.md
@@ -35,7 +35,7 @@ The last way to set a due date is by using [quick actions](../quick_actions.md),
You can see issues with their due dates in the issues list.
Overdue issues have their icon and date colored red.
-To sort issues by their due dates, select **Due date** from the dropdown menu on the right.
+To sort issues by their due dates, select **Due date** from the dropdown list on the right.
Issues are then sorted from the earliest due date to the latest.
To display issues with the latest due dates at the top, select **Sort direction** (**{sort-lowest}**).
diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md
index 9ec6d5b300c..8cd211a51c7 100644
--- a/doc/user/project/issues/managing_issues.md
+++ b/doc/user/project/issues/managing_issues.md
@@ -207,7 +207,8 @@ the appropriate project and followed up from there.
### Fields in the new issue form
-> Adding the new issue to an epic [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13847) in GitLab 13.1.
+> - Adding the new issue to an epic [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13847) in GitLab 13.1.
+> - Iteration field [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233517) in GitLab 15.6.
When you're creating a new issue, you can complete the following fields:
@@ -222,6 +223,7 @@ When you're creating a new issue, you can complete the following fields:
- [Due date](due_dates.md)
- [Milestone](../milestones/index.md)
- [Labels](../labels.md)
+- [Iteration](../../group/iterations/index.md)
## Edit an issue
@@ -340,6 +342,29 @@ To move an issue:
### Bulk move issues **(FREE SELF)**
+#### From the issues list
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15991) in GitLab 15.6.
+
+You can move multiple issues at the same time when you’re in a project.
+You can't move tasks or test cases.
+
+Prerequisite:
+
+- You must have at least the Reporter role for the project.
+
+To move multiple issues at the same time:
+
+1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, select **Issues**.
+1. Select **Edit issues**. A sidebar on the right of your screen appears.
+1. Select the checkboxes next to each issue you want to move.
+1. From the right sidebar, select **Move selected**.
+1. From the dropdown list, select the destination project.
+1. Select **Move**.
+
+#### From the Rails console
+
You can move all open issues from one project to another.
Prerequisites:
@@ -586,7 +611,7 @@ To filter the list of issues:
1. Select or type the operator to use for filtering the attribute. The following operators are
available:
- `=`: Is
- - `!=`: Is not ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18059) in GitLab 12.7)
+ - `!=`: Is not one of
1. Enter the text to filter the attribute by.
You can filter some attributes by **None** or **Any**.
1. Repeat this process to filter by multiple attributes. Multiple attributes are joined by a logical
@@ -595,6 +620,21 @@ To filter the list of issues:
GitLab displays the results on-screen, but you can also
[retrieve them as an RSS feed](../../search/index.md#retrieve-search-results-as-feed).
+### Filter with the OR operator
+
+> OR filtering for assignees was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23532) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `or_issuable_queries`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available.
+To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `or_issuable_queries`.
+The feature is not ready for production use.
+
+When this feature is enabled, you can use the OR operator (**is one of: `||`**)
+when you [filter the list of issues](#filter-the-list-of-issues).
+
+`is one of` represents an inclusive OR. For example, if you filter by `Assignee is one of Sidney Jones` and
+`Assignee is one of Zhang Wei`, GitLab shows issues where either Sidney, Zhang, or both of them are assignees.
+
### Filter issues by ID
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/39908) in GitLab 12.1.
@@ -648,7 +688,7 @@ you can see the change without having to refresh the page.
The following sections are updated in real time:
- [Assignee](#assignee)
-- Labels, [if enabled](../labels.md#real-time-changes-to-labels)
+- [Labels](../labels.md#assign-and-unassign-labels)
## Assignee
diff --git a/doc/user/project/issues/related_issues.md b/doc/user/project/issues/related_issues.md
index 53ad7196920..1a8f19b80ba 100644
--- a/doc/user/project/issues/related_issues.md
+++ b/doc/user/project/issues/related_issues.md
@@ -1,6 +1,6 @@
---
stage: Plan
-group: Project Management
+group: Product Planning
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md
index 826e0b21ea7..bb72ab0052d 100644
--- a/doc/user/project/labels.md
+++ b/doc/user/project/labels.md
@@ -28,10 +28,21 @@ You can use two types of labels in GitLab:
## Assign and unassign labels
-> Unassigning labels with the **X** button [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216881) in GitLab 13.5.
+> - Unassigning labels with the **X** button [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216881) in GitLab 13.5.
+> - Real-time updates in the sidebar [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241538) in GitLab 14.10 with a [feature flag](../../administration/feature_flags.md) named `realtime_labels`, disabled by default.
+> - Real-time updates in the sidebar [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/357370#note_991987201) in GitLab 15.1.
+> - Real-time updates in the sidebar [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/357370) in GitLab 15.5.
+> - Real-time updates in the sidebar [generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103199) in GitLab 15.6. Feature flag `realtime_labels` removed.
You can assign labels to any issue, merge request, or epic.
+Changed labels are immediately visible to other users, without refreshing the page, on the following:
+
+- Epics
+- Incidents
+- Issues
+- Merge requests
+
To assign or unassign a label:
1. In the **Labels** section of the sidebar, select **Edit**.
@@ -410,7 +421,7 @@ To subscribe to a label:
1. To the right of any label, select **Subscribe**.
1. Optional. If you are subscribing to a group label from a project, select either:
- **Subscribe at project level** to be notified about events in this project.
- - **Subscribe at group level**: to be notified about events in the whole group.
+ - **Subscribe at group level** to be notified about events in the whole group.
## Set label priority
@@ -444,23 +455,6 @@ The labels higher in the list get higher priority.
To learn what happens when you sort by priority or label priority, see
[Sorting and ordering issue lists](issues/sorting_issue_lists.md).
-## Real-time changes to labels
-
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241538) in GitLab 14.10 with a [feature flag](../../administration/feature_flags.md) named `realtime_labels`, disabled by default.
-> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/357370#note_991987201) in GitLab 15.1.
-> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/357370) in GitLab 15.5.
-
-FLAG:
-On self-managed GitLab, to prevent updating labels in real-time, you can ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `realtime_labels`.
-On GitLab.com, this feature is available.
-
-Changed labels are immediately visible to other users, without refreshing the page, on the following:
-
-- Epics
-- Incidents
-- Issues
-- Merge requests
-
## Troubleshooting
### Some label titles end with `_duplicate<number>`
diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md
index a8f1b634127..e8ec954df8f 100644
--- a/doc/user/project/members/index.md
+++ b/doc/user/project/members/index.md
@@ -45,26 +45,14 @@ flowchart RL
> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 13.11 from a form to a modal window [with a flag](../../feature_flags.md). Disabled by default.
> - Modal window [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 14.8.
-> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) in GitLab 14.9.
- [Feature flag `invite_members_group_modal`](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) removed.
+> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) in GitLab 14.9. [Feature flag `invite_members_group_modal`](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) removed.
Add users to a project so they become members and have permission
to perform actions.
-The maximum role (access level) you set depends on if you have the Owner or Maintainer role for the group. For example, the maximum
-role that can be set is:
-
-- Owner (`50`), if you have the Owner role for the project.
-- Maintainer (`40`), if you have the Maintainer role on the project.
-
-In GitLab 14.8 and earlier, direct members of a project have a maximum role of Maintainer.
-The Owner [role](../../permissions.md#project-members-permissions) can only be added at the group level.
-
Prerequisite:
-- You must have the Maintainer or Owner role:
- - To remove direct members with the Maintainer role and below, you must have the Maintainer role.
- - To remove members with the Owner role, you must have the Owner role.
+- You must have the Owner or Maintainer role.
To add a user to a project:
@@ -73,7 +61,7 @@ To add a user to a project:
1. Select **Invite members**.
1. Enter an email address and select a [role](../../permissions.md).
1. Optional. Select an **Access expiration date**.
- On that date, the user can no longer access the project.
+ From that date onwards, the user can no longer access the project.
1. Select **Invite**.
If the user has a GitLab account, they are added to the members list.
@@ -86,12 +74,22 @@ deleted after 90 days.
If the user does not have a GitLab account, they are prompted to create an account
using the email address the invitation was sent to.
+### Which roles you can assign
+
+The maximum role you can assign depends on whether you have the Owner or Maintainer
+role for the group. For example, the maximum role you can set is:
+
+- Owner (`50`), if you have the Owner role for the project.
+- Maintainer (`40`), if you have the Maintainer role on the project.
+
+In GitLab 14.8 and earlier, direct members of a project have a maximum role of Maintainer.
+The Owner [role](../../permissions.md#project-members-permissions) can be added for the group only.
+
## Add groups to a project
> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 13.11 from a form to a modal window [with a flag](../../feature_flags.md). Disabled by default.
> - Modal window [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 14.8.
-> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) in GitLab 14.9.
- [Feature flag `invite_members_group_modal`](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) removed.
+> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) in GitLab 14.9. [Feature flag `invite_members_group_modal`](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) removed.
When you add a group to a project, each user in the group gets access to the project.
Each user's access is based on:
@@ -99,19 +97,20 @@ Each user's access is based on:
- The role they're assigned in the group.
- The maximum role you choose when you invite the group.
-Prerequisite:
+Prerequisites:
- You must have the Maintainer or Owner role.
- Sharing the project with other groups must not be [prevented](../../group/access_and_permissions.md#prevent-a-project-from-being-shared-with-groups).
-To add groups to a project:
+To add a group to a project:
1. On the top bar, select **Main menu > Projects** and find your project.
1. On the left sidebar, select **Project information > Members**.
1. Select **Invite a group**.
1. Select a group.
1. Select the highest [role](../../permissions.md) for users in the group.
-1. Optional. Select an **Access expiration date**. On that date, the group can no longer access the project.
+1. Optional. Select an **Access expiration date**.
+ From that date onwards, the group can no longer access the project.
1. Select **Invite**.
The members of the group are not displayed on the **Members** tab.
@@ -169,7 +168,9 @@ group itself.
Prerequisites:
-- You must have the Maintainer or Owner role.
+- To remove direct members with the:
+ - Maintainer, Developer, Reporter, or Guest role, you must have the Maintainer role.
+ - Owner role, you must have the Owner role.
- Optional. Unassign the member from all issues and merge requests that
are assigned to them.
@@ -187,6 +188,21 @@ To remove a member from a project:
[from being forked outside their group](../../group/access_and_permissions.md#prevent-project-forking-outside-group).
1. Select **Remove member**.
+## Ensure removed users cannot invite themselves back
+
+Malicious users with the Maintainer or Owner role could exploit a race condition that allows
+them to invite themselves back to a group or project that a GitLab administrator has removed them from.
+
+To avoid this problem, GitLab administrators can:
+
+- Remove the malicious user session from the [GitLab Rails console](../../../administration/operations/rails_console.md).
+- Impersonate the malicious user to:
+ - Remove the user from the project.
+ - Log the user out of GitLab.
+- Block the malicious user account.
+- Remove the malicious user account.
+- Change the password for the malicious user account.
+
## Filter and sort members
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21727) in GitLab 12.6.
diff --git a/doc/user/project/merge_requests/approvals/index.md b/doc/user/project/merge_requests/approvals/index.md
index ea03427161e..eb460225858 100644
--- a/doc/user/project/merge_requests/approvals/index.md
+++ b/doc/user/project/merge_requests/approvals/index.md
@@ -68,7 +68,7 @@ if a user approves a merge request and is shown in the reviewer list, a green ch
After a merge request receives the [number and type of approvals](rules.md) you configure, it can merge
unless it's blocked for another reason. Merge requests can be blocked by other problems,
-such as merge conflicts, [pending discussions](../../../discussions/index.md#prevent-merge-unless-all-threads-are-resolved),
+such as merge conflicts, [unresolved threads](../../../discussions/index.md#prevent-merge-unless-all-threads-are-resolved),
or a [failed CI/CD pipeline](../merge_when_pipeline_succeeds.md).
To prevent merge request authors from approving their own merge requests,
diff --git a/doc/user/project/merge_requests/authorization_for_merge_requests.md b/doc/user/project/merge_requests/authorization_for_merge_requests.md
index ba28432e90a..52944ee3143 100644
--- a/doc/user/project/merge_requests/authorization_for_merge_requests.md
+++ b/doc/user/project/merge_requests/authorization_for_merge_requests.md
@@ -68,6 +68,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/merge_requests/changes.md b/doc/user/project/merge_requests/changes.md
index 6703cbf8b03..6e8b0cb1a75 100644
--- a/doc/user/project/merge_requests/changes.md
+++ b/doc/user/project/merge_requests/changes.md
@@ -149,7 +149,7 @@ The feature is not ready for production use.
To avoid displaying the changes that are already on target branch in the diff,
we compare the merge request's source branch with HEAD of the target branch.
-When there are conflicts between the source and target branch, we show the
-conflicts on the merge request diff:
+When there are conflicts between the source and target branch, we show an alert
+per conflicted file on the merge request diff:
-![Example of a conflict shown in a merge request diff](img/conflict_ui_v14_0.png)
+![Example of a conflict alert shown in a merge request diff](img/conflict_ui_v15_6.png)
diff --git a/doc/user/project/merge_requests/conflicts.md b/doc/user/project/merge_requests/conflicts.md
index 902095bcbce..24f22924a08 100644
--- a/doc/user/project/merge_requests/conflicts.md
+++ b/doc/user/project/merge_requests/conflicts.md
@@ -172,6 +172,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/merge_requests/drafts.md b/doc/user/project/merge_requests/drafts.md
index 2beb7406518..0bc9b337e3b 100644
--- a/doc/user/project/merge_requests/drafts.md
+++ b/doc/user/project/merge_requests/drafts.md
@@ -88,6 +88,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/merge_requests/img/conflict_ui_v14_0.png b/doc/user/project/merge_requests/img/conflict_ui_v14_0.png
deleted file mode 100644
index 92c532351cb..00000000000
--- a/doc/user/project/merge_requests/img/conflict_ui_v14_0.png
+++ /dev/null
Binary files differ
diff --git a/doc/user/project/merge_requests/img/conflict_ui_v15_6.png b/doc/user/project/merge_requests/img/conflict_ui_v15_6.png
new file mode 100644
index 00000000000..d5d5ad14edb
--- /dev/null
+++ b/doc/user/project/merge_requests/img/conflict_ui_v15_6.png
Binary files differ
diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md
index e73c339e000..8309b04758a 100644
--- a/doc/user/project/merge_requests/index.md
+++ b/doc/user/project/merge_requests/index.md
@@ -250,6 +250,28 @@ This feature works only when a merge request is merged. Selecting **Remove sourc
after merging does not retarget open merge requests. This improvement is
[proposed as a follow-up](https://gitlab.com/gitlab-org/gitlab/-/issues/321559).
+## Move sidebar actions
+
+<!-- When the `moved_mr_sidebar` feature flag is removed, delete this topic and update the steps for these actions
+like in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87727/diffs?diff_id=522279685#5d9afba799c4af9920dab533571d7abb8b9e9163 -->
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85584) in GitLab 14.10 [with a flag](../../../administration/feature_flags.md) named `moved_mr_sidebar`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `moved_mr_sidebar`.
+On GitLab.com, this feature is not available.
+
+When this feature flag is enabled, you can find the following actions in
+**Merge request actions** (**{ellipsis_v}**) on the top right:
+
+- The [notifications](../../profile/notifications.md#edit-notification-settings-for-issues-merge-requests-and-epics) toggle
+- Mark merge request as ready or [draft](../merge_requests/drafts.md)
+- Close merge request
+- [Lock discussion](../../discussions/index.md#prevent-comments-by-locking-the-discussion)
+- Copy reference
+
+When this feature flag is disabled, these actions are in the right sidebar.
+
## Merge request workflows
For a software developer working in a team:
@@ -289,3 +311,76 @@ For a web developer writing a webpage for your company's website:
- [Commits](commits.md)
- [CI/CD pipelines](../../../ci/index.md)
- [Push options](../push_options.md) for merge requests
+
+## Troubleshooting
+
+### Rebase a merge request from the Rails console **(FREE SELF)**
+
+In addition to the `/rebase` [quick action](../quick_actions.md#issues-merge-requests-and-epics),
+users with access to the [Rails console](../../../administration/operations/rails_console.md)
+can rebase a merge request from the Rails console. Replace `<username>`,
+`<namespace/project>`, and `<iid>` with appropriate values:
+
+WARNING:
+Any command that changes data directly could be damaging if not run correctly,
+or under the right conditions. We highly recommend running them in a test environment
+with a backup of the instance ready to be restored, just in case.
+
+```ruby
+u = User.find_by_username('<username>')
+p = Project.find_by_full_path('<namespace/project>')
+m = p.merge_requests.find_by(iid: <iid>)
+MergeRequests::RebaseService.new(project: m.target_project, current_user: u).execute(m)
+```
+
+### Fix incorrect merge request status **(FREE SELF)**
+
+If a merge request remains **Open** after its changes are merged,
+users with access to the [Rails console](../../../administration/operations/rails_console.md)
+can correct the merge request's status. Replace `<username>`, `<namespace/project>`,
+and `<iid>` with appropriate values:
+
+WARNING:
+Any command that changes data directly could be damaging if not run correctly,
+or under the right conditions. We highly recommend running them in a test environment
+with a backup of the instance ready to be restored, just in case.
+
+```ruby
+u = User.find_by_username('<username>')
+p = Project.find_by_full_path('<namespace/project>')
+m = p.merge_requests.find_by(iid: <iid>)
+MergeRequests::PostMergeService.new(project: p, current_user: u).execute(m)
+```
+
+Running this command against a merge request with unmerged changes causes the
+merge request to display an incorrect message: `merged into <branch-name>`.
+
+### Close a merge request from the Rails console **(FREE SELF)**
+
+If closing a merge request doesn't work through the UI or API, you may want to attempt to close it in a [Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session):
+
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
+```ruby
+u = User.find_by_username('<username>')
+p = Project.find_by_full_path('<namespace/project>')
+m = p.merge_requests.find_by(iid: <iid>)
+MergeRequests::CloseService.new(project: p, current_user: u).execute(m)
+```
+
+### Delete a merge request from the Rails console **(FREE SELF)**
+
+If deleting a merge request doesn't work through the UI or API, you may want to attempt to delete it in a [Rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session):
+
+WARNING:
+Any command that changes data directly could be damaging if not run correctly,
+or under the right conditions. We highly recommend running them in a test environment
+with a backup of the instance ready to be restored, just in case.
+
+```ruby
+u = User.find_by_username('<username>')
+p = Project.find_by_full_path('<namespace/project>')
+m = p.merge_requests.find_by(iid: <iid>)
+Issuable::DestroyService.new(project: m.project, current_user: u).execute(m)
+```
diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md
index 3b07f75a3a7..76f351f1346 100644
--- a/doc/user/project/merge_requests/revert_changes.md
+++ b/doc/user/project/merge_requests/revert_changes.md
@@ -95,6 +95,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/merge_requests/reviews/data_usage.md b/doc/user/project/merge_requests/reviews/data_usage.md
index 56fb84c8085..3a3af7a24bc 100644
--- a/doc/user/project/merge_requests/reviews/data_usage.md
+++ b/doc/user/project/merge_requests/reviews/data_usage.md
@@ -19,7 +19,7 @@ This data extraction job can take a few hours to complete (possibly up to a day)
### Generating suggestions
-Once Suggested Reviewers is enabled and the data extraction is complete, new merge requests or new commits to existing merge requests will automatically trigger a Suggested Reviewers ML model inference and generate up to 5 suggested reviewers. These suggestions are contextual to the changes in the merge request. Additional commits to merge requests may change the reviewer suggestions which will automatically update in the reviewer dropdown.
+Once Suggested Reviewers is enabled and the data extraction is complete, new merge requests or new commits to existing merge requests will automatically trigger a Suggested Reviewers ML model inference and generate up to 5 suggested reviewers. These suggestions are contextual to the changes in the merge request. Additional commits to merge requests may change the reviewer suggestions which will automatically update in the reviewer dropdown list.
## Progressive enhancement
diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md
index ce1fc94395c..4c503211513 100644
--- a/doc/user/project/merge_requests/reviews/index.md
+++ b/doc/user/project/merge_requests/reviews/index.md
@@ -39,7 +39,7 @@ Project Maintainers or Owners can enable suggested reviewers by visiting the [pr
Enabling suggested reviewers will trigger GitLab to create an ML model for your project that will be used to generate reviewers. The larger your project, the longer this can take, but usually, the model will be ready to generate suggestions within a few hours.
-No action is required once the feature is enabled. Once the model is ready, recommendations will populate the Reviewer dropdown in the right-hand sidebar of a merge request with new commits.
+No action is required once the feature is enabled. Once the model is ready, recommendations will populate the Reviewer dropdown list in the right-hand sidebar of a merge request with new commits.
## Review a merge request
diff --git a/doc/user/project/merge_requests/reviews/suggestions.md b/doc/user/project/merge_requests/reviews/suggestions.md
index 2d3682c62d4..832f78d18a1 100644
--- a/doc/user/project/merge_requests/reviews/suggestions.md
+++ b/doc/user/project/merge_requests/reviews/suggestions.md
@@ -12,7 +12,7 @@ type: index, reference
As a reviewer, you're able to suggest code changes with a Markdown syntax in merge request
diff threads. Then, the merge request author (or other users with appropriate
-[permission](../../../permissions.md)) can apply these suggestions with a click.
+[permission](../../../permissions.md)) can apply these suggestions.
This action generates a commit in the merge request, authored by the user that suggested the changes.
1. Choose a line of code to be changed, add a new comment, then select
@@ -47,8 +47,11 @@ After the author applies a suggestion:
## Multi-line suggestions
+> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/232339) in GitLab 13.11: suggestions in multi-line comments also become multi-line.
+
Reviewers can also suggest changes to multiple lines with a single suggestion
-within merge request diff threads by adjusting the range offsets. The
+within merge request diff threads by selecting and dragging selection to all
+relevant line numbers or by adjusting the range offsets. The
offsets are relative to the position of the diff thread, and specify the
range to be replaced by the suggestion when it is applied.
diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md
index e83e17072d6..9f87f1e2e0d 100644
--- a/doc/user/project/merge_requests/squash_and_merge.md
+++ b/doc/user/project/merge_requests/squash_and_merge.md
@@ -83,6 +83,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/merge_requests/versions.md b/doc/user/project/merge_requests/versions.md
index 6f29272f003..a864a9849b0 100644
--- a/doc/user/project/merge_requests/versions.md
+++ b/doc/user/project/merge_requests/versions.md
@@ -16,12 +16,12 @@ request diffs.
## Selecting a version
By default, the latest version of changes is shown. However, you
-can select an older one from version dropdown.
+can select an older one from version dropdown list.
-![Merge request versions dropdown](img/versions_dropdown.png)
+![Merge request versions dropdown list](img/versions_dropdown.png)
Merge request versions are based on push not on commit. So, if you pushed 5
-commits in a single push, it displays as a single option in the dropdown. If you
+commits in a single push, it displays as a single option in the dropdown list. If you
pushed 5 times, that counts for 5 options.
You can also compare the merge request version with an older one to see what has
@@ -76,6 +76,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/milestones/burndown_and_burnup_charts.md b/doc/user/project/milestones/burndown_and_burnup_charts.md
index 01eeee9d3b9..81b334c0a02 100644
--- a/doc/user/project/milestones/burndown_and_burnup_charts.md
+++ b/doc/user/project/milestones/burndown_and_burnup_charts.md
@@ -145,6 +145,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md
index 76c5e32eb2b..bbe4aadc50d 100644
--- a/doc/user/project/milestones/index.md
+++ b/doc/user/project/milestones/index.md
@@ -238,6 +238,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/ml/experiment_tracking/img/candidates.png b/doc/user/project/ml/experiment_tracking/img/candidates.png
new file mode 100644
index 00000000000..df70a01a2bd
--- /dev/null
+++ b/doc/user/project/ml/experiment_tracking/img/candidates.png
Binary files differ
diff --git a/doc/user/project/ml/experiment_tracking/img/experiments.png b/doc/user/project/ml/experiment_tracking/img/experiments.png
new file mode 100644
index 00000000000..a6472406b90
--- /dev/null
+++ b/doc/user/project/ml/experiment_tracking/img/experiments.png
Binary files differ
diff --git a/doc/user/project/ml/experiment_tracking/index.md b/doc/user/project/ml/experiment_tracking/index.md
new file mode 100644
index 00000000000..e274bd7f38e
--- /dev/null
+++ b/doc/user/project/ml/experiment_tracking/index.md
@@ -0,0 +1,76 @@
+---
+stage: Create
+group: Incubation
+info: Machine Learning Experiment Tracking is a GitLab Incubation Engineering program. No technical writer assigned to this group.
+---
+
+# Machine Learning Experiment Tracking **(FREE)**
+
+DISCLAIMER:
+Machine Learning Experiment Tracking is an experimental feature being developed by the Incubation Engineering Department,
+and will receive significant changes over time. This feature is being release with the aim of getting user feedback, but
+is not stable and can lead to performance degradation. See below on how to disable this feature.
+
+When creating machine learning models, data scientists often experiment with different parameters, configurations, feature
+engineering, and so on, to improve the performance of the model. Keeping track of all this metadata and the associated
+artifacts so that the data scientist can later replicate the experiment is not trivial. Machine learning experiment
+tracking enables them to log parameters, metrics, and artifacts directly into GitLab, giving easy access later on.
+
+![List of Experiments](img/experiments.png)
+
+![Experiment Candidates](img/candidates.png)
+
+## What is an experiment?
+
+An experiment is a collection of comparable model candidates. Experiments can be long lived (for example, when they represent
+a use case), or short lived (results from hyperparameter tuning triggered by a merge request), but usually hold model candidates
+that have a similar set of parameters and metrics.
+
+## Model candidate
+
+A model candidate is a variation of the training of a machine learning model, that can be eventually promoted to a version
+of the model. The goal of a data scientist is to find the model candidate whose parameter values lead to the best model
+performance, as indicated by the given metrics.
+
+Example parameters:
+
+- Algorithm (linear regression, decision tree, and so on).
+- Hyperparameters for the algorithm (learning rate, tree depth, number of epochs).
+- Features included.
+
+## Usage
+
+### User access management
+
+An experiment is always associated to a project. Only users with access to the project an experiment is associated with
+can view that experiment data.
+
+### Tracking new experiments and trials
+
+Experiment and trials can only be tracked through the [MLFlow](https://www.mlflow.org/docs/latest/tracking.html) client
+integration. More information on how to use GitLab as a backend for MLFlow Client can be found [at the documentation page](../../integrations/mlflow_client.md).
+
+### Exploring model candidates
+
+To list the current active experiments, navigate to `https/-/ml/experiments`. To display all trials
+that have been logged, along with their metrics and parameters, selecting an experiment.
+
+### Logging artifacts
+
+Trial artifacts are saved as [generic packages](../../../packages/generic_packages/index.md), and follow all their
+conventions. After an artifact is logged for a candidate, all artifacts logged for the candidate are listed in the
+package registry. The package name for a candidate is `ml_candidate_<candidate_id>`, with version `-`.
+
+### Limitations and future
+
+- Searching experiments, searching trials, visual comparison of trials, and creating, deleting and updating experiments and trials through GitLab UI is under development.
+- No support for experiment and trial metadata that do not classify as parameters or metrics.
+
+## Disabling or enabling the Feature
+
+On self-managed GitLab, ML Experiment Tracking is disabled by default. To enable the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `ml_experiment_tracking`.
+On GitLab.com, this feature is currently on private testing.
+
+## Feedback, roadmap and reports
+
+For updates on the development, feedback and bug reports, refer to the [development epic](https://gitlab.com/groups/gitlab-org/-/epics/8560).
diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md
index 5ee1fa103a4..6378d962ffe 100644
--- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md
+++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md
@@ -250,7 +250,7 @@ can use the following setup:
on the top nav.
- Select **Create Page Rule**.
- Enter the domain `www.domain.com` and select **+ Add a Setting**.
- - From the dropdown menu, choose **Forwarding URL**, then select the
+ - From the dropdown list, choose **Forwarding URL**, then select the
status code **301 - Permanent Redirect**.
- Enter the destination URL `https://domain.com`.
diff --git a/doc/user/project/pages/getting_started/pages_ci_cd_template.md b/doc/user/project/pages/getting_started/pages_ci_cd_template.md
index 01583dbe45d..caf98e8a8a4 100644
--- a/doc/user/project/pages/getting_started/pages_ci_cd_template.md
+++ b/doc/user/project/pages/getting_started/pages_ci_cd_template.md
@@ -18,9 +18,9 @@ these steps, you may have to do additional configuration for the Pages site to g
1. On the top bar, select **Main menu > Projects** and find your project.
1. On the left sidebar, select the project's name.
-1. From the **Add** (**{plus}**) dropdown, select **New file**.
-1. From the **Select a template type** dropdown, select `.gitlab-ci.yml`.
-1. From the **Apply a template** dropdown, in the **Pages** section, select the name of your SSG.
+1. From the **Add** (**{plus}**) dropdown list, select **New file**.
+1. From the **Select a template type** dropdown list, select `.gitlab-ci.yml`.
+1. From the **Apply a template** dropdown list, in the **Pages** section, select the name of your SSG.
1. In the **Commit message** box, type the commit message.
1. Select **Commit changes**.
@@ -30,5 +30,9 @@ You can watch the pipeline run by navigating to **CI/CD > Pipelines**.
When the pipeline is finished, go to **Settings > Pages** to find the link to
your Pages website.
+To view the HTML and other assets that were created for the site,
+go to the **Pipelines** tab, view the job, and on the right side,
+select **Download artifacts**.
+
For every change pushed to your repository, GitLab CI/CD runs a new pipeline
that immediately publishes your changes to the Pages site.
diff --git a/doc/user/project/pages/getting_started/pages_forked_sample_project.md b/doc/user/project/pages/getting_started/pages_forked_sample_project.md
index 33cf677e1be..69c60cab4b3 100644
--- a/doc/user/project/pages/getting_started/pages_forked_sample_project.md
+++ b/doc/user/project/pages/getting_started/pages_forked_sample_project.md
@@ -29,6 +29,10 @@ When the pipeline is finished, go to **Settings > Pages** to find the link to yo
For every change pushed to your repository, GitLab CI/CD runs a new pipeline
that immediately publishes your changes to the Pages site.
+To view the HTMl and other assets that were created for the site,
+go to the **Pipelines** tab, view the job, and on the right side,
+select **Download artifacts**.
+
You can take some **optional** further steps:
- Remove the fork relationship. If you want to contribute to the project you forked from,
diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md
index e34544c96f8..c0a1e8f16e0 100644
--- a/doc/user/project/pages/getting_started/pages_from_scratch.md
+++ b/doc/user/project/pages/getting_started/pages_from_scratch.md
@@ -420,6 +420,10 @@ Now GitLab CI/CD not only builds the website, but also:
- **Caches** dependencies installed with Bundler.
- **Continuously deploys** every push to the `main` branch.
+To view the HTMl and other assets that were created for the site,
+go to the **Pipelines** tab, view the job, and on the right side,
+select **Download artifacts**.
+
## Related topics
For more information, see the following blog posts.
@@ -427,7 +431,7 @@ For more information, see the following blog posts.
- Use GitLab CI/CD `environments` to
[deploy your web app to staging and production](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/).
- Learn how to run jobs
- [sequentially, in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2016/07/29/the-basics-of-gitlab-ci/).
+ [sequentially, in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2020/12/10/basics-of-gitlab-ci-updated/).
- Learn [how to pull specific directories from different projects](https://about.gitlab.com/blog/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/)
to deploy this website, <https://docs.gitlab.com>.
- Learn [how to use GitLab Pages to produce a code coverage report](https://about.gitlab.com/blog/2016/11/03/publish-code-coverage-report-with-gitlab-pages/).
diff --git a/doc/user/project/pages/getting_started/pages_new_project_template.md b/doc/user/project/pages/getting_started/pages_new_project_template.md
index d7e304dc6f8..e4890954d13 100644
--- a/doc/user/project/pages/getting_started/pages_new_project_template.md
+++ b/doc/user/project/pages/getting_started/pages_new_project_template.md
@@ -28,3 +28,7 @@ your Pages website.
For every change pushed to your repository, GitLab CI/CD runs a new pipeline
that immediately publishes your changes to the Pages site.
+
+To view the HTMl and other assets that were created for the site,
+go to the **Pipelines** tab, view the job, and on the right side,
+select **Download artifacts**.
diff --git a/doc/user/project/pages/getting_started/pages_ui.md b/doc/user/project/pages/getting_started/pages_ui.md
index 064255c094f..ba97fcb8749 100644
--- a/doc/user/project/pages/getting_started/pages_ui.md
+++ b/doc/user/project/pages/getting_started/pages_ui.md
@@ -41,6 +41,10 @@ To build your YAML file from the GitLab UI:
1. Commit your `.gitlab-ci.yml` to your repository. This commit triggers your first
GitLab Pages deployment.
+To view the HTMl and other assets that were created for the site,
+go to **CI/CD > Pipelines**, view the job, and on the right side,
+select **Download artifacts**.
+
## Troubleshooting
### If you can't see the "Get Started with Pages" interface
diff --git a/doc/user/project/pages/img/remove_pages_v15_3.png b/doc/user/project/pages/img/remove_pages_v15_3.png
deleted file mode 100644
index f740daf5c0b..00000000000
--- a/doc/user/project/pages/img/remove_pages_v15_3.png
+++ /dev/null
Binary files differ
diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md
index 47c0885c26b..9305b92bf7d 100644
--- a/doc/user/project/pages/index.md
+++ b/doc/user/project/pages/index.md
@@ -107,7 +107,7 @@ These GitLab Pages website examples can teach you advanced techniques to use
and adapt for your own needs:
- [Posting to your GitLab Pages blog from iOS](https://about.gitlab.com/blog/2016/08/19/posting-to-your-gitlab-pages-blog-from-ios/).
-- [GitLab CI: Run jobs sequentially, in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2016/07/29/the-basics-of-gitlab-ci/).
+- [GitLab CI: Run jobs sequentially, in parallel, or build a custom pipeline](https://about.gitlab.com/blog/2020/12/10/basics-of-gitlab-ci-updated/).
- [GitLab CI: Deployment & environments](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/).
- [Building a new GitLab docs site with Nanoc, GitLab CI, and GitLab Pages](https://about.gitlab.com/blog/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/).
- [Publish code coverage reports with GitLab Pages](https://about.gitlab.com/blog/2016/11/03/publish-code-coverage-report-with-gitlab-pages/).
diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md
index ed154c0dfca..a9b8960d629 100644
--- a/doc/user/project/pages/introduction.md
+++ b/doc/user/project/pages/introduction.md
@@ -40,20 +40,19 @@ If you are using [GitLab Pages on GitLab.com](#gitlab-pages-on-gitlabcom) to hos
Visit the [GitLab Pages group](https://gitlab.com/groups/pages) for a complete list of example projects. Contributions are very welcome.
-## Custom error codes Pages
+## Custom error codes pages
-You can provide your own 403 and 404 error pages by creating the `403.html` and
-`404.html` files respectively in the root directory of the `public/` directory
-that are included in the artifacts. Usually this is the root directory of
-your project, but that may differ depending on your static generator
-configuration.
+You can provide your own `403` and `404` error pages by creating `403.html` and
+`404.html` files in the root of the `public/` directory. Usually this is
+the root directory of your project, but that may differ
+depending on your static generator configuration.
If the case of `404.html`, there are different scenarios. For example:
- If you use project Pages (served under `/projectname/`) and try to access
`/projectname/non/existing_file`, GitLab Pages tries to serve first
`/projectname/404.html`, and then `/404.html`.
-- If you use user/group Pages (served under `/`) and try to access
+- If you use user or group Pages (served under `/`) and try to access
`/non/existing_file` GitLab Pages tries to serve `/404.html`.
- If you use a custom domain and try to access `/non/existing_file`, GitLab
Pages tries to serve only `/404.html`.
@@ -63,34 +62,34 @@ If the case of `404.html`, there are different scenarios. For example:
You can configure redirects for your site using a `_redirects` file. To learn more, read
the [redirects documentation](redirects.md).
-## GitLab Pages Access Control
+## Remove your pages
-To restrict access to your website, enable [GitLab Pages Access Control](pages_access_control.md).
+To remove your pages:
-## Unpublishing your Pages
-
-If you ever feel the need to purge your Pages content, you can do so by going
-to your project's settings through the gear icon in the top right, and then
-navigating to **Pages**. Select the **Remove pages** button to delete your Pages
-website.
-
-![Remove pages](img/remove_pages_v15_3.png)
+1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, select **Settings > Pages**.
+1. Select **Remove pages**.
## Subdomains of subdomains
When using Pages under the top-level domain of a GitLab instance (`*.example.io`), you can't use HTTPS with subdomains
of subdomains. If your namespace or group name contains a dot (for example, `foo.bar`) the domain
-`https://foo.bar.example.io` does _not_ work.
+`https://foo.bar.example.io` does **not** work.
This limitation is because of the [HTTP Over TLS protocol](https://www.rfc-editor.org/rfc/rfc2818#section-3.1). HTTP pages
work as long as you don't redirect HTTP to HTTPS.
-## GitLab Pages and subgroups
+## GitLab Pages in projects and groups
+
+You must host your GitLab Pages website in a project. This project can be
+[private, internal, or public](../../../user/public_access.md) and belong
+to a [group](../../group/index.md) or [subgroup](../../group/subgroups/index.md).
+
+For [group websites](../../project/pages/getting_started_part_one.md#user-and-group-website-examples),
+the group must be at the top level and not a subgroup.
-You must host your GitLab Pages website in a project. This project can belong to a [group](../../group/index.md) or
-[subgroup](../../group/subgroups/index.md). For
-[group websites](../../project/pages/getting_started_part_one.md#gitlab-pages-default-domain-names), the group must be
-at the top level and not a subgroup.
+For [project websites](../../project/pages/getting_started_part_one.md#project-website-examples),
+you can create your project first and access it under `http(s)://namespace.example.io/projectname`.
## Specific configuration options for Pages
@@ -129,7 +128,7 @@ pages:
See this document for a [step-by-step guide](getting_started/pages_from_scratch.md).
-### `.gitlab-ci.yml` for a repository where there's also actual code
+### `.gitlab-ci.yml` for a repository with code
Remember that GitLab Pages are by default branch/tag agnostic and their
deployment relies solely on what you specify in `.gitlab-ci.yml`. You can limit
@@ -257,26 +256,6 @@ instead. Here are some examples of what happens given the above Pages site:
Note that when `public/data/index.html` exists, it takes priority over the `public/data.html` file
for both the `/data` and `/data/` URL paths.
-## Frequently Asked Questions
-
-### Can you download your generated pages?
-
-Sure. All you need to do is download the artifacts archive from the job page.
-
-### Can you use GitLab Pages if your project is private?
-
-Yes. GitLab Pages doesn't care whether you set your project's visibility level
-to private, internal or public.
-
-### Can you create a personal or a group website?
-
-Yes. See the documentation about [GitLab Pages domain names, URLs, and base URLs](getting_started_part_one.md).
-
-### Do you need to create a user/group website before creating a project website?
-
-No, you don't. You can create your project first and access it under
-`http(s)://namespace.example.io/projectname`.
-
## Known issues
For a list of known issues, visit the GitLab [public issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues?label_name[]=Category%3APages).
diff --git a/doc/user/project/pages/pages_access_control.md b/doc/user/project/pages/pages_access_control.md
index be1ea236c87..b98772a6702 100644
--- a/doc/user/project/pages/pages_access_control.md
+++ b/doc/user/project/pages/pages_access_control.md
@@ -20,7 +20,7 @@ For a demonstration, see [Pages access controls](https://www.youtube.com/watch?v
1. Toggle the **Pages** button to enable the access control. If you don't see the toggle button,
that means it isn't enabled. Ask your administrator to [enable it](../../../administration/pages/index.md#access-control).
-1. The Pages access control dropdown allows you to set who can view pages hosted
+1. The Pages access control dropdown list allows you to set who can view pages hosted
with GitLab Pages, depending on your project's visibility:
- If your project is private:
diff --git a/doc/user/project/pages/public_folder.md b/doc/user/project/pages/public_folder.md
index b5d498402d5..a19e296b954 100644
--- a/doc/user/project/pages/public_folder.md
+++ b/doc/user/project/pages/public_folder.md
@@ -116,7 +116,7 @@ GitLab Pages supports only static sites.
```
1. Configure your Nuxt.js application for
- [Static Site Generation](https://nuxtjs.org/docs/features/deployment-targets#static-hosting).
+ [Static Site Generation](https://nuxtjs.org/docs/features/deployment-targets/#static-hosting).
### Vite
diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md
index f9dcf838c33..ab97ff08123 100644
--- a/doc/user/project/protected_branches.md
+++ b/doc/user/project/protected_branches.md
@@ -260,6 +260,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md
index 22ce81409a3..152a55d24b7 100644
--- a/doc/user/project/protected_tags.md
+++ b/doc/user/project/protected_tags.md
@@ -114,6 +114,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md
index 3471123f8b5..7b7619cfeb5 100644
--- a/doc/user/project/quick_actions.md
+++ b/doc/user/project/quick_actions.md
@@ -52,7 +52,7 @@ threads. Some quick actions might not be available to all subscription tiers.
| Command | Issue | Merge request | Epic | Action |
|:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `/add_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add one or more [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). |
+| `/add_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add one or more active [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). |
| `/approve` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Approve the merge request. |
| `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users. |
| `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself. |
@@ -65,7 +65,7 @@ threads. Some quick actions might not be available to all subscription tiers.
| `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. |
| `/clone <path/to/project> [--with_notes]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clone the issue to given project, or the current one if no arguments are given ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9421) in GitLab 13.7). Copies as much data as possible as long as the target project contains equivalent labels, milestones, and so on. Does not copy comments or system notes unless `--with_notes` is provided as an argument. |
| `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. |
-| `/confidential` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Make confidential. |
+| `/confidential` | **{check-circle}** Yes | **{dotted-circle}** No | **{check-circle}** Yes | Mark issue or epic as confidential. Support for epics [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213741) in GitLab 15.6. |
| `/copy_metadata <!merge_request>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another merge request in the project. |
| `/copy_metadata <#issue>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Copy labels and milestone from another issue in the project. |
| `/create_merge_request <branch name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Create a new merge request starting from the current issue. |
@@ -74,13 +74,13 @@ threads. Some quick actions might not be available to all subscription tiers.
| `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. |
| `/duplicate <#issue>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Close this issue and mark as a duplicate of another issue. Also, mark both as related. |
| `/epic <epic>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. |
-| `/estimate <time>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set time estimate. For example, `/estimate 1mo 2w 3d 4h 5m`. Learn more about [time tracking](time_tracking.md). |
+| `/estimate <time>` or `/estimate_time <time>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set time estimate. For example, `/estimate 1mo 2w 3d 4h 5m`. Learn more about [time tracking](time_tracking.md). Alias `/estimate_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. |
| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). |
| `/invite_email email1 email2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add up to six email participants. This action is behind feature flag `issue_email_participants` and is not yet supported in issue templates. |
| `/iteration *iteration:"iteration name"` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set iteration. For example, to set the `Late in July` iteration: `/iteration *iteration:"Late in July"` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). |
| `/label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. |
| `/lock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Lock the discussions. |
-| `/link` | **{check-circle}** Yes | ****{dotted-circle}** No | **{dotted-circle}** No | Add a link and description to [linked resources](../../operations/incident_management/linked_resources.md) in an incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/374964) in GitLab 15.5). |
+| `/link` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add a link and description to [linked resources](../../operations/incident_management/linked_resources.md) in an incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/374964) in GitLab 15.5). |
| `/merge` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), or adding to a [Merge Train](../../ci/pipelines/merge_trains.md). |
| `/milestone %milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set milestone. |
| `/move <path/to/project>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Move this issue to another project. Be careful when moving an issue to a project with different access rules. Before moving the issue, make sure it does not contain sensitive data. |
@@ -99,7 +99,7 @@ threads. Some quick actions might not be available to all subscription tiers.
| `/remove_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove one or more [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). |
| `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove due date. |
| `/remove_epic` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove from epic. |
-| `/remove_estimate` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time estimate. |
+| `/remove_estimate` or `/remove_time_estimate` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time estimate. Alias `/remove_time_estimate` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. |
| `/remove_iteration` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove iteration ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). |
| `/remove_milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove milestone. |
| `/remove_parent_epic` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove parent epic from epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10556) in GitLab 12.1). |
@@ -108,7 +108,7 @@ threads. Some quick actions might not be available to all subscription tiers.
| `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. |
| `/severity <severity>` | **{check-circle}** Yes | **{check-circle}** No | **{check-circle}** No | Set the severity. Issue type must be `Incident`. Options for `<severity>` are `S1` ... `S4`, `critical`, `high`, `medium`, `low`, `unknown`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334045) in GitLab 14.2. |
| `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\_(ツ)_/¯`. |
-| `/spend <time> [<date>]` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Add or subtract spent time. Optionally, specify the date that time was spent on. For example, `/spend 1mo 2w 3d 4h 5m 2018-08-26` or `/spend -1h 30m`. Learn more about [time tracking](time_tracking.md). |
+| `/spend <time> [<date>]` or `/spend_time <time> [<date>]` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Add or subtract spent time. Optionally, specify the date that time was spent on. For example, `/spend 1mo 2w 3d 4h 5m 2018-08-26` or `/spend -1h 30m`. Learn more about [time tracking](time_tracking.md). Alias `/spend_time` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16501) in GitLab 15.6. |
| `/submit_review` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Submit a pending review ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8041) in GitLab 12.7). |
| `/subscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Subscribe to notifications. |
| `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. |
@@ -145,6 +145,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md
index fd01dd451c2..75a25678125 100644
--- a/doc/user/project/releases/index.md
+++ b/doc/user/project/releases/index.md
@@ -462,6 +462,20 @@ In the API:
This includes associated Git-tag-names, release description, author information of the releases.
However, other repository-related information, such as [source code](release_fields.md#source-code), [release evidence](#release-evidence) are redacted.
+### Publish releases without giving access to source code
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216485) in GitLab 15.6.
+
+Releases can be made accessible to non-project members while keeping repository-related information such as
+[source code](release_fields.md#source-code) and [release evidence](#release-evidence) private. This is useful for
+projects that use releases as a way to give access to new versions of software but do not want the source code to
+be public.
+
+To make releases available publicly, set the following [project settings](../settings/index.md#project-feature-settings):
+
+- Repository is enabled and set to **Only Project Members**
+- Releases is enabled and set to **Everyone With Access**
+
### Create, update, and delete a release and its assets
- Users with at least the Developer role
diff --git a/doc/user/project/releases/release_fields.md b/doc/user/project/releases/release_fields.md
index 5ae1c69847d..c06e746268f 100644
--- a/doc/user/project/releases/release_fields.md
+++ b/doc/user/project/releases/release_fields.md
@@ -93,7 +93,7 @@ By default, GitLab fetches the release using `released_at` time. The use of the
The assets associated with a release are accessible through a permanent URL.
GitLab always redirects this URL to the actual asset
location, so even if the assets move to a different location, you can continue
-to use the same URL. This is defined during [link creation](../../../api/releases/links.md#create-a-link) or [updating](../../../api/releases/links.md#update-a-link) using the `filepath` API attribute.
+to use the same URL. This is defined during [link creation](../../../api/releases/links.md#create-a-release-link) or [updating](../../../api/releases/links.md#update-a-release-link) using the `filepath` API attribute.
The format of the URL is:
@@ -133,7 +133,7 @@ The format of the URL is:
https://host/namespace/project/-/releases/permalink/latest/downloads/:filepath
```
-If you have an asset with [`filepath`](../../../api/releases/links.md#create-a-link) for the `v11.9.0-rc2` latest release in the `gitlab-org`
+If you have an asset with [`filepath`](../../../api/releases/links.md#create-a-release-link) for the `v11.9.0-rc2` latest release in the `gitlab-org`
namespace and `gitlab-runner` project on `gitlab.com`, for example:
```json
diff --git a/doc/user/project/remote_development/index.md b/doc/user/project/remote_development/index.md
index 879978f550a..62220dd2fde 100644
--- a/doc/user/project/remote_development/index.md
+++ b/doc/user/project/remote_development/index.md
@@ -6,6 +6,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Remote Development **(FREE)**
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The feature is not ready for production use.
+
+WARNING:
+This feature is in [Alpha](../../../policy/alpha-beta-support.md#alpha-features) and subject to change without notice.
+
DISCLAIMER:
This page contains information related to upcoming products, features, and functionality.
It is important to note that the information presented is for informational purposes only.
@@ -43,7 +51,7 @@ To point a domain to your remote machine, create an `A` record from `example.rem
#### Install Certbot
-[Certbot](https://certbot.eff.org/) is a free and open-source software tool that automatically uses Let's Encrypt certificates on manually administrated websites to enable HTTPS.
+[Certbot](https://certbot.eff.org/) is a free and open-source software tool that automatically uses Let's Encrypt certificates on manually administered websites to enable HTTPS.
To install Certbot, run the following command:
@@ -89,7 +97,7 @@ docker run -d \
-v "${CERTS_DIR}/fullchain.pem:/gitlab-rd-web-ide/certs/fullchain.pem" \
-v "${CERTS_DIR}/privkey.pem:/gitlab-rd-web-ide/certs/privkey.pem" \
-v "${PROJECTS_DIR}:/projects" \
- registry.gitlab.com/gitlab-com/create-stage/editor-poc/remote-development/gitlab-rd-web-ide-docker:0.1 \
+ registry.gitlab.com/gitlab-com/create-stage/editor-poc/remote-development/gitlab-rd-web-ide-docker:0.1-alpha \
--log-level warn --domain "${DOMAIN}" --ignore-version-mismatch
```
diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md
index 9755b5cb944..6cc7394e7b3 100644
--- a/doc/user/project/repository/branches/index.md
+++ b/doc/user/project/repository/branches/index.md
@@ -124,6 +124,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md
index 910b09449d8..a1f57f51f26 100644
--- a/doc/user/project/repository/gpg_signed_commits/index.md
+++ b/doc/user/project/repository/gpg_signed_commits/index.md
@@ -253,3 +253,19 @@ If you must unverify both future and past commits,
- [OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices)
- [Creating a new GPG key with subkeys](https://www.void.gr/kargig/blog/2013/12/02/creating-a-new-gpg-key-with-subkeys/) (advanced)
- [Review existing GPG keys in your instance](../../../admin_area/credentials_inventory.md#review-existing-gpg-keys)
+
+## Troubleshooting
+
+### Fix verification problems with signed commits
+
+Commits can be signed with [X.509 certificates](../x509_signed_commits/index.md)
+or a GPG key. The verification process for both methods can fail for multiple reasons:
+
+| Value | Description | Possible Fixes |
+|-----------------------------|-------------|----------------|
+| `UNVERIFIED` | The commit signature is not valid. | Sign the commit with a valid signature. |
+| `SAME_USER_DIFFERENT_EMAIL` | The GPG key used to sign the commit does not contain the committer email, but does contain a different valid email for the committer. | Amend the commit to use an email address that matches the GPG key, or update the GPG key [to include the email address](https://security.stackexchange.com/a/261468). |
+| `OTHER_USER` | The signature and GPG key are valid, but the key belongs to a different user than the committer. | Amend the commit to use the correct email address, or amend the commit to use a GPG key associated with your user. |
+| `UNVERIFIED_KEY` | The key associated with the GPG signature has no verified email address associated with the committer. | Add and verify the email to your GitLab profile, [update the GPG key to include the email address](https://security.stackexchange.com/a/261468), or amend the commit to use a different committer email address. |
+| `UNKNOWN_KEY` | The GPG key associated with the GPG signature for this commit is unknown to GitLab. | [Add the GPG key](#add-a-gpg-key-to-your-account) to your GitLab profile. |
+| `MULTIPLE_SIGNATURES` | Multiple GPG or X.509 signatures have been found for the commit. | Amend the commit to use only one GPG or X.509 signature. |
diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md
index bcdb626d0f2..83389c15eba 100644
--- a/doc/user/project/repository/index.md
+++ b/doc/user/project/repository/index.md
@@ -153,7 +153,7 @@ contents of the file's [markup language](https://en.wikipedia.org/wiki/Lightweig
| [reStructuredText](https://docutils.sourceforge.io/rst.html) | `rst` |
| [AsciiDoc](../../asciidoc.md) | `adoc`, `ad`, `asciidoc` |
| [Textile](https://textile-lang.com/) | `textile` |
-| [Rdoc](http://rdoc.sourceforge.net/doc/index.html) | `rdoc` |
+| [Rdoc](https://rdoc.sourceforge.net/doc/index.html) | `rdoc` |
| [Org mode](https://orgmode.org/) | `org` |
| [creole](http://www.wikicreole.org/) | `creole` |
| [MediaWiki](https://www.mediawiki.org/wiki/MediaWiki) | `wiki`, `mediawiki` |
diff --git a/doc/user/project/repository/jupyter_notebooks/index.md b/doc/user/project/repository/jupyter_notebooks/index.md
index 4f6cff576ea..28f0ffb6fbd 100644
--- a/doc/user/project/repository/jupyter_notebooks/index.md
+++ b/doc/user/project/repository/jupyter_notebooks/index.md
@@ -28,24 +28,20 @@ GitLab.
> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6589) in GitLab 14.5 as an [Alpha](../../../../policy/alpha-beta-support.md#alpha-features) release [with a flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. Enabled by default.
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75500) in GitLab 14.9. Feature flag `jupyter_clean_diffs` removed.
> - [Reintroduced toggle](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85079) in GitLab 15.0 [with a flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`. Enabled by default.
-
-FLAG:
-On self-managed GitLab, by default semantic diffs are available. To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`.
-On GitLab.com, this feature is available.
+> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95373) in GitLab 15.6. Feature flag `ipynb_semantic_diff` removed.
When commits include changes to Jupyter Notebook files, GitLab:
- Transforms the machine-readable `.ipynb` file into a human-readable Markdown file.
- Displays a cleaner version of the diff that includes syntax highlighting.
- Enables switching between raw and rendered diffs on the Commit and Compare pages. (Not available on merge request pages.)
+- Renders images on the diffs.
Code suggestions are not available on diffs and merge requests for `.ipynb` files.
-![Jupyter Notebook Clean Diff](img/jupyter_notebook_diff_v14_5.png)
+Cleaner notebook diffs are not generated when the notebook is too large.
-This feature is an [Alpha](../../../../policy/alpha-beta-support.md#alpha-features) release,
-and might lead to performance degradation. On self-managed GitLab, if unexpected issues
-arise, disable the feature.
+![Jupyter Notebook Clean Diff](img/jupyter_notebook_diff_v14_5.png)
## Jupyter Git integration
diff --git a/doc/user/project/repository/mirror/index.md b/doc/user/project/repository/mirror/index.md
index fa6d8d82559..2b578977bd2 100644
--- a/doc/user/project/repository/mirror/index.md
+++ b/doc/user/project/repository/mirror/index.md
@@ -325,6 +325,9 @@ Use case: If you have multiple users using their own GitHub credentials to set u
repository mirroring, mirroring breaks when people leave the company. Use this
script to migrate disparate mirroring users and tokens into a single service account:
+WARNING:
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
+
```ruby
svc_user = User.find_by(username: 'ourServiceUser')
token = 'githubAccessToken'
diff --git a/doc/user/project/repository/push_rules.md b/doc/user/project/repository/push_rules.md
index 917fca03967..7ae085812ae 100644
--- a/doc/user/project/repository/push_rules.md
+++ b/doc/user/project/repository/push_rules.md
@@ -267,7 +267,7 @@ to use them as normal characters in a match condition.
## Related topics
-- [Server hooks](../../../administration/server_hooks.md), to create complex custom push rules
+- [Git server hooks](../../../administration/server_hooks.md) (previously called server hooks), to create complex custom push rules
- [Signing commits with GPG](gpg_signed_commits/index.md)
- [Protected branches](../protected_branches.md)
@@ -304,7 +304,7 @@ For example, to enable **Check whether the commit author is a GitLab user** and
and create a filter for allowing commits from a specific email domain only through rails console:
WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
``` ruby
Project.find_each do |p|
diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md
index c85dd4555ca..9c977e4da40 100644
--- a/doc/user/project/repository/reducing_the_repo_size_using_git.md
+++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md
@@ -72,6 +72,12 @@ To purge files from a GitLab repository:
cd project.git
```
+1. Because cloning from a bundle file sets the `origin` remote to the local bundle file, change it to the URL of your repository:
+
+ ```shell
+ git remote set-url origin https://gitlab.example.com/<namespace>/<project_name>.git
+ ```
+
1. Using either `git filter-repo` or `git-sizer`, analyze your repository
and review the results to determine which items you want to purge:
@@ -84,38 +90,39 @@ To purge files from a GitLab repository:
git-sizer
```
-1. Proceed to purging any files from the history of your repository. Because we are
- trying to remove internal refs, we rely on the `commit-map` produced by each run to tell us
- which internal refs to remove.
-
- NOTE:
- `git filter-repo` creates a new `commit-map` file every run, and overwrites the `commit-map` from
- the previous run. You need this file from **every** run. Do the next step every time you run
- `git filter-repo`.
+1. Purge the history of your repository using relevant `git filter-repo` options.
+ Two common options are:
- To purge specific files, the `--path` and `--invert-paths` options can be combined:
+ - `--path` and `--invert-paths` to purge specific files:
- ```shell
- git filter-repo --path path/to/file.ext --invert-paths
- ```
+ ```shell
+ git filter-repo --path path/to/file.ext --invert-paths
+ ```
- To generally purge all files larger than 10M, the `--strip-blobs-bigger-than` option can be used:
+ - `--strip-blobs-bigger-than` to purge all files larger than for example 10M:
- ```shell
- git filter-repo --strip-blobs-bigger-than 10M
- ```
+ ```shell
+ git filter-repo --strip-blobs-bigger-than 10M
+ ```
See the
[`git filter-repo` documentation](https://htmlpreview.github.io/?https://github.com/newren/git-filter-repo/blob/docs/html/git-filter-repo.html#EXAMPLES)
for more examples and the complete documentation.
-1. Because cloning from a bundle file sets the `origin` remote to the local bundle file, delete this `origin` remote, and set it to the URL to your repository:
+1. Because you are trying to remove internal refs,
+ you'll later rely on `commit-map` files produced by each run
+ to tell you which internal refs to remove.
+ Every `git filter-repo` run creates a new `commit-map`,
+ and overwrites the `commit-map` from the previous run.
+ You can use the following command to back up each `commit-map` file:
```shell
- git remote remove origin
- git remote add origin https://gitlab.example.com/<namespace>/<project_name>.git
+ cp .git/filter-repo/commit-map ./_filter_repo_commit_map_$(date +%s)
```
+ Repeat this step and all following steps (including the [repository cleanup](#repository-cleanup) step)
+ every time you run any `git filter-repo` command.
+
1. Force push your changes to overwrite all branches on GitLab:
```shell
diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md
index e1f05cd5501..773662edb17 100644
--- a/doc/user/project/repository/web_editor.md
+++ b/doc/user/project/repository/web_editor.md
@@ -10,13 +10,13 @@ Sometimes it's easier to make quick changes directly from the GitLab interface
than to clone the project and use the Git command-line tool. In this feature
highlight, we look at how you can create a new file, directory, branch, or
tag from the file browser. All of these actions are available from a single
-dropdown menu.
+dropdown list.
## Create a file
From a project's files page, select the '+' button to the right of the branch selector.
-Choose **New file** from the dropdown.
-![New file dropdown menu](img/web_editor_new_file_dropdown_v14_1.png)
+Choose **New file** from the dropdown list.
+![New file dropdown list](img/web_editor_new_file_dropdown_v14_1.png)
Enter a filename in the **Filename** box. Then, add file content in the editor
area. Add a descriptive commit message and choose a branch. The branch field
@@ -55,6 +55,18 @@ NOTE:
The **Set up CI/CD** button does not appear on an empty repository. For the button
to display, add a file to your repository.
+## Preview Markdown
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378966) in GitLab 15.6.
+
+To preview Markdown content in the Web Editor, select the **Preview** tab.
+In this tab, you can see a live Markdown preview that updates as you type alongside your content.
+
+To close the preview panel, do one of the following:
+
+- Select the **Write** tab.
+- From the context menu, select **Hide Live Preview**.
+
## Highlight lines
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/56159) in GitLab 13.10 for GitLab SaaS instances.
@@ -85,7 +97,7 @@ this case, you need to upload a file.
From a project's files page, select the '+' button to the right of the branch
selector. Choose **Upload file** from the dropdown:
-![Upload file dropdown menu](img/web_editor_upload_file_dropdown_v14_1.png)
+![Upload file dropdown list](img/web_editor_upload_file_dropdown_v14_1.png)
After the upload dialog pops up, there are two ways to upload your file. Either
drag and drop a file on the popup or use the **click to upload** link. After you
@@ -104,7 +116,7 @@ directory.
From a project's files page, select the plus button (`+`) to the right of the branch selector.
Choose **New directory** from the dropdown.
-![New directory dropdown](img/web_editor_new_directory_dropdown_v14_1.png)
+![New directory dropdown list](img/web_editor_new_directory_dropdown_v14_1.png)
In the new directory dialog, enter a directory name, a commit message, and choose
the target branch. Select **Create directory** to finish.
@@ -138,6 +150,7 @@ The **Create merge request** button doesn't display if:
- A branch with the same name already exists.
- A merge request already exists for this branch.
- Your project has an active fork relationship.
+- Your project is private and the issue is confidential.
To make this button appear, one possible workaround is to
[remove your project's fork relationship](../settings/index.md#remove-a-fork-relationship).
@@ -177,9 +190,9 @@ merge request is merged.
If you want to make changes to several files before creating a new merge
request, you can create a new branch upfront.
-1. From a project's files page, choose **New branch** from the dropdown.
+1. From a project's files page, choose **New branch** from the dropdown list.
- ![New branch dropdown](img/web_editor_new_branch_dropdown_v14_1.png)
+ ![New branch dropdown list](img/web_editor_new_branch_dropdown_v14_1.png)
1. Enter a new **Branch name**.
1. Optional. Change the **Create from** field to choose which branch, tag, or
@@ -202,9 +215,9 @@ Tags help you mark major milestones such as production releases and
release candidates. You can create a tag from a branch or a commit
SHA:
-1. From a project's files page, choose **New tag** from the dropdown.
+1. From a project's files page, choose **New tag** from the dropdown list.
- ![New tag dropdown](img/web_editor_new_tag_dropdown.png)
+ ![New tag dropdown list](img/web_editor_new_tag_dropdown.png)
1. Give the tag a name such as `v1.0.0`.
1. Choose the branch or SHA from which you want to create this new tag.
@@ -238,6 +251,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/project/repository/x509_signed_commits/index.md b/doc/user/project/repository/x509_signed_commits/index.md
index 4a17b2ab84c..e16f5e4defe 100644
--- a/doc/user/project/repository/x509_signed_commits/index.md
+++ b/doc/user/project/repository/x509_signed_commits/index.md
@@ -163,6 +163,11 @@ can start signing your tags:
## Troubleshooting
+For committers without administrator access, review the list of
+[verification problems with signed commits](../gpg_signed_commits/index.md#fix-verification-problems-with-signed-commits)
+for possible fixes. The other troubleshooting suggestions on this page require
+administrator access.
+
### Re-verify commits
GitLab stores the status of any checked commits in the database. You can use a
diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md
index 3b1af1f688c..922accf9d28 100644
--- a/doc/user/project/requirements/index.md
+++ b/doc/user/project/requirements/index.md
@@ -5,7 +5,7 @@ group: Certify
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Requirements Management **(ULTIMATE)**
+# Requirements management **(ULTIMATE)**
NOTE:
In 14.4, Requirements was moved under **Issues**.
@@ -120,8 +120,8 @@ You can search for a requirement from the requirements list page based on the fo
To search for a requirement:
1. In a project, go to **Issues > Requirements > List**.
-1. Select the **Search or filter results** field. A dropdown menu appears.
-1. Select the requirement author or status from the dropdown or enter plain text to search by requirement title.
+1. Select the **Search or filter results** field. A dropdown list appears.
+1. Select the requirement author or status from the dropdown list or enter plain text to search by requirement title.
1. Press <kbd>Enter</kbd> on your keyboard to filter the list.
You can also sort the requirements list by:
diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md
index a47873f5179..3c12bb9b80f 100644
--- a/doc/user/project/settings/import_export.md
+++ b/doc/user/project/settings/import_export.md
@@ -254,168 +254,3 @@ and the exports between them are compatible.
- [Project import/export administration Rake tasks](../../../administration/raketasks/project_import_export.md)
- [Group import/export](../../group/settings/import_export.md)
- [Group import/export API](../../../api/group_import_export.md)
-
-## Troubleshooting
-
-### Project fails to import due to mismatch
-
-If the [shared runners enablement](../../../ci/runners/runners_scope.md#enable-shared-runners-for-a-project)
-does not match between the exported project, and the project import, the project fails to import.
-Review [issue 276930](https://gitlab.com/gitlab-org/gitlab/-/issues/276930), and either:
-
-- Ensure shared runners are enabled in both the source and destination projects.
-- Disable shared runners on the parent group when you import the project.
-
-### Import workarounds for large repositories
-
-[Maximum import size limitations](#import-a-project-and-its-data)
-can prevent an import from being successful. If changing the import limits is not possible, you can
-try one of the workarounds listed here.
-
-#### Workaround option 1
-
-The following local workflow can be used to temporarily
-reduce the repository size for another import attempt:
-
-1. Create a temporary working directory from the export:
-
- ```shell
- EXPORT=<filename-without-extension>
-
- mkdir "$EXPORT"
- tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/
- cd "$EXPORT"/
- git clone project.bundle
-
- # Prevent interference with recreating an importable file later
- mv project.bundle ../"$EXPORT"-original.bundle
- mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz
-
- git switch --create smaller-tmp-main
- ```
-
-1. To reduce the repository size, work on this `smaller-tmp-main` branch:
- [identify and remove large files](../repository/reducing_the_repo_size_using_git.md)
- or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase)
- to reduce the number of commits.
-
- ```shell
- # Reduce the .git/objects/pack/ file size
- cd project
- git reflog expire --expire=now --all
- git gc --prune=now --aggressive
-
- # Prepare recreating an importable file
- git bundle create ../project.bundle <default-branch-name>
- cd ..
- mv project/ ../"$EXPORT"-project
- cd ..
-
- # Recreate an importable file
- tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ .
- ```
-
-1. Import this new, smaller file into GitLab.
-1. In a full clone of the original repository,
- use `git remote set-url origin <new-url> && git push --force --all`
- to complete the import.
-1. Update the imported repository's
- [branch protection rules](../protected_branches.md) and
- its [default branch](../repository/branches/default.md), and
- delete the temporary, `smaller-tmp-main` branch, and
- the local, temporary data.
-
-#### Workaround option 2
-
-NOTE:
-This workaround does not account for LFS objects.
-
-Rather than attempting to push all changes at once, this workaround:
-
-- Separates the project import from the Git Repository import
-- Incrementally pushes the repository to GitLab
-
-1. Make a local clone of the repository to migrate. In a later step, you push this clone outside of
- the project export.
-1. Download the export and remove the `project.bundle` (which contains the Git repository):
-
- ```shell
- tar -czvf new_export.tar.gz --exclude='project.bundle' @old_export.tar.gz
- ```
-
-1. Import the export without a Git repository. It asks you to confirm to import without a
- repository.
-1. Save this bash script as a file and run it after adding the appropriate origin.
-
- ```shell
- #!/bin/sh
-
- # ASSUMPTIONS:
- # - The GitLab location is "origin"
- # - The default branch is "main"
- # - This will attempt to push in chunks of 500MB (dividing the total size by 500MB).
- # Decrease this size to push in smaller chunks if you still receive timeouts.
-
- git gc
- SIZE=$(git count-objects -v 2> /dev/null | grep size-pack | awk '{print $2}')
-
- # Be conservative... and try to push 2GB at a time
- # (given this assumes each commit is the same size - which is wrong)
- BATCHES=$(($SIZE / 500000))
- TOTAL_COMMITS=$(git rev-list --count HEAD)
- if (( BATCHES > TOTAL_COMMITS )); then
- BATCHES=$TOTAL_COMMITS
- fi
-
- INCREMENTS=$(( ($TOTAL_COMMITS / $BATCHES) - 1 ))
-
- for (( BATCH=BATCHES; BATCH>=1; BATCH-- ))
- do
- COMMIT_NUM=$(( $BATCH - $INCREMENTS ))
- COMMIT_SHA=$(git log -n $COMMIT_NUM --format=format:%H | tail -1)
- git push -u origin ${COMMIT_SHA}:refs/heads/main
- done
- git push -u origin main
- git push -u origin --all
- git push -u origin --tags
- ```
-
-### Manually execute export steps
-
-Exports sometimes fail without giving enough information to troubleshoot. In these cases, it can be
-helpful to [open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
-and loop through [all the defined exporters](https://gitlab.com/gitlab-org/gitlab/-/blob/b67a5b5a12498d57cd877023b7385f7251e57de8/app/services/projects/import_export/export_service.rb#L65).
-Execute each line individually, rather than pasting the entire block at once, so you can see any
-errors each command returns.
-
-```shell
-# User needs to have permission to export
-u = User.find_by_username('someuser')
-p = Project.find_by_full_path('some/project')
-e = Projects::ImportExport::ExportService.new(p,u)
-
-e.send(:version_saver).send(:save)
-e.send(:repo_saver).send(:save)
-## continue using `e.send(:exporter_name).send(:save)` going through the list of exporters
-
-# The following line should show you the export_path similar to /var/opt/gitlab/gitlab-rails/shared/tmp/gitlab_exports/@hashed/49/94/4994....
-s = Gitlab::ImportExport::Saver.new(exportable: p, shared:p.import_export_shared)
-
-# To try and upload use:
-s.send(:compress_and_save)
-s.send(:save_upload)
-```
-
-### Import using the REST API fails when using a group access token
-
-[Group access tokens](../../group/settings/group_access_tokens.md)
-don't work for project or group import operations. When a group access token initiates an import,
-the import fails with this message:
-
-```plaintext
-Error adding importer user to Project members.
-Validation failed: User project bots cannot be added to other groups / projects
-```
-
-To use [Import REST API](../../../api/project_import_export.md),
-pass regular user account credentials such as [personal access tokens](../../profile/personal_access_tokens.md).
diff --git a/doc/user/project/settings/import_export_troubleshooting.md b/doc/user/project/settings/import_export_troubleshooting.md
new file mode 100644
index 00000000000..81ceba7139b
--- /dev/null
+++ b/doc/user/project/settings/import_export_troubleshooting.md
@@ -0,0 +1,280 @@
+---
+stage: Manage
+group: Import
+info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments"
+---
+
+# Troubleshooting file export project migrations
+
+If you have problems with [migrating projects using file exports](import_export.md), see the possible solutions below.
+
+## Troubleshooting commands
+
+Finds information about the status of the import and further logs using the JID:
+
+```ruby
+# Rails console
+Project.find_by_full_path('group/project').import_state.slice(:jid, :status, :last_error)
+> {"jid"=>"414dec93f941a593ea1a6894", "status"=>"finished", "last_error"=>nil}
+```
+
+```shell
+# Logs
+grep JID /var/log/gitlab/sidekiq/current
+grep "Import/Export error" /var/log/gitlab/sidekiq/current
+grep "Import/Export backtrace" /var/log/gitlab/sidekiq/current
+tail /var/log/gitlab/gitlab-rails/importer.log
+```
+
+## Project fails to import due to mismatch
+
+If the [shared runners enablement](../../../ci/runners/runners_scope.md#enable-shared-runners-for-a-project)
+does not match between the exported project, and the project import, the project fails to import.
+Review [issue 276930](https://gitlab.com/gitlab-org/gitlab/-/issues/276930), and either:
+
+- Ensure shared runners are enabled in both the source and destination projects.
+- Disable shared runners on the parent group when you import the project.
+
+## Import workarounds for large repositories
+
+[Maximum import size limitations](import_export.md#import-a-project-and-its-data)
+can prevent an import from being successful. If changing the import limits is not possible, you can
+try one of the workarounds listed here.
+
+### Workaround option 1
+
+The following local workflow can be used to temporarily
+reduce the repository size for another import attempt:
+
+1. Create a temporary working directory from the export:
+
+ ```shell
+ EXPORT=<filename-without-extension>
+
+ mkdir "$EXPORT"
+ tar -xf "$EXPORT".tar.gz --directory="$EXPORT"/
+ cd "$EXPORT"/
+ git clone project.bundle
+
+ # Prevent interference with recreating an importable file later
+ mv project.bundle ../"$EXPORT"-original.bundle
+ mv ../"$EXPORT".tar.gz ../"$EXPORT"-original.tar.gz
+
+ git switch --create smaller-tmp-main
+ ```
+
+1. To reduce the repository size, work on this `smaller-tmp-main` branch:
+ [identify and remove large files](../repository/reducing_the_repo_size_using_git.md)
+ or [interactively rebase and fixup](../../../topics/git/git_rebase.md#interactive-rebase)
+ to reduce the number of commits.
+
+ ```shell
+ # Reduce the .git/objects/pack/ file size
+ cd project
+ git reflog expire --expire=now --all
+ git gc --prune=now --aggressive
+
+ # Prepare recreating an importable file
+ git bundle create ../project.bundle <default-branch-name>
+ cd ..
+ mv project/ ../"$EXPORT"-project
+ cd ..
+
+ # Recreate an importable file
+ tar -czf "$EXPORT"-smaller.tar.gz --directory="$EXPORT"/ .
+ ```
+
+1. Import this new, smaller file into GitLab.
+1. In a full clone of the original repository,
+ use `git remote set-url origin <new-url> && git push --force --all`
+ to complete the import.
+1. Update the imported repository's
+ [branch protection rules](../protected_branches.md) and
+ its [default branch](../repository/branches/default.md), and
+ delete the temporary, `smaller-tmp-main` branch, and
+ the local, temporary data.
+
+### Workaround option 2
+
+NOTE:
+This workaround does not account for LFS objects.
+
+Rather than attempting to push all changes at once, this workaround:
+
+- Separates the project import from the Git Repository import
+- Incrementally pushes the repository to GitLab
+
+1. Make a local clone of the repository to migrate. In a later step, you push this clone outside of
+ the project export.
+1. Download the export and remove the `project.bundle` (which contains the Git repository):
+
+ ```shell
+ tar -czvf new_export.tar.gz --exclude='project.bundle' @old_export.tar.gz
+ ```
+
+1. Import the export without a Git repository. It asks you to confirm to import without a
+ repository.
+1. Save this bash script as a file and run it after adding the appropriate origin.
+
+ ```shell
+ #!/bin/sh
+
+ # ASSUMPTIONS:
+ # - The GitLab location is "origin"
+ # - The default branch is "main"
+ # - This will attempt to push in chunks of 500MB (dividing the total size by 500MB).
+ # Decrease this size to push in smaller chunks if you still receive timeouts.
+
+ git gc
+ SIZE=$(git count-objects -v 2> /dev/null | grep size-pack | awk '{print $2}')
+
+ # Be conservative... and try to push 2GB at a time
+ # (given this assumes each commit is the same size - which is wrong)
+ BATCHES=$(($SIZE / 500000))
+ TOTAL_COMMITS=$(git rev-list --count HEAD)
+ if (( BATCHES > TOTAL_COMMITS )); then
+ BATCHES=$TOTAL_COMMITS
+ fi
+
+ INCREMENTS=$(( ($TOTAL_COMMITS / $BATCHES) - 1 ))
+
+ for (( BATCH=BATCHES; BATCH>=1; BATCH-- ))
+ do
+ COMMIT_NUM=$(( $BATCH - $INCREMENTS ))
+ COMMIT_SHA=$(git log -n $COMMIT_NUM --format=format:%H | tail -1)
+ git push -u origin ${COMMIT_SHA}:refs/heads/main
+ done
+ git push -u origin main
+ git push -u origin --all
+ git push -u origin --tags
+ ```
+
+## Manually execute export steps
+
+You usually export a project through [the web interface](import_export.md#export-a-project-and-its-data) or through [the API](../../../api/project_import_export.md). Exporting using these
+methods can sometimes fail without giving enough information to troubleshoot. In these cases,
+[open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session)
+Execute each line individually, rather than pasting the entire block at once, so you can see any
+errors each command returns.
+
+```shell
+# User needs to have permission to export
+u = User.find_by_username('someuser')
+p = Project.find_by_full_path('some/project')
+e = Projects::ImportExport::ExportService.new(p,u)
+
+e.send(:version_saver).send(:save)
+e.send(:repo_saver).send(:save)
+## continue using `e.send(:exporter_name).send(:save)` going through the list of exporters
+
+# The following line should show you the export_path similar to /var/opt/gitlab/gitlab-rails/shared/tmp/gitlab_exports/@hashed/49/94/4994....
+s = Gitlab::ImportExport::Saver.new(exportable: p, shared:p.import_export_shared)
+
+# To try and upload use:
+s.send(:compress_and_save)
+s.send(:save_upload)
+```
+
+After the project is successfully uploaded, the exported project is located in a `.tar.gz` file in `/var/opt/gitlab/gitlab-rails/uploads/-/system/import_export_upload/export_file/`.
+
+## Import using the REST API fails when using a group access token
+
+[Group access tokens](../../group/settings/group_access_tokens.md)
+don't work for project or group import operations. When a group access token initiates an import,
+the import fails with this message:
+
+```plaintext
+Error adding importer user to Project members.
+Validation failed: User project bots cannot be added to other groups / projects
+```
+
+To use [Import REST API](../../../api/project_import_export.md),
+pass regular user account credentials such as [personal access tokens](../../profile/personal_access_tokens.md).
+
+## Troubleshooting performance issues
+
+Read through the current performance problems using the Import/Export below.
+
+### OOM errors
+
+Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](../../../administration/sidekiq/sidekiq_memory_killer.md):
+
+```shell
+SIDEKIQ_MEMORY_KILLER_MAX_RSS = 2000000
+SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS = 3000000
+SIDEKIQ_MEMORY_KILLER_GRACE_TIME = 900
+```
+
+An import status `started`, and the following Sidekiq logs signal a memory issue:
+
+```shell
+WARN: Work still in progress <struct with JID>
+```
+
+### Timeouts
+
+Timeout errors occur due to the `Gitlab::Import::StuckProjectImportJobsWorker` marking the process as failed:
+
+```ruby
+module Gitlab
+ module Import
+ class StuckProjectImportJobsWorker
+ include Gitlab::Import::StuckImportJob
+ # ...
+ end
+ end
+end
+
+module Gitlab
+ module Import
+ module StuckImportJob
+ # ...
+ IMPORT_JOBS_EXPIRATION = 15.hours.to_i
+ # ...
+ def perform
+ stuck_imports_without_jid_count = mark_imports_without_jid_as_failed!
+ stuck_imports_with_jid_count = mark_imports_with_jid_as_failed!
+
+ track_metrics(stuck_imports_with_jid_count, stuck_imports_without_jid_count)
+ end
+ # ...
+ end
+ end
+end
+```
+
+```shell
+Marked stuck import jobs as failed. JIDs: xyz
+```
+
+```plaintext
+ +-----------+ +-----------------------------------+
+ |Export Job |--->| Calls ActiveRecord `as_json` and |
+ +-----------+ | `to_json` on all project models |
+ +-----------------------------------+
+
+ +-----------+ +-----------------------------------+
+ |Import Job |--->| Loads all JSON in memory, then |
+ +-----------+ | inserts into the DB in batches |
+ +-----------------------------------+
+```
+
+### Problems and solutions
+
+| Problem | Possible solutions |
+| -------- | -------- |
+| [Slow JSON](https://gitlab.com/gitlab-org/gitlab/-/issues/25251) loading/dumping models from the database | [split the worker](https://gitlab.com/gitlab-org/gitlab/-/issues/25252) |
+| | Batch export
+| | Optimize SQL
+| | Move away from `ActiveRecord` callbacks (difficult)
+| High memory usage (see also some [analysis](https://gitlab.com/gitlab-org/gitlab/-/issues/18857) | DB Commit sweet spot that uses less memory |
+| | [Netflix Fast JSON API](https://github.com/Netflix/fast_jsonapi) may help |
+| | Batch reading/writing to disk and any SQL
+
+### Temporary solutions
+
+While the performance problems are not tackled, there is a process to workaround
+importing big projects, using a foreground import:
+
+[Foreground import](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/5384) of big projects for customers.
+(Using the import template in the [infrastructure tracker](https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues))
diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md
index 4407986f354..a872a339433 100644
--- a/doc/user/project/settings/index.md
+++ b/doc/user/project/settings/index.md
@@ -45,7 +45,7 @@ If you're an instance administrator, you can administer all project topics from
## Add a compliance framework to a project **(PREMIUM)**
-[Compliance frameworks](../../group/manage.md#compliance-frameworks) can be assigned to projects within group that has a
+[Compliance frameworks](../../group/compliance_frameworks.md) can be assigned to projects within group that has a
compliance framework using either:
- The GitLab UI:
@@ -91,9 +91,12 @@ Use the toggles to enable or disable features in the project.
| **Wiki** | ✓ | Enables a separate system for [documentation](../wiki/index.md). |
| **Snippets** | ✓ | Enables [sharing of code and text](../../snippets.md). |
| **Pages** | ✓ | Allows you to [publish static websites](../pages/index.md). |
-| **Operations** | ✓ | Control access to Operations-related features, including [Operations Dashboard](../../../operations/index.md), [Environments and Deployments](../../../ci/environments/index.md), [Feature Flags](../../../operations/feature_flags.md). |
| **Metrics Dashboard** | ✓ | Control access to [metrics dashboard](../integrations/prometheus.md). |
| **Releases** | ✓ | Control access to [Releases](../releases/index.md). |
+| **Environments** | ✓ | Control access to [Environments and Deployments](../../../ci/environments/index.md). |
+| **Feature flags** | ✓ | Control access to [Feature Flags](../../../operations/feature_flags.md). |
+| **Monitor** | ✓ | Control access to [Monitor](../../../operations/index.md) features. |
+| **Infrastructure** | ✓ | Control access to [Infrastructure](../../infrastructure/index.md) features. |
When you disable a feature, the following additional features are also disabled:
@@ -277,7 +280,7 @@ To delete a project:
1. In the "Delete project" section, select **Delete project**.
1. Confirm the action when asked to.
-This action deletes a project including all associated resources (issues, merge requests, and so on).
+This action deletes a project including all associated resources (such as issues and merge requests).
WARNING:
The default deletion behavior for projects was changed to [delayed project deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/32935)
diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md
index a0eac9376f2..0200e9c4e7d 100644
--- a/doc/user/project/web_ide/index.md
+++ b/doc/user/project/web_ide/index.md
@@ -202,12 +202,12 @@ left.
## Switching merge requests
To switch between your authored and assigned merge requests, select the
-dropdown in the top of the sidebar to open a list of merge requests. You must commit or discard all your changes before switching to a different merge
+dropdown list in the top of the sidebar to open a list of merge requests. You must commit or discard all your changes before switching to a different merge
request.
## Switching branches
-To switch between branches of the current project repository, select the dropdown
+To switch between branches of the current project repository, select the dropdown list
in the top of the sidebar to open a list of branches.
You must commit or discard all your changes before switching to a
different branch.
@@ -459,3 +459,12 @@ The Web IDE has a few limitations:
- If the terminal displays **Connection Failure**, then the terminal could not
connect to the runner. Try to stop and restart the terminal. If the
problem persists, double check your runner configuration.
+
+## VSCode Reimplementation
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95169) in GitLab 15.4 [with a flag](../../../administration/feature_flags.md) named `vscode_web_ide`. Disabled by default.
+
+As announced in [this blog post](https://about.gitlab.com/blog/2022/05/23/the-future-of-the-gitlab-web-ide/),
+the current implementation of the Web IDE will be replaced with a [VSCode inspired implementation](https://gitlab.com/groups/gitlab-org/-/epics/7683).
+
+This effort is currently under development. Follow [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7683) for updates and more information.
diff --git a/doc/user/project/working_with_projects.md b/doc/user/project/working_with_projects.md
index 705e49df039..067a0303277 100644
--- a/doc/user/project/working_with_projects.md
+++ b/doc/user/project/working_with_projects.md
@@ -516,7 +516,7 @@ If a project or repository has been updated but the state is not reflected in th
You can do so through [a Rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session) and one of the following:
WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
```ruby
## Clear project cache
@@ -545,7 +545,7 @@ end
If a project cannot be deleted, you can attempt to delete it through [Rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session).
WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
```ruby
project = Project.find_by_full_path('<project_path>')
@@ -569,7 +569,7 @@ To toggle a specific feature, you can [start a Rails console session](../../admi
and run the following function:
WARNING:
-Any command that changes data directly could be damaging if not run correctly, or under the right conditions. We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case.
+Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore.
```ruby
projects = Group.find_by_name('_group_name').projects
diff --git a/doc/user/public_access.md b/doc/user/public_access.md
index 703932e50f6..bdc711f2098 100644
--- a/doc/user/public_access.md
+++ b/doc/user/public_access.md
@@ -18,6 +18,11 @@ for your GitLab instance). For example, <https://gitlab.com/public>.
You can control the visibility of individual features with
[project feature settings](permissions.md#project-features).
+The visibility setting of a project must be at least as restrictive
+as the visibility of its parent group.
+For example, a private group can include only private projects,
+while a public group can include private, internal, and public projects.
+
## Public projects and groups
Public projects can be cloned **without any** authentication over HTTPS.
@@ -73,11 +78,12 @@ Prerequisite:
## Change group visibility
-Prerequisite:
+Prerequisites:
- You must have the Owner role for a group.
- Subgroups and projects must already have visibility settings that are at least as
- restrictive as the new setting of the parent group.
+ restrictive as the new setting of the parent group. For example, you cannot set a group
+ to private if a subgroup or project in that group is public.
1. On the top bar, select **Main menu > Groups** and find your project.
1. On the left sidebar, select **Settings > General**.
@@ -101,6 +107,6 @@ important to describe those, too. Think of things that may go wrong and include
This is important to minimize requests for support, and to avoid doc comments with
questions that you know someone might ask.
-Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+Each scenario can be a third-level heading, for example `### Getting error message X`.
If you have none to add when creating a doc, leave this section in place
but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/user/reserved_names.md b/doc/user/reserved_names.md
index f8c735eaea8..1ab22fb846d 100644
--- a/doc/user/reserved_names.md
+++ b/doc/user/reserved_names.md
@@ -26,7 +26,7 @@ under the `TOP_LEVEL_ROUTES`, `PROJECT_WILDCARD_ROUTES` and `GROUP_ROUTES` lists
## Reserved project names
-It is currently not possible to create a project with the following names:
+It is not possible to create a project with the following names:
- `\-`
- `badges`
@@ -52,7 +52,7 @@ It is currently not possible to create a project with the following names:
## Reserved group names
-Currently, the following names are reserved as top level groups:
+The following names are reserved as top level groups:
- `\-`
- `.well-known`
diff --git a/doc/user/search/advanced_search.md b/doc/user/search/advanced_search.md
index 86b59572e77..925fc7e6036 100644
--- a/doc/user/search/advanced_search.md
+++ b/doc/user/search/advanced_search.md
@@ -37,9 +37,39 @@ You can use Advanced Search in:
## Syntax
-See [Advanced Search syntax](global_search/advanced_search_syntax.md) for more information.
+Advanced Search uses [Elasticsearch syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html#simple-query-string-syntax). The syntax supports both exact and fuzzy search queries.
-## Search by ID
+<!-- markdownlint-disable -->
-- To search by issue ID, use the `#` prefix followed by the issue ID (for example, [`#23456`](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=%2323456&group_id=9970&project_id=278964)).
-- To search by merge request ID, use the `!` prefix followed by the merge request ID (for example, [`!23456`](https://gitlab.com/search?snippets=&scope=merge_requests&repository_ref=&search=%2123456&group_id=9970&project_id=278964)).
+| Syntax | Description | Example |
+|--------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `"` | Exact search | [`"gem sidekiq"`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=%22gem+sidekiq%22) |
+| <code>&#124;</code> | Or | [<code>display &#124; banner</code>](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+%7C+banner) |
+| `+` | And | [`display +banner`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=display+%2Bbanner&snippets=) |
+| `-` | Exclude | [`display -banner`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+-banner) |
+| `*` | Partial | [`bug error 50*`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=bug+error+50%2A&snippets=) |
+| `\` | Escape | [`\*md`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=%5C*md&group_id=9970&project_id=278964) |
+| `#` | Issue ID | [`#23456`](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=%2323456&group_id=9970&project_id=278964) |
+| `!` | Merge request ID | [`!23456`](https://gitlab.com/search?snippets=&scope=merge_requests&repository_ref=&search=%2123456&group_id=9970&project_id=278964) |
+
+### Code search
+
+| Syntax | Description | Example |
+|--------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `filename:` | Filename | [`filename:*spec.rb`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=filename%3A*spec.rb&group_id=9970&project_id=278964) |
+| `path:` | Repository location | [`path:spec/workers/`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=path%3Aspec%2Fworkers&snippets=) |
+| `extension:` | File extension without `.` | [`extension:js`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=extension%3Ajs&snippets=) |
+| `blob:` | Git object ID | [`blob:998707*`](https://gitlab.com/search?snippets=false&scope=blobs&repository_ref=&search=blob%3A998707*&group_id=9970) |
+
+`extension:` and `blob:` return exact matches only.
+
+### Examples
+
+| Query | Description |
+|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------|
+| [`rails -filename:gemfile.lock`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=rails+-filename%3Agemfile.lock&snippets=) | Returns `rails` in all files except the `gemfile.lock` file. |
+| [`RSpec.describe Resolvers -*builder`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=RSpec.describe+Resolvers+-*builder) | Returns `RSpec.describe Resolvers` that does not start with `builder`. |
+| [<code>bug &#124; (display +banner)</code>](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+%7C+%28display+%2Bbanner%29&group_id=9970&project_id=278964) | Returns `bug` or both `display` and `banner`. |
+| [<code>helper -extension:yml -extension:js</code>](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=helper+-extension%3Ayml+-extension%3Ajs&snippets=) | Returns `helper` in all files except files with a `.yml` or `.js` extension. |
+
+<!-- markdownlint-enable -->
diff --git a/doc/user/search/global_search/advanced_search_syntax.md b/doc/user/search/global_search/advanced_search_syntax.md
index cd4bff0a20e..c34c5c0c8fc 100644
--- a/doc/user/search/global_search/advanced_search_syntax.md
+++ b/doc/user/search/global_search/advanced_search_syntax.md
@@ -1,52 +1,11 @@
---
-stage: Data Stores
-group: Global Search
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '../advanced_search.md'
+remove_date: '2023-02-02'
---
-# Advanced Search syntax **(PREMIUM)**
+This document was moved to [another location](../advanced_search.md).
-With [Advanced Search](../advanced_search.md), you can perform a thorough
-search of your entire GitLab instance.
-
-The Advanced Search syntax supports fuzzy or exact search queries with prefixes,
-boolean operators, and more. Advanced Search uses
-[Elasticsearch's syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html#simple-query-string-syntax).
-
-WARNING:
-Advanced Search searches default project branches only.
-
-## General search
-
-<!-- markdownlint-disable -->
-
-| Use | Description | Example |
-|-----|--------------|-----------------------------------------------------------------------------------------------------------------------------------------------|
-| `"` | Exact search | [`"gem sidekiq"`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=%22gem+sidekiq%22) |
-| <code>&#124;</code> | Or | [<code>display &#124; banner</code>](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+%7C+banner) |
-| `+` | And | [`display +banner`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=display+%2Bbanner&snippets=) |
-| `-` | Exclude | [`display -banner`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+-banner) |
-| `*` | Partial | [`bug error 50*`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=bug+error+50%2A&snippets=) |
-| `\` | Escape | [`\*md`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=%5C*md&group_id=9970&project_id=278964) |
-
-## Code search
-
-| Use | Description | Example |
-|--------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `filename:` | Filename | [`filename:*spec.rb`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=filename%3A*spec.rb&group_id=9970&project_id=278964) |
-| `path:` | Repository location | [`path:spec/workers/`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=path%3Aspec%2Fworkers&snippets=) |
-| `extension:` | File extension, without the `.` | [`extension:js`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=extension%3Ajs&snippets=) |
-| `blob:` | Git object ID | [`blob:998707*`](https://gitlab.com/search?snippets=false&scope=blobs&repository_ref=&search=blob%3A998707*&group_id=9970) |
-
-`extension` and `blob` return exact matches only.
-
-## Examples
-
-| Example | Description |
-|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------|
-| [`rails -filename:gemfile.lock`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=rails+-filename%3Agemfile.lock&snippets=) | Show _rails_ in all files except the _`gemfile.lock`_ file. |
-| [`RSpec.describe Resolvers -*builder`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=RSpec.describe+Resolvers+-*builder) | Show all _RSpec.describe Resolvers_ that don't start with _builder_. |
-| [<code>bug &#124; (display +banner)</code>](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+%7C+%28display+%2Bbanner%29&group_id=9970&project_id=278964) | Show _bug_ **or** _display_ **and** _banner_. |
-| [<code>helper -extension:yml -extension:js</code>](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=helper+-extension%3Ayml+-extension%3Ajs&snippets=) | Show _helper_ in all files, except for files with _`.yml`_ **or** _`.js`_ extensions. |
-
-<!-- markdownlint-enable -->
+<!-- This redirect file can be deleted after <2023-02-02>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md
index bf233bef6a2..f9e61ad78ad 100644
--- a/doc/user/shortcuts.md
+++ b/doc/user/shortcuts.md
@@ -272,7 +272,8 @@ These shortcuts are available when editing a file with the
| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>8</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>8</kbd> | Unordered list |
| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>9</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>9</kbd> | Task list |
| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>b</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>b</kbd> | Blockquote |
-| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>c</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>c</kbd> | Code block |
+| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>c</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>c</kbd> | Code block |
+| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>h</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>h</kbd> | Highlight |
| <kbd>Command</kbd> + <kbd>,</kbd> | <kbd>Control</kbd> + <kbd>,</kbd> | Subscript |
| <kbd>Command</kbd> + <kbd>.</kbd> | <kbd>Control</kbd> + <kbd>.</kbd> | Superscript |
| <kbd>Tab</kbd> | <kbd>Tab</kbd> | Indent list |
diff --git a/doc/user/ssh.md b/doc/user/ssh.md
index be76ce487b6..85332446334 100644
--- a/doc/user/ssh.md
+++ b/doc/user/ssh.md
@@ -252,6 +252,29 @@ To generate ED25519_SK or ECDSA_SK SSH keys, you must use OpenSSH 8.2 or later:
A public and private key are generated.
[Add the public SSH key to your GitLab account](#add-an-ssh-key-to-your-gitlab-account).
+## Generate an SSH key pair with a password manager
+
+### Generate an SSH key pair with 1Password
+
+You can use [1Password](https://1password.com/) and the [1Password browser extension](https://support.1password.com/getting-started-browser/) to either:
+
+- Automatically generate a new SSH key.
+- Use an existing SSH in your 1Password vault to authenticate with GitLab.
+
+1. Sign in to GitLab.
+1. On the top bar, in the top right corner, select your avatar.
+1. Select **Preferences**.
+1. On the left sidebar, select **SSH Keys**.
+1. Select **Key**, and you should see the 1Password helper appear.
+1. Select the 1Password icon and unlock 1Password.
+1. You can then select **Create SSH Key** or select an existing SSH key to fill in the public key.
+1. In the **Title** box, type a description, like `Work Laptop` or
+ `Home Workstation`.
+1. Optional. Update **Expiration date** to modify the default expiration date.
+1. Select **Add key**.
+
+To learn more about using 1Password with SSH keys, see [1Password's documentation](https://developer.1password.com/docs/ssh/get-started).
+
## Add an SSH key to your GitLab account
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271239) in GitLab 15.4, default expiration date suggested in UI.
@@ -499,4 +522,4 @@ You can troubleshoot this by trying the following:
- Verify your IDO/U2F hardware security key supports
the key type provided.
- Verify the version of OpenSSH is 8.2 or greater by
- running `ssh -v`.
+ running `ssh -V`.
diff --git a/doc/user/tasks.md b/doc/user/tasks.md
index 10a86724bf1..5c9290e57cb 100644
--- a/doc/user/tasks.md
+++ b/doc/user/tasks.md
@@ -18,7 +18,7 @@ For the latest updates, check the [Tasks Roadmap](https://gitlab.com/groups/gitl
FLAG:
On self-managed GitLab, by default this feature is available. To hide the feature,
-ask an administrator to [disable the feature flags](../administration/feature_flags.md) named `work_items` and `work_items_hierarchy`.
+ask an administrator to [disable the feature flags](../administration/feature_flags.md) named `work_items`.
On GitLab.com, this feature is available.
Use tasks to track steps needed for the [issue](project/issues/index.md) to be closed.
@@ -50,9 +50,26 @@ Prerequisites:
To create a task:
1. In the issue description, in the **Tasks** section, select **Add**.
+1. Select **New task**.
1. Enter the task title.
1. Select **Create task**.
+## Add existing tasks to an issue
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381868) in GitLab 15.6.
+
+Prerequisites:
+
+- You must have at least the Guest role for the project, or the project must be public.
+
+To add a task:
+
+1. In the issue description, in the **Tasks** section, select **Add**.
+1. Select **Existing task**.
+1. Search tasks by title.
+1. Select one or multiple tasks to add to the issue.
+1. Select **Add task**.
+
## Edit a task
Prerequisites:
@@ -159,6 +176,30 @@ To set a start date:
The due date must be the same or later than the start date.
If you select a start date to be later than the due date, the due date is then changed to the same day.
+## Add a task to a milestone
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367463) in GitLab 15.5 [with a flag](../administration/feature_flags.md) named `work_items_mvc_2`. Disabled by default.
+
+FLAG:
+On self-managed GitLab, by default this feature is not available. To make it available per group, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `work_items_mvc_2`. On GitLab.com, this feature is not available. The feature is not ready for production use.
+
+You can add a task to a [milestone](project/milestones/index.md).
+You can see the milestone title when you view a task.
+If you create a task for an issue that already belongs to a milestone,
+the new task inherits the milestone.
+
+Prerequisites:
+
+- You must have at least the Reporter role for the project.
+
+To add a task to a milestone:
+
+1. In the issue description, in the **Tasks** section, select the title of the task you want to edit.
+ The task window opens.
+1. Next to **Milestone**, select **Add to milestone**.
+If a task already belongs to a milestone, the dropdown list shows the current milestone.
+1. From the dropdown list, select the milestone to be associated with the task.
+
## Set task weight **(PREMIUM)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362550) in GitLab 15.3.
diff --git a/doc/user/usage_quotas.md b/doc/user/usage_quotas.md
index 1b265e86dd4..7df5ade215d 100644
--- a/doc/user/usage_quotas.md
+++ b/doc/user/usage_quotas.md
@@ -7,52 +7,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Storage usage quota **(FREE)**
-> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/13294) in GitLab 12.0.
-> - Moved to GitLab Free.
-
-## Namespace storage limit
-
-Namespaces on GitLab SaaS have a storage limit. For more information, see our [pricing page](https://about.gitlab.com/pricing/).
-This limit is not visible on the Usage quotas page, but will be prior to the limit being [applied](#namespace-storage-limit-application-schedule). Self-managed deployments are not affected.
-
-Storage types that add to the total namespace storage are:
-
-- Git repository
-- Git LFS
-- Artifacts
-- Container registry
-- Package registry
-- Dependency proxy
-- Wiki
-- Snippets
-
-If your total namespace storage exceeds the available namespace storage quota, all projects under the namespace are locked.
-A locked project cannot push to the repository, run pipelines and jobs, or build and push packages.
-
-To prevent exceeding the namespace storage quota, you can:
-
-- Reduce storage consumption by following the suggestions in the [Manage Your Storage Usage](#manage-your-storage-usage) section of this page.
-- Apply for [GitLab for Education](https://about.gitlab.com/solutions/education/join/), [GitLab for Open Source](https://about.gitlab.com/solutions/open-source/join/), or [GitLab for Startups](https://about.gitlab.com/solutions/startups/) if you meet the eligibility requirements.
-- Consider using a [self-managed instance](../subscriptions/self_managed/index.md) of GitLab which does not have these limits on the free tier.
-- [Purchase additional storage](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer) units at $60/year for 10GB of storage.
-- [Start a trial](https://about.gitlab.com/free-trial/) or [upgrade to GitLab Premium or Ultimate](https://about.gitlab.com/pricing) which include higher limits and features that enable growing teams to ship faster without sacrificing on quality.
-- [Talk to an expert](https://page.gitlab.com/usage_limits_help.html) to learn more about your options and ask questions.
-
-### Namespace storage limit application schedule
-
-Information on when namespace-level storage limits will be applied is available on these FAQ pages for the [Free](https://about.gitlab.com/pricing/faq-efficient-free-tier/#storage-limits-on-gitlab-saas-free-tier) and [Paid](https://about.gitlab.com/pricing/faq-paid-storage-transfer/) tier.
-
-## Project storage limit
-
-Projects on GitLab SaaS have a 10GB storage limit on their Git repository and LFS storage.
-After namespace-level storage limits are applied, the project limit will be removed. A namespace has either a namespace-level storage limit or a project-level storage limit, but not both.
-
-When a project's repository and LFS reaches the quota, the project is locked.
-You cannot push changes to a locked project. To monitor the size of each
-repository in a namespace, including a breakdown for each project,
-[view storage usage](#view-storage-usage). To allow a project's repository and LFS to exceed the free quota
-you must purchase additional storage. For more details, see [Excess storage usage](#excess-storage-usage).
-
## View storage usage
Prerequisites:
@@ -72,7 +26,7 @@ is updated every 90 minutes.
If your namespace shows `'Not applicable.'`, push a commit to any project in the
namespace to recalculate the storage.
-## Storage usage statistics
+### Storage usage statistics
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68898) project-level graph in GitLab 14.4 [with a flag](../administration/feature_flags.md) named `project_storage_ui`. Disabled by default.
> - Enabled on GitLab.com in GitLab 14.4.
@@ -97,7 +51,22 @@ Depending on your role, you can also use the following methods to manage or redu
- [Reduce container registry storage](packages/container_registry/reduce_container_registry_storage.md).
- [Reduce wiki repository size](../administration/wikis/index.md#reduce-wiki-repository-size).
-## Excess storage usage
+## Manage your transfer usage
+
+Depending on your role, to manage your transfer usage you can [reduce Container Registry data transfers](packages/container_registry/reduce_container_registry_data_transfer.md).
+
+## Project storage limit
+
+Projects on GitLab SaaS have a 10GB storage limit on their Git repository and LFS storage.
+After namespace-level storage limits are applied, the project limit will be removed. A namespace has either a namespace-level storage limit or a project-level storage limit, but not both.
+
+When a project's repository and LFS reaches the quota, the project is locked.
+You cannot push changes to a locked project. To monitor the size of each
+repository in a namespace, including a breakdown for each project,
+[view storage usage](#view-storage-usage). To allow a project's repository and LFS to exceed the free quota
+you must purchase additional storage. For more details, see [Excess storage usage](#excess-storage-usage).
+
+### Excess storage usage
Excess storage usage is the amount that a project's repository and LFS exceeds the [project storage limit](#project-storage-limit). If no
purchased storage is available the project is locked. You cannot push changes to a locked project.
@@ -112,7 +81,7 @@ The **Storage** tab of the **Usage Quotas** page warns you of the following:
- Projects that are locked because purchased storage available is zero. Locked projects are
marked with an information icon (**{information-o}**) beside their name.
-### Excess storage example
+#### Excess storage example
The following example describes an excess storage scenario for a namespace:
@@ -141,8 +110,34 @@ available decreases. All projects remain unlocked because 40 GB purchased storag
| Yellow | 5 GB | 0 GB | 10 GB | Not locked |
| **Totals** | **45 GB** | **10 GB** | - | - |
-## Manage your transfer usage
+## Namespace storage limit
-Depending on your role, you can use the following methods to manage or reduce your transfer:
+Namespaces on GitLab SaaS have a storage limit. For more information, see our [pricing page](https://about.gitlab.com/pricing/).
+This limit is not visible on the **Usage quotas** page, but will be prior to the limit being [applied](#namespace-storage-limit-application-schedule). Self-managed deployments are not affected.
+
+Storage types that add to the total namespace storage are:
+
+- Git repository
+- Git LFS
+- Artifacts
+- Container registry
+- Package registry
+- Dependency proxy
+- Wiki
+- Snippets
+
+If your total namespace storage exceeds the available namespace storage quota, all projects under the namespace are locked.
+A locked project cannot push to the repository, run pipelines and jobs, or build and push packages.
+
+To prevent exceeding the namespace storage quota, you can:
+
+- Reduce storage consumption by following the suggestions in the [Manage Your Storage Usage](#manage-your-storage-usage) section of this page.
+- Apply for [GitLab for Education](https://about.gitlab.com/solutions/education/join/), [GitLab for Open Source](https://about.gitlab.com/solutions/open-source/join/), or [GitLab for Startups](https://about.gitlab.com/solutions/startups/) if you meet the eligibility requirements.
+- Consider using a [self-managed instance](../subscriptions/self_managed/index.md) of GitLab which does not have these limits on the free tier.
+- [Purchase additional storage](../subscriptions/gitlab_com/index.md#purchase-more-storage-and-transfer) units at $60/year for 10GB of storage.
+- [Start a trial](https://about.gitlab.com/free-trial/) or [upgrade to GitLab Premium or Ultimate](https://about.gitlab.com/pricing/) which include higher limits and features that enable growing teams to ship faster without sacrificing on quality.
+- [Talk to an expert](https://page.gitlab.com/usage_limits_help.html) to learn more about your options and ask questions.
+
+### Namespace storage limit application schedule
-- [Reduce Container Registry data transfers](packages/container_registry/reduce_container_registry_data_transfer.md).
+Information on when namespace-level storage limits will be applied is available on these FAQ pages for the [Free](https://about.gitlab.com/pricing/faq-efficient-free-tier/#storage-limits-on-gitlab-saas-free-tier) and [Paid](https://about.gitlab.com/pricing/faq-paid-storage-transfer/) tier.
diff --git a/doc/user/workspace/index.md b/doc/user/workspace/index.md
index d7e014672aa..5bcd96cd4a5 100644
--- a/doc/user/workspace/index.md
+++ b/doc/user/workspace/index.md
@@ -15,7 +15,7 @@ The development, release, and timing of any products, features, or functionality
sole discretion of GitLab Inc.
NOTE:
-Workspace is currently in development.
+Workspace is in development.
Workspace will be above the [top-level namespaces](../namespace/index.md) for you to manage
everything you do as a GitLab administrator, including: