diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-07-19 17:16:28 +0300 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-07-19 17:16:28 +0300 |
| commit | e4384360a16dd9a19d4d2d25d0ef1f2b862ed2a6 (patch) | |
| tree | 2fcdfa7dcdb9db8f5208b2562f4b4e803d671243 /.gitlab | |
| parent | ffda4e7bcac36987f936b4ba515995a6698698f0 (diff) | |
Add latest changes from gitlab-org/gitlab@16-2-stable-eev16.2.0-rc42
Diffstat (limited to '.gitlab')
40 files changed, 951 insertions, 810 deletions
diff --git a/.gitlab/CODEOWNERS b/.gitlab/CODEOWNERS index 5db7ba5b109..38b60413db5 100644 --- a/.gitlab/CODEOWNERS +++ b/.gitlab/CODEOWNERS @@ -20,12 +20,10 @@ GITALY_SERVER_VERSION @project_278964_bot6 @gitlab-org/maintainers/rails-backend /doc/**/*.jpg /doc/**/*.png /data/deprecations/*.yml -/data/removals/**/*.yml ## Technical writing files that do not need `*` rule approval, ## but still require an approval from a TW team DRI for each file. /data/deprecations/templates/ @marcel.amirault @sarahgerman @gitlab-org/tw-leadership -/data/removals/templates/ @marcel.amirault @sarahgerman @gitlab-org/tw-leadership .markdownlint.yml @marcel.amirault @eread @aqualls @gitlab-org/tw-leadership /doc/.markdownlint/ @marcel.amirault @eread @aqualls @gitlab-org/tw-leadership /doc/.vale/ @marcel.amirault @eread @aqualls @gitlab-org/tw-leadership @@ -61,6 +59,10 @@ GITALY_SERVER_VERSION @project_278964_bot6 @gitlab-org/maintainers/rails-backend /ee/app/finders/ /rubocop/rubocop-migrations.yml +[Create::IDE - Remote Development Scripts] @gitlab-org/maintainers/remote-development/backend +# This entry must occur before `/scripts/` in order to be matched first +/scripts/remote_development/ + [Engineering Productivity] @gl-quality/eng-prod /.gitlab-ci.yml /.gitlab/ci/ @@ -74,6 +76,7 @@ GITALY_SERVER_VERSION @project_278964_bot6 @gitlab-org/maintainers/rails-backend /.gitlab/ci/test-on-gdk/ @gl-quality/eng-prod @gl-quality/qe-maintainers Dangerfile /danger/ +/gems/gem.gitlab-ci.yml /tooling/danger/ /scripts/ /scripts/frontend/ @gl-quality/eng-prod @gitlab-org/maintainers/frontend @@ -83,6 +86,7 @@ Dangerfile ^[Backend Static Code Analysis] @gl-quality/eng-prod @dstull @splattael .rubocop*.yml .rubocop_todo/ +/gems/config/rubocop.yml /rubocop/ /spec/rubocop/ @@ -392,29 +396,35 @@ lib/gitlab/checks/** ^[Documentation Directories] @gl-docsteam /doc/ /data/deprecations/ -/data/removals/ ^[Documentation Pages] # This block is managed by the rake script at lib/tasks/gitlab/tw/codeowners.rake, manual updates will be overwritten! # Begin rake-managed-docs-block +/doc/administration/analytics/ @lciutacu /doc/administration/application_settings_cache.md @jglassman1 -/doc/administration/audit_event_streaming.md @eread +/doc/administration/audit_event_streaming/ @eread /doc/administration/audit_events.md @eread /doc/administration/audit_reports.md @eread /doc/administration/auditor_users.md @jglassman1 /doc/administration/auth/ @jglassman1 +/doc/administration/backup_restore/ @axil /doc/administration/cicd.md @marcel.amirault /doc/administration/clusters/ @phillipwells /doc/administration/compliance.md @eread /doc/administration/configure.md @axil /doc/administration/consul.md @axil +/doc/administration/credentials_inventory.md @jglassman1 +/doc/administration/custom_project_templates.md @aqualls @msedlakjakubowski +/doc/administration/diff_limits.md @aqualls @msedlakjakubowski /doc/administration/docs_self_host.md @axil /doc/administration/encrypted_configuration.md @axil /doc/administration/environment_variables.md @axil /doc/administration/external_pipeline_validation.md @marcel.amirault +/doc/administration/external_users.md @jglassman1 /doc/administration/feature_flags.md @axil /doc/administration/file_hooks.md @eread @ashrafkhamis /doc/administration/geo/ @axil +/doc/administration/geo_sites.md @axil /doc/administration/get_started.md @kpaizee /doc/administration/git_protocol.md @aqualls @msedlakjakubowski /doc/administration/gitaly/ @eread @@ -432,14 +442,20 @@ lib/gitlab/checks/** /doc/administration/issue_closing_pattern.md @aqualls /doc/administration/job_artifacts.md @marcel.amirault /doc/administration/job_logs.md @fneill +/doc/administration/labels.md @msedlakjakubowski /doc/administration/lfs/ @aqualls @msedlakjakubowski /doc/administration/libravatar.md @axil +/doc/administration/license.md @fneill +/doc/administration/license_file.md @fneill /doc/administration/load_balancer.md @axil /doc/administration/logs/ @axil /doc/administration/logs/index.md @msedlakjakubowski /doc/administration/maintenance_mode/ @axil /doc/administration/merge_request_diffs.md @aqualls @msedlakjakubowski +/doc/administration/merge_requests_approvals.md @aqualls @msedlakjakubowski +/doc/administration/moderate_users.md @jglassman1 /doc/administration/monitoring/github_imports.md @eread @ashrafkhamis +/doc/administration/monitoring/health_check.md @msedlakjakubowski /doc/administration/monitoring/index.md @msedlakjakubowski /doc/administration/monitoring/ip_allowlist.md @jglassman1 /doc/administration/monitoring/performance/gitlab_configuration.md @msedlakjakubowski @@ -472,11 +488,36 @@ lib/gitlab/checks/** /doc/administration/reference_architectures/ @axil /doc/administration/reply_by_email.md @msedlakjakubowski /doc/administration/reply_by_email_postfix_setup.md @axil +/doc/administration/reporting/ @phillipwells +/doc/administration/reporting/spamcheck.md @axil /doc/administration/repository_checks.md @eread /doc/administration/repository_storage_paths.md @eread /doc/administration/repository_storage_types.md @eread /doc/administration/restart_gitlab.md @axil +/doc/administration/review_abuse_reports.md @phillipwells /doc/administration/server_hooks.md @eread +/doc/administration/settings/account_and_limit_settings.md @aqualls @msedlakjakubowski +/doc/administration/settings/deprecated_api_rate_limits.md @aqualls @msedlakjakubowski +/doc/administration/settings/git_lfs_rate_limits.md @aqualls @msedlakjakubowski +/doc/administration/settings/gitaly_timeouts.md @eread +/doc/administration/settings/import_export_rate_limits.md @eread @ashrafkhamis +/doc/administration/settings/incident_management_rate_limits.md @msedlakjakubowski +/doc/administration/settings/index.md @aqualls @msedlakjakubowski +/doc/administration/settings/instance_template_repository.md @aqualls @msedlakjakubowski +/doc/administration/settings/package_registry_rate_limits.md @phillipwells +/doc/administration/settings/project_integration_management.md @eread @ashrafkhamis +/doc/administration/settings/push_event_activities_limit.md @aqualls @msedlakjakubowski +/doc/administration/settings/rate_limit_on_issues_creation.md @msedlakjakubowski +/doc/administration/settings/rate_limit_on_notes_creation.md @msedlakjakubowski +/doc/administration/settings/rate_limit_on_pipelines_creation.md @marcel.amirault +/doc/administration/settings/rate_limit_on_projects_api.md @lciutacu +/doc/administration/settings/rate_limit_on_users_api.md @jglassman1 +/doc/administration/settings/rate_limits_on_git_ssh_operations.md @aqualls @msedlakjakubowski +/doc/administration/settings/scim_setup.md @jglassman1 +/doc/administration/settings/security_and_compliance.md @rdickenson +/doc/administration/settings/terraform_limits.md @phillipwells +/doc/administration/settings/third_party_offers.md @lciutacu +/doc/administration/settings/visibility_and_access_controls.md @aqualls @msedlakjakubowski /doc/administration/sidekiq/ @axil /doc/administration/sidekiq/sidekiq_memory_killer.md @jglassman1 /doc/administration/silent_mode/ @axil @@ -506,6 +547,7 @@ lib/gitlab/checks/** /doc/api/commits.md @aqualls @msedlakjakubowski /doc/api/container_registry.md @marcel.amirault /doc/api/custom_attributes.md @msedlakjakubowski +/doc/api/database_migrations.md @aqualls /doc/api/dependencies.md @rdickenson /doc/api/dependency_proxy.md @marcel.amirault /doc/api/deploy_keys.md @phillipwells @@ -595,7 +637,7 @@ lib/gitlab/checks/** /doc/api/project_aliases.md @aqualls @msedlakjakubowski /doc/api/project_badges.md @aqualls @msedlakjakubowski /doc/api/project_clusters.md @phillipwells -/doc/api/project_import_export.md @aqualls @msedlakjakubowski +/doc/api/project_import_export.md @eread @ashrafkhamis /doc/api/project_job_token_scopes.md @marcel.amirault /doc/api/project_level_variables.md @marcel.amirault /doc/api/project_relations_export.md @eread @ashrafkhamis @@ -690,12 +732,10 @@ lib/gitlab/checks/** /doc/development/distribution/ @axil /doc/development/documentation/ @sselhorn /doc/development/export_csv.md @eread @ashrafkhamis +/doc/development/fe_guide/ @sselhorn /doc/development/fe_guide/customizable_dashboards.md @lciutacu -/doc/development/fe_guide/dark_mode.md @sselhorn -/doc/development/fe_guide/graphql.md @sselhorn /doc/development/fe_guide/merge_request_widget_extensions.md @aqualls /doc/development/fe_guide/source_editor.md @aqualls @msedlakjakubowski -/doc/development/fe_guide/view_component.md @sselhorn /doc/development/feature_categorization/ @sselhorn /doc/development/feature_development.md @sselhorn /doc/development/feature_flags/ @sselhorn @@ -716,6 +756,7 @@ lib/gitlab/checks/** /doc/development/integrations/ @eread @ashrafkhamis /doc/development/integrations/secure.md @rdickenson /doc/development/integrations/secure_partner_integration.md @rdickenson +/doc/development/internal_analytics/ @lciutacu /doc/development/internal_api/ @aqualls @msedlakjakubowski /doc/development/internal_users.md @sselhorn /doc/development/issuable-like-models.md @msedlakjakubowski @@ -735,6 +776,7 @@ lib/gitlab/checks/** /doc/development/packages/cleanup_policies.md @marcel.amirault /doc/development/packages/dependency_proxy.md @marcel.amirault /doc/development/permissions.md @jglassman1 +/doc/development/permissions/ @jglassman1 /doc/development/policies.md @jglassman1 /doc/development/project_templates.md @aqualls @msedlakjakubowski /doc/development/prometheus_metrics.md @msedlakjakubowski @@ -744,8 +786,6 @@ lib/gitlab/checks/** /doc/development/search/ @ashrafkhamis /doc/development/sec/ @rdickenson /doc/development/secure_coding_guidelines.md @sselhorn -/doc/development/service_ping/ @lciutacu -/doc/development/snowplow/ @lciutacu /doc/development/spam_protection_and_captcha/ @phillipwells /doc/development/sql.md @aqualls /doc/development/testing_guide/ @sselhorn @@ -774,8 +814,6 @@ lib/gitlab/checks/** /doc/integration/mattermost/ @axil /doc/integration/partner_marketplace.md @fneill /doc/integration/recaptcha.md @phillipwells -/doc/integration/security_partners/ @rdickenson -/doc/integration/slash_commands.md @eread @ashrafkhamis /doc/integration/sourcegraph.md @aqualls @msedlakjakubowski /doc/integration/trello_power_up.md @eread @ashrafkhamis /doc/integration/vault.md @phillipwells @@ -784,8 +822,6 @@ lib/gitlab/checks/** /doc/operations/index.md @msedlakjakubowski /doc/policy/ @axil /doc/raketasks/ @axil -/doc/raketasks/generate_sample_prometheus_data.md @msedlakjakubowski -/doc/raketasks/migrate_snippets.md @ashrafkhamis /doc/raketasks/spdx.md @rdickenson /doc/raketasks/x509_signatures.md @aqualls @msedlakjakubowski /doc/security/ @jglassman1 @@ -803,53 +839,21 @@ lib/gitlab/checks/** /doc/tutorials/boards_for_teams/ @msedlakjakubowski /doc/tutorials/compliance_pipeline/ @eread /doc/tutorials/configure_gitlab_runner_to_use_gke/ @fneill +/doc/tutorials/container_scanning/ @rdickenson /doc/tutorials/convert_personal_namespace_to_group/ @lciutacu +/doc/tutorials/dependency_scanning.md @rdickenson /doc/tutorials/fuzz_testing/ @rdickenson /doc/tutorials/move_personal_project_to_group/ @lciutacu +/doc/tutorials/protected_workflow/ @aqualls /doc/tutorials/scan_result_policy/ @rdickenson /doc/update/ @axil /doc/update/background_migrations.md @aqualls -/doc/user/admin_area/analytics/ @lciutacu -/doc/user/admin_area/credentials_inventory.md @jglassman1 -/doc/user/admin_area/custom_project_templates.md @aqualls @msedlakjakubowski -/doc/user/admin_area/diff_limits.md @aqualls @msedlakjakubowski -/doc/user/admin_area/external_users.md @jglassman1 -/doc/user/admin_area/geo_sites.md @axil -/doc/user/admin_area/labels.md @msedlakjakubowski -/doc/user/admin_area/license.md @fneill -/doc/user/admin_area/license_file.md @fneill -/doc/user/admin_area/merge_requests_approvals.md @aqualls @msedlakjakubowski -/doc/user/admin_area/moderate_users.md @jglassman1 -/doc/user/admin_area/monitoring/ @msedlakjakubowski -/doc/user/admin_area/reporting/git_abuse_rate_limit.md @phillipwells -/doc/user/admin_area/reporting/spamcheck.md @axil -/doc/user/admin_area/review_abuse_reports.md @phillipwells -/doc/user/admin_area/settings/account_and_limit_settings.md @aqualls @msedlakjakubowski /doc/user/admin_area/settings/continuous_integration.md @marcel.amirault -/doc/user/admin_area/settings/deprecated_api_rate_limits.md @aqualls @msedlakjakubowski /doc/user/admin_area/settings/email.md @msedlakjakubowski /doc/user/admin_area/settings/external_authorization.md @jglassman1 /doc/user/admin_area/settings/files_api_rate_limits.md @aqualls @msedlakjakubowski -/doc/user/admin_area/settings/git_lfs_rate_limits.md @aqualls @msedlakjakubowski -/doc/user/admin_area/settings/gitaly_timeouts.md @eread -/doc/user/admin_area/settings/import_export_rate_limits.md @eread @ashrafkhamis -/doc/user/admin_area/settings/incident_management_rate_limits.md @msedlakjakubowski -/doc/user/admin_area/settings/index.md @aqualls @msedlakjakubowski -/doc/user/admin_area/settings/instance_template_repository.md @aqualls @msedlakjakubowski -/doc/user/admin_area/settings/package_registry_rate_limits.md @phillipwells -/doc/user/admin_area/settings/project_integration_management.md @eread @ashrafkhamis -/doc/user/admin_area/settings/push_event_activities_limit.md @aqualls @msedlakjakubowski -/doc/user/admin_area/settings/rate_limit_on_issues_creation.md @msedlakjakubowski -/doc/user/admin_area/settings/rate_limit_on_notes_creation.md @msedlakjakubowski -/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md @marcel.amirault -/doc/user/admin_area/settings/rate_limit_on_projects_api.md @lciutacu -/doc/user/admin_area/settings/rate_limit_on_users_api.md @jglassman1 -/doc/user/admin_area/settings/scim_setup.md @jglassman1 -/doc/user/admin_area/settings/security_and_compliance.md @rdickenson -/doc/user/admin_area/settings/terraform_limits.md @phillipwells -/doc/user/admin_area/settings/third_party_offers.md @lciutacu +/doc/user/admin_area/settings/slack_app.md @eread @ashrafkhamis /doc/user/admin_area/settings/usage_statistics.md @lciutacu -/doc/user/admin_area/settings/visibility_and_access_controls.md @aqualls @msedlakjakubowski /doc/user/analytics/ @lciutacu /doc/user/analytics/ci_cd_analytics.md @phillipwells /doc/user/application_security/ @rdickenson @@ -958,6 +962,7 @@ lib/gitlab/checks/** /doc/user/report_abuse.md @phillipwells /doc/user/reserved_names.md @lciutacu /doc/user/search/ @ashrafkhamis +/doc/user/search/command_palette.md @sselhorn /doc/user/shortcuts.md @ashrafkhamis /doc/user/snippets.md @aqualls @msedlakjakubowski /doc/user/ssh.md @jglassman1 @@ -1104,7 +1109,6 @@ lib/gitlab/checks/** /ee/app/assets/javascripts/access_tokens/ /ee/app/assets/javascripts/audit_events/components/tokens/ /ee/app/assets/javascripts/audit_events/token_utils.js -/ee/app/assets/javascripts/batch_comments/ /ee/app/assets/javascripts/groups/settings/components/ /ee/app/assets/javascripts/pages/admin/application_settings/general/components/ /ee/app/assets/javascripts/pages/groups/omniauth_callbacks/ @@ -1236,7 +1240,6 @@ lib/gitlab/checks/** /app/controllers/projects/triggers_controller.rb /app/controllers/projects/variables_controller.rb /app/models/commit_status.rb -/app/models/external_pull_request.rb /app/models/generic_commit_status.rb /app/models/namespace_ci_cd_setting.rb /app/models/project_ci_cd_setting.rb @@ -1449,6 +1452,37 @@ ee/lib/ee/api/entities/project.rb /ee/spec/elastic/migrate/ /ee/spec/support/elastic.rb +[Create::IDE - Remote Development Backend] @gitlab-org/maintainers/remote-development/backend +/ee/app/models/remote_development/ +/ee/app/policies/remote_development/ +/ee/app/finders/remote_development/ +/ee/app/graphql/types/remote_development/ +/ee/app/graphql/mutations/remote_development/ +/ee/app/graphql/resolvers/remote_development/ +/ee/app/controllers/remote_development/ +/ee/app/services/remote_development/ +/ee/spec/frontend/remote_development/ +/ee/spec/features/remote_development/ +/ee/spec/models/remote_development/ +/ee/spec/policies/remote_development/ +/ee/spec/requests/api/graphql/mutations/remote_development/ +/ee/spec/requests/api/graphql/remote_development/ +/ee/spec/finders/remote_development/ +/ee/spec/support/shared_contexts/remote_development/ +/ee/spec/graphql/types/remote_development/ +/ee/spec/factories/remote_development/ +/ee/spec/lib/remote_development/ +/ee/spec/fixtures/remote_development/ +/ee/spec/controllers/remote_development/ +/ee/spec/services/remote_development/ +/ee/lib/remote_development/ +/qa/qa/specs/features/ee/browser_ui/3_create/remote_development/ + +[Create::IDE - Remote Development Frontend] @gitlab-org/maintainers/remote-development/frontend +/ee/app/views/remote_development/ +/ee/app/assets/javascripts/remote_development/ +/ee/app/assets/javascripts/pages/remote_development/ + # JiHu GitLab rules. See https://gitlab.com/gitlab-jh/gitlab-jh-enablement/-/issues/213#note_1024367528 [JH Frontend] @jihulab/maintainers/frontend diff --git a/.gitlab/ci/benchmark.gitlab-ci.yml b/.gitlab/ci/benchmark.gitlab-ci.yml new file mode 100644 index 00000000000..5949a9cd6a9 --- /dev/null +++ b/.gitlab/ci/benchmark.gitlab-ci.yml @@ -0,0 +1,17 @@ +benchmark-markdown: + extends: + - .single-db-ci-connection + - .rails-job-base + - .use-pg14 + - .benchmark:rules:benchmark-markdown + stage: benchmark + needs: + - setup-test-env + script: + - section_start "gitaly-test-spawn" "Spawning Gitaly"; scripts/gitaly-test-spawn; section_end "gitaly-test-spawn"; # Do not use 'bundle exec' here + - bundle exec rake benchmark:banzai &> benchmark-markdown.txt + artifacts: + when: always + paths: + - benchmark-markdown.txt + allow_failure: true diff --git a/.gitlab/ci/caching.gitlab-ci.yml b/.gitlab/ci/caching.gitlab-ci.yml index ae4bb94b301..96c2b0d9322 100644 --- a/.gitlab/ci/caching.gitlab-ci.yml +++ b/.gitlab/ci/caching.gitlab-ci.yml @@ -45,6 +45,7 @@ cache-assets:test as-if-foss: extends: - .cache-assets-base - .as-if-foss + - .caching:rules:cache-assets-as-if-foss cache-assets:production: extends: diff --git a/.gitlab/ci/docs.gitlab-ci.yml b/.gitlab/ci/docs.gitlab-ci.yml index 6c426749af3..6ad916d919a 100644 --- a/.gitlab/ci/docs.gitlab-ci.yml +++ b/.gitlab/ci/docs.gitlab-ci.yml @@ -42,7 +42,7 @@ review-docs-cleanup: docs-lint links: extends: - .docs:rules:docs-lint - image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-docs/lint-html:alpine-3.17-ruby-3.2.2-c24946ab + image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-docs/lint-html:alpine-3.18-ruby-3.2.2-4207821e stage: lint needs: [] script: @@ -52,13 +52,13 @@ docs-lint links: - mv doc/ /tmp/gitlab-docs/content/ee - cd /tmp/gitlab-docs # Build HTML from Markdown - - bundle exec nanoc + - make compile # Check the internal links and anchors (in parallel) - "parallel time bundle exec nanoc check ::: internal_links internal_anchors" .docs-markdown-lint-image: # When updating the image version here, update it in /scripts/lint-doc.sh too. - image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-docs/lint-markdown:alpine-3.17-vale-2.24.0-markdownlint-0.33.0-markdownlint2-0.6.0 + image: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-docs/lint-markdown:alpine-3.18-vale-2.27.0-markdownlint-0.35.0-markdownlint2-0.8.1 docs-lint markdown: extends: @@ -123,4 +123,3 @@ docs-lint deprecations-and-removals: needs: [] script: - bundle exec rake gitlab:docs:check_deprecations - - bundle exec rake gitlab:docs:check_removals diff --git a/.gitlab/ci/frontend.gitlab-ci.yml b/.gitlab/ci/frontend.gitlab-ci.yml index 34c29a9e902..279a7067a74 100644 --- a/.gitlab/ci/frontend.gitlab-ci.yml +++ b/.gitlab/ci/frontend.gitlab-ci.yml @@ -371,28 +371,6 @@ bundle-size-review: paths: - bundle-size-review/ -.startup-css-check-base: - extends: - - .frontend-test-base - script: - - run_timed_command "yarn generate:startup_css" - - yarn check:startup_css - -startup-css-check: - extends: - - .startup-css-check-base - - .frontend:rules:default-frontend-jobs - needs: ["compile-test-assets", "rspec-all frontend_fixture"] - -startup-css-check as-if-foss: - extends: - - .startup-css-check-base - - .as-if-foss - - .frontend:rules:default-frontend-jobs-as-if-foss - needs: - - job: "compile-test-assets as-if-foss" - - job: "rspec-all frontend_fixture as-if-foss" - .compile-storybook-base: extends: - .frontend-test-base diff --git a/.gitlab/ci/gitlab-gems.gitlab-ci.yml b/.gitlab/ci/gitlab-gems.gitlab-ci.yml new file mode 100644 index 00000000000..eb0c8b8536e --- /dev/null +++ b/.gitlab/ci/gitlab-gems.gitlab-ci.yml @@ -0,0 +1,22 @@ +include: + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "activerecord-gitlab" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "click_house-client" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "gitlab-rspec" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "gitlab-schema-validation" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "gitlab-utils" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "ipynbdiff" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "rspec_flaky" diff --git a/.gitlab/ci/global.gitlab-ci.yml b/.gitlab/ci/global.gitlab-ci.yml index b5578c25518..c501d930352 100644 --- a/.gitlab/ci/global.gitlab-ci.yml +++ b/.gitlab/ci/global.gitlab-ci.yml @@ -213,296 +213,192 @@ - *node-modules-cache # We don't push this cache as it's already rebuilt by `update-assets-compile-*-cache` - *storybook-node-modules-cache-push -.use-pg12: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-12-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine +.pg-base-variables: variables: POSTGRES_HOST_AUTH_METHOD: trust - PG_VERSION: "12" -.use-pg13: +.db-services: services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] + - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-${PG_VERSION}-pgvector-0.4.1 + command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off", "-c", "max_locks_per_transaction=256"] alias: postgres - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 alias: rediscluster # configure connections in config/redis.yml + +.db-services-with-redis-6: + services: + - !reference [.db-services, services] - name: redis:6.2-alpine + +.db-services-with-redis-7: + services: + - !reference [.db-services, services] + - name: redis:7.0-alpine + +.use-pg12: + extends: + - .pg-base-variables + - .db-services-with-redis-6 + variables: + PG_VERSION: "12" + +.use-pg13: + extends: + - .pg-base-variables + - .db-services-with-redis-6 variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "13" .use-pg14: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.2-alpine + extends: + - .pg-base-variables + - .db-services-with-redis-6 variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "14" .use-pg15: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-15-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.2-alpine + extends: + - .pg-base-variables + - .db-services-with-redis-7 variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "15" -.use-pg12-es7-ee: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-12-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine - - name: elasticsearch:7.17.6 - command: ["elasticsearch", "-E", "discovery.type=single-node", "-E", "xpack.security.enabled=false"] - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image +.zoekt-variables: variables: - POSTGRES_HOST_AUTH_METHOD: trust - PG_VERSION: "12" ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 -.use-pg13-es7-ee: +.zoekt-services: services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.2-alpine - - name: elasticsearch:7.17.6 - command: ["elasticsearch", "-E", "discovery.type=single-node", "-E", "xpack.security.enabled=false"] - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 alias: zoekt-ci-image - variables: - POSTGRES_HOST_AUTH_METHOD: trust - PG_VERSION: "13" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 -.use-pg14-es7-ee: +.es7-base: + extends: + - .pg-base-variables + - .zoekt-variables services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.2-alpine + - !reference [.db-services-with-redis-6, services] + - !reference [.zoekt-services, services] - name: elasticsearch:7.17.6 command: ["elasticsearch", "-E", "discovery.type=single-node", "-E", "xpack.security.enabled=false"] - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image + +.use-pg12-es7-ee: + extends: .es7-base + variables: + PG_VERSION: "12" + +.use-pg13-es7-ee: + extends: .es7-base + variables: + PG_VERSION: "13" + +.use-pg14-es7-ee: + extends: .es7-base variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "14" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 .use-pg15-es7-ee: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-15-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.2-alpine - - name: elasticsearch:7.17.6 - command: ["elasticsearch", "-E", "discovery.type=single-node", "-E", "xpack.security.enabled=false"] - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image + extends: .es7-base variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "15" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 -.use-pg13-es8-ee: +.es8-base: + extends: + - .pg-base-variables + - .zoekt-variables services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine + - !reference [.db-services-with-redis-6, services] + - !reference [.zoekt-services, services] - name: elasticsearch:8.6.2 - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image variables: - POSTGRES_HOST_AUTH_METHOD: trust - PG_VERSION: "13" ES_SETTING_DISCOVERY_TYPE: "single-node" ES_SETTING_XPACK_SECURITY_ENABLED: "false" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 + +.use-pg13-es8-ee: + extends: .es8-base + variables: + PG_VERSION: "13" .use-pg14-es8-ee: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine - - name: elasticsearch:8.6.2 - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image + extends: .es8-base variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "14" - ES_SETTING_DISCOVERY_TYPE: "single-node" - ES_SETTING_XPACK_SECURITY_ENABLED: "false" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 .use-pg15-es8-ee: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-15-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine - - name: elasticsearch:8.6.2 - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image + extends: .es8-base variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "15" - ES_SETTING_DISCOVERY_TYPE: "single-node" - ES_SETTING_XPACK_SECURITY_ENABLED: "false" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 -.use-pg13-opensearch1-ee: +.os1-base: + extends: + - .pg-base-variables + - .zoekt-variables services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine + - !reference [.db-services-with-redis-6, services] + - !reference [.zoekt-services, services] - name: opensearchproject/opensearch:1.3.5 alias: elasticsearch command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"] - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image - variables: - POSTGRES_HOST_AUTH_METHOD: trust - PG_VERSION: "13" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 -.use-pg13-opensearch2-ee: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-13-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine - - name: opensearchproject/opensearch:2.2.1 - alias: elasticsearch - command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"] - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image +.use-pg13-opensearch1-ee: + extends: .os1-base variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "13" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 .use-pg14-opensearch1-ee: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine - - name: opensearchproject/opensearch:1.3.5 - alias: elasticsearch - command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"] - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image + extends: .os1-base variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "14" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 -.use-pg14-opensearch2-ee: +.use-pg15-opensearch1-ee: + extends: .os1-base + variables: + PG_VERSION: "15" + +.os2-base: + extends: + - .pg-base-variables + - .zoekt-variables services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine + - !reference [.db-services-with-redis-6, services] + - !reference [.zoekt-services, services] - name: opensearchproject/opensearch:2.2.1 alias: elasticsearch command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"] - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image + +.use-pg13-opensearch2-ee: + extends: .os2-base + variables: + PG_VERSION: "13" + +.use-pg14-opensearch2-ee: + extends: .os2-base variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "14" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 -.use-pg15-opensearch1-ee: - services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-15-pgvector-0.4.1 - command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] - alias: postgres - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 - alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine - - name: opensearchproject/opensearch:1.3.5 - alias: elasticsearch - command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"] - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image +.use-pg15-opensearch2-ee: + extends: .os2-base variables: - POSTGRES_HOST_AUTH_METHOD: trust PG_VERSION: "15" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 -.use-pg15-opensearch2-ee: +.use-pg14-clickhouse23: services: - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-15-pgvector-0.4.1 + - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:postgres-14-pgvector-0.4.1 command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] alias: postgres - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:redis-cluster-6.2.12 alias: rediscluster # configure connections in config/redis.yml - - name: redis:6.0-alpine - - name: opensearchproject/opensearch:2.2.1 - alias: elasticsearch - command: ["bin/opensearch", "-E", "discovery.type=single-node", "-E", "plugins.security.disabled=true"] - - name: ${REGISTRY_HOST}/${REGISTRY_GROUP}/gitlab-build-images:zoekt-ci-image-1.1 - alias: zoekt-ci-image + - name: redis:6.2-alpine + - name: clickhouse/clickhouse-server:23-alpine + alias: clickhouse variables: POSTGRES_HOST_AUTH_METHOD: trust - PG_VERSION: "15" - ZOEKT_INDEX_BASE_URL: http://zoekt-ci-image:6060 - ZOEKT_SEARCH_BASE_URL: http://zoekt-ci-image:6070 + PG_VERSION: "14" + CLICKHOUSE_USER: clickhouse + CLICKHOUSE_PASSWORD: clickhouse + CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1 + CLICKHOUSE_DB: gitlab_clickhouse_test .use-kaniko: image: diff --git a/.gitlab/ci/package-and-test-nightly/main.gitlab-ci.yml b/.gitlab/ci/package-and-test-nightly/main.gitlab-ci.yml index 34b4e853415..bb0de4a79e2 100644 --- a/.gitlab/ci/package-and-test-nightly/main.gitlab-ci.yml +++ b/.gitlab/ci/package-and-test-nightly/main.gitlab-ci.yml @@ -2,6 +2,16 @@ include: - local: .gitlab/ci/qa-common/main.gitlab-ci.yml - local: .gitlab/ci/qa-common/rules.gitlab-ci.yml - local: .gitlab/ci/qa-common/variables.gitlab-ci.yml + - component: "gitlab.com/gitlab-org/quality/pipeline-common/allure-report@7.3.0" + inputs: + job_name: "e2e-test-report" + job_stage: "report" + aws_access_key_id_variable_name: "QA_ALLURE_AWS_ACCESS_KEY_ID" + aws_secret_access_key_variable_name: "QA_ALLURE_AWS_SECRET_ACCESS_KEY" + gitlab_auth_token_variable_name: "PROJECT_TOKEN_FOR_CI_SCRIPTS_API_USAGE" + allure_results_glob: "gitlab-qa-run-*/**/allure-results" + allure_job_name: "${QA_RUN_TYPE}" + allure_ref_slug: "${CI_COMMIT_REF_SLUG}" workflow: rules: @@ -97,12 +107,22 @@ integration-opensearch-compatibility-version-2: OPENSEARCH_VERSION: "2.0.1" QA_SCENARIO: "Test::Integration::Opensearch" +# ------------------------------------------ +# Relative url +# ------------------------------------------ +relative-url: + extends: + - .qa + - .ee + - .parallel + variables: + QA_SCENARIO: Test::Instance::RelativeUrl + # ========================================== # Post test stage # ========================================== e2e-test-report: - extends: - - .e2e-test-report + extends: .rules:report:allure-report upload-knapsack-report: extends: diff --git a/.gitlab/ci/package-and-test/main.gitlab-ci.yml b/.gitlab/ci/package-and-test/main.gitlab-ci.yml index fcea49e3103..45e07ccf659 100644 --- a/.gitlab/ci/package-and-test/main.gitlab-ci.yml +++ b/.gitlab/ci/package-and-test/main.gitlab-ci.yml @@ -4,6 +4,16 @@ include: - local: .gitlab/ci/qa-common/main.gitlab-ci.yml - local: .gitlab/ci/qa-common/rules.gitlab-ci.yml - local: .gitlab/ci/qa-common/variables.gitlab-ci.yml + - component: "gitlab.com/gitlab-org/quality/pipeline-common/allure-report@7.3.1" + inputs: + job_name: "e2e-test-report" + job_stage: "report" + aws_access_key_id_variable_name: "QA_ALLURE_AWS_ACCESS_KEY_ID" + aws_secret_access_key_variable_name: "QA_ALLURE_AWS_SECRET_ACCESS_KEY" + gitlab_auth_token_variable_name: "PROJECT_TOKEN_FOR_CI_SCRIPTS_API_USAGE" + allure_results_glob: "gitlab-qa-run-*/**/allure-results" + allure_job_name: "${QA_RUN_TYPE}" + allure_ref_slug: "${CI_COMMIT_REF_SLUG}" # ========================================== # Prepare stage @@ -81,6 +91,7 @@ instance-ff-inverse: # ------------------------------------------ # ========== instance =========== + instance: extends: - .parallel @@ -151,38 +162,6 @@ praefect-selective-parallel: variables: QA_TESTS: "" -# ========== relative-url =========== - -relative-url: - extends: - - .qa - - .parallel - variables: - QA_SCENARIO: Test::Instance::RelativeUrl - rules: - - !reference [.rules:test:qa-parallel, rules] - - if: $QA_SUITES =~ /Test::Instance::All/ - -relative-url-selective: - extends: .qa - variables: - QA_SCENARIO: Test::Instance::RelativeUrl - rules: - - !reference [.rules:test:qa-selective, rules] - - if: $QA_SUITES =~ /Test::Instance::All/ - -relative-url-selective-parallel: - extends: - - .qa - - .parallel - variables: - QA_SCENARIO: Test::Instance::RelativeUrl - rules: - - !reference [.rules:test:qa-selective-parallel, rules] - - if: $QA_SUITES =~ /Test::Instance::All/ - variables: - QA_TESTS: "" - # ========== decomposition-single-db =========== decomposition-single-db: @@ -259,7 +238,9 @@ decomposition-multiple-db-selective-parallel: # ========== object-storage =========== object-storage: - extends: .qa + extends: + - .qa + - .failure-videos parallel: 2 variables: QA_SCENARIO: Test::Instance::Image @@ -295,7 +276,9 @@ object-storage-selective-parallel: # ========== object-storage-aws =========== object-storage-aws: - extends: object-storage + extends: + - object-storage + - .failure-videos variables: AWS_S3_ACCESS_KEY: $QA_AWS_S3_ACCESS_KEY AWS_S3_BUCKET_NAME: $QA_AWS_S3_BUCKET_NAME @@ -325,7 +308,9 @@ object-storage-aws-selective-parallel: # ========== object-storage-gcs =========== object-storage-gcs: - extends: object-storage + extends: + - object-storage + - .failure-videos variables: GCS_BUCKET_NAME: $QA_GCS_BUCKET_NAME GOOGLE_PROJECT: $QA_GOOGLE_PROJECT @@ -383,7 +368,9 @@ group-saml: - !reference [.rules:test:manual, rules] oauth: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::OAuth rules: @@ -401,7 +388,9 @@ instance-saml: - !reference [.rules:test:manual, rules] jira: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::Jira rules: @@ -419,7 +408,9 @@ integrations: - !reference [.rules:test:manual, rules] ldap-no-server: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::LDAPNoServer rules: @@ -428,7 +419,9 @@ ldap-no-server: - !reference [.rules:test:manual, rules] ldap-tls: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::LDAPTLS rules: @@ -437,7 +430,9 @@ ldap-tls: - !reference [.rules:test:manual, rules] ldap-no-tls: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::LDAPNoTLS rules: @@ -455,7 +450,9 @@ mtls: - !reference [.rules:test:manual, rules] mattermost: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::Mattermost rules: @@ -464,7 +461,9 @@ mattermost: - !reference [.rules:test:manual, rules] registry: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::Registry rules: @@ -473,7 +472,9 @@ registry: - !reference [.rules:test:manual, rules] registry-with-cdn: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::RegistryWithCDN GCS_CDN_BUCKET_NAME: $QA_GCS_CDN_BUCKET_NAME @@ -490,7 +491,9 @@ registry-with-cdn: - !reference [.rules:test:manual, rules] repository-storage: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Instance::RepositoryStorage rules: @@ -499,7 +502,9 @@ repository-storage: - !reference [.rules:test:manual, rules] service-ping-disabled: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::ServicePingDisabled rules: @@ -508,7 +513,9 @@ service-ping-disabled: - !reference [.rules:test:manual, rules] smtp: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::SMTP rules: @@ -517,7 +524,9 @@ smtp: - !reference [.rules:test:manual, rules] cloud-activation: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Instance::Image QA_RSPEC_TAGS: --tag cloud_activation @@ -527,7 +536,9 @@ cloud-activation: - !reference [.rules:test:manual, rules] large-setup: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Instance::Image QA_RSPEC_TAGS: --tag can_use_large_setup @@ -566,7 +577,9 @@ registry-object-storage-tls: GITLAB_QA_OPTS: --omnibus-config registry_object_storage $EXTRA_GITLAB_QA_OPTS importers: - extends: .qa + extends: + - .qa + - .failure-videos variables: QA_SCENARIO: Test::Integration::Import QA_MOCK_GITHUB: "true" @@ -620,9 +633,7 @@ update-ee-to-ce: # Post test stage # ========================================== e2e-test-report: - extends: - - .e2e-test-report - - .rules:report:allure-report + extends: .rules:report:allure-report upload-knapsack-report: extends: diff --git a/.gitlab/ci/preflight.gitlab-ci.yml b/.gitlab/ci/preflight.gitlab-ci.yml index 968402b2ea5..6a2ea85f393 100644 --- a/.gitlab/ci/preflight.gitlab-ci.yml +++ b/.gitlab/ci/preflight.gitlab-ci.yml @@ -24,9 +24,6 @@ - .ruby-cache - .preflight:rules:rails-production-server-boot - .use-pg13 - variables: - BUNDLE_WITHOUT: "development:test" - BUNDLE_WITH: "production" # Test the puma configuration present in `config/puma.rb.example` rails-production-server-boot-puma-example: diff --git a/.gitlab/ci/qa-common/main.gitlab-ci.yml b/.gitlab/ci/qa-common/main.gitlab-ci.yml index 3b6f42ba57e..603ac12c464 100644 --- a/.gitlab/ci/qa-common/main.gitlab-ci.yml +++ b/.gitlab/ci/qa-common/main.gitlab-ci.yml @@ -9,7 +9,6 @@ include: ref: 7.2.3 file: - /ci/base.gitlab-ci.yml - - /ci/allure-report.yml - /ci/knapsack-report.yml stages: @@ -86,15 +85,6 @@ stages: - qa/knapsack/*.json expire_in: 1 day -.e2e-test-report: - extends: - - .generate-allure-report-base - stage: report - variables: - GITLAB_AUTH_TOKEN: $PROJECT_TOKEN_FOR_CI_SCRIPTS_API_USAGE - ALLURE_PROJECT_PATH: $CI_PROJECT_PATH - ALLURE_MERGE_REQUEST_IID: $CI_MERGE_REQUEST_IID - .upload-knapsack-report: extends: - .generate-knapsack-report-base @@ -148,6 +138,7 @@ stages: STATUS_SYM: ☠️ STATUS: failed TYPE: "($QA_RUN_TYPE) " + ALLURE_JOB_NAME: $QA_RUN_TYPE when: always script: - | @@ -158,6 +149,12 @@ stages: - bundle exec prepare-stage-reports --input-files "${QA_RSPEC_XML_FILE_PATTERN}" - !reference [.notify-slack-qa, script] +.failure-videos: + variables: + USE_SELENOID: "true" + QA_RECORD_VIDEO: "true" + QA_REMOTE_GRID: "selenoid:4444" + # ========================================== # Pre stage # ========================================== diff --git a/.gitlab/ci/qa-common/variables.gitlab-ci.yml b/.gitlab/ci/qa-common/variables.gitlab-ci.yml index 817f2e0020a..fe980293f3a 100644 --- a/.gitlab/ci/qa-common/variables.gitlab-ci.yml +++ b/.gitlab/ci/qa-common/variables.gitlab-ci.yml @@ -3,12 +3,6 @@ variables: REGISTRY_HOST: "registry.gitlab.com" REGISTRY_GROUP: "gitlab-org" - SKIP_OMNIBUS_TRIGGER: "true" - OMNIBUS_GITLAB_CACHE_UPDATE: "false" - OMNIBUS_GITLAB_RUBY3_BUILD: "false" - OMNIBUS_GITLAB_RUBY2_BUILD: "false" - OMNIBUS_GITLAB_CACHE_EDITION: "GITLAB" - OMNIBUS_GITLAB_BUILD_ON_ALL_OS: "false" ALLURE_JOB_NAME: $CI_PROJECT_NAME COLORIZED_LOGS: "true" QA_LOG_LEVEL: "info" @@ -17,3 +11,4 @@ variables: # run all tests by default when package-and-test is included natively in other projects # this will be overridden when selective test execution is used in gitlab canonical project QA_RUN_ALL_TESTS: "true" + USE_OLD_RUBY_VERSION: "true" diff --git a/.gitlab/ci/qa.gitlab-ci.yml b/.gitlab/ci/qa.gitlab-ci.yml index 6d181a59214..eae3b2f2a36 100644 --- a/.gitlab/ci/qa.gitlab-ci.yml +++ b/.gitlab/ci/qa.gitlab-ci.yml @@ -15,10 +15,6 @@ .e2e-trigger-base: extends: .production # this makes sure GITLAB_ALLOW_SEPARATE_CI_DATABASE is passed to the child pipeline stage: qa - needs: - - build-assets-image - - build-qa-image - - e2e-test-pipeline-generate variables: # This is needed by `trigger-omnibus-env` (`.gitlab/ci/package-and-test/main.gitlab-ci.yml`). PARENT_PIPELINE_ID: $CI_PIPELINE_ID @@ -27,7 +23,6 @@ RUN_WITH_BUNDLE: "true" # instructs pipeline to install and run gitlab-qa gem via bundler QA_PATH: qa # sets the optional path for bundler to run from DYNAMIC_PIPELINE_YML: package-and-test-pipeline.yml # yml files are generated by scripts/generate-e2e-pipeline script - EXTRA_GITLAB_QA_OPTS: --set-feature-flags super_sidebar_nav_enrolled=enabled inherit: variables: - CHROME_VERSION @@ -36,9 +31,6 @@ - DEBIAN_VERSION - REGISTRY_GROUP - REGISTRY_HOST - - OMNIBUS_GITLAB_CACHE_EDITION - - OMNIBUS_GITLAB_RUBY3_BUILD - - OMNIBUS_GITLAB_RUBY2_BUILD trigger: strategy: depend forward: @@ -102,9 +94,9 @@ trigger-omnibus: TOP_UPSTREAM_SOURCE_PROJECT: $CI_PROJECT_PATH SECURITY_SOURCES: $SECURITY_SOURCES CACHE_UPDATE: $OMNIBUS_GITLAB_CACHE_UPDATE - RUBY3_BUILD: $OMNIBUS_GITLAB_RUBY3_BUILD - RUBY2_BUILD: $OMNIBUS_GITLAB_RUBY2_BUILD - CACHE_EDITION: $OMNIBUS_GITLAB_CACHE_EDITION + USE_OLD_RUBY_VERSION: "true" + CACHE_EDITION: "GITLAB_RUBY3_0" + NEXT_RUBY_VERSION: "3.1.4" BUILD_ON_ALL_OS: $OMNIBUS_GITLAB_BUILD_ON_ALL_OS SKIP_QA_TEST: "true" ee: $EE @@ -135,7 +127,6 @@ e2e:package-and-test-ee: variables: RELEASE: "${REGISTRY_HOST}/${REGISTRY_GROUP}/build/omnibus-gitlab-mirror/gitlab-ee:${CI_COMMIT_SHA}" QA_RUN_TYPE: e2e-package-and-test - ALLURE_JOB_NAME: e2e-package-and-test PIPELINE_NAME: E2E Omnibus GitLab EE e2e:package-and-test-ce: @@ -151,17 +142,19 @@ e2e:package-and-test-ce: RELEASE: ${REGISTRY_HOST}/${REGISTRY_GROUP}/build/omnibus-gitlab-mirror/gitlab-ce:${CI_COMMIT_SHA} GITLAB_QA_IMAGE: ${CI_REGISTRY_IMAGE}/gitlab-ce-qa:${CI_COMMIT_SHA} QA_RUN_TYPE: e2e-package-and-test-ce - ALLURE_JOB_NAME: e2e-package-and-test-ce PIPELINE_NAME: E2E Omnibus GitLab CE e2e:package-and-test-old-nav: extends: - .e2e-trigger-base - .qa:rules:package-and-test-old-nav + needs: + - build-qa-image + - trigger-omnibus + - e2e-test-pipeline-generate variables: RELEASE: "${REGISTRY_HOST}/${REGISTRY_GROUP}/build/omnibus-gitlab-mirror/gitlab-ee:${CI_COMMIT_SHA}" QA_RUN_TYPE: e2e-package-and-test-old-nav - ALLURE_JOB_NAME: e2e-package-and-test-old-nav PIPELINE_NAME: E2E Omnibus Old Navigation QA_SUPER_SIDEBAR_ENABLED: "false" EXTRA_GITLAB_QA_OPTS: "" @@ -177,7 +170,6 @@ e2e:package-and-test-nightly: variables: GITLAB_SEMVER_VERSION: $GITLAB_SEMVER_VERSION QA_RUN_TYPE: nightly - ALLURE_JOB_NAME: nightly PIPELINE_NAME: E2E Omnibus GitLab Nightly DYNAMIC_PIPELINE_YML: package-and-test-nightly-pipeline.yml @@ -190,7 +182,6 @@ e2e:test-on-gdk: - build-gdk-image - e2e-test-pipeline-generate variables: - ALLURE_JOB_NAME: e2e-test-on-gdk QA_RUN_TYPE: e2e-test-on-gdk PIPELINE_NAME: E2E GDK DYNAMIC_PIPELINE_YML: test-on-gdk-pipeline.yml diff --git a/.gitlab/ci/rails.gitlab-ci.yml b/.gitlab/ci/rails.gitlab-ci.yml index 0df6ef0cacd..9338f5cf7e5 100644 --- a/.gitlab/ci/rails.gitlab-ci.yml +++ b/.gitlab/ci/rails.gitlab-ci.yml @@ -115,6 +115,18 @@ rspec background_migration pg14 praefect: - .praefect-with-db - .rails:rules:praefect-with-db +rspec migration pg14 clusterwide-db: + extends: + - rspec migration pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db + +rspec background_migration pg14 clusterwide-db: + extends: + - rspec background_migration pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db + rspec unit pg14: extends: - .rspec-base-pg14 @@ -139,6 +151,12 @@ rspec unit pg14 single-db-ci-connection: - .single-db-ci-connection-rspec - .rails:rules:single-db-ci-connection +rspec unit pg14 clusterwide-db: + extends: + - rspec unit pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db + rspec unit pg14 praefect: extends: - rspec unit pg14 @@ -169,6 +187,12 @@ rspec integration pg14 single-db-ci-connection: - .single-db-ci-connection-rspec - .rails:rules:single-db-ci-connection +rspec integration pg14 clusterwide-db: + extends: + - rspec integration pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db + rspec integration pg14 praefect: extends: - rspec integration pg14 @@ -201,6 +225,12 @@ rspec system pg14 single-db-ci-connection: - .single-db-ci-connection-rspec - .rails:rules:single-db-ci-connection +rspec system pg14 clusterwide-db: + extends: + - rspec system pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db + rspec system pg14 praefect: extends: - rspec system pg14 @@ -236,6 +266,11 @@ rspec fast_spec_helper: # Load fast_spec_helper as well just in case there are no specs available. - bin/rspec --dry-run spec/fast_spec_helper.rb $fast_spec_helper_specs +rspec unit clickhouse: + extends: + - .rspec-base-pg14-clickhouse23 + - .rails:rules:clickhouse-changes + gitlab:setup: extends: .db-job-base variables: @@ -275,7 +310,7 @@ rspec:deprecations: script: - grep -h -R "keyword" deprecations/ | awk '{$1=$1};1' | sort | uniq -c | sort - grep -R "keyword" deprecations/ | wc - - run_timed_command "fail_on_warnings bundle exec rubocop --only Lint/LastKeywordArgument --parallel" + - run_timed_command "fail_on_warnings bundle exec rubocop --config .rubocop.yml --only Lint/LastKeywordArgument --parallel" artifacts: expire_in: 31d when: always @@ -326,6 +361,10 @@ rspec:artifact-collector remainder: optional: true - job: rspec background_migration pg14 # 4 jobs optional: true + rules: + - !reference ['.rails:rules:ee-and-foss-integration', rules] + - !reference ['.rails:rules:ee-and-foss-migration', rules] + - !reference ['.rails:rules:ee-and-foss-background-migration', rules] rspec:artifact-collector as-if-foss unit: extends: @@ -351,6 +390,10 @@ rspec:artifact-collector as-if-foss remainder: optional: true - job: rspec background_migration pg14-as-if-foss # 4 jobs optional: true + rules: + - !reference ['.rails:rules:as-if-foss-integration', rules] + - !reference ['.rails:rules:as-if-foss-migration', rules] + - !reference ['.rails:rules:as-if-foss-background-migration', rules] rspec:artifact-collector single-redis: extends: @@ -370,6 +413,7 @@ rspec:artifact-collector system single-redis: rspec:artifact-collector ee single-redis: extends: - .artifact-collector + - .rails:rules:single-redis needs: - job: rspec-ee unit pg14 single-redis # 18 jobs optional: true @@ -392,6 +436,12 @@ rspec:artifact-collector ee: optional: true - job: rspec-ee system pg14 # 10 jobs optional: true + rules: + - !reference ['.rails:rules:ee-only-migration', rules] + - !reference ['.rails:rules:ee-only-background-migration', rules] + - !reference ['.rails:rules:ee-only-unit', rules] + - !reference ['.rails:rules:ee-only-integration', rules] + - !reference ['.rails:rules:ee-only-system', rules] rspec:coverage: extends: @@ -569,25 +619,37 @@ rspec migration pg14-as-if-foss single-db: extends: - rspec migration pg14-as-if-foss - .single-db-rspec - - .rails:rules:single-db + - .rails:rules:single-db-as-if-foss rspec background_migration pg14-as-if-foss single-db: extends: - rspec background_migration pg14-as-if-foss - .single-db-rspec - - .rails:rules:single-db + - .rails:rules:single-db-as-if-foss rspec migration pg14-as-if-foss single-db-ci-connection: extends: - rspec migration pg14-as-if-foss - .single-db-ci-connection-rspec - - .rails:rules:single-db-ci-connection + - .rails:rules:single-db-ci-connection-as-if-foss rspec background_migration pg14-as-if-foss single-db-ci-connection: extends: - rspec background_migration pg14-as-if-foss - .single-db-ci-connection-rspec - - .rails:rules:single-db-ci-connection + - .rails:rules:single-db-ci-connection-as-if-foss + +rspec migration pg14-as-if-foss clusterwide-db: + extends: + - rspec migration pg14-as-if-foss + - .clusterwide-db + - .rails:rules:clusterwide-db + +rspec background_migration pg14-as-if-foss clusterwide-db: + extends: + - rspec background_migration pg14-as-if-foss + - .clusterwide-db + - .rails:rules:clusterwide-db rspec unit pg14-as-if-foss: extends: @@ -599,13 +661,19 @@ rspec unit pg14-as-if-foss single-db: extends: - rspec unit pg14-as-if-foss - .single-db-rspec - - .rails:rules:single-db + - .rails:rules:single-db-as-if-foss rspec unit pg14-as-if-foss single-db-ci-connection: extends: - rspec unit pg14-as-if-foss - .single-db-ci-connection-rspec - - .rails:rules:single-db-ci-connection + - .rails:rules:single-db-ci-connection-as-if-foss + +rspec unit pg14-as-if-foss clusterwide-db: + extends: + - rspec unit pg14-as-if-foss + - .clusterwide-db + - .rails:rules:clusterwide-db rspec integration pg14-as-if-foss: extends: @@ -617,13 +685,19 @@ rspec integration pg14-as-if-foss single-db: extends: - rspec integration pg14-as-if-foss - .single-db-rspec - - .rails:rules:single-db + - .rails:rules:single-db-as-if-foss rspec integration pg14-as-if-foss single-db-ci-connection: extends: - rspec integration pg14-as-if-foss - .single-db-ci-connection-rspec - - .rails:rules:single-db-ci-connection + - .rails:rules:single-db-ci-connection-as-if-foss + +rspec integration pg14-as-if-foss clusterwide-db: + extends: + - rspec integration pg14-as-if-foss + - .clusterwide-db + - .rails:rules:clusterwide-db rspec system pg14-as-if-foss: extends: @@ -635,13 +709,19 @@ rspec system pg14-as-if-foss single-db: extends: - rspec system pg14-as-if-foss - .single-db-rspec - - .rails:rules:single-db + - .rails:rules:single-db-as-if-foss rspec system pg14-as-if-foss single-db-ci-connection: extends: - rspec system pg14-as-if-foss - .single-db-ci-connection-rspec - - .rails:rules:single-db-ci-connection + - .rails:rules:single-db-ci-connection-as-if-foss + +rspec system pg14-as-if-foss clusterwide-db: + extends: + - rspec system pg14-as-if-foss + - .clusterwide-db + - .rails:rules:clusterwide-db rspec-ee migration pg14: extends: @@ -681,6 +761,18 @@ rspec-ee background_migration pg14 single-db-ci-connection: - .single-db-ci-connection-rspec - .rails:rules:single-db-ci-connection +rspec-ee migration pg14 clusterwide-db: + extends: + - rspec-ee migration pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db + +rspec-ee background_migration pg14 clusterwide-db: + extends: + - rspec-ee background_migration pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db + rspec-ee migration pg14 praefect: extends: - rspec migration pg14 @@ -722,6 +814,12 @@ rspec-ee unit pg14 single-db-ci-connection: - .single-db-ci-connection-rspec - .rails:rules:single-db-ci-connection +rspec-ee unit pg14 clusterwide-db: + extends: + - rspec-ee unit pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db + rspec-ee integration pg14: extends: - .rspec-ee-base-pg14 @@ -751,6 +849,12 @@ rspec-ee integration pg14 single-db-ci-connection: - .single-db-ci-connection-rspec - .rails:rules:single-db-ci-connection +rspec-ee integration pg14 clusterwide-db: + extends: + - rspec-ee integration pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db + rspec-ee system pg14: extends: - .rspec-ee-base-pg14 @@ -779,6 +883,12 @@ rspec-ee system pg14 single-db-ci-connection: - rspec-ee system pg14 - .single-db-ci-connection-rspec - .rails:rules:single-db-ci-connection + +rspec-ee system pg14 clusterwide-db: + extends: + - rspec-ee system pg14 + - .clusterwide-db + - .rails:rules:clusterwide-db # EE: default refs (MRs, default branch, schedules) jobs # ################################################## @@ -1085,7 +1195,7 @@ rspec-ee system pg15 es8: needs: ["setup-test-env", "retrieve-tests-metadata", "compile-test-assets", "detect-tests"] script: - !reference [.base-script, script] - - rspec_fail_fast "${MATCHING_TESTS_PATH}" "--tag ~quarantine --tag ~zoekt" + - rspec_fail_fast "${MATCHING_TESTS_PATH}" "--fail-fast=${RSPEC_FAIL_FAST_THRESHOLD} --tag ~quarantine --tag ~zoekt --tag ~click_house" rspec fail-fast: extends: diff --git a/.gitlab/ci/rails/rspec-foss-impact.gitlab-ci.yml.erb b/.gitlab/ci/rails/rspec-foss-impact.gitlab-ci.yml.erb index dc469c30207..4584cba0282 100644 --- a/.gitlab/ci/rails/rspec-foss-impact.gitlab-ci.yml.erb +++ b/.gitlab/ci/rails/rspec-foss-impact.gitlab-ci.yml.erb @@ -39,7 +39,7 @@ dont-interrupt-me: RSPEC_TESTS_MAPPING_ENABLED: "true" script: - !reference [.base-script, script] - - rspec_paralellized_job "--tag ~quarantine --tag ~level:background_migration --tag ~zoekt" + - rspec_paralellized_job "--fail-fast=${RSPEC_FAIL_FAST_THRESHOLD} --tag ~quarantine --tag ~level:background_migration --tag ~zoekt --tag ~click_house" artifacts: expire_in: 7d paths: @@ -54,7 +54,7 @@ rspec migration foss-impact: <% end %> script: - !reference [.base-script, script] - - rspec_paralellized_job "--tag ~quarantine --tag ~zoekt" + - rspec_paralellized_job "--fail-fast=${RSPEC_FAIL_FAST_THRESHOLD} --tag ~quarantine --tag ~zoekt --tag ~click_house" <% end %> <% if rspec_files_per_test_level[:background_migration][:files].size > 0 %> diff --git a/.gitlab/ci/rails/shared.gitlab-ci.yml b/.gitlab/ci/rails/shared.gitlab-ci.yml index 6ea6d8e523c..9c2b0406f02 100644 --- a/.gitlab/ci/rails/shared.gitlab-ci.yml +++ b/.gitlab/ci/rails/shared.gitlab-ci.yml @@ -24,14 +24,16 @@ include: # Only install knapsack after bundle install! Otherwise oddly some native # gems could not be found under some circumstance. No idea why, hours wasted. - run_timed_command "gem install knapsack --no-document" - - echo -e "\e[0Ksection_start:`date +%s`:gitaly-test-spawn[collapsed=true]\r\e[0KStarting Gitaly" - section_start "gitaly-test-spawn" "Spawning Gitaly"; scripts/gitaly-test-spawn; section_end "gitaly-test-spawn" # Do not use 'bundle exec' here - - echo -e "\e[0Ksection_end:`date +%s`:gitaly-test-spawn\r\e[0K" .no-redis-cluster: variables: USE_REDIS_CLUSTER: "false" +.clusterwide-db: + variables: + CLUSTERWIDE_DB: "true" + .single-db: variables: DECOMPOSED_DB: "false" @@ -74,12 +76,15 @@ include: # spec/lib, yet background migration tests are also sitting there, # and they should run on their own jobs so we don't need to run them # in unit tests again. - - rspec_paralellized_job "--tag ~quarantine --tag ~level:background_migration" + - rspec_paralellized_job "--fail-fast=${RSPEC_FAIL_FAST_THRESHOLD} --tag ~quarantine --tag ~level:background_migration --tag ~click_house" after_script: - echo -e "\e[0Ksection_start:`date +%s`:report_results_section[collapsed=true]\r\e[0KReport results" - | if [ "$CREATE_RAILS_TEST_FAILURE_ISSUES" == "true" ]; then - bundle exec relate-failure-issue --input-files "rspec/rspec-*.json" --system-log-files "log" --project "gitlab-org-sandbox/rails-test-failures" --token "${RAILS_TEST_FAILURES_PROJECT_TOKEN}"; + bundle exec relate-failure-issue --input-files "rspec/rspec-*.json" --system-log-files "log" --project "gitlab-org/gitlab" --token "${TEST_FAILURES_PROJECT_TOKEN}"; + fi + if [ "$CREATE_RAILS_SLOW_TEST_ISSUES" == "true" ]; then + bundle exec slow-test-issues --input-files "rspec/rspec-*.json" --project "gitlab-org-sandbox/rails-test-failures" --token "${RAILS_TEST_FAILURES_PROJECT_TOKEN}"; fi - echo -e "\e[0Ksection_end:`date +%s`:report_results_section\r\e[0K" @@ -105,7 +110,7 @@ include: .rspec-base-migration: script: - !reference [.base-script, script] - - rspec_paralellized_job "--tag ~quarantine --tag ~zoekt" + - rspec_paralellized_job "--fail-fast=${RSPEC_FAIL_FAST_THRESHOLD} --tag ~quarantine --tag ~zoekt --tag ~click_house" after_script: - !reference [.rspec-base, after_script] @@ -124,6 +129,16 @@ include: - .rspec-base - .use-pg14 +.rspec-base-pg14-clickhouse23: + extends: + - .rspec-base + - .use-pg14-clickhouse23 + script: + - cp config/click_house.yml.example config/click_house.yml + - 'sed -i "s|url:.*$|url: http://clickhouse:8123|g" config/click_house.yml' + - !reference [.base-script, script] + - rspec_paralellized_job "--fail-fast=${RSPEC_FAIL_FAST_THRESHOLD} --tag click_house" + .rspec-base-pg14-as-if-foss: extends: - .rspec-base diff --git a/.gitlab/ci/release-environments/main.gitlab-ci.yml b/.gitlab/ci/release-environments/main.gitlab-ci.yml index 6c28ba3e2dd..7eb67509301 100644 --- a/.gitlab/ci/release-environments/main.gitlab-ci.yml +++ b/.gitlab/ci/release-environments/main.gitlab-ci.yml @@ -14,7 +14,7 @@ review-build-cng: review-deploy-env: allow_failure: true stage: deploy - needs: ["release-environments-build-cng"] + needs: ["review-build-cng"] variables: DEPLOY_ENV: deploy.env script: diff --git a/.gitlab/ci/review-apps/main.gitlab-ci.yml b/.gitlab/ci/review-apps/main.gitlab-ci.yml index 3522dac39e4..c5f56076ce7 100644 --- a/.gitlab/ci/review-apps/main.gitlab-ci.yml +++ b/.gitlab/ci/review-apps/main.gitlab-ci.yml @@ -44,6 +44,11 @@ review-build-cng: variables: HOST_SUFFIX: "${CI_ENVIRONMENT_SLUG}" DOMAIN: "-${CI_ENVIRONMENT_SLUG}.${REVIEW_APPS_DOMAIN}" + GITLAB_HELM_CHART_PROJECT_URL: "https://gitlab.com/gitlab-org/charts/gitlab" + GITLAB_HELM_REPO_URL: "https://charts.gitlab.io" + GITLAB_REPO_URL: ${CI_PROJECT_URL} + GITLAB_IMAGE_REPOSITORY: "registry.gitlab.com/gitlab-org/build/cng-mirror" + GITLAB_IMAGE_SUFFIX: "ee" GITLAB_HELM_CHART_REF: "75b1486a9aec212d0f49ef1251526d8e51004bbc" # 7.0.1: https://gitlab.com/gitlab-org/charts/gitlab/-/commit/75b1486a9aec212d0f49ef1251526d8e51004bbc environment: name: review/${CI_COMMIT_REF_SLUG}${SCHEDULE_TYPE} # No separator for SCHEDULE_TYPE so it's compatible as before and looks nice without it @@ -115,6 +120,7 @@ review-deploy-sample-projects: .review-stop-base: extends: .review-workflow-base + timeout: 15min environment: action: stop variables: diff --git a/.gitlab/ci/review-apps/qa.gitlab-ci.yml b/.gitlab/ci/review-apps/qa.gitlab-ci.yml index ccbdfd3c298..a9ab031e115 100644 --- a/.gitlab/ci/review-apps/qa.gitlab-ci.yml +++ b/.gitlab/ci/review-apps/qa.gitlab-ci.yml @@ -1,6 +1,18 @@ include: - local: .gitlab/ci/qa-common/main.gitlab-ci.yml - template: Verify/Browser-Performance.gitlab-ci.yml + - component: "gitlab.com/gitlab-org/quality/pipeline-common/allure-report@7.3.0" + inputs: + job_name: "e2e-test-report" + job_stage: "post-qa" + aws_access_key_id_variable_name: "QA_ALLURE_AWS_ACCESS_KEY_ID" + aws_secret_access_key_variable_name: "QA_ALLURE_AWS_SECRET_ACCESS_KEY" + gitlab_auth_token_variable_name: "PROJECT_TOKEN_FOR_CI_SCRIPTS_API_USAGE" + allure_results_glob: "qa/tmp/allure-results" + allure_ref_slug: "${CI_COMMIT_REF_SLUG}" + allure_project_path: "${CI_PROJECT_PATH}" + allure_merge_request_iid: "${CI_MERGE_REQUEST_IID}" + allure_job_name: "${QA_RUN_TYPE}" .test-variables: variables: @@ -108,13 +120,7 @@ browser_performance: URL: environment_url.txt e2e-test-report: - extends: - - .e2e-test-report - - .rules:prepare-report - stage: post-qa - variables: - ALLURE_RESULTS_GLOB: qa/tmp/allure-results - ALLURE_JOB_NAME: $QA_RUN_TYPE + extends: .rules:prepare-report upload-knapsack-report: extends: @@ -144,7 +150,6 @@ notify-slack: variables: QA_RSPEC_XML_FILE_PATTERN: ${CI_PROJECT_DIR}/qa/tmp/rspec-*.xml RUN_WITH_BUNDLE: "true" - ALLURE_JOB_NAME: $QA_RUN_TYPE when: on_failure export-test-metrics: diff --git a/.gitlab/ci/review.gitlab-ci.yml b/.gitlab/ci/review.gitlab-ci.yml index 68bfa682d0f..224312bd8ee 100644 --- a/.gitlab/ci/review.gitlab-ci.yml +++ b/.gitlab/ci/review.gitlab-ci.yml @@ -1,4 +1,5 @@ review-cleanup: + timeout: 15min extends: - .default-retry - .review:rules:review-cleanup diff --git a/.gitlab/ci/rules.gitlab-ci.yml b/.gitlab/ci/rules.gitlab-ci.yml index 1c1f6cfa96a..295f22258e9 100644 --- a/.gitlab/ci/rules.gitlab-ci.yml +++ b/.gitlab/ci/rules.gitlab-ci.yml @@ -2,7 +2,7 @@ # Conditions # ############## .if-not-canonical-namespace: &if-not-canonical-namespace - if: '$CI_PROJECT_NAMESPACE !~ /^gitlab(-org)?($|\/)/' + if: '$CI_PROJECT_NAMESPACE !~ /^gitlab(-org|-cn)?($|\/)/' .if-not-ee: &if-not-ee # Only consider FOSS not EE @@ -21,7 +21,7 @@ if: '$FORCE_GITLAB_CI' .if-default-refs: &if-default-refs - if: '$CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH || $CI_COMMIT_REF_NAME =~ /^[\d-]+-stable(-ee)?$/ || $CI_COMMIT_REF_NAME =~ /^\d+-\d+-auto-deploy-\d+$/ || $CI_COMMIT_REF_NAME =~ /^security\// || $CI_COMMIT_REF_NAME == "ruby2" || ($CI_MERGE_REQUEST_EVENT_TYPE == "merged_result" || $CI_MERGE_REQUEST_EVENT_TYPE == "detached") || $CI_COMMIT_TAG || $FORCE_GITLAB_CI' + if: '$CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH || $CI_COMMIT_REF_NAME =~ /^[\d-]+-stable(-ee)?$/ || $CI_COMMIT_REF_NAME =~ /^\d+-\d+-auto-deploy-\d+$/ || $CI_COMMIT_REF_NAME =~ /^security\// || $CI_COMMIT_REF_NAME == "ruby3_1" || ($CI_MERGE_REQUEST_EVENT_TYPE == "merged_result" || $CI_MERGE_REQUEST_EVENT_TYPE == "detached") || $CI_COMMIT_TAG || $FORCE_GITLAB_CI' .if-default-branch-refs: &if-default-branch-refs if: '$CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH && $CI_MERGE_REQUEST_IID == null' @@ -51,10 +51,10 @@ if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "release-tools/update-gitaly" || $CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /stable-(ee|jh)$/' .if-merge-request-targeting-stable-branch: &if-merge-request-targeting-stable-branch - if: '($CI_MERGE_REQUEST_EVENT_TYPE == "merged_result" || $CI_MERGE_REQUEST_EVENT_TYPE == "detached") && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /^[\d-]+-stable(-ee)?$/' + if: '($CI_MERGE_REQUEST_EVENT_TYPE == "merged_result" || $CI_MERGE_REQUEST_EVENT_TYPE == "detached") && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /^[\d-]+-stable(-ee|-jh)?$/' -.if-merge-request-labels-run-in-ruby2: &if-merge-request-labels-run-in-ruby2 - if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-in-ruby2/' +.if-merge-request-labels-run-in-ruby3_1: &if-merge-request-labels-run-in-ruby3_1 + if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-in-ruby3_1/' .if-merge-request-labels-as-if-foss: &if-merge-request-labels-as-if-foss if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-as-if-foss/' @@ -77,6 +77,9 @@ .if-merge-request-labels-run-single-db: &if-merge-request-labels-run-single-db if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-single-db/' +.if-merge-request-labels-run-clusterwide-db: &if-merge-request-labels-run-clusterwide-db + if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-clusterwide-db/' + .if-merge-request-labels-run-review-app: &if-merge-request-labels-run-review-app if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-review-app/' @@ -113,8 +116,8 @@ .if-default-branch-schedule-nightly: &if-default-branch-schedule-nightly if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "schedule" && $SCHEDULE_TYPE == "nightly"' -.if-ruby2-branch-schedule-nightly: &if-ruby2-branch-schedule-nightly - if: '$CI_COMMIT_BRANCH == "ruby2" && $CI_PIPELINE_SOURCE == "schedule" && $SCHEDULE_TYPE == "nightly"' +.if-ruby3_1-branch-schedule-nightly: &if-ruby3_1-branch-schedule-nightly + if: '$CI_COMMIT_BRANCH == "ruby3_1" && $CI_PIPELINE_SOURCE == "schedule" && $SCHEDULE_TYPE == "nightly"' .if-security-schedule: &if-security-schedule if: '$CI_PROJECT_NAMESPACE == "gitlab-org/security" && $CI_PIPELINE_SOURCE == "schedule"' @@ -149,8 +152,8 @@ .if-dot-com-gitlab-org-and-security-tag: &if-dot-com-gitlab-org-and-security-tag if: '$CI_SERVER_HOST == "gitlab.com" && $CI_PROJECT_NAMESPACE =~ /^gitlab-org($|\/security$)/ && $CI_COMMIT_TAG' -.if-ruby2-branch: &if-ruby2-branch - if: '$CI_COMMIT_BRANCH == "ruby2"' +.if-ruby3_1-branch: &if-ruby3_1-branch + if: '$CI_COMMIT_BRANCH == "ruby3_1" || $CI_MERGE_REQUEST_LABELS =~ /pipeline:run-in-ruby3_1/' #################### # Changes patterns # @@ -193,10 +196,13 @@ - "lib/gitlab/setup_helper.rb" .workhorse-patterns: &workhorse-patterns + - ".gitlab/ci/workhorse.gitlab-ci.yml" - "GITLAB_WORKHORSE_VERSION" - "workhorse/**/*" - - ".gitlab/ci/workhorse.gitlab-ci.yml" + - "scripts/gitaly-test-build" + - "scripts/gitaly-test-spawn" - "spec/support/gitlab-git-test.git/**/*" + - "spec/support/helpers/gitaly_setup.rb" .yaml-lint-patterns: &yaml-lint-patterns - "**/*.{yml,yaml}{,.*}" @@ -226,9 +232,7 @@ .docs-deprecations-and-removals-patterns: &docs-deprecations-and-removals-patterns - "doc/update/deprecations.md" - - "doc/update/removals.md" - "data/deprecations/**/*" - - "data/removals/**/*" - "tooling/docs/**/*" - "lib/tasks/gitlab/docs/compile_deprecations.rake" @@ -311,9 +315,6 @@ .lib-gitlab-patterns: &lib-gitlab-patterns - "{,ee/,jh/}lib/{,ee/,jh/}gitlab/**/*" -.startup-css-patterns: &startup-css-patterns - - "{,ee/,jh/}app/assets/stylesheets/startup/**/*" - # Backend patterns + .ci-patterns .backend-patterns: &backend-patterns - "{,jh/}Gemfile{,.lock}" @@ -412,14 +413,13 @@ - ".stylelintrc" - "Dockerfile.assets" - "vendor/assets/**/*" - - ".{eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}" - - ".gitlab-ci.yml" + - ".{eslintrc.yml,eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}" - "*_VERSION" - "{,jh/}Gemfile{,.lock}" - "Rakefile" - "tests.yml" - "config.ru" - - "{,ee/,jh/}{app,bin,config,db,generator_templates,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*" + - "{,ee/,jh/}{app,bin,config,db,generator_templates,haml_lint,lib,locale,public,scripts,storybook,symbol,vendor}/**/*" - "doc/api/graphql/reference/*" # Files in this folder are auto-generated # CI changes - ".gitlab-ci.yml" @@ -448,6 +448,9 @@ # CI changes - ".gitlab-ci.yml" - ".gitlab/ci/**/*" + # Mapped patterns (see tests.yml) + - "data/whats_new/*.yml" + - "doc/index.md" # Backstage changes - "Dangerfile" - "danger/**/*" @@ -455,9 +458,6 @@ - "{,ee/,jh/}rubocop/**/*" - "{,ee/,jh/}spec/**/*" - "{,spec/}tooling/**/*" - # Mapped patterns (see tests.yml) - - "data/whats_new/*.yml" - - "doc/index.md" # .code-patterns + .qa-patterns .code-qa-patterns: &code-qa-patterns @@ -468,7 +468,7 @@ - ".stylelintrc" - "Dockerfile.assets" - "vendor/assets/**/*" - - ".{eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}" + - ".{eslintrc.yml,eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}" - "*_VERSION" - "{,jh/}Gemfile{,.lock}" - "Rakefile" @@ -495,7 +495,7 @@ - ".stylelintrc" - "Dockerfile.assets" - "vendor/assets/**/*" - - ".{eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}" + - ".{eslintrc.yml,eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}" - ".gitlab-ci.yml" - "*_VERSION" - "{,jh/}Gemfile{,.lock}" @@ -533,8 +533,7 @@ - ".stylelintrc" - "Dockerfile.assets" - "vendor/assets/**/*" - - ".{eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}" - - ".gitlab-ci.yml" + - ".{eslintrc.yml,eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}" - "*_VERSION" - "{,jh/}Gemfile{,.lock}" - "Rakefile" @@ -545,6 +544,9 @@ # CI changes - ".gitlab-ci.yml" - ".gitlab/ci/**/*" + # Mapped patterns (see tests.yml) + - "data/whats_new/*.yml" + - "doc/index.md" # Backstage changes - "Dangerfile" - "danger/**/*" @@ -555,9 +557,6 @@ # QA changes - ".dockerignore" - "{,jh/}qa/**/*" - # Mapped patterns (see tests.yml) - - "data/whats_new/*.yml" - - "doc/index.md" # Workhorse changes - "GITLAB_WORKHORSE_VERSION" - "workhorse/**/*" @@ -617,6 +616,8 @@ .as-if-jh-default-exclusion-rules: rules: + - if: '$ADD_JH_FILES_TOKEN == null' + when: never - <<: *if-security-merge-request when: never - <<: *if-merge-request-targeting-stable-branch @@ -872,10 +873,12 @@ - <<: *if-dot-com-gitlab-org-schedule variables: ARCH: amd64,arm64 - - <<: *if-ruby2-branch + - <<: *if-ruby3_1-branch .build-images:rules:build-qa-image-as-if-foss: rules: + - <<: *if-jh + when: never - !reference [".build-images:rules:build-qa-image-merge-requests", "rules"] - <<: *if-default-branch-schedule-nightly variables: @@ -915,7 +918,7 @@ - <<: *if-merge-request-targeting-stable-branch changes: *setup-test-env-patterns allow_failure: true - - <<: *if-ruby2-branch + - <<: *if-ruby3_1-branch allow_failure: true # We include the job under the matching conditions below, but unlike in .qa:rules:e2e:test-on-gdk we don't need to # set OMNIBUS_GITLAB_BUILD_ON_ALL_OS when testing against GDK @@ -950,7 +953,7 @@ when: never - <<: *if-merge-request-targeting-stable-branch changes: *setup-test-env-patterns - - <<: *if-ruby2-branch + - <<: *if-ruby3_1-branch - <<: *if-merge-request-labels-run-review-app - <<: *if-auto-deploy-branches - <<: *if-default-refs @@ -1013,6 +1016,12 @@ when: manual allow_failure: true +.caching:rules:cache-assets-as-if-foss: + rules: + - <<: *if-jh + when: never + - !reference [".caching:rules:cache-assets", "rules"] + .caching:rules:packages-cleanup: rules: # The new strategy to cache assets as generic packages is experimental and can be disabled by removing the `CACHE_ASSETS_AS_PACKAGE` variable @@ -1146,16 +1155,20 @@ - <<: *if-merge-request-targeting-stable-branch changes: *setup-test-env-patterns - <<: *if-merge-request-labels-run-review-app - - <<: *if-merge-request-labels-run-all-e2e - <<: *if-auto-deploy-branches - - <<: *if-ruby2-branch - - <<: *if-default-branch-schedule-nightly + - <<: *if-ruby3_1-branch - <<: *if-default-refs changes: *ci-build-images-patterns - <<: *if-default-refs changes: *code-qa-patterns - <<: *if-default-refs changes: *workhorse-patterns + # Rules to support .qa:rules:package-and-test + - <<: *if-default-branch-schedule-nightly + - <<: *if-merge-request-labels-run-all-e2e + - <<: *if-dot-com-gitlab-org-and-security-merge-request-manual-ff-package-and-e2e + changes: *feature-flag-development-config-patterns + - <<: *if-force-ci .frontend:rules:compile-production-assets-as-if-foss: rules: @@ -1175,16 +1188,13 @@ .frontend:rules:compile-test-assets-as-if-foss: rules: - - <<: *if-not-ee - when: never + - !reference [".strict-ee-only-rules", rules] - <<: *if-merge-request-labels-as-if-foss - <<: *if-merge-request-labels-run-all-rspec - <<: *if-merge-request-labels-frontend-and-feature-flag - <<: *if-default-refs changes: *code-backstage-qa-patterns - <<: *if-default-refs - changes: *startup-css-patterns - - <<: *if-default-refs changes: *workhorse-patterns .frontend:rules:default-frontend-jobs: @@ -1206,8 +1216,6 @@ - <<: *if-merge-request-labels-as-if-foss - <<: *if-merge-request-labels-run-all-rspec - <<: *if-merge-request - changes: *startup-css-patterns - - <<: *if-merge-request changes: *frontend-patterns-for-as-if-foss .frontend:rules:frontend_fixture-as-if-foss: @@ -1419,7 +1427,7 @@ allow_failure: true - <<: *if-dot-com-gitlab-org-schedule allow_failure: true - - <<: *if-ruby2-branch + - <<: *if-ruby3_1-branch # Rules to support .qa:rules:package-and-test-ee - <<: *if-merge-request changes: *dependency-patterns @@ -1456,7 +1464,10 @@ - <<: *if-merge-request-targeting-stable-branch changes: *setup-test-env-patterns allow_failure: true - - <<: *if-ruby2-branch + - <<: *if-ruby3_1-branch + variables: + USE_OLD_RUBY_VERSION: 'false' + CACHE_EDITION: 'GITLAB_RUBY3_1' allow_failure: true - <<: *if-merge-request changes: *dependency-patterns @@ -1512,8 +1523,6 @@ .qa:rules:package-and-test-ee: rules: - - <<: *if-default-branch-schedule-nightly # already executed in the 2-hourly schedule - when: never - !reference [".qa:rules:package-and-test-common", rules] - !reference [".qa:rules:package-and-test-schedule", rules] - !reference [".qa:rules:code-merge-request-manual", rules] @@ -1595,6 +1604,10 @@ changes: *setup-test-env-patterns - <<: *if-merge-request-labels-run-all-rspec +.rails:rules:clusterwide-db: + rules: + - <<: *if-merge-request-labels-run-clusterwide-db + .rails:rules:single-redis: rules: - <<: *if-merge-request-labels-run-single-db @@ -1615,6 +1628,12 @@ changes: *decomposed-db-models-patterns - <<: *if-default-branch-schedule-nightly +.rails:rules:single-db-as-if-foss: + rules: + - <<: *if-jh + when: never + - !reference [".rails:rules:single-db", "rules"] + .rails:rules:db:check-migrations-single-db: rules: - <<: *if-merge-request-labels-run-single-db @@ -1636,6 +1655,12 @@ changes: *decomposed-db-models-patterns - <<: *if-default-branch-schedule-nightly +.rails:rules:single-db-ci-connection-as-if-foss: + rules: + - <<: *if-jh + when: never + - !reference [".rails:rules:single-db-ci-connection", "rules"] + .rails:rules:db:check-migrations-single-db-ci-connection: rules: - <<: *if-merge-request-labels-run-single-db @@ -1831,8 +1856,7 @@ .rails:rules:as-if-foss-migration: rules: - - <<: *if-not-ee - when: never + - !reference [".strict-ee-only-rules", rules] - <<: *if-merge-request-labels-pipeline-expedite when: never - <<: *if-merge-request-labels-run-all-rspec @@ -1857,8 +1881,7 @@ .rails:rules:as-if-foss-unit: rules: - - <<: *if-not-ee - when: never + - !reference [".strict-ee-only-rules", rules] - <<: *if-fork-merge-request when: never - <<: *if-merge-request-labels-pipeline-expedite @@ -1869,8 +1892,7 @@ .rails:rules:as-if-foss-integration: rules: - - <<: *if-not-ee - when: never + - !reference [".strict-ee-only-rules", rules] - <<: *if-fork-merge-request when: never - <<: *if-merge-request-labels-pipeline-expedite @@ -1881,8 +1903,7 @@ .rails:rules:as-if-foss-system: rules: - - <<: *if-not-ee - when: never + - !reference [".strict-ee-only-rules", rules] - <<: *if-fork-merge-request when: never - <<: *if-merge-request-labels-pipeline-expedite @@ -1968,7 +1989,7 @@ - <<: *if-merge-request-labels-pipeline-expedite when: never - <<: *if-default-branch-schedule-nightly - - <<: *if-ruby2-branch-schedule-nightly + - <<: *if-ruby3_1-branch-schedule-nightly - <<: *if-merge-request-labels-run-all-rspec .rails:rules:rspec-coverage: @@ -2045,6 +2066,11 @@ changes: *code-backstage-patterns when: always +.rails:rules:clickhouse-changes: + rules: + - <<: *if-merge-request + changes: ["**/*click_house*"] + ######################### # Static analysis rules # ######################### @@ -2099,8 +2125,7 @@ .static-analysis:rules:static-analysis-as-if-foss: rules: - - <<: *if-not-ee - when: never + - !reference [".strict-ee-only-rules", rules] - <<: *if-merge-request-labels-as-if-foss changes: *code-backstage-qa-patterns - <<: *if-security-merge-request @@ -2110,6 +2135,17 @@ - <<: *if-merge-request changes: *static-analysis-patterns +.static-analysis:rules:trigger-depsaster: + rules: + - if: $ENABLE_DEPSASTER != 'true' + when: never + - <<: *if-fork-merge-request + when: never + - <<: *if-not-ee + when: never + - <<: *if-merge-request + changes: ["**/Gemfile.checksum"] + .semgrep-appsec-custom-rules:rules: rules: - <<: *if-not-ee @@ -2128,87 +2164,19 @@ - <<: *if-merge-request changes: *code-backstage-qa-patterns -####################### -# Vendored gems rules # -####################### - -.vendor:rules:mail-smtp_pool: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/mail-smtp_pool/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:attr_encrypted: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/attr_encrypted/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:microsoft_graph_mailer: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/microsoft_graph_mailer/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:ipynbdiff: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/ipynbdiff/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:omniauth-azure-oauth2: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/omniauth-azure-oauth2/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:omniauth_crowd: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/omniauth_crowd/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:omniauth-gitlab: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/omniauth-gitlab/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:omniauth-salesforce: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/omniauth-salesforce/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:devise-pbkdf2-encryptable: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/devise-pbkdf2-encryptable/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:gitlab_active_record: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/gitlab_active_record/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:bundler-checksum: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/bundler-checksum/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:cloud_profiler_agent: - rules: - - <<: *if-merge-request - changes: ["vendor/gems/cloud_profiler_agent/**/*"] - - <<: *if-merge-request-labels-run-all-rspec - -.vendor:rules:sidekiq-reliable-fetch: +.ping-appsec-for-dependency-review:rules: rules: + # Requiring $DEPENDENCY_REVIEW_PAT prevents the bot from running on forks or CE + # Without it the script would fail too. + - if: $ENABLE_DEPSCORE != 'true' + when: never + - if: "$DEPENDENCY_REVIEW_PAT == null" + when: never + - <<: *if-not-ee + when: never + # Run only when the merge request have dependency file modifications - <<: *if-merge-request - changes: ["vendor/gems/sidekiq-reliable-fetch/**/*"] - - <<: *if-merge-request-labels-run-all-rspec + changes: *dependency-patterns ################## # Releases rules # @@ -2356,11 +2324,9 @@ - <<: *if-merge-request-labels-run-review-app - <<: *if-dot-com-gitlab-org-merge-request changes: *ci-review-patterns - allow_failure: true - <<: *if-dot-com-gitlab-org-merge-request changes: *frontend-build-patterns variables: *review-change-pattern - allow_failure: true - <<: *if-dot-com-gitlab-org-merge-request changes: *controllers-patterns variables: *review-change-pattern @@ -2378,7 +2344,6 @@ allow_failure: true - <<: *if-dot-com-gitlab-org-merge-request changes: *qa-patterns - allow_failure: true - <<: *if-dot-com-gitlab-org-merge-request changes: *code-patterns when: manual @@ -2505,7 +2470,7 @@ .setup:rules:verify-ruby-3.0: rules: - - <<: *if-merge-request-labels-run-in-ruby2 + - <<: *if-merge-request-labels-run-in-ruby3_1 .setup:rules:verify-tests-yml: rules: @@ -2533,9 +2498,9 @@ - ".gitlab/ci/test-metadata.gitlab-ci.yml" - "scripts/rspec_helpers.sh" -####################### +################### # Preflight rules # -####################### +################### .preflight:rules:rails-production-server-boot: rules: @@ -2578,6 +2543,11 @@ changes: - ".gitlab/ci/test-metadata.gitlab-ci.yml" - "scripts/rspec_helpers.sh" + - <<: *if-merge-request + changes: + - "gems/gitlab-rspec/**/*" + - "gems/rspec_flaky/**/*" + - "scripts/flaky_examples/prune-old-flaky-examples" ################### # workhorse rules # @@ -2665,3 +2635,14 @@ - <<: *if-merge-request-labels-pipeline-expedite when: never - !reference [".releases:rules:canonical-dot-com-gitlab-stable-branch-only-setup-test-env-patterns", rules] + +################### +# Benchmark rules # +################### +.benchmark:rules:benchmark-markdown: + rules: + - <<: *if-default-refs + changes: *setup-test-env-patterns + when: manual + - <<: *if-merge-request-labels-run-all-rspec + when: manual diff --git a/.gitlab/ci/setup.gitlab-ci.yml b/.gitlab/ci/setup.gitlab-ci.yml index 85409d37fa4..ddd479385fc 100644 --- a/.gitlab/ci/setup.gitlab-ci.yml +++ b/.gitlab/ci/setup.gitlab-ci.yml @@ -57,7 +57,7 @@ verify-ruby-3.0: - .setup:rules:verify-ruby-3.0 stage: prepare script: - - echo 'Please remove label ~"pipeline:run-in-ruby2" so we do test against Ruby 3.0 (default version) before merging the merge request' + - echo 'Please remove label ~"pipeline:run-in-ruby3_1" so we do test against Ruby 3.0 (default version) before merging the merge request' - exit 1 verify-tests-yml: @@ -185,15 +185,15 @@ trigger-omnibus-env: before_script: - source scripts/utils.sh script: + # Note that OMNIBUS_GITLAB_CACHE_UPDATE is not used in the code, but it is actually used in the 2-hourly maintenance pipeline schedule. - | SECURITY_SOURCES=$([[ ! "$CI_PROJECT_NAMESPACE" =~ ^gitlab-org\/security ]] || echo "true") echo "SECURITY_SOURCES=${SECURITY_SOURCES:-false}" > $BUILD_ENV echo "OMNIBUS_GITLAB_CACHE_UPDATE=${OMNIBUS_GITLAB_CACHE_UPDATE:-false}" >> $BUILD_ENV for version_file in *_VERSION; do echo "$version_file=$(cat $version_file)" >> $BUILD_ENV; done - echo "OMNIBUS_GITLAB_RUBY3_BUILD=${OMNIBUS_GITLAB_RUBY3_BUILD:-false}" >> $BUILD_ENV - echo "OMNIBUS_GITLAB_RUBY2_BUILD=${OMNIBUS_GITLAB_RUBY2_BUILD:-false}" >> $BUILD_ENV - echo "OMNIBUS_GITLAB_CACHE_EDITION=${OMNIBUS_GITLAB_CACHE_EDITION:-GITLAB}" >> $BUILD_ENV echo "OMNIBUS_GITLAB_BUILD_ON_ALL_OS=${OMNIBUS_GITLAB_BUILD_ON_ALL_OS:-false}" >> $BUILD_ENV + echo "USE_OLD_RUBY_VERSION=${USE_OLD_RUBY_VERSION:-false}" >> $BUILD_ENV + echo "NEXT_RUBY_VERSION=${NEXT_RUBY_VERSION}" >> $BUILD_ENV echo "GITLAB_ASSETS_TAG=$(assets_image_tag)" >> $BUILD_ENV echo "EE=$([[ $FOSS_ONLY == '1' ]] && echo 'false' || echo 'true')" >> $BUILD_ENV target_branch_name="${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-${CI_COMMIT_REF_NAME}}" @@ -214,3 +214,5 @@ trigger-omnibus-env as-if-foss: - .qa:rules:package-and-test-ce needs: - compile-production-assets as-if-foss + variables: + FOSS_ONLY: "1" diff --git a/.gitlab/ci/static-analysis.gitlab-ci.yml b/.gitlab/ci/static-analysis.gitlab-ci.yml index b351a63ecf0..40582170cc9 100644 --- a/.gitlab/ci/static-analysis.gitlab-ci.yml +++ b/.gitlab/ci/static-analysis.gitlab-ci.yml @@ -134,10 +134,10 @@ rubocop: unset CI_SLACK_WEBHOOK_URL run_timed_command "fail_on_warnings bundle exec rake rubocop:check:graceful" else - cat "${RSPEC_CHANGED_FILES_PATH}" | ruby -e 'print $stdin.read.split(" ").select { |f| File.exist?(f) }.join(" ")' > "$RUBOCOP_TARGET_FILES" + select_existing_files < "${RSPEC_CHANGED_FILES_PATH}" > "${RUBOCOP_TARGET_FILES}" # Skip running RuboCop if there's no target files if [ -s "${RUBOCOP_TARGET_FILES}" ]; then - run_timed_command "fail_on_warnings bundle exec rubocop --parallel --force-exclusion $(cat ${RUBOCOP_TARGET_FILES})" + run_timed_command "fail_on_warnings bundle exec rubocop --config .rubocop.yml --parallel --force-exclusion $(cat ${RUBOCOP_TARGET_FILES})" else echoinfo "Nothing interesting changed for RuboCop. Skipping." fi @@ -177,7 +177,7 @@ feature-flags-usage: script: # We need to disable the cache for this cop since it creates files under tmp/feature_flags/*.used, # the cache would prevent these files from being created. - - run_timed_command "fail_on_warnings bundle exec rubocop --only Gitlab/MarkUsedFeatureFlags --cache false" + - run_timed_command "fail_on_warnings bundle exec rubocop --config .rubocop.yml --only Gitlab/MarkUsedFeatureFlags --cache false" artifacts: expire_in: 31d when: always @@ -199,7 +199,7 @@ semgrep-appsec-custom-rules: --include app --include lib --include workhorse \ --exclude '*_test.go' --exclude spec --exclude qa > gl-sast-report.json || true variables: - CUSTOM_RULES_URL: https://gitlab.com/gitlab-com/gl-security/appsec/sast-custom-rules/-/raw/main/appsec-pings/rules.yml + CUSTOM_RULES_URL: https://gitlab.com/gitlab-com/gl-security/appsec/sast-custom-rules/-/raw/main/gitlab-sast-rules/rules.yml artifacts: paths: - gl-sast-report.json @@ -217,3 +217,31 @@ ping-appsec-for-sast-findings: script: - apk add jq curl - scripts/process_custom_semgrep_results.sh + +trigger-depsaster: + extends: .static-analysis:rules:trigger-depsaster + stage: lint + variables: + MERGE_REQUEST_PROJECT_ID: $CI_MERGE_REQUEST_PROJECT_ID + MERGE_REQUEST_IID: $CI_MERGE_REQUEST_IID + trigger: + project: "gitlab-com/gl-security/appsec/tooling/depsaster" + allow_failure: true + +ping-appsec-for-dependency-review: + stage: lint + needs: [] + variables: + GIT_CHECKOUT: "false" + DISABLE_MENTIONS: "false" + DISABLE_SCORING: "true" + DISABLE_COMMENTING: "false" + DEPENDENCY_REVIEW_BOT_CI_REG: "${CI_REGISTRY}/gitlab-com/gl-security/appsec/tooling/depscore/master" + extends: [".ping-appsec-for-dependency-review:rules", ".use-docker-in-docker"] + before_script: + - apk add jq curl + - DEPENDENCY_REVIEW_BOT_UNAME=$(curl --header "PRIVATE-TOKEN:$DEPENDENCY_REVIEW_PAT" "https://gitlab.com/api/v4/user" | jq -r '.username') + - echo "$DEPENDENCY_REVIEW_PAT" | docker login --password-stdin -u "$DEPENDENCY_REVIEW_BOT_UNAME" -- "$DEPENDENCY_REVIEW_BOT_CI_REG" + script: + - docker run --interactive --rm "$DEPENDENCY_REVIEW_BOT_CI_REG:latest" -t "$DEPENDENCY_REVIEW_PAT" -p "$CI_PROJECT_ID" -m "$CI_MERGE_REQUEST_IID" -s "$DISABLE_SCORING" -a "$DISABLE_MENTIONS" -c "$DISABLE_COMMENTING" + allow_failure: true diff --git a/.gitlab/ci/templates/gem.gitlab-ci.yml b/.gitlab/ci/templates/gem.gitlab-ci.yml new file mode 100644 index 00000000000..46c5e1342c6 --- /dev/null +++ b/.gitlab/ci/templates/gem.gitlab-ci.yml @@ -0,0 +1,28 @@ +# The template generates jobs that trigger child pipelines for gems vendored in the main GitLab project under `gems/`. +# +# Inputs: +# - `gem_name`: The name of the gem, i.e. if the gem is located at `gems/gitlab-rspec`, `gem_name` should be set to `gitlab-rspec`. +# - `gem_path_prefix`: The prefix of the gem path, i.e. if the gem is located at `vendor/gems/gitlab-rspec`, `gem_path_prefix` should be set to `vendor/gems/`. Defaults to `gems/`. +spec: + inputs: + gem_name: + gem_path_prefix: + default: "gems/" +--- +.gems:rules:$[[inputs.gem_name]]: + rules: + - if: '$CI_MERGE_REQUEST_EVENT_TYPE == "merged_result" || $CI_MERGE_REQUEST_EVENT_TYPE == "detached"' + changes: + - "$[[inputs.gem_path_prefix]]$[[inputs.gem_name]]/**/*" + - ".gitlab/ci/gitlab-gems.gitlab-ci.yml" + - ".gitlab/ci/templates/gem.gitlab-ci.yml" + - "gems/gem.gitlab-ci.yml" + +gems $[[inputs.gem_name]]: + extends: ".gems:rules:$[[inputs.gem_name]]" + needs: [] + trigger: + include: "$[[inputs.gem_path_prefix]]$[[inputs.gem_name]]/.gitlab-ci.yml" + strategy: depend + inherit: + variables: false diff --git a/.gitlab/ci/test-on-gdk/main.gitlab-ci.yml b/.gitlab/ci/test-on-gdk/main.gitlab-ci.yml index 767dc8433d2..367579b1999 100644 --- a/.gitlab/ci/test-on-gdk/main.gitlab-ci.yml +++ b/.gitlab/ci/test-on-gdk/main.gitlab-ci.yml @@ -3,6 +3,18 @@ include: - local: .gitlab/ci/qa-common/main.gitlab-ci.yml - local: .gitlab/ci/qa-common/rules.gitlab-ci.yml - local: .gitlab/ci/qa-common/variables.gitlab-ci.yml + - component: "gitlab.com/gitlab-org/quality/pipeline-common/allure-report@7.3.0" + inputs: + job_name: "e2e-test-report" + job_stage: "report" + aws_access_key_id_variable_name: "QA_ALLURE_AWS_ACCESS_KEY_ID" + aws_secret_access_key_variable_name: "QA_ALLURE_AWS_SECRET_ACCESS_KEY" + gitlab_auth_token_variable_name: "PROJECT_TOKEN_FOR_CI_SCRIPTS_API_USAGE" + allure_results_glob: "qa/tmp/allure-results" + allure_ref_slug: "${CI_COMMIT_REF_SLUG}" + allure_project_path: "${CI_PROJECT_PATH}" + allure_merge_request_iid: "${CI_MERGE_REQUEST_IID}" + allure_job_name: "${QA_RUN_TYPE}" variables: COLORIZED_LOGS: "true" @@ -20,7 +32,6 @@ variables: tags: - e2e variables: - DOCKER_DRIVER: overlay2 DOCKER_HOST: tcp://docker:2375 QA_GENERATE_ALLURE_REPORT: "true" QA_CAN_TEST_PRAEFECT: "false" @@ -29,6 +40,7 @@ variables: RSPEC_REPORT_OPTS: "--order random --force-color --format documentation --format RspecJunitFormatter --out tmp/rspec-${CI_JOB_ID}.xml --format QA::Support::JsonFormatter --out tmp/rspec-${CI_JOB_ID}.json --format html --out tmp/rspec-${CI_JOB_ID}.htm" FF_NETWORK_PER_BUILD: 1 GDK_URL: http://gdk.test:3000 + KNAPSACK_TEST_FILE_PATTERN: "qa/specs/features/**/*_spec.rb" before_script: - echo "SUITE_RAN=true" > suite_status.env - echo -e "\e[0Ksection_start:`date +%s`:pull_image\r\e[0KPull GDK QA image" @@ -52,11 +64,11 @@ variables: - source scripts/utils.sh - cd qa && bundle install script: - - retry_exponential test_url ${GDK_URL}/users/sign_in + - retry_exponential test_url $GDK_URL/users/sign_in - echo -e "\e[0Ksection_end:`date +%s`:launch_gdk\r\e[0K" - echo -e "\e[0Ksection_start:`date +%s`:run_tests\r\e[0KRun E2E tests" - - QA_COMMAND="bundle exec bin/qa Test::Instance::All ${GDK_URL} -- ${RSPEC_TAGS} ${RSPEC_REPORT_OPTS}" - - echo "Running - '${QA_COMMAND}'" + - export QA_COMMAND="bundle exec bin/qa ${QA_SCENARIO:=Test::Instance::All} $GDK_URL $GITLAB_QA_OPTS -- $QA_TESTS $QA_RSPEC_TAGS $RSPEC_REPORT_OPTS" + - echo "Running - '$QA_COMMAND'" - eval "$QA_COMMAND" - echo -e "\e[0Ksection_end:`date +%s`:run_tests\r\e[0K" after_script: @@ -76,6 +88,11 @@ variables: when: always allow_failure: true +download-knapsack-report: + extends: + - .download-knapsack-report + - .rules:download-knapsack + cache-gems: extends: - .ruby-image @@ -105,9 +122,8 @@ gdk-qa-smoke: extends: - .gdk-qa-base variables: - TEST_GDK_TAGS: "--tag smoke" + QA_SCENARIO: Test::Instance::Smoke QA_RUN_TYPE: gdk-qa-smoke - RSPEC_TAGS: --tag smoke rules: - when: always @@ -116,37 +132,41 @@ gdk-qa-smoke-with-load-balancer: - .gdk-qa-base - .gdk-with-load-balancer-setup variables: - TEST_GDK_TAGS: "--tag smoke" + QA_SCENARIO: Test::Instance::Smoke QA_RUN_TYPE: gdk-qa-smoke - RSPEC_TAGS: --tag smoke + artifacts: + paths: + - log + reports: + dotenv: "" rules: - changes: - ".gitlab/ci/test-on-gdk/**" - "lib/gitlab/database/load_balancing/**/*" -# TODO: set non manual once smoke tests prove to be stable gdk-qa-reliable: extends: - .gdk-qa-base - parallel: 5 + - .parallel variables: + QA_SCENARIO: Test::Instance::Blocking QA_RUN_TYPE: gdk-qa-blocking - QA_KNAPSACK_REPORT_NAME: ee-instance-parallel - RSPEC_TAGS: --tag smoke --tag reliable - allow_failure: true rules: - - when: manual + - when: always gdk-qa-reliable-with-load-balancer: extends: - .gdk-qa-base - .gdk-with-load-balancer-setup - parallel: 5 + - .parallel variables: + QA_SCENARIO: Test::Instance::Blocking QA_RUN_TYPE: gdk-qa-blocking - QA_KNAPSACK_REPORT_NAME: ee-instance-parallel - RSPEC_TAGS: --tag smoke --tag reliable - allow_failure: true + artifacts: + paths: + - log + reports: + dotenv: "" rules: - changes: - ".gitlab/ci/test-on-gdk/**" @@ -155,12 +175,10 @@ gdk-qa-reliable-with-load-balancer: gdk-qa-non-blocking: extends: - .gdk-qa-base - parallel: 5 + - .parallel variables: + QA_SCENARIO: Test::Instance::NonBlocking QA_RUN_TYPE: gdk-qa-non-blocking - QA_KNAPSACK_REPORT_NAME: ee-instance-parallel - QA_SKIP_SMOKE_RELIABLE: "true" - allow_failure: true rules: - when: manual @@ -168,11 +186,7 @@ gdk-qa-non-blocking: # Post test stage # ========================================== e2e-test-report: - extends: - - .e2e-test-report - - .rules:report:allure-report - variables: - ALLURE_RESULTS_GLOB: qa/tmp/allure-results + extends: .rules:report:allure-report upload-knapsack-report: extends: diff --git a/.gitlab/ci/vendored-gems.gitlab-ci.yml b/.gitlab/ci/vendored-gems.gitlab-ci.yml index 3773a343d6d..915cca2dd29 100644 --- a/.gitlab/ci/vendored-gems.gitlab-ci.yml +++ b/.gitlab/ci/vendored-gems.gitlab-ci.yml @@ -1,103 +1,45 @@ -vendor mail-smtp_pool: - extends: - - .vendor:rules:mail-smtp_pool - needs: [] - trigger: - include: vendor/gems/mail-smtp_pool/.gitlab-ci.yml - strategy: depend - -vendor attr_encrypted: - extends: - - .vendor:rules:attr_encrypted - needs: [] - trigger: - include: vendor/gems/attr_encrypted/.gitlab-ci.yml - strategy: depend - -vendor microsoft_graph_mailer: - extends: - - .vendor:rules:microsoft_graph_mailer - needs: [] - trigger: - include: vendor/gems/microsoft_graph_mailer/.gitlab-ci.yml - strategy: depend - -vendor ipynbdiff: - extends: - - .vendor:rules:ipynbdiff - needs: [] - trigger: - include: vendor/gems/ipynbdiff/.gitlab-ci.yml - strategy: depend - -vendor omniauth-azure-oauth2: - extends: - - .vendor:rules:omniauth-azure-oauth2 - needs: [] - trigger: - include: vendor/gems/omniauth-azure-oauth2/.gitlab-ci.yml - strategy: depend - -vendor omniauth_crowd: - extends: - - .vendor:rules:omniauth_crowd - needs: [] - trigger: - include: vendor/gems/omniauth_crowd/.gitlab-ci.yml - strategy: depend - -vendor omniauth-gitlab: - extends: - - .vendor:rules:omniauth-gitlab - needs: [] - trigger: - include: vendor/gems/omniauth-gitlab/.gitlab-ci.yml - strategy: depend - -vendor omniauth-salesforce: - extends: - - .vendor:rules:omniauth-salesforce - needs: [] - trigger: - include: vendor/gems/omniauth-salesforce/.gitlab-ci.yml - strategy: depend - -vendor devise-pbkdf2-encryptable: - extends: - - .vendor:rules:devise-pbkdf2-encryptable - needs: [] - trigger: - include: vendor/gems/devise-pbkdf2-encryptable/.gitlab-ci.yml - strategy: depend - -vendor bundler-checksum: - extends: - - .vendor:rules:bundler-checksum - needs: [] - trigger: - include: vendor/gems/bundler-checksum/.gitlab-ci.yml - strategy: depend - -vendor gitlab_active_record: - extends: - - .vendor:rules:gitlab_active_record - needs: [] - trigger: - include: vendor/gems/gitlab_active_record/.gitlab-ci.yml - strategy: depend - -vendor cloud_profiler_agent: - extends: - - .vendor:rules:cloud_profiler_agent - needs: [] - trigger: - include: vendor/gems/cloud_profiler_agent/.gitlab-ci.yml - strategy: depend - -vendor sidekiq-reliable-fetch: - extends: - - .vendor:rules:sidekiq-reliable-fetch - needs: [] - trigger: - include: vendor/gems/sidekiq-reliable-fetch/.gitlab-ci.yml - strategy: depend +include: + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "mail-smtp_pool" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "attr_encrypted" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "microsoft_graph_mailer" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "omniauth-azure-oauth2" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "omniauth_crowd" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "omniauth-gitlab" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "omniauth-salesforce" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "devise-pbkdf2-encryptable" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "bundler-checksum" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "cloud_profiler_agent" + gem_path_prefix: "vendor/gems/" + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "sidekiq-reliable-fetch" + gem_path_prefix: "vendor/gems/" diff --git a/.gitlab/ci/workhorse.gitlab-ci.yml b/.gitlab/ci/workhorse.gitlab-ci.yml index 603f67079c2..00c4dc6c9a9 100644 --- a/.gitlab/ci/workhorse.gitlab-ci.yml +++ b/.gitlab/ci/workhorse.gitlab-ci.yml @@ -17,6 +17,9 @@ workhorse:verify: needs: - setup-test-env before_script: + - source scripts/utils.sh + - export BUNDLE_WITHOUT="${BUNDLE_WITHOUT}:default:test:puma:kerberos:metrics:omnibus:ed25519" + - bundle_install_script - go version - scripts/gitaly-test-build script: diff --git a/.gitlab/ci/yaml.gitlab-ci.yml b/.gitlab/ci/yaml.gitlab-ci.yml index 0420f158bbb..194b91ccb07 100644 --- a/.gitlab/ci/yaml.gitlab-ci.yml +++ b/.gitlab/ci/yaml.gitlab-ci.yml @@ -23,7 +23,7 @@ lint-pipeline-yaml: stage: lint needs: [] variables: - LINT_PATHS: .gitlab-ci.yml .gitlab/ci lib/gitlab/ci/templates data/deprecations data/removals data/whats_new + LINT_PATHS: .gitlab-ci.yml .gitlab/ci lib/gitlab/ci/templates data/deprecations data/whats_new script: - 'yamllint -d "{extends: default, rules: {line-length: disable, document-start: disable}}" $LINT_PATHS' diff --git a/.gitlab/issue_templates/Analytics_Instrumentation_Incident_Template.md b/.gitlab/issue_templates/Analytics_Instrumentation_Incident_Template.md new file mode 100644 index 00000000000..2ebd5acdcd4 --- /dev/null +++ b/.gitlab/issue_templates/Analytics_Instrumentation_Incident_Template.md @@ -0,0 +1,49 @@ +<!-- Please complete the template below as best as you can. Make sure to check if this issue has already been raised by someone else first to avoid duplication. + +For each section below, please add screenshots or links or anything that may help visual learners understand the problem better, even if this takes you an extra minute or two this is a great help to some folks. + +https://www.learning-styles-online.com/style/visual-spatial/ + +--> + +## Problem +<!-- High level overview of the Incident --> + +## Detection + +<!-- How did we learn of the problem? System Error? User? --> + +## Impact + +<!-- Who was impacted and how were they impacted? +Please include Teams, Users, Automations, Processes --> + + +## Additional information + +<!-- Please fill out any additonal information that may help with resolution --> + + +## Checklist +<!-- Please ensure all of the below steps are taken --> + +* [ ] Assigned severity tags based on this [guidance](https://about.gitlab.com/handbook/engineering/development/analytics/analytics-instrumentation/#incident-detection) +* [ ] Assigned to PM and EM of ~"group::analytics instrumentation" +* [ ] Posted link to incident in `g_analyze_analytics_instrumentation` and tagged both PM and EM of the group + + +<---- TO BE FILLED BY ASSIGNEE / RESOLUTION DRI----> +<!-- The intention of this section is to give stakeholders a high level summary of the issue with whatever information we have at the point of closing the issue or providing a quick fix. For a deeper analysis of the root cause and how we can avoid such incidents in the future please use the RCA template as outlined in the handbook incident resolution guidance and link the issue here --> + +## Summary +<!-- Overall summary of the issue --> + +## Root Cause + +<!-- To be filled by the assignee. What caused this incident? Link to RCA issue if needed --> + +## Resolution +<!-- To be filled by assignee.Has the issue been resolved? How? Please include any relevant issue and/or MR links in the description --> + + + diff --git a/.gitlab/issue_templates/Broken Master - Flaky.md b/.gitlab/issue_templates/Broken Master - Flaky.md index b87d9b5ee1f..5b01e67feab 100644 --- a/.gitlab/issue_templates/Broken Master - Flaky.md +++ b/.gitlab/issue_templates/Broken Master - Flaky.md @@ -27,4 +27,8 @@ Please refer to the [Resolution guidance](https://about.gitlab.com/handbook/engi Once the flaky failure has been fixed on the default branch, open merge requests to cherry-pick the fix to the active stable branches. +### Logs + +<!-- Add here failing job logs --> + /label ~"type::maintenance" ~"failure::flaky-test" ~"priority::3" ~"severity::3" diff --git a/.gitlab/issue_templates/Broken Master - Non-flaky.md b/.gitlab/issue_templates/Broken Master - Non-flaky.md index d2dc616ead8..0b92326ed42 100644 --- a/.gitlab/issue_templates/Broken Master - Non-flaky.md +++ b/.gitlab/issue_templates/Broken Master - Non-flaky.md @@ -21,4 +21,8 @@ Please read the below documentations for a workflow of triaging and resolving br Please refer to the [Resolution guidance](https://about.gitlab.com/handbook/engineering/workflow/#resolution-of-broken-master) to learn more about resolution of broken master. +### Logs + +<!-- Add here failing job logs --> + /label ~"master:broken" ~"Engineering Productivity" ~"priority::1" ~"severity::1" ~"type::maintenance" ~"maintenance::pipelines" diff --git a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md index c63a5cb0032..b77247954de 100644 --- a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md +++ b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md @@ -69,7 +69,6 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org t.integer :retry_count, default: 0, limit: 2, null: false t.integer :verification_retry_count, default: 0, limit: 2, null: false t.boolean :checksum_mismatch, default: false, null: false - t.boolean :force_to_redownload, default: false, null: false t.boolean :missing_on_primary, default: false, null: false t.binary :verification_checksum t.binary :verification_checksum_mismatched @@ -146,11 +145,14 @@ The Geo primary site needs to checksum every replicable so secondaries can verif enable_lock_retries! def up - create_table :cool_widget_states, id: false do |t| + create_table :cool_widget_states do |t| t.datetime_with_timezone :verification_started_at t.datetime_with_timezone :verification_retry_at t.datetime_with_timezone :verified_at - t.references :cool_widget, primary_key: true, default: nil, index: false, foreign_key: { on_delete: :cascade } + t.references :cool_widget, + null: false, + index: { unique: true }, + foreign_key: { on_delete: :cascade } t.integer :verification_state, default: 0, limit: 2, null: false t.integer :verification_retry_count, default: 0, limit: 2, null: false t.binary :verification_checksum, using: 'verification_checksum::bytea' @@ -292,6 +294,11 @@ That's all of the required database changes. # Search the codebase for examples, and consult a Geo expert if needed. end + override :verification_state_model_key + def verification_state_model_key + :cool_widget_id + end + override :verification_state_table_class def verification_state_table_class CoolWidgetState @@ -391,7 +398,7 @@ That's all of the required database changes. - [ ] Make sure a Geo secondary site can request and download Cool Widgets on the Geo primary site. You may need to make some changes to `Gitlab::GitAccessCoolWidget`. For example, see [this change for Group-level Wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/54914/diffs?commit_id=0f2b36f66697b4addbc69bd377ee2818f648dd33). -- [ ] Make sure a Geo secondary site can replicate Cool Widgets where repository does not exist on the Geo primary site. The only way to know about this is to parse the error text. You may need to make some changes to `Gitlab::CoolWidgetReplicator.no_repo_message` to return the proper error message. For example, see [this change for Group-level Wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74133). +- [ ] Make sure a Geo secondary site marks Cool Widgets as missing on primary when a repository does not exist on the Geo primary site. The only way to know about this is to parse the error text. You may need to make some changes to `Gitlab::CoolWidgetReplicator.no_repo_message` to return the proper error message. For example, see [this change for Group-level Wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74133). Alternatively, if a repository should *always* exist on the primary, then a Geo-specific workaround is to create an empty repository when verifying on primary. See examples for [project wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123869) and [design repositories](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123917). - [ ] Generate the feature flag definition files by running the feature flag commands and following the command prompts: diff --git a/.gitlab/issue_templates/Geo Replicate a new blob type.md b/.gitlab/issue_templates/Geo Replicate a new blob type.md index 91a86a96ac9..94c93bd27e4 100644 --- a/.gitlab/issue_templates/Geo Replicate a new blob type.md +++ b/.gitlab/issue_templates/Geo Replicate a new blob type.md @@ -148,14 +148,13 @@ The Geo primary site needs to checksum every replicable so secondaries can verif enable_lock_retries! def up - create_table :cool_widget_states, id: false do |t| + create_table :cool_widget_states do |t| t.datetime_with_timezone :verification_started_at t.datetime_with_timezone :verification_retry_at t.datetime_with_timezone :verified_at t.references :cool_widget, - primary_key: true, - default: nil, - index: false, + null: false, + index: { unique: true }, foreign_key: { on_delete: :cascade } t.integer :verification_state, default: 0, limit: 2, null: false t.integer :verification_retry_count, default: 0, limit: 2, null: false @@ -298,6 +297,11 @@ That's all of the required database changes. # Search the codebase for examples, and consult a Geo expert if needed. end + override :verification_state_model_key + def verification_state_model_key + :cool_widget_id + end + override :verification_state_table_class def verification_state_table_class CoolWidgetState diff --git a/.gitlab/issue_templates/Pipeline Execution Issue Implementation.md b/.gitlab/issue_templates/Pipeline Execution Issue Implementation.md new file mode 100644 index 00000000000..54a8f0e4153 --- /dev/null +++ b/.gitlab/issue_templates/Pipeline Execution Issue Implementation.md @@ -0,0 +1,55 @@ +<!-- +## Author Checklist +(_NOTE: This section can be removed when the issue is ready for creation_) +- [ ] Ensure that issue title is concise yet descriptive +- [ ] Add "~frontend", "~backend", or "~documentation" labels as appropriate +- [ ] Ensure the issue containing the feature or change proposal and related discussions is linked as related to this implementation issue. +- [ ] Aside from default labeling, please make sure to include relevant labels for `type::`, `workflow::`, and `~frontend`/`~backend` labeling. +- [ ] Issues with user-facing changes should include the `~UX` label. +--> + +## Summary + +## Proposal + +## Additional details +<!-- +_NOTE: If the issue has addressed all of these questions, this separate section can be removed._ +--> + +Some relevant technical details, if applicable, such as: + +- Does this need a ~"feature flag"? +- Does there need to be an associated ~"instrumentation" issue created related to this work? +- Is there an example response showing the data structure that should be returned (new endpoints only)? +- What permissions should be used? +- Is this EE or CE? + - [ ] EE + - [ ] CE +- Additional comments: + +## Implementation Table + +<!-- +_NOTE: If the issue is not part of an epic, the implementation table can be removed. If it is part of an epic, make sure that the implementation table below mirrors the corresponding epic's implementation table content._ +--> + + +| Group | Issue Link | +| ------ | ------ | +| ~backend | :point_left: You are here | +| ~frontend | [#123123](url) | + +<!-- +## Documentation + +_NOTE: This section is optional, but can be used for easy access to any relevant documentation URLs._ +--> + +## Links/References + + + + +/label ~"group::pipeline execution" ~"Category:Continuous Integration" ~"section::ci" ~"devops::verify" ~"workflow::planning breakdown" ~"needs weight" ~"cicd::planning" +/milestone %"Backlog" diff --git a/.gitlab/issue_templates/Pipeline Execution Refinement Spike.md b/.gitlab/issue_templates/Pipeline Execution Refinement Spike.md new file mode 100644 index 00000000000..38877ad777a --- /dev/null +++ b/.gitlab/issue_templates/Pipeline Execution Refinement Spike.md @@ -0,0 +1,33 @@ +< +## Author Checklist +(_NOTE: This section can be removed when the issue is ready for creation_) +- [ ] Set title to "Spike: Refine "<original issue title>" +- [ ] Update the slash commands below with the appropriate issue #. +- [ ] Update the weight slash command to an appropriate value based on the timebox size +- [ ] Add `~frontend`, `~backend`, or `~documentation` labels as appropriate +- [ ] Update the expected outcomes if the default ones don't capture everything that should be done in this spike. + +/blocks #<original issue> +/copy_metadata #<original issue> + +/label ~"spike" ~"group::pipeline execution" ~"section::ci" ~"devops::verify" ~"workflow::ready for development" ~"cicd::planning" +/weight ## + +## Summary - Why is this spike needed? + +## Timebox Expectations + +<How much time should be spent on this spike> + +## Expected Outcomes + +- [ ] Possible cause(s) are determined +- [ ] Technical proposal added to #<original issue> +- [ ] Weight added to #<original issue> + + +## Links/References + + + + diff --git a/.gitlab/issue_templates/Pipeline Security issue implementation b/.gitlab/issue_templates/Pipeline Security issue implementation.md index 72807e98012..72807e98012 100644 --- a/.gitlab/issue_templates/Pipeline Security issue implementation +++ b/.gitlab/issue_templates/Pipeline Security issue implementation.md diff --git a/.gitlab/issue_templates/Security developer workflow.md b/.gitlab/issue_templates/Security developer workflow.md index 294d699ea2f..9cbb74d10d8 100644 --- a/.gitlab/issue_templates/Security developer workflow.md +++ b/.gitlab/issue_templates/Security developer workflow.md @@ -24,6 +24,7 @@ MUST be linked for the release bot to know that the associated merge requests sh - [ ] Create a new branch prefixing it with `security-`. - [ ] Create a merge request targeting `master` on `gitlab.com/gitlab-org/security` and use the [Security Release merge request template]. - [ ] If this includes a breaking change, make sure to include a mention of it for the relevant versions in [`doc/update/index.md`](https://gitlab.com/gitlab-org/security/gitlab/-/blob/master/doc/update/index.md#version-specific-upgrading-instructions) + * See if the [breaking changes workflow] applies After your merge request has been approved according to our [approval guidelines] and by a team member of the AppSec team, you're ready to prepare the backports @@ -76,5 +77,6 @@ After your merge request has been approved according to our [approval guidelines [approval guidelines]: https://docs.gitlab.com/ee/development/code_review.html#approval-guidelines [issue as linked]: https://docs.gitlab.com/ee/user/project/issues/related_issues.html#add-a-linked-issue [issue really needs to follow the security release workflow]: https://gitlab.com/gitlab-org/release/docs/-/blob/master/general/security/developer.md#making-sure-the-issue-needs-to-follow-the-security-release-workflow +[breaking changes workflow]: https://gitlab.com/gitlab-org/release/docs/-/blob/master/general/security/far_reaching_impact_fixes_or_breaking_change_fixes.md /label ~security diff --git a/.gitlab/merge_request_templates/Deprecations.md b/.gitlab/merge_request_templates/Deprecations.md index 7e622dbe62c..c1321dd9f96 100644 --- a/.gitlab/merge_request_templates/Deprecations.md +++ b/.gitlab/merge_request_templates/Deprecations.md @@ -27,9 +27,9 @@ Deprecation announcements can and should be created and merged into Docs at any Please review: -- The definitions of ["Deprecation", "End of Support", and "Removal"](https://docs.gitlab.com/ee/development/deprecation_guidelines/#terminology). -- The [guidelines for deprecations](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations). -- The process for [creating a deprecation entry](https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-a-deprecation-entry). +- The definitions of ["Deprecation", "End of Support", and "Removal"](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- The [guidelines for deprecations](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes). +- The process for [creating a deprecation announcement](https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-the-announcement). They are frequently updated, and everyone should make sure they are aware of the current standards (PM, PMM, EM, and TW). @@ -38,7 +38,7 @@ They are frequently updated, and everyone should make sure they are aware of the - [ ] Set yourself as the Assignee, meaning you are the DRI. - [ ] If the deprecation is a [breaking change](https://about.gitlab.com/handbook/product/gitlab-the-product/#breaking-change), add label `breaking change`. - [ ] Confirm this MR is labeled ~"release post item::deprecation" -- [ ] Follow the process to [create a deprecation YAML file](https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-a-deprecation-entry). +- [ ] Follow the process to [create a deprecation YAML file](https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-the-announcement). - [ ] Add reviewers by the 10th. - [ ] Add scoped `devops::` and `group::` labels as necessary. - [ ] Add the appropriate milestone to this MR. @@ -48,8 +48,8 @@ They are frequently updated, and everyone should make sure they are aware of the When the content is ready for review, it must be reviewed by a Technical Writer and Engineering Manager, but can also be reviewed by Product Marketing, Product Design, and the Product Leaders for this area. Please use the -[reviewers](https://docs.gitlab.com/ee/user/project/merge_requests/reviews/) -feature for all reviews. Reviewers will then `approve` the MR and remove themselves from Reviewers when their review is complete. +[reviewers](https://docs.gitlab.com/ee/development/code_review.html#dogfooding-the-reviewers-feature) +feature for all reviews. Reviewers will then approve the MR and remove themselves from Reviewers when their review is complete. - [ ] (Recommended) PMM - [ ] (Optional) Product Designer @@ -81,7 +81,7 @@ yourself as a reviewer if it's not ready for merge yet. - All links must be full URLs, as the deprecation YAML files are used in two different projects. Do not use relative links. The generated doc is an exception to the relative link rule and currently uses absolute links only. - Make sure all links and anchors are correct. Do not link to the H1 (top) anchor on a docs page. - [ ] Code. Make sure any included code is wrapped in code blocks. -- [ ] Capitalization. Make sure to capitalize feature names. Stay consistent with the Documentation Style Guidance on [Capitalization](https://docs.gitlab.com/ee/development/documentation/styleguide.html#capitalization). +- [ ] Capitalization. Make sure to capitalize feature names. Stay consistent with the Documentation Style Guidance on [Capitalization](https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#capitalization). - [ ] Blank spaces. Remove unnecessary spaces (end of line spaces, double spaces, extra blank lines, and lines with only spaces). </details> diff --git a/.gitlab/merge_request_templates/Removals.md b/.gitlab/merge_request_templates/Removals.md deleted file mode 100644 index d2224a71e65..00000000000 --- a/.gitlab/merge_request_templates/Removals.md +++ /dev/null @@ -1,110 +0,0 @@ -<!-- Set the correct label and milestone using autocomplete for guidance. Please @mention only the DRI(s) for each stage or group rather than an entire department. --> - -/label ~"release post" ~"release post item" ~"Technical Writing" ~devops:: ~group:: ~"release post item::removal" -/label ~"type::maintenance" ~"maintenance::removal" -/milestone % -/assign `@EM/PM` (choose the DRI; remove backticks here, and below) - -**Be sure to link this MR to the relevant issues.** - -- Deprecation issue: -- Removal issue: -- MR that removed (or _will_ remove) the feature: - -If there is no relevant deprecation issue, hit pause and: - -- Review the [process for deprecating and removing features](https://about.gitlab.com/handbook/product/gitlab-the-product/#process-for-deprecating-and-removing-a-feature). -- Connect with the Product Manager DRI. - -Removals must be [announced as deprecations](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations) at least 2 milestones in advance of the planned removal date. - -If the removal creates a [breaking change](https://about.gitlab.com/handbook/product/gitlab-the-product/#deprecations-removals-and-breaking-changes), it can only be removed in a major "XX.0" release. - -**By the 10th**: Assign this MR to these team members as reviewers, and for approval: - -- Required: - - Product Group Manager or Director: `@PM` - - Engineering Manager: `@EM` - - Technical writer: `@TW` -- Optional: - - Product Designer(s): `@ProductDesigners` - - Product Marketing: `@PMM` - -**By 7:59 PM UTC 15th (11:59 AM PT)**: EM/PM assigns this MR to the TW reviewer for final review and merge: `@EM/PM` - -**By 7:59 AM UTC 18th (11:59 PM PT 17th)**: TW Reviewer updates Docs by merging this MR to `master`: `@TW` - ---- - -Please review: - -- The definitions of ["Deprecation", "End of Support", and "Removal"](https://docs.gitlab.com/ee/development/deprecation_guidelines/#terminology). -- The [guidelines for removals](https://about.gitlab.com/handbook/marketing/blog/release-posts/#removals). - -## EM/PM release post item checklist - -- [ ] Set yourself as the Assignee, meaning you are the DRI. -- [ ] If the removal is a [breaking change](https://about.gitlab.com/handbook/product/gitlab-the-product/#breaking-change), add label `breaking change`. -- [ ] Follow the process to [create a removal YAML file](https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-a-removal-entry). -- [ ] Add reviewers by the 10th. -- [ ] When ready to be merged and not later than the 15th, add the ~ready label and @ message the TW for final review and merge. - - Removal notices should not be merged before the code is removed from the product. Do not mark ~ready until the removal is complete, or you are certain it will be completed within the current milestone and released. If PMs are not sure, they should confirm with their Engineering Manager. - -## Reviewers - -When the content is ready for review, the Technical Writer and Engineering Manager _must_ -review it. Optional reviewers can include Product Marketing, Product Design, and the Product Leaders -for this area. Use the -[reviewers](https://docs.gitlab.com/ee/user/project/merge_requests/reviews/) -feature for all reviews. Reviewers will `approve` the MR and remove themselves from the reviewers list when their review is complete. - -- [ ] (Recommended) PMM -- [ ] (Optional) Product Designer -- [ ] (Optional) Group Manager or Director -- [ ] Required review and approval: [Technical Writer designated to the corresponding DevOps stage/group](https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments). - -### Tech writer review - -The TW should review according to the criteria listed below. Review a removal MR -with the same process as regular docs MRs. Add suggestions as needed, @ message -the PM to inform them the first review is complete, and remove -yourself as a reviewer if it's not yet ready for merge. - -**Removal notices should not be merged before the code is removed from the product.** - -<details> -<summary>Expand for Details</summary> - -- [ ] Title: - - Length limit: 7 words (not including articles or prepositions). - - Capitalization: ensure the title is [sentence cased](https://design.gitlab.com/content/punctuation#case). -- [ ] Consistency: - - Ensure that all resources (docs, removal, etc.) refer to the feature with the same term / feature name. -- [ ] Content: - - Make sure the removal is accurate based on your understanding. Look for typos or grammar mistakes. Work with PM and PMM to ensure a consistent GitLab style and tone for messaging, based on other features and removals. - - Review use of whitespace and bullet lists. Will the removal item be easily scannable when published? Consider adding line breaks or breaking content into bullets if you have more than a few sentences. - - Make sure there aren't acronyms readers may not understand per <https://about.gitlab.com/handbook/communication/#writing-style-guidelines>. -- [ ] Links: - - All links must be full URLs, as the removal YAML files are used in multiple projects. Do not use relative links. The generated doc is an exception to the relative link rule and currently uses absolute links only. - - Make sure all links and anchors are correct. Do not link to the H1 (top) anchor on a docs page. -- [ ] Code. Make sure any included code is wrapped in code blocks. -- [ ] Capitalization. Make sure to capitalize feature names. Stay consistent with the Documentation Style Guidance on [Capitalization](https://docs.gitlab.com/ee/development/documentation/styleguide.html#capitalization). -- [ ] Blank spaces. Remove unnecessary spaces (end of line spaces, double spaces, extra blank lines, and lines with only spaces). - -</details> - -When the PM indicates it is ready for merge and all issues have been addressed, start the merge process. - -#### Technical writer merge process - -The [removals doc's `.md` file](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/update/removals.md) -must be updated before this MR is merged: - -1. Check out the MR's branch (in the [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) project). -1. From the command line (in the branch), run `bin/rake gitlab:docs:compile_removals`. - If you want to double check that it worked, you can run `bin/rake gitlab:docs:check_removals` - to verify that the doc is up to date. -1. Commit the updated file and push the changes. -1. Set the merge request to auto-merge, or if the pipeline is already complete, merge. - -If you have trouble running the rake task, check the [troubleshooting steps](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecation-rake-task-troubleshooting). diff --git a/.gitlab/sast-ruleset.toml b/.gitlab/sast-ruleset.toml index 6bfb4618b73..e247d20da2c 100644 --- a/.gitlab/sast-ruleset.toml +++ b/.gitlab/sast-ruleset.toml @@ -7,4 +7,4 @@ type = "git" value = "https://gitlab.com/gitlab-com/gl-security/appsec/sast-custom-rules.git" ref = "refs/heads/main" - subdir = "appsec-pings" + subdir = "gitlab-sast-rules" |