Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2021-11-18 16:16:36 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2021-11-18 16:16:36 +0300
commit311b0269b4eb9839fa63f80c8d7a58f32b8138a0 (patch)
tree07e7870bca8aed6d61fdcc810731c50d2c40af47 /.gitlab
parent27909cef6c4170ed9205afa7426b8d3de47cbb0c (diff)
Add latest changes from gitlab-org/gitlab@14-5-stable-eev14.5.0-rc42
Diffstat (limited to '.gitlab')
-rw-r--r--.gitlab/CODEOWNERS13
-rw-r--r--.gitlab/ci/cache-repo.gitlab-ci.yml63
-rw-r--r--.gitlab/ci/docs.gitlab-ci.yml24
-rw-r--r--.gitlab/ci/frontend.gitlab-ci.yml14
-rw-r--r--.gitlab/ci/global.gitlab-ci.yml30
-rw-r--r--.gitlab/ci/qa-report.gitlab-ci.yml15
-rw-r--r--.gitlab/ci/qa.gitlab-ci.yml54
-rw-r--r--.gitlab/ci/rails.gitlab-ci.yml280
-rw-r--r--.gitlab/ci/reports.gitlab-ci.yml12
-rw-r--r--.gitlab/ci/review-apps/main.gitlab-ci.yml7
-rw-r--r--.gitlab/ci/review-apps/qa.gitlab-ci.yml40
-rw-r--r--.gitlab/ci/rules.gitlab-ci.yml244
-rw-r--r--.gitlab/ci/setup.gitlab-ci.yml32
-rw-r--r--.gitlab/ci/static-analysis.gitlab-ci.yml92
-rw-r--r--.gitlab/ci/workhorse.gitlab-ci.yml15
-rw-r--r--.gitlab/ci/yaml.gitlab-ci.yml2
-rw-r--r--.gitlab/issue_templates/Audit Event Proposal.md2
-rw-r--r--.gitlab/issue_templates/Bug.md6
-rw-r--r--.gitlab/issue_templates/Deprecations.md6
-rw-r--r--.gitlab/issue_templates/Empty state.md80
-rw-r--r--.gitlab/issue_templates/Experiment Implementation.md (renamed from .gitlab/issue_templates/Experimentation.md)2
-rw-r--r--.gitlab/issue_templates/Experiment Rollout.md11
-rw-r--r--.gitlab/issue_templates/Experiment Successful Cleanup.md5
-rw-r--r--.gitlab/issue_templates/Feature Flag Roll Out.md26
-rw-r--r--.gitlab/issue_templates/Feature Proposal - basic.md10
-rw-r--r--.gitlab/issue_templates/Feature Proposal - lean.md93
-rw-r--r--.gitlab/issue_templates/Feature proposal - detailed.md12
-rw-r--r--.gitlab/issue_templates/Geo Replicate a new Git repository type.md278
-rw-r--r--.gitlab/issue_templates/Geo Replicate a new blob type.md275
-rw-r--r--.gitlab/issue_templates/InfraDev.md2
-rw-r--r--.gitlab/issue_templates/Problem Validation.md8
-rw-r--r--.gitlab/issue_templates/Productivity Improvement.md2
-rw-r--r--.gitlab/issue_templates/Refactoring.md4
-rw-r--r--.gitlab/issue_templates/Security developer workflow.md2
-rw-r--r--.gitlab/issue_templates/Technical Evaluation.md2
-rw-r--r--.gitlab/merge_request_templates/Deprecations.md27
-rw-r--r--.gitlab/merge_request_templates/Documentation.md4
-rw-r--r--.gitlab/merge_request_templates/New End To End Test.md4
-rw-r--r--.gitlab/merge_request_templates/Pipeline Configuration.md4
-rw-r--r--.gitlab/merge_request_templates/Quarantine End to End Test.md2
40 files changed, 1094 insertions, 710 deletions
diff --git a/.gitlab/CODEOWNERS b/.gitlab/CODEOWNERS
index 64e74dd12e6..c6a89ddfa9c 100644
--- a/.gitlab/CODEOWNERS
+++ b/.gitlab/CODEOWNERS
@@ -26,7 +26,7 @@
/doc/administration/troubleshooting @marcel.amirault @marcia @eread
/doc/api/graphql/ @msedlakjakubowski @kpaizee
/doc/api/graphql/reference/ @kpaizee
-/doc/api/group_activity_analytics.md @msedlakjakubowski
+/doc/api/group_activity_analytics.md @fneill
/doc/ci/ @marcel.amirault @sselhorn
/doc/ci/environments/ @rdickenson
/doc/ci/services/ @sselhorn
@@ -34,7 +34,7 @@
/doc/development/ @marcia
/doc/development/documentation/ @cnorris @dianalogan
/doc/development/i18n/ @ngaskill
-/doc/development/value_stream_analytics.md @msedlakjakubowski
+/doc/development/value_stream_analytics.md @fneill
/doc/gitlab-basics/ @aqualls
/doc/install/ @marcel.amirault
/doc/operations/ @ngaskill @rdickenson
@@ -45,19 +45,19 @@
/doc/topics/autodevops/ @marcia
/doc/topics/git/ @aqualls
/doc/update/ @marcel.amirault @marcia
-/doc/user/analytics/ @msedlakjakubowski @ngaskill
+/doc/user/analytics/ @fneill @ngaskill
/doc/user/application_security/ @rdickenson
/doc/user/application_security/container_scanning/ @ngaskill
/doc/user/application_security/cluster_image_scanning/ @ngaskill
/doc/user/clusters/ @marcia
/doc/user/compliance/ @rdickenson @eread
/doc/user/group/ @msedlakjakubowski
-/doc/user/group/devops_adoption/ @msedlakjakubowski
+/doc/user/group/devops_adoption/ @fneill
/doc/user/group/epics/ @msedlakjakubowski
-/doc/user/group/insights/ @msedlakjakubowski
+/doc/user/group/insights/ @fneill
/doc/user/group/iterations/ @msedlakjakubowski
/doc/user/group/roadmap/ @msedlakjakubowski
-/doc/user/group/value_stream_analytics/ @msedlakjakubowski
+/doc/user/group/value_stream_analytics/ @fneill
/doc/user/infrastructure/ @marcia
/doc/user/packages/ @ngaskill
/doc/user/packages/infrastructure_registry/ @marcia
@@ -76,6 +76,7 @@
/doc/user/project/web_ide/index.md @aqualls
/doc/user/project/wiki/index.md @aqualls
/doc/user/search/ @marcia @aqualls
+/doc/user/workspace/ @fneill
[Docs Create]
/doc/administration/file_hooks.md @aqualls
diff --git a/.gitlab/ci/cache-repo.gitlab-ci.yml b/.gitlab/ci/cache-repo.gitlab-ci.yml
deleted file mode 100644
index 98c8c72ae3a..00000000000
--- a/.gitlab/ci/cache-repo.gitlab-ci.yml
+++ /dev/null
@@ -1,63 +0,0 @@
-# Builds a cached .tar.gz of the $CI_DEFAULT_BRANCH branch with full history and
-# uploads it to Google Cloud Storage. This archive is downloaded by a
-# script defined by a CI/CD variable named CI_PRE_CLONE_SCRIPT. This has
-# two benefits:
-#
-# 1. It speeds up builds. A 800 MB download only takes seconds.
-# 2. It significantly reduces load on the file server. Smaller deltas
-# means less time spent in git pack-objects.
-#
-# Since the destination directory of the archive depends on the project
-# ID, this is only run on GitLab.com.
-#
-# CI_REPO_CACHE_CREDENTIALS contains the Google Cloud service account
-# JSON for uploading to the gitlab-ci-git-repo-cache bucket. These
-# credentials are stored in the Production vault.
-#
-# Note that this bucket should be located in the same continent as the
-# runner, or network egress charges will apply:
-# https://cloud.google.com/storage/pricing
-cache-repo:
- extends: .cache-repo:rules
- image: gcr.io/google.com/cloudsdktool/cloud-sdk:alpine
- stage: sync
- variables:
- GIT_STRATEGY: none
- SHALLOW_CLONE_TAR_FILENAME: gitlab-master-shallow.tar
- FULL_CLONE_TAR_FILENAME: gitlab-master.tar
- before_script:
- - '[ -z "$CI_REPO_CACHE_CREDENTIALS" ] || gcloud auth activate-service-account --key-file=$CI_REPO_CACHE_CREDENTIALS'
- script:
- # Enable shallow repo caching unless the $DISABLE_SHALLOW_REPO_CACHING variable exists (in the case the shallow clone caching isn't working well)
- # The `git repack` call works around a Git bug with shallow clones: https://gitlab.com/gitlab-org/git/-/issues/86
- - if [ -z "$DISABLE_SHALLOW_REPO_CACHING" ]; then
- cd .. && rm -rf $CI_PROJECT_NAME;
- today=$(date +%Y-%m-%d);
- year=$(date +%Y);
- last_year=`expr $year - 1`;
- one_year_ago=$(echo $today | sed "s/$year/$last_year/");
- echo "Cloning $CI_REPOSITORY_URL into $CI_PROJECT_NAME with commits from $one_year_ago.";
- time git clone --progress --no-checkout --shallow-since=$one_year_ago $CI_REPOSITORY_URL $CI_PROJECT_NAME;
- cd $CI_PROJECT_NAME;
- time git repack -d;
- echo "Archiving $CI_PROJECT_NAME into /tmp/$SHALLOW_CLONE_TAR_FILENAME.";
- time git remote rm origin;
- time tar cf /tmp/$SHALLOW_CLONE_TAR_FILENAME .;
- echo "GZipping /tmp/$SHALLOW_CLONE_TAR_FILENAME.";
- time gzip /tmp/$SHALLOW_CLONE_TAR_FILENAME;
- [ -z "$CI_REPO_CACHE_CREDENTIALS" ] || (echo "Uploading /tmp/$SHALLOW_CLONE_TAR_FILENAME.gz to GCloud." && time gsutil cp /tmp/$SHALLOW_CLONE_TAR_FILENAME.gz gs://gitlab-ci-git-repo-cache/project-$CI_PROJECT_ID/$SHALLOW_CLONE_TAR_FILENAME.gz);
- fi
- # Disable the full repo caching unless the $DISABLE_SHALLOW_REPO_CACHING variable exists (in the case the shallow clone caching isn't working well)
- - if [ -n "$DISABLE_SHALLOW_REPO_CACHING" ]; then
- cd .. && rm -rf $CI_PROJECT_NAME;
- echo "Cloning $CI_REPOSITORY_URL into $CI_PROJECT_NAME.";
- time git clone --progress $CI_REPOSITORY_URL $CI_PROJECT_NAME;
- cd $CI_PROJECT_NAME;
- time git repack -d;
- echo "Archiving $CI_PROJECT_NAME into /tmp/$FULL_CLONE_TAR_FILENAME.";
- time git remote rm origin;
- time tar cf /tmp/$FULL_CLONE_TAR_FILENAME .;
- echo "GZipping /tmp/$FULL_CLONE_TAR_FILENAME.";
- time gzip /tmp/$FULL_CLONE_TAR_FILENAME;
- [ -z "$CI_REPO_CACHE_CREDENTIALS" ] || (echo "Uploading /tmp/$FULL_CLONE_TAR_FILENAME.gz to GCloud." && time gsutil cp /tmp/$FULL_CLONE_TAR_FILENAME.gz gs://gitlab-ci-git-repo-cache/project-$CI_PROJECT_ID/$FULL_CLONE_TAR_FILENAME.gz);
- fi
diff --git a/.gitlab/ci/docs.gitlab-ci.yml b/.gitlab/ci/docs.gitlab-ci.yml
index f4d8698f22d..6aa3d533315 100644
--- a/.gitlab/ci/docs.gitlab-ci.yml
+++ b/.gitlab/ci/docs.gitlab-ci.yml
@@ -44,8 +44,8 @@ docs-lint markdown:
- .default-retry
- .docs:rules:docs-lint
# When updating the image version here, update it in /scripts/lint-doc.sh too.
- image: registry.gitlab.com/gitlab-org/gitlab-docs/lint-markdown:alpine-3.14-vale-2.10.4-markdownlint-0.28.1
- stage: test
+ image: registry.gitlab.com/gitlab-org/gitlab-docs/lint-markdown:alpine-3.14-vale-2.12.0-markdownlint-0.29.0
+ stage: lint
needs: []
script:
- scripts/lint-doc.sh
@@ -53,8 +53,8 @@ docs-lint markdown:
docs-lint links:
extends:
- .docs:rules:docs-lint
- image: registry.gitlab.com/gitlab-org/gitlab-docs/lint-html:alpine-3.14-ruby-2.7.4-db71f027
- stage: test
+ image: registry.gitlab.com/gitlab-org/gitlab-docs/lint-html:alpine-3.14-ruby-2.7.4-0fae0f62
+ stage: lint
needs: []
script:
# Prepare docs for build
@@ -71,7 +71,21 @@ ui-docs-links lint:
extends:
- .docs:rules:docs-lint
- .static-analysis-base
- stage: test
+ - .ruby-cache
+ stage: lint
needs: []
script:
- bundle exec haml-lint -i DocumentationLinks
+
+docs-lint deprecations:
+ variables:
+ SETUP_DB: "false"
+ extends:
+ - .default-retry
+ - .rails-cache
+ - .default-before_script
+ - .docs:rules:deprecations
+ stage: lint
+ needs: []
+ script:
+ - bundle exec rake gitlab:docs:check_deprecations
diff --git a/.gitlab/ci/frontend.gitlab-ci.yml b/.gitlab/ci/frontend.gitlab-ci.yml
index 6974d63a49c..d3844d01213 100644
--- a/.gitlab/ci/frontend.gitlab-ci.yml
+++ b/.gitlab/ci/frontend.gitlab-ci.yml
@@ -11,7 +11,7 @@
- .default-retry
- .default-before_script
- .assets-compile-cache
- image: registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7-git-2.31-lfs-2.9-node-14.15-yarn-1.22-graphicsmagick-1.3.36
+ image: registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7-git-2.33-lfs-2.9-node-14.15-yarn-1.22-graphicsmagick-1.3.36
variables:
SETUP_DB: "false"
WEBPACK_VENDOR_DLL: "true"
@@ -133,6 +133,7 @@ update-storybook-yarn-cache:
paths:
- tmp/tests/frontend/
- knapsack/
+ - crystalball/
# Builds FOSS, and EE fixtures in the EE project.
# Builds FOSS fixtures in the FOSS project.
@@ -193,21 +194,10 @@ graphql-schema-dump:
# Disable warnings in browserslist which can break on backports
# https://github.com/browserslist/browserslist/blob/a287ec6/node.js#L367-L384
BROWSERSLIST_IGNORE_OLD_DATA: "true"
- SETUP_DB: "false"
before_script:
- - !reference [.default-before_script, before_script]
- *yarn-install
stage: test
-eslint-as-if-foss:
- extends:
- - .frontend-test-base
- - .frontend:rules:eslint-as-if-foss
- - .as-if-foss
- needs: []
- script:
- - run_timed_command "yarn run lint:eslint:all"
-
.jest-base:
extends: .frontend-test-base
script:
diff --git a/.gitlab/ci/global.gitlab-ci.yml b/.gitlab/ci/global.gitlab-ci.yml
index d0c26d60066..22bd3b1e2e9 100644
--- a/.gitlab/ci/global.gitlab-ci.yml
+++ b/.gitlab/ci/global.gitlab-ci.yml
@@ -44,6 +44,7 @@
prefix: "gitaly-binaries"
paths:
- tmp/tests/gitaly/_build/bin/
+ - tmp/tests/gitaly/_build/deps/git/install/
- tmp/tests/gitaly/config.toml
- tmp/tests/gitaly/gitaly2.config.toml
- tmp/tests/gitaly/internal/
@@ -139,6 +140,10 @@
- <<: *gitaly-binaries-cache
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
+.ruby-cache:
+ cache:
+ - *ruby-gems-cache
+
.rails-cache:
cache:
- *ruby-gems-cache
@@ -150,7 +155,12 @@
- *node-modules-cache
- *rubocop-cache
-.static-analysis-cache-push:
+.rubocop-job-cache:
+ cache:
+ - *ruby-gems-cache
+ - *rubocop-cache
+
+.rubocop-job-cache-push:
cache:
- *ruby-gems-cache # We don't push this cache as it's already rebuilt by `update-setup-test-env-cache`
- *rubocop-cache-push
@@ -203,7 +213,7 @@
- *storybook-node-modules-cache-push
.use-pg11:
- image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-11-graphicsmagick-1.3.36"
+ image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.33-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-11-graphicsmagick-1.3.36"
services:
- name: postgres:11.6
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
@@ -212,7 +222,7 @@
POSTGRES_HOST_AUTH_METHOD: trust
.use-pg12:
- image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-12-graphicsmagick-1.3.36"
+ image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.33-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-12-graphicsmagick-1.3.36"
services:
- name: postgres:12
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
@@ -221,7 +231,7 @@
POSTGRES_HOST_AUTH_METHOD: trust
.use-pg13:
- image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-13-graphicsmagick-1.3.36"
+ image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.33-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-13-graphicsmagick-1.3.36"
services:
- name: postgres:13
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
@@ -230,34 +240,34 @@
POSTGRES_HOST_AUTH_METHOD: trust
.use-pg11-ee:
- image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-11-graphicsmagick-1.3.36"
+ image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.33-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-11-graphicsmagick-1.3.36"
services:
- name: postgres:11.6
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
- name: redis:5.0-alpine
- - name: elasticsearch:7.11.1
+ - name: elasticsearch:7.14.2
command: ["elasticsearch", "-E", "discovery.type=single-node"]
variables:
POSTGRES_HOST_AUTH_METHOD: trust
.use-pg12-ee:
- image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-12-graphicsmagick-1.3.36"
+ image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.33-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-12-graphicsmagick-1.3.36"
services:
- name: postgres:12
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
- name: redis:5.0-alpine
- - name: elasticsearch:7.11.1
+ - name: elasticsearch:7.14.2
command: ["elasticsearch", "-E", "discovery.type=single-node"]
variables:
POSTGRES_HOST_AUTH_METHOD: trust
.use-pg13-ee:
- image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.31-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-13-graphicsmagick-1.3.36"
+ image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7.patched-golang-1.16-git-2.33-lfs-2.9-chrome-89-node-14.15-yarn-1.22-postgresql-13-graphicsmagick-1.3.36"
services:
- name: postgres:13
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
- name: redis:5.0-alpine
- - name: elasticsearch:7.11.1
+ - name: elasticsearch:7.14.2
command: ["elasticsearch", "-E", "discovery.type=single-node"]
variables:
POSTGRES_HOST_AUTH_METHOD: trust
diff --git a/.gitlab/ci/qa-report.gitlab-ci.yml b/.gitlab/ci/qa-report.gitlab-ci.yml
new file mode 100644
index 00000000000..61cbcfd58da
--- /dev/null
+++ b/.gitlab/ci/qa-report.gitlab-ci.yml
@@ -0,0 +1,15 @@
+test-reliability-report:
+ extends:
+ - .qa:rules:reliable-reports:schedule
+ image:
+ name: ${CI_REGISTRY_IMAGE}/gitlab-ee-qa:${CI_DEFAULT_BRANCH}
+ entrypoint: [""]
+ before_script:
+ - cd /home/gitlab/qa
+ script:
+ - echo "Generate report for 'staging-full' runs"
+ - bundle exec rake "reliable_spec_report[staging-full,30,true]"
+ - bundle exec rake "unreliable_spec_report[staging-full,30,true]"
+ - echo "Generate report for 'package-and-qa' runs"
+ - bundle exec rake "reliable_spec_report[package-and-qa,30,true]"
+ - bundle exec rake "unreliable_spec_report[package-and-qa,30,true]"
diff --git a/.gitlab/ci/qa.gitlab-ci.yml b/.gitlab/ci/qa.gitlab-ci.yml
index 88e732c2e75..74fc5f2cdc0 100644
--- a/.gitlab/ci/qa.gitlab-ci.yml
+++ b/.gitlab/ci/qa.gitlab-ci.yml
@@ -55,14 +55,18 @@ update-qa-cache:
before_script:
- source scripts/utils.sh
- install_gitlab_gem
+ - tooling/bin/find_change_diffs ${CHANGES_DIFFS_DIR}
script:
- - ./scripts/trigger-build omnibus
-
-package-and-qa:
- extends:
- - .package-and-qa-base
- - .qa:rules:package-and-qa
- # This job often times out, so temporarily use private runners and a long timeout: https://gitlab.com/gitlab-org/gitlab/-/issues/238563
+ - |
+ tooling/bin/qa/package_and_qa_check ${CHANGES_DIFFS_DIR} && exit_code=$?
+ if [ $exit_code -eq 0 ]; then
+ ./scripts/trigger-build omnibus
+ elif [ $exit_code -eq 1 ]; then
+ exit 1
+ else
+ echo "Downstream jobs will not be triggered because package_and_qa_check exited with code: $exit_code"
+ fi
+ # These jobs often time out, so temporarily use private runners and a long timeout: https://gitlab.com/gitlab-org/gitlab/-/issues/238563
tags:
- prm
timeout: 4h
@@ -71,3 +75,39 @@ package-and-qa:
artifacts: false
- job: build-assets-image
artifacts: false
+ - detect-tests
+ artifacts:
+ expire_in: 7d
+ paths:
+ - ${CHANGES_FILE}
+ - ${CHANGES_DIFFS_DIR}/*
+ variables:
+ CHANGES_FILE: tmp/changed_files.txt
+ CHANGES_DIFFS_DIR: tmp/diffs
+
+.package-and-qa-ff-base:
+ script:
+ - export GITLAB_QA_OPTIONS="--set-feature-flags $(scripts/changed-feature-flags --files $(cat $CHANGES_FILE | tr ' ' ',') --state $QA_FF_STATE)"
+ - echo $GITLAB_QA_OPTIONS
+ - ./scripts/trigger-build omnibus
+
+package-and-qa:
+ extends:
+ - .package-and-qa-base
+ - .qa:rules:package-and-qa
+
+package-and-qa-ff-enabled:
+ extends:
+ - .package-and-qa-base
+ - .package-and-qa-ff-base
+ - .qa:rules:package-and-qa:feature-flags
+ variables:
+ QA_FF_STATE: "enable"
+
+package-and-qa-ff-disabled:
+ extends:
+ - .package-and-qa-base
+ - .package-and-qa-ff-base
+ - .qa:rules:package-and-qa:feature-flags
+ variables:
+ QA_FF_STATE: "disable"
diff --git a/.gitlab/ci/rails.gitlab-ci.yml b/.gitlab/ci/rails.gitlab-ci.yml
index 00f65ab7ca8..1fb1c887e56 100644
--- a/.gitlab/ci/rails.gitlab-ci.yml
+++ b/.gitlab/ci/rails.gitlab-ci.yml
@@ -6,11 +6,6 @@
- .default-before_script
- .rails-cache
-.minimal-bundle-install:
- script:
- - export BUNDLE_WITHOUT="${BUNDLE_WITHOUT}:default:test:puma:kerberos:metrics:omnibus:ed25519"
- - bundle_install_script
-
.base-script:
script:
# Only install knapsack after bundle install! Otherwise oddly some native
@@ -26,6 +21,8 @@
.decomposed-database-rspec:
variables:
DECOMPOSED_DB: "true"
+ GITLAB_LOAD_BALANCING_REUSE_PRIMARY_ci: "main"
+ GITLAB_USE_MODEL_LOAD_BALANCING: "true"
.rspec-base:
extends: .rails-job-base
@@ -77,6 +74,12 @@
- .use-pg12
needs: ["setup-test-env", "retrieve-tests-metadata", "compile-test-assets as-if-foss", "detect-tests"]
+.rspec-base-pg12-as-if-jh:
+ extends:
+ - .rspec-base
+ - .use-pg12
+ needs: ["setup-test-env", "retrieve-tests-metadata", "compile-test-assets as-if-jh", "detect-tests", "add-jh-folder"]
+
.rspec-base-pg13:
extends:
- .rspec-base
@@ -92,6 +95,11 @@
- .rspec-base
- .use-pg12-ee
+.rspec-jh-base-pg12:
+ extends:
+ - .rspec-base-pg12-as-if-jh
+ - .use-pg12-ee
+
.rspec-ee-base-pg13:
extends:
- .rspec-base
@@ -113,6 +121,12 @@
- .rspec-ee-base-geo
- .use-pg12-ee
+.rspec-jh-base-geo-pg12:
+ extends:
+ - .rspec-jh-base-pg12
+ script:
+ - !reference [.rspec-ee-base-geo, script]
+
.rspec-ee-base-geo-pg13:
extends:
- .rspec-ee-base-geo
@@ -165,10 +179,9 @@ setup-test-env:
extends:
- .rails-job-base
- .setup-test-env-cache
- - .rails:rules:code-backstage-qa
+ - .rails:rules:setup-test-env
stage: prepare
variables:
- GITLAB_TEST_EAGER_LOAD: "0"
SETUP_DB: "false"
script:
- run_timed_command "scripts/setup-test-env"
@@ -181,6 +194,7 @@ setup-test-env:
paths:
- config/secrets.yml
- tmp/tests/gitaly/_build/bin/
+ - tmp/tests/gitaly/_build/deps/git/install
- tmp/tests/gitaly/config.toml
- tmp/tests/gitaly/gitaly2.config.toml
- tmp/tests/gitaly/internal/
@@ -223,11 +237,11 @@ update-gitaly-binaries-cache:
.coverage-base:
extends:
- .default-retry
- - .default-before_script
- .coverage-cache
- variables:
- SETUP_DB: "false"
- USE_BUNDLE_INSTALL: "false"
+ before_script:
+ - source scripts/utils.sh
+ - export BUNDLE_WITHOUT="${BUNDLE_WITHOUT}:default:test:puma:kerberos:metrics:omnibus:ed25519"
+ - bundle_install_script
rspec migration pg12:
extends:
@@ -324,26 +338,16 @@ rspec fast_spec_helper minimal:
- .minimal-rspec-tests
- .rails:rules:ee-and-foss-fast_spec_helper:minimal
-db:migrate:reset:
+db:rollback:
extends: .db-job-base
script:
- - bundle exec rake db:migrate:reset
-
-db:check-schema:
- extends:
- - .db-job-base
- - .rails:rules:ee-mr-and-default-branch-only
- script:
- - source scripts/schema_changed.sh
- - scripts/validate_migration_timestamps
+ - bundle exec rake db:migrate VERSION=20181228175414
+ - bundle exec rake db:migrate SKIP_SCHEMA_VERSION_CHECK=true
-db:check-migrations:
- extends:
- - .db-job-base
- - .rails:rules:ee-and-foss-mr-with-migration
+db:migrate:reset:
+ extends: .db-job-base
script:
- - scripts/validate_migration_schema
- allow_failure: true
+ - bundle exec rake db:migrate:reset
db:migrate-from-previous-major-version:
extends: .db-job-base
@@ -352,7 +356,8 @@ db:migrate-from-previous-major-version:
SETUP_DB: "false"
PROJECT_TO_CHECKOUT: "gitlab-foss"
TAG_TO_CHECKOUT: "v13.12.9"
- script:
+ before_script:
+ - !reference [.default-before_script, before_script]
- '[[ -d "ee/" ]] || export PROJECT_TO_CHECKOUT="gitlab"'
- '[[ -d "ee/" ]] || export TAG_TO_CHECKOUT="${TAG_TO_CHECKOUT}-ee"'
- retry 'git fetch https://gitlab.com/gitlab-org/$PROJECT_TO_CHECKOUT.git $TAG_TO_CHECKOUT'
@@ -361,13 +366,28 @@ db:migrate-from-previous-major-version:
- run_timed_command "bundle exec rake db:drop db:create db:structure:load db:migrate db:seed_fu"
- git checkout -f $CI_COMMIT_SHA
- SETUP_DB=false USE_BUNDLE_INSTALL=true bash scripts/prepare_build.sh
+ script:
- run_timed_command "bundle exec rake db:migrate"
-db:rollback:
- extends: .db-job-base
+db:check-schema:
+ extends:
+ - db:migrate-from-previous-major-version
+ - .rails:rules:ee-mr-and-default-branch-only
+ variables:
+ TAG_TO_CHECKOUT: "v14.4.0"
script:
- - bundle exec rake db:migrate VERSION=20181228175414
- - bundle exec rake db:migrate SKIP_SCHEMA_VERSION_CHECK=true
+ - run_timed_command "bundle exec rake db:migrate"
+ - scripts/schema_changed.sh
+ - scripts/validate_migration_timestamps
+
+db:check-migrations:
+ extends:
+ - .db-job-base
+ - .rails:rules:ee-and-foss-mr-with-migration
+ script:
+ - git fetch origin $CI_MERGE_REQUEST_TARGET_BRANCH_NAME:$CI_MERGE_REQUEST_TARGET_BRANCH_NAME --depth 20
+ - scripts/validate_migration_schema
+ allow_failure: true
db:gitlabcom-database-testing:
extends: .rails:rules:db:gitlabcom-database-testing
@@ -457,21 +477,38 @@ rspec:coverage:
# so we use `dependencies` here.
dependencies:
- setup-test-env
+ # FOSS/EE jobs
- rspec migration pg12
- rspec unit pg12
- rspec integration pg12
- rspec system pg12
+ # FOSS/EE minimal jobs
+ - rspec migration pg12 minimal
+ - rspec unit pg12 minimal
+ - rspec integration pg12 minimal
+ - rspec system pg12 minimal
+ # EE jobs
- rspec-ee migration pg12
- rspec-ee unit pg12
- rspec-ee integration pg12
- rspec-ee system pg12
+ # EE minimal jobs
+ - rspec-ee migration pg12 minimal
+ - rspec-ee unit pg12 minimal
+ - rspec-ee integration pg12 minimal
+ - rspec-ee system pg12 minimal
+ # Geo jobs
- rspec-ee unit pg12 geo
- rspec-ee integration pg12 geo
- rspec-ee system pg12 geo
+ # Geo minimal jobs
+ - rspec-ee unit pg12 geo minimal
+ - rspec-ee integration pg12 geo minimal
+ - rspec-ee system pg12 geo minimal
+ # Memory jobs
- memory-static
- memory-on-boot
script:
- - !reference [.minimal-bundle-install, script]
- run_timed_command "bundle exec scripts/merge-simplecov"
- run_timed_command "bundle exec scripts/gather-test-memory-data"
coverage: '/LOC \((\d+\.\d+%)\) covered.$/'
@@ -490,15 +527,67 @@ rspec:feature-flags:
- .coverage-base
- .rails:rules:rspec-feature-flags
stage: post-test
- needs: ["static-analysis"]
+ needs:
+ - job: "feature-flags-usage"
+ - job: "haml-lint foss"
+ - job: "haml-lint ee"
+ optional: true
script:
- - !reference [.minimal-bundle-install, script]
- if [ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]; then
run_timed_command "bundle exec scripts/used-feature-flags" || (scripts/slack master-broken "☠️ \`${CI_JOB_NAME}\` failed! ☠️ See ${CI_JOB_URL}" ci_failing "GitLab Bot" && exit 1);
else
run_timed_command "bundle exec scripts/used-feature-flags";
fi
+rspec:skipped-flaky-tests-report:
+ extends:
+ - .default-retry
+ - .rails:rules:skipped-flaky-tests-report
+ image: ruby:2.7-alpine
+ stage: post-test
+ # We cannot use needs since it would mean needing 84 jobs (since most are parallelized)
+ # so we use `dependencies` here.
+ dependencies:
+ # FOSS/EE jobs
+ - rspec migration pg12
+ - rspec unit pg12
+ - rspec integration pg12
+ - rspec system pg12
+ # FOSS/EE minimal jobs
+ - rspec migration pg12 minimal
+ - rspec unit pg12 minimal
+ - rspec integration pg12 minimal
+ - rspec system pg12 minimal
+ # EE jobs
+ - rspec-ee migration pg12
+ - rspec-ee unit pg12
+ - rspec-ee integration pg12
+ - rspec-ee system pg12
+ # EE minimal jobs
+ - rspec-ee migration pg12 minimal
+ - rspec-ee unit pg12 minimal
+ - rspec-ee integration pg12 minimal
+ - rspec-ee system pg12 minimal
+ # Geo jobs
+ - rspec-ee unit pg12 geo
+ - rspec-ee integration pg12 geo
+ - rspec-ee system pg12 geo
+ # Geo minimal jobs
+ - rspec-ee unit pg12 geo minimal
+ - rspec-ee integration pg12 geo minimal
+ - rspec-ee system pg12 geo minimal
+ variables:
+ SKIPPED_FLAKY_TESTS_REPORT: skipped_flaky_tests_report.txt
+ before_script:
+ - 'echo "SKIP_FLAKY_TESTS_AUTOMATICALLY: $SKIP_FLAKY_TESTS_AUTOMATICALLY"'
+ - mkdir -p rspec_flaky
+ script:
+ - find rspec_flaky/ -type f -name 'skipped_flaky_tests_*_report.txt' -exec cat {} + >> "${SKIPPED_FLAKY_TESTS_REPORT}"
+ artifacts:
+ expire_in: 31d
+ paths:
+ - ${SKIPPED_FLAKY_TESTS_REPORT}
+
# EE/FOSS: default refs (MRs, default branch, schedules) jobs #
#######################################################
@@ -577,6 +666,31 @@ rspec system pg12-as-if-foss decomposed:
- .decomposed-database-rspec
- .rails:rules:decomposed-databases
+rspec migration pg12-as-if-jh:
+ extends:
+ - .rspec-base-pg12-as-if-jh
+ - .rspec-base-migration
+ - .rails:rules:as-if-jh-rspec
+ - .rspec-migration-parallel
+
+rspec unit pg12-as-if-jh:
+ extends:
+ - .rspec-base-pg12-as-if-jh
+ - .rails:rules:as-if-jh-rspec
+ - .rspec-unit-parallel
+
+rspec integration pg12-as-if-jh:
+ extends:
+ - .rspec-base-pg12-as-if-jh
+ - .rails:rules:as-if-jh-rspec
+ - .rspec-integration-parallel
+
+rspec system pg12-as-if-jh:
+ extends:
+ - .rspec-base-pg12-as-if-jh
+ - .rails:rules:as-if-jh-rspec
+ - .rspec-system-parallel
+
rspec-ee migration pg12:
extends:
- .rspec-ee-base-pg12
@@ -684,6 +798,83 @@ rspec-ee system pg12 geo minimal:
- .minimal-rspec-tests
- .rails:rules:ee-only-system:minimal
+rspec-ee migration pg12-as-if-jh:
+ extends:
+ - .rspec-jh-base-pg12
+ - .rspec-base-migration
+ - .rails:rules:as-if-jh-rspec
+ - .rspec-ee-migration-parallel
+
+rspec-ee unit pg12-as-if-jh:
+ extends:
+ - .rspec-jh-base-pg12
+ - .rails:rules:as-if-jh-rspec
+ - .rspec-ee-unit-parallel
+
+rspec-ee integration pg12-as-if-jh:
+ extends:
+ - .rspec-jh-base-pg12
+ - .rails:rules:as-if-jh-rspec
+ - .rspec-ee-integration-parallel
+
+rspec-ee system pg12-as-if-jh:
+ extends:
+ - .rspec-jh-base-pg12
+ - .rails:rules:as-if-jh-rspec
+ - .rspec-ee-system-parallel
+
+rspec-ee unit pg12-as-if-jh geo:
+ extends:
+ - .rspec-jh-base-geo-pg12
+ - .rails:rules:as-if-jh-rspec
+ - .rspec-ee-unit-geo-parallel
+
+rspec-ee integration pg12-as-if-jh geo:
+ extends:
+ - .rspec-jh-base-geo-pg12
+ - .rails:rules:as-if-jh-rspec
+
+rspec-ee system pg12-as-if-jh geo:
+ extends:
+ - .rspec-jh-base-geo-pg12
+ - .rails:rules:as-if-jh-rspec
+
+rspec-jh migration pg12-as-if-jh:
+ extends:
+ - .rspec-jh-base-pg12
+ - .rspec-base-migration
+ - .rails:rules:as-if-jh-rspec
+
+rspec-jh unit pg12-as-if-jh:
+ extends:
+ - .rspec-jh-base-pg12
+ - .rails:rules:as-if-jh-rspec
+
+rspec-jh integration pg12-as-if-jh:
+ extends:
+ - .rspec-jh-base-pg12
+ - .rails:rules:as-if-jh-rspec
+
+rspec-jh system pg12-as-if-jh:
+ extends:
+ - .rspec-jh-base-pg12
+ - .rails:rules:as-if-jh-rspec
+
+rspec-jh unit pg12-as-if-jh geo:
+ extends:
+ - .rspec-jh-base-geo-pg12
+ - .rails:rules:as-if-jh-rspec
+
+rspec-jh integration pg12-as-if-jh geo:
+ extends:
+ - .rspec-jh-base-geo-pg12
+ - .rails:rules:as-if-jh-rspec
+
+rspec-jh system pg12-as-if-jh geo:
+ extends:
+ - .rspec-jh-base-geo-pg12
+ - .rails:rules:as-if-jh-rspec
+
db:rollback geo:
extends:
- db:rollback
@@ -884,5 +1075,24 @@ fail-pipeline-early:
- install_gitlab_gem
script:
- fail_pipeline_early
+
+rspec rspec-pg12-rerun-previous-failed-tests:
+ extends:
+ - .rspec-base-pg12
+ - .rails:rules:rerun-previous-failed-tests
+ stage: test
+ needs: ["setup-test-env", "compile-test-assets", "detect-previous-failed-tests"]
+ script:
+ - !reference [.base-script, script]
+ - rspec_rerun_previous_failed_tests tmp/previous_failed_tests/rspec_failed_files.txt
+
+rspec rspec-ee-pg12-rerun-previous-failed-tests:
+ extends:
+ - "rspec rspec-pg12-rerun-previous-failed-tests"
+ - .rspec-ee-base-pg12
+ script:
+ - !reference [.base-script, script]
+ - rspec_rerun_previous_failed_tests tmp/previous_failed_tests/rspec_ee_failed_files.txt
+
# EE: Canonical MR pipelines
##################################################
diff --git a/.gitlab/ci/reports.gitlab-ci.yml b/.gitlab/ci/reports.gitlab-ci.yml
index b581cf83d56..33efc3c4788 100644
--- a/.gitlab/ci/reports.gitlab-ci.yml
+++ b/.gitlab/ci/reports.gitlab-ci.yml
@@ -9,6 +9,7 @@ code_quality:
extends:
- .default-retry
- .use-docker-in-docker
+ stage: lint
artifacts:
paths:
- gl-code-quality-report.json # GitLab-specific
@@ -20,6 +21,7 @@ code_quality:
extends:
- .default-retry
- sast
+ stage: lint
needs: []
artifacts:
paths:
@@ -54,6 +56,7 @@ gosec-sast:
.secret-analyzer:
extends: .default-retry
+ stage: lint
needs: []
artifacts:
paths:
@@ -68,6 +71,7 @@ secret_detection:
extends:
- .default-retry
- dependency_scanning
+ stage: lint
needs: []
variables:
DS_EXCLUDED_PATHS: "qa/qa/ee/fixtures/secure_premade_reports, spec, ee/spec, tmp" # GitLab-specific
@@ -92,6 +96,13 @@ retire-js-dependency_scanning:
gemnasium-python-dependency_scanning:
rules: !reference [".reports:rules:gemnasium-python-dependency_scanning", rules]
+yarn-audit-dependency_scanning:
+ extends: .ds-analyzer
+ image: "registry.gitlab.com/gitlab-org/security-products/analyzers/npm-audit:1.4.0"
+ variables:
+ TOOL: yarn
+ rules: !reference [".reports:rules:yarn-audit-dependency_scanning", rules]
+
# Analyze dependencies for malicious behavior
# See https://gitlab.com/gitlab-com/gl-security/security-research/package-hunter
.package_hunter-base:
@@ -134,6 +145,7 @@ package_hunter-bundler:
license_scanning:
extends: .default-retry
+ stage: lint
needs: []
artifacts:
expire_in: 1 week # GitLab-specific
diff --git a/.gitlab/ci/review-apps/main.gitlab-ci.yml b/.gitlab/ci/review-apps/main.gitlab-ci.yml
index 6fe9e39cb82..670175a6f11 100644
--- a/.gitlab/ci/review-apps/main.gitlab-ci.yml
+++ b/.gitlab/ci/review-apps/main.gitlab-ci.yml
@@ -35,10 +35,11 @@ review-build-cng:
extends:
- .default-retry
image: registry.gitlab.com/gitlab-org/gitlab-build-images:gitlab-helm3.5-kubectl1.17
+ resource_group: "review/${CI_COMMIT_REF_NAME}"
variables:
HOST_SUFFIX: "${CI_ENVIRONMENT_SLUG}"
DOMAIN: "-${CI_ENVIRONMENT_SLUG}.${REVIEW_APPS_DOMAIN}"
- GITLAB_HELM_CHART_REF: "v5.2.1"
+ GITLAB_HELM_CHART_REF: "v5.4.1"
environment:
name: review/${CI_COMMIT_REF_SLUG}${FREQUENCY}
url: https://gitlab-${CI_ENVIRONMENT_SLUG}.${REVIEW_APPS_DOMAIN}
@@ -51,7 +52,6 @@ review-deploy:
- .review:rules:review-deploy
stage: deploy
needs: ["review-build-cng"]
- resource_group: "review/${CI_COMMIT_REF_NAME}"
before_script:
- export GITLAB_SHELL_VERSION=$(<GITLAB_SHELL_VERSION)
- export GITALY_VERSION=$(<GITALY_SERVER_VERSION)
@@ -101,6 +101,7 @@ review-stop:
extends:
- .review-stop-base
- .review:rules:review-stop
- stage: post-qa
+ stage: deploy
+ needs: []
script:
- delete_k8s_release_namespace
diff --git a/.gitlab/ci/review-apps/qa.gitlab-ci.yml b/.gitlab/ci/review-apps/qa.gitlab-ci.yml
index 6b9d4feb3c8..07f8bd4e5fd 100644
--- a/.gitlab/ci/review-apps/qa.gitlab-ci.yml
+++ b/.gitlab/ci/review-apps/qa.gitlab-ci.yml
@@ -14,6 +14,7 @@
GITLAB_PASSWORD: "${REVIEW_APPS_ROOT_PASSWORD}"
GITLAB_ADMIN_USERNAME: "root"
GITLAB_ADMIN_PASSWORD: "${REVIEW_APPS_ROOT_PASSWORD}"
+ GITLAB_QA_ADMIN_ACCESS_TOKEN: "${REVIEW_APPS_ROOT_TOKEN}"
GITHUB_ACCESS_TOKEN: "${REVIEW_APPS_QA_GITHUB_ACCESS_TOKEN}"
EE_LICENSE: "${REVIEW_APPS_EE_LICENSE}"
SIGNUP_DISABLED: "true"
@@ -31,9 +32,23 @@
expire_in: 7 days
when: always
+.parallel-qa-base:
+ parallel: 5
+ script:
+ - export KNAPSACK_REPORT_PATH=knapsack/master_report.json
+ - export KNAPSACK_TEST_FILE_PATTERN=qa/specs/features/**/*_spec.rb
+ - |
+ bin/test "${QA_SCENARIO}" "${CI_ENVIRONMENT_URL}" \
+ -- \
+ --color --format documentation \
+ --format RspecJunitFormatter --out tmp/rspec.xml
+ artifacts:
+ reports:
+ junit: qa/tmp/rspec.xml
+
.allure-report-base:
image:
- name: ${GITLAB_DEPENDENCY_PROXY}andrcuns/allure-report-publisher:0.3.6
+ name: ${GITLAB_DEPENDENCY_PROXY}andrcuns/allure-report-publisher:0.4.1
entrypoint: [""]
stage: post-qa
variables:
@@ -64,24 +79,23 @@ review-qa-smoke:
script:
- bin/test Test::Instance::Smoke "${CI_ENVIRONMENT_URL}"
+review-qa-reliable:
+ extends:
+ - .review-qa-base
+ - .review:rules:review-qa-reliable
+ - .parallel-qa-base
+ variables:
+ QA_RUN_TYPE: review-qa-reliable
+ QA_SCENARIO: Test::Instance::Reliable
+
review-qa-all:
extends:
- .review-qa-base
- .review:rules:review-qa-all
+ - .parallel-qa-base
variables:
QA_RUN_TYPE: review-qa-all
- parallel: 5
- script:
- - export KNAPSACK_REPORT_PATH=knapsack/master_report.json
- - export KNAPSACK_TEST_FILE_PATTERN=qa/specs/features/**/*_spec.rb
- - |
- bin/test Test::Instance::All "${CI_ENVIRONMENT_URL}" \
- -- \
- --color --format documentation \
- --format RspecJunitFormatter --out tmp/rspec.xml
- artifacts:
- reports:
- junit: qa/tmp/rspec.xml
+ QA_SCENARIO: Test::Instance::All
review-performance:
extends:
diff --git a/.gitlab/ci/rules.gitlab-ci.yml b/.gitlab/ci/rules.gitlab-ci.yml
index 8ddcf9c2094..cf53e810d01 100644
--- a/.gitlab/ci/rules.gitlab-ci.yml
+++ b/.gitlab/ci/rules.gitlab-ci.yml
@@ -58,6 +58,9 @@
.if-merge-request-labels-run-decomposed: &if-merge-request-labels-run-decomposed
if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-decomposed/'
+.if-merge-request-labels-run-review-app: &if-merge-request-labels-run-review-app
+ if: '$CI_MERGE_REQUEST_LABELS =~ /pipeline:run-review-app/'
+
.if-security-merge-request: &if-security-merge-request
if: '$CI_PROJECT_NAMESPACE == "gitlab-org/security" && $CI_MERGE_REQUEST_IID'
@@ -91,9 +94,6 @@
.if-dot-com-ee-nightly-schedule-child-pipeline: &if-dot-com-ee-nightly-schedule-child-pipeline
if: '$CI_SERVER_HOST == "gitlab.com" && $CI_PROJECT_PATH == "gitlab-org/gitlab" && $CI_PIPELINE_SOURCE == "parent_pipeline" && $FREQUENCY == "nightly"'
-.if-cache-credentials-schedule: &if-cache-credentials-schedule
- if: '$CI_REPO_CACHE_CREDENTIALS && $CI_PIPELINE_SOURCE == "schedule"'
-
.if-dot-com-gitlab-org-default-branch: &if-dot-com-gitlab-org-default-branch
if: '$CI_SERVER_HOST == "gitlab.com" && $CI_PROJECT_NAMESPACE == "gitlab-org" && $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH'
@@ -103,6 +103,9 @@
.if-dot-com-gitlab-org-and-security-merge-request: &if-dot-com-gitlab-org-and-security-merge-request
if: '$CI_SERVER_HOST == "gitlab.com" && $CI_PROJECT_NAMESPACE =~ /^gitlab-org($|\/security$)/ && $CI_MERGE_REQUEST_IID'
+.if-dot-com-gitlab-org-and-security-merge-request-manual-ff-package-and-qa: &if-dot-com-gitlab-org-and-security-merge-request-manual-ff-package-and-qa
+ if: '$CI_SERVER_HOST == "gitlab.com" && $CI_PROJECT_NAMESPACE =~ /^gitlab-org($|\/security$)/ && $CI_MERGE_REQUEST_IID && $QA_MANUAL_FF_PACKAGE_AND_QA'
+
.if-dot-com-gitlab-org-and-security-tag: &if-dot-com-gitlab-org-and-security-tag
if: '$CI_SERVER_HOST == "gitlab.com" && $CI_PROJECT_NAMESPACE =~ /^gitlab-org($|\/security$)/ && $CI_COMMIT_TAG'
@@ -112,6 +115,9 @@
.if-security-pipeline-merge-result: &if-security-pipeline-merge-result
if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH && $CI_PROJECT_NAMESPACE == "gitlab-org/security" && $GITLAB_USER_LOGIN == "gitlab-release-tools-bot"'
+.if-skip-flaky-tests-automatically: &if-skip-flaky-tests-automatically
+ if: '$SKIP_FLAKY_TESTS_AUTOMATICALLY == "true"'
+
####################
# Changes patterns #
####################
@@ -159,11 +165,19 @@
- ".markdownlint.yml"
- "scripts/lint-doc.sh"
+.docs-deprecations-patterns: &docs-deprecations-patterns
+ - "doc/deprecations/index.md"
+ - "data/deprecations/*.yml"
+ - "data/deprecations/templates/_deprecation_template.md.erb"
+ - "lib/tasks/gitlab/docs/compile_deprecations.rake"
+ - "tooling/deprecations/docs.rb"
+
.bundler-patterns: &bundler-patterns
- '{Gemfile.lock,*/Gemfile.lock,*/*/Gemfile.lock}'
.nodejs-patterns: &nodejs-patterns
- '{package.json,*/package.json,*/*/package.json}'
+ - '{yarn.lock,*/yarn.lock,*/*/yarn.lock}'
.python-patterns: &python-patterns
- '{requirements.txt,*/requirements.txt,*/*/requirements.txt}'
@@ -211,6 +225,7 @@
.startup-css-patterns: &startup-css-patterns
- "{,ee/,jh/}app/assets/stylesheets/startup/**/*"
+# Backend patterns + .ci-patterns
.backend-patterns: &backend-patterns
- "{,jh/}Gemfile{,.lock}"
- "Rakefile"
@@ -224,6 +239,7 @@
- ".gitlab/ci/**/*"
- "*_VERSION"
+# DB patterns + .ci-patterns
.db-patterns: &db-patterns
- "{,ee/,jh/}{,spec/}{db,migrations}/**/*"
- "{,ee/,jh/}{,spec/}lib/{,ee/,jh/}gitlab/database/**/*"
@@ -252,6 +268,11 @@
- "{,ee/,jh/}spec/**/*"
- "{,spec/}tooling/**/*"
+.qa-patterns: &qa-patterns
+ - ".dockerignore"
+ - "qa/**/*"
+
+# Code patterns + .ci-patterns + .workhorse-patterns
.code-patterns: &code-patterns
- "{package.json,yarn.lock}"
- ".browserslistrc"
@@ -260,9 +281,6 @@
- ".csscomb.json"
- "Dockerfile.assets"
- "vendor/assets/**/*"
- # CI changes
- - ".gitlab-ci.yml"
- - ".gitlab/ci/**/*"
- ".{eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}"
- ".{codeclimate,eslintrc,gitlab-ci,haml-lint,haml-lint_todo,rubocop,rubocop_todo,rubocop_manual_todo}.yml"
- "*_VERSION"
@@ -273,11 +291,11 @@
- "{,ee/,jh/}{app,bin,config,db,generator_templates,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*"
- "doc/api/graphql/reference/*" # Files in this folder are auto-generated
- "data/whats_new/*.yml"
+ # CI changes
+ - ".gitlab-ci.yml"
+ - ".gitlab/ci/**/*"
-.qa-patterns: &qa-patterns
- - ".dockerignore"
- - "qa/**/*"
-
+# .code-patterns + .backstage-patterns
.code-backstage-patterns: &code-backstage-patterns
- "{package.json,yarn.lock}"
- ".browserslistrc"
@@ -286,9 +304,6 @@
- ".csscomb.json"
- "Dockerfile.assets"
- "vendor/assets/**/*"
- # CI changes
- - ".gitlab-ci.yml"
- - ".gitlab/ci/**/*"
- ".{eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}"
- ".{codeclimate,eslintrc,gitlab-ci,haml-lint,haml-lint_todo,rubocop,rubocop_todo,rubocop_manual_todo}.yml"
- "*_VERSION"
@@ -299,6 +314,9 @@
- "{,ee/,jh/}{app,bin,config,db,generator_templates,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*"
- "doc/api/graphql/reference/*" # Files in this folder are auto-generated
- "data/whats_new/*.yml"
+ # CI changes
+ - ".gitlab-ci.yml"
+ - ".gitlab/ci/**/*"
# Backstage changes
- "Dangerfile"
- "danger/**/*"
@@ -307,6 +325,7 @@
- "{,ee/,jh/}spec/**/*"
- "{,spec/}tooling/**/*"
+# .code-patterns + .qa-patterns
.code-qa-patterns: &code-qa-patterns
- "{package.json,yarn.lock}"
- ".browserslistrc"
@@ -315,9 +334,6 @@
- ".csscomb.json"
- "Dockerfile.assets"
- "vendor/assets/**/*"
- # CI changes
- - ".gitlab-ci.yml"
- - ".gitlab/ci/**/*"
- ".{eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}"
- ".{codeclimate,eslintrc,gitlab-ci,haml-lint,haml-lint_todo,rubocop,rubocop_todo,rubocop_manual_todo}.yml"
- "*_VERSION"
@@ -328,10 +344,14 @@
- "{,ee/,jh/}{app,bin,config,db,generator_templates,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*"
- "doc/api/graphql/reference/*" # Files in this folder are auto-generated
- "data/whats_new/*.yml"
+ # CI changes
+ - ".gitlab-ci.yml"
+ - ".gitlab/ci/**/*"
# QA changes
- ".dockerignore"
- "qa/**/*"
+# .code-patterns + .backstage-patterns + .qa-patterns
.code-backstage-qa-patterns: &code-backstage-qa-patterns
- "{package.json,yarn.lock}"
- ".browserslistrc"
@@ -340,9 +360,39 @@
- ".csscomb.json"
- "Dockerfile.assets"
- "vendor/assets/**/*"
+ - ".{eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}"
+ - ".{codeclimate,eslintrc,gitlab-ci,haml-lint,haml-lint_todo,rubocop,rubocop_todo,rubocop_manual_todo}.yml"
+ - "*_VERSION"
+ - "{,jh/}Gemfile{,.lock}"
+ - "Rakefile"
+ - "tests.yml"
+ - "config.ru"
+ - "{,ee/,jh/}{app,bin,config,db,generator_templates,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*"
+ - "doc/api/graphql/reference/*" # Files in this folder are auto-generated
+ - "data/whats_new/*.yml"
# CI changes
- ".gitlab-ci.yml"
- ".gitlab/ci/**/*"
+ # Backstage changes
+ - "Dangerfile"
+ - "danger/**/*"
+ - "{,ee/,jh/}fixtures/**/*"
+ - "{,ee/,jh/}rubocop/**/*"
+ - "{,ee/,jh/}spec/**/*"
+ - "{,spec/}tooling/**/*"
+ # QA changes
+ - ".dockerignore"
+ - "qa/**/*"
+
+# .code-backstage-qa-patterns + .workhorse-patterns
+.setup-test-env-patterns: &setup-test-env-patterns
+ - "{package.json,yarn.lock}"
+ - ".browserslistrc"
+ - "babel.config.js"
+ - "jest.config.{base,integration,unit}.js"
+ - ".csscomb.json"
+ - "Dockerfile.assets"
+ - "vendor/assets/**/*"
- ".{eslintignore,gitattributes,nvmrc,prettierrc,stylelintrc,yamllint}"
- ".{codeclimate,eslintrc,gitlab-ci,haml-lint,haml-lint_todo,rubocop,rubocop_todo,rubocop_manual_todo}.yml"
- "*_VERSION"
@@ -353,6 +403,9 @@
- "{,ee/,jh/}{app,bin,config,db,generator_templates,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*"
- "doc/api/graphql/reference/*" # Files in this folder are auto-generated
- "data/whats_new/*.yml"
+ # CI changes
+ - ".gitlab-ci.yml"
+ - ".gitlab/ci/**/*"
# Backstage changes
- "Dangerfile"
- "danger/**/*"
@@ -363,9 +416,12 @@
# QA changes
- ".dockerignore"
- "qa/**/*"
+ # Workhorse changes
+ - "GITLAB_WORKHORSE_VERSION"
+ - "workhorse/**/*"
+ - ".gitlab/ci/workhorse.gitlab-ci.yml"
-.code-backstage-danger-patterns: &code-backstage-danger-patterns
- # Backstage changes
+.danger-patterns: &danger-patterns
- "Dangerfile"
- "danger/**/*"
- "tooling/danger/**/*"
@@ -381,6 +437,9 @@
- "config/helpers/**/*.js"
- "vendor/assets/javascripts/**/*"
+.feature-flag-development-config-patterns: &feature-flag-development-config-patterns
+ - "{,ee/}config/feature_flags/{development,ops}/*.yml"
+
################
# Shared rules #
################
@@ -402,6 +461,7 @@
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-dot-com-gitlab-org-and-security-merge-request
changes: *ci-build-images-patterns
- <<: *if-dot-com-gitlab-org-and-security-merge-request
@@ -414,18 +474,11 @@
rules:
- <<: *if-not-canonical-namespace
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-auto-deploy-branches
- changes: *ci-build-images-patterns
- changes: *code-qa-patterns
-####################
-# Cache repo rules #
-####################
-.cache-repo:rules:
- rules:
- - <<: *if-cache-credentials-schedule
- allow_failure: true
-
#############
# CNG rules #
#############
@@ -476,6 +529,11 @@
changes: *docs-patterns
when: on_success
+.docs:rules:deprecations:
+ rules:
+ - <<: *if-default-refs
+ changes: *docs-deprecations-patterns
+
##################
# GraphQL rules #
##################
@@ -495,6 +553,7 @@
rules:
- <<: *if-not-canonical-namespace
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-auto-deploy-branches
- changes: *code-qa-patterns
@@ -525,8 +584,8 @@
.frontend:rules:default-frontend-jobs:
rules:
- - <<: *if-default-refs
- changes: *code-backstage-patterns
+ - <<: *if-merge-request-labels-run-all-rspec
+ - changes: *code-backstage-patterns
.frontend:rules:default-frontend-jobs-as-if-foss:
rules:
@@ -683,6 +742,9 @@
- <<: *if-not-ee
when: never
- <<: *if-dot-com-gitlab-org-and-security-merge-request
+ changes: *feature-flag-development-config-patterns
+ when: never
+ - <<: *if-dot-com-gitlab-org-and-security-merge-request
changes: *ci-qa-patterns
allow_failure: true
- <<: *if-dot-com-gitlab-org-and-security-merge-request
@@ -695,12 +757,35 @@
- <<: *if-dot-com-gitlab-org-schedule
allow_failure: true
+.qa:rules:package-and-qa:feature-flags:
+ rules:
+ - <<: *if-not-ee
+ when: never
+ - <<: *if-dot-com-gitlab-org-and-security-merge-request-manual-ff-package-and-qa
+ changes: *feature-flag-development-config-patterns
+ when: manual
+ allow_failure: true
+ - <<: *if-dot-com-gitlab-org-and-security-merge-request
+ changes: *feature-flag-development-config-patterns
+ allow_failure: true
+
+.qa:rules:reliable-reports:schedule:
+ rules:
+ - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "schedule" && $QA_RELIABLE_REPORT == "true"'
+ allow_failure: true
+
###############
# Rails rules #
###############
+.rails:rules:setup-test-env:
+ rules:
+ - changes: *setup-test-env-patterns
+ - <<: *if-merge-request-labels-run-all-rspec
+
.rails:rules:decomposed-databases:
rules:
- <<: *if-merge-request-labels-run-decomposed
+ - <<: *if-default-branch-schedule-nightly
.rails:rules:ee-and-foss-migration:
rules:
@@ -1163,6 +1248,21 @@
- <<: *if-merge-request-labels-as-if-foss
changes: *code-backstage-patterns
+.rails:rules:as-if-jh-rspec:
+ rules:
+ - <<: *if-not-ee
+ when: never
+ - <<: *if-jh
+ when: never
+ - <<: *if-security-merge-request
+ changes: *code-backstage-patterns
+ allow_failure: true
+ - <<: *if-merge-request-labels-as-if-jh
+ allow_failure: true
+ - <<: *if-merge-request
+ changes: *ci-patterns
+ allow_failure: true
+
.rails:rules:ee-and-foss-db-library-code:
rules:
- changes: *db-library-patterns
@@ -1180,9 +1280,25 @@
.rails:rules:detect-tests:
rules:
- - changes: *code-backstage-patterns
+ - changes: *code-backstage-qa-patterns
- <<: *if-merge-request-labels-run-all-rspec
+.rails:rules:detect-previous-failed-tests:
+ rules:
+ - <<: *if-security-merge-request
+ when: never
+ - <<: *if-merge-request-labels-run-all-rspec
+ - <<: *if-merge-request
+ changes: *code-backstage-patterns
+
+.rails:rules:rerun-previous-failed-tests:
+ rules:
+ - <<: *if-security-merge-request
+ when: never
+ - <<: *if-merge-request-labels-run-all-rspec
+ - <<: *if-merge-request
+ changes: *code-backstage-patterns
+
.rails:rules:rspec-foss-impact:
rules:
- <<: *if-not-ee
@@ -1252,6 +1368,14 @@
when: never
- changes: *code-backstage-patterns
+.rails:rules:skipped-flaky-tests-report:
+ rules:
+ - <<: *if-not-ee
+ when: never
+ - <<: *if-skip-flaky-tests-automatically
+ changes: *code-backstage-patterns
+ - changes: *ci-patterns
+
#########################
# Static analysis rules #
#########################
@@ -1260,6 +1384,12 @@
rules:
- changes: *code-backstage-qa-patterns
+.static-analysis:rules:ee:
+ rules:
+ - <<: *if-not-ee
+ when: never
+ - changes: *code-backstage-qa-patterns
+
.static-analysis:rules:as-if-foss:
rules:
- <<: *if-not-ee
@@ -1370,6 +1500,12 @@
when: never
- changes: *python-patterns
+.reports:rules:yarn-audit-dependency_scanning:
+ rules:
+ - if: '$DEPENDENCY_SCANNING_DISABLED || $GITLAB_FEATURES !~ /\bdependency_scanning\b/'
+ when: never
+ - changes: *nodejs-patterns
+
.reports:rules:schedule-dast:
rules:
- if: '$DAST_DISABLED || $GITLAB_FEATURES !~ /\bdast\b/'
@@ -1405,6 +1541,7 @@
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-dot-com-gitlab-org-merge-request
changes: *ci-review-patterns
- <<: *if-dot-com-gitlab-org-merge-request
@@ -1415,13 +1552,13 @@
allow_failure: true
- <<: *if-dot-com-gitlab-org-merge-request
changes: *qa-patterns
- allow_failure: true
- <<: *if-dot-com-gitlab-org-schedule
.review:rules:review-build-cng:
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-dot-com-gitlab-org-merge-request
changes: *ci-review-patterns
- <<: *if-dot-com-gitlab-org-merge-request
@@ -1431,24 +1568,22 @@
allow_failure: true
- <<: *if-dot-com-gitlab-org-merge-request
changes: *qa-patterns
- allow_failure: true
- <<: *if-dot-com-gitlab-org-schedule-child-pipeline
.review:rules:review-deploy:
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-dot-com-gitlab-org-merge-request
changes: *ci-review-patterns
- <<: *if-dot-com-gitlab-org-merge-request
changes: *frontend-patterns
- allow_failure: true
- <<: *if-dot-com-gitlab-org-merge-request
changes: *code-patterns
allow_failure: true
- <<: *if-dot-com-gitlab-org-merge-request
changes: *qa-patterns
- allow_failure: true
- <<: *if-dot-com-gitlab-org-schedule-child-pipeline
allow_failure: true
@@ -1458,6 +1593,7 @@
when: never
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-dot-com-gitlab-org-merge-request
changes: *ci-review-patterns
- <<: *if-dot-com-gitlab-org-merge-request
@@ -1473,6 +1609,7 @@
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-dot-com-gitlab-org-merge-request
changes: *code-qa-patterns
@@ -1480,24 +1617,32 @@
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-dot-com-gitlab-org-merge-request
changes: *ci-review-patterns
- <<: *if-dot-com-gitlab-org-merge-request
changes: *frontend-patterns
- allow_failure: true
- <<: *if-dot-com-gitlab-org-merge-request
- changes: *code-qa-patterns
+ changes: *qa-patterns
+ - <<: *if-dot-com-gitlab-org-merge-request
+ changes: *code-patterns
allow_failure: true
- <<: *if-dot-com-ee-schedule-child-pipeline
allow_failure: true
# The rule needs to be duplicated between `on_success` and `on_failure`
# because the jobs `needs` the previous job to complete.
+# With `when: always`, and the `review-qa-*` jobs are manual, the `allure-report-qa-*` jobs
+# would start running before the qa jobs have started.
# See https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63844#note_599012559
.review:rules:review-qa-smoke-report:
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
+ when: on_success
+ - <<: *if-merge-request-labels-run-review-app
+ when: on_failure
- <<: *if-dot-com-gitlab-org-merge-request
changes: *ci-review-patterns
when: on_success
@@ -1521,10 +1666,16 @@
- <<: *if-dot-com-ee-schedule-child-pipeline
when: on_failure
+.review:rules:review-qa-reliable:
+ rules:
+ - when: on_success
+ allow_failure: true
+
.review:rules:review-qa-all:
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
- <<: *if-dot-com-gitlab-org-merge-request
changes: *code-patterns
when: manual
@@ -1537,11 +1688,17 @@
# The rule needs to be duplicated between `on_success` and `on_failure`
# because the jobs `needs` the previous job to complete.
+# With `when: always`, and the `review-qa-*` jobs are manual, the `allure-report-qa-*` jobs
+# would start running before the qa jobs have started.
# See https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63844#note_599012559
.review:rules:review-qa-all-report:
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
+ when: on_success
+ - <<: *if-merge-request-labels-run-review-app
+ when: on_failure
- <<: *if-dot-com-gitlab-org-merge-request
changes: *code-patterns
when: manual
@@ -1569,13 +1726,16 @@
changes: *code-qa-patterns
when: manual
allow_failure: true
- - <<: *if-dot-com-gitlab-org-schedule-child-pipeline
+ - <<: *if-dot-com-gitlab-org-schedule
allow_failure: true
.review:rules:review-stop:
rules:
- <<: *if-not-ee
when: never
+ - <<: *if-merge-request-labels-run-review-app
+ when: manual
+ allow_failure: true
- <<: *if-dot-com-gitlab-org-merge-request
changes: *code-qa-patterns
when: manual
@@ -1588,7 +1748,7 @@
.review:rules:danger-local:
rules:
- if: '$CI_MERGE_REQUEST_IID'
- changes: *code-backstage-danger-patterns
+ changes: *danger-patterns
###############
# Setup rules #
@@ -1616,7 +1776,7 @@
changes: *code-backstage-patterns
when: on_success
-.setup:rules:no_ee_check:
+.setup:rules:no-ee-check:
rules:
- <<: *if-not-foss
when: never
@@ -1624,6 +1784,14 @@
changes: *code-backstage-patterns
when: on_success
+.setup:rules:no-jh-check:
+ rules:
+ - <<: *if-jh
+ when: never
+ - <<: *if-default-refs
+ changes: *code-backstage-patterns
+ when: on_success
+
.setup:rules:verify-tests-yml:
rules:
- <<: *if-not-ee
diff --git a/.gitlab/ci/setup.gitlab-ci.yml b/.gitlab/ci/setup.gitlab-ci.yml
index eb7a5afad3d..4c674f38939 100644
--- a/.gitlab/ci/setup.gitlab-ci.yml
+++ b/.gitlab/ci/setup.gitlab-ci.yml
@@ -41,13 +41,21 @@ gitlab_git_test:
script:
- spec/support/prepare-gitlab-git-test-for-commit --check-for-changes
-no_ee_check:
+no-ee-check:
extends:
- .minimal-job
- - .setup:rules:no_ee_check
+ - .setup:rules:no-ee-check
stage: test
script:
- - scripts/no-ee-check
+ - scripts/no-dir-check ee
+
+no-jh-check:
+ extends:
+ - .minimal-job
+ - .setup:rules:no-jh-check
+ stage: test
+ script:
+ - scripts/no-dir-check jh
verify-tests-yml:
extends:
@@ -102,6 +110,23 @@ detect-tests as-if-foss:
before_script:
- '[ "$FOSS_ONLY" = "1" ] && rm -rf ee/ qa/spec/ee/ qa/qa/specs/features/ee/ qa/qa/ee/ qa/qa/ee.rb'
+detect-previous-failed-tests:
+ extends:
+ - .detect-test-base
+ - .rails:rules:detect-previous-failed-tests
+ variables:
+ PREVIOUS_FAILED_TESTS_DIR: tmp/previous_failed_tests/
+ RSPEC_PG_REGEX: /rspec .+ pg12( .+)?/
+ RSPEC_EE_PG_REGEX: /rspec-ee .+ pg12( .+)?/
+ script:
+ - source ./scripts/utils.sh
+ - source ./scripts/rspec_helpers.sh
+ - retrieve_previous_failed_tests ${PREVIOUS_FAILED_TESTS_DIR} "${RSPEC_PG_REGEX}" "${RSPEC_EE_PG_REGEX}"
+ artifacts:
+ expire_in: 7d
+ paths:
+ - ${PREVIOUS_FAILED_TESTS_DIR}
+
add-jh-folder:
extends: .setup:rules:add-jh-folder
image: ${GITLAB_DEPENDENCY_PROXY}alpine:edge
@@ -112,6 +137,7 @@ add-jh-folder:
- curl --location -o "jh-folder.tar.gz" "https://gitlab.com/gitlab-jh/gitlab/-/archive/main-jh/gitlab-main-jh.tar.gz?path=jh"
- tar -xf "jh-folder.tar.gz"
- mv gitlab-main-jh-jh/jh/ ./
+ - cp Gemfile.lock jh/
- ls -l jh/
artifacts:
expire_in: 2d
diff --git a/.gitlab/ci/static-analysis.gitlab-ci.yml b/.gitlab/ci/static-analysis.gitlab-ci.yml
index 85df68e9030..ebd223c4171 100644
--- a/.gitlab/ci/static-analysis.gitlab-ci.yml
+++ b/.gitlab/ci/static-analysis.gitlab-ci.yml
@@ -2,7 +2,7 @@
extends:
- .default-retry
- .default-before_script
- - .static-analysis-cache
+ stage: lint
needs: []
variables:
SETUP_DB: "false"
@@ -14,7 +14,7 @@
update-static-analysis-cache:
extends:
- .static-analysis-base
- - .static-analysis-cache-push
+ - .rubocop-job-cache-push
- .shared:rules:update-cache
stage: prepare
script:
@@ -23,31 +23,95 @@ update-static-analysis-cache:
static-analysis:
extends:
- .static-analysis-base
+ - .static-analysis-cache
- .static-analysis:rules:ee-and-foss
- stage: test
- parallel: 4
+ parallel: 2
script:
- run_timed_command "retry yarn install --frozen-lockfile"
- scripts/static-analysis
- artifacts:
- expire_in: 31d
- when: always
- paths:
- - tmp/feature_flags/
-static-analysis-with-database:
+static-analysis as-if-foss:
+ extends:
+ - static-analysis
+ - .static-analysis:rules:as-if-foss
+ - .as-if-foss
+
+static-verification-with-database:
extends:
- .static-analysis-base
+ - .rubocop-job-cache
- .static-analysis:rules:ee-and-foss
- .use-pg12
- stage: test
script:
- bundle exec rake lint:static_verification_with_database
variables:
SETUP_DB: "true"
-static-analysis as-if-foss:
+eslint:
extends:
- - static-analysis
- - .static-analysis:rules:as-if-foss
+ - .static-analysis-base
+ - .yarn-cache
+ - .static-analysis:rules:ee
+ needs: []
+ variables:
+ USE_BUNDLE_INSTALL: "false"
+ script:
+ - run_timed_command "retry yarn install --frozen-lockfile"
+ - run_timed_command "yarn run lint:eslint:all"
+
+eslint as-if-foss:
+ extends:
+ - eslint
+ - .frontend:rules:eslint-as-if-foss
- .as-if-foss
+
+haml-lint foss:
+ extends:
+ - .static-analysis-base
+ - .ruby-cache
+ - .static-analysis:rules:ee-and-foss
+ script:
+ - run_timed_command "bin/rake 'haml_lint[app/views]'"
+ artifacts:
+ expire_in: 31d
+ when: always
+ paths:
+ - tmp/feature_flags/
+
+haml-lint ee:
+ extends:
+ - "haml-lint foss"
+ - .static-analysis:rules:ee
+ script:
+ - run_timed_command "bin/rake 'haml_lint[ee/app/views]'"
+
+rubocop:
+ extends:
+ - .static-analysis-base
+ - .rubocop-job-cache
+ - .static-analysis:rules:ee-and-foss
+ script:
+ - run_timed_command "bundle exec rubocop --parallel"
+
+qa:testcases:
+ extends:
+ - .static-analysis-base
+ - .rubocop-job-cache
+ - .static-analysis:rules:ee-and-foss
+ script:
+ - run_timed_command "bundle exec rubocop qa/qa/specs/features/**/* --only QA/DuplicateTestcaseLink"
+
+feature-flags-usage:
+ extends:
+ - .static-analysis-base
+ - .rubocop-job-cache
+ - .static-analysis:rules:ee-and-foss
+ script:
+ # We need to disable the cache for this cop since it creates files under tmp/feature_flags/*.used,
+ # the cache would prevent these files from being created.
+ - run_timed_command "bundle exec rubocop --only Gitlab/MarkUsedFeatureFlags --cache false"
+ artifacts:
+ expire_in: 31d
+ when: always
+ paths:
+ - tmp/feature_flags/
diff --git a/.gitlab/ci/workhorse.gitlab-ci.yml b/.gitlab/ci/workhorse.gitlab-ci.yml
index 0da0a334699..47b5d29ab8e 100644
--- a/.gitlab/ci/workhorse.gitlab-ci.yml
+++ b/.gitlab/ci/workhorse.gitlab-ci.yml
@@ -9,24 +9,21 @@ workhorse:verify:
.workhorse:test:
extends: .workhorse:rules:workhorse
- services:
- - name: registry.gitlab.com/gitlab-org/build/cng/gitaly:latest
- # Disable the hooks so we don't have to stub the GitLab API
- command: ["/usr/bin/env", "GITALY_TESTING_NO_GIT_HOOKS=1", "/scripts/process-wrapper"]
- alias: gitaly
variables:
- GITALY_ADDRESS: "tcp://gitaly:8075"
+ GITALY_ADDRESS: "tcp://127.0.0.1:8075"
stage: test
- needs: []
+ needs:
+ - setup-test-env
script:
- go version
- apt-get update && apt-get -y install libimage-exiftool-perl
+ - scripts/gitaly-test-build
- make -C workhorse test
workhorse:test using go 1.16:
extends: .workhorse:test
- image: ${GITLAB_DEPENDENCY_PROXY}golang:1.16
+ image: registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7-golang-1.16-git-2.31
workhorse:test using go 1.17:
extends: .workhorse:test
- image: ${GITLAB_DEPENDENCY_PROXY}golang:1.17
+ image: registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.7-golang-1.17-git-2.31
diff --git a/.gitlab/ci/yaml.gitlab-ci.yml b/.gitlab/ci/yaml.gitlab-ci.yml
index b25ad55e0ce..590593b9d75 100644
--- a/.gitlab/ci/yaml.gitlab-ci.yml
+++ b/.gitlab/ci/yaml.gitlab-ci.yml
@@ -5,7 +5,7 @@ lint-yaml:
- .default-retry
- .yaml-lint:rules
image: pipelinecomponents/yamllint:latest
- stage: test
+ stage: lint
needs: []
variables:
LINT_PATHS: .gitlab-ci.yml .gitlab/ci lib/gitlab/ci/templates
diff --git a/.gitlab/issue_templates/Audit Event Proposal.md b/.gitlab/issue_templates/Audit Event Proposal.md
index 7a5408ca1f2..af32845bd9a 100644
--- a/.gitlab/issue_templates/Audit Event Proposal.md
+++ b/.gitlab/issue_templates/Audit Event Proposal.md
@@ -9,5 +9,5 @@
<!-- Describe the audit event you are proposing should be added, including any details of what should be captured, how, and why. -->
/label ~"Category:Audit Events"
-/label ~"feature"
+/label ~"type::feature"
/label ~"group::compliance"
diff --git a/.gitlab/issue_templates/Bug.md b/.gitlab/issue_templates/Bug.md
index 41b694fdf2c..b9fed3745d1 100644
--- a/.gitlab/issue_templates/Bug.md
+++ b/.gitlab/issue_templates/Bug.md
@@ -2,10 +2,10 @@
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
-filtered by the "regression" or "bug" label:
+filtered by the "regression" or "type::bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
-- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=bug
+- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug
and verify the issue you're about to submit isn't a duplicate.
--->
@@ -82,4 +82,4 @@ will also determine whether the bug is fixed in a more recent version. -->
<!-- If you can, link to the line of code that might be responsible for the problem. -->
-/label ~bug
+/label ~"type::bug"
diff --git a/.gitlab/issue_templates/Deprecations.md b/.gitlab/issue_templates/Deprecations.md
index ff51699c6be..caef5c64334 100644
--- a/.gitlab/issue_templates/Deprecations.md
+++ b/.gitlab/issue_templates/Deprecations.md
@@ -41,7 +41,11 @@ Which tier is this feature available in?
### Deprecation Milestone
-<!-- In which milestone will this deprecation happen? -->
+<!-- In which milestone will this deprecation be announced ? -->
+
+### Planned Removal Milestone
+
+<!-- In which milestone will the feature or functionality be removed and announced? -->
### Links
diff --git a/.gitlab/issue_templates/Empty state.md b/.gitlab/issue_templates/Empty state.md
new file mode 100644
index 00000000000..d92ea8522e1
--- /dev/null
+++ b/.gitlab/issue_templates/Empty state.md
@@ -0,0 +1,80 @@
+<!-- Before implementing a new empty state solution, make sure to read the
+Empty State region docs in Pajamas: https://design.gitlab.com/regions/empty-states -->
+
+## Description
+
+<!-- Describe the solution you're proposing for your empty state region.
+Include links to user research (if applicable). -->
+
+## Location
+
+<!-- Provide a link and location of the new empty state solution.
+For example: https://gitlab.com/gitlab-org/gitlab-services/design.gitlab.com/-/issues -->
+
+## Use case
+
+<!-- What is the use case for the solution you're proposing?
+Read the Empty State docs and select the use case below: https://design.gitlab.com/regions/empty-states -->
+
+- [ ] Blank content
+- [ ] Empty search results
+- [ ] Configuration required
+- [ ] Higher tier
+
+## Checklist
+
+<!-- Follow the steps below that correspond with the use case selected above.
+Follow the steps to complete this issue -->
+
+### Blank content
+
+- [ ] The solution follows the `Blank content` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#blank-content).
+- [ ] Follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+
+### Empty search results
+
+- [ ] The solution follows the `Empty search results` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#empty-search-results).
+- [ ] Follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+
+### Configuration required
+
+- [ ] The solution follows the `Configuration required` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#configuration-required).
+- [ ] Ask a [Growth product manager or Designer](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to review your solution.
+- [ ] Is your solution introducing a new empty states or modifying an existing one?
+ - [ ] Introducing a new empty state: Follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+ - [ ] Modifying an existing empty state: Follow the [`Experimentation` process](#experimentation) below. _Note_: If the empty state you want to replace hasn't been updated in a long time, doesn't pitch the value of the feature, or does not contain a next step action CTA, then we recommend you skip the experimentation process to implement and add tracking to your new empty state.
+
+<!-- IF experimentation -->
+#### Experimentation
+
+- [ ] Collaborate with a [Growth product manager](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to help you determine if you can validate your solution through an experiment on SaaS.
+- [ ] If an experiment is possible, create an issue using the [experiment idea template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Experiment%20Idea) and follow the template intructions. Otherwise, follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+- [ ] Ask a [Growth product manager or Designer](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to review your experiment set-up.
+- [ ] Implement and monitor the experiment following the [implementation guide](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/experiment_guide/gitlab_experiment.md#implement-an-experiment).
+- [ ] Review and discuss the findings.
+- [ ] Add the findings to the [Growth experimentation knowledge](https://about.gitlab.com/direction/growth/#growth-experiments-knowledge-base---concluded-experiments).
+
+### Higher tier
+
+- [ ] The solution follows the `Higher tier` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#higher-tier).
+- [ ] Ask a Product Manager or Designer from the [Conversion group](https://about.gitlab.com/handbook/engineering/development/growth/conversion/#group-members) to review your solution.
+- [ ] Is your solution introducing a new empty states or modifying an existing one?
+ - [ ] Introducing a new empty state: follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+ - [ ] Modifying an existing empty state, follow the [`Experimentation` process](#experimentation) below.
+
+<!-- IF experimentation -->
+#### Experimentation
+
+- [ ] Collaborate with a [Growth product manager](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to help you determine if you can validate your solution through an experiment on SaaS.
+- [ ] If an experiment is possible, create an issue using the [experiment idea template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Experiment%20Idea) and follow the template intructions. Otherwise, follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+- [ ] Add a ~"Category:Conversion Experiment" label to the experiment idea issue.
+- [ ] Ask a Product Manager or Designer from the [Conversion group](https://about.gitlab.com/handbook/engineering/development/growth/conversion/#group-members) to review your experiment set-up.
+- [ ] Implement and monitor the experiment following the [implementation guide](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/experiment_guide/gitlab_experiment.md#implement-an-experiment) .
+- [ ] Review and discuss the findings.
+- [ ] Add the findings to the [Growth experimentation knowledge](https://about.gitlab.com/direction/growth/#growth-experiments-knowledge-base---concluded-experiments).
+
+
+## After merge
+
+- [ ] Use the `Snowplow event tracking` [issue template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Snowplow%20event%20tracking) and open an issue to add Snowplow event tracking to your new empty state solution.
+ - [ ] Add your ~devops:: and ~group:: labels to the new issue.
diff --git a/.gitlab/issue_templates/Experimentation.md b/.gitlab/issue_templates/Experiment Implementation.md
index ba7839fb941..b3883f08c25 100644
--- a/.gitlab/issue_templates/Experimentation.md
+++ b/.gitlab/issue_templates/Experiment Implementation.md
@@ -1,4 +1,4 @@
-<!-- Title suggestion: Experiment: [description] -->
+<!-- Title suggestion: Experiment Implementation: [description] -->
# Experiment Summary
<!-- Quick rundown of what is being done -->
diff --git a/.gitlab/issue_templates/Experiment Rollout.md b/.gitlab/issue_templates/Experiment Rollout.md
index c5fdc739943..9209423ba33 100644
--- a/.gitlab/issue_templates/Experiment Rollout.md
+++ b/.gitlab/issue_templates/Experiment Rollout.md
@@ -105,5 +105,16 @@ In this rollout issue, ensure the scoped `experiment::` label is kept accurate.
/chatops run feature set <experiment-key> false
```
+## Experiment Successful Cleanup Concerns
+
+_Items to be considered if candidate experience is to become a permanent part of GitLab_
+
+<!--
+Add a list of items raised during MR review or otherwise that may need further thought/consideration
+before becoming permanent parts of the product.
+
+Example: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70451#note_727246104
+-->
+
/label ~"feature flag" ~"devops::growth" ~"growth experiment" ~"experiment-rollout" ~Engineering ~"workflow::scheduling" ~"experiment::pending"
/milestone %"Next 1-3 releases"
diff --git a/.gitlab/issue_templates/Experiment Successful Cleanup.md b/.gitlab/issue_templates/Experiment Successful Cleanup.md
index 42f26342342..1dd57332b8e 100644
--- a/.gitlab/issue_templates/Experiment Successful Cleanup.md
+++ b/.gitlab/issue_templates/Experiment Successful Cleanup.md
@@ -12,9 +12,10 @@ The changes need to become an official part of the product.
- [ ] Determine if tracking should be kept as is, removed, or modified.
- [ ] Ensure any relevant documentation has been updated.
- [ ] Consider changes to any `feature_category:` introduced by the experiment if ownership is changing (PM for Growth and PM for the new category as DRIs)
+- [ ] Check to see if the experiment introduced new design assets. Add them to the appropriate repos and document them if needed.
- [ ] Optional: Migrate experiment to a default enabled [feature flag](https://docs.gitlab.com/ee/development/feature_flags) for one milestone and add a changelog. Converting to a feature flag can be skipped at the ICs discretion if risk is deemed low with consideration to both SaaS and (if applicable) self managed
- [ ] In the next milestone, [remove the feature flag](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up) if applicable
- [ ] After the flag removal is deployed, [clean up the feature/experiment feature flags](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up) by running chatops command in `#production` channel
-- [ ] Ensure the corresponding [Experiment Tracking](https://gitlab.com/groups/gitlab-org/-/boards/1352542?label_name[]=devops%3A%3Agrowth&label_name[]=growth%20experiment&label_name[]=experiment%20tracking) issue is updated
+- [ ] Ensure the corresponding [Experiment Rollout](https://gitlab.com/groups/gitlab-org/-/boards/1352542?label_name[]=devops%3A%3Agrowth&label_name[]=growth%20experiment&label_name[]=experiment-rollout) issue is updated
-/label ~"feature" ~"feature::maintenance" ~"workflow::scheduling" ~"growth experiment" ~"feature flag"
+/label ~"type::maintenance" ~"workflow::scheduling" ~"growth experiment" ~"feature flag"
diff --git a/.gitlab/issue_templates/Feature Flag Roll Out.md b/.gitlab/issue_templates/Feature Flag Roll Out.md
index 00b396bac4e..bc1a23729e2 100644
--- a/.gitlab/issue_templates/Feature Flag Roll Out.md
+++ b/.gitlab/issue_templates/Feature Flag Roll Out.md
@@ -30,6 +30,17 @@ Are there any other stages or teams involved that need to be kept in the loop?
<!-- Describe the expected outcome when rolling out this feature -->
+### When is the feature viable?
+
+<!-- What are the settings we need to configure in order to have this feature viable? -->
+
+<!--
+Example below:
+
+1. Enable service ping collection
+ `ApplicationSetting.first.update(usage_ping_enabled: true)`
+-->
+
### What might happen if this goes wrong?
<!-- Should the feature flag be turned off? Any MRs that need to be rolled back? Communication that needs to happen? What are some things you can think of that could go wrong - data loss or broken pages? -->
@@ -37,6 +48,12 @@ Are there any other stages or teams involved that need to be kept in the loop?
### What can we monitor to detect problems with this?
<!-- Which dashboards from https://dashboards.gitlab.net are most relevant? -->
+_Consider mentioning checks for 5xx errors or other anomalies like an increase in redirects
+(302 HTTP response status)_
+
+### What can we check for monitoring production after rollouts?
+
+_Consider adding links to check for Sentry errors, Production logs for 5xx, 302s, etc._
## Rollout Steps
@@ -73,11 +90,14 @@ Are there any other stages or teams involved that need to be kept in the loop?
If a different developer will be covering, or an exception is needed, please inform the oncall SRE by using the `@sre-oncall` Slack alias.
- [ ] Ensure that documentation has been updated ([More info](https://docs.gitlab.com/ee/development/documentation/feature_flags.html#features-that-became-enabled-by-default)).
- [ ] Announce on [the feature issue](ISSUE LINK) an estimated time this will be enabled on GitLab.com.
-- [ ] If the feature might impact the user experience, notify `#support_gitlab-com` and your team channel ([more guidance when this is necessary in the dev docs](https://docs.gitlab.com/ee/development/feature_flags/controls.html#communicate-the-change)).
+- [ ] Notify `#support_gitlab-com` and your team channel ([more guidance when this is necessary in the dev docs](https://docs.gitlab.com/ee/development/feature_flags/controls.html#communicate-the-change)).
### Global rollout on production
-All `/chatops` commands that target production should be done in the `#production` slack channel for visibility.
+For visibility, all `/chatops` commands that target production should be:
+
+- Executed in the `#production` slack channel.
+- Cross-posted (with the command results) to the responsible team's slack channel (`#g_TEAM_NAME`).
- [ ] [Incrementally roll out](https://docs.gitlab.com/ee/development/feature_flags/controls.html#process) the feature.
- If the feature flag in code has [an actor](https://docs.gitlab.com/ee/development/feature_flags/#feature-actors), perform **actor-based** rollout.
@@ -148,5 +168,5 @@ codebase.
/chatops run feature set <feature-flag-name> false
```
-/label ~"feature flag"
+/label ~"feature flag" ~"type::feature" ~"feature::addition"
/assign DRI
diff --git a/.gitlab/issue_templates/Feature Proposal - basic.md b/.gitlab/issue_templates/Feature Proposal - basic.md
index 0c05b7a0165..980751621f0 100644
--- a/.gitlab/issue_templates/Feature Proposal - basic.md
+++ b/.gitlab/issue_templates/Feature Proposal - basic.md
@@ -1,10 +1,16 @@
<!-- This template is a great use for issues that are feature::additions or technical tasks for larger issues.-->
-### Proposal
+### Proposal
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
<!-- Consider adding related issues and epics to this issue. You can also reference the Feature Proposal Template (https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md) for additional details to consider adding to this issue. Additionally, as a data oriented organization, when your feature exits planning breakdown, consider adding the `What does success look like, and how can we measure that?` section.
-->
-/label ~feature::addition ~"group::" ~"section::" ~"Category:" ~"GitLab Core"/~"GitLab Premium"/~"GitLab Ultimate"
+<!-- Label reminders
+Use the following resources to find the appropriate labels:
+- https://gitlab.com/gitlab-org/gitlab/-/labels
+- https://about.gitlab.com/handbook/product/categories/features/
+-->
+
+/label ~"type::feature" ~feature::addition ~"group::" ~"section::" ~"Category:" ~"GitLab Core"/~"GitLab Premium"/~"GitLab Ultimate"
diff --git a/.gitlab/issue_templates/Feature Proposal - lean.md b/.gitlab/issue_templates/Feature Proposal - lean.md
index 9dd4bdc6b22..504bfbb03d8 100644
--- a/.gitlab/issue_templates/Feature Proposal - lean.md
+++ b/.gitlab/issue_templates/Feature Proposal - lean.md
@@ -1,33 +1,22 @@
-<!-- This issue template can be used a great starting point for feature requests. The last section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab. -->
+<!-- This issue template can be used as a great starting point for feature requests. The section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab.
-### Release notes
+The goal of this template is brevity for quick/smaller iterations. For a more thorough list of considerations for larger features or feature sets, you can leverage the detailed [feature proposal](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md). -->
+
+### Release notes
<!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " -->
-### Problem to solve
+### Problem to solve
<!-- What is the user problem you are trying to solve with this issue? -->
-### Proposal
+### Proposal
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
-
-
-/label ~"feature" ~"group::" ~"section::" ~"Category::" ~"GitLab Free"/~"GitLab Premium"/~"GitLab Ultimate"
-
-
-<!--- Use the following resources to find the appropriate labels:
-- https://gitlab.com/gitlab-org/gitlab/-/labels
-- https://about.gitlab.com/handbook/product/categories/features/
-
-Consider adding related issues and epics to this issue. You can also reference the Feature Proposal Template (https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md) for additional details to consider adding to this issue. Additionally, as a data oriented organization, when your feature exits planning breakdown, consider adding the `What does success look like, and how can we measure that?` section.
-
-Other sections to consider adding:
-
### Intended users
-Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later.
+<!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later.
Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/
@@ -39,68 +28,28 @@ Personas are described at https://about.gitlab.com/handbook/marketing/product-ma
* [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer)
* [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator)
* [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst)
-* [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager)
+* [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager)
* [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer)
* [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test)
-* [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
+* [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst)
+* [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor)
+-->
-### User experience goal
-
-What is the single user experience workflow this problem addresses?
-For example, "The user should be able to use the UI/API/.gitlab-ci.yml with GitLab to <perform a specific task>"
-https://about.gitlab.com/handbook/engineering/ux/ux-research-training/user-story-mapping/
-
-
-### Further details
-
-Include use cases, benefits, goals, or any other details that will help us understand the problem better.
+### Metrics
-### Permissions and Security
+<!-- How are you going to track uage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it?
-<!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)?
-Consider adding checkboxes and expectations of users with certain levels of membership https://docs.gitlab.com/ee/user/permissions.html
-* [ ] Add expected impact to members with no access (0)
-* [ ] Add expected impact to Guest (10) members
-* [ ] Add expected impact to Reporter (20) members
-* [ ] Add expected impact to Developer (30) members
-* [ ] Add expected impact to Maintainer (40) members
-* [ ] Add expected impact to Owner (50) members
+Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
-### Documentation
+-->
- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change
-
-* Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/workflow.html
-* If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html
-
-### Availability & Testing
-
-This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.
-
-What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?
-
-Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
-* Unit test changes
-* Integration test changes
-* End-to-end test change
-
-See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning
-
-### What does success look like, and how can we measure that?
-
-Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this.
-
-### What is the type of buyer?
-
-What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/
-In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#three-tiers
-
-### Is this a cross-stage feature?
-
-Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features -->
+<!-- Label reminders
+Use the following resources to find the appropriate labels:
+- https://gitlab.com/gitlab-org/gitlab/-/labels
+- https://about.gitlab.com/handbook/product/categories/features/
+-->
-/label ~documentation
-/label ~direction
+/label ~"type::feature" ~"group::" ~"section::" ~"Category::" ~"GitLab Free"/~"GitLab Premium"/~"GitLab Ultimate" ~documentation ~direction
diff --git a/.gitlab/issue_templates/Feature proposal - detailed.md b/.gitlab/issue_templates/Feature proposal - detailed.md
index 9759bb7e2dc..c787fc99333 100644
--- a/.gitlab/issue_templates/Feature proposal - detailed.md
+++ b/.gitlab/issue_templates/Feature proposal - detailed.md
@@ -31,6 +31,14 @@ Personas are described at https://about.gitlab.com/handbook/marketing/product-ma
* [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor)
-->
+### Metrics
+
+<!-- How are you going to track uage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it?
+
+Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
+
+-->
+
### User experience goal
<!-- What is the single user experience workflow this problem addresses?
@@ -112,6 +120,4 @@ Use the following resources to find the appropriate labels:
-->
/label ~devops:: ~group: ~Category:
/label ~"GitLab Free"/~"GitLab Premium"/~"GitLab Ultimate"
-/label ~feature
-/label ~documentation
-/label ~direction
+/label ~"type::feature" ~documentation ~direction
diff --git a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
index 0d822945798..71a962d1789 100644
--- a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
+++ b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
@@ -111,115 +111,9 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
- [ ] Be sure to commit the relevant changes in `ee/db/geo/structure.sql`
-### Add verification state fields on the Geo primary site
+### Add verification state to the Model
-The Geo primary site needs to checksum every replicable in order for secondaries to verify their own checksums. To do this, Geo requires fields on the Model. There are two ways to add the necessary verification state fields. If the table is large and wide, then it may be a good idea to add verification state fields to a separate table (Option 2). Consult a database expert if needed.
-
-#### Add verification state fields to the model table (Option 1)
-
-- [ ] Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationStateToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationStateToCoolWidgets < ActiveRecord::Migration[6.0]
- def change
- change_table(:cool_widgets) do |t|
- t.integer :verification_state, default: 0, limit: 2, null: false
- t.column :verification_started_at, :datetime_with_timezone
- t.integer :verification_retry_count, limit: 2, null: false
- t.column :verification_retry_at, :datetime_with_timezone
- t.column :verified_at, :datetime_with_timezone
- t.binary :verification_checksum, using: 'verification_checksum::bytea'
-
- t.text :verification_failure # rubocop:disable Migration/AddLimitToTextColumns
- end
- end
- end
- ```
-
-- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
-- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
-- [ ] Adding a `text` column also [requires](../database/strings_and_the_text_data_type.md#add-a-text-column-to-an-existing-table) setting a limit. Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationFailureLimitToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationFailureLimitToCoolWidgets < ActiveRecord::Migration[6.0]
- include Gitlab::Database::MigrationHelpers
-
- disable_ddl_transaction!
-
- CONSTRAINT_NAME = 'cool_widget_verification_failure_text_limit'
-
- def up
- add_text_limit :cool_widget, :verification_failure, 255, constraint_name: CONSTRAINT_NAME
- end
-
- def down
- remove_check_constraint(:cool_widget, CONSTRAINT_NAME)
- end
- end
- ```
-
-- [ ] Add indexes on verification fields to ensure verification can be performed efficiently. Some or all of these indexes can be omitted if the table is guaranteed to be small. Ask a database expert if you are considering omitting indexes. Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationIndexesToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationIndexesToCoolWidgets < ActiveRecord::Migration[6.0]
- include Gitlab::Database::MigrationHelpers
-
- VERIFICATION_STATE_INDEX_NAME = "index_cool_widgets_on_verification_state"
- PENDING_VERIFICATION_INDEX_NAME = "index_cool_widgets_pending_verification"
- FAILED_VERIFICATION_INDEX_NAME = "index_cool_widgets_failed_verification"
- NEEDS_VERIFICATION_INDEX_NAME = "index_cool_widgets_needs_verification"
-
- disable_ddl_transaction!
-
- def up
- add_concurrent_index :cool_widgets, :verification_state, name: VERIFICATION_STATE_INDEX_NAME
- add_concurrent_index :cool_widgets, :verified_at, where: "(verification_state = 0)", order: { verified_at: 'ASC NULLS FIRST' }, name: PENDING_VERIFICATION_INDEX_NAME
- add_concurrent_index :cool_widgets, :verification_retry_at, where: "(verification_state = 3)", order: { verification_retry_at: 'ASC NULLS FIRST' }, name: FAILED_VERIFICATION_INDEX_NAME
- add_concurrent_index :cool_widgets, :verification_state, where: "(verification_state = 0 OR verification_state = 3)", name: NEEDS_VERIFICATION_INDEX_NAME
- end
-
- def down
- remove_concurrent_index_by_name :cool_widgets, VERIFICATION_STATE_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, PENDING_VERIFICATION_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, FAILED_VERIFICATION_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, NEEDS_VERIFICATION_INDEX_NAME
- end
- end
- ```
-
-- [ ] Run database migrations:
-
- ```shell
- bin/rake db:migrate
- ```
-
-- [ ] Be sure to commit the relevant changes in `db/structure.sql`
-
-#### Add verification state fields to a separate table (Option 2)
+The Geo primary site needs to checksum every replicable so secondaries can verify their own checksums. To do this, Geo requires the Model to have an associated table to track verification state.
- [ ] Create the migration file in `db/migrate`:
@@ -273,6 +167,7 @@ The Geo primary site needs to checksum every replicable in order for secondaries
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
+
- [ ] Run database migrations:
```shell
@@ -287,7 +182,14 @@ That's all of the required database changes.
#### Step 1. Implement replication and verification
-- [ ] Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
+- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
+ - Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
+ - Include the `::Gitlab::Geo::VerificationState` concern.
+ - Delegate verification related methods to the `cool_widget_state` model.
+ - For verification, override some scopes to use the `cool_widget_states` table instead of the model table.
+ - Implement the `verification_state_object` method to return the object that holds
+ the verification details
+ - Override some methods to use the `cool_widget_states` table in verification-related queries.
Pay some attention to method `pool_repository`. Not every repository type uses repository pooling. As Geo prefers to use repository snapshotting, it can lead to data loss. Make sure to overwrite `pool_repository` so it returns nil for repositories that do not have pools.
@@ -297,6 +199,7 @@ That's all of the required database changes.
# frozen_string_literal: true
class CoolWidget < ApplicationRecord
+ ...
include ::Gitlab::Geo::ReplicableModel
include ::Gitlab::Geo::VerificationState
@@ -304,31 +207,62 @@ That's all of the required database changes.
mount_uploader :file, CoolWidgetUploader
+ has_one :cool_widget_state, autosave: false, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
+
+ delegate :verification_retry_at, :verification_retry_at=,
+ :verified_at, :verified_at=,
+ :verification_checksum, :verification_checksum=,
+ :verification_failure, :verification_failure=,
+ :verification_retry_count, :verification_retry_count=,
+ :verification_state=, :verification_state,
+ :verification_started_at=, :verification_started_at,
+ to: :cool_widget_state
+ ...
+
+ scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) }
+ scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) }
+ scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) }
+
# Override the `all` default if not all records can be replicated. For an
# example of an existing Model that needs to do this, see
# `EE::MergeRequestDiff`.
# scope :available_replicables, -> { all }
- # @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
- # @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
- def self.replicables_for_current_secondary(primary_key_in)
- # This issue template does not help you write this method.
- #
- # This method is called only on Geo secondary sites. It is called when
- # we want to know which records to replicate. This is not easy to automate
- # because for example:
- #
- # * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
- # namespace, but the nature of the relationship to a namespace varies
- # between Models.
- # * The "selective sync" feature allows admins to choose which shards to
- # replicate, per secondary site. Repositories are associated with
- # shards. Most blob types are not, but Project Uploads are.
- # * Remote stored replicables are not replicated, by default. But the
- # setting `sync_object_storage` enables replication of remote stored
- # replicables.
- #
- # Search the codebase for examples, and consult a Geo expert if needed.
+ def verification_state_object
+ cool_widget_state
+ end
+ ...
+
+ class_methods do
+ extend ::Gitlab::Utils::Override
+ ...
+
+ # @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
+ # @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
+ def replicables_for_current_secondary(primary_key_in)
+ # This issue template does not help you write this method.
+ #
+ # This method is called only on Geo secondary sites. It is called when
+ # we want to know which records to replicate. This is not easy to automate
+ # because for example:
+ #
+ # * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
+ # namespace, but the nature of the relationship to a namespace varies
+ # between Models.
+ # * The "selective sync" feature allows admins to choose which shards to
+ # replicate, per secondary site. Repositories are associated with
+ # shards. Most blob types are not, but Project Uploads are.
+ # * Remote stored replicables are not replicated, by default. But the
+ # setting `sync_object_storage` enables replication of remote stored
+ # replicables.
+ #
+ # Search the codebase for examples, and consult a Geo expert if needed.
+ end
+
+ override :verification_state_table_class
+ def verification_state_table_class
+ CoolWidgetState
+ end
end
# Geo checks this method in FrameworkRepositorySyncService to avoid
@@ -336,6 +270,11 @@ That's all of the required database changes.
def pool_repository
nil
end
+ ...
+
+ def cool_widget_state
+ super || build_cool_widget_state
+ end
...
end
@@ -343,6 +282,15 @@ That's all of the required database changes.
- [ ] Implement `CoolWidget.replicables_for_current_secondary` above.
- [ ] Ensure `CoolWidget.replicables_for_current_secondary` is well-tested. Search the codebase for `replicables_for_current_secondary` to find examples of parameterized table specs. You may need to add more `FactoryBot` traits.
+- [ ] Add the following shared examples to `ee/spec/models/ee/cool_widget_spec.rb`:
+
+ ```ruby
+ include_examples 'a replicable model with a separate table for verification state' do
+ let(:verifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is included in `available_verifiables`
+ let(:unverifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is NOT included in `available_verifiables`
+ end
+ ```
+
- [ ] Create `ee/app/replicators/geo/cool_widget_replicator.rb`. Implement the `#repository` method which should return a `<Repository>` instance, and implement the class method `.model` to return the `CoolWidget` class:
```ruby
@@ -365,6 +313,10 @@ That's all of the required database changes.
::Gitlab::GitAccessCoolWidget
end
+ def self.no_repo_message
+ git_access_class.error_message(:no_repo)
+ end
+
# The feature flag follows the format `geo_#{replicable_name}_replication`,
# so here it would be `geo_cool_widget_replication`
def self.replication_enabled_by_default?
@@ -403,6 +355,9 @@ That's all of the required database changes.
```
- [ ] Make sure a Geo secondary site can request and download Cool Widgets on the Geo primary site. You may need to make some changes to `Gitlab::GitAccessCoolWidget`. For example, see [this change for Group-level Wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/54914/diffs?commit_id=0f2b36f66697b4addbc69bd377ee2818f648dd33).
+
+- [ ] Make sure a Geo secondary site can replicate Cool Widgets where repository does not exist on the Geo primary site. The only way to know about this is to parse the error text. You may need to make some changes to `Gitlab::CoolWidgetReplicator.no_repo_message` to return the proper error message. For example, see [this change for Group-level Wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74133).
+
- [ ] Generate the feature flag definition file by running the feature flag command and following the command prompts:
```shell
@@ -529,13 +484,7 @@ That's all of the required database changes.
- [ ] Make sure the factory also allows setting a `project` attribute. If the model does not have a direct relation to a project, you can use a `transient` attribute. Check out `spec/factories/merge_request_diffs.rb` for an example.
-##### If you added verification state fields to a separate table (option 2 above), then you need to make additional model and factory changes
-
-If you did not add verification state fields to a separate table, `cool_widget_states`, then skip to [Step 2. Implement metrics gathering](#step-2-implement-metrics-gathering).
-
-Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309).
-
-- [ ] Add a `Geo::CoolWidgetState` model in `ee/app/models/geo/cool_widget_state.rb`:
+- [ ] Following [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309) add a `Geo::CoolWidgetState` model in `ee/app/models/ee/geo/cool_widget_state.rb`:
``` ruby
# frozen_string_literal: true
@@ -569,63 +518,6 @@ Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.co
end
```
-- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
- - Include the `::Gitlab::Geo::VerificationState` concern.
- - Delegate verification related methods to the `cool_widget_state` model.
- - Override some scopes to use the `cool_widget_states` table instead of the model table, for verification.
- - Override some methods to use the `cool_widget_states` table in verification related queries.
-
- ```ruby
- class CoolWidget < ApplicationRecord
- ...
- include ::Gitlab::Geo::VerificationState
-
- has_one :cool_widget_state, autosave: true, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
-
- delegate :verification_retry_at, :verification_retry_at=,
- :verified_at, :verified_at=,
- :verification_checksum, :verification_checksum=,
- :verification_failure, :verification_failure=,
- :verification_retry_count, :verification_retry_count=,
- :verification_state=, :verification_state,
- :verification_started_at=, :verification_started_at,
- to: :cool_widget_state
- ...
-
- scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) }
- scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) }
- scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) }
-
- ...
-
- class_methods do
- extend ::Gitlab::Utils::Override
- ...
- override :verification_state_table_name
- def verification_state_table_name
- 'cool_widget_states'
- end
-
- override :verification_state_model_key
- def verification_state_model_key
- 'cool_widget_id'
- end
-
- override :verification_arel_table
- def verification_arel_table
- CoolWidgetState.arel_table
- end
- end
- ...
-
- def cool_widget_state
- super || build_cool_widget_state
- end
-
- ...
- end
- ```
-
#### Step 2. Implement metrics gathering
Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus` for display in the UI, and sent to Prometheus:
diff --git a/.gitlab/issue_templates/Geo Replicate a new blob type.md b/.gitlab/issue_templates/Geo Replicate a new blob type.md
index 00a71fa406e..7c927e79e93 100644
--- a/.gitlab/issue_templates/Geo Replicate a new blob type.md
+++ b/.gitlab/issue_templates/Geo Replicate a new blob type.md
@@ -37,6 +37,7 @@ It is also a good idea to first open a proof-of-concept merge request. It can be
You can look into the following examples of MRs for implementing replication/verification for a new blob type:
- [Add db changes](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60935) and [add verification for MR diffs using SSF](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309)
- [Verify Terraform state versions](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/58800)
+- [Verify LFS objects](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63981)
### Modify database schemas to prepare to add Geo support for Cool Widgets
@@ -114,113 +115,9 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
### Add verification state fields on the Geo primary site
-The Geo primary site needs to checksum every replicable in order for secondaries to verify their own checksums. To do this, Geo requires fields on the Model. There are two ways to add the necessary verification state fields. If the table is large and wide, then it may be a good idea to add verification state fields to a separate table (Option 2). Consult a database expert if needed.
+The Geo primary site needs to checksum every replicable so secondaries can verify their own checksums. To do this, Geo requires fields on the Model. Add verification state fields to a separate table. Consult a database expert if needed.
-#### Add verification state fields to the model table (Option 1)
-
-- [ ] Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationStateToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationStateToCoolWidgets < ActiveRecord::Migration[6.0]
- def change
- change_table(:cool_widgets) do |t|
- t.integer :verification_state, default: 0, limit: 2, null: false
- t.column :verification_started_at, :datetime_with_timezone
- t.integer :verification_retry_count, limit: 2, null: false
- t.column :verification_retry_at, :datetime_with_timezone
- t.column :verified_at, :datetime_with_timezone
- t.binary :verification_checksum, using: 'verification_checksum::bytea'
-
- t.text :verification_failure # rubocop:disable Migration/AddLimitToTextColumns
- end
- end
- end
- ```
-
-- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
-- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
-- [ ] Adding a `text` column also [requires](../database/strings_and_the_text_data_type.md#add-a-text-column-to-an-existing-table) setting a limit. Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationFailureLimitToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationFailureLimitToCoolWidgets < ActiveRecord::Migration[6.0]
- include Gitlab::Database::MigrationHelpers
-
- disable_ddl_transaction!
-
- CONSTRAINT_NAME = 'cool_widget_verification_failure_text_limit'
-
- def up
- add_text_limit :cool_widget, :verification_failure, 255, constraint_name: CONSTRAINT_NAME
- end
-
- def down
- remove_check_constraint(:cool_widget, CONSTRAINT_NAME)
- end
- end
- ```
-
-- [ ] Add indexes on verification fields to ensure verification can be performed efficiently. Some or all of these indexes can be omitted if the table is guaranteed to be small. Ask a database expert if you are considering omitting indexes. Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationIndexesToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationIndexesToCoolWidgets < ActiveRecord::Migration[6.0]
- include Gitlab::Database::MigrationHelpers
-
- VERIFICATION_STATE_INDEX_NAME = "index_cool_widgets_on_verification_state"
- PENDING_VERIFICATION_INDEX_NAME = "index_cool_widgets_pending_verification"
- FAILED_VERIFICATION_INDEX_NAME = "index_cool_widgets_failed_verification"
- NEEDS_VERIFICATION_INDEX_NAME = "index_cool_widgets_needs_verification"
-
- disable_ddl_transaction!
-
- def up
- add_concurrent_index :cool_widgets, :verification_state, name: VERIFICATION_STATE_INDEX_NAME
- add_concurrent_index :cool_widgets, :verified_at, where: "(verification_state = 0)", order: { verified_at: 'ASC NULLS FIRST' }, name: PENDING_VERIFICATION_INDEX_NAME
- add_concurrent_index :cool_widgets, :verification_retry_at, where: "(verification_state = 3)", order: { verification_retry_at: 'ASC NULLS FIRST' }, name: FAILED_VERIFICATION_INDEX_NAME
- add_concurrent_index :cool_widgets, :verification_state, where: "(verification_state = 0 OR verification_state = 3)", name: NEEDS_VERIFICATION_INDEX_NAME
- end
-
- def down
- remove_concurrent_index_by_name :cool_widgets, VERIFICATION_STATE_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, PENDING_VERIFICATION_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, FAILED_VERIFICATION_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, NEEDS_VERIFICATION_INDEX_NAME
- end
- end
- ```
-
-- [ ] Run database migrations:
-
- ```shell
- bin/rake db:migrate
- ```
-
-- [ ] Be sure to commit the relevant changes in `db/structure.sql`
-
-#### Add verification state fields to a separate table (Option 2)
+#### Add verification state fields to a new table
- [ ] Create the migration file in `db/migrate`:
@@ -274,12 +171,15 @@ The Geo primary site needs to checksum every replicable in order for secondaries
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
+
- [ ] Run database migrations:
```shell
bin/rake db:migrate
```
+- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
+
- [ ] Be sure to commit the relevant changes in `db/structure.sql`
That's all of the required database changes.
@@ -288,14 +188,22 @@ That's all of the required database changes.
#### Step 1. Implement replication and verification
-- [ ] Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
+- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
+ - Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
+ - Include the `::Gitlab::Geo::VerificationState` concern.
+ - Delegate verification related methods to the `cool_widget_state` model.
+ - For verification, override some scopes to use the `cool_widget_states` table instead of the model table.
+ - Implement the `verification_state_object` method to return the object that holds
+ the verification details
+ - Override some methods to use the `cool_widget_states` table in verification-related queries.
At this point the `CoolWidget` class should look like this:
```ruby
# frozen_string_literal: true
-
+
class CoolWidget < ApplicationRecord
+ ...
include ::Gitlab::Geo::ReplicableModel
include ::Gitlab::Geo::VerificationState
@@ -303,38 +211,84 @@ That's all of the required database changes.
mount_uploader :file, CoolWidgetUploader
+ has_one :cool_widget_state, autosave: false, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
+
+ delegate :verification_retry_at, :verification_retry_at=,
+ :verified_at, :verified_at=,
+ :verification_checksum, :verification_checksum=,
+ :verification_failure, :verification_failure=,
+ :verification_retry_count, :verification_retry_count=,
+ :verification_state=, :verification_state,
+ :verification_started_at=, :verification_started_at,
+ to: :cool_widget_state
+ ...
+
+ scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) }
+ scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) }
+ scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) }
+
# Override the `all` default if not all records can be replicated. For an
# example of an existing Model that needs to do this, see
# `EE::MergeRequestDiff`.
# scope :available_replicables, -> { all }
- # @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
- # @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
- def self.replicables_for_current_secondary(primary_key_in)
- # This issue template does not help you write this method.
- #
- # This method is called only on Geo secondary sites. It is called when
- # we want to know which records to replicate. This is not easy to automate
- # because for example:
- #
- # * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
- # namespace, but the nature of the relationship to a namespace varies
- # between Models.
- # * The "selective sync" feature allows admins to choose which shards to
- # replicate, per secondary site. Repositories are associated with
- # shards. Most blob types are not, but Project Uploads are.
- # * Remote stored replicables are not replicated, by default. But the
- # setting `sync_object_storage` enables replication of remote stored
- # replicables.
- #
- # Search the codebase for examples, and consult a Geo expert if needed.
+ def verification_state_object
+ cool_widget_state
end
...
+
+ class_methods do
+ extend ::Gitlab::Utils::Override
+ ...
+
+ # @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
+ # @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
+ def self.replicables_for_current_secondary(primary_key_in)
+ # This issue template does not help you write this method.
+ #
+ # This method is called only on Geo secondary sites. It is called when
+ # we want to know which records to replicate. This is not easy to automate
+ # because for example:
+ #
+ # * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
+ # namespace, but the nature of the relationship to a namespace varies
+ # between Models.
+ # * The "selective sync" feature allows admins to choose which shards to
+ # replicate, per secondary site. Repositories are associated with
+ # shards. Most blob types are not, but Project Uploads are.
+ # * Remote stored replicables are not replicated, by default. But the
+ # setting `sync_object_storage` enables replication of remote stored
+ # replicables.
+ #
+ # Search the codebase for examples, and consult a Geo expert if needed.
+ end
+
+ override :verification_state_table_class
+ def verification_state_table_class
+ CoolWidgetState
+ end
+ end
+ ...
+
+ def cool_widget_state
+ super || build_cool_widget_state
+ end
+
+ ...
end
```
- [ ] Implement `CoolWidget.replicables_for_current_secondary` above.
- [ ] Ensure `CoolWidget.replicables_for_current_secondary` is well-tested. Search the codebase for `replicables_for_current_secondary` to find examples of parameterized table specs. You may need to add more `FactoryBot` traits.
+- [ ] Add the following shared examples to `ee/spec/models/ee/cool_widget_spec.rb`:
+
+ ```ruby
+ include_examples 'a replicable model with a separate table for verification state' do
+ let(:verifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is included in `available_verifiables`
+ let(:unverifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is NOT included in `available_verifiables`
+ end
+ ```
+
- [ ] Create `ee/app/replicators/geo/cool_widget_replicator.rb`. Implement the `#carrierwave_uploader` method which should return a `CarrierWave::Uploader`, and implement the class method `.model` to return the `CoolWidget` class:
```ruby
@@ -498,13 +452,7 @@ That's all of the required database changes.
- [ ] Make sure the factory also allows setting a `project` attribute. If the model does not have a direct relation to a project, you can use a `transient` attribute. Check out `spec/factories/merge_request_diffs.rb` for an example.
-##### If you added verification state fields to a separate table (option 2 above), then you need to make additional model and factory changes
-
-If you did not add verification state fields to a separate table, `cool_widget_states`, then skip to [Step 2. Implement metrics gathering](#step-2-implement-metrics-gathering).
-
-Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309).
-
-- [ ] Add a `Geo::CoolWidgetState` model in `ee/app/models/ee/geo/cool_widget_state.rb`:
+- [ ] Following [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309) add a `Geo::CoolWidgetState` model in `ee/app/models/ee/geo/cool_widget_state.rb`:
``` ruby
module Geo
@@ -536,63 +484,6 @@ Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.co
end
```
-- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
- - Include the `::Gitlab::Geo::VerificationState` concern.
- - Delegate verification related methods to the `cool_widget_state` model.
- - Override some scopes to use the `cool_widget_states` table instead of the model table, for verification.
- - Override some methods to use the `cool_widget_states` table in verification related queries.
-
- ```ruby
- class CoolWidget < ApplicationRecord
- ...
- include ::Gitlab::Geo::VerificationState
-
- has_one :cool_widget_state, autosave: true, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
-
- delegate :verification_retry_at, :verification_retry_at=,
- :verified_at, :verified_at=,
- :verification_checksum, :verification_checksum=,
- :verification_failure, :verification_failure=,
- :verification_retry_count, :verification_retry_count=,
- :verification_state=, :verification_state,
- :verification_started_at=, :verification_started_at,
- to: :cool_widget_state
- ...
-
- scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) }
- scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) }
- scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) }
-
- ...
-
- class_methods do
- extend ::Gitlab::Utils::Override
- ...
- override :verification_state_table_name
- def verification_state_table_name
- 'cool_widget_states'
- end
-
- override :verification_state_model_key
- def verification_state_model_key
- 'cool_widget_id'
- end
-
- override :verification_arel_table
- def verification_arel_table
- CoolWidgetState.arel_table
- end
- end
- ...
-
- def cool_widget_state
- super || build_cool_widget_state
- end
-
- ...
- end
- ```
-
#### Step 2. Implement metrics gathering
Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus` for display in the UI, and sent to Prometheus:
diff --git a/.gitlab/issue_templates/InfraDev.md b/.gitlab/issue_templates/InfraDev.md
index bc0e65c3c22..d337e75289c 100644
--- a/.gitlab/issue_templates/InfraDev.md
+++ b/.gitlab/issue_templates/InfraDev.md
@@ -53,4 +53,4 @@ See also:
-->
/label ~"infradev"
-/label ~"bug"
+/label ~"type::bug"
diff --git a/.gitlab/issue_templates/Problem Validation.md b/.gitlab/issue_templates/Problem Validation.md
index 5d417c5a26d..3f92510b6af 100644
--- a/.gitlab/issue_templates/Problem Validation.md
+++ b/.gitlab/issue_templates/Problem Validation.md
@@ -1,3 +1,7 @@
+<!-- This template is used as a starting point for understing and articulating a customer problem.
+Learn more about it in the handbook: https://about.gitlab.com/handbook/product-development-flow/#validation-phase-2-problem-validation
+-->
+
## Problem Statement
<!-- What is the problem we hope to validate? Reference how to write a real customer problem statement at https://productcoalition.com/how-to-write-a-good-customer-problem-statement-a815f80189ba for guidance. -->
@@ -45,4 +49,8 @@ For example, if the solution will take a product manager, designer, and engineer
- [ ] The problem is well described and detailed with necessary requirements for product design to understand the problem
- [ ] The problem is well described and detailed with necessary requirements for engineering to understand the problem
+## Research Issue
+
+<!-- Link to the Problem Validation Research issue that will be executed by the UX Researcher. https://gitlab.com/gitlab-org/ux-research/ -->
+
/label ~"workflow::validation backlog" ~devops:: ~category: ~group::
diff --git a/.gitlab/issue_templates/Productivity Improvement.md b/.gitlab/issue_templates/Productivity Improvement.md
index 06692d3ede8..040d8ea2f89 100644
--- a/.gitlab/issue_templates/Productivity Improvement.md
+++ b/.gitlab/issue_templates/Productivity Improvement.md
@@ -2,7 +2,7 @@
<!--
Please describe the engineering productivity problem that needs to be solved backed by charts from
-https://about.gitlab.com/handbook/engineering/quality/engineering-productivity-team/#engineering-productivity-team-metrics.
+https://about.gitlab.com/handbook/engineering/quality/engineering-productivity/#engineering-productivity-metrics.
-->
### Problem identification checklist
diff --git a/.gitlab/issue_templates/Refactoring.md b/.gitlab/issue_templates/Refactoring.md
index d9466185ff7..df18dcf7656 100644
--- a/.gitlab/issue_templates/Refactoring.md
+++ b/.gitlab/issue_templates/Refactoring.md
@@ -41,9 +41,9 @@ please list them here.
<!--
Please select the appropriate label from the following:
~"feature::addition"
- ~"feature::maintenance"
+ ~"type::maintenance"
~"tooling::pipelines"
~"tooling::workflow"
-->
-/label ~"feature::maintenance"
+/label ~"type::maintenance"
diff --git a/.gitlab/issue_templates/Security developer workflow.md b/.gitlab/issue_templates/Security developer workflow.md
index 7f2c54f4f49..6bf9e6971d7 100644
--- a/.gitlab/issue_templates/Security developer workflow.md
+++ b/.gitlab/issue_templates/Security developer workflow.md
@@ -12,7 +12,7 @@ Set the title to: `Description of the original issue`
- [ ] Make sure the issue really needs to follow the security release workflow.
- Verify if the issue you're working on `gitlab-org/gitlab` is confidential, if it's public fix should be placed on GitLab canonical and no backports are required.
- If the issue you're fixing doesn't appear to be something that can be exploited by a malicious person and is instead simply a security enhancement do not hesitate to ping `@gitlab-com/gl-security/appsec` to discuss if the issue can be fixed in the canonical repository.
-- [ ] **IMPORTANT**: Mark this [issue as linked] to the Security Release Tracking Issue. You can find it on the topic of the `#releases` Slack channel. This issue
+- [ ] **IMPORTANT**: Mark this [issue as linked] to the Security Release Tracking Issue. You can find it [here](https://gitlab.com/gitlab-org/gitlab/-/issues?sort=created_date&state=opened&label_name[]=upcoming+security+release). This issue
MUST be linked for the release bot to know that the associated merge requests should be merged for this security release.
- Fill out the [Links section](#links):
- [ ] Next to **Issue on GitLab**, add a link to the `gitlab-org/gitlab` issue that describes the security vulnerability.
diff --git a/.gitlab/issue_templates/Technical Evaluation.md b/.gitlab/issue_templates/Technical Evaluation.md
index cf939725a78..680ecb7d9a6 100644
--- a/.gitlab/issue_templates/Technical Evaluation.md
+++ b/.gitlab/issue_templates/Technical Evaluation.md
@@ -29,5 +29,5 @@
### Team
-- [ ] Add ~"workflow::planning breakdown" ~feature and the corresponding `~devops::<stage>` and `~group::<group>` labels.
+- [ ] Add ~"workflow::planning breakdown" ~"type::feature" and the corresponding `~devops::<stage>` and `~group::<group>` labels.
- [ ] Ping the PM and EM.
diff --git a/.gitlab/merge_request_templates/Deprecations.md b/.gitlab/merge_request_templates/Deprecations.md
index 8431e9ca393..1449246b9bc 100644
--- a/.gitlab/merge_request_templates/Deprecations.md
+++ b/.gitlab/merge_request_templates/Deprecations.md
@@ -6,16 +6,19 @@
**Be sure to link this MR to the relevant deprecation issue(s).**
+Deprecation announcements can and should be created and merged into Docs at any time, to optimize user awareness and planning. We encourage confirmed deprecations to be merged as soon as the required reviews are complete, even if weeks ahead of the target milestone's release post. For the announcement to be included in a specific release post and that release's documentation packages, this MR must be reviewed/merged per the due dates below:
+
**By the 10th**: Assign this MR to these team members as Reviewer and for Approval (optional unless noted as required):
- Product Marketing: `@PMM`
- Product Designer(s): `@ProductDesigners`
- Group Manager or Director: `@manager`
- Engineering Manager: `@EM` - Required
+- Technical writer: `@TW` - Required
-**By 8:00 AM PDT 15th**: PM will assign this MR to the TW reviewer: `@PM`
+**By 11:59 AM PDT 15th**: PM assigns this MR to the TW reviewer for final review and merge: `@PM`
-**By 11:59 PM PDT 15th**: TW Reviewer will perform final review and merge this MR to Master: `@TW`
+**By 11:59 PM PDT 17th**: TW Reviewer updates Docs by merging this MR to `master`: `@TW`
---
@@ -31,8 +34,9 @@ They are frequently updated, and everyone should make sure they are aware of the
## PM release post item checklist
- [ ] Set yourself as the Assignee.
+- [ ] If the deprecation is a [breaking change](https://about.gitlab.com/handbook/product/gitlab-the-product/#breaking-change), add label `breaking change`.
- [ ] Follow the process to [create a deprecation YAML file](https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-a-deprecation-entry).
-- [ ] Add reviewers by the 10th
+- [ ] Add reviewers by the 10th.
- [ ] When ready to be merged and not later than the 15th, add the ~ready label and @ message the TW for final review and merge.
## Reviewers
@@ -78,5 +82,18 @@ yourself as a reviewer if it's not ready for merge yet.
</details>
-When the PM indicates it is ready for merge, all issues have been addressed merge this MR.
- - You must merge this MR by the 15th so the Release Post TW lead can run the [deprecations in Docs rake task](https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-deprecations-doc) on the 16th
+When the PM indicates it is ready for merge and all issues have been addressed, start the merge process.
+
+#### Technical writer merge process
+
+The [deprecations doc's `.md` file](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/update/deprecations.md)
+must be updated before this MR is merged:
+
+1. Check out the MR's branch (in the [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) project).
+1. From the command line (in the branch), run `bin/rake gitlab:docs:compile_deprecations`.
+ If you want to double check that it worked, you can run `bin/rake gitlab:docs:check_deprecations`
+ to verify that the doc is up to date.
+1. Commit the updated file and push the changes.
+1. Set the MR to merge when the pipeline succeeds (or merge if the pipeline is already complete).
+
+If you have trouble running the rake task, check the [troubleshooting steps](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecation-rake-task-troubleshooting).
diff --git a/.gitlab/merge_request_templates/Documentation.md b/.gitlab/merge_request_templates/Documentation.md
index 66c1eff412b..893ae7b93b5 100644
--- a/.gitlab/merge_request_templates/Documentation.md
+++ b/.gitlab/merge_request_templates/Documentation.md
@@ -20,10 +20,10 @@
If you are only adding documentation, do not add any of the following labels:
-- `~"feature"`
+- `~"type::feature"`
- `~"frontend"`
- `~"backend"`
-- `~"bug"`
+- `~"type::bug"`
- `~"database"`
These labels cause the MR to be added to code verification QA issues.
diff --git a/.gitlab/merge_request_templates/New End To End Test.md b/.gitlab/merge_request_templates/New End To End Test.md
index f9664c6315f..9ecf8999f66 100644
--- a/.gitlab/merge_request_templates/New End To End Test.md
+++ b/.gitlab/merge_request_templates/New End To End Test.md
@@ -24,5 +24,5 @@ Please link to the respective test case in the testcases project
/label ~"Quality:test-gap" ~"Quality:EE test gaps"
-<!-- Select the appropriate feature label, ~"feature::addition" for tests added for new features, ~"feature::maintenance" for tests added for existing features -->
-/label ~"feature::addition" ~"feature::maintenance"
+<!-- Select the appropriate feature label, ~"feature::addition" for tests added for new features, ~"type::maintenance" for tests added for existing features -->
+/label ~"feature::addition" ~"type::maintenance"
diff --git a/.gitlab/merge_request_templates/Pipeline Configuration.md b/.gitlab/merge_request_templates/Pipeline Configuration.md
index 920abf086cb..62210028c18 100644
--- a/.gitlab/merge_request_templates/Pipeline Configuration.md
+++ b/.gitlab/merge_request_templates/Pipeline Configuration.md
@@ -33,6 +33,6 @@ This will help keep track of expected cost increases to the [GitLab project aver
### Post-merge
-- [ ] Consider communicating these changes to the broader team following the [communication guideline for pipeline changes](https://about.gitlab.com/handbook/engineering/quality/engineering-productivity-team/#pipeline-changes)
+- [ ] Consider communicating these changes to the broader team following the [communication guideline for pipeline changes](https://about.gitlab.com/handbook/engineering/quality/engineering-productivity/#pipeline-changes)
-/label ~tooling ~"tooling::pipelines" ~"Engineering Productivity"
+/label ~"type::tooling" ~"tooling::pipelines" ~"Engineering Productivity"
diff --git a/.gitlab/merge_request_templates/Quarantine End to End Test.md b/.gitlab/merge_request_templates/Quarantine End to End Test.md
index 4edfd2a8c8e..4caebb7f1bb 100644
--- a/.gitlab/merge_request_templates/Quarantine End to End Test.md
+++ b/.gitlab/merge_request_templates/Quarantine End to End Test.md
@@ -30,7 +30,7 @@ the noise (due to constantly failing tests, flaky tests, and so on) so that new
- [ ] To ensure a faster turnaround, ask in the `#quality` Slack channel for someone to review and merge the merge request, rather than assigning it directly.
<!-- Base labels. -->
-/label ~"Quality" ~"QA" ~"feature" ~"feature::maintenance"
+/label ~"Quality" ~"QA" ~"type::maintenance"
<!--
Choose the stage that appears in the test path, e.g. ~"devops::create" for