Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2021-11-18 16:16:36 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2021-11-18 16:16:36 +0300
commit311b0269b4eb9839fa63f80c8d7a58f32b8138a0 (patch)
tree07e7870bca8aed6d61fdcc810731c50d2c40af47 /.gitlab/issue_templates
parent27909cef6c4170ed9205afa7426b8d3de47cbb0c (diff)
Add latest changes from gitlab-org/gitlab@14-5-stable-eev14.5.0-rc42
Diffstat (limited to '.gitlab/issue_templates')
-rw-r--r--.gitlab/issue_templates/Audit Event Proposal.md2
-rw-r--r--.gitlab/issue_templates/Bug.md6
-rw-r--r--.gitlab/issue_templates/Deprecations.md6
-rw-r--r--.gitlab/issue_templates/Empty state.md80
-rw-r--r--.gitlab/issue_templates/Experiment Implementation.md (renamed from .gitlab/issue_templates/Experimentation.md)2
-rw-r--r--.gitlab/issue_templates/Experiment Rollout.md11
-rw-r--r--.gitlab/issue_templates/Experiment Successful Cleanup.md5
-rw-r--r--.gitlab/issue_templates/Feature Flag Roll Out.md26
-rw-r--r--.gitlab/issue_templates/Feature Proposal - basic.md10
-rw-r--r--.gitlab/issue_templates/Feature Proposal - lean.md93
-rw-r--r--.gitlab/issue_templates/Feature proposal - detailed.md12
-rw-r--r--.gitlab/issue_templates/Geo Replicate a new Git repository type.md278
-rw-r--r--.gitlab/issue_templates/Geo Replicate a new blob type.md275
-rw-r--r--.gitlab/issue_templates/InfraDev.md2
-rw-r--r--.gitlab/issue_templates/Problem Validation.md8
-rw-r--r--.gitlab/issue_templates/Productivity Improvement.md2
-rw-r--r--.gitlab/issue_templates/Refactoring.md4
-rw-r--r--.gitlab/issue_templates/Security developer workflow.md2
-rw-r--r--.gitlab/issue_templates/Technical Evaluation.md2
19 files changed, 347 insertions, 479 deletions
diff --git a/.gitlab/issue_templates/Audit Event Proposal.md b/.gitlab/issue_templates/Audit Event Proposal.md
index 7a5408ca1f2..af32845bd9a 100644
--- a/.gitlab/issue_templates/Audit Event Proposal.md
+++ b/.gitlab/issue_templates/Audit Event Proposal.md
@@ -9,5 +9,5 @@
<!-- Describe the audit event you are proposing should be added, including any details of what should be captured, how, and why. -->
/label ~"Category:Audit Events"
-/label ~"feature"
+/label ~"type::feature"
/label ~"group::compliance"
diff --git a/.gitlab/issue_templates/Bug.md b/.gitlab/issue_templates/Bug.md
index 41b694fdf2c..b9fed3745d1 100644
--- a/.gitlab/issue_templates/Bug.md
+++ b/.gitlab/issue_templates/Bug.md
@@ -2,10 +2,10 @@
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
-filtered by the "regression" or "bug" label:
+filtered by the "regression" or "type::bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
-- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=bug
+- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug
and verify the issue you're about to submit isn't a duplicate.
--->
@@ -82,4 +82,4 @@ will also determine whether the bug is fixed in a more recent version. -->
<!-- If you can, link to the line of code that might be responsible for the problem. -->
-/label ~bug
+/label ~"type::bug"
diff --git a/.gitlab/issue_templates/Deprecations.md b/.gitlab/issue_templates/Deprecations.md
index ff51699c6be..caef5c64334 100644
--- a/.gitlab/issue_templates/Deprecations.md
+++ b/.gitlab/issue_templates/Deprecations.md
@@ -41,7 +41,11 @@ Which tier is this feature available in?
### Deprecation Milestone
-<!-- In which milestone will this deprecation happen? -->
+<!-- In which milestone will this deprecation be announced ? -->
+
+### Planned Removal Milestone
+
+<!-- In which milestone will the feature or functionality be removed and announced? -->
### Links
diff --git a/.gitlab/issue_templates/Empty state.md b/.gitlab/issue_templates/Empty state.md
new file mode 100644
index 00000000000..d92ea8522e1
--- /dev/null
+++ b/.gitlab/issue_templates/Empty state.md
@@ -0,0 +1,80 @@
+<!-- Before implementing a new empty state solution, make sure to read the
+Empty State region docs in Pajamas: https://design.gitlab.com/regions/empty-states -->
+
+## Description
+
+<!-- Describe the solution you're proposing for your empty state region.
+Include links to user research (if applicable). -->
+
+## Location
+
+<!-- Provide a link and location of the new empty state solution.
+For example: https://gitlab.com/gitlab-org/gitlab-services/design.gitlab.com/-/issues -->
+
+## Use case
+
+<!-- What is the use case for the solution you're proposing?
+Read the Empty State docs and select the use case below: https://design.gitlab.com/regions/empty-states -->
+
+- [ ] Blank content
+- [ ] Empty search results
+- [ ] Configuration required
+- [ ] Higher tier
+
+## Checklist
+
+<!-- Follow the steps below that correspond with the use case selected above.
+Follow the steps to complete this issue -->
+
+### Blank content
+
+- [ ] The solution follows the `Blank content` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#blank-content).
+- [ ] Follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+
+### Empty search results
+
+- [ ] The solution follows the `Empty search results` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#empty-search-results).
+- [ ] Follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+
+### Configuration required
+
+- [ ] The solution follows the `Configuration required` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#configuration-required).
+- [ ] Ask a [Growth product manager or Designer](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to review your solution.
+- [ ] Is your solution introducing a new empty states or modifying an existing one?
+ - [ ] Introducing a new empty state: Follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+ - [ ] Modifying an existing empty state: Follow the [`Experimentation` process](#experimentation) below. _Note_: If the empty state you want to replace hasn't been updated in a long time, doesn't pitch the value of the feature, or does not contain a next step action CTA, then we recommend you skip the experimentation process to implement and add tracking to your new empty state.
+
+<!-- IF experimentation -->
+#### Experimentation
+
+- [ ] Collaborate with a [Growth product manager](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to help you determine if you can validate your solution through an experiment on SaaS.
+- [ ] If an experiment is possible, create an issue using the [experiment idea template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Experiment%20Idea) and follow the template intructions. Otherwise, follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+- [ ] Ask a [Growth product manager or Designer](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to review your experiment set-up.
+- [ ] Implement and monitor the experiment following the [implementation guide](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/experiment_guide/gitlab_experiment.md#implement-an-experiment).
+- [ ] Review and discuss the findings.
+- [ ] Add the findings to the [Growth experimentation knowledge](https://about.gitlab.com/direction/growth/#growth-experiments-knowledge-base---concluded-experiments).
+
+### Higher tier
+
+- [ ] The solution follows the `Higher tier` specifications [in Pajamas](https://design.gitlab.com/regions/empty-states#higher-tier).
+- [ ] Ask a Product Manager or Designer from the [Conversion group](https://about.gitlab.com/handbook/engineering/development/growth/conversion/#group-members) to review your solution.
+- [ ] Is your solution introducing a new empty states or modifying an existing one?
+ - [ ] Introducing a new empty state: follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+ - [ ] Modifying an existing empty state, follow the [`Experimentation` process](#experimentation) below.
+
+<!-- IF experimentation -->
+#### Experimentation
+
+- [ ] Collaborate with a [Growth product manager](https://about.gitlab.com/handbook/engineering/development/growth/#stable-counterparts) to help you determine if you can validate your solution through an experiment on SaaS.
+- [ ] If an experiment is possible, create an issue using the [experiment idea template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Experiment%20Idea) and follow the template intructions. Otherwise, follow the instructions from the [`After merge` section](#after-merge) below to add Snowplow tracking.
+- [ ] Add a ~"Category:Conversion Experiment" label to the experiment idea issue.
+- [ ] Ask a Product Manager or Designer from the [Conversion group](https://about.gitlab.com/handbook/engineering/development/growth/conversion/#group-members) to review your experiment set-up.
+- [ ] Implement and monitor the experiment following the [implementation guide](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/experiment_guide/gitlab_experiment.md#implement-an-experiment) .
+- [ ] Review and discuss the findings.
+- [ ] Add the findings to the [Growth experimentation knowledge](https://about.gitlab.com/direction/growth/#growth-experiments-knowledge-base---concluded-experiments).
+
+
+## After merge
+
+- [ ] Use the `Snowplow event tracking` [issue template](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Snowplow%20event%20tracking) and open an issue to add Snowplow event tracking to your new empty state solution.
+ - [ ] Add your ~devops:: and ~group:: labels to the new issue.
diff --git a/.gitlab/issue_templates/Experimentation.md b/.gitlab/issue_templates/Experiment Implementation.md
index ba7839fb941..b3883f08c25 100644
--- a/.gitlab/issue_templates/Experimentation.md
+++ b/.gitlab/issue_templates/Experiment Implementation.md
@@ -1,4 +1,4 @@
-<!-- Title suggestion: Experiment: [description] -->
+<!-- Title suggestion: Experiment Implementation: [description] -->
# Experiment Summary
<!-- Quick rundown of what is being done -->
diff --git a/.gitlab/issue_templates/Experiment Rollout.md b/.gitlab/issue_templates/Experiment Rollout.md
index c5fdc739943..9209423ba33 100644
--- a/.gitlab/issue_templates/Experiment Rollout.md
+++ b/.gitlab/issue_templates/Experiment Rollout.md
@@ -105,5 +105,16 @@ In this rollout issue, ensure the scoped `experiment::` label is kept accurate.
/chatops run feature set <experiment-key> false
```
+## Experiment Successful Cleanup Concerns
+
+_Items to be considered if candidate experience is to become a permanent part of GitLab_
+
+<!--
+Add a list of items raised during MR review or otherwise that may need further thought/consideration
+before becoming permanent parts of the product.
+
+Example: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70451#note_727246104
+-->
+
/label ~"feature flag" ~"devops::growth" ~"growth experiment" ~"experiment-rollout" ~Engineering ~"workflow::scheduling" ~"experiment::pending"
/milestone %"Next 1-3 releases"
diff --git a/.gitlab/issue_templates/Experiment Successful Cleanup.md b/.gitlab/issue_templates/Experiment Successful Cleanup.md
index 42f26342342..1dd57332b8e 100644
--- a/.gitlab/issue_templates/Experiment Successful Cleanup.md
+++ b/.gitlab/issue_templates/Experiment Successful Cleanup.md
@@ -12,9 +12,10 @@ The changes need to become an official part of the product.
- [ ] Determine if tracking should be kept as is, removed, or modified.
- [ ] Ensure any relevant documentation has been updated.
- [ ] Consider changes to any `feature_category:` introduced by the experiment if ownership is changing (PM for Growth and PM for the new category as DRIs)
+- [ ] Check to see if the experiment introduced new design assets. Add them to the appropriate repos and document them if needed.
- [ ] Optional: Migrate experiment to a default enabled [feature flag](https://docs.gitlab.com/ee/development/feature_flags) for one milestone and add a changelog. Converting to a feature flag can be skipped at the ICs discretion if risk is deemed low with consideration to both SaaS and (if applicable) self managed
- [ ] In the next milestone, [remove the feature flag](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up) if applicable
- [ ] After the flag removal is deployed, [clean up the feature/experiment feature flags](https://docs.gitlab.com/ee/development/feature_flags/controls.html#cleaning-up) by running chatops command in `#production` channel
-- [ ] Ensure the corresponding [Experiment Tracking](https://gitlab.com/groups/gitlab-org/-/boards/1352542?label_name[]=devops%3A%3Agrowth&label_name[]=growth%20experiment&label_name[]=experiment%20tracking) issue is updated
+- [ ] Ensure the corresponding [Experiment Rollout](https://gitlab.com/groups/gitlab-org/-/boards/1352542?label_name[]=devops%3A%3Agrowth&label_name[]=growth%20experiment&label_name[]=experiment-rollout) issue is updated
-/label ~"feature" ~"feature::maintenance" ~"workflow::scheduling" ~"growth experiment" ~"feature flag"
+/label ~"type::maintenance" ~"workflow::scheduling" ~"growth experiment" ~"feature flag"
diff --git a/.gitlab/issue_templates/Feature Flag Roll Out.md b/.gitlab/issue_templates/Feature Flag Roll Out.md
index 00b396bac4e..bc1a23729e2 100644
--- a/.gitlab/issue_templates/Feature Flag Roll Out.md
+++ b/.gitlab/issue_templates/Feature Flag Roll Out.md
@@ -30,6 +30,17 @@ Are there any other stages or teams involved that need to be kept in the loop?
<!-- Describe the expected outcome when rolling out this feature -->
+### When is the feature viable?
+
+<!-- What are the settings we need to configure in order to have this feature viable? -->
+
+<!--
+Example below:
+
+1. Enable service ping collection
+ `ApplicationSetting.first.update(usage_ping_enabled: true)`
+-->
+
### What might happen if this goes wrong?
<!-- Should the feature flag be turned off? Any MRs that need to be rolled back? Communication that needs to happen? What are some things you can think of that could go wrong - data loss or broken pages? -->
@@ -37,6 +48,12 @@ Are there any other stages or teams involved that need to be kept in the loop?
### What can we monitor to detect problems with this?
<!-- Which dashboards from https://dashboards.gitlab.net are most relevant? -->
+_Consider mentioning checks for 5xx errors or other anomalies like an increase in redirects
+(302 HTTP response status)_
+
+### What can we check for monitoring production after rollouts?
+
+_Consider adding links to check for Sentry errors, Production logs for 5xx, 302s, etc._
## Rollout Steps
@@ -73,11 +90,14 @@ Are there any other stages or teams involved that need to be kept in the loop?
If a different developer will be covering, or an exception is needed, please inform the oncall SRE by using the `@sre-oncall` Slack alias.
- [ ] Ensure that documentation has been updated ([More info](https://docs.gitlab.com/ee/development/documentation/feature_flags.html#features-that-became-enabled-by-default)).
- [ ] Announce on [the feature issue](ISSUE LINK) an estimated time this will be enabled on GitLab.com.
-- [ ] If the feature might impact the user experience, notify `#support_gitlab-com` and your team channel ([more guidance when this is necessary in the dev docs](https://docs.gitlab.com/ee/development/feature_flags/controls.html#communicate-the-change)).
+- [ ] Notify `#support_gitlab-com` and your team channel ([more guidance when this is necessary in the dev docs](https://docs.gitlab.com/ee/development/feature_flags/controls.html#communicate-the-change)).
### Global rollout on production
-All `/chatops` commands that target production should be done in the `#production` slack channel for visibility.
+For visibility, all `/chatops` commands that target production should be:
+
+- Executed in the `#production` slack channel.
+- Cross-posted (with the command results) to the responsible team's slack channel (`#g_TEAM_NAME`).
- [ ] [Incrementally roll out](https://docs.gitlab.com/ee/development/feature_flags/controls.html#process) the feature.
- If the feature flag in code has [an actor](https://docs.gitlab.com/ee/development/feature_flags/#feature-actors), perform **actor-based** rollout.
@@ -148,5 +168,5 @@ codebase.
/chatops run feature set <feature-flag-name> false
```
-/label ~"feature flag"
+/label ~"feature flag" ~"type::feature" ~"feature::addition"
/assign DRI
diff --git a/.gitlab/issue_templates/Feature Proposal - basic.md b/.gitlab/issue_templates/Feature Proposal - basic.md
index 0c05b7a0165..980751621f0 100644
--- a/.gitlab/issue_templates/Feature Proposal - basic.md
+++ b/.gitlab/issue_templates/Feature Proposal - basic.md
@@ -1,10 +1,16 @@
<!-- This template is a great use for issues that are feature::additions or technical tasks for larger issues.-->
-### Proposal
+### Proposal
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
<!-- Consider adding related issues and epics to this issue. You can also reference the Feature Proposal Template (https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md) for additional details to consider adding to this issue. Additionally, as a data oriented organization, when your feature exits planning breakdown, consider adding the `What does success look like, and how can we measure that?` section.
-->
-/label ~feature::addition ~"group::" ~"section::" ~"Category:" ~"GitLab Core"/~"GitLab Premium"/~"GitLab Ultimate"
+<!-- Label reminders
+Use the following resources to find the appropriate labels:
+- https://gitlab.com/gitlab-org/gitlab/-/labels
+- https://about.gitlab.com/handbook/product/categories/features/
+-->
+
+/label ~"type::feature" ~feature::addition ~"group::" ~"section::" ~"Category:" ~"GitLab Core"/~"GitLab Premium"/~"GitLab Ultimate"
diff --git a/.gitlab/issue_templates/Feature Proposal - lean.md b/.gitlab/issue_templates/Feature Proposal - lean.md
index 9dd4bdc6b22..504bfbb03d8 100644
--- a/.gitlab/issue_templates/Feature Proposal - lean.md
+++ b/.gitlab/issue_templates/Feature Proposal - lean.md
@@ -1,33 +1,22 @@
-<!-- This issue template can be used a great starting point for feature requests. The last section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab. -->
+<!-- This issue template can be used as a great starting point for feature requests. The section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab.
-### Release notes
+The goal of this template is brevity for quick/smaller iterations. For a more thorough list of considerations for larger features or feature sets, you can leverage the detailed [feature proposal](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md). -->
+
+### Release notes
<!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " -->
-### Problem to solve
+### Problem to solve
<!-- What is the user problem you are trying to solve with this issue? -->
-### Proposal
+### Proposal
<!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. -->
-
-
-/label ~"feature" ~"group::" ~"section::" ~"Category::" ~"GitLab Free"/~"GitLab Premium"/~"GitLab Ultimate"
-
-
-<!--- Use the following resources to find the appropriate labels:
-- https://gitlab.com/gitlab-org/gitlab/-/labels
-- https://about.gitlab.com/handbook/product/categories/features/
-
-Consider adding related issues and epics to this issue. You can also reference the Feature Proposal Template (https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md) for additional details to consider adding to this issue. Additionally, as a data oriented organization, when your feature exits planning breakdown, consider adding the `What does success look like, and how can we measure that?` section.
-
-Other sections to consider adding:
-
### Intended users
-Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later.
+<!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later.
Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/
@@ -39,68 +28,28 @@ Personas are described at https://about.gitlab.com/handbook/marketing/product-ma
* [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer)
* [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator)
* [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst)
-* [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager)
+* [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager)
* [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer)
* [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test)
-* [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
+* [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
* [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
* [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst)
+* [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor)
+-->
-### User experience goal
-
-What is the single user experience workflow this problem addresses?
-For example, "The user should be able to use the UI/API/.gitlab-ci.yml with GitLab to <perform a specific task>"
-https://about.gitlab.com/handbook/engineering/ux/ux-research-training/user-story-mapping/
-
-
-### Further details
-
-Include use cases, benefits, goals, or any other details that will help us understand the problem better.
+### Metrics
-### Permissions and Security
+<!-- How are you going to track uage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it?
-<!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)?
-Consider adding checkboxes and expectations of users with certain levels of membership https://docs.gitlab.com/ee/user/permissions.html
-* [ ] Add expected impact to members with no access (0)
-* [ ] Add expected impact to Guest (10) members
-* [ ] Add expected impact to Reporter (20) members
-* [ ] Add expected impact to Developer (30) members
-* [ ] Add expected impact to Maintainer (40) members
-* [ ] Add expected impact to Owner (50) members
+Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
-### Documentation
+-->
- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change
-
-* Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/workflow.html
-* If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html
-
-### Availability & Testing
-
-This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.
-
-What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?
-
-Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
-* Unit test changes
-* Integration test changes
-* End-to-end test change
-
-See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning
-
-### What does success look like, and how can we measure that?
-
-Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this.
-
-### What is the type of buyer?
-
-What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/
-In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#three-tiers
-
-### Is this a cross-stage feature?
-
-Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features -->
+<!-- Label reminders
+Use the following resources to find the appropriate labels:
+- https://gitlab.com/gitlab-org/gitlab/-/labels
+- https://about.gitlab.com/handbook/product/categories/features/
+-->
-/label ~documentation
-/label ~direction
+/label ~"type::feature" ~"group::" ~"section::" ~"Category::" ~"GitLab Free"/~"GitLab Premium"/~"GitLab Ultimate" ~documentation ~direction
diff --git a/.gitlab/issue_templates/Feature proposal - detailed.md b/.gitlab/issue_templates/Feature proposal - detailed.md
index 9759bb7e2dc..c787fc99333 100644
--- a/.gitlab/issue_templates/Feature proposal - detailed.md
+++ b/.gitlab/issue_templates/Feature proposal - detailed.md
@@ -31,6 +31,14 @@ Personas are described at https://about.gitlab.com/handbook/marketing/product-ma
* [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor)
-->
+### Metrics
+
+<!-- How are you going to track uage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it?
+
+Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md
+
+-->
+
### User experience goal
<!-- What is the single user experience workflow this problem addresses?
@@ -112,6 +120,4 @@ Use the following resources to find the appropriate labels:
-->
/label ~devops:: ~group: ~Category:
/label ~"GitLab Free"/~"GitLab Premium"/~"GitLab Ultimate"
-/label ~feature
-/label ~documentation
-/label ~direction
+/label ~"type::feature" ~documentation ~direction
diff --git a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
index 0d822945798..71a962d1789 100644
--- a/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
+++ b/.gitlab/issue_templates/Geo Replicate a new Git repository type.md
@@ -111,115 +111,9 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
- [ ] Be sure to commit the relevant changes in `ee/db/geo/structure.sql`
-### Add verification state fields on the Geo primary site
+### Add verification state to the Model
-The Geo primary site needs to checksum every replicable in order for secondaries to verify their own checksums. To do this, Geo requires fields on the Model. There are two ways to add the necessary verification state fields. If the table is large and wide, then it may be a good idea to add verification state fields to a separate table (Option 2). Consult a database expert if needed.
-
-#### Add verification state fields to the model table (Option 1)
-
-- [ ] Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationStateToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationStateToCoolWidgets < ActiveRecord::Migration[6.0]
- def change
- change_table(:cool_widgets) do |t|
- t.integer :verification_state, default: 0, limit: 2, null: false
- t.column :verification_started_at, :datetime_with_timezone
- t.integer :verification_retry_count, limit: 2, null: false
- t.column :verification_retry_at, :datetime_with_timezone
- t.column :verified_at, :datetime_with_timezone
- t.binary :verification_checksum, using: 'verification_checksum::bytea'
-
- t.text :verification_failure # rubocop:disable Migration/AddLimitToTextColumns
- end
- end
- end
- ```
-
-- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
-- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
-- [ ] Adding a `text` column also [requires](../database/strings_and_the_text_data_type.md#add-a-text-column-to-an-existing-table) setting a limit. Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationFailureLimitToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationFailureLimitToCoolWidgets < ActiveRecord::Migration[6.0]
- include Gitlab::Database::MigrationHelpers
-
- disable_ddl_transaction!
-
- CONSTRAINT_NAME = 'cool_widget_verification_failure_text_limit'
-
- def up
- add_text_limit :cool_widget, :verification_failure, 255, constraint_name: CONSTRAINT_NAME
- end
-
- def down
- remove_check_constraint(:cool_widget, CONSTRAINT_NAME)
- end
- end
- ```
-
-- [ ] Add indexes on verification fields to ensure verification can be performed efficiently. Some or all of these indexes can be omitted if the table is guaranteed to be small. Ask a database expert if you are considering omitting indexes. Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationIndexesToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationIndexesToCoolWidgets < ActiveRecord::Migration[6.0]
- include Gitlab::Database::MigrationHelpers
-
- VERIFICATION_STATE_INDEX_NAME = "index_cool_widgets_on_verification_state"
- PENDING_VERIFICATION_INDEX_NAME = "index_cool_widgets_pending_verification"
- FAILED_VERIFICATION_INDEX_NAME = "index_cool_widgets_failed_verification"
- NEEDS_VERIFICATION_INDEX_NAME = "index_cool_widgets_needs_verification"
-
- disable_ddl_transaction!
-
- def up
- add_concurrent_index :cool_widgets, :verification_state, name: VERIFICATION_STATE_INDEX_NAME
- add_concurrent_index :cool_widgets, :verified_at, where: "(verification_state = 0)", order: { verified_at: 'ASC NULLS FIRST' }, name: PENDING_VERIFICATION_INDEX_NAME
- add_concurrent_index :cool_widgets, :verification_retry_at, where: "(verification_state = 3)", order: { verification_retry_at: 'ASC NULLS FIRST' }, name: FAILED_VERIFICATION_INDEX_NAME
- add_concurrent_index :cool_widgets, :verification_state, where: "(verification_state = 0 OR verification_state = 3)", name: NEEDS_VERIFICATION_INDEX_NAME
- end
-
- def down
- remove_concurrent_index_by_name :cool_widgets, VERIFICATION_STATE_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, PENDING_VERIFICATION_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, FAILED_VERIFICATION_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, NEEDS_VERIFICATION_INDEX_NAME
- end
- end
- ```
-
-- [ ] Run database migrations:
-
- ```shell
- bin/rake db:migrate
- ```
-
-- [ ] Be sure to commit the relevant changes in `db/structure.sql`
-
-#### Add verification state fields to a separate table (Option 2)
+The Geo primary site needs to checksum every replicable so secondaries can verify their own checksums. To do this, Geo requires the Model to have an associated table to track verification state.
- [ ] Create the migration file in `db/migrate`:
@@ -273,6 +167,7 @@ The Geo primary site needs to checksum every replicable in order for secondaries
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
+
- [ ] Run database migrations:
```shell
@@ -287,7 +182,14 @@ That's all of the required database changes.
#### Step 1. Implement replication and verification
-- [ ] Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
+- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
+ - Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
+ - Include the `::Gitlab::Geo::VerificationState` concern.
+ - Delegate verification related methods to the `cool_widget_state` model.
+ - For verification, override some scopes to use the `cool_widget_states` table instead of the model table.
+ - Implement the `verification_state_object` method to return the object that holds
+ the verification details
+ - Override some methods to use the `cool_widget_states` table in verification-related queries.
Pay some attention to method `pool_repository`. Not every repository type uses repository pooling. As Geo prefers to use repository snapshotting, it can lead to data loss. Make sure to overwrite `pool_repository` so it returns nil for repositories that do not have pools.
@@ -297,6 +199,7 @@ That's all of the required database changes.
# frozen_string_literal: true
class CoolWidget < ApplicationRecord
+ ...
include ::Gitlab::Geo::ReplicableModel
include ::Gitlab::Geo::VerificationState
@@ -304,31 +207,62 @@ That's all of the required database changes.
mount_uploader :file, CoolWidgetUploader
+ has_one :cool_widget_state, autosave: false, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
+
+ delegate :verification_retry_at, :verification_retry_at=,
+ :verified_at, :verified_at=,
+ :verification_checksum, :verification_checksum=,
+ :verification_failure, :verification_failure=,
+ :verification_retry_count, :verification_retry_count=,
+ :verification_state=, :verification_state,
+ :verification_started_at=, :verification_started_at,
+ to: :cool_widget_state
+ ...
+
+ scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) }
+ scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) }
+ scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) }
+
# Override the `all` default if not all records can be replicated. For an
# example of an existing Model that needs to do this, see
# `EE::MergeRequestDiff`.
# scope :available_replicables, -> { all }
- # @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
- # @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
- def self.replicables_for_current_secondary(primary_key_in)
- # This issue template does not help you write this method.
- #
- # This method is called only on Geo secondary sites. It is called when
- # we want to know which records to replicate. This is not easy to automate
- # because for example:
- #
- # * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
- # namespace, but the nature of the relationship to a namespace varies
- # between Models.
- # * The "selective sync" feature allows admins to choose which shards to
- # replicate, per secondary site. Repositories are associated with
- # shards. Most blob types are not, but Project Uploads are.
- # * Remote stored replicables are not replicated, by default. But the
- # setting `sync_object_storage` enables replication of remote stored
- # replicables.
- #
- # Search the codebase for examples, and consult a Geo expert if needed.
+ def verification_state_object
+ cool_widget_state
+ end
+ ...
+
+ class_methods do
+ extend ::Gitlab::Utils::Override
+ ...
+
+ # @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
+ # @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
+ def replicables_for_current_secondary(primary_key_in)
+ # This issue template does not help you write this method.
+ #
+ # This method is called only on Geo secondary sites. It is called when
+ # we want to know which records to replicate. This is not easy to automate
+ # because for example:
+ #
+ # * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
+ # namespace, but the nature of the relationship to a namespace varies
+ # between Models.
+ # * The "selective sync" feature allows admins to choose which shards to
+ # replicate, per secondary site. Repositories are associated with
+ # shards. Most blob types are not, but Project Uploads are.
+ # * Remote stored replicables are not replicated, by default. But the
+ # setting `sync_object_storage` enables replication of remote stored
+ # replicables.
+ #
+ # Search the codebase for examples, and consult a Geo expert if needed.
+ end
+
+ override :verification_state_table_class
+ def verification_state_table_class
+ CoolWidgetState
+ end
end
# Geo checks this method in FrameworkRepositorySyncService to avoid
@@ -336,6 +270,11 @@ That's all of the required database changes.
def pool_repository
nil
end
+ ...
+
+ def cool_widget_state
+ super || build_cool_widget_state
+ end
...
end
@@ -343,6 +282,15 @@ That's all of the required database changes.
- [ ] Implement `CoolWidget.replicables_for_current_secondary` above.
- [ ] Ensure `CoolWidget.replicables_for_current_secondary` is well-tested. Search the codebase for `replicables_for_current_secondary` to find examples of parameterized table specs. You may need to add more `FactoryBot` traits.
+- [ ] Add the following shared examples to `ee/spec/models/ee/cool_widget_spec.rb`:
+
+ ```ruby
+ include_examples 'a replicable model with a separate table for verification state' do
+ let(:verifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is included in `available_verifiables`
+ let(:unverifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is NOT included in `available_verifiables`
+ end
+ ```
+
- [ ] Create `ee/app/replicators/geo/cool_widget_replicator.rb`. Implement the `#repository` method which should return a `<Repository>` instance, and implement the class method `.model` to return the `CoolWidget` class:
```ruby
@@ -365,6 +313,10 @@ That's all of the required database changes.
::Gitlab::GitAccessCoolWidget
end
+ def self.no_repo_message
+ git_access_class.error_message(:no_repo)
+ end
+
# The feature flag follows the format `geo_#{replicable_name}_replication`,
# so here it would be `geo_cool_widget_replication`
def self.replication_enabled_by_default?
@@ -403,6 +355,9 @@ That's all of the required database changes.
```
- [ ] Make sure a Geo secondary site can request and download Cool Widgets on the Geo primary site. You may need to make some changes to `Gitlab::GitAccessCoolWidget`. For example, see [this change for Group-level Wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/54914/diffs?commit_id=0f2b36f66697b4addbc69bd377ee2818f648dd33).
+
+- [ ] Make sure a Geo secondary site can replicate Cool Widgets where repository does not exist on the Geo primary site. The only way to know about this is to parse the error text. You may need to make some changes to `Gitlab::CoolWidgetReplicator.no_repo_message` to return the proper error message. For example, see [this change for Group-level Wikis](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74133).
+
- [ ] Generate the feature flag definition file by running the feature flag command and following the command prompts:
```shell
@@ -529,13 +484,7 @@ That's all of the required database changes.
- [ ] Make sure the factory also allows setting a `project` attribute. If the model does not have a direct relation to a project, you can use a `transient` attribute. Check out `spec/factories/merge_request_diffs.rb` for an example.
-##### If you added verification state fields to a separate table (option 2 above), then you need to make additional model and factory changes
-
-If you did not add verification state fields to a separate table, `cool_widget_states`, then skip to [Step 2. Implement metrics gathering](#step-2-implement-metrics-gathering).
-
-Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309).
-
-- [ ] Add a `Geo::CoolWidgetState` model in `ee/app/models/geo/cool_widget_state.rb`:
+- [ ] Following [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309) add a `Geo::CoolWidgetState` model in `ee/app/models/ee/geo/cool_widget_state.rb`:
``` ruby
# frozen_string_literal: true
@@ -569,63 +518,6 @@ Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.co
end
```
-- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
- - Include the `::Gitlab::Geo::VerificationState` concern.
- - Delegate verification related methods to the `cool_widget_state` model.
- - Override some scopes to use the `cool_widget_states` table instead of the model table, for verification.
- - Override some methods to use the `cool_widget_states` table in verification related queries.
-
- ```ruby
- class CoolWidget < ApplicationRecord
- ...
- include ::Gitlab::Geo::VerificationState
-
- has_one :cool_widget_state, autosave: true, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
-
- delegate :verification_retry_at, :verification_retry_at=,
- :verified_at, :verified_at=,
- :verification_checksum, :verification_checksum=,
- :verification_failure, :verification_failure=,
- :verification_retry_count, :verification_retry_count=,
- :verification_state=, :verification_state,
- :verification_started_at=, :verification_started_at,
- to: :cool_widget_state
- ...
-
- scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) }
- scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) }
- scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) }
-
- ...
-
- class_methods do
- extend ::Gitlab::Utils::Override
- ...
- override :verification_state_table_name
- def verification_state_table_name
- 'cool_widget_states'
- end
-
- override :verification_state_model_key
- def verification_state_model_key
- 'cool_widget_id'
- end
-
- override :verification_arel_table
- def verification_arel_table
- CoolWidgetState.arel_table
- end
- end
- ...
-
- def cool_widget_state
- super || build_cool_widget_state
- end
-
- ...
- end
- ```
-
#### Step 2. Implement metrics gathering
Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus` for display in the UI, and sent to Prometheus:
diff --git a/.gitlab/issue_templates/Geo Replicate a new blob type.md b/.gitlab/issue_templates/Geo Replicate a new blob type.md
index 00a71fa406e..7c927e79e93 100644
--- a/.gitlab/issue_templates/Geo Replicate a new blob type.md
+++ b/.gitlab/issue_templates/Geo Replicate a new blob type.md
@@ -37,6 +37,7 @@ It is also a good idea to first open a proof-of-concept merge request. It can be
You can look into the following examples of MRs for implementing replication/verification for a new blob type:
- [Add db changes](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60935) and [add verification for MR diffs using SSF](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309)
- [Verify Terraform state versions](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/58800)
+- [Verify LFS objects](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63981)
### Modify database schemas to prepare to add Geo support for Cool Widgets
@@ -114,113 +115,9 @@ Geo secondary sites have a [Geo tracking database](https://gitlab.com/gitlab-org
### Add verification state fields on the Geo primary site
-The Geo primary site needs to checksum every replicable in order for secondaries to verify their own checksums. To do this, Geo requires fields on the Model. There are two ways to add the necessary verification state fields. If the table is large and wide, then it may be a good idea to add verification state fields to a separate table (Option 2). Consult a database expert if needed.
+The Geo primary site needs to checksum every replicable so secondaries can verify their own checksums. To do this, Geo requires fields on the Model. Add verification state fields to a separate table. Consult a database expert if needed.
-#### Add verification state fields to the model table (Option 1)
-
-- [ ] Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationStateToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationStateToCoolWidgets < ActiveRecord::Migration[6.0]
- def change
- change_table(:cool_widgets) do |t|
- t.integer :verification_state, default: 0, limit: 2, null: false
- t.column :verification_started_at, :datetime_with_timezone
- t.integer :verification_retry_count, limit: 2, null: false
- t.column :verification_retry_at, :datetime_with_timezone
- t.column :verified_at, :datetime_with_timezone
- t.binary :verification_checksum, using: 'verification_checksum::bytea'
-
- t.text :verification_failure # rubocop:disable Migration/AddLimitToTextColumns
- end
- end
- end
- ```
-
-- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
-- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
-- [ ] Adding a `text` column also [requires](../database/strings_and_the_text_data_type.md#add-a-text-column-to-an-existing-table) setting a limit. Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationFailureLimitToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationFailureLimitToCoolWidgets < ActiveRecord::Migration[6.0]
- include Gitlab::Database::MigrationHelpers
-
- disable_ddl_transaction!
-
- CONSTRAINT_NAME = 'cool_widget_verification_failure_text_limit'
-
- def up
- add_text_limit :cool_widget, :verification_failure, 255, constraint_name: CONSTRAINT_NAME
- end
-
- def down
- remove_check_constraint(:cool_widget, CONSTRAINT_NAME)
- end
- end
- ```
-
-- [ ] Add indexes on verification fields to ensure verification can be performed efficiently. Some or all of these indexes can be omitted if the table is guaranteed to be small. Ask a database expert if you are considering omitting indexes. Create the migration file in `db/migrate`:
-
- ```shell
- bin/rails generate migration AddVerificationIndexesToCoolWidgets
- ```
-
-- [ ] Replace the contents of the migration file with:
-
- ```ruby
- # frozen_string_literal: true
-
- class AddVerificationIndexesToCoolWidgets < ActiveRecord::Migration[6.0]
- include Gitlab::Database::MigrationHelpers
-
- VERIFICATION_STATE_INDEX_NAME = "index_cool_widgets_on_verification_state"
- PENDING_VERIFICATION_INDEX_NAME = "index_cool_widgets_pending_verification"
- FAILED_VERIFICATION_INDEX_NAME = "index_cool_widgets_failed_verification"
- NEEDS_VERIFICATION_INDEX_NAME = "index_cool_widgets_needs_verification"
-
- disable_ddl_transaction!
-
- def up
- add_concurrent_index :cool_widgets, :verification_state, name: VERIFICATION_STATE_INDEX_NAME
- add_concurrent_index :cool_widgets, :verified_at, where: "(verification_state = 0)", order: { verified_at: 'ASC NULLS FIRST' }, name: PENDING_VERIFICATION_INDEX_NAME
- add_concurrent_index :cool_widgets, :verification_retry_at, where: "(verification_state = 3)", order: { verification_retry_at: 'ASC NULLS FIRST' }, name: FAILED_VERIFICATION_INDEX_NAME
- add_concurrent_index :cool_widgets, :verification_state, where: "(verification_state = 0 OR verification_state = 3)", name: NEEDS_VERIFICATION_INDEX_NAME
- end
-
- def down
- remove_concurrent_index_by_name :cool_widgets, VERIFICATION_STATE_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, PENDING_VERIFICATION_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, FAILED_VERIFICATION_INDEX_NAME
- remove_concurrent_index_by_name :cool_widgets, NEEDS_VERIFICATION_INDEX_NAME
- end
- end
- ```
-
-- [ ] Run database migrations:
-
- ```shell
- bin/rake db:migrate
- ```
-
-- [ ] Be sure to commit the relevant changes in `db/structure.sql`
-
-#### Add verification state fields to a separate table (Option 2)
+#### Add verification state fields to a new table
- [ ] Create the migration file in `db/migrate`:
@@ -274,12 +171,15 @@ The Geo primary site needs to checksum every replicable in order for secondaries
```
- [ ] If deviating from the above example, then be sure to order columns according to [our guidelines](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/ordering_table_columns.md).
+
- [ ] Run database migrations:
```shell
bin/rake db:migrate
```
+- [ ] If `cool_widgets` is a high-traffic table, follow [the database documentation to use `with_lock_retries`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/migration_style_guide.md#when-to-use-the-helper-method)
+
- [ ] Be sure to commit the relevant changes in `db/structure.sql`
That's all of the required database changes.
@@ -288,14 +188,22 @@ That's all of the required database changes.
#### Step 1. Implement replication and verification
-- [ ] Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
+- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
+ - Include `Gitlab::Geo::ReplicableModel` in the `CoolWidget` class, and specify the Replicator class `with_replicator Geo::CoolWidgetReplicator`.
+ - Include the `::Gitlab::Geo::VerificationState` concern.
+ - Delegate verification related methods to the `cool_widget_state` model.
+ - For verification, override some scopes to use the `cool_widget_states` table instead of the model table.
+ - Implement the `verification_state_object` method to return the object that holds
+ the verification details
+ - Override some methods to use the `cool_widget_states` table in verification-related queries.
At this point the `CoolWidget` class should look like this:
```ruby
# frozen_string_literal: true
-
+
class CoolWidget < ApplicationRecord
+ ...
include ::Gitlab::Geo::ReplicableModel
include ::Gitlab::Geo::VerificationState
@@ -303,38 +211,84 @@ That's all of the required database changes.
mount_uploader :file, CoolWidgetUploader
+ has_one :cool_widget_state, autosave: false, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
+
+ delegate :verification_retry_at, :verification_retry_at=,
+ :verified_at, :verified_at=,
+ :verification_checksum, :verification_checksum=,
+ :verification_failure, :verification_failure=,
+ :verification_retry_count, :verification_retry_count=,
+ :verification_state=, :verification_state,
+ :verification_started_at=, :verification_started_at,
+ to: :cool_widget_state
+ ...
+
+ scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) }
+ scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) }
+ scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) }
+
# Override the `all` default if not all records can be replicated. For an
# example of an existing Model that needs to do this, see
# `EE::MergeRequestDiff`.
# scope :available_replicables, -> { all }
- # @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
- # @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
- def self.replicables_for_current_secondary(primary_key_in)
- # This issue template does not help you write this method.
- #
- # This method is called only on Geo secondary sites. It is called when
- # we want to know which records to replicate. This is not easy to automate
- # because for example:
- #
- # * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
- # namespace, but the nature of the relationship to a namespace varies
- # between Models.
- # * The "selective sync" feature allows admins to choose which shards to
- # replicate, per secondary site. Repositories are associated with
- # shards. Most blob types are not, but Project Uploads are.
- # * Remote stored replicables are not replicated, by default. But the
- # setting `sync_object_storage` enables replication of remote stored
- # replicables.
- #
- # Search the codebase for examples, and consult a Geo expert if needed.
+ def verification_state_object
+ cool_widget_state
end
...
+
+ class_methods do
+ extend ::Gitlab::Utils::Override
+ ...
+
+ # @param primary_key_in [Range, CoolWidget] arg to pass to primary_key_in scope
+ # @return [ActiveRecord::Relation<CoolWidget>] everything that should be synced to this node, restricted by primary key
+ def self.replicables_for_current_secondary(primary_key_in)
+ # This issue template does not help you write this method.
+ #
+ # This method is called only on Geo secondary sites. It is called when
+ # we want to know which records to replicate. This is not easy to automate
+ # because for example:
+ #
+ # * The "selective sync" feature allows admins to choose which namespaces # to replicate, per secondary site. Most Models are scoped to a
+ # namespace, but the nature of the relationship to a namespace varies
+ # between Models.
+ # * The "selective sync" feature allows admins to choose which shards to
+ # replicate, per secondary site. Repositories are associated with
+ # shards. Most blob types are not, but Project Uploads are.
+ # * Remote stored replicables are not replicated, by default. But the
+ # setting `sync_object_storage` enables replication of remote stored
+ # replicables.
+ #
+ # Search the codebase for examples, and consult a Geo expert if needed.
+ end
+
+ override :verification_state_table_class
+ def verification_state_table_class
+ CoolWidgetState
+ end
+ end
+ ...
+
+ def cool_widget_state
+ super || build_cool_widget_state
+ end
+
+ ...
end
```
- [ ] Implement `CoolWidget.replicables_for_current_secondary` above.
- [ ] Ensure `CoolWidget.replicables_for_current_secondary` is well-tested. Search the codebase for `replicables_for_current_secondary` to find examples of parameterized table specs. You may need to add more `FactoryBot` traits.
+- [ ] Add the following shared examples to `ee/spec/models/ee/cool_widget_spec.rb`:
+
+ ```ruby
+ include_examples 'a replicable model with a separate table for verification state' do
+ let(:verifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is included in `available_verifiables`
+ let(:unverifiable_model_record) { build(:cool_widget) } # add extra params if needed to make sure the record is NOT included in `available_verifiables`
+ end
+ ```
+
- [ ] Create `ee/app/replicators/geo/cool_widget_replicator.rb`. Implement the `#carrierwave_uploader` method which should return a `CarrierWave::Uploader`, and implement the class method `.model` to return the `CoolWidget` class:
```ruby
@@ -498,13 +452,7 @@ That's all of the required database changes.
- [ ] Make sure the factory also allows setting a `project` attribute. If the model does not have a direct relation to a project, you can use a `transient` attribute. Check out `spec/factories/merge_request_diffs.rb` for an example.
-##### If you added verification state fields to a separate table (option 2 above), then you need to make additional model and factory changes
-
-If you did not add verification state fields to a separate table, `cool_widget_states`, then skip to [Step 2. Implement metrics gathering](#step-2-implement-metrics-gathering).
-
-Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309).
-
-- [ ] Add a `Geo::CoolWidgetState` model in `ee/app/models/ee/geo/cool_widget_state.rb`:
+- [ ] Following [the example of Merge Request Diffs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63309) add a `Geo::CoolWidgetState` model in `ee/app/models/ee/geo/cool_widget_state.rb`:
``` ruby
module Geo
@@ -536,63 +484,6 @@ Otherwise, you can follow [the example of Merge Request Diffs](https://gitlab.co
end
```
-- [ ] Add the following lines to the `cool_widget` model to accomplish some important tasks:
- - Include the `::Gitlab::Geo::VerificationState` concern.
- - Delegate verification related methods to the `cool_widget_state` model.
- - Override some scopes to use the `cool_widget_states` table instead of the model table, for verification.
- - Override some methods to use the `cool_widget_states` table in verification related queries.
-
- ```ruby
- class CoolWidget < ApplicationRecord
- ...
- include ::Gitlab::Geo::VerificationState
-
- has_one :cool_widget_state, autosave: true, inverse_of: :cool_widget, class_name: 'Geo::CoolWidgetState'
-
- delegate :verification_retry_at, :verification_retry_at=,
- :verified_at, :verified_at=,
- :verification_checksum, :verification_checksum=,
- :verification_failure, :verification_failure=,
- :verification_retry_count, :verification_retry_count=,
- :verification_state=, :verification_state,
- :verification_started_at=, :verification_started_at,
- to: :cool_widget_state
- ...
-
- scope :with_verification_state, ->(state) { joins(:cool_widget_state).where(cool_widget_states: { verification_state: verification_state_value(state) }) }
- scope :checksummed, -> { joins(:cool_widget_state).where.not(cool_widget_states: { verification_checksum: nil } ) }
- scope :not_checksummed, -> { joins(:cool_widget_state).where(cool_widget_states: { verification_checksum: nil } ) }
-
- ...
-
- class_methods do
- extend ::Gitlab::Utils::Override
- ...
- override :verification_state_table_name
- def verification_state_table_name
- 'cool_widget_states'
- end
-
- override :verification_state_model_key
- def verification_state_model_key
- 'cool_widget_id'
- end
-
- override :verification_arel_table
- def verification_arel_table
- CoolWidgetState.arel_table
- end
- end
- ...
-
- def cool_widget_state
- super || build_cool_widget_state
- end
-
- ...
- end
- ```
-
#### Step 2. Implement metrics gathering
Metrics are gathered by `Geo::MetricsUpdateWorker`, persisted in `GeoNodeStatus` for display in the UI, and sent to Prometheus:
diff --git a/.gitlab/issue_templates/InfraDev.md b/.gitlab/issue_templates/InfraDev.md
index bc0e65c3c22..d337e75289c 100644
--- a/.gitlab/issue_templates/InfraDev.md
+++ b/.gitlab/issue_templates/InfraDev.md
@@ -53,4 +53,4 @@ See also:
-->
/label ~"infradev"
-/label ~"bug"
+/label ~"type::bug"
diff --git a/.gitlab/issue_templates/Problem Validation.md b/.gitlab/issue_templates/Problem Validation.md
index 5d417c5a26d..3f92510b6af 100644
--- a/.gitlab/issue_templates/Problem Validation.md
+++ b/.gitlab/issue_templates/Problem Validation.md
@@ -1,3 +1,7 @@
+<!-- This template is used as a starting point for understing and articulating a customer problem.
+Learn more about it in the handbook: https://about.gitlab.com/handbook/product-development-flow/#validation-phase-2-problem-validation
+-->
+
## Problem Statement
<!-- What is the problem we hope to validate? Reference how to write a real customer problem statement at https://productcoalition.com/how-to-write-a-good-customer-problem-statement-a815f80189ba for guidance. -->
@@ -45,4 +49,8 @@ For example, if the solution will take a product manager, designer, and engineer
- [ ] The problem is well described and detailed with necessary requirements for product design to understand the problem
- [ ] The problem is well described and detailed with necessary requirements for engineering to understand the problem
+## Research Issue
+
+<!-- Link to the Problem Validation Research issue that will be executed by the UX Researcher. https://gitlab.com/gitlab-org/ux-research/ -->
+
/label ~"workflow::validation backlog" ~devops:: ~category: ~group::
diff --git a/.gitlab/issue_templates/Productivity Improvement.md b/.gitlab/issue_templates/Productivity Improvement.md
index 06692d3ede8..040d8ea2f89 100644
--- a/.gitlab/issue_templates/Productivity Improvement.md
+++ b/.gitlab/issue_templates/Productivity Improvement.md
@@ -2,7 +2,7 @@
<!--
Please describe the engineering productivity problem that needs to be solved backed by charts from
-https://about.gitlab.com/handbook/engineering/quality/engineering-productivity-team/#engineering-productivity-team-metrics.
+https://about.gitlab.com/handbook/engineering/quality/engineering-productivity/#engineering-productivity-metrics.
-->
### Problem identification checklist
diff --git a/.gitlab/issue_templates/Refactoring.md b/.gitlab/issue_templates/Refactoring.md
index d9466185ff7..df18dcf7656 100644
--- a/.gitlab/issue_templates/Refactoring.md
+++ b/.gitlab/issue_templates/Refactoring.md
@@ -41,9 +41,9 @@ please list them here.
<!--
Please select the appropriate label from the following:
~"feature::addition"
- ~"feature::maintenance"
+ ~"type::maintenance"
~"tooling::pipelines"
~"tooling::workflow"
-->
-/label ~"feature::maintenance"
+/label ~"type::maintenance"
diff --git a/.gitlab/issue_templates/Security developer workflow.md b/.gitlab/issue_templates/Security developer workflow.md
index 7f2c54f4f49..6bf9e6971d7 100644
--- a/.gitlab/issue_templates/Security developer workflow.md
+++ b/.gitlab/issue_templates/Security developer workflow.md
@@ -12,7 +12,7 @@ Set the title to: `Description of the original issue`
- [ ] Make sure the issue really needs to follow the security release workflow.
- Verify if the issue you're working on `gitlab-org/gitlab` is confidential, if it's public fix should be placed on GitLab canonical and no backports are required.
- If the issue you're fixing doesn't appear to be something that can be exploited by a malicious person and is instead simply a security enhancement do not hesitate to ping `@gitlab-com/gl-security/appsec` to discuss if the issue can be fixed in the canonical repository.
-- [ ] **IMPORTANT**: Mark this [issue as linked] to the Security Release Tracking Issue. You can find it on the topic of the `#releases` Slack channel. This issue
+- [ ] **IMPORTANT**: Mark this [issue as linked] to the Security Release Tracking Issue. You can find it [here](https://gitlab.com/gitlab-org/gitlab/-/issues?sort=created_date&state=opened&label_name[]=upcoming+security+release). This issue
MUST be linked for the release bot to know that the associated merge requests should be merged for this security release.
- Fill out the [Links section](#links):
- [ ] Next to **Issue on GitLab**, add a link to the `gitlab-org/gitlab` issue that describes the security vulnerability.
diff --git a/.gitlab/issue_templates/Technical Evaluation.md b/.gitlab/issue_templates/Technical Evaluation.md
index cf939725a78..680ecb7d9a6 100644
--- a/.gitlab/issue_templates/Technical Evaluation.md
+++ b/.gitlab/issue_templates/Technical Evaluation.md
@@ -29,5 +29,5 @@
### Team
-- [ ] Add ~"workflow::planning breakdown" ~feature and the corresponding `~devops::<stage>` and `~group::<group>` labels.
+- [ ] Add ~"workflow::planning breakdown" ~"type::feature" and the corresponding `~devops::<stage>` and `~group::<group>` labels.
- [ ] Ping the PM and EM.