Add latest changes from gitlab-org/gitlab@master

11 files changed, 439 insertions, 11 deletions

diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md
index 99d2bf76a49..4cf296ac1f3 100644
--- a/doc/api/graphql/index.md
+++ b/doc/api/graphql/index.md
@@ -69,12 +69,13 @@ However, GitLab sometimes changes the GraphQL API in a way that is not backward-
 can include removing or renaming fields, arguments, or other parts of the schema.
 When creating a breaking change, GitLab follows a [deprecation and removal process](#deprecation-and-removal-process).
 
-Learn more about [breaking changes](../../development/deprecation_guidelines/index.md).
+To avoid having a breaking change affect your integrations, you should
+familiarize yourself with the [deprecation and removal process](#deprecation-and-removal-process) and
+frequently [verify your API calls against the future breaking-change schema](#verify-against-the-future-breaking-change-schema).
 
 Fields behind a feature flag and disabled by default do not follow the deprecation and removal process, and can be removed at any time without notice.
 
-To avoid having a breaking change affect your integrations, you should
-familiarize yourself with the deprecation and removal process.
+Learn more about [breaking changes](../../development/deprecation_guidelines/index.md).
 
 WARNING:
 GitLab makes all attempts to follow the [deprecation and removal process](#deprecation-and-removal-process).
@@ -82,6 +83,18 @@ On rare occasions, GitLab might make immediate breaking changes to the GraphQL
 API to patch critical security or performance concerns if the deprecation
 process would pose significant risk.
 
+### Verify against the future breaking-change schema
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353642) in GitLab 15.6.
+
+You can make calls against the GraphQL API as if all deprecated items were already removed.
+This way, you can verify API calls ahead of a [breaking-change release](#deprecation-and-removal-process)
+before the items are actually removed from the schema.
+
+To make these calls, add a
+`remove_deprecated=true` query parameter to the GitLab GraphQL API endpoint (for example,
+`https://gitlab.com/api/graphql?remove_deprecated=true` for GitLab SaaS GraphQL).
+
 ### Deprecation and removal process
 
 The deprecation and removal process for the GitLab GraphQL API aligns with the wider GitLab
@@ -99,14 +112,13 @@ Items are marked as deprecated in:
 - The [deprecation feature removal schedule](../../update/deprecations.md), which is linked from release posts.
 - Introspection queries of the GraphQL API.
 
+The deprecation message provides an alternative for the deprecated schema item,
+if applicable.
+
 NOTE:
 If you use the GraphQL API, we recommend you remove the deprecated schema from your GraphQL
 API calls as soon as possible to avoid experiencing breaking changes.
-To verify your API calls against the schema without the deprecated schema items, you can add a
-`?remove_deprecated=true` query parameter. You should only use this parameter for verification purposes.
-
-The deprecation message provides an alternative for the deprecated schema item,
-if applicable.
+You should [verify your API calls against the schema without the deprecated schema items](#verify-against-the-future-breaking-change-schema).
 
 #### Deprecation example
 
diff --git a/doc/architecture/blueprints/pods/index.md b/doc/architecture/blueprints/pods/index.md
index 527720d003a..3ba319d169b 100644
--- a/doc/architecture/blueprints/pods/index.md
+++ b/doc/architecture/blueprints/pods/index.md
@@ -319,6 +319,11 @@ The Pods architecture will impact many features requiring some of them to be rew
 This is the list of known affected features with the proposed solutions.
 
 - [Pods: Git Access](pods-feature-git-access.md)
+- [Pods: Data Migration](pods-feature-data-migration.md)
+- [Pods: Database Sequences](pods-feature-database-sequences.md)
+- [Pods: GraphQL](pods-feature-graphql.md)
+- [Pods: Organizations](pods-feature-organizations.md)
+- [Pods: Router Endpoints Classification](pods-feature-router-endpoints-classification.md)
 
 ## Links
 
diff --git a/doc/architecture/blueprints/pods/pods-feature-data-migration.md b/doc/architecture/blueprints/pods/pods-feature-data-migration.md
new file mode 100644
index 00000000000..fad6bca45fa
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-data-migration.md
@@ -0,0 +1,82 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Data migration'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: Data migration
+
+It is essential for Pods architecture to provide a way to migrate data out of big Pods
+into smaller ones. This describes various approaches to provide this type of split.
+
+## 1. Definition
+
+## 2. Data flow
+
+## 3. Proposal
+
+### 3.1. Split large Pods
+
+A single Pod can only be divided into many Pods. This is based on principle
+that it is easier to create exact clone of an existing Pod in many replicas
+out of which some will be made authoritative once migrated. Keeping those
+replicas up-to date with Pod 0 is also much easier due to pre-existing
+replication solutions that can replicate the whole systems: Geo, PostgreSQL
+physical replication, etc.
+
+1. All data of an organization needs to not be divided across many Pods.
+1. Split should be doable online.
+1. New Pods cannot contain pre-existing data.
+1. N Pods contain exact replica of Pod 0.
+1. The data of Pod 0 is live replicated to as many Pods it needs to be split.
+1. Once consensus is achieved between Pod 0 and N-Pods the organizations to be migrated away
+   are marked as read-only cluster-wide.
+1. The `routes` is updated on for all organizations to be split to indicate an authorative
+   Pod holding the most recent data, like `gitlab-org` on `pod-100`.
+1. The data for `gitlab-org` on Pod 0, and on other non-authoritative N-Pods are dormant
+   and will be removed in the future.
+1. All accesses to `gitlab-org` on a given Pod are validated about `pod_id` of `routes`
+   to ensure that given Pod is authoritative to handle the data.
+
+### 3.2. Migrate organization from an existing Pod
+
+This is different to split, as we intend to perform logical and selective replication
+of data belonging to a single organization.
+
+Today this type of selective replication is only implemented by Gitaly where we can migrate
+Git repository from a single Gitaly node to another with minimal downtime.
+
+In this model we would require identifying all resources belonging to a given organization:
+database rows, object storage files, Git repositories, etc. and selectively copy them over
+to another (likely) existing Pod importing data into it. Ideally ensuring that we can
+perform logical replication live of all changed data, but change similarly to split
+which Pod is authoritative for this organization.
+
+1. It is hard to identify all resources belonging to organization.
+1. It requires either downtime for organization or a robust system to identify
+   live changes made.
+1. It likely will require a full database structure analysis (more robust than project import/export)
+   to perform selective PostgreSQL logical replication.
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-database-sequences.md b/doc/architecture/blueprints/pods/pods-feature-database-sequences.md
new file mode 100644
index 00000000000..0a8bb4d250e
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-database-sequences.md
@@ -0,0 +1,94 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Database Sequences'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: Database Sequences
+
+GitLab today ensures that every database row create has unique ID, allowing
+to access Merge Request, CI Job or Project by a known global ID.
+
+Pods will use many distinct and not connected databases, each of them having
+a separate IDs for most of entities.
+
+It might be desirable to retain globally unique IDs for all database rows
+to allow migrating resources between Pods in the future.
+
+## 1. Definition
+
+## 2. Data flow
+
+## 3. Proposal
+
+This are some preliminary ideas how we can retain unique IDs across the system.
+
+### 3.1. UUID
+
+Instead of using incremental sequences use UUID (128 bit) that is stored in database.
+
+- This might break existing IDs and requires adding UUID column for all existing tables.
+- This makes all indexes larger as it requires storing 128 bit instead of 32/64 bit in index.
+
+### 3.2. Use Pod index encoded in ID
+
+Since significant number of tables already use 64 bit ID numbers we could use MSB to encode
+Pod ID effectively enabling
+
+- This might limit amount of Pods that can be enabled in system, as we might decide to only
+  allocate 1024 possible Pod numbers.
+- This might make IDs to be migratable between Pods, since even if entity from Pod 1 is migrated to Pod 100
+  this ID would still be unique.
+- If resources are migrated the ID itself will not be enough to decode Pod number and we would need
+  lookup table.
+- This requires updating all IDs to 32 bits.
+
+### 3.3. Allocate sequence ranges from central place
+
+Each Pod might receive its own range of the sequences as they are consumed from a centrally managed place.
+Once Pod consumes all IDs assigned for a given table it would be replenished and a next range would be allocated.
+Ranges would be tracked to provide a faster lookup table if a random access pattern is required.
+
+- This might make IDs to be migratable between Pods, since even if entity from Pod 1 is migrated to Pod 100
+  this ID would still be unique.
+- If resources are migrated the ID itself will not be enough to decode Pod number and we would need
+  much more robust lookup table as we could be breaking previously assigned sequence ranges.
+- This does not require updating all IDs to 64 bits.
+- This adds some performance penalty to all `INSERT` statements in Postgres or at least from Rails as we need to check for the sequence number and potentially wait for our range to be refreshed from the ID server
+- The available range will need to be stored and incremented in a centralized place so that concurrent transactions cannot possibly get the same value.
+
+### 3.4. Define only some tables to require unique IDs
+
+Maybe this is acceptable only for some tables to have a globally unique IDs. It could be projects, groups
+and other top-level entities. All other tables like `merge_requests` would only offer Pod-local ID,
+but when referenced outside it would rather use IID (an ID that is monotonic in context of a given resource, like project).
+
+- This makes the ID 10000 for `merge_requests` be present on all Pods, which might be sometimes confusing
+  as for uniqueness of the resource.
+- This might make random access by ID (if ever needed) be impossible without using composite key, like: `project_id+merge_request_id`.
+- This would require us to implement a transformation/generation of new ID if we need to migrate records to another pod. This can lead to very difficult migration processes when these IDs are also used as foreign keys for other records being migrated.
+- If IDs need to change when moving between pods this means that any links to records by ID would no longer work even if those links included the `project_id`.
+- If we plan to allow these ids to not be unique and change the unique constraint to be based on a composite key then we'd need to update all foreign key references to be based on the composite key
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-graphql.md b/doc/architecture/blueprints/pods/pods-feature-graphql.md
new file mode 100644
index 00000000000..5f8a39c0b3f
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-graphql.md
@@ -0,0 +1,94 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: GraphQL'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: GraphQL
+
+GitLab exensively uses GraphQL to perform efficient data query operations.
+GraphQL due to it's nature is not directly routable. The way how GitLab uses
+it calls the `/api/graphql` endpoint, and only query or mutation of body request
+might define where the data can be accessed.
+
+## 1. Definition
+
+## 2. Data flow
+
+## 3. Proposal
+
+There are at least two main ways to implement GraphQL in Pods architecture.
+
+### 3.1. GraphQL routable by endpoint
+
+Change `/api/graphql` to `/api/organization/<organization>/graphql`.
+
+- This breaks all existing usages of `/api/graphql` endpoint
+  since the API URI is changed.
+
+### 3.2. GraphQL routable by body
+
+As part of router parse GraphQL body to find a routable entity, like `project`.
+
+- This still makes the GraphQL query be executed only in context of a given Pod
+  and not allowing the data to be merged.
+
+```json
+# Good example
+{
+  project(fullPath:"gitlab-org/gitlab") {
+    id
+    description
+  }
+}
+
+# Bad example, since Merge Request is not routable
+{
+  mergeRequest(id: 1111) {
+    iid
+    description
+  }
+}
+```
+
+### 3.3. Merging GraphQL Proxy
+
+Implement as part of router GraphQL Proxy which can parse body
+and merge results from many Pods.
+
+- This might make pagination hard to achieve, or we might assume that
+  we execute many queries of which results are merged across all Pods.
+
+```json
+{
+  project(fullPath:"gitlab-org/gitlab"){
+    id, description
+  }
+  group(fullPath:"gitlab-com") {
+    id, description
+  }
+}
+```
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-organizations.md b/doc/architecture/blueprints/pods/pods-feature-organizations.md
new file mode 100644
index 00000000000..a0a87458767
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-organizations.md
@@ -0,0 +1,58 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Organizations'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: Organizations
+
+One of the major designs of Pods architecture is strong isolation between Groups.
+Organizations as described by this blueprint provides a way to have plausible UX
+for joining together many Groups that are isolated from the rest of systems.
+
+## 1. Definition
+
+Pods do require that all groups and projects of a single organization can
+only be stored on a single Pod since a Pod can only access data that it holds locally
+and has very limited capabilities to read information from other Pods.
+
+Pods with Organizations do require strong isolation between organizations.
+
+It will have significant implications on various user-facing features,
+like Todos, dropdowns allowing to select projects, references to other issues
+or projects, or any other social functions present at GitLab. Today those functions
+were able to reference anything in the whole system. With the introduction of
+organizations such will be forbidden.
+
+This problem definition aims to answer effort and implications required to add
+strong isolation between organizations to the system. Including features affected
+and their data processing flow. The purpose is to ensure that our solution when
+implemented consistently avoids data leakage between organizations residing on
+a single Pod.
+
+## 2. Data flow
+
+## 3. Proposal
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-router-endpoints-classification.md b/doc/architecture/blueprints/pods/pods-feature-router-endpoints-classification.md
new file mode 100644
index 00000000000..c672342fff9
--- /dev/null
+++ b/doc/architecture/blueprints/pods/pods-feature-router-endpoints-classification.md
@@ -0,0 +1,46 @@
+---
+stage: enablement
+group: pods
+comments: false
+description: 'Pods: Router Endpoints Classification'
+---
+
+DISCLAIMER:
+This page may contain information related to upcoming products, features and
+functionality. It is important to note that the information presented is for
+informational purposes only, so please do not rely on the information for
+purchasing or planning purposes. Just like with all projects, the items
+mentioned on the page are subject to change or delay, and the development,
+release, and timing of any products, features, or functionality remain at the
+sole discretion of GitLab Inc.
+
+This document is a work-in-progress and represents a very early state of the
+Pods design. Significant aspects are not documented, though we expect to add
+them in the future. This is one possible architecture for Pods, and we intend to
+contrast this with alternatives before deciding which approach to implement.
+This documentation will be kept even if we decide not to implement this so that
+we can document the reasons for not choosing this approach.
+
+# Pods: Router Endpoints Classification
+
+Classification of all endpoints is essential to properly route request
+hitting load balancer of a GitLab installation to a Pod that can serve it.
+
+Each Pod should be able to decode each request and classify for which Pod
+it belongs to.
+
+GitLab currently implements houndreds of endpoints. This document tries
+to describe various techniques that can be implemented to allow the Rails
+to provide this information efficiently.
+
+## 1. Definition
+
+## 2. Data flow
+
+## 3. Proposal
+
+## 4. Evaluation
+
+## 4.1. Pros
+
+## 4.2. Cons
diff --git a/doc/architecture/blueprints/pods/pods-feature-template.md b/doc/architecture/blueprints/pods/pods-feature-template.md
index fd9813710f2..dfae21b5406 100644
--- a/doc/architecture/blueprints/pods/pods-feature-template.md
+++ b/doc/architecture/blueprints/pods/pods-feature-template.md
@@ -2,7 +2,7 @@
 stage: enablement
 group: pods
 comments: false
-description: 'Pods architecture: Problem A'
+description: 'Pods: Problem A'
 ---
 
 This document is a work-in-progress and represents a very early state of the
@@ -12,7 +12,7 @@ contrast this with alternatives before deciding which approach to implement.
 This documentation will be kept even if we decide not to implement this so that
 we can document the reasons for not choosing this approach.
 
-# Pods: Problem A
+# Pods: A
 
 > TL;DR
 
diff --git a/doc/development/work_items.md b/doc/development/work_items.md
index b71f6e86033..a417e1d1349 100644
--- a/doc/development/work_items.md
+++ b/doc/development/work_items.md
@@ -55,6 +55,7 @@ To avoid confusion and ensure communication is efficient, we will use the follow
 | legacy issue view | The existing view used to render issues and incidents | | |
 | issue             | The existing issue model | | |
 | issuable          | Any model currently using the issueable module (issues, epics and MRs) | _Incidents are an **issuable**_ | _Incidents are a **work item type**_ |
+| widget            | A UI element to present or allow interaction with specific work item data | | |
 
 Some terms have been used in the past but have since become confusing and are now discouraged.
 
@@ -138,6 +139,20 @@ To introduce a new WIT there are two options:
 
 ### Work item type widgets
 
+A widget is a single component that can exist on a work item. This component can be used on one or
+many work item types and can be lightly customized at the point of implementation.
+
+A widget contains both the frontend UI (if present) and the associated logic for presenting and
+managing any data used by the widget. There can be a one-to-many connection between the data model
+and widgets. It means there can be multiple widgets that use or manage the same data, and they could
+be present at the same time (for example, a read-only summary widget and an editable detail widget,
+or two widgets showing two different filtered views of the same model).
+
+Widgets should be differentiated by their **purpose**. When possible, this purpose should be
+abstracted to the highest reasonable level to maximize reusability. For example, the widget for
+managing "tasks" was built as "child items". Rather than managing one type of child, it's abstracted
+up to managing any children.
+
 All WITs will share the same pool of predefined widgets and will be customized by
 which widgets are active on a specific WIT. Every attribute (column or association)
 will become a widget with self-encapsulated functionality regardless of the WIT it belongs to.
diff --git a/doc/update/deprecations.md b/doc/update/deprecations.md
index e27f21d98fa..353fbf191b5 100644
--- a/doc/update/deprecations.md
+++ b/doc/update/deprecations.md
@@ -112,7 +112,12 @@ WARNING:
 This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/).
 Review the details carefully before upgrading.
 
-The command to [register](https://docs.gitlab.com/runner/register/) a runner, `gitlab-runner register` is deprecated. GitLab plans to introduce a new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/) in GitLab 15.8, which introduces a new method for registering runners and eliminates the legacy [runner registration token](https://docs.gitlab.com/ee/security/token_overview.html#runner-registration-tokens).
+The command to [register](https://docs.gitlab.com/runner/register/) a runner, `gitlab-runner register` is deprecated.
+GitLab plans to introduce a new [GitLab Runner token architecture](https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/) in GitLab 15.8,
+which introduces a new method for registering runners and eliminates the legacy
+[runner registration token](https://docs.gitlab.com/ee/security/token_overview.html#runner-registration-tokens).
+The new method will involve passing a [runner authentication token](https://docs.gitlab.com/ee/security/token_overview.html#runner-authentication-tokens-also-called-runner-tokens)
+to a new `gitlab-runner deploy` command.
 
 </div>
 
diff --git a/doc/user/tasks.md b/doc/user/tasks.md
index 5b8e824cb4c..5c9290e57cb 100644
--- a/doc/user/tasks.md
+++ b/doc/user/tasks.md
@@ -50,9 +50,26 @@ Prerequisites:
 To create a task:
 
 1. In the issue description, in the **Tasks** section, select **Add**.
+1. Select **New task**.
 1. Enter the task title.
 1. Select **Create task**.
 
+## Add existing tasks to an issue
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381868) in GitLab 15.6.
+
+Prerequisites:
+
+- You must have at least the Guest role for the project, or the project must be public.
+
+To add a task:
+
+1. In the issue description, in the **Tasks** section, select **Add**.
+1. Select **Existing task**.
+1. Search tasks by title.
+1. Select one or multiple tasks to add to the issue.
+1. Select **Add task**.
+
 ## Edit a task
 
 Prerequisites: