diff options
Diffstat (limited to 'doc/architecture/blueprints/pods')
5 files changed, 124 insertions, 5 deletions
diff --git a/doc/architecture/blueprints/pods/index.md b/doc/architecture/blueprints/pods/index.md index 0a36de5790f..077303be30f 100644 --- a/doc/architecture/blueprints/pods/index.md +++ b/doc/architecture/blueprints/pods/index.md @@ -1,8 +1,8 @@ --- status: accepted creation-date: "2022-09-07" -authors: [ "@fzimmer", "@DylanGriffith" ] -coach: "@kamil" +authors: [ "@ayufan", "@fzimmer", "@DylanGriffith" ] +coach: "@ayufan" approvers: [ "@fzimmer" ] owning-stage: "~devops::enablement" participating-stages: [] @@ -150,6 +150,14 @@ At this moment, GitLab.com has "social-network"-like capabilities that may not f We should evaluate if the SMB and mid market segment is interested in these features, or if not having them is acceptable in most cases. +### Self-managed + +For reasons of consistency, it is expected that self-managed instances will +adopt the pods architecture as well. To expand, self-managed instances can +continue with just a single Pod while supporting the option of adding additional +Pods. Organizations, and possible User decomposition will also be adopted for +self-managed instances. + ## High-level architecture problems to solve A number of technical issues need to be resolved to implement Pods (in no particular order). This section will be expanded. @@ -325,9 +333,11 @@ This is the list of known affected features with the proposed solutions. - [Pods: Organizations](pods-feature-organizations.md) - [Pods: Router Endpoints Classification](pods-feature-router-endpoints-classification.md) - [Pods: Schema changes (Postgres and Elasticsearch migrations)](pods-feature-schema-changes.md) +- [Pods: Backups](pods-feature-backups.md) - [Pods: Global Search](pods-feature-global-search.md) - [Pods: CI Runners](pods-feature-ci-runners.md) - [Pods: Admin Area](pods-feature-admin-area.md) +- [Pods: Secrets](pods-feature-secrets.md) - [Pods: Container Registry](pods-feature-container-registry.md) - [Pods: Contributions: Forks](pods-feature-contributions-forks.md) - [Pods: Personal Namespaces](pods-feature-personal-namespaces.md) diff --git a/doc/architecture/blueprints/pods/pods-feature-backups.md b/doc/architecture/blueprints/pods/pods-feature-backups.md new file mode 100644 index 00000000000..5e4de42f473 --- /dev/null +++ b/doc/architecture/blueprints/pods/pods-feature-backups.md @@ -0,0 +1,61 @@ +--- +stage: enablement +group: pods +comments: false +description: 'Pods: Backups' +--- + +This document is a work-in-progress and represents a very early state of the +Pods design. Significant aspects are not documented, though we expect to add +them in the future. This is one possible architecture for Pods, and we intend to +contrast this with alternatives before deciding which approach to implement. +This documentation will be kept even if we decide not to implement this so that +we can document the reasons for not choosing this approach. + +# Pods: Backups + +Each pods will take its own backups, and consequently have its own isolated +backup / restore procedure. + +## 1. Definition + +GitLab Backup takes a backup of the PostgreSQL database used by the application, +and also Git repository data. + +## 2. Data flow + +Each pod has a number of application databases to back up (e.g. `main`, and `ci`). + +Additionally, there may be cluster-wide metadata tables (e.g. `users` table) +which is directly accesible via PostgreSQL. + +## 3. Proposal + +### 3.1. Cluster-wide metadata + +It is currently unknown how cluster-wide metadata tables will be accessible. We +may choose to have cluster-wide metadata tables backed up separately, or have +each pod back up its copy of cluster-wide metdata tables. + +### 3.2 Consistency + +#### 3.2.1 Take backups independently + +As each pod will communicate with each other via API, and there will be no joins +to the users table, it should be acceptable for each pod to take a backup +independently of each other. + +#### 3.2.2 Enforce snapshots + +We can require that each pod take a snapshot for the PostgreSQL databases at +around the same time to allow for a consistent-enough backup. + +## 4. Evaluation + +As the number of pods increases, it will likely not be feasible to take a +snapshot at the same time for all pods. Hence taking backups independently is +the better option. + +## 4.1. Pros + +## 4.2. Cons diff --git a/doc/architecture/blueprints/pods/pods-feature-secrets.md b/doc/architecture/blueprints/pods/pods-feature-secrets.md new file mode 100644 index 00000000000..f18a41dc0fb --- /dev/null +++ b/doc/architecture/blueprints/pods/pods-feature-secrets.md @@ -0,0 +1,48 @@ +--- +stage: enablement +group: pods +comments: false +description: 'Pods: Secrets' +--- + +This document is a work-in-progress and represents a very early state of the +Pods design. Significant aspects are not documented, though we expect to add +them in the future. This is one possible architecture for Pods, and we intend to +contrast this with alternatives before deciding which approach to implement. +This documentation will be kept even if we decide not to implement this so that +we can document the reasons for not choosing this approach. + +# Pods: Secrets + +Where possible, each pod should have its own distinct set of secrets. +However, there will be some secrets that will be required to be the same for all +pods in the cluster + +## 1. Definition + +GitLab has a lot of +[secrets](https://docs.gitlab.com/charts/installation/secrets.html) that needs +to be configured. + +Some secrets are for inter-component communication, e.g. `GitLab Shell secret`, +and used only within a pod. + +Some secrets are used for features, e.g. `ci_jwt_signing_key`. + +## 2. Data flow + +## 3. Proposal + +1. Secrets used for features will need to be consistent across all pods, so that the UX is consistent. + 1. This is especially true for the `db_key_base` secret which is used for + encrypting data at rest in the database - so that projects that are + transferred to another pod will continue to work. We do not want to have + to re-encrypt such rows when we move projects/groups between pods. +1. Secrets which are used for intra-pod communication only should be uniquely generated + per-pod. + +## 4. Evaluation + +## 4.1. Pros + +## 4.2. Cons diff --git a/doc/architecture/blueprints/pods/proposal-stateless-router-with-buffering-requests.md b/doc/architecture/blueprints/pods/proposal-stateless-router-with-buffering-requests.md index ab19b652f93..adc523e90c2 100644 --- a/doc/architecture/blueprints/pods/proposal-stateless-router-with-buffering-requests.md +++ b/doc/architecture/blueprints/pods/proposal-stateless-router-with-buffering-requests.md @@ -555,9 +555,9 @@ sequenceDiagram They get a 404. -### Experience for non-logged in users +### Experience for non-authenticated users -Flow is similar to logged in users except global routes like `/dashboard` will +Flow is similar to authenticated users except global routes like `/dashboard` will redirect to the login page as there is no default organization to choose from. ### A new customers signs up diff --git a/doc/architecture/blueprints/pods/proposal-stateless-router-with-routes-learning.md b/doc/architecture/blueprints/pods/proposal-stateless-router-with-routes-learning.md index c99b02a35e9..1156e65f6aa 100644 --- a/doc/architecture/blueprints/pods/proposal-stateless-router-with-routes-learning.md +++ b/doc/architecture/blueprints/pods/proposal-stateless-router-with-routes-learning.md @@ -578,7 +578,7 @@ sequenceDiagram They get a 404. -### Experience for non-logged in users +### Experience for non-authenticated users Flow is similar to logged in users except global routes like `/dashboard` will redirect to the login page as there is no default organization to choose from. |