diff options
Diffstat (limited to 'doc/architecture/blueprints/ci_data_decay/index.md')
| -rw-r--r-- | doc/architecture/blueprints/ci_data_decay/index.md | 60 |
1 files changed, 10 insertions, 50 deletions
diff --git a/doc/architecture/blueprints/ci_data_decay/index.md b/doc/architecture/blueprints/ci_data_decay/index.md index 221c2364f79..b7c3bdde2f8 100644 --- a/doc/architecture/blueprints/ci_data_decay/index.md +++ b/doc/architecture/blueprints/ci_data_decay/index.md @@ -1,8 +1,11 @@ --- -stage: none -group: unassigned -comments: false -description: 'CI/CD data time decay' +status: ready +creation-date: "2021-09-10" +authors: [ "@grzesiek" ] +coach: "@kamil" +approvers: [ "@jporter", "@cheryl.li" ] +owning-stage: "~devops::verify" +participating-stages: [] --- # CI/CD data time decay @@ -23,18 +26,6 @@ the data storage for pipeline builds remains almost the same since 2012. In ia separate database. Now we want to improve the architecture of GitLab CI/CD product to enable further scaling. -_Disclaimer: The following contains information related to upcoming products, -features, and functionality._ - -_It is important to note that the information presented is for informational -purposes only. Please do not rely on this information for purchasing or -planning purposes._ - -_As with all projects, the items mentioned in this document and linked pages are -subject to change or delay. The development, release and timing of any -products, features, or functionality remain at the sole discretion of GitLab -Inc._ - ## Goals **Implement a new architecture of CI/CD data storage to enable scaling.** @@ -67,7 +58,7 @@ When a build gets archived it will not be possible to retry it, but we still do keep all the processing metadata in the database, and it consumes resources that are scarce in the primary database. -In order to improve performance and make it easier to scale CI/CD data storage +To improve performance and make it easier to scale CI/CD data storage we might want to follow these three tracks described below.  @@ -210,7 +201,7 @@ We accept the possible necessity of building a separate API endpoint / endpoints needed to access pipeline data through the API. In the new API users might need to provide a time range in which the data has -been created to search through their pipelines / builds. In order to make it +been created to search through their pipelines / builds. To make it efficient it might be necessary to restrict access to querying data residing in more than two partitions at once. We can do that by supporting time ranges spanning the duration that equals to the builds archival policy. @@ -246,35 +237,4 @@ In progress. - 2022-04-30: Additional [benchmarking started](https://gitlab.com/gitlab-org/gitlab/-/issues/361019) to evaluate impact. - 2022-06-31: [Pipeline partitioning design](pipeline_partitioning.md) document [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87683) merged. - 2022-09-01: Engineering effort started to implement partitioning. - -## Who - -Proposal: - -<!-- vale gitlab.Spelling = NO --> - -| Role | Who -|------------------------------|-------------------------| -| Author | Grzegorz Bizon | -| Engineering Leader | Cheryl Li | -| Product Manager | Jackie Porter | -| Architecture Evolution Coach | Kamil Trzciński | - -DRIs: - -| Role | Who -|------------------------------|------------------------| -| Leadership | Cheryl Li | -| Product | Jackie Porter | -| Engineering | Grzegorz Bizon | - -Domain experts: - -| Area | Who -|------------------------------|------------------------| -| Verify / Pipeline execution | Fabio Pitino | -| Verify / Pipeline execution | Marius Bobin | -| Verify / Pipeline insights | Maxime Orefice | -| PostgreSQL Database | Andreas Brandl | - -<!-- vale gitlab.Spelling = YES --> +- 2022-11-01: The fastest growing CI table partitioned: `ci_builds_metadata`. |