diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-08-18 11:17:02 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-08-18 11:17:02 +0300 |
commit | b39512ed755239198a9c294b6a45e65c05900235 (patch) | |
tree | d234a3efade1de67c46b9e5a38ce813627726aa7 /doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md | |
parent | d31474cf3b17ece37939d20082b07f6657cc79a9 (diff) |
Add latest changes from gitlab-org/gitlab@15-3-stable-eev15.3.0-rc42
Diffstat (limited to 'doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md')
-rw-r--r-- | doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md | 16 |
1 files changed, 13 insertions, 3 deletions
diff --git a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md index 60b20c50696..868dae4fc6c 100644 --- a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md +++ b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md @@ -306,9 +306,19 @@ We also need to build a proof of concept for removing data on the PostgreSQL side (using foreign keys with `ON DELETE CASCADE`) and removing data through Rails associations, as this might be an important area of uncertainty. -We need to [better understand](https://gitlab.com/gitlab-org/gitlab/-/issues/360148) -how unique constraints we are currently using will perform when using the -partitioned schema. +We [learned](https://gitlab.com/gitlab-org/gitlab/-/issues/360148) that `PostgreSQL` +does not allow to create a single index (unique or otherwise) across all partitions of a table. + +One solution to solve this problem is to embed the partitioning key inside the uniqueness constraint. + +This might mean prepending the partition ID in a hexadecimal format before the token itself and storing +the concatenated string in a database. To do that we would need to reserve an appropriate number of +leading bytes in a token to accommodate for the maximum number of partitions we may have in the future. +It seems that reserving four characters, what would translate into 16-bits number in base-16, +might be sufficient. The maximum number we can encode this way would be FFFF, what is 65535 in decimal. + +This would provide a unique constraint per-partition which +is sufficient for global uniqueness. We have also designed a query analyzer that makes it possible to detect direct usage of zero partitions, legacy tables that have been attached as first |