Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitaly.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorPavlo Strokov <pstrokov@gitlab.com>2021-11-05 18:14:13 +0300
committerStan Hu <stanhu@gmail.com>2021-11-19 20:45:29 +0300
commit22062dc07ec38be6c5372702f1ce269404e8302e (patch)
tree2d475b5a2de6cdb64195839ebd4fb88bdfbe3611
parentf60e6616b8a6b65a601cae2468a80d99f4142bfd (diff)
sql-migrate: Update storage_repositories tablesh-fix-link-repository-id-migration-for-14-4
The batch update query introduced to mitigate limitation of the PostgreSQL on amount of payload size that can be send by the NOTIFY function was missing a condition in the update statements. Because of that the payload contained changes not for the N storage-repository entries, but for (N * num_of_storages) storage-repository entries. So initial size of 150 becomes 450 if 3 storages used. The change also includes significantly reduced batch size. The calculation was done on the test data similar to the production used data. The approximate payload size for single row is about 470 bytes. As max payload size is 8k bytes we are allowed to use no more than 16~17 entries. To be more realistic we reduce it to 14. Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/3806 Changelog: fixed
-rw-r--r--internal/praefect/datastore/migrations/20210906145021_link_repository_id.go13
1 files changed, 7 insertions, 6 deletions
diff --git a/internal/praefect/datastore/migrations/20210906145021_link_repository_id.go b/internal/praefect/datastore/migrations/20210906145021_link_repository_id.go
index 15deb0fe2..8ee84d968 100644
--- a/internal/praefect/datastore/migrations/20210906145021_link_repository_id.go
+++ b/internal/praefect/datastore/migrations/20210906145021_link_repository_id.go
@@ -16,21 +16,22 @@ DO $BODY$
UPDATE storage_repositories
SET repository_id = sub.repository_id
FROM (
- SELECT storage_repositories.virtual_storage, storage_repositories.relative_path, repositories.repository_id
+ SELECT storage_repositories.virtual_storage, storage_repositories.storage, storage_repositories.relative_path, repositories.repository_id
FROM storage_repositories JOIN repositories USING (virtual_storage, relative_path)
WHERE storage_repositories.repository_id IS NULL
- LIMIT 150
+ LIMIT 14
) AS sub
- WHERE storage_repositories.virtual_storage = sub.virtual_storage
- AND storage_repositories.relative_path = sub.relative_path
- RETURNING storage_repositories.repository_id
+ WHERE storage_repositories.virtual_storage = sub.virtual_storage
+ AND storage_repositories.storage = sub.storage
+ AND storage_repositories.relative_path = sub.relative_path
+ RETURNING storage_repositories.repository_id
)
SELECT COUNT(*) INTO count_val FROM updated_rows;
EXIT WHEN count_val = 0;
END LOOP;
END
$BODY$
-LANGUAGE plpgsql;
+LANGUAGE plpgsql
-- +migrate StatementEnd`,
`
UPDATE repository_assignments