Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitaly.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSami Hiltunen <shiltunen@gitlab.com>2021-12-17 12:28:58 +0300
committerSami Hiltunen <shiltunen@gitlab.com>2021-12-17 12:43:27 +0300
commit8f4dbe26d8a9fda9e95686be917c67464c12381c (patch)
tree979212e70dd84bf5d9a45c1349c7db00d0641819
parent61471fff6a262bc2c9c5270016f8a677e48638bb (diff)
Add versioning to Praefect disk storage path schemassmh-version-replica-paths
Praefect is about to begin generating unique relative paths for repositories to avoid on disk conflicts. The current format of `@repositories/xx/yy/<repo-id>` is sufficient for now and guarantees unique storage directories for repositories. The uniqueness of the paths is guaranteed by the repository ID sequence stored in Postgres. While this is sufficient for now, it doesn't leave room for evolving the schema of paths in the future. For example, if we wanted to ensure each replica lands in a unique directory to avoid the same races we had with repositories landing in the same directories, we'd have to define a new top level directory in the storage to avoid conflicts with the existing paths. Having to define new top level directories in the storage might be more confusing than having a single Praefect owned directory that can contain the disk path changes. This commit changes the top level directory in the storage from @repositories to @praefect. This clearly marks the repository as Praefect owned. A version number is added in to the path to allow for changing the expectations of the uniquness of the directories. For example, when we in future make replicas land in unique directories as well, we can do so by incrementing the path format version and giving the replicas paths like `@praefect/v2/xx/yy/<replica_id>`. If we didn't increment the version, the replica ids could conflict with the repository ids in the current path schema. Praefect is not yet generating unique paths, so this is a good time to change the format before creating the @repositories directory in the storages. Nothing would break even if we did but we can directly go to a format that can handle evolving the schema in the future.
-rw-r--r--internal/praefect/praefectutil/replica_path.go7
-rw-r--r--internal/praefect/praefectutil/replica_path_test.go4
2 files changed, 7 insertions, 4 deletions
diff --git a/internal/praefect/praefectutil/replica_path.go b/internal/praefect/praefectutil/replica_path.go
index 5f35e1e81..2b93f65c4 100644
--- a/internal/praefect/praefectutil/replica_path.go
+++ b/internal/praefect/praefectutil/replica_path.go
@@ -8,7 +8,10 @@ import (
// DeriveReplicaPath derives a repository's disk storage path from its repository ID. The repository ID
// is hashed with SHA256 and the first four hex digits of the hash are used as the two subdirectories to
-// ensure even distribution into subdirectories. The format is @repositories/ab/cd/<repository-id>.
+// ensure even distribution into subdirectories. The format is @praefect/v<version number>/ab/cd/<repository-id>.
+// The prefix '@praefect' marks the directory clearly as Praefect owned. The <version> component leaves room for
+// evolving our disk storage schema in future. If we make changes that may conflict with existing storage paths,
+// we can simply increment the version.
func DeriveReplicaPath(repositoryID int64) string {
hasher := sha256.New()
// String representation of the ID is used to make it easier to derive the replica paths with
@@ -16,5 +19,5 @@ func DeriveReplicaPath(repositoryID int64) string {
// an error.
hasher.Write([]byte(strconv.FormatInt(repositoryID, 10)))
hash := hasher.Sum(nil)
- return fmt.Sprintf("@repositories/%x/%x/%d", hash[0:1], hash[1:2], repositoryID)
+ return fmt.Sprintf("@praefect/v1/%x/%x/%d", hash[0:1], hash[1:2], repositoryID)
}
diff --git a/internal/praefect/praefectutil/replica_path_test.go b/internal/praefect/praefectutil/replica_path_test.go
index 6084c13ae..4761d8440 100644
--- a/internal/praefect/praefectutil/replica_path_test.go
+++ b/internal/praefect/praefectutil/replica_path_test.go
@@ -7,6 +7,6 @@ import (
)
func TestDeriveReplicaPath(t *testing.T) {
- require.Equal(t, "@repositories/6b/86/1", DeriveReplicaPath(1))
- require.Equal(t, "@repositories/d4/73/2", DeriveReplicaPath(2))
+ require.Equal(t, "@praefect/v1/6b/86/1", DeriveReplicaPath(1))
+ require.Equal(t, "@praefect/v1/d4/73/2", DeriveReplicaPath(2))
}