From 94cd775a6c52a99caeb1278c3d8044ee109e2d3e Mon Sep 17 00:00:00 2001 From: Taylor Blau Date: Fri, 20 May 2022 19:17:35 -0400 Subject: pack-mtimes: support reading .mtimes files To store the individual mtimes of objects in a cruft pack, introduce a new `.mtimes` format that can optionally accompany a single pack in the repository. The format is defined in Documentation/technical/pack-format.txt, and stores a 4-byte network order timestamp for each object in name (index) order. This patch prepares for cruft packs by defining the `.mtimes` format, and introducing a basic API that callers can use to read out individual mtimes. Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano --- object-store.h | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) (limited to 'object-store.h') diff --git a/object-store.h b/object-store.h index bd2322ed8c..3c98028ce6 100644 --- a/object-store.h +++ b/object-store.h @@ -115,12 +115,20 @@ struct packed_git { freshened:1, do_not_close:1, pack_promisor:1, - multi_pack_index:1; + multi_pack_index:1, + is_cruft:1; unsigned char hash[GIT_MAX_RAWSZ]; struct revindex_entry *revindex; const uint32_t *revindex_data; const uint32_t *revindex_map; size_t revindex_size; + /* + * mtimes_map points at the beginning of the memory mapped region of + * this pack's corresponding .mtimes file, and mtimes_size is the size + * of that .mtimes file + */ + const uint32_t *mtimes_map; + size_t mtimes_size; /* something like ".git/objects/pack/xxxxx.pack" */ char pack_name[FLEX_ARRAY]; /* more */ }; -- cgit v1.2.3 From b757353676d6ffe1ac275366fa8b5b42b5d9727d Mon Sep 17 00:00:00 2001 From: Taylor Blau Date: Fri, 20 May 2022 19:17:52 -0400 Subject: builtin/pack-objects.c: --cruft without expiration Teach `pack-objects` how to generate a cruft pack when no objects are dropped (i.e., `--cruft-expiration=never`). Later patches will teach `pack-objects` how to generate a cruft pack that prunes objects. When generating a cruft pack which does not prune objects, we want to collect all unreachable objects into a single pack (noting and updating their mtimes as we accumulate them). Ordinary use will pass the result of a `git repack -A` as a kept pack, so when this patch says "kept pack", readers should think "reachable objects". Generating a non-expiring cruft packs works as follows: - Callers provide a list of every pack they know about, and indicate which packs are about to be removed. - All packs which are going to be removed (we'll call these the redundant ones) are marked as kept in-core. Any packs the caller did not mention (but are known to the `pack-objects` process) are also marked as kept in-core. Packs not mentioned by the caller are assumed to be unknown to them, i.e., they entered the repository after the caller decided which packs should be kept and which should be discarded. Since we do not want to include objects in these "unknown" packs (because we don't know which of their objects are or aren't reachable), these are also marked as kept in-core. - Then, we enumerate all objects in the repository, and add them to our packing list if they do not appear in an in-core kept pack. This results in a new cruft pack which contains all known objects that aren't included in the kept packs. When the kept pack is the result of `git repack -A`, the resulting pack contains all unreachable objects. Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano --- object-store.h | 2 ++ 1 file changed, 2 insertions(+) (limited to 'object-store.h') diff --git a/object-store.h b/object-store.h index 3c98028ce6..1c58d25355 100644 --- a/object-store.h +++ b/object-store.h @@ -339,6 +339,8 @@ int repo_has_object_file_with_flags(struct repository *r, */ int has_loose_object_nonlocal(const struct object_id *); +int has_loose_object(const struct object_id *); + /** * format_object_header() is a thin wrapper around s xsnprintf() that * writes the initial " " part of the loose object -- cgit v1.2.3