diff options
| author | Derrick Stolee <stolee@gmail.com> | 2018-07-12 22:39:29 +0300 |
|---|---|---|
| committer | Junio C Hamano <gitster@pobox.com> | 2018-07-20 21:27:28 +0300 |
| commit | fe1ed56f5e482507b54a4fb491273f122c5fd9ea (patch) | |
| tree | 2f1e2521f84234aaae3eba17b459fe5d26155e56 /packfile.h | |
| parent | 3227565cfdeabcb06eede914d38c8729e6ff1434 (diff) | |
midx: sort and deduplicate objects from packfiles
Before writing a list of objects and their offsets to a multi-pack-index,
we need to collect the list of objects contained in the packfiles. There
may be multiple copies of some objects, so this list must be deduplicated.
It is possible to artificially get into a state where there are many
duplicate copies of objects. That can create high memory pressure if we
are to create a list of all objects before de-duplication. To reduce
this memory pressure without a significant performance drop,
automatically group objects by the first byte of their object id. Use
the IDX fanout tables to group the data, copy to a local array, then
sort.
Copy only the de-duplicated entries. Select the duplicate based on the
most-recent modified time of a packfile containing the object.
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Diffstat (limited to 'packfile.h')
| -rw-r--r-- | packfile.h | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/packfile.h b/packfile.h index d2ad30300a..b0eed44c0b 100644 --- a/packfile.h +++ b/packfile.h @@ -69,6 +69,8 @@ extern int open_pack_index(struct packed_git *); */ extern void close_pack_index(struct packed_git *); +extern uint32_t get_pack_fanout(struct packed_git *p, uint32_t value); + extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *); extern void close_pack_windows(struct packed_git *); extern void close_pack(struct packed_git *); |