| Age | Commit message (Collapse) | Author |
|---|
|
Currently we extract each Gitaly's packed binaries serially. This task
consumes a majority of the time spent during initialization. By
extracting these files in parallel we can measureably reduce startup
time.
On a 4-core system this improves startup time by ~10%, using the same
benchmarking process as 96438c249 (gitaly: Don't block on preloading
licensedb, 2023-09-20):
Benchmark 1: ./gitaly-par serve config.toml
Time (mean ± σ): 228.5 ms ± 4.2 ms [User: 285.8 ms, System: 76.8 ms]
Range (min … max): 222.7 ms … 237.0 ms 13 runs
Benchmark 2: ./gitaly-st serve config.toml
Time (mean ± σ): 254.5 ms ± 6.9 ms [User: 315.1 ms, System: 75.6 ms]
Range (min … max): 246.7 ms … 272.2 ms 11 runs
Summary
./gitaly-par serve config.toml ran
1.11 ± 0.04 times faster than ./gitaly-st serve config.toml
On a 16-core system this improves to ~20%:
Benchmark 1: ./gitaly-par serve config.toml
Time (mean ± σ): 234.7 ms ± 6.0 ms [User: 326.4 ms, System: 169.5 ms]
Range (min … max): 228.6 ms … 247.5 ms 12 runs
Benchmark 2: ./gitaly-st serve config.toml
Time (mean ± σ): 282.9 ms ± 10.4 ms [User: 377.1 ms, System: 156.3 ms]
Range (min … max): 266.3 ms … 302.0 ms 11 runs
Summary
'./gitaly-par serve config.toml' ran
1.21 ± 0.05 times faster than './gitaly-st serve config.toml'
This does place more demand on the disk, but only momentarily. When the
host is under heavy io pressure, simulated here with `stress-ng --iomix
5` parallel extraction's performance advantage is extended to ~36% on
the 16-core system:
Benchmark 1: ./gitaly-par serve config.toml
Time (mean ± σ): 545.9 ms ± 159.9 ms [User: 731.7 ms, System: 177.7 ms]
Range (min … max): 326.9 ms … 913.7 ms 10 runs
Benchmark 2: ./gitaly-st serve config.toml
Time (mean ± σ): 740.9 ms ± 242.8 ms [User: 977.4 ms, System: 150.7 ms]
Range (min … max): 378.9 ms … 1029.0 ms 10 runs
Summary
'./gitaly-par serve config.toml' ran
1.36 ± 0.60 times faster than './gitaly-st serve config.toml'
Changelog: performance
|
|
Remove the now-unused gitaly-git2o command as well as its supporting
infrastructure.
|
|
While dc76390d6 (Enable gitaly-gpg to be built in bundled binary in
Makefile, 2023-06-09) added gitaly-gpg to the bundled binary in the
Makefile, there were a couple of places where we missed adding
gitaly-gpg so it gets recognized as a binary in the packed binaries
executable.
|
|
We're about to release Gitaly v16.0. As we've landed a bunch of
previously announced removals it's thus time to bump our Go module
version from v15 to v16.
|
|
Bulk update all file and executable permissions to use the new perm
package.
Strictly speaking changing os.ModePerm to perm.PublicFile is a
permission change (it goes from executable to normal), but since
ModePerm is only used in tests this should be safe.
|
|
Reformat surces with the newly upgraded gofumpt v0.4.0.
|
|
We don't properly verify the error code when closing the file that
corresponds to the packed binary. While this doesn't really matter in
the first place, fixing this one case lets us drop another exception to
our linting rules.
|
|
With the auxiliary binaries packed into the gitaly binary and deployed
as a single unit, we no longer need to maintain backwards compatibility
between them and Gitaly. This commit drops the now unnecessary major
version suffix from the gitaly-git2go binary which was needed due to the
major version upgrades causing the type paths to change and thus breaking
the Gob protocol between the binaries.
|
|
We have to maintain backwards compatibility between Gitaly and its
auxiliary binaries as they may originate from different builds during
an upgrade. This has caused incidents in the past and slows down
iteration velocity. To avoid the incidents and th burden of backwards
compatiblity, it would be great if we could guarantee Gitaly
only ever invokes the auxiliary binaries from the same build. We now
have the infrastructure in place to locate binaries on disk and to
pack the auxiliary binaries in the main Gitaly binary to ship them
as a single deployment. This commit hooks wires up the final changes to
ensure the correct binaries get invoked always.
The Gitaly binary contains the auxiliary binaries packed into itself. On
start up, Gitaly will unpack the binaries into the runtime directory.
BinaryPath helper used to locate binaries is updated to correctly locate
the unpacked binaries from the runtime directory. Since each Gitaly process
has its own runtime directory, the upgrades do not affect the binaries
unpacked there. Replacing the gitaly binary on the disk itself is atomic.
The combination ensures that the correct binaries are always invoked.
The testing infrastructure is updated to also locate the packed binaries
to the runtime directory. The auxiliary binaries are not always built prior
to tests but as a part of them. Most of the tests thus do not exercise
the packing or unpacking but only the fact that certain binaries are used
from the runtime directory.
The packed binaries are still installed so they'll be present during the
upgrade if the old Gitaly uses them. We can stop installing them in the
next release.
Changelog: fixed
|
|
Gitaly needs to carefully maintain backwards compatibility with its
auxiliary commands to avoid issues during upgrades. Omnibus and source
deployments upgrade the binaries in place which can cause a running
Gitaly to invoke a binary from the newer release. This has been the
root cause in multiple production issues. To avoid having to maintain
backwards compatibility between Gitaly and its auxiliary binaries, we
can pin the binaries in a manner that Gitaly only ever invokes the
auxiliary binaries from the same build as the main binary. To do so,
we can pack the auxiliary binaries into the main Gitaly binary and
unpack them into a temporary directory when starting up. This way the
upgrades happen atomically, as replacing the gitaly binary is an
atomic operation and it contains the auxiliary binaries as well.
This commit adds the functionality to pack and unpack binaries in the
main Gitaly binary. When building the Gitaly binary, the auxiliary
binaries are taken from _build/bin and embedded in the main binary.
UnpackAuxiliaryBinaries can then be called on starting Gitaly to unpack
the binaries into a temporary directory where they can be invoked from.
In order to embed the binaries in Gitaly during the build, they have to
be built prior to building Gitaly. Makefile is updated to set the packed
binaries as dependencies of Gitaly's build. Gitaly depends on the fully
built binaries so the GNU build-id remains intact in the packed binaries.
The packed binaries are set also as dependencies for the prepare-test target
so the binaries exist for the tests that exercise the packing related code.
In similar fashion, the lint target requires the packed binaries to be
present as otherwise the linter will fail as it will find build failures
in Gitaly due to the embedding failing due to missing binaries. The packed
executables are added as a dependency to the lint target. Same goes for the
notice target which needs to have the packed binaries in place so the build
succeeds. Packed binaries are also added as dependency for the notice target.
|
