Pin auxiliary binaries the main Gitaly binary invokes

We have to maintain backwards compatibility between Gitaly and its
auxiliary binaries as they may originate from different builds during
an upgrade. This has caused incidents in the past and slows down
iteration velocity. To avoid the incidents and th burden of backwards
compatiblity, it would be great if we could guarantee Gitaly
only ever invokes the auxiliary binaries from the same build. We now
have the infrastructure in place to locate binaries on disk and to
pack the auxiliary binaries in the main Gitaly binary to ship them
as a single deployment. This commit hooks wires up the final changes to
ensure the correct binaries get invoked always.

The Gitaly binary contains the auxiliary binaries packed into itself. On
start up, Gitaly will unpack the binaries into the runtime directory.
BinaryPath helper used to locate binaries is updated to correctly locate
the unpacked binaries from the runtime directory. Since each Gitaly process
has its own runtime directory, the upgrades do not affect the binaries
unpacked there. Replacing the gitaly binary on the disk itself is atomic.
The combination ensures that the correct binaries are always invoked.

The testing infrastructure is updated to also locate the packed binaries
to the runtime directory. The auxiliary binaries are not always built prior
to tests but as a part of them. Most of the tests thus do not exercise
the packing or unpacking but only the fact that certain binaries are used
from the runtime directory.

The packed binaries are still installed so they'll be present during the
upgrade if the old Gitaly uses them. We can stop installing them in the
next release.

Changelog: fixed

1 files changed, 1 insertions, 1 deletions

diff --git a/packed_binaries.go b/packed_binaries.go
index 1eafb3639..16c02f2e4 100644
--- a/packed_binaries.go
+++ b/packed_binaries.go
@@ -14,7 +14,7 @@ const buildDir = "_build/bin"
 //go:embed _build/bin/gitaly-hooks _build/bin/gitaly-ssh _build/bin/gitaly-git2go-v15 _build/bin/gitaly-lfs-smudge
 //
 // packedBinariesFS contains embedded binaries. If you modify the above embeddings, you must also update
-// GITALY_PACKED_EXECUTABLES in Makefile.
+// GITALY_PACKED_EXECUTABLES in Makefile and packedBinaries in internal/gitaly/config/config.go.
 var packedBinariesFS embed.FS
 
 // UnpackAuxiliaryBinaries unpacks the packed auxiliary binaries of Gitaly into destination directory.