| Age | Commit message (Collapse) | Author |
|---|
|
Changelog: changed
|
|
|
|
The plan is to introduce a new locator wrapper that will write backup
manifest files. In order to properly locate the manifest file, we need
to add the backup ID and repository.
|
|
Locators only use repositories to determine paths and so do not need the
concrete repository types.
|
|
Previously locators only needed the specific step being backed up, but
soon we will be saving manifest files and this will require all steps
such that they can be persisted.
|
|
featureflag: Remove the `GitV242` flag
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6378
Merged-by: Christian Couder <chriscool@tuxfamily.org>
Approved-by: Christian Couder <chriscool@tuxfamily.org>
Co-authored-by: Karthik Nayak <knayak@gitlab.com>
|
|
service: Simplify injection of dependencies for servers
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6377
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: karthik nayak <knayak@gitlab.com>
Reviewed-by: James Liu <jliu@gitlab.com>
Reviewed-by: karthik nayak <knayak@gitlab.com>
|
|
The featureflag `GitV242` was introduced to rollout git v2.42. Since the
rollout was completed in the previous release (16.4), we can safely
remove the flag.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
ssh: Unify logic to handle git-upload-pack(1) and git-upload-archive(1)
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6379
Merged-by: James Liu <jliu@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: James Liu <jliu@gitlab.com>
Reviewed-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Reviewed-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Reviewed-by: James Liu <jliu@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
While we already have a `service.Dependencies` type around for quite a
long time, we still pass in dependencies explicitly when constructing
the actual server. This makes it harder than necessary to make a server
require more dependencies as you will have to adjust all callsites where
the server is currently getting constructed.
Simplify the code to instead inject the `service.Dependencies` type into
the server directly. This will allow us to propagate dependencies more
readily in the future.
|
|
Implement resizable semaphore data structure
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6366
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: James Liu <jliu@gitlab.com>
Reviewed-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Reviewed-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Reviewed-by: James Liu <jliu@gitlab.com>
Co-authored-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
|
|
With the preceding refactorings we have moved the only user of the
`largeBufferReaderFrom` structure to a different place, but didn't yet
move the implementation of the buffer itself. Let's do so now.
|
|
With the preceding refactorings of `monitorStdinCommand()` its current
name is not quite accurate anymore: it does not only set up monitoring
anymore, but also runs the command and handles various bits and pieces
of its lifecycle.
Let's rename the function to `runUploadCommand()`. At a later point we
can also move this functionality into a shared place such that it is
reusable for the smarthttp service. While at it, document what the
functions intent is.
|
|
The logic to run git-upload-pack(1) and git-upload-archive(1) is quite
complex. Despite the need to splice several standard streams, we also
need to handle the case where negotiation of the data that is to be sent
does not finish, which is done to address time-of-check-time-of-use
attacks.
Right now, much of the logic to do this setup is split up across the
callers and `monitorStdinCommand()`. As there are multiple callers of
the latter function, this means that some of the logic is duplicated.
Furthermore, it is quite hard to change `monitorStdinCommand()` to alter
its semantics, as it is not at all a self-contained building block.
Refactor this to move much of the logic into `monitorStdinCommand()`,
which brings us a bunch of advantages:
- The monitoring logic is more self-contained so that we can amend
it at a later point. This was the original motivation of this
patch as it will be required to fix cases where we don't correctly
detect that the mentioned negotiation has completed.
- It allows us to use the same counting writer for SSHUploadArchive
that we already use for SSHUploadPack. We thus get better insight
into how much data we're typically returning in this RPC.
- It allows us to use the same large buffer reader that we already
use for SSHUploadPack. This reader uses a larger-than-usual buffer
so that we can avoid many syscalls and should thus help to make
SSHUploadArchive more efficient.
- We can reuse the bits to detect cancellation via our pktline
monitor in case negotiation does not finish in time.
- We can reuse the error handling where the remote side hangs up
unexpectedly, which is thrown by Git's pktline protocol in case
the remote side hangs up in the middle of a write. This allows us
to label user-cancelled requests as `codes.Canceled` instead of
`codes.Internal` in SSHUploadArchive.
So overall we gain quite a lot of benefits with this refactoring while
also simplifying our code base.
Changelog: fixed
|
|
The `command.ExitStatus()` function is responsible for extracting an
exit code from an `exec.ExitError`. The way this is done is extremely
fragile though as we try to directly cast the passed-in error to the
target error type. It's quite likely though that the error may be
wrapped somewhere in the callchain, which would break the functionality.
Refactor the function to instead use `errors.As()` such that it knows to
peel the error chain correctly.
|
|
hook: Fix pack-objects hook test with context cancellation
Closes #5548
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6372
Merged-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
Add benchmark for BackupRepository RPC
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6305
Merged-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Reviewed-by: Will Chandler <wchandler@gitlab.com>
Co-authored-by: Will Chandler <wchandler@gitlab.com>
Co-authored-by: James Fargher <jfargher@gitlab.com>
|
|
|
|
|
|
log: Roundtrip `log.Logger` type
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6376
Merged-by: Will Chandler <wchandler@gitlab.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Reviewed-by: karthik nayak <knayak@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
[ci skip]
|
|
[ci skip]
|
|
trace2: Don't round offset event times
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6375
Merged-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Co-authored-by: Will Chandler <wchandler@gitlab.com>
|
|
In a prior interation, the field "BackoffBackoff" has a typo. It should
be "BackoffFactor".
|
|
limiter.AdaptiveLimit is used to store the adaptive concurrency limit.
It is updated by the adaptive calculator. This commit implements a hook
system to AdaptiveLimit. It allows callers to register hooks. When the
limit's current value is updated, those hooks are triggered. This system
allows other components to shrink or grow accordingly. They are
essential for queues in limiter, which will be modified in the later
commits.
|
|
This commit implements `resizableSemaphore`, which provides a way to
bound concurrent access to resources. It allows a certain number of
concurrent access to the resources. When the concurrency reaches the
semaphore size, the callers are blocked until a resource is available
again. The size of the semaphore can be resized freely in an atomic
manner.
This struct is not intended to serve as a general-purpose data structure
but is specifically designed for flexible concurrency control with
resizable capacity.
|
|
Remove some more uses of the `logrus` package in our codebase where it's
not strictly required.
|
|
Introduce our own wrapper type for the `logrus.Fields` type. This allows
us to get rid of the logrus import in many places and thus is another
step towards getting rid of the `logrus` dependency altogether in our
code base.
|
|
The `log.Logger` type still converts to a `logrus.Entry` when calling
functions like `WithField()` or `WithError()`. This was hard to change
though because a lot of callsites still relied on this conversion. By
now we have have fixed all blockers though, which allows us to finally
do the conversion and have the `log.Logger` type roundtrip to itself.
To do so, we inline the `logrus.FieldLogger` interface and adapt the
specific functions to return a `Logger` instead of a `logrus.Entry`.
This change also demonstrates that all internal interfaces now use the
interface exclusively and that we are free to adapt the interface as
required from now on.
Note that we also have to add two additional functions to the interface
to create the logging interceptors for the gRPC server. This is because
we have no way to create these interceptors manually from a `log.Logger`
anymore and thus need its help to do so for us.
|
|
Stop using logrus' `NullLogger` in favor of our own internal logger that
can be set up with the testhelper package.
|
|
Refactor out testhelper functions that create a logger to return our own
internal `log.LogrusLogger` instead of a `logrus.FieldLogger`. Add an
abstraction layer around adding hooks into the logger such that tests
don't need to reach into the logrus logger in general.
|
|
Add support for forking a single branch
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6325
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Reviewed-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Reviewed-by: Ash McKenzie <amckenzie@gitlab.com>
Reviewed-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Co-authored-by: Ash McKenzie <amckenzie@gitlab.com>
|
|
commit: Gracefully handle invalid arguments in RawBlame
Closes #5594
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6361
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: James Liu <jliu@gitlab.com>
|
|
When passing a path to git-blame(1) that would escape the repository
root it will return an error. Let's detect this early such that we can
gracefully return an `InvalidArgument` error.
|
|
log: Wrap the logrus logger
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/6364
Merged-by: James Fargher <jfargher@gitlab.com>
Approved-by: James Fargher <jfargher@gitlab.com>
Reviewed-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Reviewed-by: James Liu <jliu@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
