| Age | Commit message (Collapse) | Author |
|---|
|
This adds the proto definition for SSHUploadPackWithSidechannel along
with the accompanying auto-generated code.
|
|
As the last client of the deprecated fields now aligned
with the new schema we remove deprecated fields and
all the code related to them.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/3912
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/1746
Changelog: deprecated
|
|
The CommitsBetween RPC has been deprecated in favor of ListCommits,
which is a more flexible alternative. Rails has been converted to
ListCommits in f1f39db10ee (Use the ListCommits RPC for
Gitlab::Git::Commit.between, 2021-08-05) already, and with 96717bb
(gitaly: Convert from CommitsBetween to ListCommits, 2021-11-12) the
last existing callsite was removed from the Elasticsearch indexer.
Remove the RPC call completely.
Changelog: removed
|
|
Transactional votes currently only contain information about the vote
that we actually want to cast, but it doesn't contain any sort of
metadata which would allow us to more precisely track the nature of a
specific vote. Most importantly, while we already use "preparatory" and
"committing" votes in almost all places, they are only preparatory and
committing votes by convention: there is no metadata which would allow
us to discern these cases.
Amend the voting RPC call to use "phases", which currently knows three
different phases:
- "unknown": this is the current default which we'll need to support
until all voting callsites have been converted to use proper
phases.
- "prepared": the change has been prepared to be committed. Locks
have been taken already, but no change has yet been committed to
disk such that they would be visible to the user.
- "committed": changes have been committed to disk and are thus both
persistent and visible to the user.
With these phases, we can eventually build additional logic into
Praefect which makes use of them. This includes creating less
replication jobs if no change was committed, making two-phase voting a
necessity instead of a convention, and potentially others.
Changelog: added
|
|
Add 'praefect metadata' subcommand
Closes #3481
See merge request gitlab-org/gitaly!4122
|
|
Praefect has methods in the datastore to retrieve a repository's
metadata. This commit adds RPC definitions that allow for retrieving
the metadata from cluster. This RPC endpoint is going to be used by
a command line tool to retrieve a repository's metadata and by Gitaly
tests that run with Praefect in front and peek in to the filesystem.
Changelog: added
|
|
In order to keep the default branch up-to-date for incremental backups,
we need to start updating HEAD when applying bundles. Usually this would
be done in git via git-clone. Since we need to apply many bundles,
git-clone is no longer usable.
|
|
This removes the server side implementation of PackObjectsHook.
Changelog: removed
|
|
Gitaly contains CloneFromPool and CloneFromPoolInternal RPCs that were
added over two years ago in 495a384d41e8b4ed5a6e3e45375eda6aecd4f4fc.
They don't appear to be used anywhere in Gitaly or Rails. This commit
removes the unused RPCs.
Changelog: removed
|
|
This reverts commit f80312d8c (repository: Remove deprecated and unused
fields from GetRawChanges, 2021-11-11), which has removed `OldPath` and
`NewPath` from the RawChanges RPC's response. As it turns out, these
deprecated fields are still used by the Elasticsearch indexer. The
indexer will be converted to use new paths in v14.6 such that we can
reintroduce the removal in v14.7 in Gitaly.
|
|
The "old_path" and "new_path" fields of the GetRawChanges responses have
been deprecated ever since we have moved the protobuf definitions into
our repo in b9c9aec5c (Start preparation for migrating .proto files,
2019-07-05). They had originally been deprecated given that they were
using strings as types, and invalid Unicode characters in strings get
dropped when serializing and deserializing Protobuf messages. They had
thus been replaced with two fields "old_path_bytes" and "new_path_bytes"
and downstream callers have long been converted to use these.
Let's drop the old and deprecated fields -- they're not used by
anything, and neither should they ever be used.
|
|
The ListNewCommits RPC used to list all commits which were returned by
`git rev-list $newrev --not --all`, that is all commits which aren't
referenced by any existing ref. This RPC was replaced by ListCommits,
which is a lot more flexible and allows passing an arbitrary number of
revisions, including pseudo-refs "--not" and "--all". Downstream has
been converted in e8a98485fe7 (ff: Remove `:list_commits` feature flag,
2021-07-23) to exclusively use ListCommits, so ListNewCommits is now
unused.
Remove ListNewCommits.
Changelog: removed
|
|
The ref definitions import "blob.proto", which isn't used by anything.
Remove the import to squelch a warning generated by protoc.
|
|
The UnlikRepostioryFromObjectPool RPC is not used by anything anymore,
and the implementation from it is dangerous given that it doesn't
actually unlink a repository from its object pool: it only tries to
remove a remote named after the pool's project path, which we wouldn't
ever create in the first place. The RPC has thus been deprecated in
release v14.3.
Remove the RPC and its backing code.
Changelog: removed
|
|
This reverts commit 369fbdb16 (Merge branch
'pks-objectpool-drop-ondisk-remotes' into 'master', 2021-10-25), which
introduced a regression in Rails' test suite.
|
|
find_refs_by_id: Use 'refname' as the default sort order
See merge request gitlab-org/gitaly!4005
|
|
The default sort order was '-refname' (reverse lexicographic based on the refname) for FindRefsByOID. The general use case is to use the alphabetical 'refname' (lexicographic based on the refname) sort order, so this MR changes that.
The `git for-each-ref` command that performs the search doesn't need to be passed 'refname' explicitly, as 'refname' is the default for `git for-each-ref`. So the code can be simplified a bit.
Changelog: changed
|
|
The UnlikRepostioryFromObjectPool RPC is not used by anything anymore,
and the implementation from it is dangerous given that it doesn't
actually unlink a repository from its object pool: it only tries to
remove a remote named after the pool's project path, which we wouldn't
ever create in the first place. The RPC has thus been deprecated in
release v14.3.
Remove the RPC and its backing code.
Changelog: removed
|
|
Adds the proto definition and generated go and ruby code for a new RPC
FindRefsByOID used to find refs that point to a specific object ID with
some flags: patterns the refs need to match, a limit of how many results
are returned, and a field by which to sort the results.
See https://git-scm.com/docs/git-for-each-ref as a reference for how
these flags are used in git for-each-ref.
Changelog: added
|
|
Contributes to https://gitlab.com/gitlab-org/gitlab/-/issues/340591
**Problem**
FindAllTags request does not support pagination parameters. Lack of
pagination causes performance issues for repositories with many tags.
**Solution**
Add pagination parameter support similar to what we use for
FindLocalBranches.
Changelog: added
|
|
The FetchInternalRemote RPC had been used internally to replicate
repositories across different Gitaly nodes. At some point in time, this
was converted to do a direct fetch though without an additional RPC call
because as it turned out, doing inter-Gitaly mutating RPC calls is
problematic in combination with transactions given that the remote side
would now try to cast votes against another Gitaly. Nowadays, there are
no callers left which use this RPC.
Remove the deprecated `FetchInternalRemote()` call. The backing logic
which is still internally called remains though for use in other parts
of Gitaly. We may eventually want to find a better place for it to live.
Changelog: removed
|
|
In order to gracefully handle incremental backups where nothing has
changed, we need to determine when `git bundle create` failed because
the bundle was empty. This isn't an error for incremental backups, it
simply means there were no updates.
The FailedPrecondition GRPC code was chosen here since it best matches
the litmus test:
> (c) Use FailedPrecondition if the client should not retry until
> the system state has been explicitly fixed. E.g., if an "rmdir"
> fails because the directory is non-empty, FailedPrecondition
> should be returned since the client should not retry unless
> they have first fixed up the directory by deleting files from it.
Changelog: changed
|
|
'qmnguyen0711/1219-create-postuploadpackwithsidechannel-postuploadpack-replacement-using-sidechannel' into 'master'
Create PostUploadPackWithSidechannel
See merge request gitlab-org/gitaly!3883
|
|
Issue: https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1219
Changelog: added
|
|
Add FetchBundle RPC
See merge request gitlab-org/gitaly!3808
|
|
Unrevert RepositoryExists interceptor changes
See merge request gitlab-org/gitaly!3887
|
|
FetchBundle will allow updating an existing repository with the objects
and refs of a bundle, such as an incremental backup. Right now the only
other RPC is `CreateRespositoryFromBundle` but this requires the
repository to not already exist. For incremental backups, there will be
a series of bundles that need to be applied to a repository in a
specific order.
|
|
ref: Drop ListNewBlobs RPC
See merge request gitlab-org/gitaly!3893
|
|
In release 14.2, we have converted `UserSquash()` to never use worktrees
anymore. As a result, the old `IsSquashInProgress()` RPC doesn't really
do anything anymore and always returns the equivalent of `false`.
Upstream has thus been converted in b5b78c33c9f (merge_request: Drop
checks whether a squash is in progress, 2021-08-20) to not use the RPC
anymore, leaving it without callers.
Remove `IsSquashInProgress()`.
Changelog: removed
|
|
The `ListNewBlobs()` RPC call has been replaced by the more flexible
alternative `ListBlobs()`, which can do the same but across multiple new
refs at once. The last callsite in Rails has been removed in 0bc19a62ac3
(gitaly_client: Remove unused `#list_new_blobs` function, 2021-09-15),
so it is not used anywhere anymore.
Remove the RPC call.
Changelog: deprecated
|
|
Remove the deprecated `SetConfig()` RPC call. Modifying the on-disk
remote is not supported anymore by Gitaly.
Changelog: deprecated
|
|
Remove the deprecated `DeleteConfig()` RPC call. Modifying the on-disk
remote is not supported anymore by Gitaly.
Changelog: deprecated
|
|
This reverts commit 03276deb1449303a92b130741e4d1e3ed6053d68.
|
|
With Go 1.16, the ioutil package was deprecated. In addition to being
moved into the os package, `ioutil.ReadDir()` was also changed to not
stat(3P) all dir entries anymore. As a result, the caller now has to
do so manually. This is a performance improvement in some cases where
the caller didn't require any of the file information, but really only
wanted to read the directory's entries.
Adapt trivial usecases of `ioutil.ReadDir()` which do not require this
information with usage of `os.ReadDir()`. This leaves a few callsites of
the old `ioutil.ReadDir()` function for future conversion.
|
|
With Go 1.16, the ioutil package was deprecated. Replace our usage of
`ioutil.ReadAll()` with `io.ReadAll()` to adapt accordingly.
|
|
This reverts commit fdda523bafba3e028d7981c52719e7c1c34ae750.
|
|
This reverts commit 7560fb6cf7a21c7b31c133943f69cc5c9ff5329c, reversing
changes made to ccd8ea3e1436d6464ac5f67e6bebd1ae317f4d50. This change
has caused issues with QA tests where hundreds of tests now fail with
repositories not being found anymore.
|
|
ref: Deprecate `ListNewBlobs()` RPC
See merge request gitlab-org/gitaly!3873
|
|
objectpool: Deprecate `UnlinkRepositoryFromObjectPool()`
See merge request gitlab-org/gitaly!3874
|
|
Support marking RPC methods as intercepted
See merge request gitlab-org/gitaly!3858
|
|
The `UnlinkRepositoryFromObjectPool()` function is supposed to unlink a
repository from its object pool. But tall it does is to call `Unlink()`,
which in turn only removes the corresponding remote which pointed to the
repository from the object pool's repository. In fact, it's not even
really doing that given that we never create this remote in the first
place. As can be guessed by this no-op action, this RPC call is never
executed by anything.
Deprecate the RPC such that we can remove it in release 14.4.
Changelog: deprecate
|
|
The `ListNewBlobs()` RPC call has been replaced upstream by the more
flexible alternative `ListBlobs()`, which can do the same but across
multiple new refs as once. Let's deprecate this RPC such that we can
remove it in v14.4.
Changelog: deprecated
|
|
Praefect intercepts RepositoryExists calls and does not pass them through
the proxy to Gitalys. As we can now mark a single method from a service as
intercepted, let's do so for RepositoryExists. This allows us to remove its
operation type annotations which are unused and to clearly indicate it is
intercepted by Praefect.
|
|
Praefect currently supports marking an entire service as being intercepted.
When a service is intercepted, it's not required to have operation type
annotations which are used for proxying as they are not needed. This commit
adds support for marking a single method from a service as intercepted. This
allows for dropping annotations for methods which do not get proxied.
|
|
Praefect intercepts RepositoryExists calls to check whether a repository
should exist from the database rather than relaying to a Gitaly node.
Currently this is handled by a special interceptor that only handles
RepositoryExists calls. There's a better way by registering a handler
directly for the method. This way the interceptor doesn't end up in the
call chain of every RPC and it's easier to add more methods to intercept.
This commit moves the handling from an interceptor to a registered method.
|
|
The FetchInternalRemote RPC isn't used by anything anymore. Let's
deprecate it such that we can remove it in the next release.
Changelog: deprecation
|
|
When executing the formatter, we use `find_go_sources` to find the
set of Go files which should be formatted. This function excludes a
bunch of files which shouldn't ever be formatted, like for example
generated protobuf files. These exclusions have been too broad though
and caused us to never format sources of our protoc-gen-gitaly command,
which is also contained in "proto/go/".
Fix this by only excluding "proto/go/gitalypb/", which contains all
generated sources, and run the formatter on the newly found sources.
|
|
go.mod: Drop direct dependency on golang/protobuf
See merge request gitlab-org/gitaly!3809
|
|
When `UserMergeBranch` fails during hook execution, then we will not
cause the RPC to fail but instead embed the error in the response's
`PreReceiveError` field. This has led to quite some problems: Praefect
cannot determine failing RPC calls, we need to do weird games where we
need to make sure to pass through Rails' error messages verbosely, and
most importantly the upstream caller cannot easily deduce what really
has failed without comparing returned error messages. As a result, this
pattern which we use across many of our user-facing RPCs is quite
lacking.
To fix this, we can use gRPC's rich error model. This error model allows
us to embed arbitrary proto messages into the returned error, which
grants us the ability to return structured errors to the caller. Like
this, we can embed various information into errors in a structured way
which can then be extracted by upstream callers simply by unmarshalling
those error details.
Convert UserMergeBranch to use this rich error model as a proof concept.
For now, rich errors only include details in case the access checks help
to keep this change as focussed as possible. If this new model proves
viable, we may extend this to also include additional information in
different error cases, like for example the set of conflicting files in
case a merge fails.
Changelog: changed
|
|
The golang/protobuf dependency in deprecated in favor of
google.golang.org/protobuf. Migrate our usage of the plugin package to
use the new pluginpb package instead.
|
