| Age | Commit message (Collapse) | Author |
|---|
|
The `NewRepositoryName()` function computes a random repository hash for
the caller. This function has two different modes: one to create a bare
repository path with ".git" suffix, and one non-bare one without it.
There is basically not a single instance though where we'd want this
function to create a repository path without the ".git" suffix. In
production systems we always have that suffix as we only ever have bare
repositories. Which means that tests which do use a non-bare repository
path don't match our typical hashed storage layout. This is something we
should avoid though: it's preferable to have our test data as close to
production data as possible.
In the preceding commits we have already converted all remaining callers
that create non-bare repository paths to stop doing that. Let's thus
drop the parameter and unconditionally create bare repository paths so
that noone can be tempted to create a non-bare one anymore.
|
|
Some of our backup tests create non-bare repository paths which are
lacking the ".git" suffix in their relative path. This is not what we'd
typically have in production, and the suffix does make a difference in
backup tests given that we explicitly strip it in some cases when we
compute backup paths.
Convert the tests to create bare repository paths.
|
|
We're manually overriding relative paths in multiple different tests
when calling to `gittest.CreateRepository()`. The consequence is that we
are testing with a storage layout that does not match what we'd have in
production. This is important in the context of backups though given
that backup locations are derived from the repository's relative path.
Refactor the tests to not override the relative path anymore so that we
instead generate them in the `gittest` package via `NewRepositoryName()`.
|
|
Praefect: Fix transaction voter state race
Closes #4602
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5012
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: Justin Tobler <jtobler@gitlab.com>
|
|
When creating a new subtransaction previous voter state needs to be
propagated forward so the new subtransaction can be aware of previously
canceled voters. While reading the voter result state of the previous
subtransaction, the subtransaction could also have its voter state
updated through a call to `updateVoterState()`.
This change moves voter propagation logic to a new subtransaction method
`getPropagatedVoters()`. This method locks its associated subtransaction
to prevent concurrent writes to its voter result state.
|
|
Update pg_query to 2.2.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5007
Merged-by: Toon Claes <toon@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>
|
|
Merge with `security/master` after v15.3.5 security release
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5010
Merged-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: GitLab Release Tools Bot <delivery-team+release-tools@gitlab.com>
Co-authored-by: John Skarbek <jskarbek@gitlab.com>
Co-authored-by: Karthik Nayak <knayak@gitlab.com>
|
|
While we've already merged security/master into our tree via 08c88a2c8
(Merge branch 'sync-canonical-with-security-changes' into 'master',
2022-11-02), that commit merged with 702f036ed (Update changelog for
15.3.5, 2022-11-02). This is not the commit that ultimately landed in
the security/master branch though, but it instead is ef59e59f1 (Update
changelog for 15.3.5, 2022-11-02), which is a rebased version of the
former commit.
Because we have merged the wrong commit we now still see merge conflicts
in the Auto-Update bot for Gitaly. Let's fix this by merging the correct
commit, even though this doesn't bring any changes into our tree.
|
|
[ci skip]
|
|
Add git 2.38 as a build target in Makefile
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4943
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: John Cai <jcai@gitlab.com>
|
|
This fixes build issues with macOS SDK 13.0:
https://github.com/pganalyze/pg_query/issues/263
Changelog: fixed
|
|
Syncing master into gitaly
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5006
Merged-by: Stan Hu <stanhu@gmail.com>
Approved-by: John Skarbek <jskarbek@gitlab.com>
Co-authored-by: GitLab Release Tools Bot <delivery-team+release-tools@gitlab.com>
Co-authored-by: Karthik Nayak <knayak@gitlab.com>
|
|
[ci skip]
|
|
[ci skip]
|
|
[ci skip]
|
|
The `CreateRepositoryFromSnapshot` RPC gets the tar file from an HTTP
URL. It receives the file by doing a simple HTTP GET call using the Go's
standard library client.
To avoid DNS rebinding issues here, we use the provided resolved address
and do a custom DNS mapping by overriding the `DialContext` function of
the HTTP client.
Let's also add a test wherein we provide a random hostname but the
correct resolved address (IP). With HTTP, this should work.
Signed-off-by: Karthik Nayak <knayak@gitlab.com>
|
|
In `FetchRemote()` use `GetURLResolveConfig()` to get the configPair and
modified URL to avoid DNS rebinding. Unfortunately, because of the
structure of the code, the `git.Cmd` is not exposed on this level, which
means the added code is not testable.
|
|
In `updateRemoteMirror()` use `GetURLAndResolveConfig()` to get the
configPair and modified URL to avoid DNS rebinding. Unfortunately,
because of the structure of the code, the `git.Cmd` is not exposed on
this level, which means the added code is not testable.
|
|
In the `cloneFromURLCommand()` function lets use the
`GetURLAndResolveConfig()` function to obtain configPair and the
modified URL to prevent DNS rebinding. Add tests for the same.
|
|
Use the `GetURLAndResolveConfig()` to get modified URL and any
configPair if necessary to avoid DNS rebinding. Use this modified URL
instead and add the configPair to the git subcommand.
To make it easier to test, move the command instantiation part to a new
separate function `findRemoteRootRefCmd`. This makes it easier to test
the changes to the `git.Cmd`.
|
|
To facilitate the creation of the ConfigPair for the
`http.curloptResolve` flag or modification of the URL for avoiding DNS
rebinding we introduce the `GetURLAndCurloptResolveConfig()` function.
This function takes in the remoteURL and the resolvedAddress and then
provides the modified URL and configPair to be passed onto Git.
For HTTP/HTTPS URLs we leave the remoteURL as is and use the
`http.curloptResolve` flag which was introduced in Git. This is done by
providing a new ConfigPair with the value in the format of
`HOST:PORT:ADDRESS[,ADDRESS]`.
For SSH/Git protocol URLs we simply replace the hostname with the
provided resolved address.
The function does basic validation of the provided remoteURL and
resolved address and throws errors if any validation failed.
|
|
This field was introduced to handle DNS rebinding issues. But it was
never utilized by rails.
From the previous commit c21d6b31ab (proto: Add resolved_address along
remote URLs) we introduced a new field `resolved_address` which will be
used for DNS rebinding issues.
So mark the old `http_host` field as deprecated.
|
|
For the following protobuf definitions:
1. UpdateRemoteMirrorRequest
2. FindRemoteRootRefRequest
3. CreateRepositoryFromURLRequest
4. Remote
add a new field `resolved_address` which would hold the corresponding IP
address of the remote URL provided. This helps us prevent DNS rebinding.
Changelog: security
|
|
Add a git execution environment for v2.38, as well as a feature flag to
enable it.
|
|
ci: Introduce "analyze" stage
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5001
Merged-by: Toon Claes <toon@gitlab.com>
Approved-by: Pavlo Strokov <pstrokov@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
|
|
Remove rugged wrappers in Gitaly
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4992
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Co-authored-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
|
|
go: Update module gitlab.com/gitlab-org/labkit to v1.16.1
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5004
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
[ci skip]
|
|
[ci skip]
|
|
[ci skip]
|
|
objectpool: Revert error if object does not exist on fetch
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/5003
Merged-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
[ci skip]
|
|
|
|
In 3bfe68576 (objectpool: Raise error if object pool does not exist on
fetch, 2022-10-26), we have changed `FetchIntoObjectPool()` to raise an
error if the object pool didn't exist yet instead of silently creating
it. This fixes a bug in Praefect, which needs to know about all RPCs
which create a repository in order to record the new repository.
This change unfortunately breaks some tests in Rails. So let's revert it
for the time being so that we ca first amend these tests.
Changelog: fixed
|
|
objectpool: More test refactorings
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4973
Merged-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
All jobs that either test or analyze are part of the same "test" stage.
Given that the number of jobs has grown quite large by now this makes it
hard to find the job one is searching for.
Introduce a new "analyze" stage to split up those two groups of jobs.
|
|
The recently-added job that runs our tests on macOS is not grouped
together with all the other tests that run tests. Reorder it to improve
locality.
|
|
Praefect: Replicate to existing repository
Closes #4288
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4968
Merged-by: Sami Hiltunen <shiltunen@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Approved-by: Sami Hiltunen <shiltunen@gitlab.com>
Co-authored-by: Justin Tobler <jtobler@gitlab.com>
|
|
Changelog: other
|
|
go: Update github.com/ProtonMail/go-crypto digest to cf6655e
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4987
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Pavlo Strokov <pstrokov@gitlab.com>
Approved-by: John Cai <jcai@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
RuggedInterceptor was used to initialize some data when ruby sidecar
starts to handle a gRPC request. Those data were used in Gitlab's
repository wrapper around Rugged. That wrapper was removed which made
this interceptor redundant.
Changelog: other
|
|
After the wrappers for Rugged were removed, related supporting methods
become obsolete. One notable change in this commit is that we don't
need to create Gitlab::Git::Repository in find_license. Instead, we
can extract the repo path directly from the call argument.
Changelog: other
|
|
Those classes wrap around rugged data and provide some advanced
sugar-coating methods. Both FindLicense and gitaly-linguist use the
underlying gems exclusively. Those wrappers become obsolete now.
Changelog: other
|
|
Changelog: other
|
|
FeatureFlags is a wrapper for feature flags passed from Gitaly via gRPC
metadata. As there is barely anything left in ruby sidecar, it becomes
redundant.
Changelog: other
|
|
onboarding: Add gitaly retrospective repository
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4942
Merged-by: John Cai <jcai@gitlab.com>
Approved-by: John Cai <jcai@gitlab.com>
Co-authored-by: Karthik Nayak <knayak@gitlab.com>
|
|
At this point, nearly all services, except for FindLicense and
gitlay-linguist were removed from Ruby sidecar. We should remove the
redundant service clients in Gitaly.
Changelog: other
|
|
helper: Extra error creation functions
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4975
Merged-by: Pavlo Strokov <pstrokov@gitlab.com>
Approved-by: karthik nayak <knayak@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
|
|
The new functions ErrDataLossf(), ErrUnknownf() and
ErrUnimplementedf() will be in use soonish. They are
doing the same as other already existing code-based
errors.
The documentation updated with a short description
why helper wrappers should be used for error creation.
Part of https://gitlab.com/gitlab-org/gitaly/-/issues/4471
|
