| Age | Commit message (Collapse) | Author |
|---|
|
|
|
Now that we've gotten rid of ObjectInfoReader, remove the calls to it
and use the existing ObjectReader to request metadata.
|
|
Modify places where we still call ObjectInfoReader and replace with
ObjectReader. Also change a callsite where RequestRevision is still
being called to calling RequestContents.
|
|
Now that ObjectReader can request both contents and info, we don't need
both in the TreeEntry functions. Get rid of the ObjectInfoReaders.
|
|
Now that git cat-file has a --batch-command option, we no longer need to
keep both an ObjectReader and an ObjectInfoReader around. The
ObjectReader can provide both object metadata as well as content.
|
|
Move the Info() function over to ObjectReader. This also means that
instead of two queues, we only need one to issue commands to in
`cat-file --batch-command` mode.
|
|
Modify requestQueue to be able to request both object metadata as well
as content. Under `cat-file --batch-command` mode, issuing a
info deadbeef
command will return the object metadata, while
contents deadbeef
will return the object metadata as well as the contents. This will allow
the transition to only using one requestQueue instead of two.
|
|
Revert "Merge branch 'pks-revert-delete-refs-fix' into 'master'"
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4923
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: John Cai <jcai@gitlab.com>
|
|
Update .tool-versions to Go 1.18.7
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4938
Merged-by: James Fargher <proglottis@gmail.com>
Approved-by: James Fargher <proglottis@gmail.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>
|
|
Security announcement:
https://groups.google.com/g/golang-announce/c/xtuG5faxtaU
Part of https://gitlab.com/groups/gitlab-org/-/epics/8843
|
|
This reverts commit e49ea29543b2d8e71bfe4bdc3b295f785bd24fb1, reversing
changes made to 99b5528b66b23c1a8399027ecdef306267e668ae.
|
|
git: Validate \\ in refname
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4921
Merged-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Co-authored-by: John Cai <jcai@gitlab.com>
|
|
into 'master'
tools/protolint: Update module github.com/yoheimuta/protolint to v0.41.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4926
Merged-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
ruby: Update dependency gitlab-labkit to '~> 0.25'
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4854
Merged-by: Stan Hu <stanhu@gmail.com>
Approved-by: Stan Hu <stanhu@gmail.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
|
|
Rails is currently blocked on the version upgrade to Redis v5.0.0 and
newer, but the upcoming bump to gitlab-labkit Gem will cause us to pull
in this major Redis version upgrade.
Constrain the Redis Gem version to '~> 4.7.1' to not pull in the new
version until Rails has upgraded. This is the same version that Rails
currently uses.
|
|
Revert "Merge branch 'qmnguyen0711/add-limit-offset-to-search-files-by-names' into 'master'"
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4931
Merged-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Co-authored-by: James Fargher <proglottis@gmail.com>
|
|
'qmnguyen0711/add-limit-offset-to-search-files-by-names' into 'master'"
This reverts merge request !4911
|
|
Disable building of libgit2 tests
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4927
Merged-by: James Fargher <proglottis@gmail.com>
Approved-by: James Fargher <proglottis@gmail.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>
|
|
The tests require Python, which is not present on Cloud Native GitLab
images.
Relates to
https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4912#note_1132067823
|
|
Makefile: Upgrade libgit2 to v1.5.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4912
Merged-by: Sami Hiltunen <shiltunen@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
'master'
Add limit and offset to SearchFilesByName RPC
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4911
Merged-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Approved-by: karthik nayak <knayak@gitlab.com>
|
|
In the current implementation of SearchFilesByName RPC, all matched
files are returned back. In most cases, clients don't need all of them.
Instead, they perform pagination at their side. It makes sense to add
pagination from Gitaly side so that this RPC returns a reasonable subset
of files. This improvement would remove redundant payload in the
response.
To keep backward-compatibility, Gitaly doesn't enforce default limit or
offset. It's client's call.
Issue: https://gitlab.com/gitlab-org/gitaly/-/issues/4449
Changelog: added
|
|
go: Update module github.com/rubenv/sql-migrate to v1.2.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4867
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Approved-by: John Cai <jcai@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
tools: Keep package name consistent with module name
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4925
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: blanet <moweng.xx@alibaba-inc.com>
|
|
go: Update module github.com/go-git/go-git/v5 to v5.4.2
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4909
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
|
|
The new tools mechanism is great, this commit just changes the package
name for external tools from `gofumpt` to its own module name, to fix a
small nits from MR#4910.
Signed-off-by: blanet <moweng.xx@alibaba-inc.com>
|
|
Upstream has released libgit2 v1.5.0 on July 14th already, but so far we
had been blocked from upgrading as Git2go didn't yet support it. This
has now changed with the release of Git2go v34, so let's upgrade both so
that we run with the latest version.
Notable upstream changes include:
- Initial work on support for SHA256 as object hash.
- Fixes for CVE 2022-24765 and CVE 2022-29187, which could lead to
arbitrary code execution in repositories not owned by the current
user. libgit2 was not directly impacted, but the fixes now align
behaviour with Git and thus refuses to open repositories owned by
a different user.
- Several fixes for MIDX files, which might be beneficial with our
plans to start using them.
- The rename-detection limit for merges was bumped from 200 to 1000
files to match Git's behaviour.
There are many other changes, but due to our limited use of libgit2 most
of them aren't relevant to us.
|
|
go: Update module github.com/go-enry/go-enry/v2 to v2.8.3
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4917
Merged-by: James Fargher <proglottis@gmail.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
Update SAST to use Semgrep instead of Gosec
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4913
Merged-by: Justin Tobler <jtobler@gitlab.com>
Approved-by: Philippe Lafoucrière <plafoucriere@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
Co-authored-by: Will Chandler <wchandler@gitlab.com>
|
|
Makefile: Track Go tool versions via separate Go modules
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4910
Merged-by: Toon Claes <toon@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
With Gosec phased out in favor of the much faster Semgrep-based SAST
scanner, the performance implications of scanning the go cache are
minimal. Jobs which moved the cache[0] those that left it in-place[1]
are all taking roughly 20 to 30 seconds. We are also not seeing
vulnerabilities in depencies being reported against Gitaly itself, which
Gosec had been doing[2].
Remove the before_script to move the cache out of tree from the project
source.
[0] https://gitlab.com/gitlab-org/gitaly/-/jobs/3138452280#L29
[1] https://gitlab.com/gitlab-org/gitaly/-/jobs/3138531794#L29
[2] https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4341
|
|
As of GitLab 15.4 Gosec is not longer a supported scanner for SAST[0].
Switch over the the recommended Semgrep-based job.
[0] https://docs.gitlab.com/ee/update/deprecations#sast-analyzer-consolidation-and-cicd-template-changes
|
|
As of aa31a30ac3b4 (Removes SAST_DEFAULT_ANALYZERS variable,
2021-06-08), GitLab no longer checks the 'SAST_DEFAULT_ANALYZERS'
variable.
In addition, with 0f577f559cbb (Make SAST_DISABLE_DIND true, 2020-05-14)
the `SAST_DISABLE_DIND` now defaults to `true` and no longer needs to be
explicitly set.
Remove these variables from our .gitlab-ci.yml as they do nothing.
|
|
Makefile: Add `lint-fix` command
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4900
Merged-by: karthik nayak <knayak@gitlab.com>
Approved-by: John Cai <jcai@gitlab.com>
|
|
Improve NTP connectivity error message
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4916
Merged-by: Will Chandler <wchandler@gitlab.com>
Approved-by: Sami Hiltunen <shiltunen@gitlab.com>
Approved-by: karthik nayak <knayak@gitlab.com>
Approved-by: Will Chandler <wchandler@gitlab.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>
|
|
In ae728915 (refs: Return structured errors for DeleteRefs 2022-05-12),
we started to validate ref format and return a structured error. The
existence of a backslash is not something we currently validate, and
instead we rely on the `prepare` step to catch this error. We should
catch this in the ref validation step instead.
Changelog: changed
|
|
Update go-licenses to v1.4.0 by executing the following commands:
```
$ cd tools/go-licenses
$ go get github.com/google/go-licenses@latest
$ go mod tidy
```
There are no noteworthy updates in this
release.
|
|
Update gotestsum to v1.8.2 by executing the following commands:
```
$ cd tools/gotestsum
$ go get gotest.tools/gotestsum@latest
$ go mod tidy
```
There are no noteworthy updates in this release.
|
|
Now that most dependency versions are tracked via `go.mod` files it
makes more sense to keep the remaining versions in our Makefile close to
where they are used. So let's regroup them for improved locality.
|
|
Right now we track versions of our Go tooling directly in our Makefile.
While this is simple, it has several drawbacks:
- We're susceptible to supply-chain attacks in case an adversary
manages to replace the code used to build any of our tools.
- We cannot use proper dependencies in our Makefile, which adds the
need for `*.version` files.
- It is hard to build the tools outside of our Makefile as we don't
have a way to properly pull in the correct version.
- Upgrading our tooling requires us to manually hunt down new
releases for all of our tools.
We can fix these issues by following the approach that is efficially
recommended by the Go project [1]: every tool has its own Go module in
`tools/` with a "tool.go" file that imports the tool of interest. Like
this we can use Go's normal tooling to keep track of versions:
- We record hashes of the tool's sources as well as all of its
dependencies, making supply-chain attacks almost impossible.
- We can now provide proper dependencies in our Makefile: every tool
depends on "tool.go", "go.mod" and "go.sum". If any of them
changes we need to rebuild.
- The tools can be installed in the correct version simply by using
`go install` with the correct `go.mod` file.
- Upgrading tools is as simple as running `go get -u`, so no more
manual hunting for new versions.
While these benefits are great on their own already, we can go even
further with this refactoring: now that each tool has its own `go.mod`
file we can adapt the Renovate bot to pick up these files. This means
that we don't have to remember upgrading at all anymore, but instead the
bot will automatically upgrade them for us.
[1]: https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
|
|
Reorder the recipes so that our tools-related ones are grouped together.
|
|
With recent changes `gofmt` [1] started reformatting godoc comments.
This causes a problem wherein it reformats `//nolint: staticcheck` to
`// nolint: staticcheck`.
But it does ignore directives [2]. So let's change all our nolint to
directive format. This avoids the conflict with `gofmt`.
This fix was done by running `grep -r --include="*.go" -E "//nolint:
.*"` and manually fixing the issues.
[1]:
https://github.com/golangci/golangci-lint/issues/1658#issuecomment-1183148066
[2]:
https://github.com/golangci/golangci-lint/issues/3098#issuecomment-1214364533
|
|
Similar to the lint command, add a new `lint-fix` command which will
write the fixes back to the files.
|
|
Makefile: Update gofumpt to v0.4.0
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4907
Merged-by: John Cai <jcai@gitlab.com>
Approved-by: karthik nayak <knayak@gitlab.com>
Approved-by: John Cai <jcai@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
ref: Revert fix for DeleteRefs that broke QA
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4919
Merged-by: Toon Claes <toon@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
With 80d62284b (ref: Return structured error when update error is not
ErrAlreadyLocked, 2022-09-28) we have fixed a case where we still
returned successfully on error even though the feature flag to use
structured errors was enabled. This fix caused breakage in our QA jobs
though:
1st Try error in ./spec/lib/gitlab/git/repository_spec.rb:463:
expected Gitlab::Git::Repository::GitError, got #<Gitlab::Git::CommandError: 13:unable to prepare: state update to "prepare" failed: EOF, stderr: "fatal: invalid ref format: refs\\heads\\fix\n".> with backtrace:
downstream QA
Revert this change for now to unblock deployments.
|
|
On hosts where pool.ntp.org is not reachable, it's not obvious which
NTP server the clock synchronization check attempted to use and that
`NTP_HOST` can be set. This commit improves the error message.
For example, with `NTP_HOST` set to `example.com`, we can see:
Before:
```
FAIL: clock synchronization: praefect: query ntp: read udp 192.168.1.109:56948->93.184.216.34:123: i/o timeout
```
After:
```
FAIL: clock synchronization: praefect: query ntp host example.com: read udp 192.168.1.109:56948->93.184.216.34:123: i/o timeout
```
Or if `NTP_HOST` is not set:
Before:
```
FAIL: clock synchronization: praefect: query ntp host: read udp 192.168.1.109:56948->51.255.142.175:123: i/o timeout
```
After:
```
FAIL: clock synchronization: praefect: query ntp host pool.ntp.org: read udp 192.168.1.109:56948->51.255.142.175:123: i/o timeout (NTP_HOST was not set)
```
Changelog: changed
|
