| Age | Commit message (Collapse) | Author |
|---|
|
We've implemented a bunch of fixes to the language calculation. To
ensure we recalculate repositories using these new rules, bump up the
version in the cache file.
|
|
When the user has modified .gitattributes since last time the stats
where calculated, the numbers might be completely different for files
that aren't modified. To overcome any miscalculation in this case, just
do a full recalculation when .gitattributes was modified.
|
|
The functionality of newLanguageStats() was a bit overloaded. Split it
up and stop returning pointers.
|
|
Users can override [1] the linguist behavior by writing things in
gitattributes(5). This change copies that behavior into the go
implementation.
[1]: https://github.com/github/linguist/blob/master/docs/overrides.md
|
|
Files that are either documentation, vendored, or generated should be
excluded from the stats.
|
|
In 5da3e9a23 (gitpipe: Add LsTree that calls git-ls-tree(1), 2022-06-14)
we've added a skipResult function to the gitpipe.lsTreeConfig, but there
was no way to actually use that feature outside the gitpipe package.
This change adds LsTreeWithSkip() so a user of LsTree() can specify a
skip function, similar to DiffTreeWithSkip().
|
|
As an extra step to prep for more features, this change moves the
determining the language of a single file into a separate file and
struct.
|
|
We're about to make to override language statistics using
gitattributes(5). This change would make the inner loop for handling
each file a lot more complex.
To prepare for this added complexity, refactor the code so it's easier
to extend.
|
|
To make it easier to split out code in smaller functions, move some
variables we need to the linguist.Instance struct. This will allow
future refactoring.
|
|
Now linguist no longer loads colors when it's created, there's no more
need to create an instance when Gitaly is started. So remove linguist as
a dependency.
|
|
Now we're no longer loading language colors from a json file, it also
does not make sense to make the path of this json file configurable. So
remove it from the config.
|
|
We're about to go all-in on go-enry for the language detection. Which
means we'll soon no longer can load the language colors from a json file
provided by the linguist gem. We could copy the file, but go-enry
provides colors as well. So use those instead.
This also made it possible so turn the Color() function into a normal
function, instead of a receiver function.
This has a little disadvantage: it's no longer possible to load a custom
set of colors. We believe this isn't used, so we're willing to give up
on that.
|
|
To have a little bit of insurance the colors on the languages don't
change at random, test a handful of languages and their color.
|
|
The linguist Instance does not use the git.CommandFactory variable, so
we can safely remove it.
|
|
Refactor remaining users of the ioutil package
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4899
Merged-by: Sami Hiltunen <shiltunen@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
|
|
Go 1.15 has deprecated the `ioutil` package and made replacements
available in the `io` or `os` packages. While we have already migrated
most callsites away, one notable omission was `ioutil.ReadDir()` as this
function doesn't have an exact replacement: the new `os.ReadDir()`
function does not stat all directory entries anymore.
Refactor the cleanup of stale cached files in the `cache` package to use
the new function.
|
|
Go 1.15 has deprecated the `ioutil` package and made replacements
available in the `io` or `os` packages. While we have already migrated
most callsites away, one notable omission was `ioutil.ReadDir()` as this
function doesn't have an exact replacement: the new `os.ReadDir()`
function does not stat all directory entries anymore.
Refactor the cleanup of stale leases in the `cache` package to use the
new function.
|
|
Go 1.15 has deprecated the `ioutil` package and made replacements
available in the `io` or `os` packages. While we have already migrated
most callsites away, one notable omission was `ioutil.ReadDir()` as this
function doesn't have an exact replacement: the new `os.ReadDir()`
function does not stat all directory entries anymore.
Refactor the cleanup of stale keep-around references in the
`GarbageCollect` RPC to use the new function.
|
|
Go 1.15 has deprecated the `ioutil` package and made replacements
available in the `io` or `os` packages. While we have already migrated
most callsites away, one notable omission was `ioutil.ReadDir()` as this
function doesn't have an exact replacement: the new `os.ReadDir()`
function does not stat all directory entries anymore.
Refactor the cleanup of stale worktrees in the `housekeeping` package
to use the new function.
|
|
Go 1.15 has deprecated the `ioutil` package and made replacements
available in the `io` or `os` packages. While we have already migrated
most callsites away, one notable omission was `ioutil.ReadDir()` as this
function doesn't have an exact replacement: the new `os.ReadDir()`
function does not stat all directory entries anymore.
Refactor `stats.GetPackfiles()` to use the new function and manually
stat the packfiles at callsites as required.
|
|
Go 1.15 has deprecated the `ioutil` package and made replacements
available in the `io` or `os` packages. While we have already migrated
most callsites away, one notable omission was `ioutil.ReadDir()` as this
function doesn't have an exact replacement: the new `os.ReadDir()`
function does not stat all directory entries anymore.
Refactor the `tempdir` package's clean walker to use the new function
and manually stat the directory entries as required.
|
|
[ci skip]
|
|
[ci skip]
|
|
[ci skip]
|
|
proto: Deprecate embedded errors
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4894
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: karthik nayak <knayak@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
|
|
While the embedded error in the UserDeleteBranchResponse message has
been deprecated already, there is no information when the field will be
removed. Add it so we don't forget to clean it up.
|
|
The UserCherryPick RPC was converted to return structured errors instead
of embedding errors in a successful response. These embedded errors are
thus never set by Gitaly anymore and should be phased out.
Mark them as deprecated and slate them for removal in release 16.0.
Change: deprecated
|
|
The UserCreateBranch RPC was converted to return structured errors
instead of embedding errors in a successful response. These embedded
errors are thus never set by Gitaly anymore and should be phased out.
Mark them as deprecated and slate them for removal in release 16.0.
Change: deprecated
|
|
The UserCreateTag RPC was converted to return structured errors instead
of embedding errors in a successful response. These embedded errors are
thus never set by Gitaly anymore and should be phased out.
Mark them as deprecated and slate them for removal in release 16.0.
Change: deprecated
|
|
praefect: fix race condition on logger
Closes #4457 and #4473
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4886
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: Will Chandler <wchandler@gitlab.com>
|
|
The `praefect` command has a global logger that is passed to any
subcommands executed. When executing normally this is fine as each
process will only run a single subcommand. However, in testing this
creates a risk of race conditions when tests are run in parallel, as
each test goroutine will share this logger.
To resolve race detector failures on `TestAddRepositories_Exec()` and
`TestAddRepositories_Exec()`, we will now run these tests serially. This
has minimal impact on test execution time, increasing from 4.663s to
5.129s.
In the medium-term we should replace the global logger with a local that
allows parallel testing. Issue #4500[0] has been opened for this.
[0] https://gitlab.com/gitlab-org/gitaly/-/issues/4500
|
|
make: use 'override' when adding to TEST_OPTIONS
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4890
Merged-by: John Cai <jcai@gitlab.com>
Approved-by: Pavlo Strokov <pstrokov@gitlab.com>
Approved-by: John Cai <jcai@gitlab.com>
Co-authored-by: Will Chandler <wchandler@gitlab.com>
|
|
operations: Always use structured errors in UserCreateTag RPC
Closes #4372
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4882
Merged-by: John Cai <jcai@gitlab.com>
Approved-by: John Cai <jcai@gitlab.com>
Co-authored-by: Patrick Steinhardt <psteinhardt@gitlab.com>
|
|
doc: Clarify GitLab/Gitaly boundary for RPCs
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4878
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Andras Horvath <ahorvath@gitlab.com>
Approved-by: Mark Wood <mwood@gitlab.com>
|
|
Bump .tool-versions to use Go 1.18.6
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4892
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>
|
|
The GitLab Development Kit is moving to 1.18.6, so let's make all the
versions consistent to reduce the number of interpreters needed for
development.
Part of https://gitlab.com/groups/gitlab-org/-/epics/8843
|
|
go: Update module github.com/google/go-cmp to v0.5.9
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4863
Merged-by: Toon Claes <toon@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Co-authored-by: GitLab Renovate Bot <gitlab-bot@gitlab.com>
|
|
When `TEST_OPTIONS` is specified as an argument to `make`, any options
specified in the `Makefile` will be overwritten with the user-provided
value. In several targets we append arguments to `TEST_OPTIONS`, making
them vulnerable to being squashed inadvertently.
For example, if running `make race-go TEST_OPTIONS='-run=<TEST>
-count=20'` to reproduce a race condition, the intended command would
be:
gotestsum --format short -- -run <TEST> -count 20 -race ./...
But because `TEST_OPTIONS` is specified, this is actually executed as:
gotestsum --format short -- -run <TEST> -count 20 ./...
Resolve this by using the `override` directive in targets that attempt
to append to `TEST_OPTIONS`. This will allow the user to add custom
options without squashing the target's additions.
|
|
'4455-feature-flag-roll-out-user_create_branch_structured_errors' into 'master'
operations: Structured errors in UserCreateBranch
Closes #4455
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4883
Merged-by: Toon Claes <toon@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Co-authored-by: Karthik Nayak <knayak@gitlab.com>
|
|
Revert "Merge branch...
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4888
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Co-authored-by: karthik nayak <knayak@gitlab.com>
|
|
objectpool: Always prune refs on fetches to fix D/F conflicts
Closes #4394
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4885
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Justin Tobler <jtobler@gitlab.com>
|
|
With 018958fb1 (objectpool: Fix conflicting references when fetching
into pools, 2022-07-27) we have started to prune references in object
pools that have been removed in their primary pool member. This is a
necessity to avoid conflicting references in the object pool and its
member, which would otherwise break updating such pools completely.
This code has been rolled out to production on August 11th without any
observed negative fallout, and the flag was subsequently enabled by
default. Remove the feature flag guarding it.
Changelog: changed
|
|
golangci-lint: Stop excluding many `Close` and `Serve` functions from errcheck linter
See merge request https://gitlab.com/gitlab-org/gitaly/-/merge_requests/4810
Merged-by: Patrick Steinhardt <psteinhardt@gitlab.com>
Approved-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Approved-by: Pavlo Strokov <pstrokov@gitlab.com>
|
|
'4452-feature-flag-roll-out-simplify_find_local_branches_response' into 'master'"
This reverts merge request !4884
|
|
We don't properly verify the error code when closing the file that
corresponds to the packed binary. While this doesn't really matter in
the first place, fixing this one case lets us drop another exception to
our linting rules.
|
|
Many of our tests don't properly verify the error returned by the server
factory's `Serve()` function. Fix this shortcoming, which surfaces that
we mistakenly close some listeners manually that should indeed be closed
by gRPC itself. Drop the corresponding linting exception.
|
|
The `http.Server.Serve()` function will return an error when the HTTP
server has either failed unexpectedly or in case it wasn't properly shut
down. We don't check this error in many places though, which both causes
the errcheck linter to complain and hides issues we have with some of
our test setups.
Fix the missing error checks via a new `testhelper.MustServe()` helper
function that does the error checking for us. Drop the corresponding
linting exceptions.
|
|
The `grpc.Server.Serve()` function will return an error when the gRPC
server has either failed unexpectedly or in case it wasn't properly shut
down. We don't check this error in many places though, which both causes
the errcheck linter to complain and hides issues we have with some of
our test setups.
Fix the missing error checks via a new `testhelper.MustServe()` helper
function that does the error checking for us. This surfaces some test
errors for tests that forget to shut down the server or that manually
close the listener, which should be handled by gRPC itself. All of those
errors are fixed while at it.
Drop the corresponding linting exceptions.
|
|
We don't properly check the error when closing the `SidechannelWaiter`
in many places. Fix this and drop the corresponding linting exceptions.
|
