| Age | Commit message (Collapse) | Author |
|---|
|
Continuing the work of the parent commit, this change deduplicates the
server and client setup. As such the tests have very little boilerplate
left. The next item is probably the context.Context.
Visually the code is less cluttered now, and less seems to be going on.
|
|
This change introduces the suite utilities in one test to showcase how
we can use it to remove boilerplate from our test suite. As first
iteration the setup and teardown of the repository was choosen, though
it can be expanded trivially to include the service server and client.
As with many things, there's a tradeoff when executing tests. For
automated testing `make test` will work as expected, though `go test
-run=` will not work as easily as before. Currently I don't see a way
around that, as the regexp support for the `run` argument requires a
match on the suite name too. `-m` does work.
With the small commit we've got now, upon merging we'd devise a strategy
to include this throughout the Gitaly code base. Though it must be noted
that this can be done piece meal.
Additionally the output changes slightly with this change
```
=== RUN TestRepositoryService
=== RUN TestRepositoryService/TestGetInfoAttributesExisting
--- PASS: TestRepositoryService (0.02s)
--- PASS: TestRepositoryService/TestGetInfoAttributesExisting (0.02s)
--- PASS: TestRepositoryTestSuite/TestGetInfoAttributesExisting (0.02s)
```
|
|
Gitaly-git2go apply subcommand
See merge request gitlab-org/gitaly!2676
|
|
Introduction of in-memory cache for reads distribution
Closes #1466 and #3053
See merge request gitlab-org/gitaly!2738
|
|
Implements apply subcommand for gitaly-git2go that allows for
applying patches from diffs. Similar to 'git am', three-way merge
is attempted as a fallback if the patch does not apply cleanly.
|
|
To provide more visibility if reads distributions caching
enabled or not we log a message about it on startup.
It helps to understand if configuration used to start service
allows caching or not.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
To detect if cache was disabled we could search for corresponding
log message.
Unused parameters removed from 'tryCache' method.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/1466
|
|
|
|
With enabled distributed_reads feature each read operation leads to
a database query execution to get state of the storages for particular
repository. More read calls leads to more database access operations,
so the pressure to it increases in linear (or even worse).
To mitigate this problem it was decided to introduce an in-memory cache
added before accessing the database. Invalidation happens on receiving
notification events from the database. The events are send by the
triggers attached to the repositories (delete) and storage_repositories
(insert, delete, update) tables.
To monitor the cache a new counter was added: gitaly_praefect_uptodate_storages_cache_access_total.
It tracks amount of cache hits, misses and populates and evicts per virtual
repository. And to track an error rate of the notifications processing the
gitaly_praefect_uptodate_storages_errors_total was added with type set to one of:
retrieve, notification_decode.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/3053
|
|
tests: Convert remaining tests which write data into the repo
See merge request gitlab-org/gitaly!2822
|
|
Update Rouge gem to v3.25.0
See merge request gitlab-org/gitaly!2829
|
|
hooks: Improved validation and testing
See merge request gitlab-org/gitaly!2824
|
|
We used to write most of our test data into the Gitaly repository
directly. This slowed done tests, left behind state and slowed down
several tools. This antipattern was corrected to instead write all data
into temporary directories.
In order to avoid such tests creeping into our code base again, this
commit changes our CI such that it makes most of our directories
read-only before executing tests. The exception here is our build
directory which is required to be writable for test preparations as well
as the Ruby directory.
|
|
Our gitignore files contain quite a lot of rules for testdata. As we
shouldn't generate any data inside of the repository during tests
anymore, this commit cleans up our gitignore files to get rid of them.
|
|
Our test repositories are currently located in `internal/testhelper`.
They're quite hard to discover there and are intermingled with any
static test data which is part of the repo. This commit instead moves
the test repository into `_build`.
|
|
Inside our "tempdir" tests, we're using a "clean" directory to store
temporary storages, which is currently created in our "testdata". This
commit instead moves it into our global temporary test directory.
|
|
In order to test relative symlinks in combination with sockets, we
create a relative symlink in our testdata directory. This commit instead
moves it into the global temporary test directory.
|
|
We're creating a set of custom storages in our "namespace" tests. These
storages are currently created in "testdata/", but should instead be
created in a temporary directory
Fix tests to use `ioutil.TempDir()`. As we don't have a test context
available at time of creation, we cannot use `testhelper.TempDir()`.
|
|
We're currently creating the "gitlab-shell" test directory in
"testdata". This commit instead moves it into the global temporary test
directory.
|
|
In order to test relative symlinks in combination with sockets, we
create a relative symlink in our testdata directory. This commit instead
moves it into the global temporary test directory.
|
|
In order to capture the environment for both hooks and Git itsel, we're
writing a custom scripts which write the environment into `tesdata/`.
This commit converts them to instead use the global temporary test
directory.
|
|
Instead of creating "gitlab-shell" in testdata, this commit moves it
into the global temporary test directory.
|
|
We're currently creating the test Git repository in testdata. This
commit instead moves it into the global temporary test directory.
|
|
Instead of creating a temporary directory in "testdata/", this let's
just use a global one which is likely in "/tmp".
|
|
If calling `log.NewHookLogger()`, then we'll use the environment
variable GITALY_LOG_DIR to figure out where to write the logfile to. But
in case it's unset, we end up simply writing into the current directory,
wherever that is.
Fix the issue by creating a nil-logger in case the variable is unset or
empty.
|
|
CI: also test the 'next' branch in git.git
See merge request gitlab-org/gitaly!2716
|
|
Enable security related CI jobs
See merge request gitlab-org/gitaly!2785
|
|
Referencing an alias for the job rules generated
an incompatible list, so I had to give up on
DRYing the rules.
|
|
Add HasBranches method to Repository
See merge request gitlab-org/gitaly!2806
|
|
[ci skip]
|
|
Upgrades git2go to v30.0.18 which contains the bindings for ApplyToTree.
|
|
Makefile: Don't write proto.diff into source directory
See merge request gitlab-org/gitaly!2823
|
|
Prepare for global reference-transaction hook executions
See merge request gitlab-org/gitaly!2825
|
|
Repositories service has some duplicated code with `git.LocalRepository`.
This changes the service to reuse the existing implementation to check
if the repository contains branches.
|
|
Change remote repository to dial in the constructor once and to
reuse the connection in method calls. Deduplicates some of the code
and simplifies the struct as we have to hold less information.
|
|
Adds HasBranches to Repository for checking whether the repository
has local branches or not. Some operations service calls need to
know whether a repository has branches or not.
|
|
Wrong flag names in FetchRemote ported to Go
Closes #1466
See merge request gitlab-org/gitaly!2817
|
|
Improve on a62c6639 ("ci: Add scheduled jobs testing against Git
master", 2020-09-02) by also testing the 'next' branch in git.git.
This'll give us more advanced warning of any breakages, and the 'next'
branch is really quite stable (e.g. Google runs it internally, it's in
Debian experimental etc.).
We could also add "seen" here, but that branch is far more likely to
produce false positives.
Testing this requires pushing a commit which removes the schedule
rule:
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 9940ca11..9302df43 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -175,8 +175,6 @@ nightly:git:
- GO_VERSION: "1.14"
GIT_VERSION: ["master", "next"]
TARGET: [ test, test-with-proxies, test-with-praefect ]
- rules:
- - if: '$CI_PIPELINE_SOURCE == "schedule"'
cover:
<<: *cache_definition
As can be seen on
https://gitlab.com/gitlab-org/gitaly/-/pipelines/219137470 it passes
now.
There was an issue where the "next" branch on our mirror wasn't being
updated, but as of writing it's what's in github.com/git/git's "next":
https://gitlab.com/gitlab-org/gitlab-git/-/tree/next
The "seen" branch is still stale from 4 months ago. Maybe someone
kicked or setup a rule for "next", but not "seen":
https://gitlab.com/gitlab-org/gitlab-git/-/tree/seen ?
On naming: This used to be "test:master" before this change, and we
have a "test" target that's testing the matrix of go versions. I
originally went for "test:git_git" (as in git.git) here.
Let's try make it obvious that we're testing various git versions in
the name, but also that this is the nightly CI job, not run in the
regular pipeline. Originally we changed the test target name here, but
now we can keep "nightly:git".
|
|
Rename the job added in a62c66394 (ci: Add scheduled jobs testing
against Git master, 2020-09-02).
This is per the discussion in
https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2716 and on
slack where Patrick wanted to have "nightly" in the job name
somewhere.
|
|
Update LabKit library to v1.0.0
See merge request gitlab-org/gitaly!2827
|
|
Fix hook std(err|out) hook handling to be like Ruby
See merge request gitlab-org/gitaly!2804
|
|
docs: Remove text that Gitaly doesn't use Git2Go
See merge request gitlab-org/gitaly!2815
|
|
|
|
nodes: Use context to perform database queries
See merge request gitlab-org/gitaly!2796
|
|
This matches what is used in GitLab EE/CE and saves a little disk space
in an Omnibus install.
|
|
Document the ASDF_DATA_DIR workaround
See merge request gitlab-org/gitaly!2828
|
|
|
|
LabKit has reached a 1.0.0 milestone :tada:
See https://gitlab.com/gitlab-org/labkit/-/releases/v1.0.0
|
|
In the Ruby code in GitLab::Git::Hook we return either stderr if
there's anything there, or fall back to stdout.
This behavior dates back to gitlab-org/gitlab@926a8ab4765 ("Handle
custom Git hook result in GitLab UI", 2016-07-04). The current Ruby
code Gitaly runs was copy/pasted into the tree in 5d0dc144 ("Implement
CommitService.FindCommits", 2017-08-14).
When this was ported to Go in f7bb6743 ("operations: Port
UserMergeBranch from Ruby to Go", 2020-09-09) this logic error of
always using stdout was introduced. Fix it.
This makes the test added in the previous commit work. I'm fixing this
to be bug-compatible with the Ruby implementation which uses/used
".blank?".
I asked @vsizov whether he could recall whether the ".blank?" test was
needed or just some mistaken idiom for "empty string". He thought it
was safe to change it in Go to just use the len() test, but wasn't
sure.
I'm keeping the ".blank?" emulation here not so much out of the
paranoia of being bug-compatible, but because it's easier to run the
tests without things being conditionally different under the in-flight
features to migrate functionality to Go.
|
