| Age | Commit message (Collapse) | Author |
|---|
|
Use gitlab-dangerfiles for review roulette
See merge request gitlab-org/gitaly!3734
|
|
Now that we have a gemfile it's easier to use the standard ruby image
and install our dependencies to run danger.
|
|
We should keep version of the database up to date with
the officially supported version of it for the GitLab.
The source is https://docs.gitlab.com/omnibus/package-information/postgresql_versions.html
That is why we update is to the 12.6 version for the
gitaly project.
|
|
As we always require a PostgreSQL DB instance to be
running during the test execution it makes no more
sense to guard SQL dependent tests with a special
build tag. The change removes the flag from the code.
As we modify files that has a lint issues we should
fix them as well otherwise the lint job will fail.
We still need praefect_sql_test CI job because it
verifies the service works properly with PgBouncer
instance between service and database.
|
|
To get more realistic tests we replace in-memory implementation
of the replication events queue with Postgres based.
It will require Postgres database to be available during unit-test
execution, but now it is not a problem as it is required for the
test-with-postgres task already. The caller is now responsible for
providing PGPORT, PGHOST and PGUSER env vars properly configured
before the test run.
As the tests run in parallel for each packages and each package has
its own database there is no problem of concurrent data change
until we have parallel tests running inside one package.
The name of the database is generated based on the package name
where from the setup function is called.
|
|
|
|
|
|
Upgrade the Git version we test against to v2.32.0. Given that we do not
yet bump the minimum required version, we also keep our jobs which test
against v2.31.1.
|
|
Our code navigation CI job is currently broken, where we always get an
Internal Server Error when uploading artifact created by lsif-go. As it
turns out, this breakage is caused by an upstream change which has
caused a change in the output format.
Fix the error by pinning the lsif-go version to v1.3.1, which is the
most recent version which still works without issue.
|
|
Our CI is still using the Praefect binary which we install into the
source directory. Convert it to instead use the new location, which is
in our build directory.
|
|
ci: Fix binaries job being broken by missing Makefile target
See merge request gitlab-org/gitaly!3427
|
|
We're currently running the QA pipeline with the default Community
Edition, which is missing tests for Enterprise Edition-only features.
This commit thus switches the pipeline over to EE to enable better test
coverage.
|
|
It's currently more complicated than necessary to trigger a QA run with
changes which are currently in review. As a result, we tend to just not
bother and wait for QA to crash at a later stage, when changes had been
merged to the master branch already and when GitLab is about to bump the
Gitaly server version. While we certainly don't want all MRs to be
tested by QA, it would be reasonable to do so for MRs which e.g.
deprecate existing features or which implement significant changes like
for example the recent Go ports.
Improve the situation by adding a new manual "qa" job. If executed, then
it triggers a QA pipeline with the tip of the MR as Gitaly server
version.
|
|
With commit 5bf73d7a0 (Makefile: Drop assembly targets, 2021-04-28),
we've dropped both the assemble targets and the binaries target which
uses them. While the commit (probably validly) claims that no downstream
distributions or documentation use those targets, it was overlooked that
Gitaly itself does use it: when creating tags, the "binaries" CI job
compiles Gitaly with `make binaries` and then makes resulting binaries
available as artifacts, which is now broken.
Instead of bringing back the assemble targets, this commit instead
implements the job on top of `make build`: it does the same than `make
binaries` did, with the minor difference that the directory where
binaries end up in changes and that we don't create checksums.
Checksumming is thus now moved into CI and paths are adjusted to
accomodate for those changes.
|
|
Our nightly job tests against git's next and master branches to verify
that there's not going to be any new incompatibilities with the upcoming
git version. But since we have started to apply custom patches to git,
this nightly job has started to fail because the patch does not apply on
top of these branches.
Override GIT_PATCHES for this job such that we test against unpatched
versions of git.
|
|
Bump the Go versions used to test Gitaly to 1.15 and 1.16.
|
|
We're about to bump the minimum required git version to v2.31.0. As a
preparatory step, this commit updates CI to not test the older git
v2.29.0 anymore.
|
|
Git has released the new bugfix release v2.31.1. Upgrade our CI to use
that new version wherever we were using v2.31.0 previously.
|
|
Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com>
|
|
With git v2.31.0 having been released, this commit switches our default
git version from v2.29.0 to v2.31.0. Given that v2.29.0 is still the
minimum required version, jobs which have been previously testing with
v2.30.1 have been converted to test with v2.29.0 to maintain that we
keep supporting the minimum required version.
|
|
Our merge request pipelines currently only test against Git v2.29.0,
and given that this is our minimum required version, we should continue
testing against it. But in order to ensure that we're compatible against
a broader range of versions and to enable a smooth upgrade path in case
we're going to bump the minimum version, we should also test against
newer git versions.
Add tests against the latest released version to improve our test
coverage.
|
|
The libgit2 and git dependencies need to be cloned, built and installed.
Over time, this has resulted in some clutter in our build directories,
where we have `_build/src/{git,libgit2}`, `_build/{git,libgit2}` and
`_build/src/libgit2/build`. This commit ends this wild growth and
instead introduces a new `_build/deps` directory which hosts a source,
build and install directory for each dependency.
|
|
With multistage deployments we have both old and a new versions
of the praefect service running and sharing the same database.
We must be sure the changes done to the database schema doesn't
affect the old version of the praefect functionality and it
operates without issues.
The new job was added to the pipeline to verify compatibility of
the previous release and latest changes done to schema migration.
The pipeline runs only if a new migration was added/modified.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/3338
|
|
Refactor linting to remove "new-from-rev" and use big allowlist instead
See merge request gitlab-org/gitaly!3019
|
|
The Docker images are generated for developers, but none of the Gitaly
developers use it. In fact, the images were broken for a while. With
this change the Gitaly project will no longer create our own docker
images, but we can still use the CNG images.
|
|
Since my 688682aa0 (Lint: add a "lint-strict" CI target, 2021-01-06)
we've been showing the (!) checkmark next to all MRs instead of the
green/red OK/fail checkmark.
Since these strict lint failures are just something we want to keep an
eye on and not really actionable for any one MR author let's move them
to the nightly run accessible at
https://gitlab.com/gitlab-org/gitaly/-/pipeline_schedules
|
|
Refactor the pseudo-target I added in 688682aa0 (Lint: add a
"lint-strict" CI target, 2021-01-06) so you can run this with "make
lint-strict".
|
|
As noted in
https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2962#note_479374433
I think it's more time-efficient to solve most of these issues in
batches. This adds a CI target for that, so we can see what the full
set of things the lint is complaining about is, regardless of how old
they are in the codebase.
We can then use that output to inform edits to .golangci.yml, i.e. to
make certain checks the default.
Or possibly to do choose another approach, such as getting rid of
the--new-from-rev default option in 20ef26439 (golangci: Enable error
and documentation linters, 2020-12-17), add more lints under
"lint-strict", and then only add new default lints once we have them
passing in the codebase in general.
|
|
|
|
We used to write most of our test data into the Gitaly repository
directly. This slowed done tests, left behind state and slowed down
several tools. This antipattern was corrected to instead write all data
into temporary directories.
In order to avoid such tests creeping into our code base again, this
commit changes our CI such that it makes most of our directories
read-only before executing tests. The exception here is our build
directory which is required to be writable for test preparations as well
as the Ruby directory.
|
|
CI: also test the 'next' branch in git.git
See merge request gitlab-org/gitaly!2716
|
|
Enable security related CI jobs
See merge request gitlab-org/gitaly!2785
|
|
Referencing an alias for the job rules generated
an incompatible list, so I had to give up on
DRYing the rules.
|
|
Improve on a62c6639 ("ci: Add scheduled jobs testing against Git
master", 2020-09-02) by also testing the 'next' branch in git.git.
This'll give us more advanced warning of any breakages, and the 'next'
branch is really quite stable (e.g. Google runs it internally, it's in
Debian experimental etc.).
We could also add "seen" here, but that branch is far more likely to
produce false positives.
Testing this requires pushing a commit which removes the schedule
rule:
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 9940ca11..9302df43 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -175,8 +175,6 @@ nightly:git:
- GO_VERSION: "1.14"
GIT_VERSION: ["master", "next"]
TARGET: [ test, test-with-proxies, test-with-praefect ]
- rules:
- - if: '$CI_PIPELINE_SOURCE == "schedule"'
cover:
<<: *cache_definition
As can be seen on
https://gitlab.com/gitlab-org/gitaly/-/pipelines/219137470 it passes
now.
There was an issue where the "next" branch on our mirror wasn't being
updated, but as of writing it's what's in github.com/git/git's "next":
https://gitlab.com/gitlab-org/gitlab-git/-/tree/next
The "seen" branch is still stale from 4 months ago. Maybe someone
kicked or setup a rule for "next", but not "seen":
https://gitlab.com/gitlab-org/gitlab-git/-/tree/seen ?
On naming: This used to be "test:master" before this change, and we
have a "test" target that's testing the matrix of go versions. I
originally went for "test:git_git" (as in git.git) here.
Let's try make it obvious that we're testing various git versions in
the name, but also that this is the nightly CI job, not run in the
regular pipeline. Originally we changed the test target name here, but
now we can keep "nightly:git".
|
|
Rename the job added in a62c66394 (ci: Add scheduled jobs testing
against Git master, 2020-09-02).
This is per the discussion in
https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2716 and on
slack where Patrick wanted to have "nightly" in the job name
somewhere.
|
|
The no-proto-changes target checks whether there's any changes to either
our generated Go or Ruby proto files. To do so, it's currently writing
into "proto.diff" in the repo's root directory. Let's instead write it
into our build directory to keep our repo as clean as possible.
|
|
With the recent introduction of the MergeRequest pipelines workflow, our
MRs stopped executing the Danger job. The reason is that it's only been
running for branches, which we now won't build anymore due to the
changed workflow.
Fix the issue by only running Danger on merqe requests now, which we
probably should've done in the first place anyway.
|
|
There were two statements of `include:` in `.gitlab-ci.yml`, the latter
ruled out the first. So bring them together and keep them closer to the
top for better visibility.
Related issue:
https://gitlab.com/gitlab-org/gitaly/-/issues/3264
|
|
As we're about to drop support for any Git version older than v2.29.0,
this commit prepares our CI setup by dropping all jobs which use an
older version than that.
|
|
Configure pipelines for merged results
See merge request gitlab-org/gitaly!2698
|
|
Both jobs use the same build image as the configured default image, so
let's just drop the explicit statements.
|
|
Instead of manually specifying the Go version to 1.14 in several build
images, this instead creates a default GO_VERSION variable which is set
to 1.14.
|
|
The current image used by our objectinfo fuzzer uses an image which
isn't built anymore, causing it to fail now. Upgrade it to a more recent
version to fix this.
|
|
The golangci-lint project only supported Go 1.13 until recently, but
this has since changed as they now support Go up to version 1.15. Let's
upgrade our CI job to thus also use the latest version supported by
Gitaly, which is 1.14.
|
|
This CI/CD configuration change is need to enable the checkbox
"Enable merge trains and pipelines for merged results".
More info at:
https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results
|
|
Part of https://gitlab.com/groups/gitlab-org/-/epics/2380
|
|
With Git v2.29.0 having been released, let's upgrade to that version.
|
|
or to the PgBouncer instance set in front of it. That is why
proxy_host and proxy_port configuration settings were added.
All connections need to be done via proxy if it is configured.
Exception to it is a listening connection that must be a direct
connection to the database instance.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/3119
|
|
Add fuzz testing to objectinfo parser
See merge request gitlab-org/gitaly!2481
|
|
As it is hard to reason about what is the difference between
local run and run on CI in the Docker container the first
step to identify what is the actual difference between two.
If it is only a formatting then it is possible that locally
generated version was formatted and committed.
To see the difference a new artifact 'proto.diff' would be
available after failed execution of the 'no-changes' task.
It will show delta between locally generated and committed
files vs files generated on CI during verification.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2855
|
