| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
'darwinjs-git-characteristics-that-make-horizontal-scaling-difficult' into 'master'
Integrating gitaly history information
See merge request gitlab-org/gitaly!3398
|
|
Restore repositories as per backup.rake
See merge request gitlab-org/gitaly!3383
|
|
Always cast reference transaction votes via backchannels
Closes #3262
See merge request gitlab-org/gitaly!3437
|
|
|
|
Changelog: added
|
|
This commit removes the backchannel voting feature flag, causing Gitaly
to always cast votes to Praefect via a backchannel.
Historically, Praefect sent dialing information to Gitaly when a reference
transaction was started. Gitaly would then dial back to Praefect to cast its
vote. This proved difficult as in some setups Gitaly was not able to connect
back to Praefect. An alternative approach was implemented where Gitaly
sends votes back to Praefect via the same connection Praefect established to the
Gitaly node. To facilitate this, the connection between Praefect and Gitaly was
multiplexed to allow bidirectional gRPC invocations. This gRPC stream back from
Gitaly to Praefect is the backchannel connection.
As Gitaly should be upgraded prior to the Praefects, there may still be older
Praefects calling an upgraded Gitaly. As such, the old implementation is still
left in Gitaly to avoid RPCs failing during the upgrade. Removing the old
implementation thus has to be done over multiple releases.
Changelog: changed
|
|
ci: Fix binaries job being broken by missing Makefile target
See merge request gitlab-org/gitaly!3427
|
|
git: Reject pushes into internal ref namespaces
See merge request gitlab-org/gitaly!3426
|
|
|
|
Move CreateTag to the gittest package
See merge request gitlab-org/gitaly!3430
|
|
CreateTag function is used to create tags and should
be moved into the gittest package. It not only more
consistent, but also breaks a circular dependency that
occurs if we replace MustRunCommand with the gittest.Exec
function. This replacement is done as well to break
dependency on the global config.Config variable.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Gitaly repositories host several internal ref namespaces which are used
by several different parts of our application stack. These include ref
namespaces for merge requests, pipelines and environments. Given that
all of these represent internal state and are thus managed by GitLab,
the expectation is that they shouldn't be modified by users. As it turns
out, we do not have any restrictions in place though when the user
performs a push: they can simply overwrite these references with their
own commits.
Fix the issue by hiding those refs for git-receive-pack(1). The effect
is twofold: first, these references aren't announced in the initial ref
advertisement. Second, git will deny all pushes into these namespaces.
While the second property is what we're after, the first one is handy
for us, too. For almost all repositories, the number of these special
refs outweighs the number of normal refs by quite a bit. As an example,
the initial ref advertisement of gitlab-org/gitlab results in 125,000
refs to be advertised. Of these refs, 115,000 are environments,
pipelines and merge requests, and only about 10,000 are real references.
Advertising this many refs makes the packfile negotiation much slower.
So by hiding these refs from the initial ref advertisement, not only
do we properly protect these internal refs, but we should also see a
speedup of that initial ref advertisement and thus the packfile
negotiation. Win-win.
Changelog: fixed
|
|
We have a bunch of internal reference namespaces which we need to treat
specially in some contexts. This list is currently defined in the
cleanup service as it is the only current user. Given that we're about
to introduce a second user, move this list into the `internal/git`
package to make it available for reuse in a central location.
|
|
[ci skip]
|
|
Refactoring of the NewHealthServerWithListener
See merge request gitlab-org/gitaly!3428
|
|
proto: Drop deprecated LFS pointer RPC
See merge request gitlab-org/gitaly!3424
|
|
ci: Run QA pipeline with EE-only tests
See merge request gitlab-org/gitaly!3431
|
|
With the introduction of ListLFSPointers, it was thought that it could
also replace usage of GetLFSPointers. But as it turns out, this is not
easily doable: callers rely on the fact that even if given for example
the object ID of a tree containing LFS pointers, that there is no walk
and thus no result. This cannot be implemented via git-rev-list(1),
given that its "--no-walk" flag just keeps us from walking down commit
parents, but it will still traverse down the root tree of each commit.
So instead of awkwardly trying to somehow retrofit it into
ListLFSPointers, let's just un-deprecate GetLFSPointers.
|
|
The GetAllLFSPointers RPC has been deprecated and all of its callers
converted to ListLFSPointers in v13.11. Given that no users remain, we
can now remove this RPC altogether.
Changelog: removed
|
|
The GetNewLFSPointers RPC has been deprecated and all of its callers
converted to ListLFSPointers in v13.11. Given that no users remain, we
can now remove this RPC altogether.
Changelog: removed
|
|
We're currently running the QA pipeline with the default Community
Edition, which is missing tests for Enterprise Edition-only features.
This commit thus switches the pipeline over to EE to enable better test
coverage.
|
|
Replace MustRunCommand with Exec|ExecStream in gittest package
See merge request gitlab-org/gitaly!3419
|
|
ci: Add manual job to trigger QA
See merge request gitlab-org/gitaly!3387
|
|
Usages of the MustRunCommand replaced with Exec
in order to break dependency on the global
config.Config variable.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Usages of the MustRunCommand replaced with Exec
in order to break dependency on the global
config.Config variable. The change includes only the calls
that don't require change of the function signatures.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Usages of the MustRunCommand replaced with Exec
in order to break dependency on the global
config.Config variable. All dependent functions
were changed as well. The same as the call site
for them.
In order to verify the list of the parameters used to
run git binary during service method execution the git
binary path replaced and that spyGitCfg used to run the
service. But the actual git still need to be used in other
cases. It was not an issue before as MustRunCommand uses
a global config.Config variable with proper git bin path set.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Usages of the MustRunCommand replaced with Exec
in order to break dependency on the global
config.Config variable.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Function initRepoAt used by InitBareRepoAt and
InitRepoWithWorktreeAtStorage to setup repositories
on the storage. By adding config.Cfg as an additional
parameter and replacing the MustRunCommand with Exec
we break dependency on the global config.Config variable.
The change also removes unused InitBareRepo and related
test for it.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Usages of the MustRunCommand replaced with Exec
in order to break dependency on the global
config.Config variable.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
config.Cfg is passed as an input parameter. It is used as a
replacement for the gloabl config.Config variable.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Usages of the MustRunCommand replaced with ExecStream
in order to break dependency on the global config.Config
variable.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Usages of the MustRunCommand replaced with Exec or ExecStream
in order to break dependency on the global config.Config variable.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/2699
|
|
Sometimes the test requires a gRPC server with only
a health service registered. It doesn't matter if
there are some kind of interceptors on it or not.
It is only required to check the flow is working
and the request reaches the service.
Now it is not required to explicitly stop the server
as it would be terminated automatically at the end
of the test execution. And as there are no other
use of the returned server the function doesn't return
it anymore.
|
|
Update dashboard URL and screenshots
Closes #3589
See merge request gitlab-org/gitaly!3418
|
|
It's currently more complicated than necessary to trigger a QA run with
changes which are currently in review. As a result, we tend to just not
bother and wait for QA to crash at a later stage, when changes had been
merged to the master branch already and when GitLab is about to bump the
Gitaly server version. While we certainly don't want all MRs to be
tested by QA, it would be reasonable to do so for MRs which e.g.
deprecate existing features or which implement significant changes like
for example the recent Go ports.
Improve the situation by adding a new manual "qa" job. If executed, then
it triggers a QA pipeline with the tip of the MR as Gitaly server
version.
|
|
Makefile: Unset PROFILE envvar before building git
See merge request gitlab-org/gitaly!3414
|
|
Update google-protobuf to 3.15.8 to support Ruby 3.0
Closes #3583
See merge request gitlab-org/gitaly!3399
|
|
Fix failing TestServerFactory
See merge request gitlab-org/gitaly!3425
|
|
With commit 5bf73d7a0 (Makefile: Drop assembly targets, 2021-04-28),
we've dropped both the assemble targets and the binaries target which
uses them. While the commit (probably validly) claims that no downstream
distributions or documentation use those targets, it was overlooked that
Gitaly itself does use it: when creating tags, the "binaries" CI job
compiles Gitaly with `make binaries` and then makes resulting binaries
available as artifacts, which is now broken.
Instead of bringing back the assemble targets, this commit instead
implements the job on top of `make build`: it does the same than `make
binaries` did, with the minor difference that the directory where
binaries end up in changes and that we don't create checksums.
Checksumming is thus now moved into CI and paths are adjusted to
accomodate for those changes.
|
|
TestServerFactory fails if run alone.
It depends on the ruby for no reason as all
we want to do is to check that request reaches
the gitaly service through praefect. We change
the test to trigger FindCommit RPC that is go-based.
|
|
[ci skip]
|
|
Makefile: Drop assembly targets
See merge request gitlab-org/gitaly!3420
|
|
operations: Fix flaky UserRebaseConfirmable tests
See merge request gitlab-org/gitaly!3421
|
|
cache: Fix flaky cache walker test
See merge request gitlab-org/gitaly!3423
|
|
remote: Add RemoteUrl parameter to FindRemoteRootRef
See merge request gitlab-org/gitaly!3412
|
|
|
|
Historically, the GDK used Gitaly's "assemble" target to install all
runtime dependencies into a single directory. This usage has long been
replaced by a simple `make -C /path/to/gitaly` invocation in a6e99ef7
(Move gitaly to top level, 2019-12-03). This implicitly executes the
"install" target with the install directory set to Gitaly's top-level
directory, so none of the assembly targets are involved.
Scanning through downstream distributions and documentation in
gitlab-org/gitlab didn't surface any other users of these targets, so we
may assume that they are not used anymore, making them safe for removal.
|
|
Our tests exercising the UserRebaseConfirmable RPC are failing in CI for
the Go implementation. The failure is a simple race: we're awaiting that
the RPC wraps up and returns an EOF error, but it's not fast enough to
do so in one second. This most frequently happens in the code coverage
job, and most notably it only happens for the Go port. This is a bit
puzzling considering that the Go port is supposed to be faster given
that it's not creating a worktree like the Ruby implementation does. But
the fact that it is the code coverage job which is hitting the issue
hints that we hit in Go only because there's more instrumentation to be
able to determine which code paths have been hit, which naturally
doesn't exist in the Ruby port.
In any case, it doesn't really help to assert that the RPC finishes in a
fixed amount of time here. So let's fix flakiness by just dropping the
timeouts.
|
