| Age | Commit message (Collapse) | Author |
|---|
|
Instead of always issuing "update" commands, use the "create" and
"delete" verbs too.
This doesn't make any difference in the result as git does the same
dispatching internally, but makes things a bit more obvious if you're
tracing the output.
|
|
Change the two users of Update(ref, <nullsha1>, <oldvalue>) to use the
shorter form of Delete(ref, <oldvalue>) instead, which is now just as
safe since we're making it take a mandatory oldvalue after the last
commit's refactoring.
|
|
Change the only users of the updateref.Delete() method (aside from its
own tests) to use the safer "Update" instead.
Why not teach Delete() to also take <oldvalue>? The next commit will
do that & change these to use *.Delete() back, but this refactoring
was easier to do atomically this way.
|
|
hooks: Convert GL_ values to be injected via the hooks payload
Closes #3201
See merge request gitlab-org/gitaly!2913
|
|
Enable feature flag gitaly_go_user_update_submodule by default
Closes #3290
See merge request gitlab-org/gitaly!2917
|
|
|
|
UserTag tests: minor cleanups & improvements
See merge request gitlab-org/gitaly!2905
|
|
Disable hooks when fetching
See merge request gitlab-org/gitaly!2923
|
|
Amend test code I added in 5896fdaf4 (UserCreateTag tests: test nested
annotated tags, 2020-12-08) to be less expensive. As pointed out in
[1] we're not getting anything extra from testing 10 nested levels,
and spending a lot of time doing so.
Also change the loop test from "<" to "<=" so the nested level refers
to the level of nested tags. I.e. tag->tag->commit is just 1 level of
nesting, not 2 or 3.
1. https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2905#note_465020107
|
|
Follow-up the last commit (see its commit message) and fix another
assignment to testCase, this one was added in 2ad030438 (Implement
UserCreateTag RPC, 2017-09-26). While we're at it let's be consistent
here with later tests I've added and test the entire response, not
just response.Tag.
That re-alignment of "[]byte(inputTagName)" is "make format" being
stupid, why not align all the key-values just because there's a tiny
comment in the middle, but I guess it's something that makes sense for
larger comments...
|
|
Fix a logic error of mine in b1f09d511 (UserCreateTag: fix internal
error in tree/blob tag creation, 2020-12-04).
It makes no sense to be assigning to our test data itself during the
loop. As @toon points out in [1] the testCase variable is a reference.
This didn't introduce a logic error in the test itself, since we're
only using each item in the testCase array once, so it's OK for the
test results if we mutate it as we go along. But let's change this to
something more sensible anyway & change our transitory responseOk
variable instead.
This anti-pattern was itself copied from earlier code in
2ad030438 (Implement UserCreateTag RPC, 2017-09-26). A follow-up
commit will fix that test case.
1. https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2881#note_463363831
|
|
Amend the test I added in 5896fdaf4 (UserCreateTag tests: test nested
annotated tags, 2020-12-08) to use the hook setup boilerplate like the
tests before it.
I should have added this to 5896fdaf4 (part of
gitlab-org/gitaly!2881), but I didn't because I ran into an issue in
some WIP code with these hooks that turned out to be trivial. See [1].
1. https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2905#note_465020107
|
|
Change the pre-receive hook script to use an "expected_object_type"
variable like the update script.
Initially I'd intended to change this script further in a follow-up
commit, but that commit turned out not to make any sense and has been
rebased out. Still, the consistency here looks better.
|
|
|
|
With the same reasoning as the preceding commit, this disables the
reference-transaction hook for `FetchInternalRemote`.
Note that the test change to create a single commit in the source
repository is sufficient to demonstrate that hooks aren't executed
anymore. Previously, there simply weren't any changes which were
fetched, which caused us to invoke no hooks at all. With this change, we
now create a commit and would thus update a reference in the target
repository, but as the test setup was incomplete and didn't set up the
hooks service at all, it would've failed to execute the hook now. But as
we now disabled hooks, it continues to succeed.
|
|
When executing git-fetch(1), git will currently create one referecne
transaction per reference that is to be modified. This is quite
inefficient when used together with the reference-transaction hook, as
it effectively means that we execute the hook once for each reference.
If fetching hundreds of thousands of references, this directly
translates to hundreds of thousands of spawned processes, which means an
extreme performance hit.
Fix the issue for `FetchIntoObjectPool` by disabling the
reference-transaction hook for now. This is only a stop-gap measure
until we land the equivalent of `git fetch --atomic`, which creates a
single transaction only.
|
|
The updateref package will currently alwyas set up the Updater in such a
way that it uses the reference-transaction hook. This may not be wanted
in some contexts, which is why this commit now sets up a new option to
disable hooks altogether.
The way Options are implemented is really quite simplistic and extremely
limited. It's likely we won't ever add new options though, which makes
it fit for the current task at hand but for nothing else. It should be
easy enough to extend though, and using options is preferred to
introducing a new required parameter as the latter would require us to
change all callers.
|
|
Enable Go port of UserCommitFiles by default
See merge request gitlab-org/gitaly!2922
|
|
Run housekeeping on pool repository
See merge request gitlab-org/gitaly!2916
|
|
Enables the Go port of UserCommitFiles by default. The port has been
successfully tested in production without issues.
|
|
https://gitlab.com/gitlab-org/gitaly/-/merge_requests/2885 added
housekeeping on the origin repository during a FetchIntoObjectPool RPC,
but not on the pool repository itself.
We now remove housekeeping from origin repo and only clean up the pool
repository.
Relates to https://gitlab.com/gitlab-org/gitaly/-/issues/3305
|
|
Clean up of the internals of the Coordinator type
See merge request gitlab-org/gitaly!2920
|
|
Removal of unused RepositoryStore dependency
See merge request gitlab-org/gitaly!2919
|
|
Revert "Merge branch 'sh-create-fork-with-object-pool' into 'master'"
See merge request gitlab-org/gitaly!2918
|
|
The method directRepositoryScopedMessage is used only in
one place inside of the StreamDirector method. And as the
method is already has a verification of the targetRepo before
directRepositoryScopedMessage method call it has no sense to
do it once again inside it.
|
|
The grpcCall struct was introduced in the c954a53a1 (Praefect:
proper multi-virtual storage support, 2020-04-28) commit
to reduce the number of parameters used in methods. It contains
all info needed to execute proxy operation and there is no need
to pass target repository or anything like that as a separate
parameter because those are already present in the grpcCall.
This commit removes redundant parameters from the methods and
substitutes their usage with usage of the grpcCall parameter
that has the same value.
|
|
As the strategy of interacting with the repository
changes there is no more need to provide RepositoryStore
into Mgr as a dependency as it has no usage.
This commit removes RepositoryStore from the list of the
input parameters of the constructor function and from the
list of fields of the Mgr struct.
|
|
This reverts merge request !2887
|
|
Handle nil index entries in resolve conflicts
See merge request gitlab-org/gitaly!2895
|
|
|
|
Stop using require.Error to compare error values
See merge request gitlab-org/gitaly!2896
|
|
[ci skip]
|
|
[ci skip]
|
|
Support cloning with an object pool in CreateFork
See merge request gitlab-org/gitaly!2887
|
|
This is needed to support fast forking. When an object pool is provided,
forking can cheap in terms of disk space and time since the clone only
needs to fetch the references.
The `git clone` can take in a `--reference <repository>` flag and will
output an alternates file with absolute paths. To maintain the use of
relative paths, after the fork is successful we recreate the alternates
file.
Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/24523
|
|
As the preceding commit has established that we can correctly use the
fallback mechanism for the new ReceiveHooksPayload, this commit now
converts all users to inject the payload instead of the GL_ environment
variables.
|
|
This commit converts the hooks to use the ReceiveHooksPayload instead of
relying on the GL_ values. As callers don't yet inject the payload but
instead still inject the old GL_ environment variables, this nicely
demonstrates that the fallback mechanism works as expected.
|
|
Next to the values we already inject via the HooksPayload, there's also
some information which only applies to the git-receive-pack(1) hooks
pre-receive, update and post-receive. This includes information about
the user and about the protocol which is used for the push, both which
are used to establish whether the current push is allowed to proceed.
This commit thus introduces a new optional ReceiveHooksPayload structure
to the HooksPayload, which contains this information.
As with the other fields, we also need to have fallback code in place to
enable zero-downtime upgrades where the server is still running the old
code, while the gitaly-hooks executable was already updated. We thus
have a fallback in place to read the GL_PROTOCOL, GL_ID and GL_USERNAME
values in case no payload was provided.
|
|
Same as with GL_REPOSITORY, we already have information about the
GL_PROJECT_PATH available via the protobuf Repo. So this commit stops
injecting the variable explicitly into the hooks environment and instead
extends `customHooksEnv()` to derive it from the Repo.
|
|
Now that we use the Repo protobuf to derive the value of GL_REPOSITORY,
this commit converts all callsites to stop injecting this value
directly. Instead, we derive it via a new function `customHooksEnv()`
such that custom hooks still get the expected environment.
|
|
One of the GL_ values is GL_REPOSITORY, which is used both for
authentication against GitLab's API as well as for the custom hooks
environment. It's currently passed as an explicit variable when invoking
hooks, but this is in fact not at all necessary as we already have the
`gitalypb.Repo`.
Simplify the code to not expect GL_REPOSITORY as an explicit variable
anymore, instead deriving it from the Repo protobuf.
|
|
The transaction fields for the HooksPayload are currently missing
documentation, which this commit adds.
|
|
Right now, our custom hook tests expect the same environment for the
invoked custom hooks as the environment that we provide. This is going
to change though as we move more variables into the hooks payload. As
such, the input environment is only going to contain the payload, while
the custom hooks environment will stay the same.
Let's prepare for this change by converting tests to discern between
actual and expected environment to make it easier to change.
|
|
|
|
hooks: Clean up unused leftover Ruby variables
Closes #2679
See merge request gitlab-org/gitaly!2900
|
|
Fix tests for cgroups that depends on linux-only library
See merge request gitlab-org/gitaly!2902
|
|
[ci skip]
|
|
Revert "On each read/write operation praefect requires to know which"
See merge request gitlab-org/gitaly!2907
|
|
This reverts commit 09c6d25de370446ac855a8241d8f821ed3f1ceec
|
|
gitaly node is a primary. For mutator operations it used to
define from what node the response will be returned back to
the client. For the read operations it is used to redirect request
to or as a fallback option for reads distribution in case it
is enabled. The default strategy for defining the primary is
an 'sql' which means the primary is tracked inside Postgres
database and praefect issues select statement into it each time
it needs to define current primary. It creates a high load
on the database when there are too many read operations (the
outcome of the performance testing).
To resolve this problem we change logic of retrieving set of
up to date storages to return all storages including primary.
So now we don't need to know the current primary and use any
storage that has latest generation of the repository to serve
the requests. As this information is cached by the in-memory
cache praefect won't create a high load on the database anymore.
This change also makes check IsLatestGeneration for the primary
node redundant as it won't be present in the set of consistent
storages if its generation not the latest one.
Closes: https://gitlab.com/gitlab-org/gitaly/-/issues/3337
|
